Sounds like a bug. Can you provide the following information: 1. version of Pig 2. complete exception stack 3. try the following command line option: * -t all * -t ColumnMapKeyPrune 4. full script if possible
Thanks Daniel On Thu, Aug 11, 2011 at 8:15 AM, rob parker <[email protected]> wrote: > I am trying to use the PIG SUM function to sum a group of integers created > by a UDF and I am getting > > Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to > java.lang.Integer > > A = FOREACH Z GENERATE > FLATTEN(group) AS ( > key1:chararray, > key2:chararray, > key3:chararray, > year:int, > month:int, > day:int, > myType:chararray), > FLATTEN(myUDF(Y)) AS ( > startTime:chararray, > endTime:chararray, > quantity:int, > firstRecordTimestamp:chararray, > lastRecordTimestamp:chararray, > firstFileTimestamp:chararray, > lastFileTimestamp:chararray) ; > > > B = LOAD '$computeUOMmap' USING PigStorage(',') AS (myType:chararray, > derivedType:chararray); > > C = JOIN A BY myType, B BY myType USING 'replicated'; > > D = GROUP C BY ( > A::key1, > A::key2, > B::derivedType); > > > E = FOREACH D GENERATE > FLATTEN (group) AS (key1, key2, derivedType), > SUM(C.quantity) AS quantity:int, > MAX(C.lastRecordTimestamp) AS lastRecordTimestamp:chararray, > MIN(C.firstRecordTimestamp) AS firstRecordTimestamp:chararray, > MAX(C.lastFileTimestamp) AS lastFileTimestamp:chararray, > MIN(C.firstFileTimestamp) AS firstFileTimestamp:chararray; > > ================================ > describe A; > A: {key1: chararray,key2: chararray,key3: chararray,year: int,month: > int,day: int,myType: chararray,startTime: chararray,endTime: > chararray,quantity: int,firstRecordTimestamp: chararray,lastRecordTimestamp: > chararray,firstFileTimestamp: chararray,lastFileTimestamp: chararray} > > describe B; > B: {myType: chararray,derivedType: chararray} > > describe C; > C: {A::key1: chararray,A::key2: chararray,A::key3: chararray,A::year: > int,A::month: int,A::day: int,A::myType: chararray,A::startTime: > chararray,A::endTime: chararray,A::quantity: int,A::firstRecordTimestamp: > chararray,A::lastRecordTimestamp: chararray,A::firstFileTimestamp: > chararray,A::lastFileTimestamp: chararray,B::myType: > chararray,B::derivedType: chararray} > > describe D; > D: {group: (A::key1: chararray,A::key2: chararray,B::derivedType: > chararray),C: {(A::key1: chararray,A::key2: chararray,A::key3: > chararray,A::year: int,A::month: int,A::day: int,A::myType: > chararray,A::startTime: chararray,A::endTime: chararray,A::quantity: > int,A::firstRecordTimestamp: chararray,A::lastRecordTimestamp: > chararray,A::firstFileTimestamp: chararray,A::lastFileTimestamp: > chararray,B::myType: chararray,B::derivedType: chararray)}} > > > I do not understand why SUM thinks that quantity is a string? > > When I comment out the SUM line, output is produced correctly. > > Thanks, Rob >
