What is the describe output?
Are you setting the outputSchema appropriately in ISOToUnixInSecond ?
(btw.. you probably want to call that InSeconds, plural)
D
On Wed, Dec 15, 2010 at 2:00 PM, John Hui <john.m....@gmail.com> wrote:
> To give more context, the ISOToUnixInSecond return UnixTime in second. The
> return value of this function is Long
>
> 75 @Override
> 76 public Long exec(Tuple input) throws IOException
> 77 {
> 78 if (input == null || input.size() < 1) {
> 79 return null;
> 80 }
> 81
> 82 // Set the time to default or the output is in UTC
> 83 DateTimeZone.setDefault(DateTimeZone.UTC);
> 84
> 85 DateTime result = new DateTime(input.get(0).toString());
> 86
> 87 return (result.getMillis() / 1000);
> 88 }
>
>
> I think somehow pig is not converting the Long into the proper long type in
> the pig script.
>
> ISOToUnixInSecond('$STARTDATETIME') AS startTime:long
>
> Hence during the comparion, it treat the Long as a string value ...
>
> On Wed, Dec 15, 2010 at 4:28 PM, John Hui <john.m....@gmail.com> wrote:
>
>> This is actually, please ignore the code section below, Thanks!
>>
>> 7 eventData = FOREACH flattenData GENERATE B1::event AS event,
>> B1::publication AS publication, B1::deviceType AS deviceType, B1::adID AS
>> adID, B1::mcc AS mcc, B1::event_timestamp AS eventTimestamp:long,
>> ISOToUnixInSecond('$STARTDATETIME') AS startTime:long;
>> 8
>> 9 eventData = FILTER eventData BY (event == 'adImpression') AND
>> (eventTimesta mp <= startTime);
>> 10
>> 11 DESCRIBE eventData;
>> 12
>> 13 B = GROUP eventData BY (event, publication, deviceType, adID, mcc);
>>
>>
>> On Wed, Dec 15, 2010 at 4:21 PM, John Hui <john.m....@gmail.com> wrote:
>>
>>> I am having a hard time getting comparison to work. I am comparing from
>>> two long values but I keep on getting a cast long to String error
>>>
>>> Backend error message
>>> ---------------------
>>> java.lang.ClassCastException: java.lang.Long cannot be cast to
>>> java.lang.String
>>> at java.lang.String.compareTo(String.java:92)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr.doComparison(LTOrEqualToExpr.java:152)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr.getNext(LTOrEqualToExpr.java:114)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:83)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:272)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:255)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
>>> at
>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
>>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:644)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:320)
>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>> at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>>> at org.apache.hadoop.mapred.Child.main(Child.java:211
>>>
>>>
>>> this is my pig code
>>>
>>> 7 eventData = FOREACH flattenData GENERATE B1::event AS event,
>>> B1::publicatio n AS publication, B1::deviceType AS deviceType, B1::adID
>>> AS adID, B1::mcc A S mcc, B1::event_timestamp AS eventTimestamp:long,
>>> ISOToUnixInSecond('$STAR TDATETIME') AS startTime:long;
>>> 8 9 ILLUSTRATE eventData;
>>> 10 11 eventData = FILTER eventData BY (startTime <= startTime);
>>> 12 --eventData = FILTER eventData BY (event == 'adImpression') AND
>>> (eventTimes tamp <= startTime);
>>> 13
>>> 14 DESCRIBE eventData;
>>>
>>> Here is the output
>>>
>>> eventData: {event: chararray,publication: chararray,deviceType:
>>> chararray,adID: chararray,mcc: chararray,eventTimestamp: long,startTime:
>>> long}
>>>
>>>
>>>
>>
>
