[
https://issues.apache.org/jira/browse/PIG-425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olga Natkovich reassigned PIG-425:
----------------------------------
Assignee: Alan Gates (was: Shravan Matthur Narayanamurthy)
I verified that all the tests run with the patch. However, we are not going to
commit the patch since IndexTuple is going away and Alan will integrate this
changes together with his NULL work on join.
> Split -> distinct or order -> cogroup fails
> -------------------------------------------
>
> Key: PIG-425
> URL: https://issues.apache.org/jira/browse/PIG-425
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: 425.patch
>
>
> A script like:
> {code}
> \a = load 'myfile' as (name:chararray, age:int, gpa:double);
> split a into a1 if age > 50, a2 if name < 'm';
> b2 = distinct a2;
> b1 = order a1 by name;
> c = cogroup b2 by name, b1 by name;
> d = foreach c generate flatten(group), COUNT($1), COUNT($2);
> store d into 'OUTPATH';
> {code}
> Will abort with the error:
> {code}
> 08/09/09 11:46:50 ERROR mapReduceLayer.Launcher: Error message from task
> (map) tip_200809080906_0185_m_000000java.lang.ClassCastException:
> org.apache.pig.data.DefaultTuple cannot be cast to
> org.apache.pig.data.IndexedTuple
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:81)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:135)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:75)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
> {code}
> The issue is that the RearrangeAdjuster in MRCompiler is not properly seeing
> this as a cogroup and moving the localrearrnge out of the reduce and into the
> map.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
