Error in logical plan generated
-------------------------------
Key: PIG-1787
URL: https://issues.apache.org/jira/browse/PIG-1787
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Anitha Raju
Here is a sample pig script:
set default_parallel 2
ALLDATA = load 'sample.txt' using PigStorage() as (id, spaceid, type, pcid);
C1 = filter ALLDATA by (type == 'p' and
(spaceid == '1196250013'
or spaceid == '1196250024'
or spaceid == '1196250011'));
C2 = group C1 by pcid;
C3 = foreach C2 generate flatten(group) as (pc_id), COUNT(C1) as tot;
C4 = order C3 by tot desc;
C5 = limit C4 3;
C6 = join C5 by pc_id, C1 by pcid;
dump C6;
sample.txt:
1 1196250013 p 1234
2 1196250024 p 2314
3 1196250011 t 1111
4 1111111111 p 1231
5 1196250013 p 1254
6 1196250024 p 9007
This fails with the error
java.io.IOException: Type mismatch in key from map: expected
org.apache.pig.impl.io.NullableLongWritable, recieved
org.apache.pig.impl.io.NullableBytesWritable
when both pc_id and pcid are of type bytearray.
The script seems to work when
a) replicated join is substituted in the place of the regular join
b) pcid is cast to long in the loader
c) doing a dump of any statement before C6
d) setting default_parallel to 1 or removing it.
One possible cause seems to be with the logical plan generation during the
projection operation in C4 as can be observed from the describe statement.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
