PushUpFilter: Filter after skew LOJoin generates an error
---------------------------------------------------------
Key: PIG-910
URL: https://issues.apache.org/jira/browse/PIG-910
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.3.0
Reporter: Daniel Dai
Fix For: 0.4.0
The script is:
A = load 'a' as (user:chararray, action, timespent, query_term, ip_addr,
timestamp, estimated_revenue, page_info, page_links);
alpha = load 'b' as (user:chararray, action, timespent, query_term, ip_addr,
timestamp, estimated_revenue, page_info, page_links);
C = join A by user, alpha by user using "skewed" parallel 40;
D = filter C by A::user == 'aaa';
store D into 'L5out';
Stack trace is:
ERROR 1093: Swap supports swap of operators with at most one input. Found first
operator with 2 inputs.
org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2151:
Internal error while pushing filters up.
at
org.apache.pig.impl.logicalLayer.optimizer.PushUpFilter.transform(PushUpFilter.java:
325)
at
org.apache.pig.impl.logicalLayer.optimizer.LogicalOptimizer.optimize(LogicalOptimizer.java:185)
at org.apache.pig.PigServer.compileLp(PigServer.java:854)
at org.apache.pig.PigServer.compileLp(PigServer.java:791)
at org.apache.pig.PigServer.execute(PigServer.java:760)
at org.apache.pig.PigServer.access$100(PigServer.java:95)
at org.apache.pig.PigServer$Graph.execute(PigServer.java:943)
at org.apache.pig.PigServer.executeBatch(PigServer.java:255)
at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java: 112)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java: 169)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java: 141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:320)
Caused by: org.apache.pig.impl.plan.PlanException: ERROR 1093: Swap supports
swap of operators with at most one input. Found first operator with 2 inputs.
at org.apache.pig.impl.plan.OperatorPlan.swap(OperatorPlan.java:964)
at
org.apache.pig.impl.logicalLayer.optimizer.PushUpFilter.transform(PushUpFilter.java:
310)
... 12 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
