[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988270#action_12988270 ]
Thejas M Nair commented on PIG-1812: ------------------------------------ +1 > Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN > -------------------------------------------- > > Key: PIG-1812 > URL: https://issues.apache.org/jira/browse/PIG-1812 > Project: Pig > Issue Type: Bug > Affects Versions: 0.7.0, 0.8.0 > Environment: RHEL, Pig 0.8.0 > Reporter: xianyu > Assignee: Daniel Dai > Attachments: PIG-1812-1.patch, PIG-1812-2.patch > > > Hi, > I have the following input files: > pkg.txt > a 3 {(123,1.0),(236,2.0)} > a 3 {(236,1.0)} > model.txt > a 123 2 0.33 > a 236 2 0.5 > My script is listed below: > A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, > cat_bag:{t:(id:chararray, wht:float)}); > M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, > cat_id:chararray, ts:int, score:double); > B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray); > B = distinct B; > H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id); > H2 = foreach H1 { > I = order M by ts; > J = order B by ts; > generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as > tsorig, I.ts as tsmap; > } > dump H2; > When running this script, I got a warning about "Encountered Warning > DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below: > Pig Stack Trace > --------------- > ERROR 2043: Unexpected error during execution. > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to > open iterator for alias H2 > at org.apache.pig.PigServer.openIterator(PigServer.java:764) > at > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90) > at org.apache.pig.Main.run(Main.java:500) > at org.apache.pig.Main.main(Main.java:107) > Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2 > at org.apache.pig.PigServer.storeEx(PigServer.java:888) > at org.apache.pig.PigServer.store(PigServer.java:826) > at org.apache.pig.PigServer.openIterator(PigServer.java:738) > ... 7 more > Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: > Unexpected error during execution. > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208) > at org.apache.pig.PigServer.storeEx(PigServer.java:884) > ... 9 more > Caused by: java.lang.ClassCastException: > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad > cannot be cast to > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41) > at > org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69) > at > org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71) > at > org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52) > at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378) > ... 11 more > But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct > B;" this script can run smoothly. I have also tried other reducer side > operations like ORDER, it seems that they will also trigger above error. This > is really very confusing. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.