[ https://issues.apache.org/jira/browse/PIG-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228133#comment-13228133 ]
Thejas M Nair commented on PIG-2570: ------------------------------------ It is possible to have multiple scalar variables in the limit expression, this will result in limit having multiple 'soft' predecessors. This case also needs to be handled in the patch. > LimitOptimizer fails with dynamic LIMIT argument > ------------------------------------------------ > > Key: PIG-2570 > URL: https://issues.apache.org/jira/browse/PIG-2570 > Project: Pig > Issue Type: Bug > Reporter: Bill Graham > Assignee: Bill Graham > Fix For: 0.10, 0.11 > > Attachments: PIG-2570-1.patch, data1.txt, limit-fails.pig > > > The following script fails with the exception shown below. Passing {{-t > LimitOptimizer}} makes it work, as does adding an {{ORDER}} clause before the > limit. > {noformat} > A = LOAD 'data1.txt' AS > (owner:chararray,pet:chararray,age:int,phone:chararray); > B = group A all; > C = foreach B generate SUM(A.age) as total; > D = foreach A generate owner, age/(double)C.total AS percentAge; > F = LIMIT D C.total/8; > DUMP F; > {noformat} > {noformat} > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: > Error processing rule LimitOptimizer. Try -t LimitOptimizer > at > org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:287) > at org.apache.pig.PigServer.compilePp(PigServer.java:1317) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1254) > at org.apache.pig.PigServer.storeEx(PigServer.java:953) > ... 14 more > Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2243: > Attempt to remove operator LOLimit that is still softly connected in the plan > at > org.apache.pig.newplan.BaseOperatorPlan.remove(BaseOperatorPlan.java:174) > at > org.apache.pig.newplan.BaseOperatorPlan.removeAndReconnect(BaseOperatorPlan.java:449) > at > org.apache.pig.newplan.logical.rules.LimitOptimizer$OptimizeLimitTransformer.transform(LimitOptimizer.java:124) > at > org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110) > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira