[ https://issues.apache.org/jira/browse/PIG-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166760#comment-15166760 ]
Xuefu Zhang commented on PIG-4781: ---------------------------------- +1. > Fix remaining unit failure about "TestCollectedGroup" for spark engine > ---------------------------------------------------------------------- > > Key: PIG-4781 > URL: https://issues.apache.org/jira/browse/PIG-4781 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: liyunzhang_intel > Assignee: liyunzhang_intel > Fix For: spark-branch > > Attachments: PIG-4781.patch > > > in > https://builds.apache.org/job/Pig-spark/lastUnsuccessfulBuild/#showFailuresLink, > it shows that following unit test fails: > org.apache.pig.test.TestCollectedGroup.testMapsideGroupWithMergeJoin > This fails because currently we use regular join to implement merge join. > the exeception is > {code} > Caused by: > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompilerException: > ERROR 2171: Expected one but found more then one root physical operator in > physical physicalPlan. > at > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompiler.visitCollectedGroup(SparkCompiler.java:512) > at > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POCollectedGroup.visit(POCollectedGroup.java:93) > at > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompiler.compile(SparkCompiler.java:259) > at > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompiler.compile(SparkCompiler.java:240) > at > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompiler.compile(SparkCompiler.java:240) > at > org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkCompiler.compile(SparkCompiler.java:165) > at > org.apache.pig.backend.hadoop.executionengine.spark.SparkLauncher.compile(SparkLauncher.java:425) > at > org.apache.pig.backend.hadoop.executionengine.spark.SparkLauncher.launchPig(SparkLauncher.java:150) > at > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:301) > at org.apache.pig.PigServer.launchPlan(PigServer.java:1390) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375) > at org.apache.pig.PigServer.storeEx(PigServer.java:1034) > ... 27 more > {code} > After we implement Merge join, this unit test can be fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)