[ https://issues.apache.org/jira/browse/PIG-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527997#comment-14527997 ]
liyunzhang_intel commented on PIG-4421: --------------------------------------- [~mohitsabharwal]: TestSkewedJoin.testSkewedJoinKeyPartition still fails in PIG-4421_5.patch because currently we use regular join to replace skewedjoin. so we can not fix all the unit test failures about skewedJoin. TestJoinSmoke.testSkewedJoinWithGroup passes in PIG-4421_5.patch. TestPigServerLocal#testNonExistingSecondDirectoryInSkewJoin passes in PIG-4421_5.patch. I will add an @Ignore to disable TestSkewedJoin.testSkewedJoinKeyPartition in next patch. > implement visitSkewedJoin in SparkCompiler > ------------------------------------------ > > Key: PIG-4421 > URL: https://issues.apache.org/jira/browse/PIG-4421 > Project: Pig > Issue Type: Sub-task > Components: spark > Reporter: liyunzhang_intel > Assignee: liyunzhang_intel > Fix For: spark-branch > > Attachments: PIG-4421.patch, PIG-4421_2.patch, PIG-4421_3.patch, > PIG-4421_4.patch, PIG-4421_5.patch > > > If visitSkewedJoin is not implemented, following unittests will fail. > org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithGroup > org.apache.pig.test.TestSkewedJoin.testSkewedJoinMapKey > org.apache.pig.test.TestSkewedJoin.testSkewedJoinManyReducers > org.apache.pig.test.TestSkewedJoin.testNonExistingInputPathInSkewJoin > org.apache.pig.test.TestSkewedJoin.testSkewedJoinOneValue > org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithNoProperties > org.apache.pig.test.TestSkewedJoin.testSkewedJoinEmptyInput > org.apache.pig.test.TestSkewedJoin.testSkewedJoinNullKeys > org.apache.pig.test.TestSkewedJoin.testSkewedJoinOuter > org.apache.pig.test.TestSkewedJoin.testRecursiveFileListing > org.apache.pig.test.TestSkewedJoin.testSkewedJoinReducers > org.apache.pig.test.TestJoinSmoke.testSkewedJoinWithGroup > org.apache.pig.test.TestJoinSmoke.testSkewedJoinOuter -- This message was sent by Atlassian JIRA (v6.3.4#6332)