[
https://issues.apache.org/jira/browse/PIG-4374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated PIG-4374:
----------------------------------
Attachment: PIG-4374_6_patch_jekins.png
PIG-4374_6_patch_jekins.png shows all failures of unit tests. Compared the test
result with https://builds.apache.org/job/Pig-spark/97/testReport/
(125failures), following unit tests failures are added:
// merge join
org.apache.pig.test.TestMergeJoin.testExpressionFail
//skewedjoin
org.apache.pig.test.TestJoinSmoke.testSkewedJoinWithGroup
org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithGroup
org.apache.pig.test.TestSkewedJoin.testSkewedJoinManyReducers
org.apache.pig.test.TestSkewedJoin.testSkewedJoinOneValue
org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithNoProperties
org.apache.pig.test.TestSkewedJoin.testSkewedJoinEmptyInput
// a complicated case using both 'group' and 'split'
org.apache.pig.test.TestScalarAliases.testScalarAliasesSplitClause
following unit tests failures are deleted:
org.apache.pig.test.TestLimitVariable.testNestedLimitVariable1
org.apache.pig.test.TestLimitVariable.testZeroLimitVariable
org.apache.pig.test.TestEvalPipeline2.testPushUpFilterScalar
org.apache.pig.test.TestSample.testSample_VariableAll
> Add SparkPlan in spark package
> ------------------------------
>
> Key: PIG-4374
> URL: https://issues.apache.org/jira/browse/PIG-4374
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Attachments: PIG-4374_1.patch, PIG-4374_2.patch.1,
> PIG-4374_3.patch.1, PIG-4374_4.patch, PIG-4374_4.patch_Jenkins_result.png,
> PIG-4374_6.patch, PIG-4374_6_patch_jekins.png, Pig-spark #76 [Jenkins .png,
> jenkins_PIG-4374_1_patch.png.png
>
>
> in current code, there are following class in mapreduce and tez package.
> mapreduce:
> MRCompiler
> MROperPlan MROpPlanVisitor
> MapReduceOper
> tez:
> TezCompiler
> TezOperPlan TezOpPlanVisitor
> TezOperator
> Following class needs to be added in spark package:
> SparkCompiler
> SparkOperPlan SparkOpPlanVisitor
> SparkOperator
> Current code need to be refactored because of above adding classes. Some unit
> tests like TestStoreInstances fail because of it.
> following are the error of unit test TestStoreInstances:
> Error Message
> num jobs expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: num jobs expected:<1> but was:<0>
> at
> org.apache.pig.test.TestStoreInstances.testBackendStoreCommunication(TestStoreInstances.java:122)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
