[
https://issues.apache.org/jira/browse/PIG-4374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated PIG-4374:
----------------------------------
Description:
in current code, there are following class in mapreduce and tez package.
mapreduce:
MRCompiler
MROperPlan MROpPlanVisitor
MapReduceOper
tez:
TezCompiler
TezOperPlan TezOpPlanVisitor
TezOperator
Following class needs to be added in spark package:
SparkCompiler
SparkOperPlan SparkOpPlanVisitor
SparkOperator
Current code need to be refactored because of above adding classes. Some unit
tests like TestStoreInstances fail because of it.
following are the error of unit test TestStoreInstances:
Error Message
num jobs expected:<1> but was:<0>
Stacktrace
junit.framework.AssertionFailedError: num jobs expected:<1> but was:<0>
at
org.apache.pig.test.TestStoreInstances.testBackendStoreCommunication(TestStoreInstances.java:122)
was:
in current code, there are following class in mapreduce and tez package.
mapreduce:
MRCompiler
MROperPlan MROpPlanVisitor
MapReduceOper
tez:
TezCompiler
TezOperPlan TezOpPlanVisitor
TezOperator
Following class needs to be added in spark package:
SparkCompiler
SparkOperPlan SparkOpPlanVisitor
SparkOperator
Current code need to be refactored because of above adding classes.
> Need add SparkPlan in spark package
> -----------------------------------
>
> Key: PIG-4374
> URL: https://issues.apache.org/jira/browse/PIG-4374
> Project: Pig
> Issue Type: Bug
> Components: spark
> Reporter: liyunzhang_intel
>
> in current code, there are following class in mapreduce and tez package.
> mapreduce:
> MRCompiler
> MROperPlan MROpPlanVisitor
> MapReduceOper
> tez:
> TezCompiler
> TezOperPlan TezOpPlanVisitor
> TezOperator
> Following class needs to be added in spark package:
> SparkCompiler
> SparkOperPlan SparkOpPlanVisitor
> SparkOperator
> Current code need to be refactored because of above adding classes. Some unit
> tests like TestStoreInstances fail because of it.
> following are the error of unit test TestStoreInstances:
> Error Message
> num jobs expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: num jobs expected:<1> but was:<0>
> at
> org.apache.pig.test.TestStoreInstances.testBackendStoreCommunication(TestStoreInstances.java:122)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
