[
https://issues.apache.org/jira/browse/PIG-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15687248#comment-15687248
]
Nandor Kollar commented on PIG-4952:
------------------------------------
[~kellyzly] could you please explain, what should ParallelismSetter do in Spark
mode? As far as I understood, those Spark operations, which form the stage
boundary, the partition count should be a configuration option (like in your
example for join). I took a look at the current converter implementations, and
found 1-2 operations where this is not yet implemented, will attach a patch
soon.
> Calculate the value of parallism for spark mode
> -----------------------------------------------
>
> Key: PIG-4952
> URL: https://issues.apache.org/jira/browse/PIG-4952
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: Nandor Kollar
> Fix For: spark-branch
>
>
> Calculate the value of parallism for spark mode like what
> org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter
> does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
