[
https://issues.apache.org/jira/browse/PIG-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liyunzhang_intel updated PIG-4783:
----------------------------------
Attachment: PIG-4783.patch
Changes in PIG-4783.patch:
1.add a class JobGraphBuilder.java which is responsible for executing spark job
and collect job statistics
2.move functions to execute spark job and collect job statistics to
JobGraphBuilder
3.add a class UDFJarsFinder.java which finds udf jars to be downloaded with
spark job on every nodes
4.make "conf" as a property of SparkPigStats to avoid passing configuration
when calling SparkPigStats#addJobStats
> Refactor SparkLauncher for spark engine
> ---------------------------------------
>
> Key: PIG-4783
> URL: https://issues.apache.org/jira/browse/PIG-4783
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-4783.patch
>
>
> Currently, the code of SparkLauncher is too big. We can put some function
> which executes the spark plan and collects job statistics to other class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
