[
https://issues.apache.org/jira/browse/OOZIE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Satish Subhashrao Saley updated OOZIE-2547:
-------------------------------------------
Description:
Currently, we pass jars using --jars option while submitting spark job. Also,
we add spark.yarn.dist.files option in case of yarn-client mode.
Instead of that, we can have only --files option and pass on the files which
are present in mapreduce.job.cache.files. While doing so, we make sure that
spark won't make another copy of the files if files exist on the hdfs.
was:
Currently, we pass jars using --jars option while submitting spark job. Also,
we add spark.yarn.dist.files option in case of yarn-client mode.
We can have only --files option and pass on the files which are present in
> Add mapreduce.job.cache.files to spark action
> ---------------------------------------------
>
> Key: OOZIE-2547
> URL: https://issues.apache.org/jira/browse/OOZIE-2547
> Project: Oozie
> Issue Type: Bug
> Reporter: Satish Subhashrao Saley
> Assignee: Satish Subhashrao Saley
> Priority: Minor
>
> Currently, we pass jars using --jars option while submitting spark job. Also,
> we add spark.yarn.dist.files option in case of yarn-client mode.
> Instead of that, we can have only --files option and pass on the files which
> are present in mapreduce.job.cache.files. While doing so, we make sure that
> spark won't make another copy of the files if files exist on the hdfs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
