GitHub user kishorvpatil opened a pull request: https://github.com/apache/spark/pull/15627
[SPARK-18099][YARN] Fail if same files added to distributed cache for --files and --archives ## What changes were proposed in this pull request? During spark-submit, if yarn dist cache is instructed to add same file under --files and --archives, This code change ensures the spark yarn distributed cache behaviour is retained i.e. to warn and fail if same files is mentioned in both --files and --archives. ## How was this patch tested? Manually tested: 1. if same jar is mentioned in --jars and --files it will continue to submit the job. - basically functionality [SPARK-14423] #12203 is unchanged 2. if same file is mentioned in --files and --archives it will fail to submit the job. Please review https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before opening a pull request. ⦠under archives and files You can merge this pull request into a Git repository by running: $ git pull https://github.com/kishorvpatil/spark spark18099 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/15627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #15627 ---- commit 9bb16236ad7bbb982e0ffaa73899ebc11df9e6ee Author: Kishor Patil <kpa...@yahoo-inc.com> Date: 2016-10-25T18:19:46Z Dist cache yarn during submit should throw error for adding same file under archives and files ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org