[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-212227437 @HyukjinKwon @vanzin Thanks. I have updated PR description. But @vanzin have merged to master before. So I think it does not matter for this PR. --- If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12473 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-212226185 @vanzin Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-212224734 LGTM, merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-212223875 @HyukjinKwon the merge scripts clean up "@" references from the PR summary. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-212185623 (This is super minor but I remember I was told it might be better if those cc are added in comments not in the description because PR description is the place

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211722592 **[Test build #56188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56188/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211722649 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211722647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211720533 @jerryshao Yes, what you said is right. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211720423 **[Test build #56188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56188/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211717311 So my understanding is that actually supporting **different** HDFS other than default one, not **multiple** HDFS, is that right? --- If your project is set up for

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211716533 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211716518 **[Test build #56186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56186/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211716532 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211714034 **[Test build #56186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56186/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211703497 **[Test build #56182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56182/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211703519 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211703518 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211702442 @vanzin yes, I update code with your comments.Thanks. @tgravescs I test this on my spark using spark-shell and spark-submit, I update it.Thanks. --- If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211702239 **[Test build #56182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56182/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211502073 test says NA, what testing have you done with this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211500988 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211500993 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211500511 **[Test build #56064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56064/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12473#issuecomment-211487353 **[Test build #56064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56064/consoleFull)** for PR 12473 at commit

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12473#discussion_r60098422 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -356,8 +361,8 @@ private[spark] class Client( logInfo("Preparing

[GitHub] spark pull request: [SPARK-14705][YARN]support Multiple FileSystem...

2016-04-18 Thread lianhuiwang
GitHub user lianhuiwang opened a pull request: https://github.com/apache/spark/pull/12473 [SPARK-14705][YARN]support Multiple FileSystem for YARN STAGING DIR ## What changes were proposed in this pull request? In SPARK-13063, It makes the SPARK YARN STAGING DIR as configurable.