[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-22 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15888 Merged into master, branch-2.1 and branch-2.0. Thanks for reviewing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68924/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68924/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68924/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68889/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68889/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68889/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68883/ Test FAILed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68883/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68883/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 looks good. We don't have this but how about adding tests for `sparkCheckInstall`? we could at least check for the error cases it has right now? --- If your project is set up for it, you can

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68787/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68787 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68787/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68787/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 I think what you have here is reasonable but what about checking for spark.submit.deployMode from the sparkConfig list here too? I'm all for fixing this ASAP - thinking longer term, I

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-17 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15888 @felixcheung What about transferring ```SparkConf``` from JVM to R via environment when JVM creates R process? I guess it can work but need some time to verify and test. Since it's critical bug

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 That might be a reasonable approach. In fact, thinking more about this I think it really should instantiate a sparkConf to read any conf file etc as well (which could set a value for master

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68720/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68720/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15888 @felixcheung That's OK, and will make the fix more simple. I think ```sparkCheckInstall``` can only handle some conditions by arguments passed in. However, if we can not judge the deploy mode

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68720/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15888 We do try to maintain compatibility with sparkR.init() And we should avoid adding parameter in the middle of the parameter list - it breaks users depending on parameter order.

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68709/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68709/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68709/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/15888 cc @junyangq --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15888 cc @felixcheung @shivaram @zjffdu @sun-rui I'm not sure this fix affects other scenarios, but it can fix my bug. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68659/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68659/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68658/ Test PASSed. ---

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15888 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68658/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68659/consoleFull)** for PR 15888 at commit

[GitHub] spark issue #15888: [SPARK-18444][SPARKR] SparkR running in yarn-cluster mod...

2016-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15888 **[Test build #68658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68658/consoleFull)** for PR 15888 at commit