[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172090506 Yeah, it should be safe to backport, only the tests actually call `awaitTermination` for this thread. --- If your project is set up for it, you can reply to this e

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172073097 I think it is safe to back-port. I'll do that now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172067841 @srowen @zsxwing Is it safe to merge https://issues.apache.org/jira/browse/SPARK-12701 to branch 1.6? Or, we should revert it from branch 1.6? --- If your project is se

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172057917 Ah, I know what's going on @yhuai and @srowen . The cherry pick is fine, but I had [SPARK-12701](https://issues.apache.org/jira/browse/SPARK-12701) merged earlier

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172043417 @yhuai it's also possibly my cherry-pick to 1.6 was somehow not correct: https://github.com/apache/spark/commit/0c67993cf25c681611c55fd056808beee048129b If i

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-15 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-172038918 @BryanCutler can you take a look at the failed test (https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-1.6-test-maven-with-yarn-

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-14 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171751655 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-14 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171614056 Merged to master, 1.6 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10714 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-13 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171224237 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171105206 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171105208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171105026 **[Test build #49263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49263/consoleFull)** for PR 10714 at commit [`4684a58`](https://g

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171074720 **[Test build #49263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49263/consoleFull)** for PR 10714 at commit [`4684a58`](https://gi

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171068918 jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171068070 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171068076 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171067874 **[Test build #49256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49256/consoleFull)** for PR 10714 at commit [`4684a58`](https://g

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171040477 **[Test build #49256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49256/consoleFull)** for PR 10714 at commit [`4684a58`](https://gi

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171023079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171023075 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-171017718 Thanks @srowen , I think your suggestion makes it easier to follow what is going on, so I made that change. I also changed the check for appending for only if posi

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/10714#discussion_r49440804 --- Diff: core/src/main/scala/org/apache/spark/util/logging/FileAppender.scala --- @@ -58,20 +58,29 @@ private[spark] class FileAppender(inputStream: InputSt

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170835329 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170835328 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170835179 **[Test build #49219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49219/consoleFull)** for PR 10714 at commit [`20d8e7b`](https://g

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170813362 **[Test build #49219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49219/consoleFull)** for PR 10714 at commit [`20d8e7b`](https://gi

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170750310 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170750304 **[Test build #49198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49198/consoleFull)** for PR 10714 at commit [`dc65737`](https://g

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170750313 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10714#issuecomment-170750040 **[Test build #49198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49198/consoleFull)** for PR 10714 at commit [`dc65737`](https://gi

[GitHub] spark pull request: [SPARK-9844][CORE] File appender race conditio...

2016-01-11 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/10714 [SPARK-9844][CORE] File appender race condition during shutdown When an Executor process is destroyed, the FileAppender that is asynchronously reading the stderr stream of the process can throw