[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-04-05 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/5174 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-04-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5174#issuecomment-89007048 [Test build #29616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29616/consoleFull) for PR 5174 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-04-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5174#issuecomment-89007060 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5174#issuecomment-85719232 [Test build #29114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29114/consoleFull) for PR 5174 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85704201 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85717870 I've opened #5174 for `branch-1.2` (1.2.2). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/5174 [SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes (branch-1.2) ExecutorClassLoader does not ensure proper cleanup of network connections that it opens. If it

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85704462 This goes into master and 1.3. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85707764 Thanks for the review. I'll open a separate PR for `branch-1.2` (1.2.2). --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4944 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/5174#issuecomment-85744586 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/5174#issuecomment-85744571 [Test build #29114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29114/consoleFull) for PR 5174 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-23 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85264973 Yeah it looks okay to me but I also would feel more comfortable if a second core committer took a look. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-85258875 Ping @andrewor14 (as requested). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-17 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-82451448 I think this should be good to merge, but I'm going to leave it open for a little while longer in order to allow others to comment. --- If your project is set up for

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-81276944 [Test build #28633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28633/consoleFull) for PR 4944 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-15 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-81248275 [Test build #28633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28633/consoleFull) for PR 4944 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-15 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-81267408 I've updated this with a new commit that adds another layer of `try-catch` blocks to handle errors when reading the status code. I also made a slight change to how we

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-81276962 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-13 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4944#discussion_r26367472 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,30 +73,54 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-12 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-78329159 /cc @mccheah @pwendell @holdenk @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-11 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/4944#discussion_r26256855 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,30 +73,54 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4944#discussion_r26260907 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,30 +73,54 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-11 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-78342955 This looks like the right thing to do. Thanks for submitting this! @ash211 @MingyuKim care to take a look? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-11 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/4944#discussion_r26262063 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,30 +73,54 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-11 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4944#discussion_r26261715 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,30 +73,54 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77786580 I'm going to close this PR and re-open against master, since it looks like the conflicts shouldn't be too bad. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-77798728 [Test build #28378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28378/consoleFull) for PR 4944 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-77803785 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4944#issuecomment-77803782 [Test build #28378 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28378/consoleFull) for PR 4944 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread JoshRosen
Github user JoshRosen closed the pull request at: https://github.com/apache/spark/pull/4935 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77798705 I've opened an updated PR against the master branch at #4944 and added a regression test (which was slightly non-trivial to write). --- If your project is set up for

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-08 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/4944 [SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes (master branch PR) ExecutorClassLoader does not ensure proper cleanup of network connections that it opens.

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77657753 This actually turns out to be a much more subtle issue than I originally anticipated, since we might need to do other work to handle HttpUrlConnection's KeepAlive

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77652552 /cc @pwendell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25986036 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987903 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987502 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77661325 [Test build #28358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28358/consoleFull) for PR 4935 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77661330 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/4935 [SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes ExecutorClassLoader does not ensure proper cleanup of network connections that it opens. If it fails to

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4935#issuecomment-77652458 [Test build #28358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28358/consoleFull) for PR 4935 at commit

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25986151 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25987271 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri:

[GitHub] spark pull request: [SPARK-6209] Clean up connections in ExecutorC...

2015-03-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4935#discussion_r25988751 --- Diff: repl/src/main/scala/org/apache/spark/repl/ExecutorClassLoader.scala --- @@ -71,27 +72,64 @@ class ExecutorClassLoader(conf: SparkConf, classUri: