Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74637135
@pwendell @JoshRosen @MattWhelan Let me propose a 'real' fix I think. This
should be possible to make correct with a few more lines of code.
---
If your project is set
GitHub user MattWhelan opened a pull request:
https://github.com/apache/spark/pull/4627
SPARK-5841: remove DiskBlockManager shutdown hook on stop
After a call to stop, the shutdown hook is redundant, and causes a
memory leak.
You can merge this pull request into a Git
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74562181
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74562163
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74562692
[Test build #27565 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27565/consoleFull)
for PR 4627 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74561352
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74571626
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4627
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74607180
I saw it during shutdown from a failed Jenkins test run:
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74613715
@pwendell I suppose we could revert. A bandaid patch would be to just wrap
this in a try block and ignore exceptions thrown when removing the hook (see
also:
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74607089
Ah, how did you trigger that BTW? Yes I will help get it patched. I imagine
the stop logic must be factored into a method called both by the hook and by
stop, which can
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74606437
It looks like `removeShutdownHook` here can get called _during_ shutdown,
which leads to some exceptions that might prevent other cleanup from taking
place:
Github user MattWhelan commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74608953
Kinda weird that it would pass sometimes and fail others. I'll submit a
fix tomorrow.
---
If your project is set up for it, you can reply to this email and have
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74609998
@JoshRosen should we revert this in 1.3 then? I might create a release
candidate soon to kick off community testing.
---
If your project is set up for it, you can
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4627#issuecomment-74571622
[Test build #27565 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27565/consoleFull)
for PR 4627 at commit
15 matches
Mail list logo
