Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178793683
Quoting from the JIRA ticket:
> In our particular case, this behavior manifests since the
currentTaskAttemptId() method is returning -1 for each Spark
Github user nongli commented on a diff in the pull request:
https://github.com/apache/spark/pull/11012#discussion_r51628572
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -304,10 +309,9 @@ private[spark] class MemoryStore(blockManager:
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178785689
> An alternative option might be to track unroll/pending unroll memory
based on block ID rather than task attempt ID.
I think you need to account for this on
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178782148
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178782116
/cc @JoshRosen
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178790263
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178790259
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178819290
> @zsxwing, do you know why the streaming longevity / memory leak tests
didn't catch this leak?
If I understand correctly, this issue only happens when a
Github user nongli commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178824893
This looks right to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11012#discussion_r51633108
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -304,10 +309,9 @@ private[spark] class MemoryStore(blockManager:
Github user budde commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178853877
From Jenkins output:
>Fetching upstream changes from https://github.com/apache/spark.git
> git --version # timeout=10
> git fetch --tags --progress
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/11012#discussion_r51645315
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -304,10 +309,9 @@ private[spark] class MemoryStore(blockManager:
BlockManager,
Github user budde commented on a diff in the pull request:
https://github.com/apache/spark/pull/11012#discussion_r51643796
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -304,10 +309,9 @@ private[spark] class MemoryStore(blockManager:
BlockManager,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178892226
**[Test build #50612 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50612/consoleFull)**
for PR 11012 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178887920
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178935410
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178935407
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178935050
**[Test build #50612 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50612/consoleFull)**
for PR 11012 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178945649
**[Test build #50629 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50629/consoleFull)**
for PR 11012 at commit
Github user budde commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178940544
Looks like a bunch of Spark SQL/Hive tests are failing due to this error:
>Caused by: sbt.ForkMain$ForkError: org.apache.spark.SparkException: Job
aborted due to
Github user budde commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178929153
Latest change is looking good on my end. No unroll memory is being leaked.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178944716
**[Test build #50627 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50627/consoleFull)**
for PR 11012 at commit
Github user budde commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178766741
Updated PR with new implementation that uses a counter variable instead of
requiring the whole method to be atomic.
---
If your project is set up for it, you can reply
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178776979
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178776981
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178918227
@budde thanks for providing so much detail in the JIRA and fixing this
issue. I believe the latest changes are correct and seem more performant than
what you had
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178932261
**[Test build #2498 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2498/consoleFull)**
for PR 11012 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178932447
**[Test build #2499 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2499/consoleFull)**
for PR 11012 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178941583
no it's unrelated. I manually triggered a few extra builds. retest this
please
---
If your project is set up for it, you can reply to this email and have your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178980161
**[Test build #50629 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50629/consoleFull)**
for PR 11012 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178980260
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178982671
Merged into master and 1.6. Thanks again for catching this issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11012
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178978030
**[Test build #2499 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2499/consoleFull)**
for PR 11012 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178965433
**[Test build #2498 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2498/consoleFull)**
for PR 11012 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178979130
**[Test build #50627 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50627/consoleFull)**
for PR 11012 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178979204
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178979203
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user budde commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178314141
Pinging @andrewor14 , the original implementor of unrollSafely(), for any
potential feedback.
---
If your project is set up for it, you can reply to this email and have
GitHub user budde opened a pull request:
https://github.com/apache/spark/pull/11012
[SPARK-13122] Fix race condition in MemoryStore.unrollSafely()
https://issues.apache.org/jira/browse/SPARK-13122
A race condition can occur in MemoryStore's unrollSafely() method if two
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178299115
**[Test build #50520 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50520/consoleFull)**
for PR 11012 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178346293
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178346294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11012#issuecomment-178345954
**[Test build #50520 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50520/consoleFull)**
for PR 11012 at commit
44 matches
Mail list logo
