Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-60237457
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58754077
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58687609
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58273902
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58275181
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21417/consoleFull)
for PR 2524 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58281930
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21417/consoleFull)
for PR 2524 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-58281935
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57806876
ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57299654
OK, Jenkins said OK
Finished the modification,
1. Removed the option for the user to choose whether the accumulator
accepts duplication (this may
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57303226
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21037/consoleFull)
for PR 2524 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57311210
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57311196
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21037/consoleFull)
for PR 2524 at commit
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57323970
added a test case for result stage
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57248974
I think it should work...I'm trying this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user markhamstra commented on a diff in the pull request:
https://github.com/apache/spark/pull/2524#discussion_r18193968
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -112,6 +112,10 @@ class DAGScheduler(
// stray messages
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57260140
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21011/consoleFull)
for PR 2524 at commit
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/2524#discussion_r18128796
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -112,6 +112,10 @@ class DAGScheduler(
// stray messages to
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57074221
Let's not de-duplicate in shuffle stages please. That complicates the patch
a lot and I'm not sure why people would necessarily use it.
Also, why did you add a
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57074233
Basically it would be great to get a really simple patch that *only* fixes
SPARK-3628 and adds no new data structures in DAGScheduler.
---
If your project is set up for
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57074388
the drawbacks for us not to de-duplicate in shuffle stage is that, it makes
accumulator usage to be very tricky...
it sounds like you are not encouraged to use
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-57074398
I can simply monitor the accumulator update in TaskSetManager, just not
sure if that can maximumly resolve the problem.
---
If your project is set up for it, you
Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/2524#issuecomment-56959242
BTW, if we don't want to de-duplicate in shuffle stages, we can just move
the necessary part to TaskSetManager
---
If your project is set up for it, you can reply to
22 matches
Mail list logo
