[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2015-01-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3515 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2015-01-19 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-70468111 Looks like this has gone stale so I'd like to close this issue pending an update form @JoshRosen --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-12-01 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-65183834 I haven't had a chance to dig into this much more, but perhaps this could be due to streaming checkpointing; if RDDs' dependencies change after checkpointing, then that

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-12-01 Thread CodEnFisH
Github user CodEnFisH commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-65181037 I reproduced the failed testing locally and took a look at the log. The failed test case ("awaitTermination with error in task") is to check if task failure is

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64976003 It looks like the tests have failed twice with the same error, so this looks like it might be a legitimate bug. That would be surprising, since it would seem to indica

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64975039 **[Test build #23954 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23954/consoleFull)** for PR 3515 at commit [`8aee34d`](https://git

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64975041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64972773 [Test build #23954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23954/consoleFull) for PR 3515 at commit [`8aee34d`](https://githu

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64966682 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/3515#discussion_r21056495 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -401,7 +370,7 @@ class DAGScheduler( val s = stages.head

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64966679 **[Test build #23950 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23950/consoleFull)** for PR 3515 at commit [`1ab3d6d`](https://git

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3515#discussion_r21055836 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -401,7 +370,7 @@ class DAGScheduler( val s = stages.head

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3515#issuecomment-64963143 [Test build #23950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23950/consoleFull) for PR 3515 at commit [`1ab3d6d`](https://githu

[GitHub] spark pull request: [SPARK-4654] Clean up DAGScheduler getMissingP...

2014-11-29 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/3515 [SPARK-4654] Clean up DAGScheduler getMissingParentStages / stageDependsOn methods DAGScheduler has getMissingParentStages() and stageDependsOn() methods which are suspiciously similar to getPare