Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-178132065
I did not merge this into 1.6 or before because of 2 reasons:
- It doesn't merge cleanly, and more importantly
- This changes internal semantics and it's not t
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/10934
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-178130645
Merged into master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have thi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51463360
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,75 @@ class CheckpointSuite extends TestSuiteBase w
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177353166
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177353167
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177353119
**[Test build #50448 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50448/consoleFull)**
for PR 10934 at commit
[`20e4509`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177327392
**[Test build #50448 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50448/consoleFull)**
for PR 10934 at commit
[`20e4509`](https://gi
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51352293
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,75 @@ class CheckpointSuite extends TestSuiteBase with
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177081591
LGTM, I'll merge this once you address the minor comments
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51341040
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,75 @@ class CheckpointSuite extends TestSuiteBase w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51341044
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,75 @@ class CheckpointSuite extends TestSuiteBase w
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177049310
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177049306
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177048766
**[Test build #50421 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50421/consoleFull)**
for PR 10934 at commit
[`97e39c0`](https://g
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51334790
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,70 @@ class CheckpointSuite extends TestSuiteBase with
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177032299
**[Test build #50421 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50421/consoleFull)**
for PR 10934 at commit
[`97e39c0`](https://gi
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-177020382
Looks great! I only have documentation and test suggestions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51331064
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1578,6 +1582,11 @@ abstract class RDD[T: ClassTag](
if (!doCheckpointCalled
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51331038
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var ch
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330972
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,70 @@ class CheckpointSuite extends TestSuiteBase w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330867
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,70 @@ class CheckpointSuite extends TestSuiteBase w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330811
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,70 @@ class CheckpointSuite extends TestSuiteBase w
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330708
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -243,6 +244,10 @@ class JobGenerator(jobScheduler: JobS
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330737
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobScheduler.scala
---
@@ -210,6 +210,9 @@ class JobScheduler(val ssc: StreamingC
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330625
--- Diff: core/src/test/scala/org/apache/spark/CheckpointSuite.scala ---
@@ -512,6 +512,21 @@ class CheckpointSuite extends SparkFunSuite with
RDDCheckpoi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330500
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1578,6 +1582,11 @@ abstract class RDD[T: ClassTag](
if (!doCheckpointCalled
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51330247
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1697,6 +1706,8 @@ abstract class RDD[T: ClassTag](
*/
object RDD {
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51329576
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var ch
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r51328868
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var ch
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175464973
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175464971
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175464840
**[Test build #50173 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50173/consoleFull)**
for PR 10934 at commit
[`ef3983b`](https://g
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175414765
**[Test build #50173 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50173/consoleFull)**
for PR 10934 at commit
[`ef3983b`](https://gi
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175408940
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175406790
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175406784
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175360742
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175360743
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175360587
**[Test build #50141 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50141/consoleFull)**
for PR 10934 at commit
[`36cba8c`](https://g
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50934112
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,33 @@ class CheckpointSuite extends TestSuiteBase with
D
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50933818
--- Diff:
streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala ---
@@ -821,6 +821,33 @@ class CheckpointSuite extends TestSuiteBase with
D
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50933676
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -243,6 +243,10 @@ class JobGenerator(jobScheduler: JobSchedul
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50933356
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var checkpoi
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50930243
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -243,6 +243,10 @@ class JobGenerator(jobScheduler: JobSche
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50929995
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var check
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r5092
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
---
@@ -243,6 +243,10 @@ class JobGenerator(jobScheduler: JobSchedul
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50929833
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1578,6 +1582,11 @@ abstract class RDD[T: ClassTag](
if (!doCheckpointCalled) {
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/10934#discussion_r50929760
--- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala ---
@@ -1535,6 +1535,10 @@ abstract class RDD[T: ClassTag](
private[spark] var checkpoi
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175314567
**[Test build #50141 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50141/consoleFull)**
for PR 10934 at commit
[`36cba8c`](https://gi
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/10934#issuecomment-175310767
/cc @andrewor14 @tdas
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have thi
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/10934
[SPARK-6847][Core][Streaming]Fix stack overflow issue when updateStateByKey
is followed by a checkpointed dstream
Add a local property to indicate if checkpointing all RDDs that are marked
with th
52 matches
Mail list logo