spark git commit: [SPARK-9504] [STREAMING] [TESTS] Fix o.a.s.streaming.StreamingContextSuite.stop gracefully again
Repository: spark
Updated Branches:
refs/heads/master 2b67fdb60 -> d34bac0e1
[SPARK-9504] [STREAMING] [TESTS] Fix o.a.s.streaming.StreamingContextSuite.stop
gracefully again
The test failure is here:
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/3150/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=centos/testReport/junit/org.apache.spark.streaming/StreamingContextSuite/stop_gracefully/
There is a race condition in TestReceiver that it may add 1 record and increase
`TestReceiver.counter` after stopping `BlockGenerator`. This PR just adds
`join` to wait the pushing thread.
Author: zsxwing
Closes #7934 from zsxwing/SPARK-9504-2 and squashes the following commits:
cfd7973 [zsxwing] Wait for the thread to make sure we won't change
TestReceiver.counter after stopping BlockGenerator
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d34bac0e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d34bac0e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d34bac0e
Branch: refs/heads/master
Commit: d34bac0e156432ca6a260db73dbe1318060e309c
Parents: 2b67fdb
Author: zsxwing
Authored: Tue Aug 4 20:09:15 2015 -0700
Committer: Tathagata Das
Committed: Tue Aug 4 20:09:15 2015 -0700
--
.../scala/org/apache/spark/streaming/StreamingContextSuite.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/d34bac0e/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
--
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
index b7db280..7423ef6 100644
---
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
+++
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
@@ -789,7 +789,8 @@ class TestReceiver extends
Receiver[Int](StorageLevel.MEMORY_ONLY) with Logging
}
def onStop() {
-// no clean to be done, the receiving thread should stop on it own
+// no clean to be done, the receiving thread should stop on it own, so
just wait for it.
+receivingThreadOption.foreach(_.join())
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-9504] [STREAMING] [TESTS] Fix o.a.s.streaming.StreamingContextSuite.stop gracefully again
Repository: spark
Updated Branches:
refs/heads/branch-1.5 d196d3607 -> 6e72d24e2
[SPARK-9504] [STREAMING] [TESTS] Fix o.a.s.streaming.StreamingContextSuite.stop
gracefully again
The test failure is here:
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/3150/AMPLAB_JENKINS_BUILD_PROFILE=hadoop1.0,label=centos/testReport/junit/org.apache.spark.streaming/StreamingContextSuite/stop_gracefully/
There is a race condition in TestReceiver that it may add 1 record and increase
`TestReceiver.counter` after stopping `BlockGenerator`. This PR just adds
`join` to wait the pushing thread.
Author: zsxwing
Closes #7934 from zsxwing/SPARK-9504-2 and squashes the following commits:
cfd7973 [zsxwing] Wait for the thread to make sure we won't change
TestReceiver.counter after stopping BlockGenerator
(cherry picked from commit d34bac0e156432ca6a260db73dbe1318060e309c)
Signed-off-by: Tathagata Das
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6e72d24e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/6e72d24e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/6e72d24e
Branch: refs/heads/branch-1.5
Commit: 6e72d24e24d0e6d140e1b25597edae3f3054f98a
Parents: d196d36
Author: zsxwing
Authored: Tue Aug 4 20:09:15 2015 -0700
Committer: Tathagata Das
Committed: Tue Aug 4 20:09:27 2015 -0700
--
.../scala/org/apache/spark/streaming/StreamingContextSuite.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/spark/blob/6e72d24e/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
--
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
index b7db280..7423ef6 100644
---
a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
+++
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
@@ -789,7 +789,8 @@ class TestReceiver extends
Receiver[Int](StorageLevel.MEMORY_ONLY) with Logging
}
def onStop() {
-// no clean to be done, the receiving thread should stop on it own
+// no clean to be done, the receiving thread should stop on it own, so
just wait for it.
+receivingThreadOption.foreach(_.join())
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
