Repository: spark Updated Branches: refs/heads/master adce5ee72 -> 8290004d9
[SPARK-13693][STREAMING][TESTS] Stop StreamingContext before deleting checkpoint dir ## What changes were proposed in this pull request? Stop StreamingContext before deleting checkpoint dir to avoid the race condition that deleting the checkpoint dir and writing checkpoint happen at the same time. The flaky test log is here: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7/256/testReport/junit/org.apache.spark.streaming/MapWithStateSuite/_It_is_not_a_test_/ ## How was this patch tested? unit tests Author: Shixiong Zhu <shixi...@databricks.com> Closes #11531 from zsxwing/SPARK-13693. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8290004d Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/8290004d Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/8290004d Branch: refs/heads/master Commit: 8290004d94760c22d6d3ca8dda3003ac8644422f Parents: adce5ee Author: Shixiong Zhu <shixi...@databricks.com> Authored: Sat Mar 5 15:26:27 2016 -0800 Committer: Shixiong Zhu <shixi...@databricks.com> Committed: Sat Mar 5 15:26:27 2016 -0800 ---------------------------------------------------------------------- .../test/scala/org/apache/spark/streaming/MapWithStateSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/8290004d/streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala ---------------------------------------------------------------------- diff --git a/streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala b/streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala index b6d6585..4034009 100644 --- a/streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala +++ b/streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala @@ -43,10 +43,10 @@ class MapWithStateSuite extends SparkFunSuite } after { + StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) } if (checkpointDir != null) { Utils.deleteRecursively(checkpointDir) } - StreamingContext.getActive().foreach { _.stop(stopSparkContext = false) } } override def beforeAll(): Unit = { --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
