Repository: spark Updated Branches: refs/heads/master db36e1e75 -> e164a04b2
[SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails ## What changes were proposed in this pull request? This PR moves `ssc.stop()` into `finally` for `StreamingContextSuite.createValidCheckpoint` to avoid leaking a StreamingContext since leaking a StreamingContext will fail a lot of tests and make us hard to find the real failure one. ## How was this patch tested? Jenkins unit tests Author: Shixiong Zhu <shixi...@databricks.com> Closes #14354 from zsxwing/ssc-leak. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e164a04b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e164a04b Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e164a04b Branch: refs/heads/master Commit: e164a04b2ba3503e5c14cd1cd4beb40e0b79925a Parents: db36e1e Author: Shixiong Zhu <shixi...@databricks.com> Authored: Mon Jul 25 18:26:29 2016 -0700 Committer: Tathagata Das <tathagata.das1...@gmail.com> Committed: Mon Jul 25 18:26:29 2016 -0700 ---------------------------------------------------------------------- .../org/apache/spark/streaming/StreamingContextSuite.scala | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/e164a04b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala ---------------------------------------------------------------------- diff --git a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala index 806e181..f1482e5 100644 --- a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala +++ b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala @@ -819,10 +819,13 @@ class StreamingContextSuite extends SparkFunSuite with BeforeAndAfter with Timeo ssc.checkpoint(checkpointDirectory) ssc.textFileStream(testDirectory).foreachRDD { rdd => rdd.count() } ssc.start() - eventually(timeout(10000 millis)) { - assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1) + try { + eventually(timeout(30000 millis)) { + assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1) + } + } finally { + ssc.stop() } - ssc.stop() checkpointDirectory } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
