GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/20221
[SPARK-23019][Core] Wait until SparkContext.stop() finished in SparkLauncherSuite ## What changes were proposed in this pull request? In current code ,the function `waitFor` in `BaseSuite` https://github.com/apache/spark/blob/cfcd746689c2b84824745fa6d327ffb584c7a17d/launcher/src/test/java/org/apache/spark/launcher/BaseSuite.java#L53 only wait until DAGScheduler is stopped, while SparkContext.clearActiveContext may not be called yet. Thus, in the Jenkins test https://amplab.cs.berkeley.edu/jenkins/job/spark-branch-2.3-test-maven-hadoop-2.6/ , `JdbcRDDSuite` failed because the previous test `SparkLauncherSuite` exit before SparkContext.stop() is finished. To repo: ``` $ build/sbt > project core > testOnly *SparkLauncherSuite *JavaJdbcRDDSuite ``` To Fix: Wait for a reasonable amount of time to avoid creating two active SparkContext in JVM in SparkLauncherSuite. Can' come up with any better solution for now. ## How was this patch tested? Unit test You can merge this pull request into a Git repository by running: $ git pull https://github.com/gengliangwang/spark SPARK-23019 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20221 ---- commit 16b985c1018c0f5f54ef1062bdd7960cc4a87b39 Author: Wang Gengliang <ltnwgl@...> Date: 2018-01-10T13:12:10Z SPARK-23019: Wait until SparkContext.stop() finished in SparkLauncherSuite ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org