Repository: spark Updated Branches: refs/heads/master 0ad93b093 -> 34c3bc9f1
[SPARK-25618][SQL][TEST] Reduce time taken to execute KafkaContinuousSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? In this test, i have reduced the test time to 20 secs from 1 minute while reducing the sleep time from 1 sec to 100 milliseconds. With this change, i was able to run the test in 20+ seconds consistently on my laptop. I would like see if it passes in jenkins consistently. ## How was this patch tested? Its a test fix. Closes #22900 from dilipbiswal/SPARK-25618. Authored-by: Dilip Biswal <dbis...@us.ibm.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/34c3bc9f Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/34c3bc9f Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/34c3bc9f Branch: refs/heads/master Commit: 34c3bc9f1e2750bbcb91a8706ab78c6a58113350 Parents: 0ad93b0 Author: Dilip Biswal <dbis...@us.ibm.com> Authored: Wed Oct 31 02:57:39 2018 -0700 Committer: Dongjoon Hyun <dongj...@apache.org> Committed: Wed Oct 31 02:57:39 2018 -0700 ---------------------------------------------------------------------- .../apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/34c3bc9f/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala ---------------------------------------------------------------------- diff --git a/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala b/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala index 39c4e3f..491a9c6 100644 --- a/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala +++ b/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDontFailOnDataLossSuite.scala @@ -221,7 +221,7 @@ class KafkaSourceStressForDontFailOnDataLossSuite extends StreamTest with KafkaM .as[(String, String)] val query = startStream(kafka.map(kv => kv._2.toInt)) - val testTime = 1.minutes + val testTime = 20.seconds val startTime = System.currentTimeMillis() // Track the current existing topics val topics = mutable.ArrayBuffer[String]() @@ -252,7 +252,7 @@ class KafkaSourceStressForDontFailOnDataLossSuite extends StreamTest with KafkaM testUtils.createTopic(topic, partitions = 1, overwrite = true) logInfo(s"Create topic $topic") case 3 => - Thread.sleep(1000) + Thread.sleep(100) case _ => // Push random messages for (topic <- topics) { val size = Random.nextInt(10) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
