[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/22662 [SPARK-25627][TEST] Reduce test time for ContinuousStressSuite ## What changes were proposed in this pull request? This goes to reduce test time for ContinuousStressSuite - from 8 mins 13 sec to 43 seconds. The approach taken by this is to reduce the triggers and epochs to wait and to reduce the expected rows accordingly. ## How was this patch tested? Existing tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/viirya/spark-1 SPARK-25627 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22662.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22662 commit df420b7f1e4fef3354352c15541931ef0f46f02a Author: Liang-Chi Hsieh Date: 2018-10-07T08:25:07Z Reduce test time for ContinuousStressSuite. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r224977443 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite extends ContinuousSuiteBase { testStream(df, useV2Sink = true)( StartStream(Trigger.Continuous(2012)), --- End diff -- just curios: Is this value still `2012`? Can a reduction to `1012` reduce time? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r224994803 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite extends ContinuousSuiteBase { testStream(df, useV2Sink = true)( StartStream(Trigger.Continuous(2012)), --- End diff -- I think 2012 is safer here as it only increases to epoch 1. Smaller value might make this test flaky. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22662#discussion_r225000849 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -259,10 +259,10 @@ class ContinuousStressSuite extends ContinuousSuiteBase { testStream(df, useV2Sink = true)( StartStream(Trigger.Continuous(2012)), --- End diff -- Got it, thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22662 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org