[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-07 Thread viirya 
GitHub user viirya opened a pull request:

https://github.com/apache/spark/pull/22662

[SPARK-25627][TEST] Reduce test time for ContinuousStressSuite

## What changes were proposed in this pull request?

This goes to reduce test time for ContinuousStressSuite - from 8 mins 13 
sec to 43 seconds.

The approach taken by this is to reduce the triggers and epochs to wait and 
to reduce the expected rows accordingly.

## How was this patch tested?

Existing tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/viirya/spark-1 SPARK-25627

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/22662.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #22662


commit df420b7f1e4fef3354352c15541931ef0f46f02a
Author: Liang-Chi Hsieh 
Date:   2018-10-07T08:25:07Z

Reduce test time for ContinuousStressSuite.




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-13 Thread kiszk 
Github user kiszk commented on a diff in the pull request:

https://github.com/apache/spark/pull/22662#discussion_r224977443
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala
 ---
@@ -259,10 +259,10 @@ class ContinuousStressSuite extends 
ContinuousSuiteBase {
 testStream(df, useV2Sink = true)(
   StartStream(Trigger.Continuous(2012)),
--- End diff --

just curios: Is this value still `2012`? Can a reduction to `1012` reduce 
time?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-14 Thread viirya 
Github user viirya commented on a diff in the pull request:

https://github.com/apache/spark/pull/22662#discussion_r224994803
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala
 ---
@@ -259,10 +259,10 @@ class ContinuousStressSuite extends 
ContinuousSuiteBase {
 testStream(df, useV2Sink = true)(
   StartStream(Trigger.Continuous(2012)),
--- End diff --

I think 2012 is safer here as it only increases to epoch 1. Smaller value 
might make this test flaky.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-14 Thread kiszk 
Github user kiszk commented on a diff in the pull request:

https://github.com/apache/spark/pull/22662#discussion_r225000849
  
--- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala
 ---
@@ -259,10 +259,10 @@ class ContinuousStressSuite extends 
ContinuousSuiteBase {
 testStream(df, useV2Sink = true)(
   StartStream(Trigger.Continuous(2012)),
--- End diff --

Got it, thanks


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22662: [SPARK-25627][TEST] Reduce test time for Continuo...

2018-10-22 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/22662


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org