[ 
https://issues.apache.org/jira/browse/SPARK-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14134859#comment-14134859
 ] 

Michael Armbrust commented on SPARK-3534:
-----------------------------------------

sbt/sbt -Phive catalyst/test sql/test hive/test



> Avoid running MLlib and Streaming tests when testing SQL PRs
> ------------------------------------------------------------
>
>                 Key: SPARK-3534
>                 URL: https://issues.apache.org/jira/browse/SPARK-3534
>             Project: Spark
>          Issue Type: Bug
>          Components: Project Infra, SQL
>            Reporter: Michael Armbrust
>            Priority: Blocker
>
> We are bumping up against the 120 minute time limit for tests pretty 
> regularly now.  Since we have decreased the number of shuffle partitions and 
> up-ed the parallelism I don't think there is much low hanging fruit to speed 
> up the SQL tests. (The tests that are listed as taking 2-3 minutes are 
> actually 100s of tests that I think are valuable).  Instead I propose we 
> avoid running tests that we don't need to.
> This will have the added benefit of eliminating failures in SQL due to flaky 
> streaming tests.
> Note that this won't fix the full builds that are run for every commit.  
> There I think we just just up the test timeout.
> cc: [~joshrosen] [~pwendell]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to