[spark] branch branch-3.3 updated: [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number of partitions in TPC-DS build to avoid out-of-memory

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.3 by this push:
     new 84ed6142ef5 [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number 
of partitions in TPC-DS build to avoid out-of-memory
84ed6142ef5 is described below

commit 84ed6142ef5f4caa4ef94685a8fc5e0105231aea
Author: Hyukjin Kwon <gurwls...@apache.org>
AuthorDate: Tue Jul 26 15:03:17 2022 +0900

    [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number of partitions in 
TPC-DS build to avoid out-of-memory
    
    ### What changes were proposed in this pull request?
    
    This PR increases the number of partitions further more (see also 
https://github.com/apache/spark/pull/37270)
    
    ### Why are the changes needed?
    
    To make the build pass.
    
    At least, two builds 
(https://github.com/apache/spark/runs/7500542538?check_suite_focus=true and 
https://github.com/apache/spark/runs/7511748355?check_suite_focus=true) passed 
after https://github.com/apache/spark/pull/37273. I assume that the number of 
partitions helps, and this PR increases some more.
    
    ### Does this PR introduce _any_ user-facing change?
    No, test and dev-only.
    
    ### How was this patch tested?
    
    It's tested in 
https://github.com/LuciferYang/spark/runs/7497163716?check_suite_focus=true
    
    Closes #37286 from HyukjinKwon/SPARK-39856-follwup.
    
    Authored-by: Hyukjin Kwon <gurwls...@apache.org>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
    (cherry picked from commit 72b55ccf8327c00e173ab6130fdb428ad0d5aacc)
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
index f3eaa898e59..9affe827bc1 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala
@@ -62,7 +62,7 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase 
with SQLQueryTestHelp
 
   // To make output results deterministic
   override protected def sparkConf: SparkConf = super.sparkConf
-    .set(SQLConf.SHUFFLE_PARTITIONS.key, 16.toString)
+    .set(SQLConf.SHUFFLE_PARTITIONS.key, 32.toString)
 
   protected override def createSparkSession: TestSparkSession = {
     new TestSparkSession(new SparkContext("local[1]", 
this.getClass.getSimpleName, sparkConf))


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org