This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 72b55ccf832 [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number of partitions in TPC-DS build to avoid out-of-memory 72b55ccf832 is described below commit 72b55ccf8327c00e173ab6130fdb428ad0d5aacc Author: Hyukjin Kwon <gurwls...@apache.org> AuthorDate: Tue Jul 26 15:03:17 2022 +0900 [SPARK-39856][SQL][TESTS][FOLLOW-UP] Increase the number of partitions in TPC-DS build to avoid out-of-memory ### What changes were proposed in this pull request? This PR increases the number of partitions further more (see also https://github.com/apache/spark/pull/37270) ### Why are the changes needed? To make the build pass. At least, two builds (https://github.com/apache/spark/runs/7500542538?check_suite_focus=true and https://github.com/apache/spark/runs/7511748355?check_suite_focus=true) passed after https://github.com/apache/spark/pull/37273. I assume that the number of partitions helps, and this PR increases some more. ### Does this PR introduce _any_ user-facing change? No, test and dev-only. ### How was this patch tested? It's tested in https://github.com/LuciferYang/spark/runs/7497163716?check_suite_focus=true Closes #37286 from HyukjinKwon/SPARK-39856-follwup. Authored-by: Hyukjin Kwon <gurwls...@apache.org> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala index f3eaa898e59..9affe827bc1 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala @@ -62,7 +62,7 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase with SQLQueryTestHelp // To make output results deterministic override protected def sparkConf: SparkConf = super.sparkConf - .set(SQLConf.SHUFFLE_PARTITIONS.key, 16.toString) + .set(SQLConf.SHUFFLE_PARTITIONS.key, 32.toString) protected override def createSparkSession: TestSparkSession = { new TestSparkSession(new SparkContext("local[1]", this.getClass.getSimpleName, sparkConf)) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org