This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 51194d883c6 [SPARK-39519][SPARK-39387][SQL][TESTS] Ignore flaky test `: BytesColumnVector should not throw RuntimeException due to overflow` 51194d883c6 is described below commit 51194d883c63187423b9069cf1d2f040207566f6 Author: yangjie01 <yangji...@baidu.com> AuthorDate: Fri Jun 24 11:58:28 2022 +0900 [SPARK-39519][SPARK-39387][SQL][TESTS] Ignore flaky test `: BytesColumnVector should not throw RuntimeException due to overflow` ### What changes were proposed in this pull request? This pr aims to ignore flaky test `SPARK-39387: BytesColumnVector should not throw RuntimeException due to overflow` in `OrcQuerySuite` to ensure test stability when using Java 11. ### Why are the changes needed? "SPARK-39387: BytesColumnVector should not throw RuntimeException due to overflow" will prepare a large amount of test data (100MB * 22) and involve the allocation of multiple large arrays(`sharedBuffer` in `BytesColumnVector#allocateBuffer` method), the biggest is 1GB(From the description of https://github.com/apache/spark/pull/36772, it is expected). So when using Java 11 to test whole `sql` module, this suite often triggers OOM. This pr aims to ensure the stability of the test when using Java 11. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA Closes #36954 from LuciferYang/NewRatio. Authored-by: yangjie01 <yangji...@baidu.com> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- .../apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala index 275f20a08e6..2152cbd263a 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala @@ -833,7 +833,9 @@ abstract class OrcQuerySuite extends OrcQueryTest with SharedSparkSession { } } - test("SPARK-39387: BytesColumnVector should not throw RuntimeException due to overflow") { + // SPARK-39519: Ignore this case because it requires more than 4g heap memory to ensure test + // stability when use Java 11. Should test it manually when upgrading `hive-storage-api` + ignore("SPARK-39387: BytesColumnVector should not throw RuntimeException due to overflow") { withTempPath { dir => val path = dir.getCanonicalPath val df = spark.range(1, 22, 1, 1).map { _ => --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org