This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.3 by this push: new e2f82fdaeba [SPARK-39019][TESTS] Use `withTempPath` to clean up temporary data directory after `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` e2f82fdaeba is described below commit e2f82fdaeba8d56cf6fa2f75d6d52b5b4f9da4ea Author: yangjie01 <yangji...@baidu.com> AuthorDate: Tue Apr 26 15:52:22 2022 -0700 [SPARK-39019][TESTS] Use `withTempPath` to clean up temporary data directory after `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` ### What changes were proposed in this pull request? `SPARK-37463: read/write Timestamp ntz to Orc with different time zone` use the absolute path to save the test data, and does not clean up the test data after the test. This pr change to use `withTempPath` to ensure the data directory is cleaned up after testing. ### Why are the changes needed? Clean up the temporary data directory after test. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? - Pass GA - Manual test: Run ``` mvn clean install -pl sql/core -am -DskipTests mvn clean test -pl sql/core -Dtest=none -DwildcardSuites=org.apache.spark.sql.execution.datasources.orc.OrcV1QuerySuite git status ``` **Before** ``` sql/core/ts_ntz_orc/ ls sql/core/ts_ntz_orc _SUCCESS part-00000-9523e257-5024-4980-8bb3-12070222b0bd-c000.snappy.orc ``` **After** No residual `sql/core/ts_ntz_orc/` Closes #36352 from LuciferYang/SPARK-39019. Authored-by: yangjie01 <yangji...@baidu.com> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> (cherry picked from commit 97449d23a3b2232e14e63c6645919c5d93e4491c) Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- .../sql/execution/datasources/orc/OrcQuerySuite.scala | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala index f093a5fc735..e44fb63c757 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcQuerySuite.scala @@ -814,16 +814,19 @@ abstract class OrcQuerySuite extends OrcQueryTest with SharedSparkSession { | timestamp_ntz '2021-03-14 02:15:00.0' as ts_ntz3 |""".stripMargin - val df = sql(sqlText) + withTempPath { dir => + val path = dir.getCanonicalPath + val df = sql(sqlText) - df.write.mode("overwrite").orc("ts_ntz_orc") + df.write.mode("overwrite").orc(path) - val query = "select * from `orc`.`ts_ntz_orc`" + val query = s"select * from `orc`.`$path`" - DateTimeTestUtils.outstandingZoneIds.foreach { zoneId => - DateTimeTestUtils.withDefaultTimeZone(zoneId) { - withAllNativeOrcReaders { - checkAnswer(sql(query), df) + DateTimeTestUtils.outstandingZoneIds.foreach { zoneId => + DateTimeTestUtils.withDefaultTimeZone(zoneId) { + withAllNativeOrcReaders { + checkAnswer(sql(query), df) + } } } } --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org