This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new e9697130a1ac [MINOR][TEST] Add output/exception to error message when schema not matched in `TPCDSQueryTestSuite` e9697130a1ac is described below commit e9697130a1acd2293d52ce72b9bddf6a203e3e8c Author: Liang-Chi Hsieh <vii...@gmail.com> AuthorDate: Mon Feb 5 09:43:44 2024 -0800 [MINOR][TEST] Add output/exception to error message when schema not matched in `TPCDSQueryTestSuite` ### What changes were proposed in this pull request? This patch adds output/exception string to the error message when output schema not match expected schema in `TPCDSQueryTestSuite`. ### Why are the changes needed? We have used `TPCDSQueryTestSuite` for testing TPCDS query results. The test suite checks output schema and then output result. If any exception happens during query execution, it will handle the exception and return an empty schema and exception class + message as output. So, when any exception happens, the test suite just fails on schema check and never uses/shows the exception, e.g., ``` java.lang.Exception: Expected "struct<[count(1):bigint]>", but got "struct<[]>" Schema did not match ``` We cannot see what exception was happened there from the log. It is somehow inconvenient for debugging. ### Does this PR introduce _any_ user-facing change? No, test only. ### How was this patch tested? Existing tests ### Was this patch authored or co-authored using generative AI tooling? No Closes #45025 from viirya/minor_ouput_exception. Authored-by: Liang-Chi Hsieh <vii...@gmail.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .../test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala index ef7bdc2b079e..bde615552987 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSQueryTestSuite.scala @@ -139,7 +139,14 @@ class TPCDSQueryTestSuite extends QueryTest with TPCDSBase with SQLQueryTestHelp (segments(1).trim, segments(2).replaceAll("\\s+$", "")) } - assertResult(expectedSchema, s"Schema did not match\n$queryString") { + val notMatchedSchemaOutput = if (schema == emptySchema) { + // There might be exception. See `handleExceptions`. + s"Schema did not match\n$queryString\nOutput/Exception: $outputString" + } else { + s"Schema did not match\n$queryString" + } + + assertResult(expectedSchema, notMatchedSchemaOutput) { schema } if (shouldSortResults) { --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org