rdtr opened a new pull request, #13557: URL: https://github.com/apache/gluten/pull/13557
## What changes are proposed in this pull request? `Complex field and type inferring` and `SPARK-4228 DataFrame to JSON` are excluded from `GlutenJsonV1Suite`, `GlutenJsonV2Suite` and `GlutenJsonLegacyTimeParserSuite` in the spark40, spark41 and spark42 `VeloxTestSettings`, with the comment `FIXME: Array direct selection fails`. They are not excluded for Spark 3.4 or 3.5. Both tests pass now on 4.0, 4.1 and 4.2, so this removes the 18 excludes. The `SPARK-39731` exclude in `GlutenJsonV2Suite` stays. The JSON scan itself falls back, but the array access in the first test runs in Velox. For `select arrayOfString[0], arrayOfString[1], arrayOfString[2] from jsonTable`, the executed plan on Spark 4.2 is `VeloxColumnarToRow` over `ProjectExecTransformer [arrayOfString#8070[0] AS arrayOfString[0]#8074, ...]`. ## How was this patch tested? Ran `GlutenJsonV1Suite`, `GlutenJsonV2Suite` and `GlutenJsonLegacyTimeParserSuite` with `SPARK_ANSI_SQL_MODE=false` and `-Dspark.test.home` set: - Spark 4.0: 150, 152 and 150 tests, no failures. - Spark 4.1: 456 succeeded, 0 failed. - Spark 4.2: 462 succeeded, 3 failed. The 3 failures are `TIME type roundtrip with JSON datasource - representative precisions and custom format` in each suite. It is not excluded and fails the same way without this change. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
