yaooqinn opened a new pull request, #12124: URL: https://github.com/apache/gluten/pull/12124
## What changes are proposed in this pull request? Extend `ColumnarCachedBatchE2ESuite` with three lifecycle invariant tests for `ColumnarCachedBatchSerializer`, exercising the cached-batch wire format across SQLConf transitions: 1. **cross-config (forward)**: build with `spark.gluten.sql.columnar.maxBatchSize`-stats path enabled, read with it disabled. The wire format is build-time-decided; a v2-with-stats payload must survive a reader-time downgrade and partition pruning must still engage. 2. **cross-config (reverse)**: build with stats disabled, read with stats enabled. The legacy v1 payload (`stats=null`) at build time must NOT be retro-fitted by the reader; the query must fall back to a full scan. 3. **cross-build-cycle**: same logical query rebuilt twice with different stats settings. Round 2 must re-honor its own SQLConf rather than reuse stale gate state from round 1. Each test asserts (a) result row-count correctness, (b) the cached batches are served by `ColumnarCachedBatchSerializer` (R-path), and (c) when `expectPrune=true`, that `InMemoryTableScanExec.numOutputRows` reflects partition pruning (R-prune). The shared assertion logic is factored into a private helper `assertGlutenCachedPlanAndPrune(df, expectPrune)`. Its scaladoc documents an intentional asymmetry: the reverse "no prune" direction is not observable through `numOutputRows` on the Gluten native scan path — the existing baseline test "numOutputRows reflects post-filter row count" already notes that `outRows` may legitimately be 0 even under full pruning, because the surviving row is delivered via the native scan metrics path. The `expectPrune=false` branch therefore intentionally performs path-only verification. ## How was this patch tested? `ColumnarCachedBatchE2ESuite` — 15 succeeded / 0 failed / 2 canceled (pre-existing baseline) locally on Spark 3.5, Velox backend. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.7 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
