longvu-db opened a new pull request, #55536: URL: https://github.com/apache/spark/pull/55536
### What changes were proposed in this pull request? Reorganize and expand the CACHE TABLE test coverage in `DataSourceV2DataFrameSuite` to match all five scenarios from the design doc Section [5] (CACHE TABLE impact on reads). **Moved tests:** Three existing cache-pinning tests are relocated to the bottom of the suite, grouped under a section comment for discoverability: - `"cached DSv2 table DataFrame is refreshed and reused after insert"` (Scenario 2) - `"SPARK-54022: caching table via Dataset API should pin table state"` (Scenario 1+2) - `"SPARK-54022: caching a query via Dataset API should not pin table state"` **New tests:** - **Scenario 3** (`"SPARK-54022: cached table pinned against external schema change"`): External ADD COLUMN via catalog API is invisible to the cached table. - **Scenario 4** (`"SPARK-54022: session schema change invalidates cache"`): Session `ALTER TABLE ADD COLUMN` invalidates and rebuilds cache with the new 3-column schema. - **Scenario 5** (`"SPARK-54022: cached table after external drop and recreate sees empty table"`): External drop+recreate via catalog API produces a new table with a different ID; query sees the new empty table. ### Why are the changes needed? The existing tests covered scenarios 1 and 2 (external data write pinning and session write invalidation), but did not cover: - External schema changes with cache pinning (scenario 3) - Session schema changes invalidating cache (scenario 4) - External drop and recreate of a cached table (scenario 5) These scenarios are important to verify the correctness of DSv2 cache behavior as described in the design doc. ### Does this PR introduce _any_ user-facing change? No. This PR only adds and reorganizes tests. ### How was this patch tested? New and moved tests in `DataSourceV2DataFrameSuite`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-opus-4-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
