viirya opened a new pull request, #58215: URL: https://github.com/apache/spark/pull/58215
### What changes were proposed in this pull request? Add a test to `ArrowCachedBatchSerializerSuite` covering deeply nested types in the Arrow cache column-pruning read path added by #58177 (SPARK-58918): `array<struct<...>>`, `struct<struct<struct<...>>>`, and `map<int, array<int>>`, each selected alone, mixed with primitives, reordered, and with other columns pruned in between, under both the row and vectorized read paths. ### Why are the changes needed? The pruning read copies each selected top-level column's entire buffer subtree, so `readProjectedRecordBatch`'s buffer-span arithmetic (`fieldBufferCount` / `fieldNodeCount`) recurses through nested types to arbitrary depth. The original test only exercised one level of nesting (`array<int>`, `struct` of primitives), so an off-by-one in the recursive span over a deeper subtree -- which would shift the buffers of a following selected column and produce wrong values -- was not covered. The new test fails if that recursion is broken (verified locally by making the buffer count ignore children). ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? The new test; full `ArrowCachedBatchSerializerSuite` passes. ### Was this patch authored or co-authored using generative AI tooling? Yes, this pull request and its description were written by Claude Code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
