MaxGekk opened a new pull request, #58850: URL: https://github.com/apache/spark/pull/58850
### What changes were proposed in this pull request? Two tests in `ArrowCachedBatchSerializerSuite` for the Arrow cache serializer (SPARK-57268) over `TIME` and `INTERVAL DAY TO SECOND`. `TIME(p)` for p in 0, 3, 6 and 9, each at three null patterns (every 31st row null, no nulls, all nulls), a thousand rows in one partition. Each case checks that the cached frame answers what the uncached one did, that the cached `InMemoryRelation` keeps `TimeType(p)` with the declared precision, and that the serializer's own columnar read path (`convertCachedBatchToColumnarBatch`) yields a `TimeNanoVector` per batch whose row and null counts add up to the input's. The values are truncated to the declared precision, so a precision loss could not hide behind a value the type would round anyway. `INTERVAL DAY TO SECOND` with whole microseconds of both signs, at the same three null patterns, checking the same three things against a `DurationVector`. Two small helpers carry the mechanics: `cachedRelation`, which populates the cache and returns the `InMemoryRelation` behind a frame (it has to run before anything else forces the frame's memoized `queryExecution`, which is why the expected rows are built from the values rather than from an uncached `collect`), and `cachedVectors`, which reads the relation back through the serializer and reports the Arrow vector class, row count and null count per batch. ### Why are the changes needed? The suite round-trips `TIME` once, at precision 6, over two non-null rows, and checks its `LongColumnStats`. Every precision of `TIME` is written to the same `TimeNanoVector`, and the precision travels only in the Arrow field metadata, so a regression that dropped it on the way back through the cache would leave the values intact and fail no existing test. The all-null column takes a branch of its own in the vector (no validity buffer), and negative intervals were not covered at all. `TIME` is about to be enabled by default, and the Arrow cache is one of the paths it takes. ### Does this PR introduce _any_ user-facing change? No. Test-only. ### How was this patch tested? The two new tests, and the whole `ArrowCachedBatchSerializerSuite` (79 tests), pass locally; `dev/scalastyle` passes. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Fable 5.1) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
