Seanium opened a new pull request, #11348: URL: https://github.com/apache/arrow-rs/pull/11348
# Which issue does this PR close? Part of https://github.com/apache/datafusion/issues/25484. # Rationale for this change `StatisticsConverter` currently returns null bounds for INT96 timestamps, preventing pruning even when a file advertises the new timestamp column order. # What changes are included in this PR? - Convert row-group and page INT96 bounds directly to the requested timestamp unit, preserving timezone and the reader's pre-epoch rounding. - Discard both bounds when either overflows the target unit. The reader wraps out-of-range values, so keeping only the opposite bound could incorrectly prune matching rows. - Reject deprecated signed INT96 bounds. - Document that callers must verify `INT96_TIMESTAMP_ORDER` in the file footer before using these bounds for pruning. This updates the existing converter APIs. The direct-from-bytes API proposed in #11285 is separate and should use the same conversion rules if it lands first. # Are these changes tested? New real-file statistics tests cover all four timestamp units, timezones, null groups, negative sub-second timestamps, exact i64 boundaries, overflow on either endpoint, and agreement with decoded values. An additional test covers missing, invalid, and deprecated bounds. Existing INT64 timestamp behavior remains covered by the statistics suite. # Are there any significant user-facing changes? INT96 timestamp statistics become available when representable. No public API changes. ## AI assistance Implementation, tests, and review were AI-assisted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
