mbutrovich opened a new pull request, #2301: URL: https://github.com/apache/iceberg-rust/pull/2301
## Which issue does this PR close? - Closes #2299. ## What changes are included in this PR? - Add `coerce_int96_timestamps()` to patch the Arrow schema before reading, using arrow-rs's schema hint mechanism (`ArrowReaderOptions::with_schema`) to read INT96 columns at the resolution specified by the Iceberg table schema - `timestamp`/`timestamptz` → microsecond, `timestamp_ns`/`timestamptz_ns` → nanosecond, per the [Iceberg spec](https://iceberg.apache.org/spec/#primitive-types) - Falls back to microsecond when no field ID is available (matching Iceberg Java's `TimestampInt96Reader` behavior) - Applied after all three schema resolution branches (with field IDs, name mapping, positional fallback) so the fix covers both native and migrated tables - Handles INT96 inside nested types (structs) via Parquet column path matching ## Are these changes tested? - `test_read_int96_timestamps_with_field_ids` — files with embedded field IDs (branch 1) - `test_read_int96_timestamps_without_field_ids` — migrated files without field IDs (branches 2/3) - `test_read_int96_timestamps_in_struct` — INT96 inside a struct field - All three tests use dates outside the i64 nanosecond range (~1677-2262) to confirm the overflow is avoided - [Apache DataFusion Comet](https://github.com/apache/datafusion-comet) used the repro test in [apache/datafusion-comet#3856](https://github.com/apache/datafusion-comet/issues/3856) and it passes with this change -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
