andygrove opened a new pull request, #3411: URL: https://github.com/apache/datafusion-comet/pull/3411
## Summary - Add `ArrowConstantColumnReader` and `ArrowRowIndexColumnReader` that create Arrow Java vectors directly instead of going through native mutable buffers - `NativeBatchReader` now uses these new readers for partition columns, missing columns, and row index columns - Set `isFfiSafe=true` on `CometScanWrapper` so that `ScanExec` no longer deep-copies scan data The existing `ConstantColumnReader`, `RowIndexColumnReader`, and `MetadataColumnReader` are unchanged and still used by `BatchReader` (deprecated `native_comet` path) and Iceberg. ## Test plan - [ ] Partition column reading tests - [ ] Missing column / schema evolution tests - [ ] Row index column tests - [ ] Iceberg scan tests (`CometIcebergNativeSuite`) - [ ] Parquet read tests (`ParquetReadSuite`) - [ ] Exchange / shuffle tests (verify FFI safe default doesn't break) - [ ] Fuzz tests (`CometFuzzTestSuite`, `CometFuzzIcebergSuite`) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
