adriangb opened a new pull request, #18721:
URL: https://github.com/apache/datafusion/pull/18721
## Summary
This PR moves statistics handling from individual `FileSource`
implementations into `FileScanConfig`, simplifying the `FileSource` trait
interface.
### Changes
- **FileSource trait simplification**: Removed `statistics()`,
`with_statistics()`, and `with_projection()` methods
- **FileScanConfig enhancement**: Added `statistics` field and
`statistics()` method
- **FileSource implementations updated**: Removed `projected_statistics`
field from all implementations:
- ParquetSource
- CsvSource
- JsonSource
- AvroSource
- ArrowFileSource and ArrowStreamFileSource
- MockSource (test utility)
- **Test utilities**: Updated assertions to use `config.statistics()`
instead of `file_source.statistics()`
- **Proto serialization**: Updated to use `config.statistics()`
### Benefits
1. **Simpler trait interface**: `FileSource` implementations no longer need
to manage statistics
2. **Centralized statistics**: All statistics are now managed consistently
in `FileScanConfig`
3. **Cleaner API**: Statistics lifecycle is clearer and less error-prone
4. **Reduced code duplication**: Removes ~140 lines of boilerplate across
implementations
### Related
This is part of the projection refactoring work in #18627. This PR extracts
just the statistics-related changes to make review easier. The full projection
refactoring will come in subsequent PRs.
## Test plan
- [x] All modified file source implementations compile
- [x] Test utilities updated and compile
- [ ] CI tests pass (will verify after PR creation)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
