qzyu999 opened a new pull request, #3738: URL: https://github.com/apache/iceberg-python/pull/3738
## Summary Extract `PyArrowFile`, `PyArrowFileIO`, `PyArrowLocalFileSystem` and their helper functions into a new `pyiceberg/io/_pyarrow_fileio.py` module. The original `pyiceberg/io/pyarrow.py` re-exports all public names for full backward compatibility. This is the first step toward decomposing the 3,100+ line `pyarrow.py` monolith into focused modules by concern (tracked in #3737). ## Changes - **New file:** `pyiceberg/io/_pyarrow_fileio.py` (584 lines) containing the FileIO implementation - **Modified:** `pyiceberg/io/pyarrow.py` (-537 lines) replaced inline definitions with re-exports - **Removed unused imports** from `pyarrow.py` that were only needed by the FileIO section ## What stays the same - All existing `from pyiceberg.io.pyarrow import PyArrowFileIO` imports continue to work - Zero behavior change pure code movement with re-exports - Pickle/unpickle of `PyArrowFileIO` works unchanged - No new dependencies ## Why `pyarrow.py` handles 6 unrelated concerns in one file. The FileIO concern (filesystem abstraction for S3/GCS/Azure/HDFS/local) has zero coupling to schema conversion, expression translation, or statistics collection. Extracting it: 1. Reduces cognitive load when working on scan/write logic 2. Establishes the pattern for subsequent extractions (schema, expressions, stats, write, scan) 3. Creates a clean seam point for future alternative FileIO implementations ## Testing Full unit test suite passes: **3803 passed, 3 skipped, 0 failures, 0 errors** (Windows local testing requires #3721 + #3722 applied on top; Linux CI should pass without them.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
