quinnj opened a new pull request, #609: URL: https://github.com/apache/arrow-julia/pull/609
## Summary - replace the Arrow 2.x implementation with a validated core, new IPC reader and writer, C Data and C Stream support, and a materialized public table facade - add `Tables.Scan` pushdown, sparse byte-range reads, footer statistics, and the CloudStore extension - add ArrowStrings 1.0, prepare ArrowTypes 2.4, and keep the public Arrow API small and namespace-qualified - add seeded property tests, focused rewrite regressions, Windows coverage, conformance oracles, and a trim-safe compile gate - rewrite the user documentation, migration guide, changelog, and ASF/General release workflow ## Dependency This PR is blocked by JuliaData/Tables.jl#380. The branch temporarily pins its exact current head, `64268c6a316e380cc3da26965f440a5433ebc1f7`, because `Tables.Scan` is not in a registered Tables.jl release yet. After Tables.jl releases that API, this PR must remove the temporary source and CI pins and raise the Tables.jl compatibility floor before merge and release. ## Breaking changes This is the Arrow.jl 3.0 rewrite. Important changes include: - Julia 1.10 is now required. - `Arrow.Table` and `Arrow.Stream` return materialized Julia vectors instead of lazy Arrow vector wrappers. - writing is eager; incremental writers, append, curried writes, and several Arrow 2.x writer options were removed. - `Arrow.write(io, table)` now writes file format by default; use `file=false` for stream format. - ArrowTypes-based custom serialization is no longer part of Arrow.jl. - most APIs are no longer exported and must be called through `Arrow`. See `CHANGELOG.md` and `docs/src/migration.md` for the complete list and migration guidance. ## Validation - Arrow root tests pass on Julia 1.10 and Julia 1.12: 3,260 and 3,261 counted assertions, plus all four acceptance batteries. - ArrowStrings 1.0 passes 2,515 tests on Julia 1.10 and Julia 1.12. - ArrowTypes 2.4 passes 133 tests on Julia 1.0 and Julia 1.12. - 20,000 deterministic malformed-input probes produced no raw exception leaks. - Apache Arrow corpus: 275 pass, 0 fail, 36 declared skips. - PyArrow/nanoarrow oracle: 170 pass, 0 fail, 43 declared skips. - C Data oracle: 143 pass, 0 fail, 9 declared skips. - Documenter, JuliaFormatter, Apache RAT, TOML/YAML parsing, release shell syntax, and `git diff --check` pass. Co-authored by Codex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
