fboudra commented on PR #50409: URL: https://github.com/apache/arrow/pull/50409#issuecomment-5712871976
@mroeschke splitted into stacked PRs to make the review hopefully more easier | # | Branch | Description | |---|--------|-------------| | 1 | `pr-50409-limited-api-replacements` | Use more CPython Limited APIs in python/pyarrow | | 2 | `pr-50409-remove-benchmark` | Remove pyarrow.benchmark and simplify parse helpers | | 3 | `pr-50409-limited-api-fast-paths` | Use the CPython Limited API in the remaining fast paths | | 4 | `pr-50409-build-limited-api` | Build pyarrow with Py_LIMITED_API (cp311-abi3) | | 5 | `pr-50409-symbol-audit-script` | Add a limited-API symbol audit script for pyarrow | | 6 | `pr-50409-ci-abi3-wheels` | CI: ship cp311-abi3 wheels with symbol audit and perf gates | 1. https://github.com/apache/arrow/pull/51372 Limited API replacements, macros become calls. Source-only, no build-flag change: `PyBytes_AS_STRING`/`PyBytes_GET_SIZE`/`Py_TYPE()->tp_name` etc. become the equivalent Limited API calls. Full-API build and tests are unaffected (the Limited API is a subset of the full API), so this PR alone must be green on an unmodified tree. 2. https://github.com/apache/arrow/pull/51373 Remove pyarrow.benchmark and clean up the parse helpers. Deletions (`benchmark.cc`, `benchmark.pxi`, `benchmark.py`, microbenchmarks) plus three small simplifications in `helpers.cc`/`common.h` (`PyBytes_AsStdStringView`, ParseBinary debug check, ParseUuid). 3. https://github.com/apache/arrow/pull/51374 Remaining fast paths, still a full-API build. Fast paths in `common.*`, `iterators.h`, `numpy_convert.cc` and `python_to_arrow.cc`, `PyGILState_Check` replaced by `Py_IsInitialized`, pythoncapi_compat shim guards, last blockers. No `Py_LIMITED_API` define yet. 4. https://github.com/apache/arrow/pull/51375 The switch. `Py_LIMITED_API` define in `python/CMakeLists.txt` plus the fixes it requires: refcount/stable-API semantics, vendored pythoncapi_compat fixes, `Py_REFCNT` compat header, Decimal `'#'` specifiers, `.abi3.so` module naming, immortal-object test fix, and one unrelated one-file fix (honour `TZDB` in the vendored tz database). It is the first PR that delivers working cp311-abi3 builds. 5. https://github.com/apache/arrow/pull/51376 Audit script. New file only (`python/scripts/audit_limited_api_symbols.py`), no wiring. 6. https://github.com/apache/arrow/pull/51377 CI. Symbol-audit gate in the xlinux wheel build, abi3 tag plumbing (`SKBUILD_WHEEL_PY_API`), wheel matrix 49 tasks down to 7 on cp311-abi3 (3.14 build interpreter), perf gate (`perf_gate.py` plus committed reference), audit fallback to the `python3.11` binary, PEP 440 local- segment fix, freethreading row removal. CI files only, no Python code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
