james-willis opened a new pull request, #101: URL: https://github.com/apache/spark-connect-rust/pull/101
### What changes were proposed in this pull request? Add a free-threaded (no-GIL, PEP 703) wheel build for `pyspark-client-rust`. This is the second of the two steps in [SPARK-59432](https://issues.apache.org/jira/browse/SPARK-59432); the companion change declares `gil_used = false` on the module. - `release.yml`: add a Linux `cp313t` / `cp314t` wheel build leg alongside the existing abi3 leg. abi3 and the free-threaded ABI are mutually exclusive, so these are separate, version-specific wheels; both land in the same `dist/` and are uploaded together, and pip selects the right one per interpreter. macOS/Windows free-threaded legs can follow. - New `freethreaded.yml` (runs on push / pull_request): compiles `pyspark-rs` against a free-threaded interpreter and imports the built extension. ### Why are the changes needed? The current wheels are `cp39-abi3`. abi3 is incompatible with the free-threaded ABI, so no wheel is selectable on a `3.13t` / `3.14t` interpreter — a `gil_used = false` declaration has no runtime effect until a free-threaded wheel exists. This PR makes that wheel exist. `freethreaded.yml` also fills a real CI gap: `pyspark-rs` is excluded from the workspace `default-members`, so no existing job compiles it against a free-threaded interpreter — and compiling against `Py_GIL_DISABLED` is exactly what enforces that every `#[pyclass]` is `Sync`. A green build in this workflow is that proof. ### Does this PR introduce _any_ user-facing change? Yes, additive: free-threaded (`cp313t` / `cp314t`) Linux wheels become available on release, so users on free-threaded CPython can install `pyspark-client-rust`. No change for existing (GIL-enabled / abi3) users. Note that the GIL only actually stays off once the module declares `gil_used = false` (companion change); with just this PR the free-threaded wheel loads but CPython re-enables the GIL on import. ### How was this patch tested? `freethreaded.yml` builds `pyspark-rs` against a free-threaded 3.13t interpreter (proving the `Sync` requirement holds) and imports the resulting extension. The `cp313t` / `cp314t` wheels building across the Linux targets runs in `release.yml`'s dry-run path. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Anthropic) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
