HyukjinKwon opened a new pull request, #79: URL: https://github.com/apache/spark-connect-rust/pull/79
## What changes were proposed in this pull request? Hardens `.github/workflows/release.yml` so a committer can cut a release directly (via a `v*` tag push or a `dry_run=false` dispatch) without the risk of a botched, irreversible publish. - **Dry-run now actually rehearses the crate publish.** Previously a `dry_run=true` dispatch only built wheels — the crate publish was never exercised, so the first proof that `cargo publish` worked was the real, irreversible release. `publish-crates` now always runs and, in dry-run, executes `cargo publish --dry-run -p apache-spark-connect-proto -p apache-spark-connect-core -p apache-spark-connect`. cargo (>= 1.90) orders and verify-builds these together against the local workspace, so it works even before anything exists on crates.io. The real publish drops the manual `sleep 30` loop and lets cargo order the crates and wait on the index. - **Idempotent re-runs.** The real crate publish skips any crate whose current version is already on crates.io, and wheels/sdist upload with `twine upload --skip-existing`. A re-run after a partial failure no longer aborts on "already exists". - **Portable, complete artifacts.** Wheels are built from a target matrix: Linux `x86_64` + `aarch64` cross-linked against an old glibc via `--zig` (manylinux2014), macOS `universal2`, and Windows `x64`, plus a source distribution (sdist) as the install fallback. This replaces the single `ubuntu-latest` wheel whose glibc floor tracked the runner (and which PyPI could reject), and the arm64-only macOS wheel. - **`concurrency` guard** so two releases never overlap and a publish in flight is never cancelled. - **Stronger version gate.** `validate-version` now also asserts the internal path-dep version pins (`spark-connect-core`/`spark-connect` → `proto`) track the workspace version. - **Crate metadata.** The three published crates inherit `repository`/`homepage`/`authors` from the workspace, fixing the `manifest has no ... repository` warning so the crates.io pages link back to the repo. The workflow stays first-party-actions-only (`actions/*` + `twine`/`maturin` via pip), consistent with the existing ASF Actions-policy note in the file. ## Why are the changes needed? A "dry run" that skips the crate publish gives false confidence: crates.io versions are permanent (yank-only), and the previous workflow offered no rehearsal and no safe recovery from a partial publish. The Linux wheel was also tied to the runner's glibc with no aarch64/sdist coverage. ## Does this PR introduce _any_ user-facing change? No. This only changes the release/CI workflow and crate packaging metadata; it does not change any client API. ## How was this patch tested? Locally rehearsed the crate side offline against a populated cargo cache (network to crates.io is the only piece a runner adds): - `cargo package -p apache-spark-connect-proto -p apache-spark-connect-core -p apache-spark-connect` packaged all three and verify-built each in dependency order, resolving `core`/`connect` against the **locally-packaged** siblings — the same resolution `cargo publish --dry-run` uses. - Inspected the shipped `Cargo.toml`: path deps stripped, `version` pins kept, and `repository`/`homepage`/`authors` now present (no more "manifest has no repository" warning). - `validate-version` version/tag/path-pin logic checked against the current `4.2.0`. - `release.yml` validated as well-formed YAML. Wheel/sdist build steps use standard `maturin` invocations and are exercised by the workflow's own dry-run. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
