Hyukjin Kwon created SPARK-59077:
------------------------------------
Summary: Make the spark-connect-rust release workflow safe for a
direct release
Key: SPARK-59077
URL: https://issues.apache.org/jira/browse/SPARK-59077
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Hyukjin Kwon
The release workflow (.github/workflows/release.yml) can publish the Rust
crates to crates.io and the Python wheel to PyPI, but a dry_run=true dispatch
only builds wheels -- it never rehearses the crate publish, so the first proof
that "cargo publish" works is the irreversible real release. Other gaps make a
direct release risky:
- No idempotent recovery: a re-run after a partial publish aborts on "already
exists" (crates) and on duplicate upload (PyPI).
- The single ubuntu-latest wheel's glibc floor tracks the runner, with no
aarch64 build and no sdist fallback.
- No concurrency guard, so a tag push and a manual dispatch can publish at the
same time.
- The published crates lack repository/homepage metadata (crates.io "manifest
has no repository" warning).
Harden the workflow so a committer can cut a release directly:
- Rehearse the crate publish in dry-run via workspace-ordered "cargo publish
--dry-run" (cargo >= 1.90 orders and verify-builds the crates against the local
workspace, so it works even before anything is on crates.io).
- Make crate and wheel publishing idempotent (skip crates already on crates.io;
twine upload --skip-existing).
- Build portable manylinux2014 (x86_64 + aarch64 via --zig) and macOS
universal2 wheels, plus a source distribution.
- Add a concurrency guard that never cancels a publish in flight.
- Extend the version gate to also check the internal path-dependency version
pins.
- Inherit repository/homepage/authors on the published crates from the
workspace.
PR: https://github.com/apache/spark-connect-rust/pull/79
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
