HyukjinKwon commented on code in PR #84:
URL: https://github.com/apache/spark-connect-rust/pull/84#discussion_r3885943595
##########
.github/workflows/release.yml:
##########
@@ -57,118 +92,107 @@ concurrency:
group: release
cancel-in-progress: false
+env:
+ # The release source is always fetched from this canonical apache repository.
+ UPSTREAM_REPO: apache/spark-connect-rust
+
jobs:
- validate-version:
- name: Validate version
+ resolve:
+ name: Resolve release mode
runs-on: ubuntu-latest
+ # In the canonical apache repo, only dry runs are allowed (release-version
and
+ # rc-count empty). A fork may perform real runs. This mirrors apache/spark.
+ if: >-
+ github.repository != 'apache/spark-connect-rust' ||
+ (inputs.release-version == '' && inputs.rc-count == '')
+ outputs:
+ mode: ${{ steps.r.outputs.mode }}
+ version: ${{ steps.r.outputs.version }}
+ rc_tag: ${{ steps.r.outputs.rc_tag }}
+ branch: ${{ steps.r.outputs.branch }}
Review Comment:
**Source is a moving branch name, not a pinned commit.** `build-wheels`,
`build-sdist`, and `publish-rc` each independently `actions/checkout` this
branch. If a commit lands on the branch mid-run, the wheels, the sdist, and the
RC tag (created in `publish-rc`) can come from different commits — so the tag's
source and the attached wheels diverge, and at finalize `cargo publish` (from
the tag) ships different source than the PyPI wheels. The version assertion
only catches version *mismatches*, not same-version/different-commit drift.
Consider resolving the branch to a SHA once in `resolve` (output it) and
checking out that SHA everywhere.
##########
.github/workflows/release.yml:
##########
@@ -17,36 +17,71 @@
# under the License.
#
-# Release workflow for the Rust crates (crates.io) and the Python wheel (PyPI).
+# Release workflow for pyspark-client-rust (the Rust crates on crates.io + the
+# Python wheel on PyPI). Modeled on Apache Spark's release workflow: it is
+# DISPATCHED FROM A COMMITTER'S FORK (which holds the publish tokens), takes
the
+# same inputs, and follows the same release-candidate-then-finalize flow.
#
-# Following the Apache Spark model, releases are cut by a committer from THEIR
-# OWN FORK, not from the canonical apache/ repository:
+# Inputs (identical to apache/spark's release workflow):
+# branch apache/spark-connect-rust branch the release source is
fetched
+# from (always from apache, never the fork's own copy).
Default
+# master. Unused by finalize, which uses the RC tag as its
source.
+# release-version Final version, e.g. 4.2.0. Leave empty (with rc-count) for
a dry
+# run. Must equal the Cargo.toml/pyproject.toml version on
`branch`
+# (asserted) -- the RC number never appears in the artifact
version.
+# rc-count RC number, e.g. 1. Leave empty (with release-version) for
a dry run.
+# finalize true => convert the named RC into the OFFICIAL release
+# (IRREVERSIBLE). Default false.
#
-# 1. Enable GitHub Actions on your fork.
-# 2. Add the release secrets to the fork:
-# - CARGO_REGISTRY_TOKEN (crates.io API token)
-# - PYPI_API_TOKEN (PyPI API token)
-# 3. Bump the version in Cargo.toml and pyproject.toml (they must match).
-# 4. Dispatch this workflow with dry_run=true to rehearse (default), then
push
-# a matching `v<version>` tag (or dispatch with dry_run=false) to publish.
+# The three modes are derived from the inputs (see the `resolve` job):
#
-# A dry run fully rehearses BOTH sides without publishing: it packages and
-# verify-builds every crate in dependency order (`cargo publish --dry-run`) and
-# builds every wheel and the sdist. A tag push (or an explicit dry_run=false
-# dispatch) additionally publishes to crates.io and PyPI. Both publish steps
are
-# idempotent, so a re-run after a partial failure skips what already landed.
+# * Dry run (release-version AND rc-count empty): build every wheel + sdist
and
+# `cargo publish --dry-run`. Creates no tag, publishes nothing. This is the
+# testable path and also runs on a schedule to keep the release path
healthy.
+#
+# * Cut an RC (release-version + rc-count, finalize=false): build the wheels
+
+# sdist -- versioned as the FINAL release-version, since the RC number
lives
+# only in the tag/release name, never in the artifact version -- then
+# auto-create and push the tag `v<version>-rc<rc>` to this fork and attach
the
+# artifacts to a GitHub *pre-release* of that tag. crates.io/PyPI are NOT
+# touched, so an RC stays fully deletable.
+#
+# * Finalize (finalize=true, naming an existing RC): check out the RC tag,
+# `twine upload` the EXACT wheels+sdist from that RC's GitHub pre-release
to
+# PyPI, `cargo publish` the identical source to crates.io, then delete
every
+# `v<version>-rc*` GitHub pre-release (and its tag) from this fork. GitHub
+# Releases are only RC staging; the official release lives on PyPI +
crates.io.
+#
+# Required fork secrets for a real run: CARGO_REGISTRY_TOKEN (crates.io) and
+# PYPI_API_TOKEN (PyPI). Without them, only dry runs work. The default
GITHUB_TOKEN
+# creates/deletes the RC tags and GitHub Releases on the fork.
+#
+# To drop an RC without finalizing: delete its GitHub pre-release and the
+# `v<version>-rc<n>` tag from the fork (nothing was published to
PyPI/crates.io).
name: Release
on:
+ schedule:
+ # Periodic dry run (runs only in the repo hosting the workflow; GitHub
does not
+ # run scheduled workflows in forks), to catch release-path breakage early.
+ - cron: '0 7 */2 * *'
workflow_dispatch:
inputs:
- dry_run:
- description: "Build and validate artifacts but do not publish"
- type: boolean
- default: true
- push:
- tags: ["v*"]
+ branch:
+ description: 'Branch to release. Leave release-version/rc-count empty
to launch a dryrun. Dispatch this workflow only in the forked repository.'
+ required: true
+ default: master
+ release-version:
+ description: 'Release version (e.g. 4.2.0). Leave it empty to launch a
dryrun.'
+ required: false
+ rc-count:
+ description: 'RC number (e.g. 1). Leave it empty to launch a dryrun.'
+ required: false
+ finalize:
Review Comment:
**`finalize` is an untyped string input.** Without `type: boolean`, this
renders as a free-text field defaulting to the string `"false"`, and the gate
is `[ "$FIN" = "true" ]` (L134). A committer who types `True`/`yes`/`TRUE` gets
`MODE=rc` (since `release-version` is set) — the workflow silently re-cuts the
RC instead of finalizing, with no error, despite the IRREVERSIBLE intent. Use
`type: boolean`.
##########
.github/workflows/release.yml:
##########
@@ -233,28 +257,197 @@ jobs:
name: wheels-sdist
path: dist/*.tar.gz
- publish-wheels:
- name: Publish wheel + sdist to PyPI
- needs: [build-wheels, build-sdist]
- if: github.event_name == 'push' || (github.event_name ==
'workflow_dispatch' && inputs.dry_run == false)
+ # ---- Dry run: rehearse the crates.io packaging without publishing ----
+ dryrun-crates:
+ name: Dry-run crates.io packaging
+ needs: resolve
+ if: needs.resolve.outputs.mode == 'dryrun'
Review Comment:
**Dry-run performs no version validation.** The `Assert the branch version
equals release-version` step is gated `if: mode == 'rc'`, so the scheduled
every-other-day dry run never checks that `Cargo.toml` and `pyproject.toml`
agree. A branch with `Cargo.toml=4.3.0` but `pyproject.toml=4.2.0` passes the
dry run green — undercutting the stated "keep the release path healthy"
purpose. Consider an unconditional Cargo==pyproject agreement check
(independent of `release-version`) that also runs in dry-run mode.
##########
.github/workflows/release.yml:
##########
@@ -57,118 +92,107 @@ concurrency:
group: release
cancel-in-progress: false
+env:
+ # The release source is always fetched from this canonical apache repository.
+ UPSTREAM_REPO: apache/spark-connect-rust
+
jobs:
- validate-version:
- name: Validate version
+ resolve:
+ name: Resolve release mode
runs-on: ubuntu-latest
+ # In the canonical apache repo, only dry runs are allowed (release-version
and
+ # rc-count empty). A fork may perform real runs. This mirrors apache/spark.
+ if: >-
+ github.repository != 'apache/spark-connect-rust' ||
+ (inputs.release-version == '' && inputs.rc-count == '')
+ outputs:
+ mode: ${{ steps.r.outputs.mode }}
+ version: ${{ steps.r.outputs.version }}
+ rc_tag: ${{ steps.r.outputs.rc_tag }}
+ branch: ${{ steps.r.outputs.branch }}
steps:
- - uses: actions/checkout@v4
- - name: Cargo and pyproject versions agree (and match the tag, if any)
+ - name: Determine mode and validate inputs
+ id: r
+ env:
+ IN_BRANCH: ${{ inputs.branch }}
+ IN_VERSION: ${{ inputs.release-version }}
+ IN_RC: ${{ inputs.rc-count }}
+ IN_FINALIZE: ${{ inputs.finalize }}
run: |
set -euo pipefail
- CARGO_VER=$(grep -m1 '^version' Cargo.toml | awk -F'"' '{print $2}')
- PYPROJECT_VER=$(grep -m1 '^version' pyproject.toml | awk -F'"'
'{print $2}')
- echo "Cargo.toml=${CARGO_VER} pyproject.toml=${PYPROJECT_VER}"
- if [[ "${CARGO_VER}" != "${PYPROJECT_VER}" ]]; then
- echo "::error::Cargo.toml (${CARGO_VER}) and pyproject.toml
(${PYPROJECT_VER}) versions differ"
- exit 1
- fi
- # Internal path-dependency version pins (e.g. spark-connect-core ->
proto)
- # must track the workspace version, or `cargo publish` embeds a stale
- # requirement into the published crate.
- BAD=$(grep -rEn 'path = "\.\./spark-connect[^"]*"'
crates/*/Cargo.toml \
- | grep -E 'version = "' \
- | grep -v "version = \"${CARGO_VER}\"" || true)
- if [[ -n "${BAD}" ]]; then
- echo "::error::internal path-dep version pin(s) do not match
${CARGO_VER}:"
- echo "${BAD}"
+ BRANCH="${IN_BRANCH:-master}"
+ RV="${IN_VERSION:-}"
+ RC="${IN_RC:-}"
+ FIN="${IN_FINALIZE:-false}"
+
+ # release-version and rc-count must be provided together (or
neither).
+ if { [ -n "$RV" ] && [ -z "$RC" ]; } || { [ -z "$RV" ] && [ -n "$RC"
]; }; then
+ echo "::error::Provide BOTH 'release-version' and 'rc-count', or
leave BOTH empty for a dry run."
exit 1
fi
- if [[ "${GITHUB_REF}" == refs/tags/* ]]; then
- TAG="${GITHUB_REF##*/}"
- if [[ "${TAG}" != "v${CARGO_VER}" ]]; then
- echo "::error::Pushed tag ${TAG} does not match version
v${CARGO_VER}"
+
+ if [ "$FIN" = "true" ]; then
+ if [ -z "$RV" ] || [ -z "$RC" ]; then
+ echo "::error::finalize=true requires 'release-version' and
'rc-count' naming the RC to promote."
exit 1
fi
+ MODE=finalize
+ elif [ -n "$RV" ]; then
+ MODE=rc
+ else
+ MODE=dryrun
fi
- publish-crates:
- name: Publish Rust crates to crates.io
- needs: validate-version
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
- - name: Install Rust toolchain
- run: |
- rustup toolchain install stable --profile minimal
- rustup default stable
- - name: Install protoc
- run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
- - name: cargo publish (dependency-ordered; dry-run unless releasing)
- env:
- CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
- # true only for an explicit dry-run dispatch; tag pushes and an
explicit
- # dry_run=false dispatch both evaluate to "false" and publish for
real.
- DRY_RUN: ${{ github.event_name == 'workflow_dispatch' &&
inputs.dry_run }}
- run: |
- set -euo pipefail
- # Leaf-first; cargo (>=1.90) orders and verify-builds these together,
- # resolving siblings from the local workspace.
- CRATES=(apache-spark-connect-proto apache-spark-connect-core
apache-spark-connect)
- VER=$(grep -m1 '^version' Cargo.toml | awk -F'"' '{print $2}')
+ RC_TAG=""
+ [ -n "$RV" ] && RC_TAG="v${RV}-rc${RC}"
- if [[ "${DRY_RUN}" == "true" ]]; then
- # Rehearse the real publish. Because siblings resolve from the
local
- # workspace, this works even before anything exists on crates.io.
- pkgs=(); for c in "${CRATES[@]}"; do pkgs+=(-p "$c"); done
- echo "::group::cargo publish --dry-run ${pkgs[*]}"
- cargo publish --dry-run "${pkgs[@]}"
- echo "::endgroup::"
- echo "Dry run only — nothing published."
- exit 0
- fi
+ {
+ echo "mode=$MODE"
+ echo "version=$RV"
+ echo "rc_tag=$RC_TAG"
+ echo "branch=$BRANCH"
+ } >> "$GITHUB_OUTPUT"
+ echo "Resolved: mode=$MODE version='${RV}' rc='${RC}'
tag='${RC_TAG}' branch='${BRANCH}'"
- # Real publish. Skip any crate whose current version is already on
- # crates.io so a re-run after a partial failure is idempotent.
- pkgs=()
- for c in "${CRATES[@]}"; do
- if curl -sf -H "User-Agent: spark-connect-rust-release
([email protected])" \
- "https://crates.io/api/v1/crates/${c}/${VER}"; >/dev/null; then
- echo "${c} ${VER} is already on crates.io — skipping"
- else
- pkgs+=(-p "$c")
- fi
- done
- if [[ ${#pkgs[@]} -eq 0 ]]; then
- echo "All crates already published at ${VER}; nothing to do."
- exit 0
+ - name: Check out release source (for the version assertion)
+ if: steps.r.outputs.mode == 'rc'
+ uses: actions/checkout@v4
+ with:
+ repository: apache/spark-connect-rust
+ ref: ${{ steps.r.outputs.branch }}
+
+ - name: Assert the branch version equals release-version
+ if: steps.r.outputs.mode == 'rc'
+ run: |
+ set -euo pipefail
+ V="${{ steps.r.outputs.version }}"
+ CARGO_VER=$(grep -m1 '^version' Cargo.toml | awk -F'"' '{print $2}')
+ PYPROJECT_VER=$(grep -m1 '^version' pyproject.toml | awk -F'"'
'{print $2}')
+ echo "branch '${{ steps.r.outputs.branch }}': Cargo=${CARGO_VER}
pyproject=${PYPROJECT_VER}; release-version=${V}"
+ if [ "$CARGO_VER" != "$V" ] || [ "$PYPROJECT_VER" != "$V" ]; then
Review Comment:
**Dropped path-dependency pin validation (regression).** This step checks
`Cargo.toml`/`pyproject.toml` equal `release-version`, but the error message on
the next line promises to verify "the internal path-dep pins" — and nothing
does. The old `validate-version` job ran `grep -rEn 'path =
"\.\./spark-connect..."' | grep -v "version = \"${CARGO_VER}\""` to catch a
crate pinning e.g. `apache-spark-connect-proto = "4.1.0"` while the workspace
is `4.2.0`. Without it, finalize's `cargo publish` can ship
`apache-spark-connect-core 4.2.0` with a stale hard dep on
`apache-spark-connect-proto 4.1.0`, breaking downstream consumers. Re-add the
pin check here (and ideally in the finalize assertion at L373 too).
##########
.github/workflows/release.yml:
##########
@@ -57,118 +92,107 @@ concurrency:
group: release
cancel-in-progress: false
+env:
+ # The release source is always fetched from this canonical apache repository.
+ UPSTREAM_REPO: apache/spark-connect-rust
Review Comment:
**`UPSTREAM_REPO` is dead code.** It's declared as the canonical-repo source
of truth, but every checkout hardcodes the literal `repository:
apache/spark-connect-rust` (L161/194/246/269) and the `resolve` `if:` (L106)
re-hardcodes it. Either reference `${{ env.UPSTREAM_REPO }}` in those places or
drop the env var — as-is a repo rename must touch ~6 spots and the unused var
misleads readers into thinking it's authoritative.
##########
.github/workflows/release.yml:
##########
@@ -233,28 +257,197 @@ jobs:
name: wheels-sdist
path: dist/*.tar.gz
- publish-wheels:
- name: Publish wheel + sdist to PyPI
- needs: [build-wheels, build-sdist]
- if: github.event_name == 'push' || (github.event_name ==
'workflow_dispatch' && inputs.dry_run == false)
+ # ---- Dry run: rehearse the crates.io packaging without publishing ----
+ dryrun-crates:
+ name: Dry-run crates.io packaging
+ needs: resolve
+ if: needs.resolve.outputs.mode == 'dryrun'
runs-on: ubuntu-latest
steps:
- - uses: actions/setup-python@v5
+ - uses: actions/checkout@v4
with:
- python-version: "3.11"
+ repository: apache/spark-connect-rust
+ ref: ${{ needs.resolve.outputs.branch }}
+ - name: Install Rust toolchain
+ run: |
+ rustup toolchain install stable --profile minimal
+ rustup default stable
+ - name: Install protoc
+ run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
+ - name: cargo publish --dry-run (dependency-ordered)
+ run: |
+ set -euo pipefail
+ cargo publish --dry-run \
+ -p apache-spark-connect-proto \
+ -p apache-spark-connect-core \
+ -p apache-spark-connect
+
+ # ---- Cut an RC: tag + GitHub pre-release with the artifacts (no
crates.io/PyPI) ----
+ publish-rc:
+ name: Cut RC GitHub pre-release
+ needs: [resolve, build-wheels, build-sdist]
+ if: needs.resolve.outputs.mode == 'rc'
+ runs-on: ubuntu-latest
+ permissions:
+ contents: write # push the RC tag + create the GitHub Release on the
fork
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ repository: apache/spark-connect-rust
+ ref: ${{ needs.resolve.outputs.branch }}
+ fetch-depth: 0 # full history so the tag push carries all reachable
objects
- uses: actions/download-artifact@v4
with:
- # Matches every wheels-<target> artifact plus wheels-sdist.
pattern: wheels-*
path: dist
merge-multiple: true
- - name: Publish to PyPI
- # Fork-based release: authenticate with the committer's PyPI token via
twine
- # (avoids third-party publish actions; ASF Actions policy allowlists
actions/*).
+ - name: Create and push the RC tag to this fork
+ env:
+ GH_TOKEN: ${{ github.token }}
+ TAG: ${{ needs.resolve.outputs.rc_tag }}
+ VERSION: ${{ needs.resolve.outputs.version }}
+ run: |
+ set -euo pipefail
+ git config user.name "${{ github.actor }}"
+ git config user.email "${{ github.actor }}@users.noreply.github.com"
+ if git rev-parse -q --verify "refs/tags/${TAG}" >/dev/null; then
+ echo "Local tag ${TAG} already exists; reusing."
+ else
+ git tag -a "${TAG}" -m "Release candidate ${TAG} of
pyspark-client-rust ${VERSION}"
+ fi
+ # Push the RC tag (and the objects it reaches) to THIS fork, so the
GitHub
+ # pre-release below can hang off it. Force so re-cutting the same RC
is idempotent.
+ git push --force
"https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git";
"refs/tags/${TAG}"
+ - name: Create the GitHub pre-release with the artifacts
+ env:
+ GH_TOKEN: ${{ github.token }}
+ TAG: ${{ needs.resolve.outputs.rc_tag }}
+ VERSION: ${{ needs.resolve.outputs.version }}
+ run: |
+ set -euo pipefail
+ ls -l dist
+ NOTES="Release candidate ${TAG} of pyspark-client-rust
${VERSION}."$'\n\n'"> Not published to PyPI or crates.io. Install directly from
a wheel/sdist asset below, e.g. \`pip install <asset-url>\`. This pre-release
is deleted when the RC is finalized (or dropped)."
+ # Idempotent: a re-run updates the existing release and clobbers its
assets.
+ if gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}" >/dev/null
2>&1; then
+ gh release upload "${TAG}" dist/* --repo "${GITHUB_REPOSITORY}"
--clobber
+ else
+ gh release create "${TAG}" dist/* --repo "${GITHUB_REPOSITORY}" \
+ --prerelease --title "${TAG}" --notes "${NOTES}"
+ fi
+
+ # ---- Finalize: promote the named RC into the official release
(IRREVERSIBLE) ----
+ finalize:
+ name: Finalize release to PyPI + crates.io (IRREVERSIBLE)
+ needs: resolve
+ if: needs.resolve.outputs.mode == 'finalize'
+ runs-on: ubuntu-latest
+ permissions:
+ contents: write # delete the RC pre-releases + tags on the fork
+ steps:
+ - name: Abort window
+ run: |
+ echo
"=============================================================================="
+ echo " CONVERTING RC ${{ needs.resolve.outputs.rc_tag }} INTO THE
OFFICIAL RELEASE"
+ echo " ${{ needs.resolve.outputs.version }} -> PyPI + crates.io.
THIS IS IRREVERSIBLE."
+ echo " Cancel this workflow now if you did not intend to finalize."
+ echo " Continuing in 60 seconds..."
+ echo
"=============================================================================="
+ sleep 60
+ - name: Verify the RC pre-release exists
+ env:
+ GH_TOKEN: ${{ github.token }}
+ TAG: ${{ needs.resolve.outputs.rc_tag }}
+ run: |
+ set -euo pipefail
+ if ! gh release view "${TAG}" --repo "${GITHUB_REPOSITORY}"
>/dev/null 2>&1; then
+ echo "::error::RC GitHub pre-release ${TAG} not found on
${GITHUB_REPOSITORY}. Cut the RC first."
+ exit 1
+ fi
+ - name: Check out the RC tag (the exact released source)
+ uses: actions/checkout@v4
+ with:
+ # The RC tag lives on the fork where the RC was cut (this
repository).
+ repository: ${{ github.repository }}
+ ref: ${{ needs.resolve.outputs.rc_tag }}
+ fetch-depth: 0
+ - name: Assert the RC source version equals release-version
+ run: |
+ set -euo pipefail
+ V="${{ needs.resolve.outputs.version }}"
+ CARGO_VER=$(grep -m1 '^version' Cargo.toml | awk -F'"' '{print $2}')
+ PYPROJECT_VER=$(grep -m1 '^version' pyproject.toml | awk -F'"'
'{print $2}')
+ echo "RC tag source: Cargo=${CARGO_VER} pyproject=${PYPROJECT_VER};
release-version=${V}"
+ if [ "$CARGO_VER" != "$V" ] || [ "$PYPROJECT_VER" != "$V" ]; then
+ echo "::error::RC tag ${{ needs.resolve.outputs.rc_tag }} declares
Cargo=${CARGO_VER} / pyproject=${PYPROJECT_VER}, not ${V}."
+ exit 1
+ fi
+ - uses: actions/setup-python@v5
+ with:
+ python-version: "3.11"
+ - name: Download the RC artifacts (the exact tested wheels + sdist)
+ env:
+ GH_TOKEN: ${{ github.token }}
+ TAG: ${{ needs.resolve.outputs.rc_tag }}
+ run: |
+ set -euo pipefail
+ mkdir -p dist
+ gh release download "${TAG}" --repo "${GITHUB_REPOSITORY}" --dir
dist --pattern '*'
+ ls -l dist
+ - name: Publish wheels + sdist to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: |
+ set -euo pipefail
python -m pip install --upgrade twine
# --skip-existing makes a re-run after a partial upload safe.
twine upload --skip-existing dist/*
+ - name: Install Rust toolchain
+ run: |
+ rustup toolchain install stable --profile minimal
+ rustup default stable
+ - name: Install protoc
+ run: sudo apt-get update && sudo apt-get install -y protobuf-compiler
+ - name: Publish crates to crates.io (from the identical RC source)
+ env:
+ CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
+ run: |
+ set -euo pipefail
+ CRATES=(apache-spark-connect-proto apache-spark-connect-core
apache-spark-connect)
+ VER="${{ needs.resolve.outputs.version }}"
+ # Skip any crate already on crates.io at this version so a re-run is
idempotent.
+ pkgs=()
+ for c in "${CRATES[@]}"; do
+ if curl -sf -H "User-Agent: spark-connect-rust-release
([email protected])" \
+ "https://crates.io/api/v1/crates/${c}/${VER}"; >/dev/null; then
+ echo "${c} ${VER} is already on crates.io — skipping"
+ else
+ pkgs+=(-p "$c")
+ fi
+ done
+ if [ ${#pkgs[@]} -eq 0 ]; then
+ echo "All crates already published at ${VER}; nothing to do."
+ else
+ # cargo publishes in dependency order and waits for the index
between crates.
+ cargo publish "${pkgs[@]}"
+ fi
+ - name: Remove all RC GitHub pre-releases + tags for this version
+ env:
+ GH_TOKEN: ${{ github.token }}
+ VERSION: ${{ needs.resolve.outputs.version }}
+ run: |
+ set -euo pipefail
+ # Every v<version>-rcN pre-release is RC staging; the official
release now lives
+ # on PyPI + crates.io, so delete them all (and their tags) from this
fork.
+ # Escape dots: grep -E treats '.' as any char, so an unescaped 4.2.0
would also
+ # match tags like v4x2x0-rcN.
+ ESC_V="${VERSION//./\\.}"
+ mapfile -t RCS < <(gh release list --repo "${GITHUB_REPOSITORY}"
--limit 200 \
+ --json tagName --jq '.[].tagName' | grep -E "^v${ESC_V}-rc[0-9]+$"
|| true)
Review Comment:
**`|| true` masks a `gh` failure, not just grep's no-match.** With `set -o
pipefail` above, a transient `gh release list` failure (rate limit/network)
makes the pipeline exit non-zero, but `|| true` swallows that too — `RCS` ends
up empty, the step prints "No ... releases to remove", and finalize goes green
while every `v<version>-rcN` pre-release + tag is left behind, contradicting
the documented cleanup. Capture `gh release list` output in its own step/var
first (let it fail loudly), then apply `grep ... || true` only around the match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
