This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/texera.git
commit 31f1e7bf8adc9efcc3334ad4dcce5df469218c19 Author: Yicong Huang <[email protected]> AuthorDate: Mon Jul 27 15:23:36 2026 -0700 ci: automate backport labeling and open PRs for failed backports (#6941) ### What changes were proposed in this PR? This PR overhauls the release-backport lifecycle so it stops depending on people remembering. Three problems today, each addressed below. **Problem 1 — Backports are forgotten.** The `release/*` label is added by hand, so authors and committers routinely forget to request a backport, and fixes silently never reach the release branch. _Solution:_ Backporting becomes **opt-out**. A new `backport-auto-label` workflow labels every `fix:` PR into `main` with the configured `release/*` targets and requests review from each branch's **release manager**. `.github/release-branches.yml` is the single source of truth mapping each release branch to its manager and an `actively-supporting` flag (`release/v1.2` → `xuang7`, active; `release/v1.1` → `bobbai00`, inactive), parsed by a stdlib-only helper. Only actively-supporting branches are auto-labeled; an inactive branch stays a valid manual target but isn't offered by default. Remove a label to decline — a removal is remembered via the timeline, so a later edit never silently re-adds it. **Problem 2 — Failed backports are lost.** When a backport conflicts, the only trace is a PR comment that sinks to the bottom and gets forgotten; nobody is on the hook to finish it. And a conflicting cherry-pick still burns the full build matrix. _Solution:_ The pre-merge backport is split into a fast, git-only **apply-check** that gates the expensive build (a conflict is reported in seconds and no longer spins up the stacks), and `backport` is **removed from the Required Checks aggregator** so it's advisory, not a merge gate. Post-merge, `direct-backport-push` classifies each target from that signal: **green** cherry-picks straight to the release branch; **red** (conflict, or applies-but-won't-build) auto-opens a **draft** backport PR `fix(scope, vX.Y): …` on `backport/<PR>-<slug>-<target>`, with the conflicted tree committed, **assigned to the author** with the release manager as reviewer — an actionable task instead of a lost comment. **Problem 3 — Fixes for main-only features get backported anyway.** A fix for a feature that only exists on `main` shouldn't go to the release at all, but it still gets labeled and attempted. _Solution:_ **Automatic feature-absent skip** — before labeling a target (and again on the red path), skip it when every file the PR *modifies* (added files excluded) is absent on that release branch: the feature clearly isn't there. Any ambiguity keeps the label and lets the apply-check decide. For what automation can't catch (new code inside an existing file, or pure judgment), a **`no-backport-needed`** label is a hard manual veto honored end-to-end (auto-label, pre-merge precheck, and post-merge push). Note: a red apply-check now shows as a (non-blocking) red check on the PR — the intended "this needs a manual backport" signal, not a merge blocker. ### Any related issues, documentation, discussions? Resolves #6940. ### How was this PR tested? Static: `actionlint` on all workflows; YAML parses; every `github-script` block passes `node --check`; `bash -n` on the scripts; unit-checked the parser, branch-slug derivation, `fix(scope, vX.Y)` title injection, and the feature-absent git guard. End-to-end on a fork test harness ([`Yicong-Huang/texera`](https://github.com/Yicong-Huang/texera), throwaway `release/test-clean` + `release/test-conflict` branches, docs-only fixes to skip the heavy build): | Scenario | Result | Evidence | | --- | --- | --- | | `fix:` PR into main → auto-labeled with both targets | ✅ | [PR #19](https://github.com/Yicong-Huang/texera/pull/19) | | apply-check: green on clean target, red on conflicting one | ✅ | [run](https://github.com/Yicong-Huang/texera/actions/runs/30303217100) | | red apply-check does **not** block merge (Required Checks green) | ✅ | [run](https://github.com/Yicong-Huang/texera/actions/runs/30303217100) | | green target → cherry-picked to the release branch (author preserved) | ✅ | [commit `ef1df63`](https://github.com/Yicong-Huang/texera/commit/ef1df63cc6429b3ea9429a5375007422861934e8) · [run](https://github.com/Yicong-Huang/texera/actions/runs/30303765765) | | red target → draft PR `fix(scope, vX.Y): …`, conflict markers committed, assigned to author | ✅ | [PR #22](https://github.com/Yicong-Huang/texera/pull/22) | | fix touching only files absent on the release branch → not labeled | ✅ | [PR #20](https://github.com/Yicong-Huang/texera/pull/20) | | `no-backport-needed` → all backport work vetoed | ✅ | [PR #21](https://github.com/Yicong-Huang/texera/pull/21) | | remove a label then edit → not re-added (opt-out remembered) | ✅ | [PR #23](https://github.com/Yicong-Huang/texera/pull/23) | Not exercised on the fork: the request-review success path (the test manager was the PR author, so it correctly took the skip branch) and a full heavy-build green path (docs-only by design). ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]> --- .github/release-branches.yml | 46 ++++ .github/scripts/create-backport-branch.sh | 114 ++++++++++ .github/scripts/release_branches.py | 115 ++++++++++ .github/workflows/backport-auto-label.yml | 208 ++++++++++++++++++ .github/workflows/direct-backport-push.yml | 340 +++++++++++++++++++++++++---- .github/workflows/required-checks.yml | 81 ++++++- 6 files changed, 857 insertions(+), 47 deletions(-) diff --git a/.github/release-branches.yml b/.github/release-branches.yml new file mode 100644 index 0000000000..afc0f14747 --- /dev/null +++ b/.github/release-branches.yml @@ -0,0 +1,46 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Active release branches and their release managers. This file is the single +# source of truth for the backport automation. Listing a branch here: +# - auto-labels new `fix:` PRs with the branch's `release/*` label +# (.github/workflows/backport-auto-label.yml), so backports are opt-out +# rather than easy-to-forget opt-in; +# - requests review from the branch's release manager on those PRs; +# - drives the post-merge backport (.github/workflows/direct-backport-push.yml): +# a green pre-merge backport check cherry-picks straight to the branch, a +# red one opens a draft backport PR assigned to the author with the manager +# as reviewer. +# +# The label name equals the branch name (e.g. branch `release/v1.2` <-> label +# `release/v1.2`). +# +# `actively-supporting` controls the default: only actively-supporting branches +# are auto-labeled on new `fix:` PRs. An inactive branch stays a valid backport +# target — a maintainer can still add its label by hand and the apply-check / +# post-merge backport run as usual — it just isn't offered by default. Omitting +# the field defaults to true. When a branch reaches end-of-life, drop its entry +# entirely. +# +# `manager` is a GitHub username. It may be omitted (leave the branch as a +# backport target without an auto-requested reviewer), but is recommended. +targets: + - branch: release/v1.2 + manager: xuang7 + actively-supporting: true + - branch: release/v1.1 + manager: bobbai00 + actively-supporting: false diff --git a/.github/scripts/create-backport-branch.sh b/.github/scripts/create-backport-branch.sh new file mode 100755 index 0000000000..68f22d9ece --- /dev/null +++ b/.github/scripts/create-backport-branch.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash + +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Builds and pushes the branch behind an auto-opened backport PR, used when the +# pre-merge backport check was red so a straight cherry-pick to the release +# branch is unsafe. The cherry-pick is committed even when it conflicts: the +# tree carries the conflict markers, and the human resolves them in the PR +# rather than starting the backport from scratch (the same approach the common +# backport bots take). +# +# Usage: create-backport-branch.sh <merge-sha> <target-branch> <pr-number> +# Writes to $GITHUB_OUTPUT (or stdout when unset): branch, version, +# had_conflict, conflict_files, subject. + +set -euo pipefail + +MERGE_SHA="${1:?merge sha is required}" +TARGET_BRANCH="${2:?target branch is required}" +PR_NUMBER="${3:?pr number is required}" + +log() { printf '[open-backport-pr %s] %s\n' "${TARGET_BRANCH}" "$*"; } +out() { printf '%s\n' "$*" >> "${GITHUB_OUTPUT:-/dev/stdout}"; } + +version="${TARGET_BRANCH#release/}" + +# Human-readable slug from the squash subject: drop the trailing "(#N)" and the +# conventional-commit "type(scope): " prefix, then kebab-case and truncate. +subject="$(git log -1 --format=%s "${MERGE_SHA}")" +desc="$(printf '%s' "${subject}" \ + | sed -E 's/ \(#[0-9]+\)$//; s/^[a-z]+(\([^)]*\))?!?: *//')" +slug="$(printf '%s' "${desc}" \ + | tr '[:upper:]' '[:lower:]' \ + | tr -c 'a-z0-9' '-' \ + | sed -E 's/-+/-/g; s/^-//; s/-$//' \ + | cut -c1-40 \ + | sed -E 's/-$//')" +[[ -n "${slug}" ]] || slug="backport" +branch="backport/${PR_NUMBER}-${slug}-${version}" +log "branch=${branch}" + +git config user.name "github-actions[bot]" +git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + +original_author="$(git log -1 --format='%an <%ae>' "${MERGE_SHA}")" +merge_message="$(git log -1 --format=%B "${MERGE_SHA}")" + +git fetch --no-tags origin "${TARGET_BRANCH}" + +# Feature-absent guard: if every file this commit modifies or deletes is +# missing on the target branch, the fix targets code that does not exist on +# the release (e.g. a fix for a main-only feature). Skip instead of opening a +# doomed PR. Added files don't count — a backportable fix may add files too. +preexisting="$(git diff-tree --no-commit-id --name-only -r --diff-filter=MD "${MERGE_SHA}")" +if [[ -n "${preexisting}" ]]; then + any_present=0 + while IFS= read -r path; do + [[ -z "${path}" ]] && continue + if git cat-file -e "origin/${TARGET_BRANCH}:${path}" 2>/dev/null; then + any_present=1 + break + fi + done <<< "${preexisting}" + if [[ "${any_present}" -eq 0 ]]; then + log "all modified files absent on ${TARGET_BRANCH}; feature not on release — skipping PR" + out "feature_absent=true" + out "version=${version}" + exit 0 + fi +fi +out "feature_absent=false" + +git checkout -B "${branch}" "origin/${TARGET_BRANCH}" + +# --no-commit keeps full control of the message/author in both the clean and +# the conflicted case. On conflict the working tree keeps the markers; staging +# everything turns the unmerged entries into a normal (dirty) commit. +had_conflict=false +conflict_files="" +if ! git cherry-pick --no-commit "${MERGE_SHA}"; then + had_conflict=true + conflict_files="$(git diff --name-only --diff-filter=U | tr '\n' ' ')" + log "cherry-pick conflicted in: ${conflict_files}" + git add -A +fi +# Clear any cherry-pick sequencer state so the manual commit below is clean. +git cherry-pick --quit 2>/dev/null || true + +new_message="$(printf '%s' "${merge_message}" \ + | python3 .github/scripts/compose-backport-message.py "${MERGE_SHA}")" +printf '%s\n' "${new_message}" | git commit -F - --author="${original_author}" + +git push --force origin "HEAD:${branch}" +log "pushed ${branch}=$(git rev-parse HEAD)" + +out "branch=${branch}" +out "version=${version}" +out "had_conflict=${had_conflict}" +out "conflict_files=${conflict_files}" +out "subject=${subject}" diff --git a/.github/scripts/release_branches.py b/.github/scripts/release_branches.py new file mode 100644 index 0000000000..f0f2e61d49 --- /dev/null +++ b/.github/scripts/release_branches.py @@ -0,0 +1,115 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Reads .github/release-branches.yml and emits the release-branch -> manager +# mapping as JSON, so the backport workflows have one parser instead of each +# hand-rolling YAML handling in shell/JS. Uses only the standard library +# (GitHub runners have python3 but not necessarily PyYAML), parsing the small, +# fixed schema this file uses: +# +# targets: +# - branch: release/v1.2 # inline comments allowed +# manager: xuang7 # optional +# actively-supporting: true # optional, defaults to true +# +# Usage: +# release_branches.py [path] -> JSON array +# [{"branch","manager","active"}] +# release_branches.py [path] --targets -> JSON array of branch names +# release_branches.py [path] --manager B -> manager for branch B (or empty) + +import json +import sys + +DEFAULT_PATH = ".github/release-branches.yml" + + +def _strip(value): + # Drop an inline "# ..." comment (the values here never contain '#'), + # surrounding whitespace, and optional quotes. + value = value.split("#", 1)[0].strip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'": + value = value[1:-1] + return value + + +def parse(path): + entries = [] + current = None + in_targets = False + with open(path, encoding="utf-8") as handle: + for raw in handle: + line = raw.rstrip("\n") + stripped = line.strip() + if not stripped or stripped.startswith("#"): + continue + + indent = len(line) - len(line.lstrip()) + if indent == 0: + # A new top-level key ends the targets block. + in_targets = stripped.split(":", 1)[0].strip() == "targets" + current = None + continue + if not in_targets: + continue + + item = stripped + if item.startswith("-"): + # `active` defaults to true; an entry is only inactive when it + # explicitly says so. + current = {"branch": "", "manager": "", "active": True} + entries.append(current) + item = item[1:].strip() + if not item: + continue + if current is None or ":" not in item: + continue + key, value = item.split(":", 1) + key = key.strip() + if key in ("branch", "manager"): + current[key] = _strip(value) + elif key == "actively-supporting": + current["active"] = _strip(value).lower() in ("true", "yes", "1") + + return [e for e in entries if e["branch"]] + + +def main(argv): + args = list(argv) + manager_of = None + mode = "entries" + if "--targets" in args: + args.remove("--targets") + mode = "targets" + if "--manager" in args: + idx = args.index("--manager") + manager_of = args[idx + 1] + del args[idx : idx + 2] + mode = "manager" + path = args[0] if args else DEFAULT_PATH + + entries = parse(path) + if mode == "targets": + print(json.dumps([e["branch"] for e in entries])) + elif mode == "manager": + match = next((e for e in entries if e["branch"] == manager_of), None) + print(match["manager"] if match else "") + else: + print(json.dumps(entries)) + + +if __name__ == "__main__": + main(sys.argv[1:]) diff --git a/.github/workflows/backport-auto-label.yml b/.github/workflows/backport-auto-label.yml new file mode 100644 index 0000000000..f10959b8c4 --- /dev/null +++ b/.github/workflows/backport-auto-label.yml @@ -0,0 +1,208 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Makes backporting opt-out instead of easy-to-forget opt-in: when a `fix:` PR +# targeting main opens, label it with every active release branch from +# .github/release-branches.yml and request review from each branch's release +# manager. The author (or a committer) removes a release/* label to decline the +# backport for that branch — a removal is remembered (via the timeline), so a +# later edit never silently re-adds a label the author took off. + +name: Backport Auto Label + +on: + pull_request_target: + types: + - opened + - reopened + - edited + +permissions: + contents: read + pull-requests: write + +jobs: + auto-label: + # Only PRs into main get backport labels. A PR already targeting a + # release/* branch is itself a backport and must not be re-labeled. + if: ${{ github.event.pull_request.base.ref == 'main' }} + runs-on: ubuntu-latest + steps: + # Check out the base commit (a trusted commit already on main) so the + # release-branches config and its parser come from the base repo, never + # from the (untrusted) PR head. pull_request_target runs with write + # permissions, so we must not execute PR-controlled code here. + - name: Checkout base + uses: actions/checkout@v7 + with: + ref: ${{ github.event.pull_request.base.sha }} + persist-credentials: false + + - name: Read release branches + id: targets + run: | + entries=$(python3 .github/scripts/release_branches.py .github/release-branches.yml) + echo "entries=${entries}" >> "$GITHUB_OUTPUT" + echo "Release branch targets: ${entries}" + + - name: Label fix PRs and request release-manager review + uses: actions/github-script@v9 + env: + ENTRIES: ${{ steps.targets.outputs.entries }} + with: + # Add the label with the fine-grained PAT (falling back to the + # default token) so the resulting `labeled` event retriggers the + # backport pre-merge check in Required Checks. Labels applied by the + # default GITHUB_TOKEN do not trigger downstream workflows. + github-token: ${{ secrets.AUTO_MERGE_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const entries = JSON.parse(process.env.ENTRIES || "[]"); + if (entries.length === 0) { + core.info("No release branches configured; nothing to label."); + return; + } + + const { owner, repo } = context.repo; + const pr = context.payload.pull_request; + const title = pr.title || ""; + + // Conventional-commit `fix:` (optional scope, optional `!`). Other + // types (feat/chore/docs/...) are not auto-backported. + if (!/^fix(\([^)]*\))?!?:/i.test(title)) { + core.info(`PR #${pr.number} title is not a fix (\"${title}\"); skipping.`); + return; + } + + const author = pr.user?.login; + const currentLabels = new Set((pr.labels || []).map((l) => l.name)); + + // Hard manual override: a maintainer marks a PR that must never be + // backported (e.g. it fixes a feature that only exists on main). + if (currentLabels.has("no-backport-needed")) { + core.info(`PR #${pr.number} has no-backport-needed; skipping all backport labels.`); + return; + } + + // A fix for a feature that does not exist on a release branch + // should not be backported there. The reliable, cheap signal: the + // files the PR *modifies* (not the ones it adds — a backportable + // fix may add files too) don't exist on that branch at all. If + // every modified file is absent, the feature isn't there, so skip + // the target. Any ambiguity (a modified file still exists, or the + // existence check errors) keeps the label and lets the apply-check + // decide — we would rather over-label than drop a real backport. + async function featureAbsentOnBranch(prNumber, branch) { + const files = await github.paginate(github.rest.pulls.listFiles, { + owner, repo, pull_number: prNumber, per_page: 100, + }); + const preexisting = files + .filter((f) => f.status !== "added") + .map((f) => (f.status === "renamed" ? f.previous_filename : f.filename)); + if (preexisting.length === 0) return false; // pure-add fix applies cleanly + for (const path of preexisting) { + try { + await github.rest.repos.getContent({ owner, repo, path, ref: branch }); + return false; // a modified file exists on the branch + } catch (e) { + if (e.status === 404) continue; + core.warning(`getContent ${path}@${branch} failed (${e.status ?? "?"}); assuming present.`); + return false; + } + } + return true; // every modified file is absent from the branch + } + + // Labels the author (or anyone) explicitly removed. Re-adding one + // would override a deliberate opt-out, so we skip those forever. + const timeline = await github.paginate( + github.rest.issues.listEventsForTimeline, + { owner, repo, issue_number: pr.number, per_page: 100 } + ); + const removed = new Set( + timeline + .filter((e) => e.event === "unlabeled" && e.label?.name) + .map((e) => e.label.name) + ); + + async function ensureLabelExists(name) { + try { + await github.rest.issues.getLabel({ owner, repo, name }); + } catch (e) { + if (e.status !== 404) throw e; + // Match the colour of the existing release/* labels. + await github.rest.issues.createLabel({ + owner, + repo, + name, + color: "ccebc2", + description: `back porting to ${name}`, + }); + core.info(`Created missing label ${name}.`); + } + } + + for (const entry of entries) { + const label = entry.branch; + const manager = entry.manager; + + // Only actively-supporting branches are auto-labeled. An inactive + // branch stays a valid manual target (label it by hand and the + // apply-check / post-merge backport still run) — just not by + // default. + if (!entry.active) { + core.info(`${label} is not actively-supporting; not auto-labeling.`); + continue; + } + if (currentLabels.has(label)) { + core.info(`PR #${pr.number} already has ${label}.`); + continue; + } + if (removed.has(label)) { + core.info(`${label} was removed on PR #${pr.number} (opt-out); not re-adding.`); + continue; + } + if (await featureAbsentOnBranch(pr.number, label)) { + core.info(`PR #${pr.number}: all modified files absent on ${label}; feature not on that release — skipping.`); + continue; + } + + await ensureLabelExists(label); + await github.rest.issues.addLabels({ + owner, + repo, + issue_number: pr.number, + labels: [label], + }); + core.info(`Added ${label} to PR #${pr.number}.`); + + if (manager && manager !== author) { + try { + await github.rest.pulls.requestReviewers({ + owner, + repo, + pull_number: pr.number, + reviewers: [manager], + }); + core.info(`Requested review from ${manager} for ${label}.`); + } catch (e) { + core.warning( + `Could not request review from ${manager} (status ${e.status ?? "?"}): ${e.message}` + ); + } + } else if (manager === author) { + core.info(`Release manager ${manager} is the author; skipping review request.`); + } + } diff --git a/.github/workflows/direct-backport-push.yml b/.github/workflows/direct-backport-push.yml index 2456db159b..58031e26e1 100644 --- a/.github/workflows/direct-backport-push.yml +++ b/.github/workflows/direct-backport-push.yml @@ -33,12 +33,26 @@ jobs: name: Discover direct backport targets runs-on: ubuntu-latest outputs: - entries: ${{ steps.discover.outputs.entries }} - has_targets: ${{ steps.discover.outputs.has_targets }} + push_entries: ${{ steps.discover.outputs.push_entries }} + pr_entries: ${{ steps.discover.outputs.pr_entries }} + has_push: ${{ steps.discover.outputs.has_push }} + has_pr: ${{ steps.discover.outputs.has_pr }} steps: - - name: Resolve merged PRs and green targets + - name: Checkout main + uses: actions/checkout@v7 + with: + fetch-depth: 1 + - name: Read release branches + id: release_branches + run: | + entries=$(python3 .github/scripts/release_branches.py .github/release-branches.yml) + echo "entries=${entries}" >> "$GITHUB_OUTPUT" + echo "Release branches: ${entries}" + - name: Resolve merged PRs and classify targets id: discover uses: actions/github-script@v9 + env: + RELEASE_BRANCHES: ${{ steps.release_branches.outputs.entries }} with: script: | const sha = context.sha; @@ -133,7 +147,10 @@ jobs: return null; } - async function greenTargetsFor(pullRequest, requestedTargets) { + // Collect every job across the PR head's Required Checks runs so + // both the pre-flight apply-check and the full backport build can + // be inspected per target. + async function jobsForPr(pullRequest) { const buildRuns = await github.paginate( github.rest.actions.listWorkflowRuns, { @@ -144,45 +161,88 @@ jobs: per_page: 100, } ); - if (buildRuns.length === 0) { - core.warning(`No Build workflow runs found for ${pullRequest.head.sha}.`); + core.warning(`No Required Checks runs found for ${pullRequest.head.sha}.`); return []; } - const allJobs = []; for (const run of buildRuns) { const jobs = await github.paginate( github.rest.actions.listJobsForWorkflowRun, - { - owner, - repo, - run_id: run.id, - per_page: 100, - } + { owner, repo, run_id: run.id, per_page: 100 } ); allJobs.push(...jobs); } + return allJobs; + } - return requestedTargets.filter((target) => { - const prefix = `backport (${target}) / `; - const targetJobs = allJobs.filter((job) => job.name.startsWith(prefix)); - // Treat skipped as green: precheck legitimately skips stacks - // based on PR labels, matching the Required Checks aggregator. - return ( - targetJobs.length > 0 && - targetJobs.every( - (job) => job.conclusion === "success" || job.conclusion === "skipped" - ) + const isDone = (job) => job.status === "completed"; + // Skipped legs count as green: precheck legitimately skips stacks + // by label; a real conflict shows up as an apply-check *failure*. + const isGreen = (job) => + job.conclusion === "success" || job.conclusion === "skipped"; + + // Classify each requested target into: + // push — safe to cherry-pick straight to the release branch: the + // pre-flight apply-check finished green AND no finished + // backport build leg failed. + // pr — needs a human: the apply-check finished red (conflict) + // or a finished build leg failed (applies but won't build). + // skip — no *completed* signal (nothing ran, or the PR merged + // before the checks finished); acting on an in-progress + // job could push a conflict or open a spurious PR, so we + // leave it (the scheduled reconciliation, if any, or a + // re-push will pick it up once checks are done). + function classifyTargets(allJobs, requestedTargets) { + const push = []; + const pr = []; + for (const target of requestedTargets) { + const applyJob = allJobs.find( + (job) => job.name === `backport-apply-check (${target})` ); - }); + const buildJobs = allJobs.filter((job) => + job.name.startsWith(`backport (${target}) / `) + ); + + if (applyJob && isDone(applyJob) && !isGreen(applyJob)) { + pr.push(target); // cherry-pick conflicts + continue; + } + if (buildJobs.some((job) => isDone(job) && !isGreen(job))) { + pr.push(target); // applies but does not build + continue; + } + + const applyOk = applyJob && isDone(applyJob) && isGreen(applyJob); + const anyBuildDone = buildJobs.some(isDone); + // Normal path: apply-check finished green. The `!applyJob` + // clause keeps older runs (before apply-check existed) working + // off the build legs alone during rollout. + if (applyOk || (!applyJob && anyBuildDone)) { + push.push(target); + } else { + core.warning(`PR: no completed backport signal for ${target}; skipping.`); + } + } + return { push, pr }; } + const releaseManagers = new Map( + JSON.parse(process.env.RELEASE_BRANCHES || "[]").map((e) => [ + e.branch, + e.manager || "", + ]) + ); + const commits = await listPushCommits(); core.info(`Push contains ${commits.length} commit(s).`); - // One matrix entry per (PR, target) pair, in commit order. - const entries = []; + // push_entries: (PR, target) pairs cherry-picked straight to the + // release branch, in commit order. + // pr_entries: (PR, target, manager) pairs that get an auto-opened + // draft backport PR for the author to finish. + const pushEntries = []; + const prEntries = []; const seenPrNumbers = new Set(); for (const commit of commits) { const pullRequest = @@ -196,10 +256,14 @@ jobs: } seenPrNumbers.add(pullRequest.number); + const labelNames = pullRequest.labels.map((label) => label.name); + if (labelNames.includes("no-backport-needed")) { + core.info(`PR #${pullRequest.number} has no-backport-needed; skipping.`); + continue; + } + const requestedTargets = [...new Set( - pullRequest.labels - .map((label) => label.name) - .filter((name) => /^release\/.+$/.test(name)) + labelNames.filter((name) => /^release\/.+$/.test(name)) )].sort(); if (requestedTargets.length === 0) { @@ -207,28 +271,39 @@ jobs: continue; } - const greenTargets = await greenTargetsFor(pullRequest, requestedTargets); - const skippedTargets = requestedTargets.filter((target) => !greenTargets.includes(target)); - if (skippedTargets.length > 0) { - core.warning(`PR #${pullRequest.number}: skipping targets without a successful Backport run: ${skippedTargets.join(", ")}`); - } + const allJobs = await jobsForPr(pullRequest); + const { push, pr } = classifyTargets(allJobs, requestedTargets); + core.info( + `PR #${pullRequest.number}: push=[${push.join(", ")}] pr=[${pr.join(", ")}]` + ); - for (const target of greenTargets) { - entries.push({ + for (const target of push) { + pushEntries.push({ + pr_number: pullRequest.number, + merge_sha: commit.sha, + target, + }); + } + for (const target of pr) { + prEntries.push({ pr_number: pullRequest.number, merge_sha: commit.sha, target, + manager: releaseManagers.get(target) || "", }); } } - core.info(`Backport entries: ${JSON.stringify(entries)}`); - core.setOutput("entries", JSON.stringify(entries)); - core.setOutput("has_targets", entries.length > 0 ? "true" : "false"); + core.info(`Push entries: ${JSON.stringify(pushEntries)}`); + core.info(`PR entries: ${JSON.stringify(prEntries)}`); + core.setOutput("push_entries", JSON.stringify(pushEntries)); + core.setOutput("pr_entries", JSON.stringify(prEntries)); + core.setOutput("has_push", pushEntries.length > 0 ? "true" : "false"); + core.setOutput("has_pr", prEntries.length > 0 ? "true" : "false"); push-backports: needs: discover - if: ${{ needs.discover.outputs.has_targets == 'true' }} + if: ${{ needs.discover.outputs.has_push == 'true' }} runs-on: ubuntu-latest name: "backport #${{ matrix.pr_number }} to ${{ matrix.target }}" strategy: @@ -238,7 +313,7 @@ jobs: # release branch. max-parallel: 1 matrix: - include: ${{ fromJson(needs.discover.outputs.entries) }} + include: ${{ fromJson(needs.discover.outputs.push_entries) }} steps: - name: Checkout main uses: actions/checkout@v7 @@ -672,3 +747,186 @@ jobs: } else { core.info("No PR number resolved — skipping PR comment."); } + + open-backport-pr: + needs: discover + if: ${{ needs.discover.outputs.has_pr == 'true' }} + runs-on: ubuntu-latest + name: "open backport PR #${{ matrix.pr_number }} to ${{ matrix.target }}" + strategy: + fail-fast: false + matrix: + include: ${{ fromJson(needs.discover.outputs.pr_entries) }} + steps: + - name: Checkout main + uses: actions/checkout@v7 + with: + fetch-depth: 0 + # AUTO_MERGE_TOKEN so the pushed branch and the opened PR retrigger + # CI on the release branch (GITHUB_TOKEN-authored pushes/PRs do not). + token: ${{ secrets.AUTO_MERGE_TOKEN || secrets.GITHUB_TOKEN }} + - name: Create backport branch + id: branch + env: + MERGE_SHA: ${{ matrix.merge_sha }} + TARGET_BRANCH: ${{ matrix.target }} + PR_NUMBER: ${{ matrix.pr_number }} + run: | + bash ./.github/scripts/create-backport-branch.sh \ + "${MERGE_SHA}" "${TARGET_BRANCH}" "${PR_NUMBER}" + - name: Open draft backport PR + uses: actions/github-script@v9 + env: + MERGE_SHA: ${{ matrix.merge_sha }} + TARGET_BRANCH: ${{ matrix.target }} + PR_NUMBER: ${{ matrix.pr_number }} + MANAGER: ${{ matrix.manager }} + BRANCH: ${{ steps.branch.outputs.branch }} + VERSION: ${{ steps.branch.outputs.version }} + HAD_CONFLICT: ${{ steps.branch.outputs.had_conflict }} + CONFLICT_FILES: ${{ steps.branch.outputs.conflict_files }} + SUBJECT: ${{ steps.branch.outputs.subject }} + FEATURE_ABSENT: ${{ steps.branch.outputs.feature_absent }} + with: + github-token: ${{ secrets.AUTO_MERGE_TOKEN || secrets.GITHUB_TOKEN }} + script: | + const { + MERGE_SHA, TARGET_BRANCH, PR_NUMBER, MANAGER, + BRANCH, VERSION, HAD_CONFLICT, CONFLICT_FILES, SUBJECT, + FEATURE_ABSENT, + } = process.env; + const { owner, repo } = context.repo; + const prNumber = Number(PR_NUMBER); + const hadConflict = HAD_CONFLICT === "true"; + const runUrl = + `${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`; + + // The branch step skipped this target: the fix changes only files + // that don't exist on the release branch, so there is nothing to + // backport. Record it on the original PR instead of opening a PR. + if (FEATURE_ABSENT === "true") { + core.info(`Feature absent on ${TARGET_BRANCH}; not opening a backport PR.`); + try { + await github.rest.issues.createComment({ + owner, repo, issue_number: prNumber, + body: + `Not backported to \`${TARGET_BRANCH}\`: the files this fix ` + + `changes do not exist there (the feature is not on that ` + + `release). Add \`no-backport-needed\` to silence this for ` + + `future targets.`, + }); + } catch (e) { + core.warning(`Could not comment on #${prNumber}: ${e.message}`); + } + try { + await github.rest.repos.createCommitStatus({ + owner, repo, sha: MERGE_SHA, state: "success", + context: `backport/${TARGET_BRANCH}`, + description: "Skipped: feature not on release branch", + }); + } catch (e) { + core.warning(`Could not set commit status: ${e.message}`); + } + return; + } + + // Inject the release version into the conventional-commit scope so + // the backport PR reads e.g. `fix(amber, v1.2): ...`. Falls back to + // suffixing the version if the subject is not conventional. + function backportTitle(subject, version) { + const m = subject.match(/^(\w+)(?:\(([^)]*)\))?(!)?:\s*(.*)$/); + if (!m) return `${subject} (${version})`; + const [, type, scope, bang, rest] = m; + const newScope = scope ? `${scope}, ${version}` : version; + return `${type}(${newScope})${bang || ""}: ${rest}`; + } + const title = backportTitle(SUBJECT, VERSION); + + // Idempotency: on a re-run the branch is force-pushed, so an open + // PR from the same head already reflects the new commit — don't + // open a duplicate. + const existing = await github.rest.pulls.list({ + owner, repo, state: "open", head: `${owner}:${BRANCH}`, + }); + let pr = existing.data[0]; + if (pr) { + core.info(`Backport PR already open: #${pr.number}; not duplicating.`); + } else { + const conflictList = CONFLICT_FILES.trim() + ? CONFLICT_FILES.trim().split(/\s+/).map((f) => `- \`${f}\``).join("\n") + : ""; + const body = [ + `Automated backport of #${prNumber} to \`${TARGET_BRANCH}\`.`, + ``, + hadConflict + ? `⚠️ The cherry-pick **conflicted** and was committed with conflict markers. ` + + `Resolve the conflicts on this branch, then mark the PR ready for review.` + : `The cherry-pick applied cleanly but the backported tree failed its ` + + `pre-merge build. Fix the build on this branch, then mark the PR ready for review.`, + conflictList ? `\n**Conflicting files:**\n${conflictList}` : ``, + ``, + `Source: ${MERGE_SHA} · [automation run](${runUrl})`, + ].join("\n"); + + pr = (await github.rest.pulls.create({ + owner, repo, base: TARGET_BRANCH, head: BRANCH, + title, body, draft: true, + })).data; + core.info(`Opened draft backport PR #${pr.number}.`); + } + + // Assign the original author (best-effort: external authors may not + // be assignable) and label the PR with the target branch. + let author = ""; + try { + author = (await github.rest.pulls.get({ + owner, repo, pull_number: prNumber, + })).data.user?.login || ""; + } catch (e) { + core.warning(`Could not fetch PR #${prNumber} author: ${e.message}`); + } + if (author) { + try { + await github.rest.issues.addAssignees({ + owner, repo, issue_number: pr.number, assignees: [author], + }); + } catch (e) { + core.warning(`Could not assign ${author}: ${e.message}`); + } + } + if (MANAGER && MANAGER !== author) { + try { + await github.rest.pulls.requestReviewers({ + owner, repo, pull_number: pr.number, reviewers: [MANAGER], + }); + } catch (e) { + core.warning(`Could not request review from ${MANAGER}: ${e.message}`); + } + } + + // Link the backport PR back from the original PR so it is not lost. + const prUrl = `${context.serverUrl}/${owner}/${repo}/pull/${pr.number}`; + try { + await github.rest.issues.createComment({ + owner, repo, issue_number: prNumber, + body: + `Backport to \`${TARGET_BRANCH}\` needs manual work — opened ` + + `draft PR #${pr.number} (${prUrl})` + + (author ? `, assigned to @${author}` : "") + `.`, + }); + } catch (e) { + core.warning(`Could not comment on #${prNumber}: ${e.message}`); + } + + // A failure commit status on the merge commit keeps the "needs + // backport" signal visible on main next to the successful ones. + try { + await github.rest.repos.createCommitStatus({ + owner, repo, sha: MERGE_SHA, state: "failure", + context: `backport/${TARGET_BRANCH}`, + description: `Draft backport PR #${pr.number} opened`, + target_url: prUrl, + }); + } catch (e) { + core.warning(`Could not set commit status: ${e.message}`); + } diff --git a/.github/workflows/required-checks.yml b/.github/workflows/required-checks.yml index 8ee83d3813..6194d621cd 100644 --- a/.github/workflows/required-checks.yml +++ b/.github/workflows/required-checks.yml @@ -216,9 +216,16 @@ jobs: core.setOutput("run_infra", runInfra ? "true" : "false"); core.setOutput("run_pyright_language_service", runPyrightLanguageService ? "true" : "false"); - // Backport targets: all current release/* labels on the PR. - const targets = [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort(); - if (targets.length === 0) { + // Backport targets: all current release/* labels on the PR, unless + // `no-backport-needed` vetoes backporting entirely (e.g. a fix for + // a feature that does not exist on any release branch). The veto + // wins even if release/* labels are also present, so adding it + // mid-review cancels the backport apply-check/build on the re-run. + let targets = [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort(); + if (labels.includes("no-backport-needed")) { + core.info("no-backport-needed present; skipping all backport targets."); + targets = []; + } else if (targets.length === 0) { core.info("No backport targets on PR."); } else { core.info(`Backport targets: ${targets.join(", ")}`); @@ -275,13 +282,70 @@ jobs: run_pyright_language_service: ${{ needs.precheck.outputs.run_pyright_language_service == 'true' }} secrets: inherit - backport: + # Fast pre-flight: does the change even cherry-pick onto each release branch? + # This is a git-only job (no build), so a conflict is reported in seconds and + # gates the expensive backport build below — a conflicting target never spins + # up the full stack matrix. Post-merge, Direct Backport Push reads this job's + # result as the authoritative "conflicts / applies" signal. + backport-apply-check: needs: precheck if: ${{ needs.precheck.outputs.backport_targets != '[]' }} strategy: fail-fast: false matrix: target: ${{ fromJson(needs.precheck.outputs.backport_targets) }} + runs-on: ubuntu-latest + steps: + - name: Checkout PR head + uses: actions/checkout@v7 + with: + ref: refs/pull/${{ github.event.pull_request.number }}/head + fetch-depth: 0 + - name: Dry-run cherry-pick onto ${{ matrix.target }} + run: | + bash ./.github/scripts/prepare-backport-checkout.sh \ + "${{ matrix.target }}" \ + "${{ format('{0}..{1}', github.event.pull_request.base.sha, github.event.pull_request.head.sha) }}" + + # Collect the targets whose pre-flight cherry-pick succeeded. always() so a + # conflict on one target still lets the others build; an all-conflict run + # yields an empty list and skips the backport build entirely. + backport-buildable: + needs: [precheck, backport-apply-check] + if: ${{ always() && needs.precheck.outputs.backport_targets != '[]' }} + runs-on: ubuntu-latest + outputs: + targets: ${{ steps.collect.outputs.targets }} + steps: + - name: Collect targets that applied cleanly + id: collect + uses: actions/github-script@v9 + env: + REQUESTED: ${{ needs.precheck.outputs.backport_targets }} + with: + script: | + const requested = JSON.parse(process.env.REQUESTED || "[]"); + const { owner, repo } = context.repo; + const jobs = await github.paginate( + github.rest.actions.listJobsForWorkflowRun, + { owner, repo, run_id: context.runId, per_page: 100 } + ); + const buildable = requested.filter((target) => { + const job = jobs.find( + (j) => j.name === `backport-apply-check (${target})` + ); + return job && job.conclusion === "success"; + }); + core.info(`Buildable backport targets: ${JSON.stringify(buildable)}`); + core.setOutput("targets", JSON.stringify(buildable)); + + backport: + needs: [precheck, backport-buildable] + if: ${{ needs.backport-buildable.outputs.targets != '' && needs.backport-buildable.outputs.targets != '[]' }} + strategy: + fail-fast: false + matrix: + target: ${{ fromJson(needs.backport-buildable.outputs.targets) }} uses: ./.github/workflows/build.yml with: checkout_ref: refs/pull/${{ github.event.pull_request.number }}/head @@ -302,7 +366,13 @@ jobs: required-checks: # Do not rename this job — its display name is referenced in .asf.yaml. name: Required Checks - needs: [precheck, build, backport] + # The backport job is deliberately NOT a dependency here: it is an advisory + # signal, not a merge gate. A red pre-merge backport (a cherry-pick that + # conflicts, or a backported tree that fails to build) must not block the + # PR from landing on main — instead Direct Backport Push opens a draft + # backport PR for the author after merge. Keeping backport out of `needs` + # also lets this aggregator report as soon as build finishes. + needs: [precheck, build] if: always() runs-on: ubuntu-latest steps: @@ -311,7 +381,6 @@ jobs: declare -A results=( [precheck]="${{ needs.precheck.result }}" [build]="${{ needs.build.result }}" - [backport]="${{ needs.backport.result }}" ) failed=0 for job in "${!results[@]}"; do
