This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-7042-965786f2a51dc3043b6342fd539f0fddc7f00e53 in repository https://gitbox.apache.org/repos/asf/texera.git
commit 9d4f69a30f8415c8e53b65a2d619779ab80b3ad5 Author: Yicong Huang <[email protected]> AuthorDate: Wed Jul 29 15:32:18 2026 -0400 ci: speed up merge queue via HEADGREEN and stack selection (#7042) ### What changes were proposed in this PR? Two complementary merge-queue speedups. **1. Retune the `merge_queue` ruleset in `.asf.yaml`:** | Parameter | Before | After | Why | | --- | --- | --- | --- | | `grouping_strategy` | ALLGREEN | HEADGREEN | Merge a whole batch on one green head-group run instead of one green run per queued PR; also cuts flaky-test exposure from N draws per batch to 1. | | `max_entries_to_build` | 5 | 2 | Under HEADGREEN, non-head speculative builds no longer gate merging; keep one in flight for pipelining, save shared ASF runner quota (the pool has been at its 900-job limit lately; policy caps a project at 20 concurrent jobs, and one full group is already ~20). | | `min_entries_to_merge` | 1 | 2 | Batch PRs so one full CI run merges several at peak times. | | `min_entries_to_merge_wait_minutes` | 5 | 3 | Short batching window; off-peak a lone PR waits at most 3 minutes. | | `check_response_timeout_minutes` | 30 | 45 | Headroom for runner-queue delays so a late-reporting check is not misjudged as failed. | **2. Smarter stack selection for `merge_group` runs (precheck.yml).** Previously merge groups ran every stack unconditionally. Now, per group: ``` merge_group ├─ resolve the group's PRs from squash messages "(#N)" (same convention as direct-backport-push.yml) ├─ tier 1: group tree == a PR head tree that already passed Required Checks │ → byte-identical content was just validated → skip every stack ├─ tier 2: otherwise, union the labels of every PR in the group and apply │ the same LABEL_STACKS filter a PR event gets └─ any resolution failure → fail open, run every stack ``` Tier 1 typically fires for a single fresh PR based on the current `main` tip (or a stacked PR containing the group's other entries) — tree equality makes the skip content-exact, not heuristic. Backport targets stay PR-event-only; merge-group backports are still discovered post-merge by `direct-backport-push.yml`, which already iterates every commit in a multi-PR push. ### Any related issues, documentation, discussions? Closes #7041. Ruleset parameters follow the GitHub rulesets `merge_queue` schema: https://docs.github.com/en/rest/repos/rules ### How was this PR tested? - Config-only + workflow-logic change; `.asf.yaml` and both workflows pass YAML parsing, and the embedded github-script block passes `node --check`. - The merge-group path fails open on any API/parse error (single try/catch → full run), so a bad lookup can never skip CI silently. - Real merge_group behavior can only be observed on `main`; will watch the first few queue batches after merge (skip/label-union decisions are logged in the precheck step). ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Fable 5) --------- Co-authored-by: Claude Fable 5 <[email protected]> --- .asf.yaml | 10 +-- .github/workflows/precheck.yml | 125 ++++++++++++++++++++++++++++++++-- .github/workflows/required-checks.yml | 6 +- 3 files changed, 129 insertions(+), 12 deletions(-) diff --git a/.asf.yaml b/.asf.yaml index 8a23cb6f45..c0f520b666 100644 --- a/.asf.yaml +++ b/.asf.yaml @@ -83,12 +83,12 @@ github: - type: merge_queue parameters: merge_method: SQUASH - max_entries_to_build: 5 - min_entries_to_merge: 1 + max_entries_to_build: 2 + min_entries_to_merge: 2 max_entries_to_merge: 5 - min_entries_to_merge_wait_minutes: 5 - grouping_strategy: ALLGREEN - check_response_timeout_minutes: 30 + min_entries_to_merge_wait_minutes: 3 + grouping_strategy: HEADGREEN + check_response_timeout_minutes: 45 - type: pull_request parameters: allowed_merge_methods: diff --git a/.github/workflows/precheck.yml b/.github/workflows/precheck.yml index c98589060b..4ce1d24092 100644 --- a/.github/workflows/precheck.yml +++ b/.github/workflows/precheck.yml @@ -19,9 +19,12 @@ # Checks (backport preflight/build) call, so the label -> stack mapping and # the labeler-wait live in exactly one place. On PR events it waits for the # Pull Request Labeler to finish, then maps the PR's labels to the build -# stacks each requires and to the release/* backport targets. Non-PR events -# (push / merge_group / workflow_dispatch) run every stack and request no -# backports. +# stacks each requires and to the release/* backport targets. Push and +# workflow_dispatch events run every stack. Merge groups resolve the PRs in +# the group and either skip every stack (group tree identical to a PR head +# that already passed Required Checks) or apply the same label -> stack +# filter using the union of the group's PR labels; resolution failures fall +# open to a full run. Non-PR events request no backports. name: Precheck on: @@ -50,6 +53,9 @@ on: permissions: checks: read + # contents: read backs the git-data commit lookups (tree SHAs) in the + # merge-queue fast path. + contents: read pull-requests: read jobs: @@ -100,6 +106,97 @@ jobs: const eventName = context.eventName; let labels = []; + // When true, the LABEL_STACKS filter below applies to `labels`; + // when false the event runs every stack. + let applyLabelFilter = false; + + // Merge-queue path, two tiers, evaluated per group: + // + // 1. Fast path — a merge group whose squash result has the same + // git tree as a PR head that already passed Required Checks + // would re-validate byte-identical content: skip every stack. + // Tree equality only holds for a single-entry group whose PR + // was based on the then-current main (or a stacked PR that + // contains the group's other entries), so the skip is + // content-exact rather than heuristic. + // 2. Label union — otherwise, resolve every PR in the group from + // its squash commit message (PR_TITLE_AND_DESC squash titles + // end with "(#N)", same convention direct-backport-push.yml + // relies on) and apply the same label -> stack filter a PR + // event gets, using the union of all the group's PR labels. + // + // Any resolution failure falls open to a full run. + let mergeGroupAlreadyValidated = false; + if (eventName === "merge_group") { + try { + const { owner, repo } = context.repo; + const mg = context.payload.merge_group; + const { data: comparison } = await github.rest.repos.compareCommits({ + owner, + repo, + base: mg.base_sha, + head: mg.head_sha, + }); + const prNumbers = comparison.commits.map((c) => { + const m = c.commit.message.split("\n", 1)[0].match(/\(#(\d+)\)\s*$/); + if (!m) throw new Error(`no PR number in the message of ${c.sha}`); + return Number(m[1]); + }); + if (prNumbers.length === 0) { + throw new Error(`no commits between ${mg.base_sha} and ${mg.head_sha}`); + } + const pullRequests = []; + for (const pullNumber of prNumbers) { + const { data: pr } = await github.rest.pulls.get({ + owner, + repo, + pull_number: pullNumber, + }); + pullRequests.push(pr); + } + + // Tier 1: tree match against the newest entry's PR head. + const headPr = pullRequests[pullRequests.length - 1]; + const mgTree = ( + await github.rest.git.getCommit({ owner, repo, commit_sha: mg.head_sha }) + ).data.tree.sha; + const prTree = ( + await github.rest.git.getCommit({ owner, repo, commit_sha: headPr.head.sha }) + ).data.tree.sha; + if (mgTree === prTree) { + // The aggregator job in required-checks.yml publishes the + // "Required Checks" check run on the PR head. + const { data: checkData } = await github.rest.checks.listForRef({ + owner, + repo, + ref: headPr.head.sha, + check_name: "Required Checks", + }); + const check = checkData.check_runs[0]; + if (check && check.status === "completed" && check.conclusion === "success") { + mergeGroupAlreadyValidated = true; + core.info( + `Merge group tree matches PR #${headPr.number} head ${headPr.head.sha}, which already passed Required Checks; skipping all stacks.` + ); + } + } + + // Tier 2: label union across the group's PRs. + if (!mergeGroupAlreadyValidated) { + labels = [...new Set(pullRequests.flatMap((pr) => pr.labels.map((l) => l.name)))]; + applyLabelFilter = true; + core.info( + `Merge group PRs: ${prNumbers.map((n) => `#${n}`).join(", ")}; label union: ${labels.join(", ") || "(none)"}` + ); + } + } catch (e) { + core.warning(`Merge-group PR resolution failed (${e.message}); running all stacks.`); + labels = []; + applyLabelFilter = false; + mergeGroupAlreadyValidated = false; + } + } + if (eventName === "pull_request") { // Re-fetch labels: the labeler may have just added some. const { data: pr } = await github.rest.pulls.get({ @@ -108,6 +205,7 @@ jobs: pull_number: context.payload.pull_request.number, }); labels = pr.labels.map((l) => l.name); + applyLabelFilter = true; core.info(`PR labels: ${labels.join(", ") || "(none)"}`); } @@ -183,7 +281,7 @@ jobs: let runInfra = true; let runPyrightLanguageService = true; - if (eventName === "pull_request") { + if (applyLabelFilter) { const stacks = new Set(); for (const label of labels) { for (const stack of LABEL_STACKS[label] || []) { @@ -204,6 +302,18 @@ jobs: ); } + if (mergeGroupAlreadyValidated) { + runFrontend = false; + runAmber = false; + runAmberIntegration = false; + runPlatform = false; + runPlatformIntegration = false; + runPyamber = false; + runAgentService = false; + runInfra = false; + runPyrightLanguageService = false; + } + core.setOutput("run_frontend", runFrontend ? "true" : "false"); core.setOutput("run_amber", runAmber ? "true" : "false"); core.setOutput("run_amber_integration", runAmberIntegration ? "true" : "false"); @@ -219,7 +329,12 @@ jobs: // a feature that does not exist on any release branch). The veto // wins even if release/* labels are also present, so adding it // mid-review cancels the backport apply-check/build on the re-run. - let targets = [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort(); + // PR events only: merge_group also populates `labels` now, but its + // backports are discovered post-merge by direct-backport-push.yml. + let targets = + eventName === "pull_request" + ? [...new Set(labels.filter((n) => /^release\/.+$/.test(n)))].sort() + : []; if (labels.includes("no-backport-needed")) { core.info("no-backport-needed present; skipping all backport targets."); targets = []; diff --git a/.github/workflows/required-checks.yml b/.github/workflows/required-checks.yml index f2ee2efb89..ae47606443 100644 --- a/.github/workflows/required-checks.yml +++ b/.github/workflows/required-checks.yml @@ -43,8 +43,10 @@ concurrency: cancel-in-progress: ${{ github.ref != 'refs/heads/main' && !startsWith(github.ref, 'refs/heads/release/') }} jobs: - # Decide which build stacks this event needs (label union on PR events; all - # stacks on push/merge_group/dispatch). Shared with Backport Checks via the + # Decide which build stacks this event needs (label union on PR events and + # merge groups — a merge group unions the labels of every PR it contains, + # and skips every stack when its tree matches an already-green PR head; + # all stacks on push/dispatch). Shared with Backport Checks via the # reusable precheck workflow so the label -> stack mapping lives in one place # (.github/workflows/precheck.yml). The backport_targets output is unused # here — backport preflight/build moved to the Backport Checks workflow.
