This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-6824-0ba5d1a22a3baaa61a36d1bbb8c4573d40e89b5c in repository https://gitbox.apache.org/repos/asf/texera.git
commit dda9e830f54e0fa74dcf0fd5d41a3c2e1a0c989c Author: Xinyuan Lin <[email protected]> AuthorDate: Thu Jul 23 04:58:13 2026 -0700 ci: discover staged Codecov flags instead of a hardcoded matrix (#6824) ### What changes were proposed in this PR? Follow-up to #6730. The deferred `Codecov Upload` workflow hardcoded all 11 flags in its matrix and tried to download each from the source run. On a label-gated PR — e.g. a frontend-only PR that only stages `codecov-frontend` — the other 10 legs hit `##[error] Artifact not found for name: codecov-<flag>`, which looks like a name mismatch; and because the download used `continue-on-error: true`, a genuinely failed download was swallowed, the leg went green, and `notify-failure` never fired. **Fix:** add a `discover` job that lists the source run's *actual* `codecov-*` artifacts and drives the upload matrix from that (`fromJSON`), and drop `continue-on-error` on the download. - **No more spurious errors** — only the flags the source run actually built are processed; a frontend-only PR runs one clean leg. - **Failures surface** — a listed artifact that then fails to download or upload fails the leg, so `notify-failure` fires (previously it went silently green). - Also removes the hardcoded flag list, so adding a `platform` service no longer needs a matching matrix row here. Observed on the first live run after #6730 merged ([run 29984668372](https://github.com/apache/texera/actions/runs/29984668372), triggered by a frontend-only PR's checks): `frontend` uploaded fine, but 10 legs logged `Artifact not found` for the label-gated flags and the run still went green. ### Any related issues, documentation, discussions? Follow-up to #6730 (reported on #6685). ### How was this PR tested? - `workflow_run` only activates from the copy of the file on the default branch, so this can't run on its own PR. Validated by YAML lint and an adversarial review of the discover → upload → notify flow: empty-flags cleanly skips (no failure, no notify), the dynamic `fromJSON` matrix handles one or many flags, a download/upload failure now fails the leg and fires `notify-failure`, `amber-integration` (results-only) skips the coverage step via `hashFiles`, and the `workflow_dispatch` path resolves the run id. - The `workflow_dispatch` (`run_id`) entry remains for manual post-merge validation against a finished *Required Checks* run. ### Was this PR authored or co-authored using generative AI tooling? Yes. Generated-by: Claude Code (Opus 4.8 [1M context]) --- .github/workflows/build.yml | 5 +- .github/workflows/codecov-upload.yml | 148 ++++++++++++++++++++--------------- 2 files changed, 85 insertions(+), 68 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 34eeff6ae3..5084638c0e 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -787,9 +787,8 @@ jobs: CC_BRANCH: ${{ github.event.pull_request.head.ref || github.ref_name }} run: bash .github/scripts/stage-codecov.sh - name: Upload ${{ matrix.service }} Codecov artifact - # Adding a service to the matrix above also needs a matching - # { flag: <service>, coverage: true } row in codecov-upload.yml, or this - # artifact is downloaded by nothing and the service's coverage is dropped. + # codecov-upload.yml discovers codecov-* artifacts dynamically, so a new + # platform service here is picked up automatically — no matrix row to sync. if: inputs.backport_target_branch == '' && !cancelled() uses: actions/upload-artifact@v4 with: diff --git a/.github/workflows/codecov-upload.yml b/.github/workflows/codecov-upload.yml index b43e22906e..fbe442cd38 100644 --- a/.github/workflows/codecov-upload.yml +++ b/.github/workflows/codecov-upload.yml @@ -27,17 +27,20 @@ # # Each build job stages its reports as a `codecov-<flag>` artifact with a uniform # layout — coverage/ (coverage reports) and results/ (JUnit XMLs) plus pr-number / -# commit-sha / branch identifier files. The matrix below re-uploads each flag with -# `directory:` scoping so Codecov's search is confined to that flag's files (this is -# what lets the amber flag's ~8 same-basename module jacoco.xml reports upload without -# a fragile explicit file list). Codecov requires one upload per flag (flags in a -# single upload apply to all its files, and carryforward needs the flag at upload -# time), so the flags cannot be collapsed into fewer calls — the win here is the -# token, not fewer uploads. +# commit-sha / branch identifier files. The `discover` job lists which codecov-* +# artifacts the source run ACTUALLY produced (a label-gated PR only builds some of +# them) and drives the upload matrix from that, so we never chase a flag that wasn't +# built — that avoids the spurious "artifact not found" errors and lets the download +# fail loudly for a flag that should be there. Each flag re-uploads with `directory:` +# scoping so Codecov's search is confined to that flag's files (this is what lets the +# amber flag's ~8 same-basename module jacoco.xml reports upload without a fragile +# explicit file list). Codecov requires one upload per flag (flags in a single upload +# apply to all its files, and carryforward needs the flag at upload time), so the +# flags cannot be collapsed into fewer calls — the win is the token, not fewer uploads. # # NOTE: `workflow_run` only fires from the copy of this file on the default branch, so -# this has NO effect until merged to main. After merge, exercise it manually via the -# workflow_dispatch entry below, passing a finished "Required Checks" run id. +# a change here has NO effect until merged to main. After merge, exercise it manually +# via the workflow_dispatch entry below, passing a finished "Required Checks" run id. name: Codecov Upload @@ -55,66 +58,80 @@ on: permissions: contents: read - actions: read # download-artifact needs this to read artifacts from another run + actions: read # list + download artifacts from the source run jobs: - upload: - # Skip only a cancelled/skipped upstream; still run on failure so partial - # coverage is reported (mirrors the build's always()/!cancelled() staging). + discover: + # Which codecov-<flag> artifacts did the source run actually build? A label-gated + # PR only stages some of them, so we list what exists and upload exactly those — + # rather than a hardcoded flag set that throws "artifact not found" for the rest. + # Skip only a cancelled/skipped upstream; still run on failure so partial coverage + # is reported (mirrors the build's always()/!cancelled() staging). if: >- github.event_name == 'workflow_dispatch' || (github.event.workflow_run.conclusion != 'cancelled' && github.event.workflow_run.conclusion != 'skipped') runs-on: ubuntu-latest - # Job-level so the group can key on matrix.flag: one upload pass per source - # run + flag, and a re-run of the source supersedes the previous attempt. + outputs: + flags: ${{ steps.list.outputs.flags }} + steps: + - name: List codecov-* artifacts in the source run + id: list + uses: actions/github-script@v9 + env: + # Passed via env (not interpolated into the script body) so a + # workflow_dispatch-supplied run_id can't inject into the JS. + SOURCE_RUN_ID: ${{ github.event.workflow_run.id || inputs.run_id }} + with: + script: | + const runId = Number(process.env.SOURCE_RUN_ID); + if (!Number.isInteger(runId) || runId <= 0) { + core.setFailed(`Invalid source run id: '${process.env.SOURCE_RUN_ID}' (expected a positive integer)`); + return; + } + const arts = await github.paginate(github.rest.actions.listWorkflowRunArtifacts, { + owner: context.repo.owner, repo: context.repo.repo, run_id: runId, per_page: 100, + }); + const flags = [...new Set( + arts.map(a => a.name) + .filter(n => n.startsWith('codecov-')) + .map(n => n.slice('codecov-'.length)) + )].sort(); + core.info(`Source run ${runId} staged codecov flags: ${flags.join(', ') || '(none)'}`); + core.setOutput('flags', JSON.stringify(flags)); + + upload: + needs: discover + # Nothing to do when the source produced no codecov-* artifacts (also avoids an + # empty matrix, which is an error). + if: ${{ needs.discover.outputs.flags != '' && needs.discover.outputs.flags != '[]' }} + runs-on: ubuntu-latest + # Job-level so the group can key on matrix.flag: one upload pass per source run + + # flag, and a re-run of the source supersedes the previous attempt. concurrency: group: codecov-upload-${{ github.event.workflow_run.id || inputs.run_id }}-${{ matrix.flag }} cancel-in-progress: false strategy: fail-fast: false matrix: - # flag = artifact suffix (codecov-<flag>); coverage=false for the - # test-results-only flag (amber-integration has no coverage report). - # - # KEEP IN SYNC with the coverage-producing jobs in build.yml: one row per - # codecov-<flag> artifact staged there — frontend, amber, amber-integration, - # pyamber, agent-service, and every service in build.yml's `platform` matrix. - # A flag staged in build.yml but missing here is downloaded by nothing and - # its coverage is silently dropped (the reverse — a row here with no artifact - # — is safe: the download continue-on-errors and the leg skips). - include: - - { flag: frontend, coverage: true } - - { flag: amber, coverage: true } - - { flag: amber-integration, coverage: false } - - { flag: pyamber, coverage: true } - - { flag: agent-service, coverage: true } - - { flag: config-service, coverage: true } - - { flag: access-control-service, coverage: true } - - { flag: file-service, coverage: true } - - { flag: computing-unit-managing-service, coverage: true } - - { flag: workflow-compiling-service, coverage: true } - - { flag: notebook-migration-service, coverage: true } + # `|| '[]'` guards matrix evaluation if the output is ever an empty string + # (discover skipped/failed); the job `if` above still skips the empty case. + flag: ${{ fromJSON(needs.discover.outputs.flags || '[]') }} steps: - - name: Resolve source run id - id: src - shell: bash - run: echo "run_id=${{ github.event.workflow_run.id || inputs.run_id }}" >> "$GITHUB_OUTPUT" - - name: Download staged coverage - id: dl - continue-on-error: true # absent when this flag's job was label-gated out - # Layout: stage-codecov.sh roots the artifact at cc/ (coverage/, results/, - # and the pr-number/commit-sha/branch txt files), so downloading to path: cc - # restores cc/coverage/**, cc/results/**, and cc/pr-number.txt — the paths - # the steps below reference. Keep that rooting if the staging script changes. + - name: Download staged coverage for ${{ matrix.flag }} + # No continue-on-error: discover confirmed this artifact exists, so a download + # failure here is a real problem that must fail the leg and trip notify-failure + # (the old design swallowed the failure and went green with no signal). + # Layout: stage-codecov.sh roots the artifact at cc/ (coverage/, results/, and + # the pr-number/commit-sha/branch txt files), so downloading to path: cc + # restores cc/coverage/**, cc/results/**, and cc/pr-number.txt. uses: actions/download-artifact@v4 with: name: codecov-${{ matrix.flag }} path: cc - run-id: ${{ steps.src.outputs.run_id }} + run-id: ${{ github.event.workflow_run.id || inputs.run_id }} github-token: ${{ github.token }} - name: Read coverage metadata - if: steps.dl.outcome == 'success' id: meta shell: bash # Artifact content comes from a (possibly fork) PR build, so sanitize each @@ -124,12 +141,10 @@ jobs: echo "sha=$(cat cc/commit-sha.txt 2>/dev/null | tr -cd '0-9a-fA-F')" >> "$GITHUB_OUTPUT" echo "branch=$(cat cc/branch.txt 2>/dev/null | tr -cd '[:alnum:]._/:-')" >> "$GITHUB_OUTPUT" - name: Upload ${{ matrix.flag }} coverage to Codecov - # fail_ci_if_error: true so a genuine upload failure fails this leg and - # trips the notify-failure job below (Codecov posts its own comment only on - # success, so a failed coverage upload would otherwise leave the PR with no - # report and no signal). A label-gated-absent flag is not a failure — the - # download continue-on-errors and this step is skipped via the guards above. - if: steps.dl.outcome == 'success' && matrix.coverage && hashFiles('cc/coverage/**') != '' + # Every flag has coverage except the results-only amber-integration; hashFiles + # skips it there (its artifact has no coverage/ dir). fail_ci_if_error: true so + # a genuine upload failure fails the leg and trips notify-failure. + if: hashFiles('cc/coverage/**') != '' uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0 with: token: ${{ secrets.CODECOV_TOKEN }} @@ -140,10 +155,7 @@ jobs: override_branch: ${{ steps.meta.outputs.branch }} override_pr: ${{ steps.meta.outputs.pr }} - name: Upload ${{ matrix.flag }} test results to Codecov - # fail_ci_if_error: true here too (per review) so a test-results upload error - # also trips notify-failure instead of being silently dropped. Still guarded - # on hashFiles, so an absent / label-gated flag simply skips. - if: steps.dl.outcome == 'success' && hashFiles('cc/results/**') != '' + if: hashFiles('cc/results/**') != '' uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0 with: token: ${{ secrets.CODECOV_TOKEN }} @@ -156,11 +168,11 @@ jobs: override_pr: ${{ steps.meta.outputs.pr }} notify-failure: - # If any upload above failed (coverage or test-results), leave a comment on the - # PR so a human can re-dispatch it. Codecov posts its own comment on SUCCESS, so - # we only cover the failure case (per #6730 review). A delayed / queued upload - # posts nothing — the absence of a Codecov comment then signals a stuck run. - needs: upload + # If discovery or any upload above failed, comment on the PR so a human can + # re-dispatch it. Codecov posts its own comment on SUCCESS, so we only cover the + # failure case (per #6730 review). A delayed / queued upload posts nothing — the + # absence of a Codecov comment then signals a stuck run. + needs: [discover, upload] if: ${{ failure() && github.event_name == 'workflow_run' }} runs-on: ubuntu-latest permissions: @@ -169,7 +181,13 @@ jobs: pull-requests: write # comment on the PR steps: - name: Download coverage metadata - id: dl + # Best-effort (opposite of the upload job's download, which fails hard): this + # only fetches a codecov-* artifact to read the PR number for the comment. If + # it can't (e.g. the source run produced no artifacts, so the pattern matches + # nothing and download-artifact errors), continue-on-error lets the job fall + # through to the `if: pr != ''` guard and skip commenting, rather than + # hard-failing notify-failure itself — which isn't a required check, so its + # failure would be invisible noise rather than a signal. continue-on-error: true uses: actions/download-artifact@v4 with:
