This is an automated email from the ASF dual-hosted git repository.
Yicong-Huang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new 7d3b55bd3b refactor(ci): consolidate Build matrix into a single
reusable workflow (#4624)
7d3b55bd3b is described below
commit 7d3b55bd3b02a4420f428b8b275664911c15d073
Author: Yicong Huang <[email protected]>
AuthorDate: Fri May 1 16:14:42 2026 -0700
refactor(ci): consolidate Build matrix into a single reusable workflow
(#4624)
### What changes were proposed in this PR?
The Build matrix jobs (`frontend`, `scala`, `python`, `agent-service`)
were duplicated between `github-action-build.yml` and
`reusable-build.yml`, and the two had drifted — `reusable-build.yml` was
missing the recent license-check additions (npm bundle check,
pip-licenses manifest, bundled-jar diff against LICENSE-binary,
agent-service license manifest). Net change: **+238 / −416** lines.
- Rename `reusable-build.yml` → `build.yml` (workflow name `Build`). It
is now the single source of truth for the matrix steps, with the
license-check additions ported in.
- Rename `github-action-build.yml` → `required-checks.yml` (workflow
name `Required Checks`). Replace the four inline matrix jobs with a
single `build:` caller that `uses: ./.github/workflows/build.yml`. The
`backport:` caller is unchanged; the `Required Checks` aggregator job's
`needs:` shrinks from `[precheck, frontend, scala, python,
agent-service, backport]` to `[precheck, build, backport]`.
- Update `direct-backport-push.yml`'s `workflow_id` reference to the new
filename.
`.asf.yaml` continues to require only `Required Checks`, so the
display-name change (matrix children gain a `build /` prefix) does not
affect branch protection.
### Any related issues, documentation, discussions?
Closes #4623
### How was this PR tested?
YAML parses locally for all three modified workflow files. Step parity
between the new `build.yml` and the previous inline
`github-action-build.yml` matrix jobs verified by side-by-side diff. The
job will be exercised on this PR itself; matrix children appear under
`Required Checks / build / …` and the `backport:` matrix continues to
appear under `Required Checks / backport (...) / …`.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.7)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
---
.../workflows/{reusable-build.yml => build.yml} | 63 +++-
.github/workflows/direct-backport-push.yml | 2 +-
.github/workflows/github-action-build.yml | 411 ---------------------
.github/workflows/required-checks.yml | 178 +++++++++
4 files changed, 238 insertions(+), 416 deletions(-)
diff --git a/.github/workflows/reusable-build.yml b/.github/workflows/build.yml
similarity index 75%
rename from .github/workflows/reusable-build.yml
rename to .github/workflows/build.yml
index 3e34206f90..4e1b431169 100644
--- a/.github/workflows/reusable-build.yml
+++ b/.github/workflows/build.yml
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-name: Reusable Build
+name: Build
on:
workflow_call:
@@ -105,10 +105,13 @@ jobs:
run: yarn --cwd frontend install --immutable --inline-builds
--network-timeout=100000
- name: Lint with Prettier & ESLint
run: yarn --cwd frontend format:ci
- - name: Run frontend unit tests
- run: yarn --cwd frontend run test:ci
- name: Prod build
run: yarn --cwd frontend run build:ci
+ - name: Check bundled npm packages against LICENSE-binary
+ if: matrix.os == 'ubuntu-latest'
+ run: ./bin/licensing/check_binary_deps.py npm
frontend/dist/3rdpartylicenses.json
+ - name: Run frontend unit tests
+ run: yarn --cwd frontend run test:ci
scala:
if: ${{ inputs.run_scala }}
@@ -173,6 +176,41 @@ jobs:
psql -h localhost -U postgres -f sql/texera_lakefs.sql
env:
PGPASSWORD: postgres
+ - name: Build distributable bundles for license check
+ # Build every dist-producing module so the union of bundled jars can
+ # be diffed against LICENSE-binary.
+ run: sbt 'clean; ConfigService/dist; AccessControlService/dist;
FileService/dist; ComputingUnitManagingService/dist;
WorkflowCompilingService/dist; WorkflowExecutionService/dist'
+ - name: Unzip JVM distributable bundles
+ run: |
+ mkdir -p /tmp/dists
+ for zip in \
+ config-service/target/universal/config-service-*.zip \
+
access-control-service/target/universal/access-control-service-*.zip \
+ file-service/target/universal/file-service-*.zip \
+
computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip
\
+
workflow-compiling-service/target/universal/workflow-compiling-service-*.zip \
+ amber/target/universal/amber-*.zip; do
+ unzip -q "$zip" -d /tmp/dists/
+ done
+ - name: Check bundled jars against LICENSE-binary
+ run: |
+ ./bin/licensing/check_binary_deps.py jar \
+ /tmp/dists/config-service-*/lib \
+ /tmp/dists/access-control-service-*/lib \
+ /tmp/dists/file-service-*/lib \
+ /tmp/dists/computing-unit-managing-service-*/lib \
+ /tmp/dists/workflow-compiling-service-*/lib \
+ /tmp/dists/amber-*/lib
+ - name: Audit per-dep license preservation (advisory)
+ if: always()
+ run: |
+ ./bin/licensing/audit_jar_licenses.py \
+ /tmp/dists/config-service-*/lib \
+ /tmp/dists/access-control-service-*/lib \
+ /tmp/dists/file-service-*/lib \
+ /tmp/dists/computing-unit-managing-service-*/lib \
+ /tmp/dists/workflow-compiling-service-*/lib \
+ /tmp/dists/amber-*/lib
- name: Create texera_db_for_test_cases
run: psql -h localhost -U postgres -v DB_NAME=texera_db_for_test_cases
-f sql/texera_ddl.sql
env:
@@ -214,6 +252,13 @@ jobs:
python -m pip install --upgrade pip
if [ -f amber/requirements.txt ]; then pip install -r
amber/requirements.txt; fi
if [ -f amber/operator-requirements.txt ]; then pip install -r
amber/operator-requirements.txt; fi
+ if [ "${{ matrix.python-version }}" = "3.12" ]; then pip install
pip-licenses; fi
+ - name: Generate pip-licenses manifest
+ if: matrix.python-version == '3.12'
+ run: pip-licenses --format=csv --ignore-packages pip-licenses
prettytable wcwidth > /tmp/pip-licenses.csv
+ - name: Check installed Python packages against LICENSE-binary
+ if: matrix.python-version == '3.12'
+ run: ./bin/licensing/check_binary_deps.py python /tmp/pip-licenses.csv
- name: Install PostgreSQL
run: sudo apt-get update && sudo apt-get install -y postgresql
- name: Start PostgreSQL Service
@@ -254,7 +299,17 @@ jobs:
run: |
curl -fsSL https://bun.sh/install | bash -s -- bun-v${{
matrix.bun-version }}
echo "$HOME/.bun/bin" >> $GITHUB_PATH
- - name: Install dependencies
+ - name: Install production dependencies
+ run: bun install --production --frozen-lockfile
+ - name: Generate agent-service license manifest
+ if: matrix.os == 'ubuntu-latest'
+ run: |
+ mkdir -p dist
+ bun run bin/collect-licenses.ts > dist/3rdpartylicenses.json
+ - name: Check bundled agent-service packages against LICENSE-binary
+ if: matrix.os == 'ubuntu-latest'
+ run: ../bin/licensing/check_binary_deps.py agent-npm
dist/3rdpartylicenses.json
+ - name: Install development dependencies
run: bun install --frozen-lockfile
- name: Lint with Prettier
run: bun run format:check
diff --git a/.github/workflows/direct-backport-push.yml
b/.github/workflows/direct-backport-push.yml
index 37fbe04ce4..c291fc1887 100644
--- a/.github/workflows/direct-backport-push.yml
+++ b/.github/workflows/direct-backport-push.yml
@@ -129,7 +129,7 @@ jobs:
{
owner,
repo,
- workflow_id: "github-action-build.yml",
+ workflow_id: "required-checks.yml",
head_sha: pullRequest.head.sha,
per_page: 100,
}
diff --git a/.github/workflows/github-action-build.yml
b/.github/workflows/github-action-build.yml
deleted file mode 100644
index 5d5bacfe01..0000000000
--- a/.github/workflows/github-action-build.yml
+++ /dev/null
@@ -1,411 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-name: Build
-
-env:
- NX_CLOUD_ACCESS_TOKEN: ${{ secrets.NX_CLOUD_ACCESS_TOKEN }}
-
-on:
- push:
- branches:
- - 'ci-enable/**'
- - 'main'
- - 'release/**'
- pull_request:
- types:
- - opened
- - reopened
- - synchronize
- - labeled
- - unlabeled
- workflow_dispatch:
-
-permissions:
- checks: write
- contents: read
- pull-requests: read
-
-concurrency:
- group: ${{ github.workflow }}-${{ github.ref }}
- cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
-
-jobs:
- # Precheck decides which downstream jobs run for this event:
- # - run_frontend / run_scala / run_python / run_agent_service: gate the
- # main build stacks. All true today; placeholder for future path- or
- # label-based selection.
- # - backport_targets: JSON array of release/* labels currently on the PR.
- # Drives the backport matrix; empty array means no backport runs.
- precheck:
- name: Precheck
- runs-on: ubuntu-latest
- outputs:
- run_frontend: ${{ steps.decide.outputs.run_frontend }}
- run_scala: ${{ steps.decide.outputs.run_scala }}
- run_python: ${{ steps.decide.outputs.run_python }}
- run_agent_service: ${{ steps.decide.outputs.run_agent_service }}
- backport_targets: ${{ steps.decide.outputs.backport_targets }}
- steps:
- - name: Decide which jobs to run
- id: decide
- uses: actions/github-script@v8
- with:
- script: |
- const eventName = context.eventName;
-
- // Main build stacks: always run.
- const stacks = ["run_frontend", "run_scala", "run_python",
"run_agent_service"];
- for (const key of stacks) {
- core.setOutput(key, "true");
- }
-
- // Backport targets: all current release/* labels on the PR.
- let targets = [];
- if (eventName === "pull_request") {
- const labels = context.payload.pull_request.labels.map((l) =>
l.name);
- targets = [...new Set(labels.filter((n) =>
/^release\/.+$/.test(n)))].sort();
- }
-
- if (targets.length === 0) {
- core.info(`No backport targets on PR.`);
- } else {
- core.info(`Backport targets: ${targets.join(", ")}`);
- }
- core.setOutput("backport_targets", JSON.stringify(targets));
-
- cleanup-stale-backport:
- if: ${{ github.event_name == 'pull_request' && github.event.action ==
'unlabeled' && startsWith(github.event.label.name, 'release/') }}
- runs-on: ubuntu-latest
- steps:
- - name: Cancel obsolete backport check_runs for the removed target
- uses: actions/github-script@v8
- with:
- script: |
- const { owner, repo } = context.repo;
- const target = context.payload.label.name;
- const headSha = context.payload.pull_request.head.sha;
- const prefix = `backport (${target}) `;
-
- const checks = await github.paginate(
- github.rest.checks.listForRef,
- { owner, repo, ref: headSha, per_page: 100 }
- );
-
- for (const check of checks) {
- if (!check.name.startsWith(prefix)) continue;
- if (check.status === "completed" && check.conclusion ===
"cancelled") continue;
- try {
- await github.rest.checks.update({
- owner,
- repo,
- check_run_id: check.id,
- status: "completed",
- conclusion: "cancelled",
- });
- core.info(`Cancelled check ${check.name}`);
- } catch (e) {
- core.warning(`Failed to update check ${check.id}
(${check.name}): ${e.message}`);
- }
- }
-
- frontend:
- needs: precheck
- if: ${{ needs.precheck.outputs.run_frontend == 'true' }}
- name: frontend (${{ matrix.os }}, 18)
- runs-on: ${{ matrix.os }}
- strategy:
- matrix:
- os: [ubuntu-latest, windows-latest, macos-latest]
- include:
- - os: macos-latest
- arch: arm64
- - os: ubuntu-latest
- arch: x64
- - os: windows-latest
- arch: x64
- node-version:
- - 20.19.0
- steps:
- - name: Checkout Texera
- uses: actions/checkout@v5
- - name: Setup node
- uses: actions/setup-node@v5
- with:
- node-version: ${{ matrix.node-version }}
- architecture: ${{ matrix.arch }}
- - uses: actions/cache@v4
- with:
- path: frontend/.yarn/cache
- key: ${{ runner.os }}-${{ matrix.arch }}-${{ matrix.node-version
}}-yarn-cache-v4-${{ hashFiles('**/yarn.lock') }}
- restore-keys: |
- ${{ runner.os }}-${{ matrix.arch }}-${{ matrix.node-version
}}-yarn-cache-v4-
- - name: Prepare Yarn 4.14.1
- run: corepack enable && corepack prepare [email protected] --activate
- - name: Setup Python
- uses: actions/setup-python@v6
- with:
- python-version: '3.12'
- - name: Install dependency
- timeout-minutes: 20
- run: yarn --cwd frontend install --immutable --inline-builds
--network-timeout=100000
- - name: Lint with Prettier & ESLint
- run: yarn --cwd frontend format:ci
- - name: Prod build
- run: yarn --cwd frontend run build:ci
- - name: Check bundled npm packages against LICENSE-binary
- if: matrix.os == 'ubuntu-latest'
- run: ./bin/licensing/check_binary_deps.py npm
frontend/dist/3rdpartylicenses.json
- - name: Run frontend unit tests
- run: yarn --cwd frontend run test:ci
-
- scala:
- needs: precheck
- if: ${{ needs.precheck.outputs.run_scala == 'true' }}
- strategy:
- matrix:
- os: [ ubuntu-22.04 ]
- java-version: [ 11 ]
- runs-on: ${{ matrix.os }}
- env:
- JAVA_OPTS: -Xms2048M -Xmx2048M -Xss6M -XX:ReservedCodeCacheSize=256M
-Dfile.encoding=UTF-8
- JVM_OPTS: -Xms2048M -Xmx2048M -Xss6M -XX:ReservedCodeCacheSize=256M
-Dfile.encoding=UTF-8
-
- services:
- postgres:
- image: postgres
- env:
- POSTGRES_PASSWORD: postgres
- ports:
- - 5432:5432
- # Add a health check so steps wait until Postgres is ready
- options: >-
- --health-cmd="pg_isready -U postgres"
- --health-interval=10s
- --health-timeout=5s
- --health-retries=5
- steps:
- - name: Checkout
- uses: actions/checkout@v5
- - name: Setup JDK
- uses: actions/setup-java@v5
- with:
- distribution: 'temurin'
- java-version: 11
- - name: Setup Python for Scala tests
- uses: actions/setup-python@v6
- with:
- python-version: '3.11'
- - name: Show Python
- run: python --version || python3 --version
- - name: Install dependencies
- run: |
- python -m pip install --upgrade pip
- if [ -f amber/requirements.txt ]; then pip install -r
amber/requirements.txt; fi
- if [ -f amber/operator-requirements.txt ]; then pip install -r
amber/operator-requirements.txt; fi
- - name: Setup sbt launcher
- uses: sbt/setup-sbt@508b753e53cb6095967669e0911487d2b9bc9f41 # v1.1.22
- - uses: coursier/cache-action@90c37294538be80a558fd665531fcdc2b467b475 #
v8.1.0
- with:
- extraSbtFiles: '["*.sbt", "project/**.{scala,sbt}",
"project/build.properties" ]'
- - name: Lint with scalafmt
- run: sbt scalafmtCheckAll
- - name: Create Databases
- run: |
- psql -h localhost -U postgres -f sql/texera_ddl.sql
- psql -h localhost -U postgres -f sql/iceberg_postgres_catalog.sql
- psql -h localhost -U postgres -f sql/texera_lakefs.sql
- env:
- PGPASSWORD: postgres
- - name: Build distributable bundles for license check
- # Build every dist-producing module so the union of bundled jars can
- # be diffed against LICENSE-binary.
- run: sbt 'clean; ConfigService/dist; AccessControlService/dist;
FileService/dist; ComputingUnitManagingService/dist;
WorkflowCompilingService/dist; WorkflowExecutionService/dist'
- - name: Unzip JVM distributable bundles
- run: |
- mkdir -p /tmp/dists
- for zip in \
- config-service/target/universal/config-service-*.zip \
-
access-control-service/target/universal/access-control-service-*.zip \
- file-service/target/universal/file-service-*.zip \
-
computing-unit-managing-service/target/universal/computing-unit-managing-service-*.zip
\
-
workflow-compiling-service/target/universal/workflow-compiling-service-*.zip \
- amber/target/universal/amber-*.zip; do
- unzip -q "$zip" -d /tmp/dists/
- done
- - name: Check bundled jars against LICENSE-binary
- run: |
- ./bin/licensing/check_binary_deps.py jar \
- /tmp/dists/config-service-*/lib \
- /tmp/dists/access-control-service-*/lib \
- /tmp/dists/file-service-*/lib \
- /tmp/dists/computing-unit-managing-service-*/lib \
- /tmp/dists/workflow-compiling-service-*/lib \
- /tmp/dists/amber-*/lib
- - name: Audit per-dep license preservation (advisory)
- if: always()
- run: |
- ./bin/licensing/audit_jar_licenses.py \
- /tmp/dists/config-service-*/lib \
- /tmp/dists/access-control-service-*/lib \
- /tmp/dists/file-service-*/lib \
- /tmp/dists/computing-unit-managing-service-*/lib \
- /tmp/dists/workflow-compiling-service-*/lib \
- /tmp/dists/amber-*/lib
- - name: Create texera_db_for_test_cases
- run: psql -h localhost -U postgres -v DB_NAME=texera_db_for_test_cases
-f sql/texera_ddl.sql
- env:
- PGPASSWORD: postgres
- - name: Compile with sbt
- run: sbt clean package
- - name: Lint with scalafix
- run: sbt "scalafixAll --check"
- - name: Set docker-java API version
- run: |
- echo "api.version=1.52" >> ~/.docker-java.properties
- cat ~/.docker-java.properties
- - name: Run backend tests
- run: sbt test
-
- python:
- needs: precheck
- if: ${{ needs.precheck.outputs.run_python == 'true' }}
- strategy:
- matrix:
- os: [ ubuntu-latest ]
- python-version: [ '3.10', '3.11', '3.12', '3.13' ]
- runs-on: ${{ matrix.os }}
- steps:
- - name: Checkout Texera
- uses: actions/checkout@v5
- - name: Set up Python ${{ matrix.python-version }}
- uses: actions/setup-python@v6
- with:
- python-version: ${{ matrix.python-version }}
- - name: Install dependencies
- run: |
- python -m pip install --upgrade pip
- if [ -f amber/requirements.txt ]; then pip install -r
amber/requirements.txt; fi
- if [ -f amber/operator-requirements.txt ]; then pip install -r
amber/operator-requirements.txt; fi
- if [ "${{ matrix.python-version }}" = "3.12" ]; then pip install
pip-licenses; fi
- - name: Generate pip-licenses manifest
- if: matrix.python-version == '3.12'
- run: pip-licenses --format=csv --ignore-packages pip-licenses
prettytable wcwidth > /tmp/pip-licenses.csv
- - name: Check installed Python packages against LICENSE-binary
- if: matrix.python-version == '3.12'
- run: ./bin/licensing/check_binary_deps.py python /tmp/pip-licenses.csv
- - name: Install PostgreSQL
- run: sudo apt-get update && sudo apt-get install -y postgresql
- - name: Start PostgreSQL Service
- run: sudo systemctl start postgresql
- - name: Create Database and User
- run: |
- cd sql && sudo -u postgres psql -f iceberg_postgres_catalog.sql
- - name: Lint with Ruff
- run: |
- cd amber/src/main/python && ruff check . && ruff format --check .
- - name: Test with pytest
- run: |
- cd amber/src/main/python && pytest -sv
-
- agent-service:
- needs: precheck
- if: ${{ needs.precheck.outputs.run_agent_service == 'true' }}
- runs-on: ${{ matrix.os }}
- strategy:
- fail-fast: false
- matrix:
- os: [ubuntu-latest, macos-latest]
- bun-version: ['1.3.3']
- defaults:
- run:
- working-directory: agent-service
- steps:
- - name: Checkout Texera
- uses: actions/checkout@v5
- - name: Setup Bun
- run: |
- curl -fsSL https://bun.sh/install | bash -s -- bun-v${{
matrix.bun-version }}
- echo "$HOME/.bun/bin" >> $GITHUB_PATH
- - name: Install production dependencies
- run: bun install --production --frozen-lockfile
- - name: Generate agent-service license manifest
- if: matrix.os == 'ubuntu-latest'
- run: |
- mkdir -p dist
- bun run bin/collect-licenses.ts > dist/3rdpartylicenses.json
- - name: Check bundled agent-service packages against LICENSE-binary
- if: matrix.os == 'ubuntu-latest'
- run: ../bin/licensing/check_binary_deps.py agent-npm
dist/3rdpartylicenses.json
- - name: Install development dependencies
- run: bun install --frozen-lockfile
- - name: Lint with Prettier
- run: bun run format:check
- - name: Typecheck
- run: bun run typecheck
- - name: Run unit tests
- run: bun test
-
- backport:
- needs: precheck
- if: ${{ needs.precheck.outputs.backport_targets != '[]' }}
- strategy:
- fail-fast: false
- matrix:
- target: ${{ fromJson(needs.precheck.outputs.backport_targets) }}
- uses: ./.github/workflows/reusable-build.yml
- with:
- checkout_ref: refs/pull/${{ github.event.pull_request.number }}/head
- backport_target_branch: ${{ matrix.target }}
- backport_commit_range: ${{ format('{0}..{1}',
github.event.pull_request.base.sha, github.event.pull_request.head.sha) }}
- job_name_suffix: ""
- run_frontend: true
- run_scala: true
- run_python: true
- run_agent_service: true
- secrets: inherit
-
- required-checks:
- # Do not rename this job — its display name is referenced in .asf.yaml.
- name: Required Checks
- needs: [precheck, frontend, scala, python, agent-service, backport]
- if: always()
- runs-on: ubuntu-latest
- steps:
- - name: Verify all required checks succeeded or were skipped
- run: |
- declare -A results=(
- [precheck]="${{ needs.precheck.result }}"
- [frontend]="${{ needs.frontend.result }}"
- [scala]="${{ needs.scala.result }}"
- [python]="${{ needs.python.result }}"
- [agent-service]="${{ needs.agent-service.result }}"
- [backport]="${{ needs.backport.result }}"
- )
- failed=0
- for job in "${!results[@]}"; do
- r="${results[$job]}"
- echo "${job}: ${r}"
- if [[ "$r" != "success" && "$r" != "skipped" ]]; then
- failed=1
- fi
- done
- if (( failed )); then
- echo "::error::One or more required checks did not succeed."
- exit 1
- fi
- echo "All required checks succeeded or were skipped."
diff --git a/.github/workflows/required-checks.yml
b/.github/workflows/required-checks.yml
new file mode 100644
index 0000000000..e189e79c38
--- /dev/null
+++ b/.github/workflows/required-checks.yml
@@ -0,0 +1,178 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: Required Checks
+
+on:
+ push:
+ branches:
+ - 'ci-enable/**'
+ - 'main'
+ - 'release/**'
+ pull_request:
+ types:
+ - opened
+ - reopened
+ - synchronize
+ - labeled
+ - unlabeled
+ workflow_dispatch:
+
+permissions:
+ checks: write
+ contents: read
+ pull-requests: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
+
+jobs:
+ # Precheck decides which downstream jobs run for this event:
+ # - run_frontend / run_scala / run_python / run_agent_service: gate the
+ # main build stacks. All true today; placeholder for future path- or
+ # label-based selection.
+ # - backport_targets: JSON array of release/* labels currently on the PR.
+ # Drives the backport matrix; empty array means no backport runs.
+ precheck:
+ name: Precheck
+ runs-on: ubuntu-latest
+ outputs:
+ run_frontend: ${{ steps.decide.outputs.run_frontend }}
+ run_scala: ${{ steps.decide.outputs.run_scala }}
+ run_python: ${{ steps.decide.outputs.run_python }}
+ run_agent_service: ${{ steps.decide.outputs.run_agent_service }}
+ backport_targets: ${{ steps.decide.outputs.backport_targets }}
+ steps:
+ - name: Decide which jobs to run
+ id: decide
+ uses: actions/github-script@v8
+ with:
+ script: |
+ const eventName = context.eventName;
+
+ // Main build stacks: always run.
+ const stacks = ["run_frontend", "run_scala", "run_python",
"run_agent_service"];
+ for (const key of stacks) {
+ core.setOutput(key, "true");
+ }
+
+ // Backport targets: all current release/* labels on the PR.
+ let targets = [];
+ if (eventName === "pull_request") {
+ const labels = context.payload.pull_request.labels.map((l) =>
l.name);
+ targets = [...new Set(labels.filter((n) =>
/^release\/.+$/.test(n)))].sort();
+ }
+
+ if (targets.length === 0) {
+ core.info(`No backport targets on PR.`);
+ } else {
+ core.info(`Backport targets: ${targets.join(", ")}`);
+ }
+ core.setOutput("backport_targets", JSON.stringify(targets));
+
+ cleanup-stale-backport:
+ if: ${{ github.event_name == 'pull_request' && github.event.action ==
'unlabeled' && startsWith(github.event.label.name, 'release/') }}
+ runs-on: ubuntu-latest
+ steps:
+ - name: Cancel obsolete backport check_runs for the removed target
+ uses: actions/github-script@v8
+ with:
+ script: |
+ const { owner, repo } = context.repo;
+ const target = context.payload.label.name;
+ const headSha = context.payload.pull_request.head.sha;
+ const prefix = `backport (${target}) `;
+
+ const checks = await github.paginate(
+ github.rest.checks.listForRef,
+ { owner, repo, ref: headSha, per_page: 100 }
+ );
+
+ for (const check of checks) {
+ if (!check.name.startsWith(prefix)) continue;
+ if (check.status === "completed" && check.conclusion ===
"cancelled") continue;
+ try {
+ await github.rest.checks.update({
+ owner,
+ repo,
+ check_run_id: check.id,
+ status: "completed",
+ conclusion: "cancelled",
+ });
+ core.info(`Cancelled check ${check.name}`);
+ } catch (e) {
+ core.warning(`Failed to update check ${check.id}
(${check.name}): ${e.message}`);
+ }
+ }
+
+ build:
+ needs: precheck
+ uses: ./.github/workflows/build.yml
+ with:
+ run_frontend: ${{ needs.precheck.outputs.run_frontend == 'true' }}
+ run_scala: ${{ needs.precheck.outputs.run_scala == 'true' }}
+ run_python: ${{ needs.precheck.outputs.run_python == 'true' }}
+ run_agent_service: ${{ needs.precheck.outputs.run_agent_service ==
'true' }}
+ secrets: inherit
+
+ backport:
+ needs: precheck
+ if: ${{ needs.precheck.outputs.backport_targets != '[]' }}
+ strategy:
+ fail-fast: false
+ matrix:
+ target: ${{ fromJson(needs.precheck.outputs.backport_targets) }}
+ uses: ./.github/workflows/build.yml
+ with:
+ checkout_ref: refs/pull/${{ github.event.pull_request.number }}/head
+ backport_target_branch: ${{ matrix.target }}
+ backport_commit_range: ${{ format('{0}..{1}',
github.event.pull_request.base.sha, github.event.pull_request.head.sha) }}
+ job_name_suffix: ""
+ run_frontend: ${{ needs.precheck.outputs.run_frontend == 'true' }}
+ run_scala: ${{ needs.precheck.outputs.run_scala == 'true' }}
+ run_python: ${{ needs.precheck.outputs.run_python == 'true' }}
+ run_agent_service: ${{ needs.precheck.outputs.run_agent_service ==
'true' }}
+ secrets: inherit
+
+ required-checks:
+ # Do not rename this job — its display name is referenced in .asf.yaml.
+ name: Required Checks
+ needs: [precheck, build, backport]
+ if: always()
+ runs-on: ubuntu-latest
+ steps:
+ - name: Verify all required checks succeeded or were skipped
+ run: |
+ declare -A results=(
+ [precheck]="${{ needs.precheck.result }}"
+ [build]="${{ needs.build.result }}"
+ [backport]="${{ needs.backport.result }}"
+ )
+ failed=0
+ for job in "${!results[@]}"; do
+ r="${results[$job]}"
+ echo "${job}: ${r}"
+ if [[ "$r" != "success" && "$r" != "skipped" ]]; then
+ failed=1
+ fi
+ done
+ if (( failed )); then
+ echo "::error::One or more required checks did not succeed."
+ exit 1
+ fi
+ echo "All required checks succeeded or were skipped."
