This is an automated email from the ASF dual-hosted git repository.

LuciferYang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gluten.git


The following commit(s) were added to refs/heads/main by this push:
     new f6e4123d29 [GLUTEN-13541][VL][CI] Run the Velox x86, ARM and Kafka 
pipelines in contributors' forks (#13535)
f6e4123d29 is described below

commit f6e4123d29a801620520122365e0ebfa8397214d
Author: YangJie <[email protected]>
AuthorDate: Fri Nov 13 10:33:58 2026 -0500

    [GLUTEN-13541][VL][CI] Run the Velox x86, ARM and Kafka pipelines in 
contributors' forks (#13535)
    
    ## What changes are proposed in this pull request?
    
    Fork PR runs of the Velox x86, ARM and Kafka workflows used 223,000 of 
apache/gluten's 292,000 runner-minutes in the week of 2026-10-27, over the 
ASF's 250,000 weekly limit. This PR runs them in the contributor's fork on 
push, as Spark does.
    
    - A fork run tests the branch squash-merged onto the current `main`. Fork 
PRs to `main` no longer run these workflows here.
    - Fork builds restore apache/gluten's `main` ccache. With `CCACHE_BASEDIR`, 
forks with other names hit it too.
    - Fork runs of the Iceberg UT read baselines from apache/gluten.
    - `update_fork_build_status.yml` mirrors each fork run as a PR check every 
5 minutes, without `pull_request_target`. A check fails on its run's first 
failed job and lists failed jobs.
    
    Open fork PRs run again once rebased onto `main` and pushed. Fork PRs won't 
get #13611's "Report test results" check, since fork runs don't trigger 
`workflow_run`. Contributor steps are in `CONTRIBUTING.md`, background and 
follow-ups in #13541.
    
    ## How was this patch tested?
    
    Fork runs of the latest commit passed: 
[x86](https://github.com/LuciferYang/gluten/actions/runs/43160683425), 
[ARM](https://github.com/LuciferYang/gluten/actions/runs/43160683016), 
[Kafka](https://github.com/LuciferYang/gluten/actions/runs/43160683029). The 
Velox jobs on this PR show as skipped. After merge, fork PRs still list those 
skipped jobs, plus `... in fork` checks with their fork runs' results. Those 
checks can't show up before merge, because `update_fork_build_status.yml` ru 
[...]
    
    ## Was this patch authored or co-authored using generative AI tooling?
    
    Yes. Generated-by: Claude Opus 5.5
---
 .github/actions/restore-upstream-ccache/action.yml |  88 +++++++
 .github/actions/squash-merge-upstream/action.yml   |  81 +++++++
 .github/workflows/iceberg_spark_ut.yml             |  21 +-
 .github/workflows/iceberg_upstream_revision.yml    |   8 +-
 .github/workflows/update_fork_build_status.yml     |  52 +++++
 .../workflows/util/iceberg-upstream/baseline.py    |   5 +-
 .github/workflows/util/update-fork-build-status.js | 252 +++++++++++++++++++++
 .github/workflows/velox_backend_arm.yml            |  95 +++++++-
 .github/workflows/velox_backend_x86.yml            | 156 ++++++++++++-
 .github/workflows/velox_kafka.yml                  |  65 +++++-
 CONTRIBUTING.md                                    |  11 +
 dev/ci-velox-buildshared-centos-9.sh               |   4 +
 ...velox-buildstatic-centos-8-enhanced-features.sh |   4 +
 dev/ci-velox-buildstatic-centos-8.sh               |   4 +
 dev/ci-velox-buildstatic-centos-9.sh               |   4 +
 15 files changed, 824 insertions(+), 26 deletions(-)

diff --git a/.github/actions/restore-upstream-ccache/action.yml 
b/.github/actions/restore-upstream-ccache/action.yml
new file mode 100644
index 0000000000..ddd025f990
--- /dev/null
+++ b/.github/actions/restore-upstream-ccache/action.yml
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Restore ccache from apache/gluten main
+description: >-
+  Download the ccache stash that apache/infrastructure-actions/stash/save 
stored
+  for `key` on apache/gluten main. stash/restore only searches the repository 
the
+  run belongs to, so a fork run uses this after missing its own stash. Without 
a
+  stash for `key`, the newest stash for the same key prefix is used, as
+  actions/cache restore-keys would.
+inputs:
+  key:
+    description: The key the stash was saved with, ending in a hashFiles() 
hash.
+    required: true
+  path:
+    description: The directory to restore into.
+    required: true
+outputs:
+  hit:
+    description: "'true' if a stash was downloaded."
+    value: ${{ steps.download.outcome == 'success' }}
+runs:
+  using: composite
+  steps:
+    - name: Find the stash
+      id: find
+      uses: actions/github-script@v9
+      env:
+        KEY: ${{ inputs.key }}
+      with:
+        script: |
+          const repo = { owner: 'apache', repo: 'gluten' }
+          // stash/save appends the branch name to the key. Only stashes saved 
by
+          // apache/gluten's own runs on main count.
+          const usable = (a) => !a.expired && a.name.endsWith('-main')
+            && a.workflow_run.head_branch == 'main'
+            && a.workflow_run.head_repository_id == 
a.workflow_run.repository_id
+          const exact = process.env.KEY + '-main'
+          const { data } = await github.rest.actions.listArtifactsForRepo({
+            ...repo, name: exact, per_page: 10
+          })
+          let stash = data.artifacts.find(usable)
+          if (!stash) {
+            // The key changed in this branch, or main changed it and the cache
+            // workflow hasn't saved the new stash yet: take the newest one it 
saved.
+            const prefix = process.env.KEY.replace(/[0-9a-f]{64}$/, '')
+            const { data: runs } = await github.rest.actions.listWorkflowRuns({
+              ...repo, workflow_id: 'velox_backend_cache.yml', branch: 'main',
+              status: 'success', per_page: 5
+            })
+            for (const run of runs.workflow_runs) {
+              const { data: arts } = await 
github.rest.actions.listWorkflowRunArtifacts({
+                ...repo, run_id: run.id, per_page: 100
+              })
+              stash = arts.artifacts.find(a => usable(a) && 
a.name.startsWith(prefix))
+              if (stash) break
+            }
+          }
+          if (stash) {
+            console.log('Restoring ' + stash.name + ' from apache/gluten run ' 
+ stash.workflow_run.id)
+            core.setOutput('name', stash.name)
+            core.setOutput('run_id', stash.workflow_run.id)
+          } else {
+            console.log('No stash for ' + exact + ' on apache/gluten main')
+          }
+    - name: Download the stash
+      id: download
+      if: steps.find.outputs.run_id != ''
+      continue-on-error: true
+      uses: actions/download-artifact@v8
+      with:
+        name: ${{ steps.find.outputs.name }}
+        repository: apache/gluten
+        run-id: ${{ steps.find.outputs.run_id }}
+        github-token: ${{ github.token }}
+        path: ${{ inputs.path }}
diff --git a/.github/actions/squash-merge-upstream/action.yml 
b/.github/actions/squash-merge-upstream/action.yml
new file mode 100644
index 0000000000..ccdf39b24a
--- /dev/null
+++ b/.github/actions/squash-merge-upstream/action.yml
@@ -0,0 +1,81 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: Squash-merge onto apache/gluten main
+description: >-
+  In a fork, squash-merge the commit under test onto the current apache/gluten
+  main, which is what a pull request merge ref would hold. If anything in
+  `paths` changed, push the result to a refs/gluten-ci/ ref in the fork, so the
+  workflow's other jobs can check it out by commit. Use it after
+  actions/checkout with fetch-depth 0, in a job with `contents: write`.
+inputs:
+  paths:
+    description: >-
+      Extended regex for the files the workflow covers. A `paths:` filter on 
push
+      diffs against the branch's previous push, not against main, so fork runs
+      apply the list here.
+    required: true
+  paths-ignore:
+    description: Extended regex for files to leave out before matching `paths`.
+    default: '^$'
+outputs:
+  base_sha:
+    description: The apache/gluten main commit the branch was merged onto.
+    value: ${{ steps.merge.outputs.base_sha }}
+  revision:
+    description: The merged commit.
+    value: ${{ steps.merge.outputs.revision }}
+  relevant:
+    description: "'false' if nothing in `paths` changed."
+    value: ${{ steps.merge.outputs.relevant }}
+runs:
+  using: composite
+  steps:
+    - name: Squash-merge the commit under test
+      id: merge
+      shell: bash
+      env:
+        PATHS: ${{ inputs.paths }}
+        PATHS_IGNORE: ${{ inputs.paths-ignore }}
+      run: |
+        git config user.name 'github-actions[bot]'
+        git config user.email 
'41898282+github-actions[bot]@users.noreply.github.com'
+        git fetch --no-tags https://github.com/apache/gluten.git main
+        base_sha=$(git rev-parse FETCH_HEAD)
+        git checkout --quiet --detach "$base_sha"
+        if ! git merge --squash --no-commit "$GITHUB_SHA"; then
+          echo "::error::$GITHUB_REF_NAME does not merge cleanly into 
apache/gluten main ($base_sha). Rebase it and push again."
+          exit 1
+        fi
+        git commit --quiet --allow-empty -m "Squash-merge $GITHUB_SHA onto 
$base_sha"
+        echo "base_sha=$base_sha" >> "$GITHUB_OUTPUT"
+        echo "revision=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+        # Not `grep -q`: it can exit before the upstream grep is done writing, 
and under
+        # pipefail the SIGPIPE would read as no match.
+        if git diff --name-only "$base_sha" HEAD | grep -vE "$PATHS_IGNORE" | 
grep -E "$PATHS" > /dev/null; then
+          echo "relevant=true" >> "$GITHUB_OUTPUT"
+        else
+          echo "Nothing this workflow covers changed -- skipping."
+          echo "relevant=false" >> "$GITHUB_OUTPUT"
+        fi
+    # One ref per workflow and branch, overwritten by the next run. Slashes in 
the
+    # branch name are replaced so that branches `a` and `a/b` don't collide.
+    - name: Publish the merged commit in the fork
+      if: steps.merge.outputs.relevant == 'true'
+      shell: bash
+      run: |
+        workflow=${GITHUB_WORKFLOW_REF%%@*}
+        workflow=${workflow##*/}
+        git push --force origin 
"HEAD:refs/gluten-ci/${workflow%.*}/${GITHUB_REF_NAME//\//-}"
diff --git a/.github/workflows/iceberg_spark_ut.yml 
b/.github/workflows/iceberg_spark_ut.yml
index 6f8c8ec1eb..b3f4a25cee 100644
--- a/.github/workflows/iceberg_spark_ut.yml
+++ b/.github/workflows/iceberg_spark_ut.yml
@@ -23,6 +23,14 @@ on:
       native_artifact:
         type: string
         default: ''
+      # A fork push passes the commit it squash-merged onto apache/gluten 
main, and
+      # that main commit. Empty: the event's commit and pull request base.
+      revision:
+        type: string
+        default: ''
+      base_revision:
+        type: string
+        default: ''
   workflow_dispatch:
 
 permissions:
@@ -60,11 +68,12 @@ jobs:
     steps:
       - uses: actions/checkout@v7
         with:
+          ref: ${{ inputs.revision }}
           fetch-depth: 0
       - name: Identify the merged baseline and runtimes
         id: baseline
         env:
-          BASE_SHA: ${{ github.event.pull_request.base.sha }}
+          BASE_SHA: ${{ inputs.base_revision || 
github.event.pull_request.base.sha }}
         run: |
           python3 .github/workflows/util/iceberg-upstream/baseline.py prepare \
             --revision "${BASE_SHA:-$(git rev-parse HEAD^)}" >> 
"$GITHUB_OUTPUT"
@@ -77,6 +86,8 @@ jobs:
         continue-on-error: true
         env:
           GH_TOKEN: ${{ github.token }}
+          # Baselines are saved by apache/gluten runs; a fork reads them from 
there.
+          ICEBERG_BASELINE_REPOSITORY: apache/gluten
           BASELINE_KEY: ${{ steps.baseline.outputs.baseline_key }}
           BASE_SHA: ${{ steps.baseline.outputs.base_sha }}
           BASE_BRANCH: ${{ github.event.pull_request.base.ref || 
github.event.repository.default_branch }}
@@ -101,7 +112,8 @@ jobs:
     needs: prepare
     uses: ./.github/workflows/iceberg_upstream_revision.yml
     with:
-      revision: ${{ github.sha }}
+      revision: ${{ inputs.revision || github.sha }}
+      harness_revision: ${{ inputs.revision }}
       label: head
       native_image: ${{ needs.prepare.outputs.native_image }}
       test_image: ${{ needs.prepare.outputs.test_image }}
@@ -113,6 +125,7 @@ jobs:
     uses: ./.github/workflows/iceberg_upstream_revision.yml
     with:
       revision: ${{ needs.prepare.outputs.base_sha }}
+      harness_revision: ${{ inputs.revision }}
       label: base
       native_image: ${{ needs.prepare.outputs.native_image }}
       test_image: ${{ needs.prepare.outputs.test_image }}
@@ -125,6 +138,7 @@ jobs:
     steps:
       - uses: actions/checkout@v7
         with:
+          ref: ${{ inputs.revision }}
           fetch-depth: 0
       - name: Require complete test runs
         env:
@@ -153,9 +167,10 @@ jobs:
         if: always() && steps.head.outcome == 'success' && 
needs.iceberg-upstream-test.result == 'success'
         env:
           HEAD_KEY: ${{ needs.prepare.outputs.head_key }}
+          HEAD_SHA: ${{ inputs.revision || github.sha }}
         run: |
           python3 .github/workflows/util/iceberg-upstream/baseline.py record \
-            --directory iceberg-head --key "$HEAD_KEY" --revision "$GITHUB_SHA"
+            --directory iceberg-head --key "$HEAD_KEY" --revision "$HEAD_SHA"
       - name: Save the result for reuse after merge
         if: always() && steps.head-identity.outcome == 'success'
         uses: actions/upload-artifact@v7
diff --git a/.github/workflows/iceberg_upstream_revision.yml 
b/.github/workflows/iceberg_upstream_revision.yml
index 4dc933ab52..c82c7d626e 100644
--- a/.github/workflows/iceberg_upstream_revision.yml
+++ b/.github/workflows/iceberg_upstream_revision.yml
@@ -30,6 +30,10 @@ on:
       test_image:
         required: true
         type: string
+      # The commit holding the comparison harness. Empty: the event's commit.
+      harness_revision:
+        type: string
+        default: ''
       native_artifact:
         # Native libraries already built for this revision in the calling run. 
When empty,
         # they are built here.
@@ -68,7 +72,7 @@ jobs:
         if: inputs.label == 'base'
         uses: actions/checkout@v7
         with:
-          ref: ${{ github.sha }}
+          ref: ${{ inputs.harness_revision || github.sha }}
           path: iceberg-harness
       - name: Apply the current harness to the base production revision
         if: inputs.label == 'base'
@@ -121,7 +125,7 @@ jobs:
         if: inputs.label == 'base'
         uses: actions/checkout@v7
         with:
-          ref: ${{ github.sha }}
+          ref: ${{ inputs.harness_revision || github.sha }}
           path: iceberg-harness
       - name: Apply the current harness to the base production revision
         if: inputs.label == 'base'
diff --git a/.github/workflows/update_fork_build_status.yml 
b/.github/workflows/update_fork_build_status.yml
new file mode 100644
index 0000000000..a1f13a31cb
--- /dev/null
+++ b/.github/workflows/update_fork_build_status.yml
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# A pull request from a fork runs velox_backend_x86.yml, velox_backend_arm.yml 
and
+# velox_kafka.yml in the fork, on push. This workflow reports those runs as 
checks on
+# the pull request, using util/update-fork-build-status.js. It only reads run 
metadata,
+# so it never runs pull request code and needs no pull_request_target. Adapted 
from
+# apache/spark's update_build_status.yml (SPARK-60578).
+name: Update fork build status
+
+on:
+  schedule:
+    # Every 5 minutes, off the top of the hour, when GitHub delays scheduled 
runs most.
+    - cron: '2-57/5 * * * *'
+  workflow_dispatch:
+
+permissions: {}
+
+concurrency:
+  group: ${{ github.workflow }}
+  cancel-in-progress: false
+
+jobs:
+  update:
+    if: github.repository == 'apache/gluten'
+    runs-on: ubuntu-slim
+    permissions:
+      actions: read
+      checks: write
+      contents: read
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          sparse-checkout: .github/workflows/util/update-fork-build-status.js
+          sparse-checkout-cone-mode: false
+      - uses: actions/github-script@v9
+        with:
+          script: |
+            const script = 
require(`${process.env.GITHUB_WORKSPACE}/.github/workflows/util/update-fork-build-status.js`)
+            await script({ github, context, core })
diff --git a/.github/workflows/util/iceberg-upstream/baseline.py 
b/.github/workflows/util/iceberg-upstream/baseline.py
index b17080c1e4..da64292aee 100644
--- a/.github/workflows/util/iceberg-upstream/baseline.py
+++ b/.github/workflows/util/iceberg-upstream/baseline.py
@@ -374,7 +374,10 @@ if __name__ == "__main__":
                     args.key,
                     args.revision,
                     args.branch,
-                    GitHub(os.environ["GITHUB_REPOSITORY"]),
+                    GitHub(
+                        os.environ.get("ICEBERG_BASELINE_REPOSITORY")
+                        or os.environ["GITHUB_REPOSITORY"]
+                    ),
                 )
             except (ValueError, KeyError, OSError) as error:
                 args.directory.mkdir(parents=True, exist_ok=True)
diff --git a/.github/workflows/util/update-fork-build-status.js 
b/.github/workflows/util/update-fork-build-status.js
new file mode 100644
index 0000000000..204c08d158
--- /dev/null
+++ b/.github/workflows/util/update-fork-build-status.js
@@ -0,0 +1,252 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Run by update_fork_build_status.yml. For each open pull request from a 
fork, reports
+// the fork's push run of each workflow below as the check named next to it.
+
+const WORKFLOWS = [
+  { file: 'velox_backend_x86.yml', check: 'Velox Backend (x86) in fork' },
+  { file: 'velox_backend_arm.yml', check: 'Velox Backend (ARM) in fork' },
+  { file: 'velox_kafka.yml', check: 'Velox Kafka in fork' },
+];
+
+// Check runs accept fewer conclusions than a workflow run reports; anything 
else
+// means the run produced no result.
+const CONCLUSIONS = ['success', 'failure', 'neutral', 'cancelled', 
'timed_out', 'action_required', 'skipped'];
+
+// Rate limiting, which GitHub reports as 429 or as 403 with rate-limit 
headers.
+function isRateLimited(e) {
+  if (e.status == 429) return true;
+  if (e.status != 403) return false;
+  const h = (e.response && e.response.headers) || {};
+  return !!h['retry-after'] || h['x-ratelimit-remaining'] === '0' || /rate 
limit/i.test(e.message || '');
+}
+
+// 5xx and network errors are retried on a later pass.
+function isTransient(e) {
+  return !e.status || e.status >= 500;
+}
+
+const runPath = (w) => '.github/workflows/' + w.file;
+
+// Job conclusions that make the run fail. No job in these workflows sets
+// continue-on-error, so a check can turn red before the run ends.
+const FAILED = ['failure', 'timed_out'];
+
+// What a check records in its output text about its fork run, or null.
+function recordOf(check) {
+  if (check.conclusion == 'action_required') return null;
+  try {
+    const p = JSON.parse(check.output.text);
+    return p && p.run_id ? p : null;
+  } catch (e) {
+    return null;
+  }
+}
+
+// The fork run attempt a check points at, as "run_id/attempt", or null. Checks
+// written before attempts were recorded count as attempt 1.
+function attemptOf(check) {
+  const p = recordOf(check);
+  return p ? `${p.run_id}/${p.run_attempt || 1}` : null;
+}
+
+// The check status (and conclusion) for a run's current state and its failed 
jobs.
+function stateOf(run, failedJobs) {
+  if (failedJobs.length) return { status: 'completed', conclusion: 'failure' };
+  if (run.status == 'completed') {
+    return { status: 'completed', conclusion: 
CONCLUSIONS.includes(run.conclusion) ? run.conclusion : 'failure' };
+  }
+  return { status: run.status == 'in_progress' ? 'in_progress' : 'queued' };
+}
+
+function noRunSummary(w, sha) {
+  return [
+    `${w.file} runs in your fork on each push to a branch other than main. No 
run was found for ${sha}.`,
+    '',
+    '1. Enable GitHub Actions in the Actions tab of your fork, then push 
again. See [Managing GitHub Actions settings for a 
repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository).',
+    '2. If your branch predates fork runs, rebase it on the latest 
apache/gluten main and push again.',
+    "3. If this pull request comes from your fork's main branch, push it to 
another branch and open the pull request from there.",
+  ].join('\n');
+}
+
+module.exports = async ({ github, context, core }) => {
+  const repo = { owner: context.repo.owner, repo: context.repo.repo };
+
+  const listChecks = (sha) =>
+    github.paginate('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', { 
...repo, ref: sha, filter: 'all', per_page: 100 });
+
+  // The fork's push runs for the pull request's head commit. Re-polled until 
every
+  // workflow in `wanted` has one, because a run can take a few seconds to 
register.
+  const listForkRuns = async (pr, wanted, attempts) => {
+    let runs = [];
+    for (let i = 0; i < attempts; i++) {
+      try {
+        const { data } = await github.request('GET 
/repos/{owner}/{repo}/actions/runs', {
+          owner: pr.head.repo.owner.login,
+          repo: pr.head.repo.name,
+          event: 'push',
+          branch: pr.head.ref,
+          head_sha: pr.head.sha,
+          per_page: 100,
+        });
+        runs = data.workflow_runs.filter((r) => r.head_sha == pr.head.sha);
+      } catch (e) {
+        if (e.status == 404) return [];
+        throw e;
+      }
+      if (wanted.every((w) => runs.some((r) => r.path == runPath(w)))) break;
+      if (i < attempts - 1) await new Promise((r) => setTimeout(r, 3000));
+    }
+    return runs;
+  };
+
+  // Jobs of the run's latest attempt that failed. Read only while the run is 
going or
+  // when it didn't pass.
+  const failedJobsOf = async (pr, run) => {
+    if (run.status == 'completed' && ['success', 'skipped', 
'neutral'].includes(run.conclusion)) return [];
+    const jobs = await github.paginate('GET 
/repos/{owner}/{repo}/actions/runs/{run_id}/jobs', {
+      owner: pr.head.repo.owner.login,
+      repo: pr.head.repo.name,
+      run_id: run.id,
+      filter: 'latest',
+      per_page: 100,
+    });
+    return jobs.filter((j) => FAILED.includes(j.conclusion));
+  };
+
+  const checkFor = async (pr, run) => {
+    const failedJobs = await failedJobsOf(pr, run);
+    const done = run.status == 'completed';
+    const count = failedJobs.length == 1 ? '1 job failed' : 
`${failedJobs.length} jobs failed`;
+    const lines = [
+      `[See the run in ${pr.head.repo.full_name}](${run.html_url}). ` +
+        'Rerun failed jobs there if the failure is unrelated to this pull 
request.',
+    ];
+    if (failedJobs.length) {
+      lines.push('', done ? 'Failed jobs:' : 'Failed jobs so far (the run is 
still going):');
+      for (const j of failedJobs) lines.push(`- [${j.name}](${j.html_url})`);
+    }
+    return {
+      ...stateOf(run, failedJobs),
+      output: {
+        title: failedJobs.length ? count + (done ? '' : ', run still going') : 
'Fork run',
+        summary: lines.join('\n'),
+        // `done` stops further reads of a check whose run has ended.
+        text: JSON.stringify({
+          owner: pr.head.repo.owner.login,
+          repo: pr.head.repo.name,
+          run_id: run.id,
+          run_attempt: run.run_attempt,
+          done,
+        }),
+      },
+    };
+  };
+
+  const syncCheck = async (pr, w, check, run) => {
+    if (recordOf(check).done) return;
+    const next = await checkFor(pr, run);
+    const same = (k) => (check.output[k] || '') == next.output[k];
+    if (
+      check.status == next.status &&
+      check.conclusion == (next.conclusion || null) &&
+      same('title') &&
+      same('summary') &&
+      same('text')
+    ) {
+      return;
+    }
+    console.log(`#${pr.number} ${w.file}: ${next.status} ${next.conclusion || 
''} ${next.output.title}`);
+    // An update without `output` clears its text, which is how the next pass 
finds this check.
+    return github.request('PATCH 
/repos/{owner}/{repo}/check-runs/{check_run_id}', {
+      ...repo,
+      check_run_id: check.id,
+      details_url: run.html_url,
+      ...next,
+    });
+  };
+
+  const createRunCheck = async (pr, w, run) => {
+    console.log(`#${pr.number} ${w.file}: tracking ${run.html_url}`);
+    return github.rest.checks.create({
+      ...repo,
+      name: w.check,
+      head_sha: pr.head.sha,
+      details_url: run.html_url,
+      ...(await checkFor(pr, run)),
+    });
+  };
+
+  const createNoRunCheck = (pr, w) => {
+    console.log(`#${pr.number} ${w.file}: no fork run`);
+    return github.rest.checks.create({
+      ...repo,
+      name: w.check,
+      head_sha: pr.head.sha,
+      status: 'completed',
+      conclusion: 'action_required',
+      output: { title: 'No run found in your fork', summary: noRunSummary(w, 
pr.head.sha) },
+    });
+  };
+
+  const failed = [];
+  let rateLimited = false;
+  for await (const { data: prs } of 
github.paginate.iterator(github.rest.pulls.list, {
+    ...repo,
+    state: 'open',
+    base: 'main',
+    per_page: 100,
+  })) {
+    for (const pr of prs) {
+      // Branches of this repository (Dependabot) run upstream; skip deleted 
forks.
+      if (!pr.head.repo || pr.head.repo.full_name == 
`${repo.owner}/${repo.repo}`) continue;
+      try {
+        const checks = await listChecks(pr.head.sha);
+        const checksOf = (w) => checks.filter((c) => c.name == w.check);
+        // Only a workflow without any check yet can have a run that hasn't 
registered.
+        const fresh = WORKFLOWS.filter((w) => checksOf(w).length == 0);
+        const runs = await listForkRuns(pr, fresh, fresh.length ? 3 : 1);
+        for (const w of WORKFLOWS) {
+          const run = runs.filter((r) => r.path == runPath(w)).sort((a, b) => 
b.id - a.id)[0];
+          if (run) {
+            // A newer run or attempt than any check points at (a rerun, or a 
push back to
+            // an earlier commit) gets a check of its own, so a rerun shows as 
in progress
+            // instead of keeping the previous attempt's result.
+            const check = checksOf(w).find((c) => attemptOf(c) == 
`${run.id}/${run.run_attempt}`);
+            await (check ? syncCheck(pr, w, check, run) : createRunCheck(pr, 
w, run));
+          } else if (!checksOf(w).some((c) => c.conclusion == 
'action_required')) {
+            await createNoRunCheck(pr, w);
+          }
+        }
+      } catch (e) {
+        // Stop the pass instead of sending more requests; the next pass picks 
up here.
+        if (isRateLimited(e)) {
+          console.log(`#${pr.number}: rate limited, stopping this pass`);
+          rateLimited = true;
+          break;
+        }
+        if (isTransient(e)) continue;
+        console.error(`#${pr.number}: ${e.message}`);
+        failed.push(pr.number);
+      }
+    }
+    if (rateLimited) break;
+  }
+  // Keep a recurring error visible instead of hiding it in a green run.
+  if (failed.length) core.setFailed('Failed to sync #' + failed.join(', #'));
+};
diff --git a/.github/workflows/velox_backend_arm.yml 
b/.github/workflows/velox_backend_arm.yml
index abf7114891..7af5b956cc 100644
--- a/.github/workflows/velox_backend_arm.yml
+++ b/.github/workflows/velox_backend_arm.yml
@@ -19,6 +19,7 @@ on:
   pull_request:
     paths:
       - '.github/workflows/velox_backend_arm.yml'
+      - '.github/actions/**'
       - 'pom.xml'
       - 'backends-velox/**'
       - 'gluten-uniffle/**'
@@ -40,6 +41,13 @@ on:
       - '!cpp/bolt.CMakeLists.cmake'
       - 'dev/**'
       - 'build/mvn'
+  # A pull request from a fork is tested in the fork: each push to a branch
+  # other than main runs this workflow there (see detect-changes). In
+  # apache/gluten a push run skips every job.
+  push:
+    branches-ignore:
+      - main
+  workflow_dispatch:
 
 env:
   ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION: true
@@ -48,42 +56,99 @@ env:
   CCACHE_DIR: "${{ github.workspace }}/.ccache"
 
 concurrency:
-  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ 
github.workflow }}
+  # head_ref is empty on push. Keying push runs by branch lets a new push 
cancel the previous run.
+  group: >-
+    ${{ github.repository }}-${{ github.head_ref || (github.event_name == 
'push' && github.ref) ||
+    github.sha }}-${{ github.workflow }}
   cancel-in-progress: true
 
 jobs:
+  # Every other job chains off this one, so its `if` decides whether the 
workflow runs:
+  # push and workflow_dispatch only in forks, where the run tests a fork pull 
request;
+  # pull_request only in apache/gluten and only from its own branches 
(Dependabot).
+  # In a fork it squash-merges the commit under test onto apache/gluten main 
and
+  # publishes the result, which the jobs below check out by `revision`. It is 
the
+  # only job that can write, and none of its steps run in apache/gluten.
+  merge-upstream:
+    if: >-
+      (github.repository_owner != 'apache' &&
+       (github.event_name == 'push' || github.event_name == 
'workflow_dispatch')) ||
+      (github.event_name == 'pull_request' && github.repository_owner == 
'apache' &&
+       github.event.pull_request.head.repo.full_name == github.repository)
+    runs-on: ubuntu-24.04
+    permissions:
+      contents: write
+    outputs:
+      relevant: ${{ steps.merge.outputs.relevant }}
+      revision: ${{ steps.merge.outputs.revision }}
+      base_sha: ${{ steps.merge.outputs.base_sha }}
+    steps:
+      - uses: actions/checkout@v7
+        if: github.repository_owner != 'apache'
+        with:
+          fetch-depth: 0
+      - name: Squash-merge onto apache/gluten main
+        id: merge
+        if: github.repository_owner != 'apache'
+        uses: ./.github/actions/squash-merge-upstream
+        with:
+          # The `paths:` list above. Keep the two in sync.
+          paths: 
'^(\.github/workflows/velox_backend_arm\.yml$|\.github/actions/|pom\.xml$|build/mvn$|(backends-velox|gluten-(uniffle|celeborn|core|substrait|arrow|delta|iceberg|hudi|paimon|ut)|package|shims|tools/gluten-it|ep/build-velox|cpp|dev)/)'
+          paths-ignore: '^cpp/bolt/|^cpp/bolt\.CMakeLists\.cmake$'
+
   # Detect which parts of the codebase were modified so downstream jobs can
   # skip work that is irrelevant to the change.
   detect-changes:
+    needs: merge-upstream
     runs-on: ubuntu-24.04
     outputs:
+      # false only on a fork run that changes nothing in the `paths:` list 
above.
+      relevant: ${{ needs.merge-upstream.outputs.relevant }}
+      # The commit the other jobs check out: the merged commit on a fork run, 
otherwise
+      # empty, which keeps actions/checkout's default.
+      revision: ${{ needs.merge-upstream.outputs.revision }}
       cpp:      ${{ steps.filter.outputs.cpp }}
       java:     ${{ steps.filter.outputs.java }}
       tools_it: ${{ steps.filter.outputs.tools_it }}
     steps:
       - uses: actions/checkout@v7
         with:
+          ref: ${{ needs.merge-upstream.outputs.revision }}
           fetch-depth: 0
       - name: Detect changed paths
         id: filter
         run: |
-          BASE=${{ github.event.pull_request.base.sha }}
-          HEAD=${{ github.sha }}
+          if [ "${{ needs.merge-upstream.outputs.relevant }}" = "false" ]; then
+            for flag in cpp java tools_it; do
+              echo "$flag=false" >> $GITHUB_OUTPUT
+            done
+            exit 0
+          fi
+          if [ -n "${{ needs.merge-upstream.outputs.base_sha }}" ]; then
+            BASE=${{ needs.merge-upstream.outputs.base_sha }}
+            HEAD=$(git rev-parse HEAD)
+          else
+            BASE=${{ github.event.pull_request.base.sha }}
+            HEAD=${{ github.sha }}
+          fi
           changed=$(git diff --name-only "$BASE" "$HEAD")
           echo "$changed"
 
           match() { echo "$changed" | grep -qE "$1" && echo true || echo 
false; }
 
           echo "cpp=$(match      '^(cpp/|ep/build-velox/|dev/)')"              
                              >> $GITHUB_OUTPUT
-          echo "java=$(match     
'^(\.github/workflows/|pom\.xml|backends-velox/|gluten-(uniffle|celeborn|ras|core|substrait|arrow|delta|iceberg|hudi|paimon)/|gluten-ut/(common/|test/|pom\.xml)|package/|build/mvn)')"
 >> $GITHUB_OUTPUT
+          echo "java=$(match     
'^(\.github/(workflows|actions)/|pom\.xml|backends-velox/|gluten-(uniffle|celeborn|ras|core|substrait|arrow|delta|iceberg|hudi|paimon)/|gluten-ut/(common/|test/|pom\.xml)|package/|build/mvn)')"
 >> $GITHUB_OUTPUT
           echo "tools_it=$(match '^tools/gluten-it/')"                         
                              >> $GITHUB_OUTPUT
 
   build-native-lib-centos-9:
     needs: detect-changes
+    if: needs.detect-changes.outputs.relevant != 'false'
     runs-on: ubuntu-24.04-arm
     container: apache/gluten:vcpkg-centos-9
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Install Stash restore dependencies # the stash action needs gh 
and jq, absent from this container
         run: |
           case "$(uname -m)" in
@@ -102,8 +167,15 @@ jobs:
         with:
           path: '${{ env.CCACHE_DIR }}'
           key: ccache-centos9-release-default-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: '${{ env.CCACHE_DIR }}'
+          key: ccache-centos9-release-default-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: '${{ env.CCACHE_DIR }}'
@@ -147,6 +219,8 @@ jobs:
     container: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -185,6 +259,8 @@ jobs:
     container: apache/gluten:centos-9-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Install Stash restore dependencies # the stash action needs gh 
and jq, absent from this container
         run: |
           case "$(uname -m)" in
@@ -203,8 +279,15 @@ jobs:
         with:
           path: '${{ env.CCACHE_DIR }}'
           key: ccache-centos9-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: '${{ env.CCACHE_DIR }}'
+          key: ccache-centos9-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: '${{ env.CCACHE_DIR }}'
diff --git a/.github/workflows/velox_backend_x86.yml 
b/.github/workflows/velox_backend_x86.yml
index 5a4a22fcd2..c7a9ecc288 100644
--- a/.github/workflows/velox_backend_x86.yml
+++ b/.github/workflows/velox_backend_x86.yml
@@ -24,6 +24,7 @@ on:
       - '.github/workflows/iceberg_upstream_revision.yml'
       - '.github/workflows/velox_native_build.yml'
       - '.github/workflows/util/iceberg-upstream/**'
+      - '.github/actions/**'
       - 'pom.xml'
       - 'backends-velox/**'
       - 'gluten-uniffle/**'
@@ -45,6 +46,12 @@ on:
       - '!cpp/bolt.CMakeLists.cmake'
       - 'dev/**'
       - 'build/mvn'
+  # A pull request from a fork is tested in the fork: each push to a branch
+  # other than main runs this workflow there (see detect-changes). In
+  # apache/gluten a push run skips every job.
+  push:
+    branches-ignore:
+      - main
   # Nightly full run. PR runs trim the TPC matrices to representative JDKs and
   # skip the OOM / random-kill chaos jobs unless native code changed (see
   # detect-changes' `full_run` output); this scheduled run restores the full
@@ -73,10 +80,51 @@ env:
   UT_PARALLELISM: 3
 
 concurrency:
-  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ 
github.workflow }}
+  # head_ref is empty on push. Keying push runs by branch lets a new push 
cancel the previous run.
+  group: >-
+    ${{ github.repository }}-${{ github.head_ref || (github.event_name == 
'push' && github.ref) ||
+    github.sha }}-${{ github.workflow }}
   cancel-in-progress: true
 
 jobs:
+  # Every other job chains off this one, so its `if` decides whether the 
workflow runs:
+  #   • push / workflow_dispatch in a fork: the run tests a fork pull request.
+  #   • pull_request: only in apache/gluten and only from its own branches
+  #     (Dependabot); a fork's pull request is tested by the push run in the 
fork.
+  #   • schedule: only in apache/gluten. A fork with Actions enabled would
+  #     otherwise burn ~1800 runner minutes nightly on a signal nobody reads.
+  # In a fork it squash-merges the commit under test onto apache/gluten main 
and
+  # publishes the result, which the jobs below check out by `revision`. It is 
the
+  # only job that can write, and none of its steps run in apache/gluten.
+  merge-upstream:
+    if: >-
+      (github.repository_owner != 'apache' &&
+       (github.event_name == 'push' || github.event_name == 
'workflow_dispatch')) ||
+      (github.event_name == 'pull_request' && github.repository_owner == 
'apache' &&
+       github.event.pull_request.head.repo.full_name == github.repository) ||
+      (github.event_name == 'schedule' && github.repository_owner == 'apache') 
||
+      (github.event_name == 'workflow_dispatch' && github.repository_owner == 
'apache')
+    runs-on: ubuntu-24.04
+    permissions:
+      contents: write
+    outputs:
+      relevant: ${{ steps.merge.outputs.relevant }}
+      revision: ${{ steps.merge.outputs.revision }}
+      base_sha: ${{ steps.merge.outputs.base_sha }}
+    steps:
+      - uses: actions/checkout@v7
+        if: github.repository_owner != 'apache'
+        with:
+          fetch-depth: 0
+      - name: Squash-merge onto apache/gluten main
+        id: merge
+        if: github.repository_owner != 'apache'
+        uses: ./.github/actions/squash-merge-upstream
+        with:
+          # The `paths:` list above. Keep the two in sync.
+          paths: 
'^(\.github/workflows/(velox_backend_x86|iceberg_spark_ut|iceberg_upstream_revision|velox_native_build)\.yml$|\.github/workflows/util/(parallel-ut|iceberg-upstream)/|\.github/actions/|pom\.xml$|build/mvn$|(backends-velox|gluten-(uniffle|celeborn|core|substrait|arrow|delta|iceberg|hudi|paimon|ut)|package|shims|tools/gluten-it|ep/build-velox|cpp|dev)/)'
+          paths-ignore: '^cpp/bolt/|^cpp/bolt\.CMakeLists\.cmake$'
+
   # Detect which parts of the codebase were modified so downstream jobs can
   # skip work that is irrelevant to the change (e.g. a Spark-3.4-shim-only 
change does
   # not need to run spark-test-spark35/40/41; C++ changes still run 
spark-test-*
@@ -98,12 +146,15 @@ jobs:
   # widen their matrices to the extended combos that PR runs skip (alternative
   # JDKs, older Celeborn, chaos tests on java-only changes).
   detect-changes:
-    # Every job chains off this one, so the fork guard here skips the whole
-    # workflow: a fork with Actions enabled would otherwise burn ~1800 runner
-    # minutes nightly on a signal nobody reads.
-    if: github.event_name != 'schedule' || github.repository_owner == 'apache'
+    needs: merge-upstream
     runs-on: ubuntu-24.04
     outputs:
+      # false only on a fork run that changes nothing in the `paths:` list 
above.
+      relevant: ${{ needs.merge-upstream.outputs.relevant }}
+      # The commit the other jobs check out: the merged commit on a fork run, 
otherwise
+      # empty, which keeps actions/checkout's default.
+      revision: ${{ needs.merge-upstream.outputs.revision }}
+      base_sha: ${{ needs.merge-upstream.outputs.base_sha }}
       full_run: ${{ steps.filter.outputs.full_run }}
       cpp:      ${{ steps.filter.outputs.cpp }}
       java:     ${{ steps.filter.outputs.java }}
@@ -121,12 +172,15 @@ jobs:
     steps:
       - uses: actions/checkout@v7
         with:
+          ref: ${{ needs.merge-upstream.outputs.revision }}
           fetch-depth: 0
       - name: Detect changed paths
         id: filter
         run: |
           event="${{ github.event_name }}"
-          if [ "$event" = "schedule" ] || [ "$event" = "workflow_dispatch" ]; 
then
+          # A fork run, including a manual one, diffs its merged commit 
against main.
+          if [ -z "${{ needs.merge-upstream.outputs.base_sha }}" ] &&
+             { [ "$event" = "schedule" ] || [ "$event" = "workflow_dispatch" 
]; }; then
             # schedule / workflow_dispatch: no PR diff to inspect.
             # For scheduled runs, skip entirely when no commits landed in the
             # last 24 hours — avoids burning runner minutes on idle days.
@@ -149,15 +203,26 @@ jobs:
             exit 0
           fi
           echo "full_run=false" >> $GITHUB_OUTPUT
-          BASE=${{ github.event.pull_request.base.sha }}
-          HEAD=${{ github.sha }}
+          if [ "${{ needs.merge-upstream.outputs.relevant }}" = "false" ]; then
+            for flag in cpp java udf shims34 shims35 shims40 shims41 shims42 
tools_it; do
+              echo "$flag=false" >> $GITHUB_OUTPUT
+            done
+            exit 0
+          fi
+          if [ -n "${{ needs.merge-upstream.outputs.base_sha }}" ]; then
+            BASE=${{ needs.merge-upstream.outputs.base_sha }}
+            HEAD=$(git rev-parse HEAD)
+          else
+            BASE=${{ github.event.pull_request.base.sha }}
+            HEAD=${{ github.sha }}
+          fi
           changed=$(git diff --name-only "$BASE" "$HEAD")
           echo "$changed"
 
           match() { echo "$changed" | grep -qE "$1" && echo true || echo 
false; }
 
           echo "cpp=$(match      '^(cpp/|ep/build-velox/|dev/)')"              
                              >> $GITHUB_OUTPUT
-          echo "java=$(match     
'^(\.github/workflows/|pom\.xml|backends-velox/|gluten-(uniffle|celeborn|ras|core|substrait|arrow|delta|iceberg|hudi|paimon)/|gluten-ut/(common/|test/|src-iceberg-upstream/|pom\.xml)|package/|build/mvn)')"
 >> $GITHUB_OUTPUT
+          echo "java=$(match     
'^(\.github/(workflows|actions)/|pom\.xml|backends-velox/|gluten-(uniffle|celeborn|ras|core|substrait|arrow|delta|iceberg|hudi|paimon)/|gluten-ut/(common/|test/|src-iceberg-upstream/|pom\.xml)|package/|build/mvn)')"
 >> $GITHUB_OUTPUT
           echo "udf=$(match      
'^(backends-velox/src/test/scala/org/apache/gluten/expression/VeloxUdfSuite\.scala|\.github/workflows/velox_backend_x86\.yml)$')"
 >> $GITHUB_OUTPUT
           echo "shims34=$(match  
'^(shims/(spark34|common)/|gluten-ut/spark34/)')"                           >> 
$GITHUB_OUTPUT
           echo "shims35=$(match  
'^(shims/(spark35|common)/|gluten-ut/spark35/)')"                           >> 
$GITHUB_OUTPUT
@@ -168,6 +233,7 @@ jobs:
 
   build-native-lib-centos-8:
     needs: detect-changes
+    if: needs.detect-changes.outputs.relevant != 'false'
     runs-on: ubuntu-24.04
     container: apache/gluten:vcpkg-almalinux-8-gcc13
     steps:
@@ -178,14 +244,23 @@ jobs:
           dnf config-manager --add-repo 
https://cli.github.com/packages/rpm/gh-cli.repo
           dnf install -y gh --setopt=gpgcheck=0
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Get Ccache from Apache Stash
         id: ccache-stash
         uses: 
apache/infrastructure-actions/stash/restore@69afc125e535c4c41e7f1b7470f583087e0f344b
         with:
           path: '${{ env.CCACHE_DIR }}'
           key: ccache-almalinux8-release-default-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: '${{ env.CCACHE_DIR }}'
+          key: ccache-almalinux8-release-default-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: '${{ env.CCACHE_DIR }}'
@@ -264,6 +339,8 @@ jobs:
     container: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -394,6 +471,8 @@ jobs:
     container: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -474,6 +553,8 @@ jobs:
           sudo docker image prune --all --force > /dev/null
           df -h
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -592,6 +673,8 @@ jobs:
           sudo docker image prune --all --force > /dev/null
           df -h
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -634,6 +717,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -691,6 +776,8 @@ jobs:
     container: centos:8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Native Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -766,6 +853,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -837,6 +926,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -888,6 +979,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -961,6 +1054,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1024,6 +1119,8 @@ jobs:
     container: apache/gluten:centos-8-jdk8
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1069,6 +1166,7 @@ jobs:
     steps:
       - uses: actions/checkout@v7
         with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
           # 'check.py tidy commit' diffs HEAD^..HEAD, which a depth-1 clone 
lacks: the
           # diff comes back empty and the check passes without inspecting 
anything.
           fetch-depth: 2
@@ -1090,8 +1188,15 @@ jobs:
         with:
           path: '${{ env.CCACHE_DIR }}'
           key: ccache-centos9-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: '${{ env.CCACHE_DIR }}'
+          key: ccache-centos9-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: '${{ env.CCACHE_DIR }}'
@@ -1207,6 +1312,8 @@ jobs:
     uses: ./.github/workflows/iceberg_spark_ut.yml
     with:
       native_artifact: velox-native-lib-centos-8-${{github.sha}}
+      revision: ${{ needs.detect-changes.outputs.revision }}
+      base_revision: ${{ needs.detect-changes.outputs.base_sha }}
 
   build-cudf-centos-9:
     needs: detect-changes
@@ -1221,14 +1328,23 @@ jobs:
       - run: df -h | sort -k 5 -nr # check disk space for debug
 
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Get Ccache from Apache Stash
         id: ccache-stash
         uses: 
apache/infrastructure-actions/stash/restore@69afc125e535c4c41e7f1b7470f583087e0f344b
         with:
           path: '${{ env.CCACHE_DIR }}'
           key: ccache-centos9-cudf-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: '${{ env.CCACHE_DIR }}'
+          key: ccache-centos9-cudf-release-shared-${{runner.arch}}-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: '${{ env.CCACHE_DIR }}'
@@ -1277,6 +1393,8 @@ jobs:
     container: apache/gluten:centos-8-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Build with fast-build profile (Spark 4.0, Java 17)
         run: |
           cd $GITHUB_WORKSPACE/
@@ -1292,6 +1410,8 @@ jobs:
     container: apache/gluten:centos-8-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Build Spark 4.2 Velox (Java 17, Scala 2.13)
         run: |
           cd $GITHUB_WORKSPACE/
@@ -1313,6 +1433,8 @@ jobs:
     container: apache/gluten:centos-8-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1379,6 +1501,8 @@ jobs:
     container: apache/gluten:centos-8-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1436,6 +1560,8 @@ jobs:
     container: apache/gluten:centos-9-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1508,6 +1634,8 @@ jobs:
     container: apache/gluten:centos-9-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1565,6 +1693,8 @@ jobs:
     container: apache/gluten:centos-9-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
@@ -1640,6 +1770,8 @@ jobs:
     container: apache/gluten:centos-9-jdk17
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.detect-changes.outputs.revision }}
       - name: Download All Artifacts
         uses: actions/download-artifact@v8
         with:
diff --git a/.github/workflows/velox_kafka.yml 
b/.github/workflows/velox_kafka.yml
index 0acab52607..e65bd4310c 100644
--- a/.github/workflows/velox_kafka.yml
+++ b/.github/workflows/velox_kafka.yml
@@ -19,6 +19,7 @@ on:
   pull_request:
     paths:
       - '.github/workflows/velox_kafka.yml'
+      - '.github/actions/**'
       - 'pom.xml'
       - 'backends-velox/**'
       - 'gluten-kafka/**'
@@ -32,6 +33,12 @@ on:
       - 'ep/build-velox/**'
       - 'dev/**'
       - 'build/mvn'
+  # A pull request from a fork is tested in the fork: each push to a branch
+  # other than main runs this workflow there (see merge-upstream). In
+  # apache/gluten a push run skips every job.
+  push:
+    branches-ignore:
+      - main
   workflow_dispatch:
 
 permissions:
@@ -39,7 +46,10 @@ permissions:
   actions: read
 
 concurrency:
-  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ 
github.workflow }}
+  # head_ref is empty on push. Keying push runs by branch lets a new push 
cancel the previous run.
+  group: >-
+    ${{ github.repository }}-${{ github.head_ref || (github.event_name == 
'push' && github.ref) ||
+    github.sha }}-${{ github.workflow }}
   cancel-in-progress: true
 
 env:
@@ -47,12 +57,52 @@ env:
   PROMPT_ALWAYS_RESPOND: 'y'
 
 jobs:
+  # Every other job chains off this one, so its `if` decides whether the 
workflow runs:
+  # push and workflow_dispatch in forks, where the run tests a fork pull 
request;
+  # pull_request only in apache/gluten and only from its own branches 
(Dependabot);
+  # workflow_dispatch in apache/gluten.
+  # In a fork it squash-merges the commit under test onto apache/gluten main 
and
+  # publishes the result, which the jobs below check out by `revision`. It is 
the
+  # only job that can write, and none of its steps run in apache/gluten.
+  merge-upstream:
+    if: >-
+      (github.event_name == 'push' && github.repository_owner != 'apache') ||
+      (github.event_name == 'pull_request' && github.repository_owner == 
'apache' &&
+       github.event.pull_request.head.repo.full_name == github.repository) ||
+      github.event_name == 'workflow_dispatch'
+    runs-on: ubuntu-24.04
+    permissions:
+      contents: write
+    outputs:
+      # false only on a fork run that changes nothing in the `paths:` list 
above.
+      relevant: ${{ steps.merge.outputs.relevant }}
+      # The commit the other jobs check out: the merged commit on a fork run, 
otherwise
+      # empty, which keeps actions/checkout's default.
+      revision: ${{ steps.merge.outputs.revision }}
+    steps:
+      - uses: actions/checkout@v7
+        if: github.repository_owner != 'apache'
+        with:
+          fetch-depth: 0
+      - name: Squash-merge onto apache/gluten main
+        id: merge
+        if: github.repository_owner != 'apache'
+        uses: ./.github/actions/squash-merge-upstream
+        with:
+          # The `paths:` list above. Keep the two in sync.
+          paths: 
'^(\.github/workflows/velox_kafka\.yml$|\.github/actions/|pom\.xml$|build/mvn$|(backends-velox|gluten-(kafka|core|substrait|ut)|shims|cpp|ep/build-velox|dev)/)'
+          paths-ignore: '^cpp/bolt/|^cpp/bolt\.CMakeLists\.cmake$'
+
   build-native:
+    needs: merge-upstream
+    if: needs.merge-upstream.outputs.relevant != 'false'
     runs-on: ubuntu-24.04
     container: apache/gluten:vcpkg-almalinux-8-gcc13
     timeout-minutes: 180
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.merge-upstream.outputs.revision }}
       - name: Install build tools
         run: |
           dnf install -y jq bison tzdata
@@ -66,8 +116,15 @@ jobs:
         with:
           path: ${{ env.CCACHE_DIR }}
           key: ccache-almalinux8-release-default-${{ 
hashFiles('ep/build-velox/src/**') }}
+      - name: Get Ccache from apache/gluten main (fork run on Stash miss)
+        id: ccache-upstream
+        if: github.repository_owner != 'apache' && 
steps.ccache-stash.outputs.stash-hit != 'true'
+        uses: ./.github/actions/restore-upstream-ccache
+        with:
+          path: ${{ env.CCACHE_DIR }}
+          key: ccache-almalinux8-release-default-${{ 
hashFiles('ep/build-velox/src/**') }}
       - name: Get Ccache from actions/cache (fallback on Stash miss)
-        if: steps.ccache-stash.outputs.stash-hit != 'true'
+        if: steps.ccache-stash.outputs.stash-hit != 'true' && 
steps.ccache-upstream.outputs.hit != 'true'
         uses: actions/cache/restore@v4
         with:
           path: ${{ env.CCACHE_DIR }}
@@ -79,6 +136,8 @@ jobs:
         run: |
           source /opt/rh/gcc-toolset-13/enable
           export NUM_THREADS=2 CCACHE_MAXSIZE=1G
+          # As in dev/ci-velox-build*.sh, which build the stash restored above.
+          export CCACHE_BASEDIR=$GITHUB_WORKSPACE
           ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --enable_kafka=ON \
             --build_arrow=OFF --build_tests=ON --build_benchmarks=OFF 
--build_examples=OFF \
             --enable_s3=OFF --enable_gcs=OFF --enable_hdfs=OFF 
--enable_abfs=OFF
@@ -127,6 +186,8 @@ jobs:
           --health-retries 12
     steps:
       - uses: actions/checkout@v7
+        with:
+          ref: ${{ needs.merge-upstream.outputs.revision }}
       - name: Setup java
         run: |
           sudo apt-get update
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 294afc15be..412b1a5716 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -76,6 +76,15 @@ To re-trigger CH CI, please post the below comment on PR 
page:
 ##### Velox Backend CI
 To check CI failure for Velox backend, please go into the GitHub action page 
from your PR page.
 
+For a pull request from a fork, the Velox x86, ARM and Kafka workflows 
(`velox_backend_x86.yml`, `velox_backend_arm.yml` and `velox_kafka.yml`) run in 
your fork each time you push the PR branch, not in apache/gluten. To make that 
work:
+
+- Enable GitHub Actions in the Actions tab of your fork.
+- Open the pull request from a branch other than your fork's `main`.
+
+Each run tests your branch merged onto the current apache/gluten `main`, but 
it runs the workflow files from your branch. After a CI change lands on `main`, 
merge or rebase onto `main` and push to pick it up.
+
+The `Velox Backend (x86) in fork`, `Velox Backend (ARM) in fork` and `Velox 
Kafka in fork` checks on your pull request link to the runs in your fork. They 
are updated every 5 minutes, and a check fails as soon as a job in its run 
fails. To rerun failed jobs, rerun them in your fork. If you can't, a committer 
can push your branch to their own fork and link that run on the pull request.
+
 To see the perf. impact on Velox backend, you can comment `/Benchmark Velox` 
on PR page to trigger a pretest. The benchmark
 (currently TPC-H) result will be posted after completed.
 
@@ -83,6 +92,8 @@ To see the perf. impact on Velox backend, you can comment 
`/Benchmark Velox` on
 
 Please ensure no CI failure is reported for your PR.
 
+The fork runs of a PR use the workflow files from its branch, so review 
changes under `.github/` with that in mind.
+
 A PR may need several iterations in the community review process. Please 
@reviewer once your PR is ready for next round of review.
 
 For critical code change, merging the PR requires at least two committers' 
approval.
diff --git a/dev/ci-velox-buildshared-centos-9.sh 
b/dev/ci-velox-buildshared-centos-9.sh
index 02fcd50a88..4ba1d91eb9 100755
--- a/dev/ci-velox-buildshared-centos-9.sh
+++ b/dev/ci-velox-buildshared-centos-9.sh
@@ -25,6 +25,10 @@ if [ "$(uname -m)" = "aarch64" ]; then
 fi
 
 
+# Rewrite absolute paths under the checkout to relative ones, so a ccache 
saved by
+# a checkout at another path still hits.
+export CCACHE_BASEDIR=$(cd "$(dirname "$BASH_SOURCE")/.."; pwd)
+
 export VELOX_BUILD_SHARED=ON 
 
 ./dev/builddeps-veloxbe.sh --run_setup_script=OFF --build_arrow=OFF 
--build_tests=ON \
diff --git a/dev/ci-velox-buildstatic-centos-8-enhanced-features.sh 
b/dev/ci-velox-buildstatic-centos-8-enhanced-features.sh
index 176a13bd18..3d211c8c77 100755
--- a/dev/ci-velox-buildstatic-centos-8-enhanced-features.sh
+++ b/dev/ci-velox-buildstatic-centos-8-enhanced-features.sh
@@ -24,6 +24,10 @@ if [ "$(uname -m)" = "aarch64" ]; then
     export VCPKG_FORCE_SYSTEM_BINARIES=1;
 fi
 
+# Rewrite absolute paths under the checkout to relative ones, so a ccache 
saved by
+# a checkout at another path still hits.
+export CCACHE_BASEDIR=$(cd "$(dirname "$BASH_SOURCE")/.."; pwd)
+
 export NUM_THREADS=2
 ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --build_arrow=OFF 
--build_tests=ON --build_benchmarks=ON \
                            --build_examples=OFF --enable_s3=ON --enable_gcs=ON 
--enable_hdfs=ON --enable_abfs=ON --enable_enhanced_features=ON
diff --git a/dev/ci-velox-buildstatic-centos-8.sh 
b/dev/ci-velox-buildstatic-centos-8.sh
index f5cd4ed884..c82ce63260 100644
--- a/dev/ci-velox-buildstatic-centos-8.sh
+++ b/dev/ci-velox-buildstatic-centos-8.sh
@@ -24,6 +24,10 @@ if [ "$(uname -m)" = "aarch64" ]; then
     export VCPKG_FORCE_SYSTEM_BINARIES=1;
 fi
 
+# Rewrite absolute paths under the checkout to relative ones, so a ccache 
saved by
+# a checkout at another path still hits.
+export CCACHE_BASEDIR=$(cd "$(dirname "$BASH_SOURCE")/.."; pwd)
+
 export NUM_THREADS=2
 ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --build_arrow=OFF 
--build_tests=OFF --build_benchmarks=OFF \
                            --build_examples=OFF --enable_s3=ON --enable_gcs=ON 
--enable_hdfs=ON --enable_abfs=ON
diff --git a/dev/ci-velox-buildstatic-centos-9.sh 
b/dev/ci-velox-buildstatic-centos-9.sh
index 15b9767e71..fa58cba77c 100755
--- a/dev/ci-velox-buildstatic-centos-9.sh
+++ b/dev/ci-velox-buildstatic-centos-9.sh
@@ -23,6 +23,10 @@ if [ "$(uname -m)" = "aarch64" ]; then
     export VCPKG_FORCE_SYSTEM_BINARIES=1;
 fi
 
+# Rewrite absolute paths under the checkout to relative ones, so a ccache 
saved by
+# a checkout at another path still hits.
+export CCACHE_BASEDIR=$(cd "$(dirname "$BASH_SOURCE")/.."; pwd)
+
 export NUM_THREADS=2
 ./dev/builddeps-veloxbe.sh --enable_vcpkg=ON --build_arrow=OFF 
--build_tests=OFF --build_benchmarks=OFF \
                            --build_examples=OFF --enable_s3=ON --enable_gcs=ON 
--enable_hdfs=ON --enable_abfs=ON


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to