wombatu-kun commented on code in PR #20476:
URL: https://github.com/apache/hudi/pull/20476#discussion_r4420025897
##########
.github/workflows/validate_source.yml:
##########
@@ -63,6 +69,26 @@ jobs:
exit 1
fi
+ # Seconds of node tests of the fork CI status bridge and of the ci.yml
routing they check, so
+ # they run on every PR instead of path-filtering on the two workflow
files. Unlike the PR
+ # metadata checks they need no PR data, so they also run on merge-queue
groups and pushes.
+ # The tests read both workflow files through PyYAML. A pip install into
the runner's system
+ # Python can be refused (PEP 668), so the step gets its own Python.
+ - name: Set up Python for the fork_ci_status.yml tests
+ if: ${{ !cancelled() }}
+ uses: actions/setup-python@v5
Review Comment:
`actions/setup-python@v5` targets Node 20, so every `validate-pr-and-source`
run now ends with a Node 20 deprecation warning. `@v6` runs on Node 24.
##########
.github/workflows/fork_ci_status.yml:
##########
@@ -0,0 +1,232 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Adapted from apache/arrow-rs .github/workflows/fork_ci_status.yml.
+# Mirror fork results as the informational CI (fork) check after upstream PR CI
+# completes and through scheduled polling for late runs and reruns. Read fork
+# runs and write upstream checks using reviewed default-branch code.
+# No checkout, artifacts, or code from the pull request are used here.
+# Test with Python 3.11+, PyYAML, and Node.js 24:
+# node --test .github/ci/scripts/test-fork-ci-status.cjs
+# Tests mock GitHub APIs. End-to-end testing requires a test repository with
+# the upstream guards adapted and this workflow installed on its default
branch.
+name: Fork CI status
+
+on:
+ workflow_run:
+ workflows: [CI]
+ types: [completed]
+ schedule:
+ - cron: '7,22,37,52 * * * *'
+ workflow_dispatch:
+
+permissions:
+ actions: read
+ checks: write
+ pull-requests: read
+
+# Like DataFusion, group notifications separately from scheduled/manual sweeps.
+# workflow_run identifies the fork and branch rather than the PR number.
+concurrency:
+ group: ${{ github.workflow }}-${{
github.event.workflow_run.head_repository.full_name || 'update' }}-${{
github.event.workflow_run.head_branch }}
+ cancel-in-progress: ${{ github.event_name == 'workflow_run' }}
+
+jobs:
+ bridge:
+ if: github.repository == 'apache/hudi' && (github.event_name !=
'workflow_run' || github.event.workflow_run.event == 'pull_request')
+ name: Mirror the fork run
+ runs-on: ubuntu-slim
+ timeout-minutes: 10
+ steps:
+ - name: Create or update the check
+ uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3
# v9.0.0
+ with:
+ script: |
+ // Adapted from apache/datafusion#25591. Fork results are
informational:
+ // contributors control their workflows; only upstream merge-queue
CI gates merging.
+ const CHECK = 'CI (fork)';
+ const WORKFLOW = 'ci.yml';
+ const upstream = { owner: context.repo.owner, repo:
context.repo.repo };
+ const repository = upstream.owner + '/' + upstream.repo;
+ const GUIDE =
'https://github.com/apache/hudi/blob/master/.github/workflows/README.md#pull-requests-from-forks';
+ const SWEEP_SIZE = 25;
+ const listing = { ...upstream, state: 'open', base: 'master',
sort: 'updated', direction: 'desc' };
+ const isForkPr = (pr) => pr.state === 'open' && pr.head.repo &&
+ pr.head.repo.full_name.toLowerCase() !==
repository.toLowerCase() && pr.base.ref === 'master';
+ const externalId = (pr) => 'hudi-fork-ci:' + pr.number;
+ const conclusions = new Set([
+ 'action_required', 'cancelled', 'failure', 'neutral', 'skipped',
'success', 'timed_out',
+ ]);
+ const UPDATE_NOTE =
+ 'This check is informational: the contributor controls the fork
workflow. ' +
+ 'The upstream merge queue runs the required CI checks on the
proposed merge result. ' +
+ 'Active or missing runs on recently updated PRs are polled every
15 minutes; ' +
+ 'a rotating sweep also discovers reruns on older PRs. Scheduled
updates may be delayed.';
+
+ // An absent workflow/run is expected on a fork with Actions
disabled. Other
+ // API errors must fail the bridge rather than falsely reporting
missing CI.
+ async function optional(request) {
+ try {
+ return (await request()).data;
+ } catch (error) {
+ if (error.status === 404) return null;
+ throw error;
+ }
+ }
+
+ async function syncCheck(pr, includeCompleted = true) {
+ if (!isForkPr(pr)) return;
+ const fork = { owner: pr.head.repo.owner.login, repo:
pr.head.repo.name };
+ const sha = pr.head.sha;
+ const checks = await
github.paginate(github.rest.checks.listForRef, {
+ ...upstream, ref: sha, check_name: CHECK, per_page: 100,
+ });
+ const check = checks.find((item) =>
+ item.app?.slug === 'github-actions' && item.external_id ===
externalId(pr));
+ let previous = {};
+ try { previous = JSON.parse(check?.output?.text || '{}'); }
catch {}
+ // A missing run must keep being polled, even though its check
is completed.
+ if (!includeCompleted && check?.status === 'completed' &&
previous.run_id) return;
Review Comment:
A successful re-run of a failed fork run is only picked up when the rotating
slice reaches the PR, even on recently updated PRs, and at today's 574 open PRs
to master that rotation takes 23 slices, up to about six hours. Is that lag
intended, or should recent PRs also re-poll completed checks whose conclusion
is not `success`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
