comphead commented on code in PR #26683:
URL: https://github.com/apache/datafusion/pull/26683#discussion_r4566026252
##########
.github/workflows/release_version_labeler.yaml:
##########
@@ -7,20 +7,17 @@
name: Release Version Labeler
-concurrency:
- group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{
github.workflow }}
- cancel-in-progress: true
-
+# `pull_request_target` is disabled by ASF infra policy, so label on the push
of
+# the merge commit to `main` and look up the merged pull request from it.
on:
- pull_request_target:
- types:
- - closed
+ push:
branches:
- main
Review Comment:
Sound, and it applies more widely than it looks. The merge queue squashes
with the PR body as the commit message, so a skip marker anywhere in a PR
description reaches `main`. None of the last 3000 commits on `main` has one,
but this PR's own description had two until today, which I removed. Fixed in
c24e43aac: each push run now labels every PR merged to `main` in the last 7
days that has no release label, with the version that
`release_version_labeler.js` names at the PR's merge commit. So a later push
labels a merge whose own push skipped the workflow, without depending on that
push. This replaces the lookup of the pushed commits. A test checks that
`dev/update_datafusion_versions.py` still rewrites only the version constant,
since the script now also reads that constant back at older commits.
##########
ci/scripts/release_version_labeler.js:
##########
@@ -4,6 +4,27 @@
// Used by the release_version_labeler workflow
+// A push can have many commits. Look them up this many at a time, to stay well
+// below GitHub's limit of 100 concurrent requests.
+const BATCH_SIZE = 10;
+
+// A squash merge or a merge queue commit maps to the merged pull request. The
+// pull request can lag the push by a few seconds, so retry.
+async function mergedPulls({ github, context, sha }) {
+ for (let attempt = 1; ; attempt++) {
+ const { data } = await
github.rest.repos.listPullRequestsAssociatedWithCommit({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ commit_sha: sha
+ });
+ const pulls = data.filter((pull) => pull.merged_at && pull.base.ref
=== 'main');
+ if (pulls.length > 0 || attempt === 3) {
Review Comment:
Fixed by the same change in c24e43aac. The run no longer looks up the pushed
commits, so there is nothing to wait for. It lists the recently merged PRs, and
for the last 40 merges `merged_at` was always set before the push run was
created. A PR that a run still misses, or a run that fails, is labeled by the
next push.
##########
.github/workflows/fork_ci_status.yml:
##########
@@ -199,13 +213,41 @@ jobs:
}
async function notify() {
- const pr = context.payload.pull_request;
+ // The same branch can also target a release branch, which gets
no
+ // mirrored check, so look only at the pull request to `main`.
+ const pulls = await
require('./ci/scripts/pull_requests_of_run.js')({
+ github,
+ context,
+ base: 'main',
+ });
+ // A run for an older commit finds nothing, and the scheduled
+ // update creates the check of the new head.
+ const pr = pulls.find(
+ (pull) => pull.head.sha ===
context.payload.workflow_run.head_sha
+ );
+ if (!pr) {
+ core.info('No open pull request to main at this commit;
nothing to mirror.');
+ return;
+ }
// Only fork PRs to main skip jobs upstream; release branches
have
// no merge queue and keep the full run.
if (!isForkPrToMain(pr)) {
core.info('Not a fork pull request to main; nothing to
mirror.');
return;
}
+ // A pull request of the same branch to a release branch, or a
+ // reopened pull request, signals a commit that can already have
+ // the check. Update that check instead of adding a second one.
+ const { data } = await github.rest.checks.listForRef({
Review Comment:
Sound. The overlap that matters is with the scheduled update, which has its
own concurrency group. Two signals for the same branch share one, so only the
newer runs. Fixed in cb97e0b67: `createCheck()` looks for the commit's check
only after the wait, and updates it if one exists. The scheduled update's
recovery path calls `createCheck()` too, so it is covered from that side as
well. A test creates the check while the run looks for the fork run, and it
fails with the earlier order.
##########
.github/workflows/fork_ci_status.yml:
##########
@@ -199,13 +213,41 @@ jobs:
}
async function notify() {
- const pr = context.payload.pull_request;
+ // The same branch can also target a release branch, which gets
no
+ // mirrored check, so look only at the pull request to `main`.
+ const pulls = await
require('./ci/scripts/pull_requests_of_run.js')({
+ github,
+ context,
+ base: 'main',
+ });
+ // A run for an older commit finds nothing, and the scheduled
+ // update creates the check of the new head.
+ const pr = pulls.find(
+ (pull) => pull.head.sha ===
context.payload.workflow_run.head_sha
+ );
+ if (!pr) {
+ core.info('No open pull request to main at this commit;
nothing to mirror.');
+ return;
+ }
// Only fork PRs to main skip jobs upstream; release branches
have
// no merge queue and keep the full run.
if (!isForkPrToMain(pr)) {
core.info('Not a fork pull request to main; nothing to
mirror.');
return;
}
+ // A pull request of the same branch to a release branch, or a
+ // reopened pull request, signals a commit that can already have
+ // the check. Update that check instead of adding a second one.
+ const { data } = await github.rest.checks.listForRef({
Review Comment:
Fixed in cb97e0b67. The existence check now lives only in `createCheck()`,
and `notify()` calls it directly, so there is one lookup.
##########
.github/workflows/fork_ci_status.yml:
##########
@@ -199,13 +213,41 @@ jobs:
}
async function notify() {
- const pr = context.payload.pull_request;
+ // The same branch can also target a release branch, which gets
no
+ // mirrored check, so look only at the pull request to `main`.
+ const pulls = await
require('./ci/scripts/pull_requests_of_run.js')({
+ github,
+ context,
+ base: 'main',
+ });
+ // A run for an older commit finds nothing, and the scheduled
+ // update creates the check of the new head.
+ const pr = pulls.find(
+ (pull) => pull.head.sha ===
context.payload.workflow_run.head_sha
+ );
+ if (!pr) {
+ core.info('No open pull request to main at this commit;
nothing to mirror.');
+ return;
+ }
// Only fork PRs to main skip jobs upstream; release branches
have
// no merge queue and keep the full run.
if (!isForkPrToMain(pr)) {
Review Comment:
Right, fixed in cb97e0b67. I removed the guard and moved its reason into the
comment on the `base: 'main'` lookup. The scheduled update still uses
`isForkPrToMain()`.
##########
.github/workflows/labeler.yml:
##########
@@ -4,36 +4,56 @@
name: Labeler
+# `pull_request_target` is disabled by ASF infra policy. This workflow runs
from
+# the default branch when `pull_request_signal.yml` starts, so it never runs
the
+# code of a pull request. See that file for details.
concurrency:
- group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{
github.workflow }}
+ group: ${{ github.repository }}-${{
github.event.workflow_run.head_repository.full_name }}-${{
github.event.workflow_run.head_branch }}-${{ github.workflow }}
cancel-in-progress: true
on:
- pull_request_target:
- types:
- - opened
- - edited
- - synchronize
+ workflow_run:
+ workflows: ["Pull request signal"]
+ types: [requested]
+
+permissions: {}
jobs:
process:
name: Process
runs-on: ubuntu-slim
- # only run for users whose permissions allow them to update PRs
- # otherwise labeler is failing:
+ if: github.event.workflow_run.event == 'pull_request'
+ # The token of a `workflow_run` workflow can update PRs from forks.
+ # Previously `pull_request_target` was used for the same reason, see
# https://github.com/apache/datafusion/issues/3743
permissions:
contents: read
pull-requests: write
steps:
+ # This checks out the default branch, never the pull request.
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 #
v7.0.1
+ with:
+ sparse-checkout: ci/scripts
Review Comment:
Confirmed in the source of the pinned `actions/labeler`: it reads the config
from the file when the file exists, and falls back to the contents API
otherwise. Fixed in cb97e0b67 by adding `.github/workflows/labeler` to the
sparse checkout. The checkout is the default branch, so the content is the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]