This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new bfc288222a3 Document fork CI for contributors and maintainers (#74825)
bfc288222a3 is described below
commit bfc288222a33c697c7354cc919d3ed373a1fc646
Author: Jarek Potiuk <[email protected]>
AuthorDate: Sat Oct 24 22:35:08 2026 +0200
Document fork CI for contributors and maintainers (#74825)
* Document fork CI for contributors and maintainers
Contributors need to know how to set up CI in their fork, how to check that
setup, and how to join or leave the pilot, and maintainers need to know which
parts of a gated pull request's lifecycle the reconciler owns so triage does
not fight it. The previous one-line advice to enable GitHub Actions in the fork
did not say any of that.
Generated-by: Claude Opus 5
* Say when fork CI applies in the contributor docs
While the gate is off, a contributor outside the pilot who enabled fork CI
as the docs said got CI twice on every pull request. The docs now say fork CI
applies to pilot participants and use fork ci pull requests until it is
switched on for everyone, and the maintainer docs cover the needs project ci
label, the CI passed status and why removing fork ci required is not enough.
Generated-by: Claude Opus 5
* Describe apache/airflow push runs after release-branch CI moved to a
schedule
ci-arm.yml no longer lists the push branches (#75420). Point to
CANONICAL_PUSH_BRANCH_FILTERS and say that the incremental push base is only
used in apache/airflow.
Generated-by: Claude Opus 5
* Fix the fork CI section title level and label wording in the contributor
docs
The new subsection used a title style the file had not established, which
made Newsfragments a level-4 section and broke links to it. Also say when a
pull request the author drafted gets ready for maintainer review, use a
placeholder pilot instant, and describe the fork compare base as breeze's
default branch rather than the branch name.
Generated-by: Claude Opus 5
---
contributing-docs/03_contributors_quick_start.rst | 13 +++++
.../03a_contributors_quick_start_beginners.rst | 5 ++
contributing-docs/05_pull_requests.rst | 10 +++-
contributing-docs/10_working_with_git.rst | 3 +
contributing-docs/18_contribution_workflow.rst | 65 +++++++++++++++++++++-
contributing-docs/25_maintainer_pr_triage.md | 33 +++++++++++
dev/breeze/doc/ci/05_workflows.md | 25 +++++++++
.../doc/ci/08_fork_ci_for_external_contributors.md | 1 -
8 files changed, 151 insertions(+), 4 deletions(-)
diff --git a/contributing-docs/03_contributors_quick_start.rst
b/contributing-docs/03_contributors_quick_start.rst
index 5adc87ad9dc..61b1af39148 100644
--- a/contributing-docs/03_contributors_quick_start.rst
+++ b/contributing-docs/03_contributors_quick_start.rst
@@ -237,6 +237,19 @@ Forking and cloning Project
to clone the repo locally (you can also do it in your IDE - see the `Using
your IDE`_
chapter below)
+4. Add ``apache/airflow`` as the ``upstream`` remote, following the
+ `Git remote naming conventions <10_working_with_git.rst>`__:
+
+ .. code-block:: bash
+
+ git remote add upstream https://github.com/apache/airflow.git
+
+ If you use SSH with GitHub, add ``[email protected]:apache/airflow.git``
instead; ``breeze ci audit``
+ recognises both forms.
+
+ Once Breeze is installed, run ``breeze ci audit`` before your first push to
check your remotes
+ and the CI in your fork; see `CI in your fork
<18_contribution_workflow.rst#ci-in-your-fork>`__.
+
.. note::
For windows based machines, on cloning, the Git line endings may be
different from unix based systems
and might lead to unexpected behaviour on running breeze tooling. Manually
setting a property will mitigate this issue.
diff --git a/contributing-docs/03a_contributors_quick_start_beginners.rst
b/contributing-docs/03a_contributors_quick_start_beginners.rst
index a8c333233ad..fdbea14df69 100644
--- a/contributing-docs/03a_contributors_quick_start_beginners.rst
+++ b/contributing-docs/03a_contributors_quick_start_beginners.rst
@@ -71,6 +71,11 @@ Option A – Breeze on Your Laptop
git clone https://github.com/<you>/airflow.git
cd airflow
+ git remote add upstream https://github.com/apache/airflow.git
+
+Before your first push, run ``breeze ci audit`` (after installing Breeze
below) to check that your
+remotes and the CI in your fork are set up; see
+`CI in your fork <18_contribution_workflow.rst#ci-in-your-fork>`__.
2. **Set up Git hooks and install Breeze**
diff --git a/contributing-docs/05_pull_requests.rst
b/contributing-docs/05_pull_requests.rst
index 4052986d8ba..c7fee6d2677 100644
--- a/contributing-docs/05_pull_requests.rst
+++ b/contributing-docs/05_pull_requests.rst
@@ -94,8 +94,14 @@ these guidelines:
The Airflow repo uses `GitHub Actions
<https://help.github.com/en/actions>`_ to
run the tests and `codecov <https://codecov.io/gh/apache/airflow>`_ to
track
- coverage. You can set up both for free on your fork. It will help you make
sure you do not
- break the build with your PR and that you help increase coverage.
+ coverage. Fork CI is being piloted: it applies to pilot participants and
to pull requests a
+ maintainer labels ``use fork ci``, and to every external contributor only
once it is switched
+ on for everyone. Until then, enabling the ``Tests (AMD)`` workflow in your
fork outside the
+ pilot runs CI twice for each pull request, in your fork and in
apache/airflow. When fork CI
+ applies to you, enable that workflow and every push to your fork runs CI
for free; pull
+ requests labelled ``fork ci required`` take their CI result from your fork
and stay in draft
+ until that run is green. See `CI in your fork
<18_contribution_workflow.rst#ci-in-your-fork>`__
+ and run ``breeze ci audit``, which tells you whether fork CI applies to
you yet.
Also we advise to install locally `prek hooks
<08_static_code_checks.rst#prek-hooks>`_ to
apply various checks, code generation and formatting at the time you make
a local commit - which
gives you near-immediate feedback on things you need to fix before you
push your code to the PR, or in
diff --git a/contributing-docs/10_working_with_git.rst
b/contributing-docs/10_working_with_git.rst
index e51cb1a1f21..edb6b54188d 100644
--- a/contributing-docs/10_working_with_git.rst
+++ b/contributing-docs/10_working_with_git.rst
@@ -37,6 +37,9 @@ you should configure your local checkout to match:
* ``origin`` — your personal fork of ``apache/airflow`` (where you push
branches
for PRs).
+``breeze ci audit`` checks these names, offers the commands below when they
differ, and also checks
+the CI in your fork; see `CI in your fork
<18_contribution_workflow.rst#ci-in-your-fork>`__.
+
Always push PR branches to ``origin``; don't push to ``upstream`` (the branch
protection on ``apache/airflow`` will reject it anyway). Working on dedicated
branches is recommended, though developing directly on your fork's ``main`` is
diff --git a/contributing-docs/18_contribution_workflow.rst
b/contributing-docs/18_contribution_workflow.rst
index b96f4f16238..1a0953d6617 100644
--- a/contributing-docs/18_contribution_workflow.rst
+++ b/contributing-docs/18_contribution_workflow.rst
@@ -43,7 +43,8 @@ In general, your contribution includes the following stages:
2. Create a `local virtualenv <07_local_virtualenv.rst>`_,
initialize the `Breeze environment <../dev/breeze/doc/README.rst>`__, and
install `prek tool <08_static_code_checks.rst#prek-hooks>`__.
- If you want to add more changes in the future, set up your fork and enable
GitHub Actions.
+ Set up the ``upstream`` and ``origin`` remotes and check them with ``breeze
ci audit``; if
+ fork CI applies to you (see `CI in your fork`_), enable CI in your fork as
described there.
3. Join `devlist
<https://lists.apache.org/[email protected]>`__
and set up a `Slack account <https://s.apache.org/airflow-slack>`__.
@@ -126,6 +127,68 @@ to make them immediately visible in the environment.
6. Open your IDE (for example, PyCharm) and select the virtual environment you
created
as the project's default virtual environment in your IDE.
+CI in your fork
+~~~~~~~~~~~~~~~
+
+Pull requests from contributors who are not committers or collaborators are
moving to running
+their CI in the contributor's own fork, on GitHub's free runners, instead of
on the shared CI
+capacity of the Apache Software Foundation (`AIP-120
+<https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks>`__).
+CI on a public fork costs you nothing and has no minute limit; it runs on
every push to your
+fork, so you get feedback before a maintainer looks at your pull request. The
change is being
+piloted: until it is switched on for everyone, it applies to pilot
participants (see below) and
+to pull requests a maintainer labels ``use fork ci``.
+
+Set up your checkout and fork once:
+
+1. Name your remotes as described in `Git remote naming conventions
<10_working_with_git.rst>`__:
+ ``upstream`` for ``apache/airflow`` and ``origin`` for your fork.
+
+2. If fork CI applies to you, enable the ``Tests (AMD)`` workflow in your
fork: today that means
+ you joined the pilot or a maintainer labelled your pull request ``use fork
ci``; once fork CI is
+ switched on for everyone, it means you are not a committer, a collaborator
or a member of an
+ exempted organisation. GitHub disables workflows in new forks. Otherwise
keep it disabled, so
+ your pushes do not run CI twice:
+
+ .. code-block:: bash
+
+ gh workflow enable ci-amd.yml -R <you>/airflow
+
+ If the workflow is missing, your fork's ``main`` is older than it; sync it
first with
+ ``gh repo sync <you>/airflow --branch main``.
+
+3. Check everything at once. ``breeze ci audit`` prints one row per check and
the command that fixes
+ anything missing; ``--fix`` runs those commands after asking:
+
+ .. code-block:: bash
+
+ breeze ci audit
+
+When fork CI applies to your pull request, it is labelled ``fork ci required``
and kept in draft
+until the ``Tests (AMD)`` run in your fork for its latest commit is green. A
+``Fork CI / Tests (AMD)`` status on the pull request links to that run, and
one comment on the pull
+request explains the current state. Once the run is green, a pull request the
monitor drafted is
+marked ready for review and labelled ``ready for maintainer review``; a pull
request you drafted
+yourself stays a draft, and gets the label, once you mark it ready. A pull
request whose fork CI
+does not run for 7 days is closed with an explanation; you can reopen it after
setting up fork CI. If your pull request changes
+files that define what CI runs, such as ``.github/``, ``dev/breeze/`` or
``scripts/ci/``, a green
+fork run is not enough: a maintainer reviews the change and runs project CI
for it.
+
+**Joining the pilot.** Open a pull request that adds your GitHub login to the
``[pilot]`` table in
+``.github/fork-ci-gate.toml``, with a join instant after the expected merge,
for example the next day
+at 00:00 UTC:
+
+.. code-block:: toml
+
+ [pilot]
+ "your-login" = YYYY-MM-DDT00:00:00Z
+
+Every pull request you open after that instant uses fork CI; your earlier pull
requests keep project
+CI. To leave the pilot, open a pull request that removes the line. A pilot
entry gates your pull
+requests whatever your role, so a committer in the pilot also enables the
workflow;
+``breeze ci audit`` tells you which applies to you. The design is described in
+`Fork CI for external contributors
<../dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md>`__.
+
Step 3: Connect with People
---------------------------
diff --git a/contributing-docs/25_maintainer_pr_triage.md
b/contributing-docs/25_maintainer_pr_triage.md
index 046a1b20fa2..ab12c1f877d 100644
--- a/contributing-docs/25_maintainer_pr_triage.md
+++ b/contributing-docs/25_maintainer_pr_triage.md
@@ -242,6 +242,39 @@ The triage workflow uses the following labels and states:
| `suspicious changes detected` | PR (and all open PRs by the same author) was
closed because the diff contained suspicious patterns (secret exfiltration,
malicious CI modifications, etc.). |
| `closed because of open PR limit` | PR was closed in the one-time closure
that introduced the [open PR limit](32_open_pull_request_limit.rst). Applied by
`dev/close_prs_over_open_pr_limit.py`, never by the triage skill. If the author
reopens the PR, triage it like any other. |
| **Draft status** | PR was converted to draft because it does not meet
quality criteria. The contributor is asked to fix the listed issues and mark
the PR as "Ready for review" once done. |
+| `fork ci required` | PR runs its CI in the contributor's fork (AIP-120).
Applied and managed by the fork CI reconciler, never by the triage skill. |
+| `needs project ci` | Fork CI is green, but the PR changes CI-defining files,
so a maintainer must review them and add `use project ci`. Applied and removed
by the fork CI reconciler. Filter on it to find PRs waiting only on a
maintainer. |
+| `use project ci` / `use fork ci` | Set by a maintainer to override the fork
CI decision for one PR: run project CI, or use fork CI for a PR that would
otherwise run project CI. |
+
+## Pull requests gated to fork CI
+
+Pull requests labelled `fork ci required` run their CI in the contributor's
fork. The scheduled
+fork CI reconciler (`breeze ci fork-ci-reconcile`, run by
`fork-ci-monitor.yml`) owns four things
+for them, and the triage skill must treat those pull requests as out of scope
for each:
+
+- **Draft state.** The reconciler keeps the pull request in draft until the
fork run for its
+ latest commit is green, and undrafts only what it drafted.
+- **`ready for maintainer review`.** The reconciler adds it when the fork run
is green and the
+ pull request is not a draft the author made (it adds it once the author
marks it ready), and
+ removes it when a new push starts a new run.
+- **`needs project ci`.** The reconciler adds it when the fork run is green
but the pull request
+ changes CI-defining files, and removes it once the state is anything else.
These pull requests
+ stay in draft and wait for a maintainer to review the CI change and add `use
project ci`.
+- **Closing after inactivity.** The reconciler closes a pull request whose
fork CI has not run for
+ `close_after_days`.
+
+Everything else the skill does (quality criteria, pings, suspicious-change
flags) applies as
+usual. The `Fork CI / Tests (AMD)` commit status shows the fork CI state and
links to the run in
+the fork. To move a single pull request back to project CI, add `use project
ci`; the reconciler
+releases it on its next run; removing `fork ci required` by hand is not
enough, the next run
+labels a gated pull request again. Adding the label does not start project CI
by itself: re-run
+the pull request's latest `Tests (AMD)` run (`gh run rerun <run-id> -R
apache/airflow`, or
+**Re-run all jobs** on the run's page), or ask the author to push. Pull
requests that change CI-defining files stay in draft even
+when the fork run is green and are labelled `needs project ci`, for a
maintainer to review and run
+project CI with `use project ci`. When the fork run is green and trusted, the
reconciler also posts
+the `CI passed` status the merge ruleset requires, so these pull requests can
be merged or
+auto-merged without project CI. See
+[Fork CI for external
contributors](../dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md).
## For contributors
diff --git a/dev/breeze/doc/ci/05_workflows.md
b/dev/breeze/doc/ci/05_workflows.md
index c59950dc89f..eca830bcb7d 100644
--- a/dev/breeze/doc/ci/05_workflows.md
+++ b/dev/breeze/doc/ci/05_workflows.md
@@ -23,6 +23,7 @@
- [CI run types](#ci-run-types)
- [Pull request run](#pull-request-run)
+ - [Fork push run](#fork-push-run)
- [Canary run](#canary-run)
- [Workflows](#workflows)
- [Workflow Architecture Overview](#workflow-architecture-overview)
@@ -92,6 +93,30 @@ all dependencies to latest versions and build images with
them. This way we chec
dependencies are upgraded. This can also be forced by setting the `upgrade to
newer dependencies`
label in the PR if you are a committer and want to force dependency upgrade.
+Every pull request run of `ci-amd.yml` starts with the `Fork CI gate` job. For
pull requests
+whose CI runs in the contributor's fork (AIP-120), the gate skips `build-info`
and therefore
+everything after it, so the run finishes in seconds; see
+[Fork CI for external contributors](08_fork_ci_for_external_contributors.md).
+
+## Fork push run
+
+These runs are triggered by a push to any branch of a contributor's fork in
which the
+`Tests (AMD)` workflow is enabled. They run in the fork, on GitHub's public
runners and with the
+fork's token, and are how pull requests labelled `fork ci required` get their
CI result. They are
+selected like a pull request run: the changed files come from comparing the
pushed commit with
+`apache/airflow` (`main`, or the release branch the pushed code was cut from,
as breeze's default
+branch at that commit records it), so a fork whose `main` is behind does not
make the run bigger,
+and they never use self-hosted runners. The `schedule` trigger is skipped in
forks, so enabling
+the workflow does not start the canary matrix there. The fork CI reconciler
reads the result of these runs and reports it on the pull
+request as the `Fork CI / Tests (AMD)` status.
+
+In `apache/airflow` itself, pushes still only run CI for the release-prep and
providers branches
+(`CANONICAL_PUSH_BRANCH_FILTERS` in `fork_ci_gate.py`, the branches
`release-branch-ci.yml` starts
+full runs for), and they test only the changes since the last successful run
of the branch. That
+incremental base is looked up only in `apache/airflow`: a fork's push is
always compared with
+`apache/airflow` as described above. A push to any other branch produces a run
in which only the
+gate executes.
+
## Canary run
This workflow runs on a schedule on the `main` branch, and a few times a day
diff --git a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
index 6321692fc97..6e352fea115 100644
--- a/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
+++ b/dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md
@@ -818,7 +818,6 @@ as a failed check rather than a crash.
- `contributing-docs/10_working_with_git.rst`: cross-link from the naming
conventions.
- `contributing-docs/25_maintainer_pr_triage.md`: the ownership split with the
reconciler.
- `dev/breeze/doc/ci/05_workflows.md`: add the fork push run type.
-- `dev/breeze/doc/ci/03_github_variables.md`: the new gate job outputs.
- `dev/breeze/doc/08_ci_tasks.rst`: `breeze ci audit` and `breeze ci
fork-ci-reconcile`.
- `dev/breeze/doc/ci/README.md`: index this document.
