Hello everyone,

Following the discussion
<https://lists.apache.org/thread.html/6sfqor414lxvwjnw3dvp5kzckms3t7zb>on
the dev list, I would like to call for lazy consensus on AIP-118:

https://cwiki.apache.org/confluence/x/-JXwGg

Appendix with the supporting data and implementation details:

https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974672/AIP-118+Appendix+%E2%80%94+evidence+and+implementation+details

In short: the trusted part of our CI moves from GitHub-hosted runners to
self-hosted runners in Airflow's AWS account, provisioned by RunsOn
(https://runs-on.com) with GitHub Actions remaining the orchestrator. The
runners are funded by the additional AWS Open Source credits that AWS
granted for this effort. GitHub-hosted runners stay the default for PRs
from forks and remain available as a fallback for everything else.

What is included, based on the feedback in the discussion:

* Scheduled canary runs and pushes to apache/airflow branches run on
  self-hosted runners. PRs from forks keep using GitHub-hosted runners,
  with a read-only token and no secrets, as today.
* Committers can opt a PR in with the "use self-hosted runners" label.
  The label counts only when a committer added it last. Any PR can be
  forced back to GitHub-hosted runners with the "use public runners"
  label. Windows jobs stay on GitHub-hosted runners.
* RunsOn was chosen over a self-managed EKS cluster and AWS CodeBuild:
  no cluster to operate, ephemeral AMD and ARM instances, and a free
  open-source license in exchange for a public acknowledgement (a text
  line in the README sponsors section). Other Apache projects (Arrow,
  DataFusion, ASF Infra) already use it. If the license or the AWS
  credits lapse, the fallback is the EKS design from the appendix or
  another cloud provider, with a revised AIP.
* Spending is monitored against the shared AWS budget, and self-hosted
runners do not become the default for all
  PRs until we know the budget supports it.

In Addition:
* A merge queue and other hosted-runner vendors (Blacksmith, Incredibuild)
  are out of scope here and can be discussed separately.
* As Jarek suggested, it would be great if we have a small group of
volunteer owners for maintenance, incidents and spending, with
notifications in a dedicated Slack
  channel.

The implementation is in https://github.com/apache/airflow/pull/74889,
which already ran the full test suite green on RunsOn. The PR will be
merged only after this lazy consensus passes.

The lazy consensus will last 72 hours, until 2026-10-22 18:00 UTC. If
there are no objections by then, AIP-118 is accepted.


Shahar

Reply via email to