Hello everyone, Just to add a little spice.
Following our discussions during Community Over Code and the Airflow Contributor Day (which I felt was quite a success), I am even more convinced that adopting Airflow as our native AI orchestrator for various project tasks is the right path forward. The ASF is currently developing its own layer for agentic workflows, focusing on persistent AI workflows, institutional memory, privacy, LLM gateways, and governance. The Foundation’s organizational needs here are very similar to those of many corporations. While ASF is leading in this space—with foundations like Eclipse and the PSF trailing further behind—the orchestration layer itself is not yet fully ideated or developed. As the ASF builds these initial components, we have an opportunity to pioneer using them alongside Airflow to orchestrate workflows within our organization. Demonstrating that Airflow works well in this capacity could open the door for broader, ASF-wide adoption as a solution for organization-level needs. Furthermore, other organisations are closely watching ASF's progress and will likely follow our lead and learn from our experiences. While this is a bit forward-looking and the idea hasn't been widely circulated, I believe the need for the more complex orchestration Airflow offers will become obvious in a few months, even if current efforts are focused on lower-level building blocks. It would be great if we could demonstrate this capability early and propose Airflow as the basis for the ASF's internal orchestration, carrying it forward to other organisations. This would allow us to build a strong "Airflow as a Native AI orchestrator" narrative, backed by highly impactful and influential names. In November, I plan to spend significant time and energy traveling the world—from the Upstream unconference, Current, OWASP App Security, CNCF Kubecon, and Community Over Code Sydney, to all the Airflow and Python meetups I plan to visit in Japan, Korea, and Taiwan—to discuss, promote, and preach the idea of "Airflow as a native AI orchestrator," alongside Magpie discussions. Best, J. On Sun, Oct 18, 2026 at 1:09 PM Shubham Raj <[email protected]> wrote: > Dear Community Members, > > I am bringing this thread back to your attention for additional visibility. > If no further issues are raised, I can proceed to a vote on Wednesday, 21st > October. > > Best regards, > Shubham > > On Wed, 14 Oct 2026 at 03:07, Shubham Raj <[email protected]> wrote: > > > Thanks Jarek; I agree with all of this. I'll update the "Where to get > > tokens?" section of the AIP along these lines: > > > > - Treat token sources as a pool (Bedrock, Azure credits, > llm.apache.org, > > donated Google/Amazon credits) rather than a single choice. Models are > > reached only through an Airflow Connection, so switching is config, > not > > code. > > - Deterministic-first as an explicit design principle: rules start > > deterministic, and a model is consulted only for a bounded typed > > classification or to tailor a templated message. Token usage per > rule/Dag > > becomes a pilot metric next to time-to-first-review and findings > > accepted/corrected. > > - Edge executors in Azure added as a dogfooding stretch goal; initial > > pilot stays in one account to keep debugging simple. > > > > Specifically regarding llm.apache.org: for other community members, it's > > a LiteLLM gateway with an OpenAI-compatible endpoint, so common.ai > should > > work with just a connection change. It also supports script keys on > request > > ("Request a key" under My Keys), which the deployment would use rather > than > > anyone's personal PAT. Per-project budgets and usage reports are also > > traceable. One thing to note: the models on llm.apache.org today are > > three self-hosted open models: gemma4-26b, qwen3.8-27b, and qwen3-8b, I > > think the Magpie baseline, was produced with a frontier model via the > > Claude CLI. We need to assess or maybe divide the tasks based on which > > model could be good for them. Also Jarek, do you know whether a PMC > member > > can self-serve one for the project workflow? > > > > Also, For more reference, https://github.com/apache/tooling-llmao > > > > Thanks, > > Shubham > > > > On Fri, Oct 9, 2026 at 8:02 PM Jarek Potiuk <[email protected]> wrote: > > > >> Yeah. Subscription cannot be used as an API replacement "really" (at > least > >> long term and this is how subscriptions are being "targeted"). You > really > >> need to use them via the "Claude" or "Codex" CLI. While there are some > >> workarounds (like `claude -p`), this method might stop working (it was > >> scheduled for disabling - but Anthropic backed out temporarily). > >> > >> But yeah absolutely - we can use the Azure Credits, for that as well. > >> Ideally this will be **very little** credit use. We could even use Edge > >> executors for that (which is even more Dogfooding) - even if Airflow > will > >> be running in Amazon - the runners could be set-up in Azure for example. > >> Or > >> we could connect to the models Azure exposes. > >> > >> From my Magpie experiments about 80%-90% of such checks can be purely > >> deterministic, and the vast majority of the AI usage falls into two > cases: > >> > >> * generating responses (most of them can be templated - only small > amount > >> of tokens is spent to make messages "tailored" to the PR/ Security > issue) > >> * decision making on how to classify things (which can be way cheaper > now > >> with Jev - and similar solutions - Open AI just 3 days ago released > >> "Decisions API" - and pretty much everyone will have similar services > soon > >> [1] > >> > >> And i think we should treat it as a starting point - there are and will > be > >> many sources of tokens we can use: > >> > >> * We have stakeholders - Google and Amazon—who can likely donate some of > >> their credits > >> * ASF is finalizing llm service [2] for projects to use (open models on > >> rented GPUs, plus donated credits to ASF by big models). This is > >> specifically foreseen as a shared resource that ASF projects can use, > and > >> using it for "around CI" work is one of the stronger use-cases. > >> > >> I think it's important that we monitor and use the capability of our > >> subscriptions to iterate on ideas with AI - and turn them in **mostly** > >> deterministic workflows with as-small-as-possible usage of tokens and > >> monitor it. > >> > >> This is one of the things we have almost done in Magpie -> - we have a > >> SKILL that optimizes other skills by simplifying language, refactoring > and > >> extracting pieces of it - and turning everything that can be turned into > >> deterministict Python code. And I am plannning to use it on Airflow > >> AGENTS.md and SKILLs (see the other discussion). > >> > >> [1] > >> > >> > https://community.openai.com/t/decisions-api-is-now-available-in-public-beta/1403877 > >> [2] https://llm.apache.org - LLM service from ASF that is about to be > >> announced > >> [3] Skill optimization effort in Magpie > >> https://github.com/apache/magpie/issues/1342 > >> > >> J. > >> > >> > >> > >> On Fri, Oct 9, 2026 at 12:07 PM Shubham Raj <[email protected]> > >> wrote: > >> > >> > Thanks Ash for the quick response and feedback, > >> > > >> > Initially, yes (while in shadow mode), I plan to use copy/paste or GH > >> APIs. > >> > But the final goal would be like using a service account creds, which > >> will > >> > automatically post the reviews on the PR when approved, Today when we > >> use > >> > the harness to post the comments, it ends with "Drafted-by: <harness> > >> > (<model>); reviewed by <username> before posting", we will follow the > >> same > >> > thing, it's just the username would be the maintainer's name who > >> approved > >> > the HITL. > >> > > >> > Initially, I'm thinking of implementing this for committers+, and we > can > >> > decide on scaling later based on guardrails or rules implementation. > >> > > >> > Yes, those are good programs, but they are specific to users. I think > >> that > >> > would be really needed for contributors regarding their own PR > creation > >> and > >> > related work. I don't think the program provides API keys for > >> developers, > >> > which we need for the setup. Good to know we have Azure credits; this > >> adds > >> > to the viable options. > >> > > >> > Thanks, > >> > Shubham > >> > > >> > On Fri, Oct 9, 2026 at 4:18 PM Ash Berlin-Taylor <[email protected]> > >> wrote: > >> > > >> > > Love the idea of using Airflow to manage the Airflow project. > >> > > > >> > > Some clarifying questions: > >> > > > >> > > > human-approved comments, labels, draft conversion, closure where > >> > > permitted, and review dispositions; > >> > > > >> > > For the initial trail, and in the long term, who or what is making > >> > posting > >> > > comments? Initial version is “the human has to copy them etc” (and > we > >> > could > >> > > make that easier by providing `gh` commands etc) or an “AirflowBot” > GH > >> > > user/app etc? (I think I’m fine either way, I’m just unsure) > >> > > > >> > > Who would have access to see the results of these runs? Who would be > >> able > >> > > to be the Human in the HITL to approve things? Were you thinking > >> > everyone > >> > > in Triage, Committers and PMC groups? I think from reading the AIP > you > >> > have > >> > > it as Committers+? > >> > > > >> > > Another possible triage rule: history of opening PRs either too > >> quickly, > >> > > or to many and then never responding to comments? Might not be a > real > >> > case > >> > > so might not be worth it. > >> > > > >> > > > >> > > On question 4: Both Anthropic and OpenAI have “OSS” programs where > >> they > >> > > give (rolling, after re-applying) 6 month free things to > contributors > >> of > >> > > projects like Airflow. Downside to that is it would be tied to an > >> > > individual user I suspect. We also have a large number of Azure > >> credits > >> > > that have a (now 1year left) expiration on, so that might be a > useful > >> > thing > >> > > to do with them. I know Shahar is also looking at using credits for > CI > >> > > runners. > >> > > > >> > > > >> > > > On 9 Oct 2026, at 11:28, Shubham Raj <[email protected]> > wrote: > >> > > > > >> > > > Hello everyone, > >> > > > > >> > > > As a follow-up to this thread, I've written up the proposal, Here > >> is an > >> > > AIP > >> > > > < > >> > > > >> > > >> > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451979463/WIP+AIP-124+Airflow-Powered+Agentic+Workflows+for+Project+Operations > >> > > > > >> > > > , > >> > > > > >> > > > TL;DR: Move our PR triage and review assistance from a "push" > model > >> (a > >> > > > maintainer running local agent sessions) to a "pull" model. > Airflow > >> > > > continuously discovers work, handles the deterministic parts, and > >> > > surfaces > >> > > > only the decisions that need a human in a shared HITL inbox. This > >> > removes > >> > > > the single point of failure and lets more maintainers share the > >> load. > >> > > > > >> > > > *Key points:* > >> > > > > >> > > > - No new Airflow core subsystem. It reuses existing HITL, > >> common.ai, > >> > > > Assets, deferral, and the official Helm chart (requires Airflow > >> > 3.3+). > >> > > > - Runs in a project-controlled AWS account. Workflows, prompts, > >> > > > policies, and tests live in a new airflow-agentic-workflows > repo, > >> so > >> > > anyone > >> > > > can propose changes via PRs. > >> > > > - Bounded event loop, not an open-ended agent. Airflow owns the > >> state > >> > > > machine; the model returns typed results, and every write > >> requiring > >> > > > judgment is gated by a human. > >> > > > - Explicit non-goals: no automatic merging, no unattended > >> approvals > >> > or > >> > > > request-changes, no autonomous workflow approval/rerun. > >> > > > - Rollout starts in shadow mode, comparing results against > current > >> > > > maintainer practice before enabling any GitHub writes. > >> > > > > >> > > > The initial scope is PR triage and review. The security issue > >> workflow > >> > > > would follow as a separate track. > >> > > > > >> > > > *Open questions where I'd especially value input:* > >> > > > > >> > > > 1. The initial triage rules (R1-R6). More inputs would help > here; > >> are > >> > > > any rules missing or too aggressive? > >> > > > 2. Hosting and operational ownership of the AWS deployment. > >> > > > 3. The attribution model for reviews posted via a central GitHub > >> > > account > >> > > > on behalf of the approving maintainer. > >> > > > 4. Token/model source: Bedrock, llm.apache.org, or something > >> else? > >> > > > 5. What quality gate should we require before leaving shadow > mode? > >> > > > > >> > > > Please leave comments here or directly on the wiki page. Once the > >> > > > discussion settles, I'll update the AIP and we can move toward a > >> vote. > >> > > > > >> > > > Thanks, > >> > > > Shubham > >> > > > > >> > > > On Fri, Sep 11, 2026 at 3:02 PM Jarek Potiuk <[email protected]> > >> wrote: > >> > > > > >> > > >> Hi everyone, > >> > > >> > >> > > >> A quick follow-up: something I read today offers a slightly > >> different > >> > > >> perspective on the proposal to use Airflow, which might help > >> clarify > >> > my > >> > > aim: > >> > > >> > >> > > >> Read that*: AWS open-sources Pizza Bot: *email-style inbox for > >> > > background > >> > > >> AI agents - The New Stack: > >> > > >> https://thenewstack.io/aws-pizza-bot-agent-inbox/ > >> > > >> > >> > > >> I want our triage and security process to follow a similar model. > >> > Agents > >> > > >> would work mostly in the background, completing tasks > autonomously > >> and > >> > > >> surfacing to maintainers (in an inbox/to-do list fashion) only > when > >> > > human > >> > > >> decisions are required. We would have a shared `inbox` style > places > >> > > where > >> > > >> maintainers can log in and perform Human-in-the-Loop (HITL) > >> actions. > >> > > >> > >> > > >> Using Airflow to drive this allows us to tailor the experience > >> closely > >> > > to > >> > > >> our needs. It also showcases Airflow for this type of workload > and > >> > > >> Dogfooding it offers numerous advantages over using a third-party > >> > > solution > >> > > >> like Pizza Bot. Specifically, we could implement: > >> > > >> > >> > > >> - A shared inbox for the security team to review issues, make > >> triage > >> > > >> decisions, trigger fix PRs, and handle release announcements. > >> > > >> - Per-area inboxes for maintainers to make triage decisions on > new > >> > and > >> > > >> outstanding PRs > >> > > >> - Learning what Airflow still needs to support these kinds of > >> > workflows > >> > > >> > >> > > >> This proposal heavily optimizes for minimizing human interaction > by > >> > > >> automatically handling straightforward tasks and batching > >> decisions to > >> > > >> surface only high-priority items with full context. It also > >> eliminates > >> > > the > >> > > >> Single Point of Failure (SPOF)—currently myself—by enabling the > >> > broader > >> > > >> community to share the load (but only the "make decisions" part; > >> > Airflow > >> > > >> will handle the execution). > >> > > >> > >> > > >> Quick FAQ: > >> > > >> > >> > > >> *How much Magpie is involved?* > >> > > >> > >> > > >> On the runtime/execution side for Airflow: none. Magpie serves > >> purely > >> > > as a > >> > > >> starting point and interactive testing ground. It already > contains > >> > > useful > >> > > >> building blocks, mini-workflows, and integrations that we can > learn > >> > > from or > >> > > >> adapt. We can learn from or adapt these integrations and try out > >> new > >> > > things > >> > > >> However, embedding or weaving Magpie into Airflow DAGs is a > >> non-goal. > >> > We > >> > > >> might re-use some code/SKILLs initially, but making this a > >> long-term > >> > > >> dependency is not the goal. Possibly Magpie could eventually > >> implement > >> > > an > >> > > >> "export" feature to Airflow for any project wanting to automate > its > >> > > >> triaging/security issue handling with Airflow - following our > >> > blueprint. > >> > > >> However, this is not a goal - it's more of a side effect of using > >> > > similar > >> > > >> features while interactively testing things with Magpie and > >> wanting to > >> > > make > >> > > >> them into a DAG. > >> > > >> > >> > > >> *Should we analyze stats on what works first?* > >> > > >> > >> > > >> Yes. Absolutely. We have, and we will continue analyzing stats > >> while > >> > > >> incrementally moving some workflows to run via Airflow. I have > >> already > >> > > >> gathered and shared these stats from my manual runs with Magpie > >> and we > >> > > can > >> > > >> continue doing so. Look at my past messages where I shared stats, > >> drew > >> > > >> conclusions, and decided on next steps. Magpie provides the > >> tooling to > >> > > run > >> > > >> experiments and evaluate results quickly. Transitioning this to a > >> > > >> continuous, automated Airflow setup removes the single point of > >> > failure > >> > > >> (SPOF) and allows us to iterate and improve these workflows more > >> > > >> collaboratively. Since the decisions we make will have a > >> continuous, > >> > > >> immediate effect once implemented and deployed in our "Airflow > >> > driver," > >> > > we > >> > > >> will hopefully make them more consciously. > >> > > >> > >> > > >> Best regards, > >> > > >> Jarek > >> > > >> > >> > > >> > >> > > >> On Thu, Sep 10, 2026 at 7:02 PM Jarek Potiuk <[email protected]> > >> > wrote: > >> > > >> > >> > > >>> Hello everyone, > >> > > >>> > >> > > >>> Following up on today's Dev call, Shubham will share a proposal > >> > > document > >> > > >>> soon detailing our discussion from yesterday. However, I wanted > to > >> > > outline > >> > > >>> the idea here first to clear up a few misconceptions that arose > >> > during > >> > > >>> brief conversations on Slack and with PMC members. If agreed > upon, > >> > this > >> > > >>> might eventually become an AIP. > >> > > >>> > >> > > >>> > >> > > >>> *TL;DR: I propose experimenting with dogfooding Airflow > >> > deterministic + > >> > > >>> agentic workflows to handle our triaging and security issue > >> handling* > >> > > >>> > >> > > >>> *Current State* > >> > > >>> We currently use agentic automation via "Magpie" SKILLs ( > >> > > >>> https://magpie.apache.org/) for both areas, with varying > success: > >> > > >>> > >> > > >>> - Security issue handling: Working well, processing over 60 > >> issues > >> > > >>> monthly (a 10x increase over six months ago). > >> > > >>> - PR triaging: Partially effective (e.g., filtering bad PRs, > with > >> > > >>> potential to automate closing large PRs from inexperienced > >> > > contributors). > >> > > >>> > >> > > >>> *The Problem* > >> > > >>> > >> > > >>> Currently, these workflows require a "push" model where I (in > >> Claude > >> > > CLI) > >> > > >>> initiate actions, confirm proposals, and approve decisions. > >> > Regularly. > >> > > I > >> > > >>> initiate them regularly. Others might also initiate actions, but > >> very > >> > > few > >> > > >>> people have done so. This creates a Single Point of Failure > >> (SPOF). > >> > > IMHO - > >> > > >>> we need to shift to an "automated pull" model to eliminate this > >> SPOF > >> > > and > >> > > >>> allow more community members to act as Human-In-The-Loop (HITL) > >> > > reviewers > >> > > >>> without needing manual initiation. This would also allow more > >> people > >> > to > >> > > >>> contribute to and review those workflows. > >> > > >>> > >> > > >>> *The Proposal* > >> > > >>> > >> > > >>> Instead of relying on local CLI pushes, we can translate these > >> > > learnings > >> > > >>> into Airflow DAGs. Roughly 60%–80% of triage actions are > >> > deterministic > >> > > and > >> > > >>> require no human intervention. For the rest, the automated > >> workflow > >> > > would > >> > > >>> run in the background and trigger "pull" notifications (e.g., > >> Slack > >> > > alerts) > >> > > >>> when bulk human decisions are needed. > >> > > >>> > >> > > >>> *Key Benefits & Features:* > >> > > >>> > >> > > >>> - Shared UI: A cloud-hosted Airflow UI allows multiple > >> maintainers > >> > to > >> > > >>> review and approve decisions. > >> > > >>> - Community-Driven Workflows: Workflows can live in a new > Airflow > >> > repo > >> > > >>> (e.g., airflow-agentic-workflows), allowing anyone to modify > DAGs > >> via > >> > > pull > >> > > >>> requests. > >> > > >>> - No External Dependencies: While Magpie SKILLs can still be > used > >> > for > >> > > >>> local testing or syncs if preferred, using Magpie will not be > >> > required > >> > > for > >> > > >>> PMC/Dev team members. > >> > > >>> - They workflows **might** by somewhat synced with Magpie in > >> either > >> > > way > >> > > >>> (and likely I will start by transplanting parts of the process > >> from > >> > > Magpie > >> > > >>> to Airflow Dags (and implement some features for that in > Magpie) . > >> > But > >> > > >>> there will be exactly "0" Magpie components in Airflow Agentic > >> Dags. > >> > > >>> - Dogfooding: Managing Airflow with our latest Helm chart > >> provides > >> > > >>> direct operational feedback for the community—feedback that we > >> > > currently > >> > > >>> miss. > >> > > >>> - We could make this a "use case" demonstrating how Airflow > >> Agentic > >> > > >>> workflows helped solve our problem, which could confirm that > >> Airflow > >> > is > >> > > >>> ideally suited for these kinds of workflows. > >> > > >>> > >> > > >>> *Note on Workflow Architecture (Addressing Vikram's feedback):* > >> > > >>> > >> > > >>> Regarding REACT vs. REPL: Triage and security handling fit the > >> REACT > >> > > >>> pattern (Reason -> Act -> Iterate) due to the continuous stream > of > >> > > incoming > >> > > >>> PRs and issues. It's literally -> get issues, evaluate them, > make > >> > > actions > >> > > >>> (with or without human) and go back to the beginning. > >> > > >>> REPL relies more on immediate code-execution loops for task > >> > > >>> implementation, whereas our needs align with REACT's iterative > >> > > reasoning > >> > > >>> over time. > >> > > >>> > >> > > >>> Let's discuss further in this thread. Looking forward to your > >> > comments > >> > > >>> and feedback. > >> > > >>> > >> > > >>> Best regards, > >> > > >>> Jarek > >> > > >>> > >> > > >>> > >> > > > >> > > > >> > > > --------------------------------------------------------------------- > >> > > To unsubscribe, e-mail: [email protected] > >> > > For additional commands, e-mail: [email protected] > >> > > > >> > > > >> > > >> > > >
