I don't think we are there yet and reviewing it isn't the main idea. The AGENTS.md overhaul (plus inevitable follow-ups) focuses more on the preceding step: doing everything possible to ensure all (or most) of the coming PRs are better, because we instruct agents exactly what we want, what to do, what not to do and how to do it well.
This basically follows the old golden rule of health care: it's better to prevent sickness than to deal with it when it happens. Ideally all those tools and quality checks should happen long before they reach us maintainers. Dealing with review comments and similar issues just adds noise, so we should do everything possible to avoid this back-and-forth about basic quality checks. Instead, we should focus on what truly matters: is that change needed at all, and does it fit the project's roadmap? My current thinking is that the next step should also be to filter out and immediately close PRs that don't follow the correct process. Include a comment explaining why we do it and how they can fix it, perhaps by telling them to use smart agents to follow the instructions. I believe right now we should focus on removing noise rather than adding more. Therefore, IMHO, there is no point for us maintainers to even see, consider, or spend any time on PRs that do not follow all the specifications in the agentic instructions. We should remove such PRs out of the picture completely - as efficiently as possible, with as good explanation why we are doing it and with as helpful instructions for whoever submits those as possible so that they can do it properly. I plan to focus on that in the coming days, following the "no assignment" policy we discussed recently; this is the first step I intend to build on. And having good agentic instructions is a very basic foundation for it, I think J. On Fri, Feb 27, 2026 at 6:37 PM Damian Shaw <[email protected]> wrote: > What I'm interested in is an outside contributors PR being reviewed by a > bot, how this process is triggered (e.g. automatically, by the maintainer, > by the reviewer), does the bot correctly pick up on AI slop issues, how > does the contributor react to this, does this help or get in the way of > maintainers moderating slop issues, etc.? > > Damian > > -----Original Message----- > From: Jarek Potiuk <[email protected]> > Sent: Friday, February 27, 2026 11:49 AM > To: [email protected] > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > automatically > > Here it is Damian: https://github.com/apache/airflow/pull/62575 > > This is somewhat of an inception PR. It updates the agent instructions to > create a PR as we want. I used Claude Code to generate the PR and then I > asked Claude Code to create the PR—it followed the instructions it had just > created. > > After this PR: > > * The PRs will be created via the web interface - which will give > contributors a chance to review the PR before they create it (adds a bit of > **good** friction there) > * It will use our template and we will prefill the "Generated-by" > information so the user does not have to do it manually. > Which is actually pretty cool - because that will also likely increase the > likelihood we will get the actual information about the agent used (we > basically instruct the agent to tell who they are). > > J. > > > > On Thu, Feb 26, 2026 at 5:47 PM Damian Shaw <[email protected]> > wrote: > > > Would love to see an example of it working with a real PR, i.e., > > submitted by an outside contributor not a test PR, so see how the > > interaction ends up going, will consider this for other open-source > projects I maintain. > > > > Damian > > > > -----Original Message----- > > From: Amogh Desai <[email protected]> > > Sent: Thursday, February 26, 2026 3:46 AM > > To: [email protected] > > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > > automatically > > > > This is amazing, Kaxil! Thank you! > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Wed, Feb 25, 2026 at 9:38 PM Vincent Beck <[email protected]> > wrote: > > > > > Very much needed! Thank you Kaxil! > > > > > > On 2026/02/25 08:16:47 Jarek Potiuk wrote: > > > > Very cool :). > > > > > > > > On Wed, Feb 25, 2026 at 4:14 AM Kaxil Naik <[email protected]> > > wrote: > > > > > > > > > Hey all, > > > > > > > > > > tl;dr: AI coding tools now get proper Airflow guardrails via > > > AGENTS.md, and > > > > > Copilot code review will automatically flag common mistakes in > > > > > PRs > > > > > -- including AI-slop patterns as long as we add Copilot as > reviewer. > > > > > > > > > > We've been getting more AI-generated PRs lately, and reviewers > > > > > keep catching the same things: N+1 queries, architecture > > > > > boundary > > > violations, > > > > > assert in production code, fabricated diffs, etc. I wanted to > > > > > automate that. > > > > > > > > > > I've been using AGENTS.md (CLAUDE.md , Cursor rules, skills and > > > > > such) > > > with > > > > > Cursor, Claude Code, and Copilot and various other AI harnesses > > > > > since > > > last > > > > > year and have iterated on it many times. Three PRs landed today: > > > > > > > > > > - https://github.com/apache/airflow/pull/62440 -- AGENTS.md > > > overhaul. > > > > > Replaced the sparse doc-index with concrete guidance: > > > > > environment > > > setup, > > > > > architecture boundaries, coding/testing standards, commit > > > conventions. > > > > > Added nested AGENTS.md for Execution API (Cadwyn versioning) > > > > > and providers. > > > > > - https://github.com/apache/airflow/pull/62442 -- Copilot > > > > > code > > > review > > > > > instructions. Added .github/instructions/ > > > code-review.instructions.md > > > > > scoped to review only. Covers architecture violations, N+1 > > > > > queries, run_id > > > > > scoping, unbounded caches, and AI-slop signals (fabricated > > > > > diffs, narrating > > > > > comments, over-engineered solutions, etc.). > > > > > - https://github.com/apache/airflow/pull/62443 -- Symlink > > > CLAUDE.md to > > > > > AGENTS.md so Claude Code reads the same file. > > > > > > > > > > To validate the Copilot instructions, I opened a test PR ( > > > > > https://github.com/apache/airflow/pull/62447) with 8 > > > > > deliberately > > > planted > > > > > violations. Copilot caught all 8: > > > > > > > > > > 1. N+1 query (loop calling get_task_instances() per dag run) > > > > > 2. time.time() instead of time.monotonic() > > > > > 3. assert in production code > > > > > 4. Unbounded @lru_cache (no maxsize) > > > > > 5. run_id query without dag_id > > > > > 6. Narrating comments > > > > > 7. Import inside function body > > > > > 8. No tests for new behavior > > > > > > > > > > This won't replace human review, but should save time on > > > > > mechanical catches. If you have patterns you'd like Copilot to > > > > > flag, PRs to .github/instructions/code-review.instructions.md > > > > > are > > welcome. > > > > > > > > > > Regards, > > > > > Kaxil > > > > > > > > > > > > > > > -------------------------------------------------------------------- > > > - To unsubscribe, e-mail: [email protected] > > > For additional commands, e-mail: [email protected] > > > > > > > > ________________________________ > > Strike Technologies, LLC (“Strike”) is part of the GTS family of > > companies. Strike is a technology solutions provider, and is not a > > broker or dealer and does not transact any securities related business > > directly whatsoever. This communication is the property of Strike and > > its affiliates, and does not constitute an offer to sell or the > > solicitation of an offer to buy any security in any jurisdiction. It > > is intended only for the person to whom it is addressed and may > > contain information that is privileged, confidential, or otherwise > protected from disclosure. > > Distribution or copying of this communication, or the information > > contained herein, by anyone other than the intended recipient is > > prohibited. If you have received this communication in error, please > > immediately notify Strike at [email protected], and delete > and destroy any copies hereof. > > ________________________________ > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any > > attachments are intended solely for the addressee. This transmission > > is covered by the Electronic Communications Privacy Act, 18 U.S.C > > ''2510-2521. The information contained in this transmission is > > confidential in nature and protected from further use or disclosure > > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be > > subject to attorney-client or other legal privilege. Your use or > > disclosure of this information for any purpose other than that > > intended by its transmittal is strictly prohibited, and may subject > > you to fines and/or penalties under federal and state law. If you are > > not the intended recipient of this transmission, please DESTROY ALL > > COPIES RECEIVED and confirm destruction to the sender via return > transmittal. > > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS family of > companies. Strike is a technology solutions provider, and is not a broker > or dealer and does not transact any securities related business directly > whatsoever. This communication is the property of Strike and its > affiliates, and does not constitute an offer to sell or the solicitation of > an offer to buy any security in any jurisdiction. It is intended only for > the person to whom it is addressed and may contain information that is > privileged, confidential, or otherwise protected from disclosure. > Distribution or copying of this communication, or the information contained > herein, by anyone other than the intended recipient is prohibited. If you > have received this communication in error, please immediately notify Strike > at [email protected], and delete and destroy any copies hereof. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > are intended solely for the addressee. This transmission is covered by the > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > information contained in this transmission is confidential in nature and > protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > legal privilege. Your use or disclosure of this information for any purpose > other than that intended by its transmittal is strictly prohibited, and may > subject you to fines and/or penalties under federal and state law. If you > are not the intended recipient of this transmission, please DESTROY ALL > COPIES RECEIVED and confirm destruction to the sender via return > transmittal. >
