I’m also super curious from seeing what we can learn and copy other projects, great work :)
Twitter: https://twitter.com/holdenkarau Fight Health Insurance: https://www.fighthealthinsurance.com/ <https://www.fighthealthinsurance.com/?q=hk_email> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau Pronouns: she/her On Fri, Feb 27, 2026 at 10:25 AM Damian Shaw <[email protected]> wrote: > That makes a lot of sense, I will be closely following along to see where > the Airflow project lands on this to see what best practices can be applied > elsewhere. > > Thanks, > Damian > > -----Original Message----- > From: Jarek Potiuk <[email protected]> > Sent: Friday, February 27, 2026 1:08 PM > To: [email protected] > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > automatically > > I don't think we are there yet and reviewing it isn't the main idea. The > AGENTS.md overhaul (plus inevitable follow-ups) focuses more on the > preceding step: doing everything possible to ensure all (or most) of the > coming PRs are better, because we instruct agents exactly what we want, > what to do, what not to do and how to do it well. > > This basically follows the old golden rule of health care: it's better to > prevent sickness than to deal with it when it happens. > > Ideally all those tools and quality checks should happen long before they > reach us maintainers. Dealing with review comments and similar issues just > adds noise, so we should do everything possible to avoid this > back-and-forth about basic quality checks. Instead, we should focus on what > truly matters: is that change needed at all, and does it fit the project's > roadmap? > > My current thinking is that the next step should also be to filter out and > immediately close PRs that don't follow the correct process. Include a > comment explaining why we do it and how they can fix it, perhaps by telling > them to use smart agents to follow the instructions. > > I believe right now we should focus on removing noise rather than adding > more. Therefore, IMHO, there is no point for us maintainers to even see, > consider, or spend any time on PRs that do not follow all the > specifications in the agentic instructions. We should remove such PRs out > of the picture completely - as efficiently as possible, with as good > explanation why we are doing it and with as helpful instructions for > whoever submits those as possible so that they can do it properly. > > I plan to focus on that in the coming days, following the "no assignment" > policy we discussed recently; this is the first step I intend to build on. > And having good agentic instructions is a very basic foundation for it, I > think > > > J. > > > > > On Fri, Feb 27, 2026 at 6:37 PM Damian Shaw <[email protected]> > wrote: > > > What I'm interested in is an outside contributors PR being reviewed by > > a bot, how this process is triggered (e.g. automatically, by the > > maintainer, by the reviewer), does the bot correctly pick up on AI > > slop issues, how does the contributor react to this, does this help or > > get in the way of maintainers moderating slop issues, etc.? > > > > Damian > > > > -----Original Message----- > > From: Jarek Potiuk <[email protected]> > > Sent: Friday, February 27, 2026 11:49 AM > > To: [email protected] > > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > > automatically > > > > Here it is Damian: https://github.com/apache/airflow/pull/62575 > > > > This is somewhat of an inception PR. It updates the agent instructions > > to create a PR as we want. I used Claude Code to generate the PR and > > then I asked Claude Code to create the PR—it followed the instructions > > it had just created. > > > > After this PR: > > > > * The PRs will be created via the web interface - which will give > > contributors a chance to review the PR before they create it (adds a > > bit of > > **good** friction there) > > * It will use our template and we will prefill the "Generated-by" > > information so the user does not have to do it manually. > > Which is actually pretty cool - because that will also likely increase > > the likelihood we will get the actual information about the agent used > > (we basically instruct the agent to tell who they are). > > > > J. > > > > > > > > On Thu, Feb 26, 2026 at 5:47 PM Damian Shaw > > <[email protected]> > > wrote: > > > > > Would love to see an example of it working with a real PR, i.e., > > > submitted by an outside contributor not a test PR, so see how the > > > interaction ends up going, will consider this for other open-source > > projects I maintain. > > > > > > Damian > > > > > > -----Original Message----- > > > From: Amogh Desai <[email protected]> > > > Sent: Thursday, February 26, 2026 3:46 AM > > > To: [email protected] > > > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > > > automatically > > > > > > This is amazing, Kaxil! Thank you! > > > > > > Thanks & Regards, > > > Amogh Desai > > > > > > > > > On Wed, Feb 25, 2026 at 9:38 PM Vincent Beck <[email protected]> > > wrote: > > > > > > > Very much needed! Thank you Kaxil! > > > > > > > > On 2026/02/25 08:16:47 Jarek Potiuk wrote: > > > > > Very cool :). > > > > > > > > > > On Wed, Feb 25, 2026 at 4:14 AM Kaxil Naik <[email protected]> > > > wrote: > > > > > > > > > > > Hey all, > > > > > > > > > > > > tl;dr: AI coding tools now get proper Airflow guardrails via > > > > AGENTS.md, and > > > > > > Copilot code review will automatically flag common mistakes in > > > > > > PRs > > > > > > -- including AI-slop patterns as long as we add Copilot as > > reviewer. > > > > > > > > > > > > We've been getting more AI-generated PRs lately, and reviewers > > > > > > keep catching the same things: N+1 queries, architecture > > > > > > boundary > > > > violations, > > > > > > assert in production code, fabricated diffs, etc. I wanted to > > > > > > automate that. > > > > > > > > > > > > I've been using AGENTS.md (CLAUDE.md , Cursor rules, skills > > > > > > and > > > > > > such) > > > > with > > > > > > Cursor, Claude Code, and Copilot and various other AI > > > > > > harnesses since > > > > last > > > > > > year and have iterated on it many times. Three PRs landed today: > > > > > > > > > > > > - https://github.com/apache/airflow/pull/62440 -- AGENTS.md > > > > overhaul. > > > > > > Replaced the sparse doc-index with concrete guidance: > > > > > > environment > > > > setup, > > > > > > architecture boundaries, coding/testing standards, commit > > > > conventions. > > > > > > Added nested AGENTS.md for Execution API (Cadwyn > > > > > > versioning) and providers. > > > > > > - https://github.com/apache/airflow/pull/62442 -- Copilot > > > > > > code > > > > review > > > > > > instructions. Added .github/instructions/ > > > > code-review.instructions.md > > > > > > scoped to review only. Covers architecture violations, N+1 > > > > > > queries, run_id > > > > > > scoping, unbounded caches, and AI-slop signals (fabricated > > > > > > diffs, narrating > > > > > > comments, over-engineered solutions, etc.). > > > > > > - https://github.com/apache/airflow/pull/62443 -- Symlink > > > > CLAUDE.md to > > > > > > AGENTS.md so Claude Code reads the same file. > > > > > > > > > > > > To validate the Copilot instructions, I opened a test PR ( > > > > > > https://github.com/apache/airflow/pull/62447) with 8 > > > > > > deliberately > > > > planted > > > > > > violations. Copilot caught all 8: > > > > > > > > > > > > 1. N+1 query (loop calling get_task_instances() per dag run) > > > > > > 2. time.time() instead of time.monotonic() > > > > > > 3. assert in production code > > > > > > 4. Unbounded @lru_cache (no maxsize) > > > > > > 5. run_id query without dag_id > > > > > > 6. Narrating comments > > > > > > 7. Import inside function body > > > > > > 8. No tests for new behavior > > > > > > > > > > > > This won't replace human review, but should save time on > > > > > > mechanical catches. If you have patterns you'd like Copilot to > > > > > > flag, PRs to .github/instructions/code-review.instructions.md > > > > > > are > > > welcome. > > > > > > > > > > > > Regards, > > > > > > Kaxil > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > > > -- > > > > - To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > ________________________________ > > > Strike Technologies, LLC (“Strike”) is part of the GTS family of > > > companies. Strike is a technology solutions provider, and is not a > > > broker or dealer and does not transact any securities related > > > business directly whatsoever. This communication is the property of > > > Strike and its affiliates, and does not constitute an offer to sell > > > or the solicitation of an offer to buy any security in any > > > jurisdiction. It is intended only for the person to whom it is > > > addressed and may contain information that is privileged, > > > confidential, or otherwise > > protected from disclosure. > > > Distribution or copying of this communication, or the information > > > contained herein, by anyone other than the intended recipient is > > > prohibited. If you have received this communication in error, please > > > immediately notify Strike at [email protected], and delete > > and destroy any copies hereof. > > > ________________________________ > > > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any > > > attachments are intended solely for the addressee. This transmission > > > is covered by the Electronic Communications Privacy Act, 18 U.S.C > > > ''2510-2521. The information contained in this transmission is > > > confidential in nature and protected from further use or disclosure > > > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be > > > subject to attorney-client or other legal privilege. Your use or > > > disclosure of this information for any purpose other than that > > > intended by its transmittal is strictly prohibited, and may subject > > > you to fines and/or penalties under federal and state law. If you > > > are not the intended recipient of this transmission, please DESTROY > > > ALL COPIES RECEIVED and confirm destruction to the sender via return > > transmittal. > > > > > ________________________________ > > Strike Technologies, LLC (“Strike”) is part of the GTS family of > > companies. Strike is a technology solutions provider, and is not a > > broker or dealer and does not transact any securities related business > > directly whatsoever. This communication is the property of Strike and > > its affiliates, and does not constitute an offer to sell or the > > solicitation of an offer to buy any security in any jurisdiction. It > > is intended only for the person to whom it is addressed and may > > contain information that is privileged, confidential, or otherwise > protected from disclosure. > > Distribution or copying of this communication, or the information > > contained herein, by anyone other than the intended recipient is > > prohibited. If you have received this communication in error, please > > immediately notify Strike at [email protected], and delete > and destroy any copies hereof. > > ________________________________ > > > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any > > attachments are intended solely for the addressee. This transmission > > is covered by the Electronic Communications Privacy Act, 18 U.S.C > > ''2510-2521. The information contained in this transmission is > > confidential in nature and protected from further use or disclosure > > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be > > subject to attorney-client or other legal privilege. Your use or > > disclosure of this information for any purpose other than that > > intended by its transmittal is strictly prohibited, and may subject > > you to fines and/or penalties under federal and state law. If you are > > not the intended recipient of this transmission, please DESTROY ALL > > COPIES RECEIVED and confirm destruction to the sender via return > transmittal. > > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS family of > companies. Strike is a technology solutions provider, and is not a broker > or dealer and does not transact any securities related business directly > whatsoever. This communication is the property of Strike and its > affiliates, and does not constitute an offer to sell or the solicitation of > an offer to buy any security in any jurisdiction. It is intended only for > the person to whom it is addressed and may contain information that is > privileged, confidential, or otherwise protected from disclosure. > Distribution or copying of this communication, or the information contained > herein, by anyone other than the intended recipient is prohibited. If you > have received this communication in error, please immediately notify Strike > at [email protected], and delete and destroy any copies hereof. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments > are intended solely for the addressee. This transmission is covered by the > Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The > information contained in this transmission is confidential in nature and > protected from further use or disclosure under U.S. Pub. L. 106-102, 113 > U.S. Stat. 1338 (1999), and may be subject to attorney-client or other > legal privilege. Your use or disclosure of this information for any purpose > other than that intended by its transmittal is strictly prohibited, and may > subject you to fines and/or penalties under federal and state law. If you > are not the intended recipient of this transmission, please DESTROY ALL > COPIES RECEIVED and confirm destruction to the sender via return > transmittal. >
