Just to update everyone: I've auto-triaged a bunch of PRs—the tool works very well IMHO, but we will know after the authors see them and review
Some stats (I will gather more in the next days as I am adding timing and further improvements): * I triaged about 100 PRs in under an hour of elapsed time (I also corrected, improved and noted some fixes, so it will be faster) * I converted 30 of those into Drafts and closed a few * I have not marked any as ready to review yet, but I will do that tomorrow * The LLM (Claude) assessment is quite fast - faster than I thought. Parallelizing it also helps. LLM assessment takes between 20 s and 2 minutes (elapsed), but usually, only a few pull requests (15% or less) are LLM assessed in a batch, so this is not a bottleneck. I will also modify the tool to start reviewing deterministic things before LLMs complete - which should speed up the whole process even more * The LLM assessments are pretty good - but a few were significantly wrong and I would not post them. It's good we have Human-In-The-Loop and in the driver's seat. Overall - I think the tool is doing very well what I wanted. But let's see the improvements over the next few days, observe how authors react, and determine if it can actually help maintainers I added a few PRs as improvements; looking forward to reviews, : * https://github.com/apache/airflow/pull/63318 * https://github.com/apache/airflow/pull/63317 * https://github.com/apache/airflow/pull/63315 * https://github.com/apache/airflow/pull/63319 * https://github.com/apache/airflow/pull/63320 J. On Tue, Mar 10, 2026 at 10:18 PM Jarek Potiuk <[email protected]> wrote: > Lazy consensus reached. I will try it out tonight. I added more signals > (unresolved review comments) and filtering options ( > https://github.com/apache/airflow/pull/63300) that will be useful during > this phase. > > On Fri, Mar 6, 2026 at 9:08 PM Jarek Potiuk <[email protected]> wrote: > >> Hello here, >> >> I am asking a lazy consensus on the approach proposed in >> https://lists.apache.org/thread/ly6lrm2gc4p7p54vomr8621nmb1pvlsk >> regarding our approach to triaging PRs. >> >> The lazy consensus will last till Tuesday 10 pm CEST ( >> https://www.timeanddate.com/countdown/generic?iso=20260310T22&p0=262&font=cursive >> ) >> >> Summary of the proposal >> >> This is the proposed update to the PR contributing guidelines: >> >> > Start with **Draft**: Until you are sure that your PR passes all the >> quality checks and tests, keep it in **Draft** status. This will signal to >> maintainers that the PR is not yet ready for review and it will prevent >> maintainers from accidentally merging it before it's ready. Once you are >> sure that your PR is ready for review, you can mark it as "Ready for >> review" in the GitHub UI. Our regular check will convert all PRs from >> non-collaborators that do not pass our quality gates to Draft status, so if >> you see that your PR is in Draft status and you haven't set it to Draft. >> Check the comments to see what needs to be fixed. >> >> That's a "broad" description of the process; details will be worked out >> while testing the solution. >> >> The PR: https://github.com/apache/airflow/pull/62682 >> >> My testing approach is to start with individual areas, update and perfect >> the tool, gradually increase the reach of it and engage others - then we >> might think about more regular process involving more maintainers. >> >> J. >> >
