I like Rok's proposed approach.
Pedro and I discussed this topic offline. While we can have private,
internal conversations with authors regarding contribution quality within
the company, open source contributions lack direct messaging capabilities.
As a result, this interaction - unfortunately - needs to take place
publicly.
Dmitry Chirkov,
Dremio
On Mon, Sep 21, 2026 at 9:40 AM Rok Mihevc <[email protected]> wrote:
> Agreed with Pedro's proposal as well!
>
> I'd suggest another item here: let's introduce a GitHub label:
> "needs-author-engagement". Reviewer could use this label to flag PRs that
> require substantial responses from their authors before a productive review
> can continue. This label should not focus on AI use, but rather on the
> author's observable understanding of the PR's subject matter and their
> engagement with reviewer feedback. Such label would also help other
> reviewers avoid duplicate effort and serve as feedback to the author of the
> PR.
>
> Rok
>
> On Mon, Sep 21, 2026 at 6:24 PM Nic Crane <[email protected]> wrote:
>
> > Hi Pedro,
> >
> > Thanks for raising this issue, and I think it's very timely in that many
> of
> > us are experiencing similar issues. Your proposal sounds good to me.
> >
> > Nic
> >
> > On Fri, 18 Sept 2026 at 10:33, Pedro Matias <[email protected]>
> > wrote:
> >
> > > Hello all,
> > >
> > > I'd like to revive this thread. My goal is simple: I want to discuss
> > > establishing guidelines for reviewers, especially those of us who are
> not
> > > Arrow committers/PMCs, on how to handle low-quality PRs with weak
> > > engagement. The community added AI guidelines for contributors, but not
> > for
> > > reviewers: https://arrow.apache.org/docs/developers/reviewing.html
> > >
> > > Lately I've been reviewing a PR that I think fits this description.
> I've
> > > pointed the person to the AI guidelines at
> > >
> https://arrow.apache.org/docs/developers/overview.html#ai-generated-code
> > ,
> > > but the guidelines were ignored. I can continue steering the code with
> > > reviews, but I am afraid this might encourage others to repeat this
> > pattern
> > > of weak engagement.
> > >
> > > I repeat R. Tyler Croy's ask: "don't rely on everybody following the
> > rules,
> > > and come up with an agreed upon way to handle those that don't."
> > >
> > > I looked into other projects to see if any of them have something
> > similar.
> > > LLVM has guidance [0] on both how to warn the contributor and when to
> > > escalate to someone with permission to lock the conversation, as well
> as
> > a
> > > label that can be added to a low effort PR.
> > >
> > > I propose we add a section titled "Handling violations of the AI
> > > contribution guidelines" to the reviewer guidelines. I wrote a small
> > draft
> > > [1] of what it could look like. I'm happy to iterate on it if people
> > think
> > > this is something worth adding.
> > >
> > > I'm particularly curious about what committers/PMC members think should
> > be
> > > the way for read-only reviewers to escalate to maintainers. The current
> > > proposal suggests pinging someone, which might be too noisy.
> > >
> > > 0- https://llvm.org/docs/AIToolPolicy.html#handling-violations
> > >
> > > 1- When a reviewer finds that a contribution does not seem to conform
> to
> > > the guidelines for AI usage, they should respond with the following
> > > message:
> > > "
> > > This PR does not seem to meet the standards for AI generated
> > contributions.
> > > Please read the guidelines at
> > >
> https://arrow.apache.org/docs/developers/overview.html#ai-generated-code
> > > and ensure you modify your PR to conform to the rules.
> > > "
> > > If the contributor fails to adapt their work and/or engagement level to
> > > meet the guidelines' standards, maintainers may close the PR. Reviewers
> > > without permission to close the PR should escalate by pinging a
> > maintainer
> > > via comment indicating that they do not believe the change meets the
> > > standards.
> > >
> > > Best regards,
> > > Pedro Matias
> > >
> > >
> > >
> > >
> > > On Fri, Feb 13, 2026 at 3:53 PM Nic Crane <[email protected]> wrote:
> > >
> > > > On a similar note, after conversations with folks around what appear
> to
> > > be
> > > > AI-generated mailing list responses, I've also opened a PR suggesting
> > > > people disclose any AI-generated questions they post to mailing list
> > > > discussions; feel free to add any comments there (if you're a human!
> > ;) )
> > > >
> > > > https://github.com/apache/arrow/pull/49277/changes
> > > >
> > > >
> > > > On Thu, 22 Jan 2026 at 20:42, Nic Crane <[email protected]> wrote:
> > > >
> > > > > PR here for anyone interested:
> > > > https://github.com/apache/arrow/pull/48952
> > > > >
> > > > > On Thu, 22 Jan 2026 at 09:56, Nic Crane <[email protected]>
> wrote:
> > > > >
> > > > >> Thanks Andrew, I really like how you spell out the reasoning
> around
> > > it,
> > > > I
> > > > >> will see how we can incorporate some of those ideas
> > > > >>
> > > > >> On Thu, 22 Jan 2026 at 09:23, Andrew Lamb <[email protected]>
> > > wrote:
> > > > >>
> > > > >>> > We have had repeated attempts at contributions by some folks
> who
> > > > simply
> > > > >>> do not understand their generated code and when asked for
> > > > clarification,
> > > > >>> have the LLM generate more incorrect commentary. It's very
> > > > >>> Dunning-Krueger
> > > > >>> and leads to lots of frustration all around.
> > > > >>>
> > > > >>> We saw this too in DataFusion and I was pleased with what we came
> > up
> > > > with
> > > > >>> for rationale about why it is not helpful[1]. Basically the
> > reviewers
> > > > are
> > > > >>> more efficient using the LLM tools directly and the contributor
> > isn't
> > > > >>> learning anything either.
> > > > >>>
> > > > >>> Andrew
> > > > >>>
> > > > >>>
> > > > >>> [1]:
> > > > >>>
> > > > >>>
> > > >
> > >
> >
> https://datafusion.apache.org/contributor-guide/index.html#why-fully-ai-generated-prs-without-understanding-are-not-helpful
> > > > >>>
> > > > >>> On Mon, Jan 19, 2026 at 12:48 PM R Tyler Croy <
> [email protected]>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > (replies inline)
> > > > >>> >
> > > > >>> > On Sunday, January 18th, 2026 at 7:43 PM, Gang Wu <
> > > [email protected]>
> > > > >>> > wrote:
> > > > >>> >
> > > > >>> > > - Summitters should review all lines of generated code before
> > > > >>> creating
> > > > >>> > the
> > > > >>> > > PR to
> > > > >>> > > understand every piece of detail just like they are written
> by
> > > the
> > > > >>> > > submitters
> > > > >>> > > themselves.
> > > > >>> > > - AI tools are notorious for generating overly verbose
> > comments,
> > > > >>> > unnecessary
> > > > >>> > > test cases, fixing test failures using wrong approaches, etc.
> > > Make
> > > > >>> sure
> > > > >>> > > these
> > > > >>> > > are checked and fixed.
> > > > >>> > > - Reviewers are humans, so please try to break down large PRs
> > > into
> > > > >>> > smaller
> > > > >>> > > ones to make reviewers' life easier to get PRs promptly
> > reviewed.
> > > > >>> >
> > > > >>> >
> > > > >>> > Like others I think Nic's draft is a good one, I would like to
> > > offer
> > > > >>> some
> > > > >>> > thoughts as a maintainer (delta-rs) which has received
> increased
> > > > >>> > AI-assisted pull requests over the past six months.
> > > > >>> >
> > > > >>> >
> > > > >>> > The "PR may be closed without further review" statement I would
> > > > >>> strongly
> > > > >>> > encourage moving to the very beginning of the policy. I would
> > also
> > > > >>> > encourage labels being used like "ai-assisted" to signal to
> other
> > > > >>> > contributors who may or may not wish to engage in reviewing
> > > potential
> > > > >>> slop.
> > > > >>> >
> > > > >>> > We have had repeated attempts at contributions by some folks
> who
> > > > >>> simply do
> > > > >>> > not understand their generated code and when asked for
> > > clarification,
> > > > >>> have
> > > > >>> > the LLM generate more incorrect commentary. It's very
> > > > Dunning-Krueger
> > > > >>> and
> > > > >>> > leads to lots of frustration all around.
> > > > >>> >
> > > > >>> > Like most policies it's important to speak to those that are
> > acting
> > > > in
> > > > >>> > good faith but don't rely on everybody following the rules, and
> > > come
> > > > up
> > > > >>> > with an agreed upon way to handle those that don't.
> > > > >>> >
> > > > >>> >
> > > > >>> > Either way I think it's good to ship! :)
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > Cheers
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>
