Maybe good to make clear that this is an auxiliary process and committer+ must review on their own? The scenario I can think of is the author sees the AI review and diligently addresses them, and the committer chimes in later and says it's incorrect. The review content from automatic AI review is no owner in terms of responsibility.
I understand many of us are leveraging AI assisted review (and I'm almost pushed to this way due to amount of PRs to review) but I expect the final content (whether it is a code, review, written doc, whatever) would be owned by the one who posts it, and have a responsibility with it - it's not that LLM would have responsibility with it. This is great for double checking but I hope this isn't going to replace the primary review. On Fri, Oct 2, 2026 at 6:59 AM Hyukjin Kwon <[email protected]> wrote: > Hi all, > > I've been running a trial of automated AI code review on new apache/spark > PRs, and I'd like to share how it works and get your feedback. > > For now the reviews are posted from my own GitHub account (@HyukjinKwon) > while I check that it behaves well. Once it's proven, I plan to move it to > the https://github.com/spark-test account, so the reviews are clearly > separate from my personal ones. I plan to run this for the next week. > > *What it does* > > Each review is a full, multi-pass review of the PR diff, run by Claude > (via Claude Code). Findings are checked against the code before they're > posted. They're posted as a normal GitHub review: a short summary plus > inline comments, each labeled with a severity, e.g. "Non-blocking (P2)" > or "Nit (P3)". Example: > https://github.com/apache/spark/pull/59184#pullrequestreview-5379759787 > > *No LGTM, no approvals* > > - It never approves a PR and never posts "LGTM". Every review is a plain > "Comment" review, so it doesn't count as an approval or a change > request, and it has no effect on merging. > - If a review finds nothing new, it posts nothing at all: no "no > findings" or "all comments addressed" message. > - It doesn't push commits or edit anyone's PR. > - Human review and committer judgement are unchanged. Please treat its > comments like any other reviewer's: feel free to push back or ignore. > > *When a review is triggered* > > - A new PR is reviewed shortly after it's opened. Draft PRs are reviewed > once they're marked ready for review. > - After a review posts findings, the bot waits. It reviews again only > when the author pushes new commits. > - It also waits while a PR has open review comments from anyone that are > newer than the latest commit, so it doesn't pile on while the author is > still responding to reviewers. > - Replying in a thread without pushing doesn't trigger a re-review. > > *When it stops* > > - When a review finds nothing new. The PR gets a fresh review if the > author pushes again later. > - When the PR is merged or closed. > > *Early numbers* > > Over the first ~12 hours (Oct 1), it covered 10 PRs, including 5 > re-reviews after the author pushed. It posted 7 reviews with 14 findings > in total, mostly P3 nits. Four PRs came back clean and got no comment. > Several authors pushed fixes, and the follow-up reviews found nothing > new. > > *Feedback* > > Please let me know if the comments are useful, noisy, or wrong, either > here or on the PR. If you'd rather it not review your PRs, tell me and > I'll exclude them. I'm also happy to adjust when it triggers based on > what the community prefers. > > Thanks, > Hyukjin >
