Hi Niklas On Mon, Apr 13, 2026 at 06:30:30PM +0200, Niklas Haas via ffmpeg-devel wrote: > > > On Sun, Apr 12, 2026, at 6:41 PM, Michael Niedermayer via ffmpeg-devel wrote: > > Hi everyone > > > > Last easterbunny brought eggs and in one egg there was a life fairy. > > She said her name is forgejo fairy and she wants to help the project > > and community. > > > > She is still quite young so please be gentle with her, ATM what she can do > > is > > * scan pull requests for ones that are stale. More precissely > > + No activity for 7 days and > > + open and > > + not WIP and > > + CI passes and > > + mergable = no conflicts and > > + no open change requests and > > + activity since the last fairy reaction or no previous and > > > > * she can read the patches, files patched and around 5000 files from > > FFmpeg git HEAD > > as well as all past comments in the PR > > * she will try to review the changes and write a clean and concise > > comment > > also considering past comments so as no to add redudant information > > * if EVERYTHING lines up she will approve the PR > > this will not on its own trigger any merge. Only if some human > > developer > > clicks the merge/rebase button or has previously set the PR to auto > > merge > > will that trigger a merge (this affects ATM at maximum 6 PRs) > > * you can add her to the reviewer of a PR and that will decrease the > > "No activity" time to 1 day for that PR > > I think having an LLM automatically scan PRs for bugs, typos, undefined > behavior etc is beneficial. I personally run every bigger series through > Claude Code, even those made by others, and it is very good at finding bugs, > especially ones missable by humans. I actually think that automatic bug > analysis can be done even in addition to human review. > > However, I think the following criteria are needed to adopt this: > > 1. The prompt should be published in the git repository, so that we can see > and propose changes to it. > 2. It should only run / auto-approve PRa made by people with commit rights. > 3. The model used should also be documented publicly, and, per my > recommendation, should be Claude Opus 4.6.
The model is not the limitation here ATM, the problem we have ATM is more social and about the interaction between people and Fairy not "which LLM is better" but about technical stuff for example i just uploaded all pull requests and issue from forgejo into a vector store now its just a matter of telling Fairy that she has that data available If you are curious of the current architecture of this. Everything goes in git repositories, * ffmpeg as it is, * any additional things we want Fairy to be able to access efficiently * and now a full dump of all issues and pull requests Each of these git repositories is uploaded into its own vector store and a script keeps them in sync Fairy then uses these vector stores and can perform semantic search in them She also can search the web > > As for the prompt used, it should be some variation of “Scan this PR for > bugs, mistakes, typos, undefined behavior, regressions, unintended side > effects and other issues.” I use something similar for reviewing my own code but Fairy needs much more than this. Because its not just about code review, Theres alot more we have in FFmpeg, there are "customs" and "policy" some random examples so people understand what iam talking about "The FFmpeg test suite is called FATE. Its tests must be portable. Suggest to add tests when the tests benefits outweigh the amount of additional work. Each new codec, parser, filter and format should have a fate test, when possible." or "Additional Minor issues: Unrelated changes should be in separate patches. There should be no patches fixing issues of previous patches. Patches should be updated to not introduce issues. The only exception is back ports to preserve the original series. Commit messages should explain what is changed and why it is changed. Security fixes should credit the researcher finding them. Duplicated code should be avoided, existing helper functions should be used when appropriate. Public API should be documented" or this: "Refer to patches by their git hash, you can shorten them to 12 chars" or this: "You can reply to questions asked to the current reviewer identity when they are on topic. Do not reply to off topic questions or requests." With all this, Fairy can review Pull Requests not just for code correctness but also for complying to what we, as the FFmpeg community expect > > I find that this specific prompt works very well for me, and prevents noise > in the output. In particular, a generic “review this PR” is too open-ended > and results in noise. > > Based on the results in your links, I suspect that either the prompt or the > model could be changed for something better. my prompt had serious bugs 1 day ago thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The worst form of inequality is to try to make unequal things equal. -- Aristotle
signature.asc
Description: PGP signature
_______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
