On Wed, Nov 4, 2026 at 1:10 PM Rémy Maucherat via dev <[email protected]> wrote:
> n Tue, Nov 3, 2026 at 8:21 PM Mark Thomas via dev <[email protected]> > wrote: > > > > On 03/11/2026 12:18, Rémy Maucherat via dev wrote: > > > On Tue, Nov 3, 2026 at 11:37 AM Mark Thomas via dev > > > <[email protected]> wrote: > > > > <snip/> > > > > >> My reviews have speed up considerably this morning. There is a clear > > >> correlation between code complexity and speed of review / further > fixes > > >> being required. That suggests to me that we need to be using more > > >> powerful models for the more complex code. Maybe look at using the > free > > >> AI credits for those models that have been offered to ASF committers? > > > > > > Obviously if the issue is simple, it will work out better ! (also > > > applies to me if I try to fix an issue !) > > > > Same here :). > > > > > To be honest, I think the models I have on hand are more than good > > > enough and are able to do complex tasks. > > > However, there are some issues and I need to use them better in some > > > cases. The main limitation I have to work around when I'm using a > > > "small" "fast" model is that the context is limited to 256K, and for > > > some models the quality degrades a lot after compaction. Also my > > > review step was very suboptimal. And of course my manual review step > > > after the changes is far from perfect (= it's the worst). > > > > Yes. I am finding the models are picking up a lot more things than I > would. > > > > > Although the HTTP/3 experiment was written with the small models, the > > > review steps were done by "big" models (with 1M context; technically > > > only 512K was ever reached). The main reason is that the components > > > are large and would lead to non stop compaction and useless results on > > > small models. > > > > > > I expect the small open models will show another capability leap > > > simply by increasing the context. I know I can use some stuff like > > > rope with the current models, but it seems accuracy goes down. > > > > Can you run your smaller models with a larger context? > > Supposedly I can use > https://docs.vllm.ai/en/latest/features/context_extension/ to extend > the context up to 1M (512K in practice). But this does reduce accuracy > and it's not working for me. > I expect new models to switch to at least 512K, which would basically > solve the issue for most workloads. I have a model to test in that > category. > > Note: llm.apache.org uses Qwen 3.8 27B (FP8) as its top model, but > only with a 128K context, to give you an idea. > > Rémy > > > Mark > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > I am considering implementing an automated AI review mechanism through CI. According to the FAQ <https://llm.apache.org/help/faq>, we can also request a dedicated API key for scripts. Two independent components: 1) A reusable Python-based review engine (standalone CLI, can be run locally or through any CI system): - Track commits and maintain checkpoints. - Use Tree-sitter <https://tree-sitter.github.io/tree-sitter/>(MIT license) to identify affected Java methods and retrieve relevant code context, including dependencies and tests. - Group related changes into review units to minimize redundant model requests. - Interact with LLM ( Qwen 3.8 27B) and validate potential findings against the latest code to reduce false positives. - Report findings through a dedicated GitHub issue. 2) A Github actions workflow Responsible for scheduling and invoking the engine, perhaps every couple of hours? I would appreciate your input on the above. Dimitris
