n Tue, Nov 3, 2026 at 8:21 PM Mark Thomas via dev <[email protected]> wrote: > > On 03/11/2026 12:18, Rémy Maucherat via dev wrote: > > On Tue, Nov 3, 2026 at 11:37 AM Mark Thomas via dev > > <[email protected]> wrote: > > <snip/> > > >> My reviews have speed up considerably this morning. There is a clear > >> correlation between code complexity and speed of review / further fixes > >> being required. That suggests to me that we need to be using more > >> powerful models for the more complex code. Maybe look at using the free > >> AI credits for those models that have been offered to ASF committers? > > > > Obviously if the issue is simple, it will work out better ! (also > > applies to me if I try to fix an issue !) > > Same here :). > > > To be honest, I think the models I have on hand are more than good > > enough and are able to do complex tasks. > > However, there are some issues and I need to use them better in some > > cases. The main limitation I have to work around when I'm using a > > "small" "fast" model is that the context is limited to 256K, and for > > some models the quality degrades a lot after compaction. Also my > > review step was very suboptimal. And of course my manual review step > > after the changes is far from perfect (= it's the worst). > > Yes. I am finding the models are picking up a lot more things than I would. > > > Although the HTTP/3 experiment was written with the small models, the > > review steps were done by "big" models (with 1M context; technically > > only 512K was ever reached). The main reason is that the components > > are large and would lead to non stop compaction and useless results on > > small models. > > > > I expect the small open models will show another capability leap > > simply by increasing the context. I know I can use some stuff like > > rope with the current models, but it seems accuracy goes down. > > Can you run your smaller models with a larger context?
Supposedly I can use https://docs.vllm.ai/en/latest/features/context_extension/ to extend the context up to 1M (512K in practice). But this does reduce accuracy and it's not working for me. I expect new models to switch to at least 512K, which would basically solve the issue for most workloads. I have a model to test in that category. Note: llm.apache.org uses Qwen 3.8 27B (FP8) as its top model, but only with a 128K context, to give you an idea. Rémy > Mark > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
