This kind of thing used to be doable by statically reweaving code using aspectj. It worked like a charm and could be done only when needed, without polluting the main code (or when it wasn't possible to alter the target code).
It is a pity aspectj is a bit of a dead branch now, I really liked it. It's hard to replace with anything else (that I know of). Dawid On Wed, Mar 19, 2025 at 2:47 PM Adrien Grand <jpou...@gmail.com> wrote: > I could see Lucene help applications a bit more with building this sort of > thing, e.g. by allowing applications to provide a listener to > IndexWriterConfig that gets called on every segment deletion, flush, merge > and live-docs/doc-values update with the total number of docs/bytes > involved and the time it took. This should help applications track > indexing-performance-related metrics more easily, e.g. write amplification > through segment merges. > > On Wed, Mar 19, 2025 at 1:22 PM Michael Sokolov <msoko...@gmail.com> > wrote: > >> One idea I've heard batted around is to override >> IndexSearcher.createWeight in a profiling IndexSearcher and then wrap >> Weights and finally Scorers in order to emit metrics on every advance >> or other low-level operation. This could help with search profiling I >> guess although of course it will slow everything down, it might be >> helpful for counting operations. But maybe your problem is during >> indexing? >> >> On Wed, Mar 19, 2025 at 5:59 AM Robert Muir <rcm...@gmail.com> wrote: >> > >> > opentelemetry is anything but lightweight, not recommended. Don't >> > waste your time with it. if you want to expose 'metrics', it is really >> > up to the application to do that, lucene is just a library. I'd >> > recommend just exposing via HTTP and using prometheus & co. >> > >> > Debugging performance issues in the library is totally different and >> > telemetry as a word doesn't make sense for that You can setup 'perf' >> > to debug performance issues. You can also run with lightweight JFR >> > profiling built into the JVM. The nightly benchmarks already do that. >> > >> > On Wed, Mar 19, 2025 at 2:16 AM Vigya Sharma <vigya.w...@gmail.com> >> wrote: >> > > >> > > Hi All, >> > > >> > > I am debugging an unexpected throughput regression in my feature >> branch which has changes across multiple files and functions, and was >> wondering if there's a better way to identify the slowdown than adding >> elapsed time logs to infostream. >> > > >> > > Do we have support in Lucene for telemetry on internal operations? It >> would be nice to be able to add timers, counters, and metrics, to identify >> performance changes. This would hide disabled behind a gradle flag. We >> could enable it for benchmark runs and possibly chart these metrics in >> nightly runs. >> > > >> > > Has this been discussed before? I'm happy to explore this more if it >> sounds like a good idea. Any suggestions for lightweight, open-source >> telemetry libraries that I should look into? I'm considering OpenTelemetry >> but open to alternate suggestions and ideas. >> > > >> > > Vigya >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> > For additional commands, e-mail: dev-h...@lucene.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> > > -- > Adrien >
