IMO Lucene should just be a library with ~no dependencies. It just needs to expose the right callbacks to allow for easy observability higher up the stack.
Today I was looking at Solr's hooks into Lucene on this topic. SolrIndexWriter overrides a protected method doAfterFlush allowing Solr to increment a counter for every flushed segment. Pretty easy. Although it's not clear, looking at IndexWriter, how to get any information about the size of that segment in docs or bytes; if someone sees how to do that elegantly, please let me know. Without that detail, you can't compute a good write-amplification-factor. ~ David On Wed, Mar 19, 2025 at 2:16 AM Vigya Sharma <[email protected]> wrote: > Hi All, > > I am debugging an unexpected throughput regression in my feature branch > which has changes across multiple files and functions, and was wondering if > there's a better way to identify the slowdown than adding elapsed time logs > to infostream. > > Do we have support in Lucene for telemetry on internal operations? It > would be nice to be able to add timers, counters, and metrics, to > identify performance changes. This would hide disabled behind a gradle > flag. We could enable it for benchmark runs and possibly chart these > metrics in nightly runs. > > Has this been discussed before? I'm happy to explore this more if it > sounds like a good idea. Any suggestions for lightweight, open-source > telemetry libraries that I should look into? I'm considering OpenTelemetry > but open to alternate suggestions and ideas. > > Vigya >
