mikemccand commented on pull request #177: URL: https://github.com/apache/lucene/pull/177#issuecomment-866787723
> the JMH will do this too. I forget the defaults, but uses multiple jvm iterations and iterations within each jvm and warmup iterations. But it has smarts around the JIT compiler and can dump profiled assembly for its microbenchmarks. I never have noise issues with it. Excellent! > The current big "integration test" (lucene util) is useful for some things: e.g. something has to tell us there is pollution from too many java abstractions going megamorphic and so on :) +1 It really is more of an integration test, yeah. It runs many different kinds of queries/tasks, concurrently across multiple threads, trying to exercise Lucene roughly in a way that OpenSearch/Elasticsearch/Solr might. Though, it does not do concurrent indexing with searching in a single JVM, at least not with the default benchmarks. Really, distributed search engines should not do that -- they should rather use [Lucene's near-real-time segment replication](https://blog.mikemccandless.com/2017/09/lucenes-near-real-time-segment-index.html#:~:text=Lucene's%20near%2Dreal%2Dtime%20segment%20index%20replication,-%5BTL%3BDR%3A&text=Lucene%20has%20a%20unique%20write,files%20will%20never%20again%20change.), which is more efficient if you have deep replicas, and also enables strong physical isolation of indexing and searching JVMs which have very different resources requirements! OK `</soapbox>`! > But I think it would be improved by providing some more diagnostics (LogCompilation or whatever, maybe JIT stats in the JFR output). Let it be a "canary" to find little ways to improve. +1. I wonder if we could tap into those in real-time and get a sense of when the JVM really is roughly "warmed up", instead of the static "discard first N samples for each task" that we do now. Or maybe to detect mis-compilation of `readVInt`! > But we have nothing setup to do simple noise-free microbenchmarks over some specific code, e.g. like "unit tests" running different query types. And for those you don't want crazy JFR and logging and stuff as it is so targeted, you can just dump the hot assembly code instead. For now if you want to do this, you are writing one-off stuff yourself. Yeah maybe consing up a quick JMH for such cases is perfectly fine solution for we developers? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
