Hi, Next time please reply/send these technical questions to java-u...@lucene.apache.org or d...@lucene.apache.org to get more audience.
538 segments seems huge. What is your configuration to get this number of segments? Le jeu. 16 avr. 2020 à 16:28, 张海雷 <zhanghai...@bytedance.com> a écrit : > > > > Hi, all > In our Elastic Search benchmark, found out that IndexReader‘s > registerParentReader method takes a lot of CPU time > Environment: > ES Version: 7.5.1 > Lucene Version: 8.3.0 > > Index: > Shard Num 1 > Replicator’s Num:1 > Segment Count: 538 > Size:49.5GB > > Benchmark result: qps:5200,pct99 600ms > CPU Util:93% > > Use flame graph find the hot method IndexReader‘s registerParentReader. > dive into code find out that Have a race condition that on the > synchronizedSet Because of only one > Shard and one shard have too many segments, this intensify competition > > > > To verify this assumption,execute force merge on this index,after force > merge: > QPS: more then 5600 > PCT99: 60ms > CPU Util: 11% > The performance improved by a factor of 10x > > So we can use ConcurrentWeakKeyHashmap instead of synchronizedSet to > reduce lock competition? > > > > > > >