jaisonbi commented on PR #816: URL: https://github.com/apache/lucene/pull/816#issuecomment-1109236031
This issue has no relationship with "creating and destroying too many threads", but it's caused by **too many "ThreadLocal" objects created from same threads**. Based on the implementation of ThreadLocal: 1. All the "ThreadLocal" objects created from same thread will be stored in one ThreadLocalMap. 2. Each thread has it's own ThreadLocalMap. The investigation on the composition of the "ThreadLocalMap": "**management**" thread: Most entries(Each ThreadLocal object is wrapped into an Entry) are "CompressingStoredFieldsReader" ``` final CloseableThreadLocal<StoredFieldsReader> fieldsReaderLocal = new CloseableThreadLocal<StoredFieldsReader>() { @Override protected StoredFieldsReader initialValue() { return fieldsReaderOrig.clone(); } }; ``` "**write**" thread: Most entries are "PerThreadIDVersionAndSeqNoLookup" ``` static final ConcurrentMap<IndexReader.CacheKey, CloseableThreadLocal<PerThreadIDVersionAndSeqNoLookup[]>> lookupStates = ConcurrentCollections.newConcurrentMapWithAggressiveConcurrency(); ``` Either "CompressingStoredFieldsReader" or "PerThreadIDVersionAndSeqNoLookup" is created by "CloseableThreadLocal". ReentrantReadWriteLock will create it's own ThreadLocal object. So the ReentrantReadWrite lock aquired from "write" thread, it will share the same ThreadLocalMap with "PerThreadIDVersionAndSeqNoLookup". There're too many entries stored into ThreadLocalMap, this is why "ThreadLocal#remove" running with high CPU usage. So I think the heavy usage of "ThreadLocal" from Lucene's current machanism is one major reason of this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org