Thanks Adrien for quick reply! We use jboss app server to handle application level threads.
Thanks, Mukul Ranjan From: Adrien Grand [mailto:[email protected]] Sent: Friday, December 23, 2016 6:33 PM To: Mukul Ranjan <[email protected]>; Lucene Users Mailing List <[email protected]> Subject: Re: CompressingStoredFieldReader object taking lot of heap space Hi, SearcherManager only cares about managing readers/searchers in a thread-safe way, it does not know about the thread pool it runs in. How does your application handle threads? I'm thinking you might be running in a servlet container, or a similar framework that handles it for you? Le ven. 23 déc. 2016 à 13:57, Mukul Ranjan <[email protected]<mailto:[email protected]>> a écrit : Hi Adrien, We are using searcher manager for acquiring and releasing index searcher object. Is there any way we can control fixed thread pool using searcher Manager? Thanks, Mukul Ranjan From: Adrien Grand [mailto:[email protected]<mailto:[email protected]>] Sent: Thursday, December 22, 2016 11:25 PM To: Mukul Ranjan <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: CompressingStoredFieldReader object taking lot of heap space Hi Mukul, Lucene maintains a number of threadlocals alongside each segment. While these structures should remain small (a couple tens of KB at most), the fact we have them per segment and in threadlocals means that overall memory usage for these structures is linear with the number of threads that have access to the index and to the number of segments that your index has. So you plan on having access to an index from multiple thread, you should make sure it is a fixed thread pool that has a reasonable number of threads (in the order of the number of parallel threads that the host can run). Le jeu. 22 déc. 2016 à 07:25, Mukul Ranjan <[email protected]<mailto:[email protected]>> a écrit : Hi Adrien, Could you please explain the 2nd point in detail? Thanks, Mukul Ranjan -----Original Message----- From: Adrien Grand [mailto:[email protected]<mailto:[email protected]>] Sent: Wednesday, December 21, 2016 6:43 PM To: [email protected]<mailto:[email protected]> Subject: Re: CompressingStoredFieldReader object taking lot of heap space This issue has been reported a couple times and is usually due to one of these issues: - readers are not always closed - the index is accessed from a non fixed thread pool - there are too many indices open at the same time Le mer. 21 déc. 2016 à 13:37, Mukul Ranjan <[email protected]<mailto:[email protected]>> a écrit : > Hi, > > We are using lucene 5.5.2 for search. > > We ran load test for 8 hours on our setup and observed that > "org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader" > object is taking 210 MB out of 1.2 GB heap space. There are 8890 > instances of this class present in the heap. > > Could you please confirm if this is the normal behaviour or there is > some issue with the code. > > Thanks, > Mukul Ranjan > > ----- > Watch an eGain Try+Buy(tm) customer success video< > https://www.youtube.com/watch?v=ewEeyeqJRCE&feature=youtu.be/> >
