If you create SolrQueryRequest make sure you close it then, since it's necessary to release a searcher.
On Wed, Apr 19, 2017 at 12:35 PM, Elodie Sannier <elodie.sann...@kelkoo.fr> wrote: > Hello, > > We have found how to fix the problem. > When we update the original SolrQueryResponse object, we need to create > a new BasicResultContext object with the extra response. > > Simplified code : > > public class CustomSearchHandler extends SearchHandler { > > public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse > rsp) throws Exception { > > SolrQueryRequest extraReq = createExtraRequest(); > SolrQueryResponse extraRsp = new SolrQueryResponse(); > > super.handleRequestBody(extraReq, extraRsp); > > ResultContext extraRc = (ResultContext) extraRsp.getResponse(); > > // code with memory leak !! > rsp.addResponse(extraRc); > > // code without memory leak > ResultContext extraRcClone = new BasicResultContext(extraRc.get > DocList(), > rsp.getReturnFields(), req.getSearcher(), > extraRc.getQuery(), req); > rsp.addResponse(extraRcClone); > > } > > } > > We don't know why we need to create a new BasicResultContext to properly > manage searchers. Do you know why ? > > Elodie > > > On 04/07/2017 04:14 PM, Rick Leir wrote: > >> Hi Gerald >> The best solution in my mind is to look at the custom code and try to >> find a way to remove it from your system. Solr queries can be complex, and >> I hope there is a way to get the results you need. Would you like to say >> what results you want to get, and what Solr queries you have tried? >> I realize that in large organizations it is difficult to suggest change. >> Cheers -- Rick >> >> On April 7, 2017 9:08:19 AM EDT, Shawn Heisey <apa...@elyograg.org> >> wrote: >> >>> On 4/7/2017 3:09 AM, Gerald Reinhart wrote: >>> >>>> We have some custom code that extends SearchHandler to be able to >>>> >>> : >>> >>>> - do an extra request >>>> - merge/combine the original request and the extra request >>>> results >>>> >>>> On Solr 5.x, our code was working very well, now with Solr 6.x we >>>> have the following issue: the number of SolrIndexSearcher are >>>> increasing (we can see them in the admin view > Plugins/ Stats > Core >>>> >>> ). >>> >>>> As SolrIndexSearcher are accumulating, we have the following issues : >>>> - the memory used by Solr is increasing => OOM after a long >>>> period of time in production >>>> - some files in the index has been deleted from the system but >>>> the Solr JVM still hold them => ("fake") Full disk after a long >>>> >>> period >>> >>>> of time in production >>>> >>>> We are wondering, >>>> - what has changed between Solr 5.x and Solr 6.x in the >>>> management of the SolrIndexSearcher ? >>>> - what would be the best way, in a Solr plugin, to perform 2 >>>> queries and merge the results to a single SolrQueryResponse ? >>>> >>> I hesitated to send a reply because when it comes right down to it, I >>> do >>> not know a whole lot about deep Solr internals. I tend to do my work >>> with the code at a higher level, and don't dive down in the depths all >>> that often. I am slowly learning, though. You may need to wait for a >>> reply from someone who really knows those internals. >>> >>> It looks like you and I participated in a discussion last month where >>> you were facing a similar problem with searchers -- deleted index files >>> being held open. How did that turn out? Seems like if that problem >>> were solved, it would also solve this problem. >>> >>> Very likely, the fact that the plugin worked correctly in 5.x was >>> actually a bug in Solr related to reference counting, one that has been >>> fixed in later versions. >>> >>> You may need to use a paste website or a file-sharing website to share >>> all your plugin code so that people can get a look at it. The list has >>> a habit of deleting attachments. >>> >>> Thanks, >>> Shawn >>> >> > > Kelkoo SAS > Société par Actions Simplifiée > Au capital de € 4.168.964,30 > Siège social : 158 Ter Rue du Temple 75003 Paris > 425 093 069 RCS Paris > > Ce message et les pièces jointes sont confidentiels et établis à > l'attention exclusive de leurs destinataires. Si vous n'êtes pas le > destinataire de ce message, merci de le détruire et d'en avertir > l'expéditeur. > -- Sincerely yours Mikhail Khludnev