RE: How to limit the number of open searchers?
I'm curious, why can't you do a master/slave setup? It's just not all that useful for this particular application. Indexing new docs and merging segments - which as I understand is the main strength of having a write-only master - is a relatively small part of our app. What really is expensive perf-wise for us is the facet regeneration after each commit, and long-running complex queries. I'm not sure a master/slave setup would solve the issue with many open searchers - even with a long poll interval, won't the slave still potentially open many searchers if old searchers are still in use? My current plan is to create a request handler that iterates through the mbeans in req.getCore().getInfoRegistry() and gets a count of how many distinct SolrIndexSearchers there are, then returns this count. Seems really hacky, but fairly straightforward. Would love to hear any ideas on a better way to do this. -Michael
Re: How to limit the number of open searchers?
Ah, you're right. If you queries run across several commits you'll get multiple searchers open. I don't know of any good way to do what you want. I'm curious, why can't you do a master/slave setup? The other thing to think about would be the NRT stuff if you can run trunk. Best Erick On Wed, Mar 7, 2012 at 2:30 PM, Michael Ryan mr...@moreover.com wrote: Unless you have warming happening, there should only be a single searcher open at any given time. So it seems to me that maxWarmingSearchers should give you what you need. What I'm seeing is that if a query takes a very long time to run, and runs across the duration of multiple commits (I know, that itself sounds bad!), I can get into a situation where I have 2 searchers in use and 1 searcher warming, rather than 1 searcher in use and 1 searcher warming. Due to all the memory-intensive features I use, having 3 or more searchers open can cause an OutOfMemoryError. I'm not using master/slave for this application, so can't go that route. I'd like a way to see how many searchers are currently open that is external to Solr. This would allow me to block my commits until I see that there is only 1 searcher currently open. I could use JMX, but that feels like overkill - wondering if there is something simpler. -Michael
Re: How to limit the number of open searchers?
Unless you have warming happening, there should only be a single searcher open at any given time. So it seems to me that maxWarmingSearchers should give you what you need. And you can pretty easily insure this by making your poll interval (assuming master/slave) longer than your warmup time. Best Erick On Mon, Mar 5, 2012 at 2:18 PM, Michael Ryan mr...@moreover.com wrote: Is there a way to limit the number of searchers that can be open at a given time? I know there is a maxWarmingSearchers configuration that limits the number of warming searchers, but that's not quite what I'm looking for... Ideally, when I commit, I want there to only be one searcher open before the commit, so that during the commit and warming, there is a max of two searchers open. I'd be okay with delaying the commit until there is only one searcher open. Is there a way to programmatically determine how many searchers are currently open? -Michael
RE: How to limit the number of open searchers?
Unless you have warming happening, there should only be a single searcher open at any given time. So it seems to me that maxWarmingSearchers should give you what you need. What I'm seeing is that if a query takes a very long time to run, and runs across the duration of multiple commits (I know, that itself sounds bad!), I can get into a situation where I have 2 searchers in use and 1 searcher warming, rather than 1 searcher in use and 1 searcher warming. Due to all the memory-intensive features I use, having 3 or more searchers open can cause an OutOfMemoryError. I'm not using master/slave for this application, so can't go that route. I'd like a way to see how many searchers are currently open that is external to Solr. This would allow me to block my commits until I see that there is only 1 searcher currently open. I could use JMX, but that feels like overkill - wondering if there is something simpler. -Michael
Re: How to limit the number of open searchers?
what do u mean programmatically? modify codes of solr? becuase solr is
not like lucene, it only provide http interfaces for its users other than
java api
if you want to modify solr, you can find codes in SolrCore
private final LinkedListRefCountedSolrIndexSearcher _searchers = new
LinkedListRefCountedSolrIndexSearcher();
and _searcher is current searcher.
be careful to use searcherLock to synchronizing your codes.
maybe you can write your codes like:
synchronized(searcherLock){
if(_searchers.size==1){
...
}
}
On Tue, Mar 6, 2012 at 3:18 AM, Michael Ryan mr...@moreover.com wrote:
Is there a way to limit the number of searchers that can be open at a
given time? I know there is a maxWarmingSearchers configuration that
limits the number of warming searchers, but that's not quite what I'm
looking for...
Ideally, when I commit, I want there to only be one searcher open before
the commit, so that during the commit and warming, there is a max of two
searchers open. I'd be okay with delaying the commit until there is only
one searcher open. Is there a way to programmatically determine how many
searchers are currently open?
-Michael
How to limit the number of open searchers?
Is there a way to limit the number of searchers that can be open at a given time? I know there is a maxWarmingSearchers configuration that limits the number of warming searchers, but that's not quite what I'm looking for... Ideally, when I commit, I want there to only be one searcher open before the commit, so that during the commit and warming, there is a max of two searchers open. I'd be okay with delaying the commit until there is only one searcher open. Is there a way to programmatically determine how many searchers are currently open? -Michael
