Re: Disable caching
I ran into another issue that I am having issue running to ground. My implementation on Solr 4.x worked as I expected but trying to migrate this to Solr 5.x it looks like some of the faceting is delegated to DocValuesFacets which ultimately caches things at a field level in the FieldCache.DEFAULT cache. I don't see anyway to override this cache or augment the key in anyway, am I missing an extension point here? Is there another approach I should be taking in this case? On Wed, Aug 19, 2015 at 9:08 AM, Jamie Johnson wrote: > This was my original thought. We already have the thread local so should > be straight fwd to just wrap the Field name and use that as the key. Again > thanks, I really appreciate the feedback > On Aug 19, 2015 8:12 AM, "Yonik Seeley" wrote: > >> On Tue, Aug 18, 2015 at 10:58 PM, Jamie Johnson >> wrote: >> > Hmm...so I think I have things setup correctly, I have a custom >> > QParserPlugin building a custom query that wraps the query built from >> the >> > base parser and stores the user who is executing the query. I've added >> the >> > username to the hashCode and equals checks so I think everything is >> setup >> > properly. I ran a quick test and it definitely looks like my items are >> > being cached now per user, which is really great. >> > >> > The outage that I'm running into now is the FieldValueCache doesn't take >> > into account the query, so the FieldValueCache is built for user a and >> then >> > reused for user b, which is an issue for me. In short I'm back to my >> > NoOpCache for FieldValues. It's great that I'm in a better spot for the >> > others, but is there anything that can be done with FieldValues to take >> > into account the requesting user? >> >> I guess a cache implementation that gets the user through a thread >> local and either wraps the original key with an object containing the >> user, or delegates to a per-user cache underneath. >> >> -Yonik >> >
Re: Disable caching
This was my original thought. We already have the thread local so should be straight fwd to just wrap the Field name and use that as the key. Again thanks, I really appreciate the feedback On Aug 19, 2015 8:12 AM, "Yonik Seeley" wrote: > On Tue, Aug 18, 2015 at 10:58 PM, Jamie Johnson wrote: > > Hmm...so I think I have things setup correctly, I have a custom > > QParserPlugin building a custom query that wraps the query built from the > > base parser and stores the user who is executing the query. I've added > the > > username to the hashCode and equals checks so I think everything is setup > > properly. I ran a quick test and it definitely looks like my items are > > being cached now per user, which is really great. > > > > The outage that I'm running into now is the FieldValueCache doesn't take > > into account the query, so the FieldValueCache is built for user a and > then > > reused for user b, which is an issue for me. In short I'm back to my > > NoOpCache for FieldValues. It's great that I'm in a better spot for the > > others, but is there anything that can be done with FieldValues to take > > into account the requesting user? > > I guess a cache implementation that gets the user through a thread > local and either wraps the original key with an object containing the > user, or delegates to a per-user cache underneath. > > -Yonik >
Re: Disable caching
On Tue, Aug 18, 2015 at 10:58 PM, Jamie Johnson wrote: > Hmm...so I think I have things setup correctly, I have a custom > QParserPlugin building a custom query that wraps the query built from the > base parser and stores the user who is executing the query. I've added the > username to the hashCode and equals checks so I think everything is setup > properly. I ran a quick test and it definitely looks like my items are > being cached now per user, which is really great. > > The outage that I'm running into now is the FieldValueCache doesn't take > into account the query, so the FieldValueCache is built for user a and then > reused for user b, which is an issue for me. In short I'm back to my > NoOpCache for FieldValues. It's great that I'm in a better spot for the > others, but is there anything that can be done with FieldValues to take > into account the requesting user? I guess a cache implementation that gets the user through a thread local and either wraps the original key with an object containing the user, or delegates to a per-user cache underneath. -Yonik
Re: Disable caching
Hmm...so I think I have things setup correctly, I have a custom
QParserPlugin building a custom query that wraps the query built from the
base parser and stores the user who is executing the query. I've added the
username to the hashCode and equals checks so I think everything is setup
properly. I ran a quick test and it definitely looks like my items are
being cached now per user, which is really great.
The outage that I'm running into now is the FieldValueCache doesn't take
into account the query, so the FieldValueCache is built for user a and then
reused for user b, which is an issue for me. In short I'm back to my
NoOpCache for FieldValues. It's great that I'm in a better spot for the
others, but is there anything that can be done with FieldValues to take
into account the requesting user?
On Tue, Aug 18, 2015 at 9:59 PM, Yonik Seeley wrote:
> On Tue, Aug 18, 2015 at 9:51 PM, Jamie Johnson wrote:
> > Thanks, I'll try to delve into this. We are currently using the parent
> > query parser, within we could use {!secure} I think. Ultimately I would
> > want the solr qparser to actually do the work of parsing and I'd just
> wrap
> > that.
>
> Right... look at something like BoostQParserPlugin
> it should be trivial to wrap any other type of query.
>
> baseParser = subQuery(localParams.get(QueryParsing.V), null);
> Query q = baseParser.getQuery();
>
> q={!secure}my_normal_query
> OR
> q={!secure v=$qq)&qq=my_normal_query
> OR
> q={!secure}{!parent ...}
> OR
> q={!secure v=$qq}&qq={!parent. ..}
>
> -Yonik
>
>
> > Are there any examples that I could look at for this? It's not
> > clear to me what to do in the qparser once I have the user auths though.
> > Again thanks, this is really good stuff.
> > On Aug 18, 2015 8:54 PM, "Yonik Seeley" wrote:
> >
> >> On Tue, Aug 18, 2015 at 8:38 PM, Jamie Johnson
> wrote:
> >> > I really like this idea in concept. My query would literally be just
> a
> >> > wrapper at that point, what would be the appropriate place to do this?
> >>
> >> It depends on how much you are trying to make everything transparent
> >> (that there is security) or not.
> >>
> >> First approach is explicitly changing the query types (you obviously
> >> need to make sure that only trusted code can run queries against solr
> >> for this method):
> >> q=foo:bar&fq=inStock:true
> >> q={!secure id=user}foo:bar&fq={!secure id=user}inStock:true
> >> you could even make the {!secure} qparser look for global security
> >> params so you don't need to repeat them.
> >> q={!secure}foo:bar&fq={!secure}inStock:true&security_id=user
> >>
> >> Second approach would prob involve a search component, probably that
> >> runs after the query component, that would handle wrapping any queries
> >> or filters in the prepare() phase. This would be slightly more
> >> difficult since it would require ensuring that none of the solr code /
> >> features you use re-grab the "q" or "fq" parameters re-parse without
> >> the opportunity for you to wrap them again.
> >>
> >> > What would I need to do to the query to make it behave with the cache.
> >>
> >> Probably not much... record the credentials in the wrapper and use in
> >> the hashCode / equals.
> >>
> >> -Yonik
> >>
> >>
> >> > Again thanks for the idea, I think this could be a simple way to use
> the
> >> > caches.
> >> >
> >> > On Tue, Aug 18, 2015 at 8:31 PM, Yonik Seeley
> wrote:
> >> >
> >> >> On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson
> >> wrote:
> >> >> > when you say a security filter, are you asking if I can express my
> >> >> security
> >> >> > constraint as a query? If that is the case then the answer is
> no. At
> >> >> this
> >> >> > point I have a requirement to secure Terms (a nightmare I know).
> >> >>
> >> >> Heh - ok, I figured as much.
> >> >>
> >> >> So... you could also wrap the main query and any filter queries in a
> >> >> custom security query that would contain the user, and thus still be
> >> >> able to use filter and query caches unmodified. I know... that's only
> >> >> a small part of the problem though.
> >> >>
> >> >> -Yonik
> >> >>
> >>
>
Re: Disable caching
On Tue, Aug 18, 2015 at 9:51 PM, Jamie Johnson wrote:
> Thanks, I'll try to delve into this. We are currently using the parent
> query parser, within we could use {!secure} I think. Ultimately I would
> want the solr qparser to actually do the work of parsing and I'd just wrap
> that.
Right... look at something like BoostQParserPlugin
it should be trivial to wrap any other type of query.
baseParser = subQuery(localParams.get(QueryParsing.V), null);
Query q = baseParser.getQuery();
q={!secure}my_normal_query
OR
q={!secure v=$qq)&qq=my_normal_query
OR
q={!secure}{!parent ...}
OR
q={!secure v=$qq}&qq={!parent. ..}
-Yonik
> Are there any examples that I could look at for this? It's not
> clear to me what to do in the qparser once I have the user auths though.
> Again thanks, this is really good stuff.
> On Aug 18, 2015 8:54 PM, "Yonik Seeley" wrote:
>
>> On Tue, Aug 18, 2015 at 8:38 PM, Jamie Johnson wrote:
>> > I really like this idea in concept. My query would literally be just a
>> > wrapper at that point, what would be the appropriate place to do this?
>>
>> It depends on how much you are trying to make everything transparent
>> (that there is security) or not.
>>
>> First approach is explicitly changing the query types (you obviously
>> need to make sure that only trusted code can run queries against solr
>> for this method):
>> q=foo:bar&fq=inStock:true
>> q={!secure id=user}foo:bar&fq={!secure id=user}inStock:true
>> you could even make the {!secure} qparser look for global security
>> params so you don't need to repeat them.
>> q={!secure}foo:bar&fq={!secure}inStock:true&security_id=user
>>
>> Second approach would prob involve a search component, probably that
>> runs after the query component, that would handle wrapping any queries
>> or filters in the prepare() phase. This would be slightly more
>> difficult since it would require ensuring that none of the solr code /
>> features you use re-grab the "q" or "fq" parameters re-parse without
>> the opportunity for you to wrap them again.
>>
>> > What would I need to do to the query to make it behave with the cache.
>>
>> Probably not much... record the credentials in the wrapper and use in
>> the hashCode / equals.
>>
>> -Yonik
>>
>>
>> > Again thanks for the idea, I think this could be a simple way to use the
>> > caches.
>> >
>> > On Tue, Aug 18, 2015 at 8:31 PM, Yonik Seeley wrote:
>> >
>> >> On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson
>> wrote:
>> >> > when you say a security filter, are you asking if I can express my
>> >> security
>> >> > constraint as a query? If that is the case then the answer is no. At
>> >> this
>> >> > point I have a requirement to secure Terms (a nightmare I know).
>> >>
>> >> Heh - ok, I figured as much.
>> >>
>> >> So... you could also wrap the main query and any filter queries in a
>> >> custom security query that would contain the user, and thus still be
>> >> able to use filter and query caches unmodified. I know... that's only
>> >> a small part of the problem though.
>> >>
>> >> -Yonik
>> >>
>>
Re: Disable caching
Thanks, I'll try to delve into this. We are currently using the parent
query parser, within we could use {!secure} I think. Ultimately I would
want the solr qparser to actually do the work of parsing and I'd just wrap
that. Are there any examples that I could look at for this? It's not
clear to me what to do in the qparser once I have the user auths though.
Again thanks, this is really good stuff.
On Aug 18, 2015 8:54 PM, "Yonik Seeley" wrote:
> On Tue, Aug 18, 2015 at 8:38 PM, Jamie Johnson wrote:
> > I really like this idea in concept. My query would literally be just a
> > wrapper at that point, what would be the appropriate place to do this?
>
> It depends on how much you are trying to make everything transparent
> (that there is security) or not.
>
> First approach is explicitly changing the query types (you obviously
> need to make sure that only trusted code can run queries against solr
> for this method):
> q=foo:bar&fq=inStock:true
> q={!secure id=user}foo:bar&fq={!secure id=user}inStock:true
> you could even make the {!secure} qparser look for global security
> params so you don't need to repeat them.
> q={!secure}foo:bar&fq={!secure}inStock:true&security_id=user
>
> Second approach would prob involve a search component, probably that
> runs after the query component, that would handle wrapping any queries
> or filters in the prepare() phase. This would be slightly more
> difficult since it would require ensuring that none of the solr code /
> features you use re-grab the "q" or "fq" parameters re-parse without
> the opportunity for you to wrap them again.
>
> > What would I need to do to the query to make it behave with the cache.
>
> Probably not much... record the credentials in the wrapper and use in
> the hashCode / equals.
>
> -Yonik
>
>
> > Again thanks for the idea, I think this could be a simple way to use the
> > caches.
> >
> > On Tue, Aug 18, 2015 at 8:31 PM, Yonik Seeley wrote:
> >
> >> On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson
> wrote:
> >> > when you say a security filter, are you asking if I can express my
> >> security
> >> > constraint as a query? If that is the case then the answer is no. At
> >> this
> >> > point I have a requirement to secure Terms (a nightmare I know).
> >>
> >> Heh - ok, I figured as much.
> >>
> >> So... you could also wrap the main query and any filter queries in a
> >> custom security query that would contain the user, and thus still be
> >> able to use filter and query caches unmodified. I know... that's only
> >> a small part of the problem though.
> >>
> >> -Yonik
> >>
>
Re: Disable caching
On Tue, Aug 18, 2015 at 8:38 PM, Jamie Johnson wrote:
> I really like this idea in concept. My query would literally be just a
> wrapper at that point, what would be the appropriate place to do this?
It depends on how much you are trying to make everything transparent
(that there is security) or not.
First approach is explicitly changing the query types (you obviously
need to make sure that only trusted code can run queries against solr
for this method):
q=foo:bar&fq=inStock:true
q={!secure id=user}foo:bar&fq={!secure id=user}inStock:true
you could even make the {!secure} qparser look for global security
params so you don't need to repeat them.
q={!secure}foo:bar&fq={!secure}inStock:true&security_id=user
Second approach would prob involve a search component, probably that
runs after the query component, that would handle wrapping any queries
or filters in the prepare() phase. This would be slightly more
difficult since it would require ensuring that none of the solr code /
features you use re-grab the "q" or "fq" parameters re-parse without
the opportunity for you to wrap them again.
> What would I need to do to the query to make it behave with the cache.
Probably not much... record the credentials in the wrapper and use in
the hashCode / equals.
-Yonik
> Again thanks for the idea, I think this could be a simple way to use the
> caches.
>
> On Tue, Aug 18, 2015 at 8:31 PM, Yonik Seeley wrote:
>
>> On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson wrote:
>> > when you say a security filter, are you asking if I can express my
>> security
>> > constraint as a query? If that is the case then the answer is no. At
>> this
>> > point I have a requirement to secure Terms (a nightmare I know).
>>
>> Heh - ok, I figured as much.
>>
>> So... you could also wrap the main query and any filter queries in a
>> custom security query that would contain the user, and thus still be
>> able to use filter and query caches unmodified. I know... that's only
>> a small part of the problem though.
>>
>> -Yonik
>>
Re: Disable caching
I really like this idea in concept. My query would literally be just a wrapper at that point, what would be the appropriate place to do this? What would I need to do to the query to make it behave with the cache. Again thanks for the idea, I think this could be a simple way to use the caches. On Tue, Aug 18, 2015 at 8:31 PM, Yonik Seeley wrote: > On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson wrote: > > when you say a security filter, are you asking if I can express my > security > > constraint as a query? If that is the case then the answer is no. At > this > > point I have a requirement to secure Terms (a nightmare I know). > > Heh - ok, I figured as much. > > So... you could also wrap the main query and any filter queries in a > custom security query that would contain the user, and thus still be > able to use filter and query caches unmodified. I know... that's only > a small part of the problem though. > > -Yonik >
Re: Disable caching
On Tue, Aug 18, 2015 at 8:19 PM, Jamie Johnson wrote: > when you say a security filter, are you asking if I can express my security > constraint as a query? If that is the case then the answer is no. At this > point I have a requirement to secure Terms (a nightmare I know). Heh - ok, I figured as much. So... you could also wrap the main query and any filter queries in a custom security query that would contain the user, and thus still be able to use filter and query caches unmodified. I know... that's only a small part of the problem though. -Yonik
Re: Disable caching
when you say a security filter, are you asking if I can express my security constraint as a query? If that is the case then the answer is no. At this point I have a requirement to secure Terms (a nightmare I know). Our fallback is to aggregate the authorizations to a document level and secure the document which I think we wouldn't have to do anything to the caches but our customer has pushed back on this in the past. On Tue, Aug 18, 2015 at 7:46 PM, Yonik Seeley wrote: > On Tue, Aug 18, 2015 at 7:11 PM, Jamie Johnson wrote: > > Yes, my use case is security. Basically I am executing queries with > > certain auths and when they are executed multiple times with differing > > auths I'm getting cached results. > > If it's just simple stuff like top N docs returned, can't you just use > a security filter? > > The queryResult cache uses both the main query and a list of filters > (and the sort order) for the cache key. > > -Yonik >
Re: Disable caching
On Tue, Aug 18, 2015 at 7:11 PM, Jamie Johnson wrote: > Yes, my use case is security. Basically I am executing queries with > certain auths and when they are executed multiple times with differing > auths I'm getting cached results. If it's just simple stuff like top N docs returned, can't you just use a security filter? The queryResult cache uses both the main query and a list of filters (and the sort order) for the cache key. -Yonik
Re: Disable caching
Yes, my use case is security. Basically I am executing queries with certain auths and when they are executed multiple times with differing auths I'm getting cached results. One option is to have another implementation that has a number of caches based on the auths, something that I suspect we will at some point go (unless there is a better solution ;). I'd be happy to look at other options so all suggestions are appreciated. On Tue, Aug 18, 2015 at 6:56 PM, Yonik Seeley wrote: > You can comment out (some) of the caches. > > There are some caches like field caches that are more at the lucene > level and can't be disabled. > > Can I ask what you are trying to prevent from being cached and why? > Different caches are for different things, so it would seem to be an > odd usecase to disable them all. Security? > > -Yonik > > > On Tue, Aug 18, 2015 at 6:52 PM, Jamie Johnson wrote: > > I see that if Solr is in realtime mode that caching is disable within the > > SolrIndexSearcher that is created in SolrCore, but is there anyway to > > disable caching without being in realtime mode? Currently I'm > implementing > > a NoOp cache that implements SolrCache but returns null for everything > and > > doesn't return anything on the get requests, but it would be nice to not > > need to do this by being able to disable caching in general. Is this > > possible? > > > > -Jamie >
Re: Disable caching
You can comment out (some) of the caches. There are some caches like field caches that are more at the lucene level and can't be disabled. Can I ask what you are trying to prevent from being cached and why? Different caches are for different things, so it would seem to be an odd usecase to disable them all. Security? -Yonik On Tue, Aug 18, 2015 at 6:52 PM, Jamie Johnson wrote: > I see that if Solr is in realtime mode that caching is disable within the > SolrIndexSearcher that is created in SolrCore, but is there anyway to > disable caching without being in realtime mode? Currently I'm implementing > a NoOp cache that implements SolrCache but returns null for everything and > doesn't return anything on the get requests, but it would be nice to not > need to do this by being able to disable caching in general. Is this > possible? > > -Jamie
Disable caching
I see that if Solr is in realtime mode that caching is disable within the SolrIndexSearcher that is created in SolrCore, but is there anyway to disable caching without being in realtime mode? Currently I'm implementing a NoOp cache that implements SolrCache but returns null for everything and doesn't return anything on the get requests, but it would be nice to not need to do this by being able to disable caching in general. Is this possible? -Jamie
Disable caching in sort
Hello All,
We are trying to provide a personalized sort order for each user. We've a
per-computed list of user to products and if it matches the solr result set
those products need to be shown upfront. One way can be handling this in
application but pagination becomes tricky. Another way we are exploring
this is via a custom value source where we'll pass productid to
custom-score and sort based on this. We've been able to manipulate result
set using this, but sort order is getting cached. One way can be using
{!cache=false} but that would lead to performance degradation. Any other
way of achieving this?
--
Thanks & Regards,
Apoorva
RE: Disable caching on sorting to improve performance
Thanks and good call, that has been there for quite some time! I've changed it to: -Xms200m -Xmx1500m I'll look into the effect of this first. -- View this message in context: http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356p4108362.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: Disable caching on sorting to improve performance
Hi,
Sorting uses Lucene FieldCache which you cannot disable. If you have lots of
documents and sort on three fields, you must increase the heap space. Also,
note that you have defined Xmx twice here, i don't know what effect that will
have.
Cheers
-Original message-
> From:PeterKerk
> Sent: Friday 27th December 2013 12:39
> To: solr-user@lucene.apache.org
> Subject: Disable caching on sorting to improve performance
>
> I'm getting a lot of java heap memory full errors. I've now been reading into
> solr performance (in the meantime also configuring the sematext tools to try
> to drill down to the cause)
>
> I already increased the memory available to Solr:
> bash -c "cd /cygdrive/c/Databases/solr-4.3.1/example/;java
> -Dsolr.solr.home="./example-DIH/solr/" -jar -Xmx200m -Xmx1200m start.jar"
>
> And now I read:
> Factors that affect memory usage:
> http://stackoverflow.com/questions/1546898/how-to-reduce-solr-memory-usage
> I see that sorting affects memory usage. I have the feeling that is the case
> for me, because since I implemented sorting the memory errors are going
> through the roof.
>
> I read here
> http://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters how to
> do it on filter queries, but I was wondering how I can disable the caching
> on the sort parameter in below statement where it now has
> `&sort=clickcount%20desc,prijs%20desc,updatedate%20desc`
>
>
> searchquery.Append("&fl=id,artikelnummer,titel,friendlyurl,pricerange,lang,currency,createdate")
> searchquery.Append("&facet.field=pricerange")
> searchquery.Append("&facet.mincount=1")
> searchquery.Append("&facet.sort=index")
> searchquery.Append("&omitHeader=true")
> searchquery.Append("&sort=clickcount%20desc,prijs%20desc,updatedate%20desc")
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Disable caching on sorting to improve performance
I'm getting a lot of java heap memory full errors. I've now been reading into
solr performance (in the meantime also configuring the sematext tools to try
to drill down to the cause)
I already increased the memory available to Solr:
bash -c "cd /cygdrive/c/Databases/solr-4.3.1/example/;java
-Dsolr.solr.home="./example-DIH/solr/" -jar -Xmx200m -Xmx1200m start.jar"
And now I read:
Factors that affect memory usage:
http://stackoverflow.com/questions/1546898/how-to-reduce-solr-memory-usage
I see that sorting affects memory usage. I have the feeling that is the case
for me, because since I implemented sorting the memory errors are going
through the roof.
I read here
http://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters how to
do it on filter queries, but I was wondering how I can disable the caching
on the sort parameter in below statement where it now has
`&sort=clickcount%20desc,prijs%20desc,updatedate%20desc`
searchquery.Append("&fl=id,artikelnummer,titel,friendlyurl,pricerange,lang,currency,createdate")
searchquery.Append("&facet.field=pricerange")
searchquery.Append("&facet.mincount=1")
searchquery.Append("&facet.sort=index")
searchquery.Append("&omitHeader=true")
searchquery.Append("&sort=clickcount%20desc,prijs%20desc,updatedate%20desc")
--
View this message in context:
http://lucene.472066.n3.nabble.com/Disable-caching-on-sorting-to-improve-performance-tp4108356.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Disable Caching
: If you are not searching against your master, and you shouldn't (and : it sounds like you aren't), then you don't have to worry about : disabling caches - they will just remain empty. You could comment : them out, but I think that won't actually disable them. FWIW: what i generall advocate is that even if you have a machine that *should* never get queries (ie: master, repeater, backup, ... whatever) it's still a good idea to leave small caching enabled on that machine. the reason being that *IF* some rogue client mistakenly starts sending requests to this server they are not suppose to be sending requests to, then i would rather those requests get cached to help reduce the likelyood of the machine rolling over and dieing under load -- but i advise disabling all the autowarming and any explicit newSearcher warming you might normally confugre so that if there is a blip and this situation does happen, then newSearcher events aren't delayed dealing with caching warming over and over again even long after the rogue client gets stopped. Related suggestion: disable all the requestHandler names you clients nomraly query in your "master" solrconfig.xml, and only expose on using a really bizare unlikely name so you have someway to query that index for debugging ... that way if clients that normally query "http://slave:8983/solr/products/search?q=..."; get misconfigured to hit "http://master:8983/solr/products/search?q=..."; they'll get a 404. but you can still use "http://master:8983/solr/products/secret-debug-search?q=..."; as needed. -Hoss
Re: Disable Caching
Thanks for the replies. 2012/10/17 Otis Gospodnetic > Hi, > > If you are not searching against your master, and you shouldn't (and > it sounds like you aren't), then you don't have to worry about > disabling caches - they will just remain empty. You could comment > them out, but I think that won't actually disable them. > > Warmup queries you can just comment our in solrconfig.xml. > > Otis > -- > Search Analytics - http://sematext.com/search-analytics/index.html > Performance Monitoring - http://sematext.com/spm/index.html > > > On Wed, Oct 17, 2012 at 12:25 PM, Anderson vasconcelos > wrote: > > Hi > > > > I have a server that just index data and sincronize this data to others > > slaves. In my arquitecture, i have a one master server that only receive > > index requests and n slaves that receives only search requests. > > > > I wanna to disable the cache of the master server, because they not > receive > > a search request, this is the best way? I can do this? > > > > Wat about warmingSearch, i must disable this too? > > > > I'm using solr 3.6.0 > > > > Thanks >
Re: Disable Caching
Hi, If you are not searching against your master, and you shouldn't (and it sounds like you aren't), then you don't have to worry about disabling caches - they will just remain empty. You could comment them out, but I think that won't actually disable them. Warmup queries you can just comment our in solrconfig.xml. Otis -- Search Analytics - http://sematext.com/search-analytics/index.html Performance Monitoring - http://sematext.com/spm/index.html On Wed, Oct 17, 2012 at 12:25 PM, Anderson vasconcelos wrote: > Hi > > I have a server that just index data and sincronize this data to others > slaves. In my arquitecture, i have a one master server that only receive > index requests and n slaves that receives only search requests. > > I wanna to disable the cache of the master server, because they not receive > a search request, this is the best way? I can do this? > > Wat about warmingSearch, i must disable this too? > > I'm using solr 3.6.0 > > Thanks
RE: Disable Caching
Yes Anderson, you don't need cache for master, neither warming. -Original Message- From: Anderson vasconcelos [mailto:anderson.v...@gmail.com] Sent: Wednesday, October 17, 2012 9:55 PM To: solr-user Subject: Disable Caching Hi I have a server that just index data and sincronize this data to others slaves. In my arquitecture, i have a one master server that only receive index requests and n slaves that receives only search requests. I wanna to disable the cache of the master server, because they not receive a search request, this is the best way? I can do this? Wat about warmingSearch, i must disable this too? I'm using solr 3.6.0 Thanks
Re: OOME diagnosis - possible to disable caching?
On 19-Jan-09, at 2:44 PM, James Brady wrote: Hi all, I have 20 indices, each ~10GB in size, being searched by a single Solr slave instance (using the multicore features in a slightly old 1.2 dev build) I'm getting unpredictable, but inevitable, OutOfMemoryError from the slave, and I have no more physical memory to throw at the problem (HotSpot 1.6 with Xmx=4000m). At this point, I'd like to see how much memory Lucene is eating by disabling all supplemental Solr caches. Which solrconfig settings do I need to be paying attention to here? filterCache, queryResultCache and documentCache? That should be it. Note that it is unadvisable to reduce documentCache size to less than 100 or so, as Solr assumes that enough cache for the docs for one query. A heap dump should help point you in to the relevant problems. Also, it should be easy to get a rough estimate of the filtercache mem usage by looking at its size. -Mike
OOME diagnosis - possible to disable caching?
Hi all, I have 20 indices, each ~10GB in size, being searched by a single Solr slave instance (using the multicore features in a slightly old 1.2 dev build) I'm getting unpredictable, but inevitable, OutOfMemoryError from the slave, and I have no more physical memory to throw at the problem (HotSpot 1.6 with Xmx=4000m). At this point, I'd like to see how much memory Lucene is eating by disabling all supplemental Solr caches. Which solrconfig settings do I need to be paying attention to here? filterCache, queryResultCache and documentCache? I'm not faceting, sorting, highlighting or anything like that (all in an effort to get more docs in a searchable index!) Thanks! James
