Re: clearing document cache || solr 6.6

2019-01-30 Thread Shawn Heisey
On 1/30/2019 2:27 AM, sachin gk wrote: To support an existing functionality we have turned the opensearcher to false. Is there a way to flush the cache programiticaly. Executing a commit with openSearcher=true is the only way I know of without custom code. When you commit with openSearcher s

Re: clearing document cache || solr 6.6

2019-01-30 Thread Erick Erickson
I'd also ask why you care? What benefit do you think you'd get if you did explicitly flush the document cache? You seem to think there's some benefit to programmatically flushing the cache, but you haven't stated what that benefit is. I suspect that you are making some assu

Re: clearing document cache || solr 6.6

2019-01-30 Thread Walter Underwood
You don’t need to do that. When there is a commit, Solr creates a new Searcher with an empty document cache. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jan 29, 2019, at 10:27 PM, sachin gk wrote: > > Hi All, > > Is there a w

Re: clearing document cache || solr 6.6

2019-01-30 Thread sachin gk
Thanks Shawn, To support an existing functionality we have turned the opensearcher to false. Is there a way to flush the cache programiticaly. Regards, Sachin On Wed, Jan 30, 2019, 12:58 PM Shawn Heisey On 1/29/2019 11:27 PM, sachin gk wrote: > > Is there a way to clear the *document

Re: clearing document cache || solr 6.6

2019-01-29 Thread Shawn Heisey
On 1/29/2019 11:27 PM, sachin gk wrote: Is there a way to clear the *document cache* after we commit to the indexer. All Solr caches are invalidated when you issue a commit with openSearcher set to true. The default setting is true, and normally it doesn't get set to false unles

clearing document cache || solr 6.6

2019-01-29 Thread sachin gk
Hi All, Is there a way to clear the *document cache* after we commit to the indexer. -- Regards, Sachin

Re: Document Cache

2016-03-19 Thread Emir Arnautovic
Hi, Your cache will be cleared on soft commits - every two minutes. It seems that it is either configured to be huge or you have big documents and retrieving all fields or dont have lazy field loading set to true. Can you please share your document cache config and heap settings. Thanks

Re: Document Cache

2016-03-19 Thread Emir Arnautovic
Problem starts with autowarmCount="5000" - that executes 5000 queries when new searcher is created and as queries are executed, document cache is filled. If you have large queryResultWindowSize and queries return big number of documents, that will eat up memory before new search is

Re: Document Cache

2016-03-19 Thread Rallavagu
nSearcher is set to "false". What I am seeing is (from heap dump due to OutOfMemory error) that the LRUCache pertaining "Document Cache" occupies around 85% of available heap and that is causing OOM errors. So, trying to understand the behavior to address the OOM issues.

Re: Document Cache

2016-03-19 Thread Emir Arnautovic
/16 8:56 AM, Emir Arnautovic wrote: Problem starts with autowarmCount="5000" - that executes 5000 queries when new searcher is created and as queries are executed, document cache is filled. If you have large queryResultWindowSize and queries return big number of documents, that will eat

Re: Document Cache

2016-03-19 Thread Rallavagu
e executed, document cache is filled. If you have large queryResultWindowSize and queries return big number of documents, that will eat up memory before new search is executed. It probably takes some time as well. This is also combined with filter cache. How big is your index? Index is not very large.

Re: Document Cache

2016-03-19 Thread Rallavagu
On 3/18/16 8:56 AM, Emir Arnautovic wrote: Problem starts with autowarmCount="5000" - that executes 5000 queries when new searcher is created and as queries are executed, document cache is filled. If you have large queryResultWindowSize and queries return big number of documents, tha

Document Cache

2016-03-19 Thread Rallavagu
Solr 5.4 embedded Jetty Is it the right assumption that whenever a document that is returned as a response to a query is cached in "Document Cache"? Essentially, if I request for any entry like /select?q=id: will it be cached in "Document Cache"? If yes, what is the TTL? Thanks in advance

Re: Document Cache

2016-03-19 Thread Rallavagu
18/2016 8:22 AM, Rallavagu wrote: So, each soft commit would create a new searcher that would invalidate the old cache? Here is the configuration for Document Cache true In an earlier message, you indicated you're running into OOM. I think we can see why with this cache definition. The

Re: Document Cache

2016-03-19 Thread Erick Erickson
rick On Thu, Mar 17, 2016 at 1:40 PM, Rallavagu wrote: > Solr 5.4 embedded Jetty > > Is it the right assumption that whenever a document that is returned as a > response to a query is cached in "Document Cache"? > > Essentially, if I request for any entry like /select?q=

Re: Document Cache

2016-03-18 Thread Shawn Heisey
On 3/18/2016 8:22 AM, Rallavagu wrote: > So, each soft commit would create a new searcher that would invalidate > the old cache? > > Here is the configuration for Document Cache > > initialSize="10" autowarmCount="0"/> > > true In an earlier mes

Re: Document Cache

2016-03-18 Thread Rallavagu
So, each soft commit would create a new searcher that would invalidate the old cache? Here is the configuration for Document Cache autowarmCount="0"/> true Thanks On 3/18/16 12:45 AM, Emir Arnautovic wrote: Hi, Your cache will be cleared on soft commits - every two minutes.

Re: Ignoring the Document Cache per query

2015-05-29 Thread Erick Erickson
This is totally weird. The document cache should really have nothing to do with whether MLT returns documents or not AFAIK. So either I'm totally misunderstanding MLT, you're leaving out a step or there's some bug in Solr. The fact that setting the document cache to 0 changes t

Re: Ignoring the Document Cache per query

2015-05-29 Thread Bryan Bende
o a weird state. Through trial and error I narrowed down that if we set the documentCache size to 0, then this problem doesn't happen. Since we can't really figure out why this is happening in Solr, we were hoping there was some way to not use the document cache on the call where we use th

Re: Ignoring the Document Cache per query

2015-05-28 Thread Erick Erickson
disk seek. So I must be missing why you want to do this. Best, Erick On Thu, May 28, 2015 at 1:23 PM, Bryan Bende wrote: > Is there a way to the document cache on a per-query basis? > > It looks like theres {!cache=false} for preventing the filter cache from > being used for a given q

Ignoring the Document Cache per query

2015-05-28 Thread Bryan Bende
Is there a way to the document cache on a per-query basis? It looks like theres {!cache=false} for preventing the filter cache from being used for a given query, looking for the same thing for the document cache. Thanks, Bryan

Re: How to size document cache

2013-10-25 Thread Shawn Heisey
On 10/25/2013 7:48 AM, Erick Erickson wrote: > I hadn't thought about it before, but now I'm curious how > MMapDirectoryFactory plays into documentCache. Uwe, > are you listening? :) My _guess_ is that if you're using > MMapDirectoryFactory, the usefulness of the d

Re: How to size document cache

2013-10-25 Thread Erick Erickson
I hadn't thought about it before, but now I'm curious how MMapDirectoryFactory plays into documentCache. Uwe, are you listening? :) My _guess_ is that if you're using MMapDirectoryFactory, the usefulness of the document cache is lessened, kinda. Since the documents are coming f

How to size document cache

2013-10-25 Thread Matteo Grolla
Hi, I'd really appreciate if you could give me some help understanding how to tune the document cache. My thoughts: min values: max_results * max_concurrent_queries, as stated by http://wiki.apache.org/solr/SolrCaching how can I estimate max_concurrent_qu

Re: Soft Commit and Document Cache

2013-04-22 Thread Niran Fajemisin
Thanks Shawn and Mark! That was very helpful. -Niran > > From: Shawn Heisey >To: solr-user@lucene.apache.org >Sent: Monday, April 22, 2013 5:30 PM >Subject: Re: Soft Commit and Document Cache > > >On 4/22/2013 4:16 PM, Niran Fajemisi

Re: Soft Commit and Document Cache

2013-04-22 Thread Shawn Heisey
On 4/22/2013 4:16 PM, Niran Fajemisin wrote: A quick (and hopefully simply) question: Does the document cache (or any of the other caches for that matter), get invalidated after a soft commit has been performed? All Solr caches are invalidated when you issue a commit with openSearcher set

Re: Soft Commit and Document Cache

2013-04-22 Thread Mark Miller
Yup - all of the top level caches are. It's a trade off - don't NRT more than you need to. - Mark On Apr 22, 2013, at 6:16 PM, Niran Fajemisin wrote: > Hi all, > > A quick (and hopefully simply) question: Does the document cache (or any of > the other caches

Soft Commit and Document Cache

2013-04-22 Thread Niran Fajemisin
Hi all, A quick (and hopefully simply) question: Does the document cache (or any of the other caches for that matter), get invalidated after a soft commit has been performed? Thanks, Niran

RE: Disabling document cache usage

2013-01-15 Thread Markus Jelsma
Hi, Commenting them out works fine. We don't use documentCaches either as they eat too much and return only so little. Cheers -Original message- > From:Otis Gospodnetic > Sent: Tue 15-Jan-2013 17:29 > To: solr-user@lucene.apache.org > Subject: Re: Disabling doc

Re: Disabling document cache usage

2013-01-15 Thread Otis Gospodnetic
gt; > To: solr-user@lucene.apache.org > > Subject: Disabling document cache usage > > > > Hi, > > > > https://issues.apache.org/jira/browse/SOLR-2429 added the ability to > > disable filter and query caches on a request by request basis. > > > > Is there

RE: Disabling document cache usage

2013-01-15 Thread Markus Jelsma
No, SolrIndexSearcher has no mechanism to do that. The only way is to disable the cache altogether or patch it up :) -Original message- > From:Otis Gospodnetic > Sent: Tue 15-Jan-2013 16:57 > To: solr-user@lucene.apache.org > Subject: Disabling document cache u

Re: document cache

2012-05-15 Thread Erick Erickson
t; I want to know the internal mechanism how document cache works . > > specifically its flushing cycle ... > > i.e does it gets flushed  on every commit /replication . > > regards > > Rajat Rastogi > > > -- > View this message in context: > http://lucene.47

document cache

2012-05-14 Thread shinkanze
hi , I want to know the internal mechanism how document cache works . specifically its flushing cycle ... i.e does it gets flushed on every commit /replication . regards Rajat Rastogi -- View this message in context: http://lucene.472066.n3.nabble.com/document-cache-tp3983796.html Sent

Re: maximum recommended document cache size

2010-05-15 Thread Lance Norskog
quite small, usually under 10k. I > currently have a document cache size of about 15,000, and am warming up 5,000 > with a query after each indexing. Autocommit is set at 30 seconds, and my > caches are warming up easily in just a couple of seconds. I've read of > concerns regarding

maximum recommended document cache size

2010-05-13 Thread Nagelberg, Kallin
I am trying to tune my Solr setup so that the caches are well warmed after the index is updated. My documents are quite small, usually under 10k. I currently have a document cache size of about 15,000, and am warming up 5,000 with a query after each indexing. Autocommit is set at 30 seconds