Otis, This means we have to leave enough space for os cache to cache the whole index . so In case of 16 GB index ., if I am not wrong at least 16 GB memory must not be allocated to any application for os cache to utilize the memory .
>> The operating systems are very good at maintaining this cache. It > > usually better to give the Solr JVM enough memory to run comfortably > > and rely on the OS cache to optimize disk I/O, instead of giving it > > all available ram. how much ram would be good enough for the Solr JVM to run comfortably. thanks, Bharath On Tue, Nov 10, 2009 at 3:59 AM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > Bharat, > > No, you should not give the JVM so much memory. Give it enough to avoid > overly frequent GC, but don't steal memory from the OS cache. > > Otis > -- > Sematext is hiring -- http://sematext.com/about/jobs.html?mls > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR > > > > ----- Original Message ---- > > From: bharath venkatesh <bharathv6.proj...@gmail.com> > > To: solr-user@lucene.apache.org > > Sent: Sun, November 8, 2009 2:15:00 PM > > Subject: Re: tracking solr response time > > > > Thanks Lance for the clear explanation .. are you saying we should give > > solr JVM enough memory so that os cache can optimize disk I/O efficiently > .. > > that means in our case we have 16 GB index so would it be enough to > > allocated solr JVM 20GB memory and rely on the OS cache to optimize disk > I/O > > i .e cache the index in memory ?? > > > > > > below is stats related to cache > > > > > > *name: * queryResultCache *class: * org.apache.solr.search.LRUCache * > > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512, > > autowarmCount=256, > > regenerator=org.apache.solr.search.solrindexsearche...@67e112b3) > > *stats: *lookups > > : 0 > > hits : 0 > > hitratio : 0.00 > > inserts : 8 > > evictions : 0 > > size : 8 > > cumulative_lookups : 15 > > cumulative_hits : 7 > > cumulative_hitratio : 0.46 > > cumulative_inserts : 8 > > cumulative_evictions : 0 > > > > > > *name: * documentCache *class: * org.apache.solr.search.LRUCache * > > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512) > * > > stats: *lookups : 0 > > hits : 0 > > hitratio : 0.00 > > inserts : 0 > > evictions : 0 > > size : 0 > > cumulative_lookups : 744 > > cumulative_hits : 639 > > cumulative_hitratio : 0.85 > > cumulative_inserts : 105 > > cumulative_evictions : 0 > > > > > > *name: * filterCache *class: * org.apache.solr.search.LRUCache > > *version: *1.0 > > *description: * LRU Cache(maxSize=512, initialSize=512, > autowarmCount=256, > > regenerator=org.apache.solr.search.solrindexsearche...@1e3dbf67) > > *stats: *lookups > > : 0 > > hits : 0 > > hitratio : 0.00 > > inserts : 20 > > evictions : 0 > > size : 12 > > cumulative_lookups : 64 > > cumulative_hits : 60 > > cumulative_hitratio : 0.93 > > cumulative_inserts : 12 > > cumulative_evictions : 0 > > > > > > hits and hit ratio are zero for ducment cache , filter cache and query > > cache .. only commulative hits and hitratio has a non zero numbers .. > is > > this how it is supposed to be .. or do we to configure it properly ? > > > > Thanks, > > Bharath > > > > > > > > > > > > On Sat, Nov 7, 2009 at 5:47 AM, Lance Norskog wrote: > > > > > The OS cache is the memory used by the operating system (Linux or > > > Windows) to store a cache of the data stored on the disk. The cache is > > > usually by block numbers and are not correlated to files. Disk blocks > > > that are not used by programs are slowly pruned from the cache. > > > > > > The operating systems are very good at maintaining this cache. It > > > usually better to give the Solr JVM enough memory to run comfortably > > > and rely on the OS cache to optimize disk I/O, instead of giving it > > > all available ram. > > > > > > Solr has its own caches for certain data structures, and there are no > > > solid guidelines for tuning those. The solr/admin/stats.jsp page shows > > > the number of hits & deletes for the caches and most people just > > > reload that over & over. > > > > > > On Fri, Nov 6, 2009 at 3:09 AM, bharath venkatesh > > > wrote: > > > >>I have to state the obvious: you may really want to upgrade to 1.4 > when > > > > it's out > > > > > > > > when would solr 1.4 be released .. is there any beta version > available ? > > > > > > > >>We don't have the details, but a machine with 32 GB RAM and 16 GB > index > > > > should have the whole index cached by >the OS > > > > > > > > do we have to configure solr for the index to be cached by OS in a > > > > optimised way . how does this caching of index in memory happens ? > r > > > > there any docs or link which gives details regarding the same > > > > > > > >>unless something else is consuming the memory or unless something is > > > > constantly throwing data out of the OS >cache (e.g. frequent index > > > > optimization). > > > > > > > > what are the factors which would cause constantly throwing data out > of > > > the > > > > OS cache (we are doing index optimization only once in a day during > > > > midnight ) > > > > > > > > > > > > Thanks, > > > > Bharath > > > > > > > > > > > > > > > > -- > > > Lance Norskog > > > goks...@gmail.com > > > > >
