Be a little careful when extrapolating from disk to memory. Any fields where you've set stored="true" will put data in segment files with extensions .fdt and .fdx, see These are the compressed verbatim copy of the data for stored fields and have very little impact on memory required for searching. I've seen indexes where 75% of the data is stored and indexes where 5% of the data is stored.....
"Summary of File Extensions" here: http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/codecs/lucene40/package-summary.html Best, Erick On Wed, Sep 11, 2013 at 2:57 AM, prabu palanisamy <pr...@serendio.com>wrote: > @Shawn: Correctly I am trying to reduce the index size. I am working on > reindex the solr with some of the features as indexed and not stored > > @Jean: I tried with different caches. It did not show much improvement. > > > On Fri, Sep 6, 2013 at 3:17 PM, Shawn Heisey <s...@elyograg.org> wrote: > > > On 9/6/2013 2:54 AM, prabu palanisamy wrote: > > > I am currently using solr -3.5.0, indexed wikipedia dump (50 gb) with > > > java 1.6. > > > I am searching the solr with text (which is actually twitter tweets) . > > > Currently it takes average time of 210 millisecond for each post, out > of > > > which 200 millisecond is consumed by solr server (QTime). I used the > > > jconsole monitor tool. > > > > If the size of all your Solr indexes on disk is in the 50GB range of > > your wikipedia dump, then for ideal performance, you'll want to have > > 50GB of free memory so the OS can cache your index. You might be able > > to get by with 25-30GB of free memory, depending on your index > composition. > > > > Note that this is memory over and above what you allocate to the Solr > > JVM, and memory used by other processes on the machine. If you do have > > other services on the same machine, note that those programs might ALSO > > require OS disk cache RAM. > > > > http://wiki.apache.org/solr/SolrPerformanceProblems#OS_Disk_Cache > > > > Thanks, > > Shawn > > > > >
