some values in the field are up to a 1M as well
On Wed, Jun 5, 2013 at 7:27 PM, Raheel Hasan <raheelhasan....@gmail.com>wrote: > ok thanks for the reply.... The field having values like 60kb each.... > > Furthermore, I have realized that the issue is with MySQL as its not > processing this table when a "where" is applied.... > > Secondly, I have turned this field to "*stored=false*" and now the "* > select/*" is fast working again.... > > > > On Wed, Jun 5, 2013 at 6:56 PM, Shawn Heisey <s...@elyograg.org> wrote: > >> On 6/5/2013 3:08 AM, Raheel Hasan wrote: >> > Hi, >> > >> > I am trying to index a heavy dataset with 1 particular field really too >> > heavy... >> > >> > However, As I start, I get Memory warning and rollback >> (OutOfMemoryError). >> > So, I have learned that we can use -Xmx1024m option with java command to >> > start the solr and allocate more memory to the heap. >> > >> > My question is, that since this could also become insufficient later, >> so it >> > the issue related to cacheing? >> > >> > here is my cache block in solrconfig: >> > >> > <filterCache class="solr.FastLRUCache" >> > size="512" >> > initialSize="512" >> > autowarmCount="0"/> >> > >> > <queryResultCache class="solr.LRUCache" >> > size="512" >> > initialSize="512" >> > autowarmCount="0"/> >> > >> > <documentCache class="solr.LRUCache" >> > size="512" >> > initialSize="512" >> > autowarmCount="0"/> >> > >> > I am thinking like maybe I need to turn of the cache for >> "documentClass". >> > Anyone got a better idea? Or perhaps there is another issue here? >> >> Exactly how big is this field? Do you need this giant field returned >> with your results, or is it just there for searching? >> >> Caches of size 512, especially with autowarm disabled, are probably not >> a major cause for concern, unless the big field is big enough so that >> 512 of them is really really huge. If that's the case, I would reduce >> the size of your documentCache, not turn it off. >> >> The value of ramBufferSizeMB elsewhere in your config is more likely to >> affect how much RAM gets used during indexing. The default for this >> field as of Solr 4.1.0 is 100. Most people can reduce this value. >> >> I'm writing a reply to another thread where you are participating, with >> info that will likely be useful for you too. Look for that. >> >> Thanks, >> Shawn >> >> > > > -- > Regards, > Raheel Hasan > -- Regards, Raheel Hasan
