Hey so after adding those GC options, I was able to incrementally push my max (and min) memory settings up and when we got to max=min=12GB we started looking much better! One slave handles all the load with no OOMs at all! I'm watching the live tomcat log using 'tail'. Next I will convert that field type to (trie) int and reindex. I'll have to start a new index from scratch with a field type change like that so I'll have to delete the old one first on our master... It takes us a couple days to index 15 million products (some are sets so the final index size is only 8 million) so I don't want to do *that* too often as the slaves will be quite stale by the time it's done! :)
Thanks for the help! -----Original Message----- From: Robert Petersen [mailto:rober...@buy.com] Sent: Wednesday, June 30, 2010 9:49 AM To: solr-user@lucene.apache.org Subject: RE: OOM on uninvert field request At and above 4GB we get those GC errors though! Should I switch to something like this? Recommended Options To use i-cms in Java SE 6, use the following command line options: -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode \ -XX:+PrintGCDetails -XX:+PrintGCTimeStamps Caused by: java.lang.RuntimeException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:418) at org.apache.solr.handler.SnapPuller.doCommit(SnapPuller.java:467) at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:319) ... 11 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded -----Original Message----- From: Lance Norskog [mailto:goks...@gmail.com] Sent: Tuesday, June 29, 2010 8:42 PM To: solr-user@lucene.apache.org Subject: Re: OOM on uninvert field request Yes, it is better to use ints for ids than strings. Also, the Trie int fields have a compressed format that may cut the storage needs even more. 8m * 4 = 32mb, times "a few hundred", we'll say 300, is 900mb of IDs. I don't know how these fields are stored, but if they are separate objects we've blown up to several gigs (per-object overheads are surprising). 4G is probably not enough for what you want. If you watch the total memory with 'top' and hit it with different queries, you will get a stronger sense of how much memory your use cases need. On Tue, Jun 29, 2010 at 4:32 PM, Robert Petersen <rober...@buy.com> wrote: > Hello I am trying to find the right max and min settings for Java 1.6 on 20GB > index with 8 million docs, running 1.6_018 JVM with solr 1.4, and am > currently have java set to an even 4GB (export JAVA_OPTS="-Xmx4096m > -Xms4096m") for both min and max which is doing pretty well but occasionally > still getting the below OOM errors. We're running on dual quad core xeons > with 16GB memory installed. I've been getting the below OOM exceptions still > though. > > Is the memsize mentioned in the INFO for the uninvert in bytes? Ie is > memSize=29604020 mean 29MB? We have a few hundred of these fields and they > contain ints used as IDs, and so I guess could they eat all the memory to > uninvert them all after we apply load and enough queries are performed. Does > the field type matter, would int be better than string if these are lookup > ids sparsely populated across the index? BTW these are used for faceting and > filtering only. > > <dynamicField name="*_contentAttributeToken" type="string" > indexed="true" multiValued="true" stored="true" required="false"/> > > Jun 29, 2010 3:54:50 PM org.apache.solr.request.UnInvertedField uninvert > INFO: UnInverted multi-valued field > {field=768_contentAttributeToken,memSize=29604014,tindexSize=50,time=1841,phase1=1824,nTerms=1,bigTerms=0,termInstances=18,uses=0} > Jun 29, 2010 3:54:52 PM org.apache.solr.request.UnInvertedField uninvert > INFO: UnInverted multi-valued field > {field=749_contentAttributeToken,memSize=29604020,tindexSize=56,time=1847,phase1=1829,nTerms=143,bigTerms=0,termInstances=951,uses=0} > Jun 29, 2010 3:54:59 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.OutOfMemoryError: Java heap space > at > org.apache.solr.request.UnInvertedField.uninvert(UnInvertedField.java:191) > at > org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:178) > at > org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:839) > at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:250) > at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:283) > at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:166) > -- Lance Norskog goks...@gmail.com
