Guys, thanks for GC discussion; but the root of a problem is FieldCache internals.
Not enough RAM for FieldCache will cause unpredictable OOM, and it does not depend on GC. How much RAM FieldCache needs in case of 20000 different values for a Field, 200 bytes each (Unicode), and 100M documents? What if we have 100 such non-tokenized fields in a schema? SOLR has an option to warm up caches on startup which might help troubleshooting. JRockit JVM has 'realtime' version if you are interested in predictable GC (without delaying 'transaction')... GC will frequently happen even if RAM is more than enough: in case if it is heavily sparse... so that have even more RAM! -----Original Message----- From: Fuad Efendi [mailto:f...@efendi.ca] Sent: September-25-09 12:17 PM To: solr-user@lucene.apache.org Subject: RE: Solr and Garbage Collection > You are saying that I should give more memory than 12GB? Yes. Look at this: > > SEVERE: java.lang.OutOfMemoryError: Java heap space > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:3 > 61 > > ) It can't find few (!!!) contiguous bytes for .createValue(...) It can't add (Field Value, Document ID) pair to an array. GC tuning won't help in this specific case... May be SOLR/Lucene core developers may WARM FieldCache at IndexReader opening time, in the future... to have early OOM... Avoiding faceting (and sorting) on such field will only postpone OOM to unpredictable date/time... -Fuad http://www.linkedin.com/in/liferay
