You are saying that I should give more memory than 12GB? When I was with 10GB I had the exceptions that I sent. Switching to 12GB made them disappear. So I think I don't have problems with FieldCache any more. What it seems like a problem is 11% on the application time dedicated to GC. Specially when those servers are under really heavy load. I think that's why I sometimes get queries that in one moment are being executed in a few ms and a moment after 20 seconds!
It seems like I should tune my jvm, don't you think so? On Fri, Sep 25, 2009 at 1:01 PM, Fuad Efendi <f...@efendi.ca> wrote: > Give it even more memory. > > Lucene FieldCache is used to store non-tokenized single-value non-boolean > (DocumentId -> FieldValue) pairs, and it is used (in-full!) for instance > for > sorting query results. > > So that if you have 100,000,000 documents with specific heavily distributed > field values (cardinality is high! Size is 100bytes!) you need > 10,000,000,000 bytes for just this instance of FieldCache. > > GC does not play any role. FieldCache won't be GC-collected. > > > -Fuad > http://www.linkedin.com/in/liferay > > > > > -----Original Message----- > > From: Jonathan Ariel [mailto:ionat...@gmail.com] > > Sent: September-25-09 11:37 AM > > To: solr-user@lucene.apache.org; yo...@lucidimagination.com > > Subject: Re: Solr and Garbage Collection > > > > Right, now I'm giving it 12GB of heap memory. > > If I give it less (10GB) it throws the following exception: > > > > Sep 5, 2009 7:18:32 PM org.apache.solr.common.SolrException log > > SEVERE: java.lang.OutOfMemoryError: Java heap space > > at > > > > org.apache.lucene.search.FieldCacheImpl$10.createValue(FieldCacheImpl.java:3 > 61 > > ) > > at > > org.apache.lucene.search.FieldCacheImpl$Cache.get(FieldCacheImpl.java:72) > > at > > > > org.apache.lucene.search.FieldCacheImpl.getStringIndex(FieldCacheImpl.java:3 > 52 > > ) > > at > > > > org.apache.solr.request.SimpleFacets.getFieldCacheCounts(SimpleFacets.java:2 > 67 > > ) > > at > > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:185) > > at > > > > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:2 > 07 > > ) > > at > > > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:104) > > at > > > > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java > :7 > > 0) > > at > > > > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand > le > > r.java:169) > > at > > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. > ja > > va:131) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1204) > > at > > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3 > 03 > > ) > > at > > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: > 23 > > 2) > > at > > > > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler > .j > > ava:1089) > > at > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) > > at > > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > > at > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > > at > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > > at > > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > > at > > > > org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerColl > ec > > tion.java:211) > > at > > > > org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:11 > 4) > > at > > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > > at org.mortbay.jetty.Server.handle(Server.java:285) > > at > > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) > > at > > > > org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java: > 83 > > 5) > > at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) > > at > org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) > > at > org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) > > at > > > > org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:22 > 6) > > at > > > > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:4 > 42 > > ) > > > > On Fri, Sep 25, 2009 at 10:55 AM, Yonik Seeley > > <yo...@lucidimagination.com>wrote: > > > > > On Fri, Sep 25, 2009 at 9:30 AM, Jonathan Ariel <ionat...@gmail.com> > > > wrote: > > > > Hi to all! > > > > Lately my solr servers seem to stop responding once in a while. I'm > using > > > > solr 1.3. > > > > Of course I'm having more traffic on the servers. > > > > So I logged the Garbage Collection activity to check if it's because > of > > > > that. It seems like 11% of the time the application runs, it is > stopped > > > > because of GC. And some times the GC takes up to 10 seconds! > > > > Is is normal? My instances run on a 16GB RAM, Dual Quad Core Intel > Xeon > > > > servers. My index is around 10GB and I'm giving to the instances 10GB > of > > > > RAM. > > > > > > Bigger heaps lead to bigger GC pauses in general. > > > Do you mean that you are giving the JVM a 10GB heap? Were you getting > > > OOM exceptions with a smaller heap? > > > > > > -Yonik > > > http://www.lucidimagination.com > > > > > >
