Your autocommit is unrealistic. You're telling the server to commit every 5 seconds and you're overloading the system. At least that's my guess. Every time you do this, you're causing all the caches to be thrown away, any autowarming to be triggered, etc, etc, etc.
There have been significant improvements in this area on trunk, but you haven't told us what version of Solr you're using, so I'm not sure whether that helps. See NRT (Near Real Time) discussions on the JIRA. Here's what I'd do to test this: Increase the autocommit parameters significantly and see if the problem goes away (something like 10,000 docs and 60,000 milliseconds). And don't forget maxRamBufferSizeMB. That'll point you in the right direction. You really need to consider a master/slave architecture here I think, where the master only indexes and the slaves periodically get the updates. There will be some latency. Best Erick On Tue, Dec 7, 2010 at 7:01 AM, Hamid Vahedi <hvb...@yahoo.com> wrote: > Hi Peter > > Thanks a lot for reply. Actually I need real time indexing and query at the > same > time. > > Here told: > "You can run multiple Solr instances in separate JVMs, with both having > their > solr.xml configured to use the same index folder." > > Now > Q1: I'm using Tomcat now, Could you please tell me how to have separate > JVMs > with Tomcat? > > Q2:What should I set for LockType? > > Thanks in advanced > > > > > ________________________________ > From: Peter Karich <peat...@yahoo.de> > To: solr-user@lucene.apache.org > Sent: Tue, December 7, 2010 2:06:49 PM > Subject: Re: Solr & JVM performance issue after 2 days > > Hi Hamid, > > try to avoid autowarming when indexing (see solrconfig.xml: > caches->autowarm + newSearcher + maxSearcher). > If you need to query and indexing at the same time, > then probably you'll need one read-only core and one for writing with no > autowarming configured. > See: http://wiki.apache.org/solr/NearRealtimeSearchTuning > > Or replicate from the indexing-core to a different core with different > settings. > > Regards, > Peter. > > > > Hi, > > > > I am using multi-core tomcat on 2 servers. 3 language per server. > > > > I am adding documents to solr up to 200 doc/sec. when updating process is > > started, every thing is fine (update performance is max 200 ms/doc. with > about > > 800 MB memory used with minimal cpu usage). > > > > After 15-17 hours it's became so slow (more that 900 sec for update), > used > >heap > > memory is about 15GB, GC time is became more than one hour. > > > > > > I don't know what's wrong with it? Can anyone describe me what's the > problem? > > Is that came from Solr or JVM? > > > > Note: when i stop updating, CPU busy within 15-20 min. and when start > updating > > again i have same issue. but when stop tomcat service and start it again, > all > > thing is OK. > > > > I am using tomcat 6 with 18 GB memory on windows 2008 server x64. Solr > 1.4.1 > > > > thanks in advanced > > Hamid > > > -- > http://jetwick.com twitter search prototype > > > >
