Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-04-05 Thread Jan Høydahl
Hi, Just curious, was there any resolution to this? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 8. feb. 2011, at 03.40, Markus Jelsma wrote: Do you have GC logging enabled? Tail -f the log file and you'll see what CMS is telling you. Tuning the occupation

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-04-05 Thread Simon Wistow
On Wed, Apr 06, 2011 at 12:05:57AM +0200, Jan Høydahl said: Just curious, was there any resolution to this? Not really. We tuned the GC pretty aggressively - we use these options -server -Xmx20G -Xms20G -Xss10M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalMode

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-02-07 Thread Markus Jelsma
Heap usage can spike after a commit. Existing caches are still in use and new caches are being generated and/or auto warmed. Can you confirm this is the case? On Friday 28 January 2011 00:34:42 Simon Wistow wrote: On Tue, Jan 25, 2011 at 01:28:16PM +0100, Markus Jelsma said: Are you sure you

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-02-07 Thread Simon Wistow
On Mon, Feb 07, 2011 at 02:06:00PM +0100, Markus Jelsma said: Heap usage can spike after a commit. Existing caches are still in use and new caches are being generated and/or auto warmed. Can you confirm this is the case? We see spikes after replication which I suspect is, as you say, because

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-27 Thread Simon Wistow
On Tue, Jan 25, 2011 at 01:28:16PM +0100, Markus Jelsma said: Are you sure you need CMS incremental mode? It's only adviced when running on a machine with one or two processors. If you have more you should consider disabling the incremental flags. I'll test agin but we added those to get

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-25 Thread Markus Jelsma
Hi, Are you sure you need CMS incremental mode? It's only adviced when running on a machine with one or two processors. If you have more you should consider disabling the incremental flags. Cheers, On Monday 24 January 2011 19:32:38 Simon Wistow wrote: We have two slaves replicating off one

Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-24 Thread Simon Wistow
We have two slaves replicating off one master every 2 minutes. Both using the CMS + ParNew Garbage collector. Specifically -server -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing but periodically they both get into a GC storm and just keel over.

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-24 Thread Em
Hi Simon, I got no experiences with a distributed environment. However, what you are talking about reminds me on another post on the mailing list. Could it be possible that your slaves not finished their replicating until the new replication-process starts? If so, there you got the OOM :).

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-24 Thread Simon Wistow
On Mon, Jan 24, 2011 at 08:00:53PM +0100, Markus Jelsma said: Are you using 3rd-party plugins? No third party plugins - this is actually pretty much stock tomcat6 + solr from Ubuntu. The only difference is that we've adapted the directory layout to fit in with our house style

Re: Possible Memory Leaks / Upgrading to a Later Version of Solr or Lucene

2011-01-24 Thread Simon Wistow
On Mon, Jan 24, 2011 at 10:55:59AM -0800, Em said: Could it be possible that your slaves not finished their replicating until the new replication-process starts? If so, there you got the OOM :). This was one of my thoughts as well - we're currently running a slave which has no queries in it