RE: entire farm fails at the same time with OOM issues

2010-12-09 Thread Chris Hostetter
I'm not sure if you resolved this issue, but... : It has typically been when query traffic was lowest! We are at 12 GB ...that doesn't mean it couldn't have been query load related. it's possible that some unusual query (ie: trying to sort on many fields at the same time?) could have

RE: entire farm fails at the same time with OOM issues

2010-12-01 Thread Robert Petersen
-Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Tuesday, November 30, 2010 6:44 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen rober...@buy.com

Re: entire farm fails at the same time with OOM issues

2010-12-01 Thread Ken Krugler
To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues Hi Robert, I'd recommend launching Tomcat with -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=path to where you want the file to go, so then you have something to look at versus

Re: entire farm fails at the same time with OOM issues

2010-12-01 Thread Peter Karich
also try to minimize maxWarming searchers to 1(?) or 2. And decrease cache usage (especially autowarming) if possible at all. But again: only if it doesn't affect performance ... Regards, Peter. On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersenrober...@buy.com wrote: My question is this.

RE: entire farm fails at the same time with OOM issues

2010-12-01 Thread Robert Petersen
Good idea. Our farm is behind Akamai so that should be ok to do. -Original Message- From: Peter Karich [mailto:peat...@yahoo.de] Sent: Wednesday, December 01, 2010 12:21 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues also try

entire farm fails at the same time with OOM issues

2010-11-30 Thread Robert Petersen
Greetings, we are running one master and four slaves of our multicore solr setup. We just served searches for our catalog of 8 million products with this farm during black Friday and cyber Monday, our busiest days of the year, and the servers did not break a sweat! Index size is about 28GB.

Re: entire farm fails at the same time with OOM issues

2010-11-30 Thread Ken Krugler
Hi Robert, I'd recommend launching Tomcat with -XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath=path to where you want the file to go, so then you have something to look at versus a Gedankenexperiment :) -- Ken On Nov 30, 2010, at 3:04pm, Robert Petersen wrote: Greetings, we are

RE: entire farm fails at the same time with OOM issues

2010-11-30 Thread Robert Petersen
that add up to anything right? -Original Message- From: Ken Krugler [mailto:kkrugler_li...@transpac.com] Sent: Tuesday, November 30, 2010 3:12 PM To: solr-user@lucene.apache.org Subject: Re: entire farm fails at the same time with OOM issues Hi Robert, I'd recommend launching Tomcat with -XX

Re: entire farm fails at the same time with OOM issues

2010-11-30 Thread Yonik Seeley
On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen rober...@buy.com wrote: My question is this.  Why in the world would all of my slaves, after running fine for some days, suddenly all at the exact same minute experience OOM heap errors and go dead? If there is no change in query traffic when