Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
Hoping I can get a better response with a more directed question: With facet queries and the fields used, what qualifies as a large number of values? The wiki uses U.S. states as an example, so the number of unique values = 50. More to the point, is there an algorithm that I can use to

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Mike Klaas
On 4/2/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: Hoping I can get a better response with a more directed question: I haven't answered your original question as it seems that general java memory debugging techniques would be the most useful thing here. With facet queries and the fields

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Yonik Seeley
On 4/1/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: Our scenario: 150MB index, 14 documents, read/write servers in place using standard replication. Running Tomcat 5.5.17 on Redhat Enterprise Linux 4. Java configured to start with -Xmx1024m. We encounter java heap out-of-memory issues on

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
On 4/2/07, Yonik Seeley [EMAIL PROTECTED] wrote: On 4/1/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: Our scenario: 150MB index, 14 documents, read/write servers in place using standard replication. Running Tomcat 5.5.17 on Redhat Enterprise Linux 4. Java configured to start with

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
Thanks for the pointers, Mike. I'm trying to determine the math to resolve some strange numbers we're seeing. Here's the top dozen lines from a jmap analysis on a heap dump: SizeCount Class description - 428246064 1792204

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Yonik Seeley
On 4/2/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: We are doing incremental updates, and we optimize quite a bit. mergeFactor presently set to 10. maxDoc count = 144156 numDocs count = 144145 What version of Solr are you using? Another potential OOM (multiple threads generating the same

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
Sorry for the confusion. We do have caching disabled. I was asking the question because I wasn't certain if the configurable cache settings applied throughout, or if the FieldCache in lucene still came in play. The two integer-based facets are single valued per document. The string-based

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
Major version is 1.0. The bits are from a nightly build from early September 2006. We do have plans to upgrade solr soon. On 4/2/07, Yonik Seeley [EMAIL PROTECTED] wrote: On 4/2/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: We are doing incremental updates, and we optimize quite a bit.

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Jeff Rodenburg
Yonik - is this the JIRA entry you're referring to? http://issues.apache.org/jira/browse/LUCENE-754 On 4/2/07, Yonik Seeley [EMAIL PROTECTED] wrote: On 4/2/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: We are doing incremental updates, and we optimize quite a bit. mergeFactor presently set

Re: Troubleshooting java heap out-of-memory

2007-04-02 Thread Yonik Seeley
On 4/2/07, Jeff Rodenburg [EMAIL PROTECTED] wrote: Yonik - is this the JIRA entry you're referring to? http://issues.apache.org/jira/browse/LUCENE-754 Yes. But from the heap dump you provided, that doesn't look like the issue. -Yonik