Re: Memory Leak in solr 4.8.1

2015-04-09 Thread Toke Eskildsen
On Wed, 2015-04-08 at 14:00 -0700, pras.venkatesh wrote:
 1. 8 nodes, 4 shards(2 nodes per shard)
 2. each node having about 55 GB of Data, in total there is 450 million
 documents in the collection. so the document size is not huge, 

So ~120M docs/shard.

 3. The schema has 42 fields, it gets reloaded every 15 mins with about
 50,000 documents. Now we have primary Key for the index, so when there are
 any duplicates the document gets re-written.
 4. The GC policy is CMS, with heap size min and max = 8 gb and perm size =
 512 mb and RAM on the VM is 24 gb.

Do you have a large and active filter cache? Each entry is 30MB, so it
does not take many entries to fill a 8GB heap. That would match the
description of ever-running GC.

- Toke Eskildsen, State and University Library, Denmark




Re: Memory Leak in solr 4.8.1

2015-04-09 Thread pras.venkatesh
I don't have a filter cache, and have completely disabled filter cache. Since
I am not using filter queries.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Memory-Leak-in-solr-4-8-1-tp4198488p4198716.html
Sent from the Solr - User mailing list archive at Nabble.com.


Memory Leak in solr 4.8.1

2015-04-08 Thread pras.venkatesh
I have a solr cloud instance with 8 nodes, 4 shards and facing memory leak on
the JVMs

here are the details of the instance.


1. 8 nodes, 4 shards(2 nodes per shard)
2. each node having about 55 GB of Data, in total there is 450 million
documents in the collection. so the document size is not huge, 
3. The schema has 42 fields, it gets reloaded every 15 mins with about
50,000 documents. Now we have primary Key for the index, so when there are
any duplicates the document gets re-written.
4. The GC policy is CMS, with heap size min and max = 8 gb and perm size =
512 mb and RAM on the VM is 24 gb.


when users start searching in solr and not always but often the heap keeps
growing and the GC cycles are not clearing up the heap. I see GC running for
almost 100,000 ms with still not clearing up the heap.

Appreciate any advice on this.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Memory-Leak-in-solr-4-8-1-tp4198488.html
Sent from the Solr - User mailing list archive at Nabble.com.