Re: Solr slave Heap space error and index size issue
: 5-6 days after fresh index index size suddenly increased (no optimization in : between) by 150GB and then query takes long time and java heap error comes. : I run optimize in this index Its takes long time and result it increase : index size more more then 200GB and it didn't show about optimize completed. : merge factor is default as given in solr build. did you check your logs? this smells like maybe a failure during commit or optimize (OOM maybe?) that resulted in old files not being cleaned up on disk ... particularly you didn't show about optimize completed. comment. There is a CheckIndex tool that you can use (google for details in lucene-java mailing list) which *should* tell you if there are extra segments (i don't remember the details to be certain). -Hoss
Re: Solr slave Heap space error and index size issue
I don't think the index should *suddenly* increase in size if you are just adding/updating/deleting documents. It is normal that it temporarily increases during optimization. 35GB for 1.5M docs sounds a lot. You either have large fields or you store them or both? Maybe share your schema, show relevant solrconfig settings, list your index directory, share some of the stats from the Solr admin stats page, tell us about your JVM parameters, your RAM, etc. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Vikash Kontia vikash.kon...@gmail.com To: solr-user@lucene.apache.org Sent: Sunday, June 28, 2009 10:46:57 PM Subject: Solr slave Heap space error and index size issue Hi All, I have 1 master and 1 slave machine at deployment. I am using Solr1.4 nightly Build. My fresh index size is 35GB for 1.5 million documents with approx 50 fields each document I taken care of omitNorm and Stored in schema. I have approx 1 update daily and I run commit in every hr. in 5-6 days after fresh index index size suddenly increased (no optimization in between) by 150GB and then query takes long time and java heap error comes. I run optimize in this index Its takes long time and result it increase index size more more then 200GB and it didn't show about optimize completed. merge factor is default as given in solr build. For fixing this issue I have to use re-index in every week almost. I think issue with frequent update on index. Please help me for debug the issue. Or Is I am missing something in confuguration. Thanks Vikash Kontia -- View this message in context: http://www.nabble.com/Solr-slave-Heap-space-error-and-index-size-issue-tp24247690p24247690.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr slave Heap space error and index size issue
Hi All, I have 1 master and 1 slave machine at deployment. I am using Solr1.4 nightly Build. My fresh index size is 35GB for 1.5 million documents with approx 50 fields each document I taken care of omitNorm and Stored in schema. I have approx 1 update daily and I run commit in every hr. in 5-6 days after fresh index index size suddenly increased (no optimization in between) by 150GB and then query takes long time and java heap error comes. I run optimize in this index Its takes long time and result it increase index size more more then 200GB and it didn't show about optimize completed. merge factor is default as given in solr build. For fixing this issue I have to use re-index in every week almost. I think issue with frequent update on index. Please help me for debug the issue. Or Is I am missing something in confuguration. Thanks Vikash Kontia -- View this message in context: http://www.nabble.com/Solr-slave-Heap-space-error-and-index-size-issue-tp24247690p24247690.html Sent from the Solr - User mailing list archive at Nabble.com.
