[ https://issues.apache.org/jira/browse/SOLR-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harish Verma updated SOLR-4356: ------------------------------- Comment: was deleted (was: screenshots of Memory Dump) > SOLR 4.1 Out Of Memory error After commit of a few thousand Solr Docs > --------------------------------------------------------------------- > > Key: SOLR-4356 > URL: https://issues.apache.org/jira/browse/SOLR-4356 > Project: Solr > Issue Type: Improvement > Components: clients - java, Schema and Analysis, Tests > Affects Versions: 4.1 > Environment: OS = Ubuntu 12.04 > Sun JAVA 7 > Max Java Heap Space = 2GB > Apache Tomcat 7 > Hardware = {Intel core i3, 2GB RAM} > Average no of fields in a Solr Doc = 100 > Reporter: Harish Verma > Labels: performance, test > Fix For: 4.1.1 > > Attachments: memorydump1.png, memorydump2.png > > Original Estimate: 168h > Remaining Estimate: 168h > > we are testing solr 4.1 running inside tomcat 7 and java 7 with following > options > JAVA_OPTS="-Xms256m -Xmx2048m -XX:MaxPermSize=1024m -XX:+UseConcMarkSweepGC > -XX:+CMSIncrementalMode -XX:+ParallelRefProcEnabled > -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/ubuntu/OOM_HeapDump" > our source code looks like following: > /**** START *****/ > int noOfSolrDocumentsInBatch = 0; > for(int i=0 ; i<5000 ; i++) { > SolrInputDocument solrInputDocument = getNextSolrInputDocument(); > server.add(solrInputDocument); > noOfSolrDocumentsInBatch += 1; > if(noOfSolrDocumentsInBatch == 10) { > server.commit(); > noOfSolrDocumentsInBatch = 0; > } > } > /**** END *****/ > the method "getNextSolrInputDocument()" generates a solr document with 100 > fields (average). Around 50 of the fields are of "text_general" type. > Some of the "test_general" fields consist of approx 1000 words rest consists > of few words. Ouf of total fields there are around 35-40 multivalued fields > (not of type "text_general"). > We are indexing all the fields but storing only 8 fields. Out of these 8 > fields two are string type, five are long and one is boolean. So our index > size is only 394 MB. But the RAM occupied at time of OOM is around 2.5 GB. > Why the memory is so high even though the index size is small? > What is being stored in the memory? Our understanding is that after every > commit documents are flushed to the disk.So nothing should remain in RAM > after commit. > We are using the following settings: > server.commit() set waitForSearcher=true and waitForFlush=true > solrConfig.xml has following properties set: > directoryFactory = solr.MMapDirectoryFactory > maxWarmingSearchers = 1 > text_general data type is being used as supplied in the schema.xml with the > solr setup. > maxIndexingThreads = 8(default) > <autoCommit><maxTime>15000</maxTime><openSearcher>false</openSearcher></autoCommit> > We get Java heap Out Of Memory Error after commiting around 3990 solr > documents.Some of the snapshots of memory dump from profiler are attached. > can somebody please suggest what should we do to minimize/optimize the memory > consumption in our case with the reasons? > also suggest what should be optimal values and reason for following > parameters of solrConfig.xml > useColdSearcher - true/false? > maxwarmingsearchers- number > spellcheck-on/off? > omitNorms=true/false? > omitTermFreqAndPositions? > mergefactor? we are using default value 10 > java garbage collection tuning parameters ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org