Re: Problem Commiting to Large Index

2011-12-05 Thread Erick Erickson
Some details please. Are you indexing and searching on the same machine? How are you committing? After every add? Via commitWithin? Via autocommit? What version of Solr? Whatenvironment? You might review: http://wiki.apache.org/solr/UsingMailingLists Best Erick On Fri, Dec 2, 2011 at 2:35 PM, M

Problem Commiting to Large Index

2011-12-02 Thread Marty Trenouth
We are creating an index of about 500 - 600M records fairly small documents. We are currently @ 252M+ records and adding documents at a rate of about 2k - 3k per second in multithreaded 1K batches sent from multiple servers. Our commits started bombing out with out of Memory Exceptions with 14G