"unable to create new native thread" That suggests you're running out of threads, not RAM. Possibly you're using a multithreaded collector, and it's pushing you over the top of how many threads your OS lets a single process allocate? Or somehow the thread stack size is set too high?
More here: http://stackoverflow.com/questions/763579/how-many-threads-can-a-java-vm-support Michael Della Bitta ------------------------------------------------ Appinions, Inc. -- Where Influence Isn’t a Game. http://www.appinions.com On Sun, Jul 15, 2012 at 2:45 PM, Nick Koton <nick.ko...@gmail.com> wrote: >>> Solrj multi-threaded client sends several 1,000 docs/sec > >>Can you expand on that? How many threads at once are sending docs to solr? > Is each request a single doc or multiple? > I realize, after the fact, that my solrj client is much like > org.apache.solr.client.solrj.LargeVolumeTestBase. The number of threads is > configurable at run time as are the various commit parameters. Most of the > test have been in the 4-16 threads range. Most of my testing has been with > the single document SolrServer::add(SolrInputDocument doc )method. When I > realized what LargeVolumeTestBase is doing, I converted my program to use > the SolrServer::add(Collection<SolrInputDocument> docs) method with 100 > documents in each add batch. Unfortunately, the out of memory errors still > occur without client side commits. > > If you agree my three approaches to committing are logical, would it be > useful for me to try to reproduce this with "example" schema in a small > cloud configuration using LargeVolumeTestBase or the like? It will take me > a couple days to work it in. Or perhaps this sort of test is already run? > > Best > Nick > > -----Original Message----- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley > Sent: Sunday, July 15, 2012 11:05 AM > To: Nick Koton > Cc: solr-user@lucene.apache.org > Subject: Re: SOLR 4 Alpha Out Of Mem Err > > On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton <nick.ko...@gmail.com> wrote: >>> Do you have the following hard autoCommit in your config (as the >>> stock >> server does)? >>> <autoCommit> >>> <maxTime>15000</maxTime> >>> <openSearcher>false</openSearcher> >>> </autoCommit> >> >> I have tried with and without that setting. When I described running >> with auto commit, that setting is what I mean. > > OK cool. You should be able to run the stock server (i.e. with this > autocommit) and blast in updates all day long - it looks like you have more > than enough memory. If you can't, we need to fix something. You shouldn't > need explicit commits unless you want the docs to be searchable at that > point. > >> Solrj multi-threaded client sends several 1,000 docs/sec > > Can you expand on that? How many threads at once are sending docs to solr? > Is each request a single doc or multiple? > > -Yonik > http://lucidimagination.com >
