>> Solrj multi-threaded client sends several 1,000 docs/sec >Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? I realize, after the fact, that my solrj client is much like org.apache.solr.client.solrj.LargeVolumeTestBase. The number of threads is configurable at run time as are the various commit parameters. Most of the test have been in the 4-16 threads range. Most of my testing has been with the single document SolrServer::add(SolrInputDocument doc )method. When I realized what LargeVolumeTestBase is doing, I converted my program to use the SolrServer::add(Collection<SolrInputDocument> docs) method with 100 documents in each add batch. Unfortunately, the out of memory errors still occur without client side commits.
If you agree my three approaches to committing are logical, would it be useful for me to try to reproduce this with "example" schema in a small cloud configuration using LargeVolumeTestBase or the like? It will take me a couple days to work it in. Or perhaps this sort of test is already run? Best Nick -----Original Message----- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: Sunday, July 15, 2012 11:05 AM To: Nick Koton Cc: solr-user@lucene.apache.org Subject: Re: SOLR 4 Alpha Out Of Mem Err On Sun, Jul 15, 2012 at 11:52 AM, Nick Koton <nick.ko...@gmail.com> wrote: >> Do you have the following hard autoCommit in your config (as the >> stock > server does)? >> <autoCommit> >> <maxTime>15000</maxTime> >> <openSearcher>false</openSearcher> >> </autoCommit> > > I have tried with and without that setting. When I described running > with auto commit, that setting is what I mean. OK cool. You should be able to run the stock server (i.e. with this autocommit) and blast in updates all day long - it looks like you have more than enough memory. If you can't, we need to fix something. You shouldn't need explicit commits unless you want the docs to be searchable at that point. > Solrj multi-threaded client sends several 1,000 docs/sec Can you expand on that? How many threads at once are sending docs to solr? Is each request a single doc or multiple? -Yonik http://lucidimagination.com
