RE: SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-30 Thread Markus Jelsma
to partition my own input up to 50k documents and everything is running fine. Markus -Original message- > From:Erick Erickson <erickerick...@gmail.com> > Sent: Thursday 29th October 2015 22:28 > To: solr-user <solr-user@lucene.apache.org> > Subject: Re: SolrJ st

Re: SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-30 Thread Susheel Kumar
Sent: Thursday 29th October 2015 22:28 > > To: solr-user <solr-user@lucene.apache.org> > > Subject: Re: SolrJ stalls/hangs on client.add(); and doesn't return > > > > You're sending 100K docs in a single packet? It's vaguely possible that > you're > > getting a ti

Re: SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-30 Thread Yonik Seeley
On Thu, Oct 29, 2015 at 5:28 PM, Erick Erickson wrote: > Try making batches of 1,000 docs and sending them through instead. The other thing about ConcurrentUpdateSolrClient is that it will create batches itself while streaming. For example, if you call add a number of

Re: SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-30 Thread Erick Erickson
Glad you can solve it one way or the other. I do wonder, though what's really going on, the fact that your original case just hung is kind of disturbing. 50K is still a lot, and Yonik's comment is well taken. I did some benchmarking (not ConcurrentUpdateSolrServer, HttpSolrClient as I remember)

Re: SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-29 Thread Erick Erickson
You're sending 100K docs in a single packet? It's vaguely possible that you're getting a timeout although that doesn't square with no docs being indexed... Hmmm, to check you could do a manual commit. Or watch the Solr log to see if update requests ever go there. Or you're running out of memory

SolrJ stalls/hangs on client.add(); and doesn't return

2015-10-29 Thread Markus Jelsma
Hello - we have some processes periodically sending documents to 5.3.0 in local mode using ConcurrentUpdateSolrClient 5.3.0, it has queueSize 10 and threadCount 4, just chosen arbitrarily having no idea what is right. Usually its a few thousand up to some tens of thousands of rather small