Would you mind sharing details on 1. the Solr Cloud setup, how may nodes do you have at your disposal and how many shards do you have setup ? 2. The indexing technology, what are you using? Core java/.net threads ? Or a system like spark ? 3. Where do you see the exceptions? The indexer process logs or Solr cloud logs?
> On Jun 25, 2018, at 11:06 PM, solrnoobie <raphaeldegra...@yahoo.com> wrote: > > We are currently having problems in out current production setup in solr. > > What we currently have is something like this: > > - Solr 6.6.3 (cloud mode) > - 10 threads for indexing > - 900k total documents > - 500 documents per batch > > > So in each thread, the process will call a stored procedure with a lot of > resultsets (1 main table and 8 sub tables) and after the db call, the > application will assemble the documents based on the resultsets and then it > will send it to solr for indexing. > > We are having errors such as heap space error in our indexing so we decided > to lower the batch size to 50. The problem with this is that sometimes it > really does not help since 1 document can contain 1000 child documents and > it will still have the heap errors and indexing is generally slow everytime. > > So my question would be what approach should we have to resolve this kind of > problem (will queue based indexing help? what are your indexing methods in > your respective production environments?)? > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
