I may have a bit of good news. The ulimit of open files was set to 4096. I
just chose a random high limit (10) and it seems to be working better
now. I still have more testing to do though, but the initial results are
hopeful.
On Wed, Aug 14, 2013 at 4:22 PM, Kevin Osborn wrote:
> Actually
Actually, I thought it worked last night, but that may have just been a
fluke. Today, it is not working.
This is what I have done.
I have turned off autoCommit and softAutoCommit. My updates are not sending
any softCommit messages.
I am sending over data in chunks of 500 records.
At the end of
Thanks so much for your help and for the explanations. Eventually, we will
be doing several batches in parallel. But at least now I know where to look
and can do some testing on various scenarios.
Since we may be doing a lot of heavy uploading (while still doing a lot of
queries), having a autoCom
Kevin,
I wouldn't have considered using softCommits at all based on what I understand
from your use case. You appear to be loading in large batches, and softCommits
are better aligned to NRT search where there is a steady stream of smaller
updates that need to be available immediately.
As E
right, SOLR-5081 is possible but somewhat unlikely
given the fact that you actually don't have very many
nodes in your cluster.
soft commits aren't relevant to the tlog, but here's
the thing. Your tlogs may get replayed
when you restart solr. If they're large, this may take
a long time. When you s
Interesting, that did work. Do you or anyone else have any ideas or what I
should look at? While soft commit is not a requirement in my project, my
understanding is that it should help performance. On the same index, I will
be doing both a large number of queries as well as updates.
If I have to d
While I don't have a past history of this issue to use as reference, if I were
in your shoes I would consider trying your updates with softCommit disabled.
My suspicion is you're experiencing some issue with the transaction logging and
how it's managed when your hard commit occurs.
If you can
I am using Solr Cloud 4.4. It is pretty much a base configuration. We have
2 servers and 3 collections. Collection1 is 1 shard and the Collection2 and
Collection3 both have 2 shards. Both servers are identical.
So, here is my process, I do a lot of queries on Collection1 and
Collection2. I then do
