Erick,
We currently have ramBufferSizeMB at 1024M. For this indexing activity, the
cluster is "offline" thus no queries coming in so not worried about any
user impact or delays should Solr terminate and need to replay. The
thinking was that increasing these values (ramBuffer, commit times, etc)
Every time your ramBufferSizeMB limit is exceeded, a segment is
created that's eventually merged. In terms of _throughput_, making
this large usually doesn't help much after about 100M (the default).
It'd be interesting to see if it changes your I/O activity though.
BTW, I'd hard commit
We are using Solr 5.4 to index TBs of documents in a bulk fashion to get
the cluster up and running. Indexing is over HTTP round robin as directed
by zookeeper.
Each of the 13 nodes is receiving about 6-8 MB/s on the NIC but solr is
writing around 20 to 25 thousand times per second (4k block
