Hmm, merging can't read at 800 MB/sec and only write at 20 MB/sec for very long ... unless there is a huge percentage of deletes.
Also, by default CMS doesn't throttle forced merges (see CMS.get/setForceMergeMBPerSec). Maybe capture IndexWriter.setInfoStream output? Mike McCandless http://blog.mikemccandless.com On Wed, Jun 15, 2016 at 9:12 PM, Shawn Heisey <apa...@elyograg.org> wrote: > On the IRC channel, I ran into somebody who was having problems with > optimizes on their Solr indexes taking a really long time. When > investigating, they found that during the optimize, *reads* were > happening on their SSD disk at over 800MB/s, but *writes* were > proceeding at only 20 MB/s. > > Looking into ConcurrentMergeScheduler, I discovered that it does indeed > have a default write throttle of only 20 MB/s. I saw code that would > sometimes set the speed to unlimited, but had a hard time figuring out > what circumstances will result in the different settings, so based on > the user experience, I assume that the 20MB/s throttle must be applied > for Solr optimizes. > > From what I can see in the code, there's currently no way in > solrconfig.xml to configure scheduler options like the maximum write > speed. Before I an open an issue to add additional configuration > options for the merge scheduler, I thought it might be a good idea to > just double-check with everyone here to see whether there's something I > missed. > > This is likely even affecting people who are not using SSD storage. > Most modern magnetic disks can easily exceed 20MB/s on both reads and > writes. Some RAID arrays can write REALLY fast. > > Thanks, > Shawn > >
