On the IRC channel, I ran into somebody who was having problems with optimizes on their Solr indexes taking a really long time. When investigating, they found that during the optimize, *reads* were happening on their SSD disk at over 800MB/s, but *writes* were proceeding at only 20 MB/s.
Looking into ConcurrentMergeScheduler, I discovered that it does indeed have a default write throttle of only 20 MB/s. I saw code that would sometimes set the speed to unlimited, but had a hard time figuring out what circumstances will result in the different settings, so based on the user experience, I assume that the 20MB/s throttle must be applied for Solr optimizes. >From what I can see in the code, there's currently no way in solrconfig.xml to configure scheduler options like the maximum write speed. Before I an open an issue to add additional configuration options for the merge scheduler, I thought it might be a good idea to just double-check with everyone here to see whether there's something I missed. This is likely even affecting people who are not using SSD storage. Most modern magnetic disks can easily exceed 20MB/s on both reads and writes. Some RAID arrays can write REALLY fast. Thanks, Shawn
