On 1/8/2015 11:05 PM, Boogie Shafer wrote: > In the abstract, it sounds like you are seeing the difference between tuning > for latency vs tuning for throughput > > My hunch would be you are seeing more (albeit individually quicker) GC events > with your new settings during the rebuild > > I imagine that in most cases a solr rebuild is relatively rare compared to > the amount of times where a lower latency request is desired. If the rebuild > times are problematic for you, use tunings specific to that workload during > the times you need it and then switch back to your low latency settings > after. If you are doing that you can probably run with a bigger heap > temporarily during the rebuild as you aren't likely to be fielding queries > and don't benefit from having a larger OS cache available
Full rebuilds are indeed relatively rare. Avoiding long pauses and keeping query latency low are usually a lot more important than how quickly the index rebuilds. Quick rebuilds are nice, but not strictly necessary. We do incremental updates that start at the top of every minute, unless an update is already running. Exactly how long those updates take is of little importance, unless that time is easier to measure in minutes rather than seconds. If I ever find myself in a situation where completing a rebuild as fast as possible becomes extremely important, does anyone have suggestions for GC tuning options that will optimize for throughput? Thanks, Shawn
