> One of my coworker is reminding me that major compactions do have the > well know side effect of slowing down a busy system.
I think where this is going is the system IO cache problem could be solved with something like DirectIOLinuxDirectory: https://issues.apache.org/jira/browse/LUCENE-2500 Of course the issue would be integrating DIOLD or it's underlying native implementation into HDFS somehow?
