: Right, right, I meant to say that I know about that blog post.... but my Q : is: : If mlockall is such a good thing, why not have it in Lucene or Solr? Or : maybe mlockall is not such a good or simple thing?
Beyond writting that agent jar, I never put much effort into thinking about integrating it directly into Solr for a variety of reasons alluded to in the blog... http://searchhub.org/2013/05/21/mlockall-for-all/ "...it seemed like an unnecessary complication and poor substitute for disabling swap on your production servers." "There are a few important caveats to using mlockall-agent.jar, mostly because there are some important caveats to using mlockall in general (pay attention to your ulimits) and some specific caveats to using it in java (make sure your min heap = your max heap)" ...and in the FAQ at the bottom of the README... https://github.com/LucidWorks/mlockall-agent/blob/master/README.txt#L111 In particular, note the FAQ about MCL_CURRENT and the associated comments (straight from CASSANDRA-1214)... https://github.com/LucidWorks/mlockall-agent/blob/master/src/MLockAgent.java#L29 ...my understanding is that mlockall is really only a good idea *before* any data is mmapped so you don't try to lock the stuff the OS is already mmapping -- doing that from within Solr's source (after the servlet container has already started) would be risky. Assuing it was worth the techinical effort, it would convolute the build system a bit, and we'd have to make choices (similar to what Cassandra did) of how to deal with it on systems where it's not supported, or in instances where the call fails (because the ulimit isn't set high ehough) ... treating at as an "optional" performance optimization isn't neccessarily the best approach if people with platforms that do work are counting on it working. Baring any evidence from someone whose looked into it more then me, my current suggestion would be... * don't overload your machines and just disable swap when using solr -- don't worry about mlockall * if you can't disable swap, and you want to run solr with mlockall because your machines are overloaded, use mlockall-agent Once Solr evolves to the point where we don't run in a servlet container, and have our own public static void main, and ship platform native startup scripts where we can handle things like forcing heap min=max, and have overridable startup config options for things like "force_mlockall=true" then it might be worth revisiting. -Hoss http://www.lucidworks.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
