On Tue, Oct 23, 2012 at 3:52 AM, Shawn Heisey <s...@elyograg.org> wrote: > As soon as you make any change at all to an index, it's no longer > "optimized." Delete one document, add one document, anything. Most of the > time you will not see a performance increase from optimizing an index that > consists of one large segment and a bunch of very tiny segments or deleted > documents. >
I've since realized that by experimentation. I've probably saved quite a few minutes of reading time by investing hours of experiment time! > How big is your index, and did you run this right after a reboot? If you > did, then the cache will be fairly empty, and Solr has only read enough from > the index files to open the searcher.The number is probably too small to > show up on a gigabyte scale. As you issue queries, the cached amount will > get bigger. If your index is small enough to fit in the 14GB of free RAM > that you have, you can manually populate the disk cache by going to your > index directory and doing 'cat * > /dev/null' from the commandline or a > script. The first time you do it, it may go slowly, but if you immediately > do it again, it will complete VERY fast -- the data will all be in RAM. > The cat trick to get the files in RAM is great. I would not have thought that would work for binary files. The index is small, much less than the available RAM, for the time being. Therefore, there was nothing to fill it with I now understand. Both 'free' outputs were after the system had been running for some time. > The 'free -m' command in your first email shows cache usage of 1243MB, which > suggests that maybe your index is considerably smaller than your available > RAM. Having loads of free RAM is a good thing for just about any workload, > but especially for Solr.Try running the free command without the -g so you > can see those numbers in kilobytes. > > I have seen a tendency towards creating huge caches in Solr because people > have lots of memory. It's important to realize that the OS is far better at > the overall job of caching the index files than Solr itself is. Solr caches > are meant to cache result sets from queries and filters, not large sections > of the actual index contents. Make the caches big enough that you see some > benefit, but not big enough to suck up all your RAM. > I see, thanks. > If you are having warm time problems, make the autowarm counts low. I have > run into problems with warming on my filter cache, because we have filters > that are extremely hairy and slow to run. I had to reduce my autowarm count > on the filter cache to FOUR, with a cache size of 512. When it is 8 or > higher, it can take over a minute to autowarm. > I will have to experiment with the warning. Thank you for the tips. -- Dotan Cohen http://gibberish.co.il http://what-is-what.com
