On Mon, Jul 28, 2008 at 4:53 PM, Britske <[EMAIL PROTECTED]> wrote: > Each query requests at most 20 stored fields. Why doesn't help > lazyfieldloading in this situation?
It's the disk seek that kills you... loading 1 byte or 1000 bytes per document would be about the same speed. > Also, if I understand correctly, for optimal performance I need to have at > least enough RAM to put the entire Index size in OS cache (thus RAM) + the > amount of RAM that SOLR / Lucene consumes directly through the JVM? The normal usage is to just retrieve the stored fields for the top 10 (or a window of 10 or 20) documents. Under this scenario, the slowdown from not having all of the stored fields cached is usually acceptable. Faster disks (seek time) can also help. > Luckily most of the normal queries return 10 documents each, which results > in a discrepancy between total elapsed time and qTIme of about 15-30 ms. > Doesn't this seem strange, since to me it would seem logical that the > discrepancy would be at least 1/10th of fetching 100 documents. Yes, in general 1/10th the cost is what one would expect on average. But some of the docs you are trying to retrieve *will* be in cache, so it's hard to control this test. You could try forcing the index out of memory by "cat"ing some other big files multiple times and then re-trying.... or do a reboot to be sure. -Yonik
