The Solr caches are thrown away on each hard commit. The document cache could be conserved across commits. Documents in segments that still exist would be saved. Documents in segments that are removed would be thrown away.
Perhaps the document cache should be pushed down into Lucene, to handle this well? ----- Original Message ----- | From: "Mikhail Khludnev" <mkhlud...@griddynamics.com> | To: solr-user@lucene.apache.org | Sent: Thursday, September 20, 2012 11:19:58 AM | Subject: Re: MMapDirectory | | My limited understanding, confirmed by profiler though, is that doing | mmap | IO cost you a copying bytes from mmaped virtual memory into heap VM. | Just | look into java.nio.DirectByteBuffer.get(byte[], int, int) . It | happens | several times to me - we saw hotspot in profiler on mmaped IO (yep, | just | in copying bytes!!), cache them in heap and we had hotspot moved | after that. | Good sample of heap cache for mmaped data is terminfos cache with | configurable interval. | Overal question is absolutely worth to think about. | | On Thu, Sep 20, 2012 at 9:39 PM, Erick Erickson | <erickerick...@gmail.com>wrote: | | > So I just had a curiosity question pop up and wanted to check it | > out. | > Solr has the documentCache, designed to hold stored fields while | > various parts of a requestHandler do their tricks, keeping the | > stored | > content from having to be re-fetched from disk. When using | > MMapDirectory, is this even something to worry about? | > | > It seems like documentCache wouldn't be all that useful, but then I | > don't have a deep understanding here. I can imagine scenarios where | > it | > would be more efficient i.e. it's targeted to the documents | > actually | > being accessed rather than random places on disk in the fdt/fdx | > files.... | > | > Thanks, | > Erick | > | | | | -- | Sincerely yours | Mikhail Khludnev | Tech Lead | Grid Dynamics | | <http://www.griddynamics.com> | <mkhlud...@griddynamics.com> |
