Hmm, the tlog. AFAIK, that's because of "real time get" (Yonik/Mark/Whoever, please correct if wrong).
This is a feature where when fetching a document as the result of a search it insures that you get the most recent version whether it's been committed or not..... The tlog isn't relevant for searching however. Best, Erick On Tue, Oct 27, 2015 at 8:56 AM, Rallavagu <rallav...@gmail.com> wrote: > > > On 10/27/15 8:43 AM, Erick Erickson wrote: >> >> bq: So, the updated file(s) on the disk automatically read into memory >> as they are Memory mapped? > > Yes. >> >> >> Not quite sure why you care, curiosity or is there something you're >> trying to accomplish? > > This is out of curiosity. So, I can get better understanding of Solr's > memory usage (heap & mmap). > >> >> The contents of the index's segment files are read into virtual memory >> by MMapDirectory as needed to satisfy queries. Which is the point of >> autowarming BTW. > > > Ok. But, I have noticed that even "tlog" files are memory mapped (output > from "lsof") in addition to all other files under "data" directory. > >> >> commit in the following is either hard commit with openSearcher=true >> or soft commit. > > > Hard commit is setup with openSearcher=false and softCommit is setup for > every 2 min. > >> >> Segments that have been created (closed actually) after the last >> commit are _not_ read at all until the next searcher is opened via >> another commit. Nothing is done with these new segments before the new >> searcher is opened which you control with your commit strategy. > > > I see. Thanks for the insight. > >> >> Best, >> Erick >> >> On Mon, Oct 26, 2015 at 9:07 PM, Rallavagu <rallav...@gmail.com> wrote: >>> >>> Erick, Thanks for clarification. I was under impression that >>> MMapDirectory >>> is being used for both read/write operations. Now, I see how it is being >>> used. Essentially, it only reads from MMapDirectory and writes directly >>> to >>> disk. So, the updated file(s) on the disk automatically read into memory >>> as >>> they are Memory mapped? >>> >>> On 10/26/15 8:43 PM, Erick Erickson wrote: >>>> >>>> >>>> You're really looking at this backwards. The MMapDirectory stuff is >>>> for Solr (Lucene, really) _reading_ data from closed segment files. >>>> >>>> When indexing, there are internal memory structures that are flushed >>>> to disk on commit, but these have nothing to do with MMapDirectory. >>>> >>>> So the question is really moot ;) >>>> >>>> Best, >>>> Erick >>>> >>>> On Mon, Oct 26, 2015 at 5:47 PM, Rallavagu <rallav...@gmail.com> wrote: >>>>> >>>>> >>>>> All, >>>>> >>>>> Are memory mapped files (mmap) flushed to disk during "hard commit"? If >>>>> yes, >>>>> should we disable OS level (Linux for example) memory mapped flush? >>>>> >>>>> I am referring to following for mmap files for Lucene/Solr >>>>> >>>>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html >>>>> >>>>> Linux level flush >>>>> >>>>> >>>>> http://www.cyberciti.biz/faq/linux-stop-flushing-of-mmaped-pages-to-disk/ >>>>> >>>>> Solr's hard and soft commit >>>>> >>>>> >>>>> >>>>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >>>>> >>>>> Thanks in advance.
