Is it related to this config?
<!-- If true, IndexReaders will be opened/reopened from the IndexWriter
instead of from the Directory. Hosts in a master/slave setup
should have this set to false while those in a SolrCloud
cluster need to be set to true. Default: true
-->
<!--
<nrtMode>true</nrtMode>
this will be true by default.
On 10/27/15 1:24 PM, Erick Erickson wrote:
Hmm, the tlog. AFAIK, that's because of
"real time get" (Yonik/Mark/Whoever, please
correct if wrong).
This is a feature where when fetching a document as
the result of a search it insures that you get the most
recent version whether it's been committed or not.....
The tlog isn't relevant for searching however.
Best,
Erick
On Tue, Oct 27, 2015 at 8:56 AM, Rallavagu <rallav...@gmail.com> wrote:
On 10/27/15 8:43 AM, Erick Erickson wrote:
bq: So, the updated file(s) on the disk automatically read into memory
as they are Memory mapped?
Yes.
Not quite sure why you care, curiosity or is there something you're
trying to accomplish?
This is out of curiosity. So, I can get better understanding of Solr's
memory usage (heap & mmap).
The contents of the index's segment files are read into virtual memory
by MMapDirectory as needed to satisfy queries. Which is the point of
autowarming BTW.
Ok. But, I have noticed that even "tlog" files are memory mapped (output
from "lsof") in addition to all other files under "data" directory.
commit in the following is either hard commit with openSearcher=true
or soft commit.
Hard commit is setup with openSearcher=false and softCommit is setup for
every 2 min.
Segments that have been created (closed actually) after the last
commit are _not_ read at all until the next searcher is opened via
another commit. Nothing is done with these new segments before the new
searcher is opened which you control with your commit strategy.
I see. Thanks for the insight.
Best,
Erick
On Mon, Oct 26, 2015 at 9:07 PM, Rallavagu <rallav...@gmail.com> wrote:
Erick, Thanks for clarification. I was under impression that
MMapDirectory
is being used for both read/write operations. Now, I see how it is being
used. Essentially, it only reads from MMapDirectory and writes directly
to
disk. So, the updated file(s) on the disk automatically read into memory
as
they are Memory mapped?
On 10/26/15 8:43 PM, Erick Erickson wrote:
You're really looking at this backwards. The MMapDirectory stuff is
for Solr (Lucene, really) _reading_ data from closed segment files.
When indexing, there are internal memory structures that are flushed
to disk on commit, but these have nothing to do with MMapDirectory.
So the question is really moot ;)
Best,
Erick
On Mon, Oct 26, 2015 at 5:47 PM, Rallavagu <rallav...@gmail.com> wrote:
All,
Are memory mapped files (mmap) flushed to disk during "hard commit"? If
yes,
should we disable OS level (Linux for example) memory mapped flush?
I am referring to following for mmap files for Lucene/Solr
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html
Linux level flush
http://www.cyberciti.biz/faq/linux-stop-flushing-of-mmaped-pages-to-disk/
Solr's hard and soft commit
https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
Thanks in advance.
