[
https://issues.apache.org/jira/browse/LUCENE-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730590#action_12730590
]
Uwe Schindler commented on LUCENE-1741:
---------------------------------------
Javadocs state (in FileChannel#map): "For most operating systems, mapping a
file into memory is more expensive than reading or writing a few tens of
kilobytes of data via the usual read and write methods. From the standpoint of
performance it is generally only worth mapping relatively large files into
memory."
So it should be as big as possible. A second problem with too many buffers is,
that the MMU/TLB cannot handle too many of them effective.
In my opinion, maybe we could enhance MMapDirectory to work together with
FileSwitchDirectory or something like that, to only use mmap for large files
and all others handled by NIO/Simple. E.g. mapping the segments.gen file into
memory is really wasting resources. So MMapDir would only return the
MMapIndexInput, if the underlying file is > X Bytes (e.g. 8 Megabytes per
default) and fall back to SimpleFSIndexInput otherwise.
bq. In any event, Uwe can you add to the javadocs describing this false OOM
problem and what to do if you hit it?
Will do this tomorrow, will go to bed now.
Here are also some other numbers about this problem:
http://groups.google.com/group/jsr203-interest/browse_thread/thread/66f6a5042f2b0c4a/12228bbd57d1956d
> Make MMapDirectory.MAX_BBUF user configureable to support chunking the index
> files in smaller parts
> ---------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1741
> URL: https://issues.apache.org/jira/browse/LUCENE-1741
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 2.9
>
> Attachments: LUCENE-1741.patch, LUCENE-1741.patch
>
>
> This is a followup for java-user thred:
> http://www.lucidimagination.com/search/document/9ba9137bb5d8cb78/oom_with_2_9#9bf3b5b8f3b1fb9b
> It is easy to implement, just add a setter method for this parameter to
> MMapDir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
