yes, I need average query time less than 10 ms. The faster the better.
I have enough memory for lucene because I know there are not too much
data. there are not many modifications. every day there are about
hundreds of document update. if indexes are not in physical memory,
then IO operations will cost a few ms.
btw, the full gc may also add uncertainty, So I need optimize it as
much as possible.
On Mon, Jun 11, 2012 at 5:27 PM, Michael Kuhlmann <k...@solarier.de> wrote:
> You cannot guarantee this when you're running out of RAM. You'd have a
> problem then anyway.
>
> Why are you caring that much? Did you yet have performance issues? 1GB
> should load really fast, and both auto warming and OS cache should help a
> lot as well. With such an index, you usually don't need to fine tune
> performance that much.
>
> Did you think about using a SSD? Since you want to persist your index,
> you'll need to live with disk IO anyway.
>
> Greetings,
> Kuli
>
> Am 11.06.2012 11:20, schrieb Li Li:
>
>> I am sorry. I make a mistake. even use RAMDirectory, I can not
>> guarantee they are not swapped out.
>>
>> On Mon, Jun 11, 2012 at 4:45 PM, Michael Kuhlmann<k...@solarier.de>
>> wrote:
>>>
>>> Set the swapiness to 0 to avoid memory pages being swapped to disk too
>>> early.
>>>
>>> http://en.wikipedia.org/wiki/Swappiness
>>>
>>> -Kuli
>>>
>>> Am 11.06.2012 10:38, schrieb Li Li:
>>>
>>>> I have roughly read the codes of RAMDirectory. it use a list of 1024
>>>> byte arrays and many overheads.
>>>> But as far as I know, using MMapDirectory, I can't prevent the page
>>>> faults. OS will swap less frequent pages out. Even if I allocate
>>>> enough memory for JVM, I can guarantee all the files in the directory
>>>> are in memory. am I understanding right? if it is, then some less
>>>> frequent queries will be slow. How can I let them always in memory?
>>>>
>>>> On Fri, Jun 8, 2012 at 5:53 PM, Lance Norskog<goks...@gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> Yes, use MMapDirectory. It is faster and uses memory more efficiently
>>>>> than RAMDirectory. This sounds wrong, but it is true. With
>>>>> RAMDirectory, Java has to work harder doing garbage collection.
>>>>>
>>>>> On Fri, Jun 8, 2012 at 1:30 AM, Li Li<fancye...@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>> hi all
>>>>>> I want to use lucene 3.6 providing searching service. my data is
>>>>>> not very large, raw data is less that 1GB and I want to use load all
>>>>>> indexes into memory. also I need save all indexes into disk
>>>>>> persistently.
>>>>>> I originally want to use RAMDirectory. But when I read its javadoc.
>>>>>>
>>>>>> Warning: This class is not intended to work with huge indexes.
>>>>>> Everything beyond several hundred megabytes
>>>>>> will waste resources (GC cycles), because it uses an internal buffer
>>>>>> size of 1024 bytes, producing millions of byte
>>>>>> [1024] arrays. This class is optimized for small memory-resident
>>>>>> indexes. It also has bad concurrency on
>>>>>> multithreaded environments.
>>>>>> It is recommended to materialize large indexes on disk and use
>>>>>> MMapDirectory, which is a high-performance
>>>>>> directory implementation working directly on the file system cache of
>>>>>> the operating system, so copying data to
>>>>>> Java heap space is not useful.
>>>>>>
>>>>>> should I use MMapDirectory? it seems another contrib instantiated.
>>>>>> anyone test it with RAMDirectory?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lance Norskog
>>>>> goks...@gmail.com
>>>
>>>
>>>
>
