: Re: Nio File Caching & Performance Test
My search process is using MMapDirectory on a read-only index via:
-Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.MMapDirectory
Another indexing process is building the next version of the index in a
different directory. When it'
My search process is using MMapDirectory on a read-only index via:
-Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.MMapDirectory
Another indexing process is building the next version of the index in a
different directory. When it's time to switch to the new index, the search
proces
: Nio File Caching & Performance Test
Hi Robert,
I might be easily wrong, but I beleive I saw something on JIRA (or was it
bugzilla?) a long long time ago, where somebody made MMAP implementation for
really big indexes that works on 32 bit. I guess it is worth checking it.
- Original Mes
AIL PROTECTED]>
To: java-dev@lucene.apache.org; [EMAIL PROTECTED]
Sent: Tuesday, 16 May, 2006 6:10:07 PM
Subject: Re: Nio File Caching & Performance Test
On 5/16/06, Robert Engels <[EMAIL PROTECTED]> wrote:
> SO, I would like to use a memory mapped reader, but I encounter OOM errors
&
On 5/16/06, Robert Engels <[EMAIL PROTECTED]> wrote:
SO, I would like to use a memory mapped reader, but I encounter OOM errors
when mapping large files, due to running out of address space.
Pretty much all x86 servers sold are 64 bit capable now.
Run a 64 bit OS if you can :-)
Has anyone fou
Robert Engels wrote:
SO, I would like to use a memory mapped reader, but I encounter OOM errors
when mapping large files, due to running out of address space.
Has anyone found a solution for this? (A 2 gig index is not all that
large...).
A 64-bit hardware, OS and JVM solves this nicely. On 3
:55 AM
To: java-dev@lucene.apache.org
Subject: Re: Nio File Caching & Performance Test
Hi,
According to my humble tests, there is no significant improvement
either. NIO has buffer creation time costs compared to other Buffered
IOs. However, a testbed would be ideal for benchmarks.
Murat
Hi,
According to my humble tests, there is no significant improvement
either. NIO has buffer creation time costs compared to other Buffered
IOs. However, a testbed would be ideal for benchmarks.
Murat
Doug Cutting wrote:
Robert Engels wrote:
The most important statistic is that the readi
Robert Engels wrote:
The most important statistic is that the reading via the local cache, vs.
going to the OS (where the block is cached) is 3x faster (22344 vs. 68578).
With random reads, when the block may not be in the OS cache, it is 8x
faster (72766 vs. 586391).
[ ... ]
This test only dem
On May 15, 2006, at 5:41 PM, Robert Engels wrote:
As stated in the email, it is 3x faster reading from a Java local
cache,
then having Java go to the OS (where it may or may not be cached).
It avoids
the overhead/context switch into the OS.
I read that in the original mail, but your perfor
To: java-dev@lucene.apache.org
Subject: Re: Nio File Caching & Performance Test
On May 12, 2006, at 3:38 PM, Robert Engels wrote:
> I finally got around to making the NioFSDirectory with caching 1.9
> compliant. I also produced a performance test case.
How does this implementation comp
On May 12, 2006, at 3:38 PM, Robert Engels wrote:
I finally got around to making the NioFSDirectory with caching 1.9
compliant. I also produced a performance test case.
How does this implementation compare to the MMapDirectory?
I've found that the MMapDirectory is far faster than the FSDirecto
I finally got around to making the NioFSDirectory with caching 1.9
compliant. I also produced a performance test case.
Below is the results on my machine:
read random = 586391
read same = 68578
nio read random = 72766
nio max mem = 203292672
nio memory = 102453248
nio hits = 14974713
nio misses =
13 matches
Mail list logo
