On 21/09/11 15:54, David Jordan wrote:
Understood. When you say
We're thinking of using it for TDB direct mode
does that mean it is planned for the future, versus implemented today?
Yes - direct mode of TDB is used on 32 bit machines. It currently uses
ByteBuffer.allocate() i.e. in-heap byte buffers. The direct byte buffer
(different use of the word "direct"!) should be faster but we need to
test whether it does have a measurable improvement and does not cause
difficulties in some other way.
The direct byte buffer area is fixed size - hit the limit and the JVM
will die so management of the buffers needs testing.
Andy
-----Original Message-----
From: Andy Seaborne [mailto:[email protected]] On Behalf Of Andy
Seaborne
Sent: Wednesday, September 21, 2011 10:06 AM
To: [email protected]
Subject: Re: benchmarking
On 21/09/11 11:32, Dave Reynolds wrote:
On Tue, 2011-09-20 at 17:22 +0000, David Jordan wrote:
I guess I had the wrong impression of how TDB is implemented. I thought on 64
bit environments that some of the files were memory-mapped directly into the
virtual memory space of the application process.
They are memory mapped but that doesn't mean they are all in memory
unless you have enough memory available, have a cooperative OS, and
have set appropriate options such as -XX:MaxDirectMemorySize depending
on your JVM.
-XX:MaxDirectMemorySize affects a different area of memory allocation to memory
mapped files.
Direct memory is from ByteBuffer.allocateDirect(capacity). It is really plain
old malloc'ed space - it isn't in the heap, isn't a mapped memory segment and
does not move due to GC but is free'd when it's becomes unreachable. I've
assumed that the size is fixed so the process can grow the heap via sbrk(2) and
still have a contiguous heap.
Such a direct memory ByteBuffer is in the same segment of memory as the rest of
the JVM; it is faster to access than a heap ByteBuffer. We're thinking of
using it for TDB direct mode.
Memory mapped files are also direct ByteBuffers but they aren't from allocated
space. There are two kinds of direct memory buffers in Java.
The OS manages memory mapped files via segments of the adress space - the
Java process does not allocated them and they live in memory mapped segments of
your procress.
Andy
