More test data...
Running the previously mentioned measurement (allocating direct buffers
randomly sized between 256KB and 1MB for 60 seconds) with a single
allocating tread that presents allocation pressure which is still
acceptable for original code (two threads are already too much an lead
to OOME), the results, using 4-core i7 CPU and
-XX:MaxDirectMemorySize=128m are:
- 482403 allocations satisfied without helping ReferenceHandler
- 75 allocations satisfied while helping ReferenceHandler but before
System.gc()
- 2373 allocations satisfied while helping ReferenceHandler, after
System.gc() but before any sleeps
- no sleeps
avg. allocation: 0.12 ms/op (original code: 0.6 ms/op)
This test may be regarded as an edge test. No current real-world
application exhibits substantialy higher allocation pressure. If it did,
it would throw OOME, so it would be unusable. Above numbers show that
majority of allocations (99.5%) happen without helping ReferenceHandler
thread. Only every 200th disturbs the ReferenceHandler thread, and it
does exactly what ReferenceHandler does asynchronously and because the
ReferenceHandler thread is asleep. So it actually increases the
promptness of Reference enqueue-ing - not something that could hurt. We
can reasonably expect that in current real-world applications helping
ReferenceHandler would happen even less frequently, and could not
negatively impact application behaviour. What applications will see is
up to 5x improvement in throughput of allocation (a result of using
atomic operations for reservation and less sleeping).
For comparison, here are also the results for 2 allocating threads
(higher allocation pressure than any current real-world application):
- 734916 allocations satisfied without helping ReferenceHandler
- 3112 allocations satisfied while helping ReferenceHandler but before
System.gc()
- 3817 allocations satisfied while helping ReferenceHandler, after
System.gc() but before any sleeps
- no sleeps
avg. allocation: 0.16 ms/op (per thread)
This is still 99.1% allocations without disturbing the peaceful flow of
ReferenceHandler thread.
Regards, Peter
On 10/07/2013 12:56 AM, Peter Levart wrote:
Hi Again,
The result of my experimentation is as follows:
Letting ReferenceHandler thread alone to en-queue References and
execute Cleaners is not enough to prevent OOMEs when allocation is
performed in large number of threads even if I let Cleaners do only
synchronous announcing of what will be freed (very fast), delegate the
actual de-allocation to a background thread and base reservation
waiting on announced free space (still wait that space is deallocated
and unreserved before satisfying reservation request, but wait as long
as it takes if the announced free space is enough for reservation
request).
ReferenceHandler thread, when it finds that it has no more pending
References, parks and waits for notification from VM. The VM promptly
process references (hooks them on the pending list), but with
saturated CPUs, waking-up the ReferenceHandler thread and re-gaining
the lock takes too much time. During that time allocating threads can
reserve the whole permitted space and OOME must be thrown. So I'm back
to strategy #1 - helping ReferenceHandler thread.
It's not so much about helping to achieve better throughput (as I
noted deallocating can not be effectively parallelized) but to
overcome the latency of waking-up the ReferenceHandler thread. Here's
my attempt at doing this:
http://cr.openjdk.java.net/~plevart/jdk8-tl/DyrectBufferAlloc/webrev.01/
This is much simplified from my 1st submission of similar strategy. I
tried to be as undisruptive to current logic of Reference processing
as possible, but of course you decide if this is still too risky for
inclusion into JDK8. Cleaner is unchanged - it processes it's thunk
synchronously and ReferenceHandler thread invokes it directly.
ReferenceHandler logic is the same - I just factored-out the content
of the loop into a private method to be able to call it from nio Bits
where the bulk of change lies.
The (un)reservation logic is re-implemented with atomic operations -
no locks. When large number of threads are competing for reservation,
locking overhead can be huge and can slow-down unreservation (which
must use the same lock as reservation). The reservation re-try logic
1st tries to satisfy the reservation request while helping
ReferenceHandler thread in en-queue-ing References and executing
Cleaners until the list of pending references is exhausted. If this
does not succeed, it triggers VM to process references (System.gc())
and then enters similar re-try loop but introducing exponentially
increasing back-off delay every time the chain of pending references
is exhausted, starting with 1ms sleep and doubling. This gives VM time
to process the references. Maximum number of sleeps is 9, giving max.
accumulated sleep time of 0.5 s. This means that a request that
rightfully throws OOME will do so after 0.5 s sleep.
I did the following measurement: Using LongAdders (to avoid
Heisenberg) I counted various exit paths from Bits.reserveMemory()
during a test that spawned 128 allocating threads on a 4-core i7
machine, allocating direct buffers randomly sized between 256KB and
1MB for 60 seconds, using -XX:MaxDirectMemorySize=512m:
Total of 909960 allocations were performed:
- 247993 were satisfied before attempting to help ReferenceHandler thread
- 660184 were satisfied while helping ReferenceHandler thread but
before triggering System.gc()
- 1783 were satisfied after triggering System.gc() but before doing
any sleep
- no sleeping has been performed
The same test, just changing -XX:MaxDirectMemorySize=128m (that means
1MB per thread each allocating direct buffers randomly sized between
256KB and 1MB):
Total of 579943 allocations were performed:
- 131547 were satisfied before attempting to help ReferenceHandler thread
- 438345 were satisfied while helping ReferenceHandler thread but
before triggering System.gc()
- 10016 were satisfied after triggering System.gc() but before doing
any sleep
- 34 were satisfied after sleep(1)
- 1 was satisfied after sleep(1) followed by sleep(2)
That's it. I think this is good enough for testing on large scale. I
have also included a modified DirectBufferAllocTest as a unit test,
but I don't know if it's suitable since it takes 60s to run. The run
time could be lowered with less probability to catch OOMEs.
So what do you think? Is this still too risky for JDK8?
Regards, Peter
On 10/06/2013 01:19 PM, Peter Levart wrote:
Hi,
I agree the problem with de-allocation of native memory blocks should
be studied deeply and this takes time.
What I have observed so far on Linux platform (other platforms may
behave differently) is the following:
Deallocation of native memory with Unsafe.freeMemory(address) can
take various amounts of time. It can grow to a constant amount of
several milliseconds to free a 1MB block, for example, when there's
already lots of blocks allocated and multiple threads are constantly
allocating more. I'm not sure yet about the main reasons for that,
but it could either be a contention with allocation from multiple
threads, interaction with GC, or even the algorithm used in the
native allocator. Deallocation is also not very parallelizable. My
observation is that deallocating with 2 threads (on a 4 core CPU)
does not help much.
Current scheme of deallocating in ReferenceHandler thread means that
a lot of "pending" Cleaner objects can accumulate and although VM has
promptly processed Cleaner PhantomReferences (hooked them on the
pending list), a lot of work is still to be done to actually free the
native blocks. This clogs ReferenceHandler thread and affects other
Reference processing. It also presents difficulties for back-off
strategy for allocating native memory. The strategy has no
information that would be needed to decide whether to wait more or to
fail with OOME.
I'm currently experimenting with approach where Cleaner and
ReferenceHandler code stays as is, but the Cheaner's thunk (the
Deallocator in DirectByteBuffer) is modified so that it performs some
actions synchronously (announcing what will be de-allocated) and
delegates the actual deallocation and unreservation to a background
thread. Reservation strategy has more information to base it's
back-off strategy that way. I'll let you know if I get some results
from that.
Regards, Peter
On 10/04/2013 08:39 PM, mark.reinh...@oracle.com wrote:
2013/10/2 15:13 -0700,alan.bate...@oracle.com:
BTW: Is this important enough to attempt to do this late in 8? I just
wonder about a significant change like switching to weak references and
whether it would be more sensible to hold it back to do early in 9.
I share your concern. This is extraordinarily sensitive code.
Now is not the time to rewrite it for JDK 8.
- Mark
