Hi Michael,
On 04/17/2019 03:59 AM, Michael Pollmeier wrote:
Hi Per,
While testing different JVMs I realized that it's fixed in openjdk 11,
e.g. openjdk version "11.0.1" 2018-10-16 LTS (zulu build), maybe by this
commit: https://hg.openjdk.java.net/jdk/jdk/rev/6464882498b5
That bug only affected ZGC (which was introduced in 11), so it didn't
change the behavior of any other GC.
That's great to know, but is it worth updating the javadocs of older
versions?
To reproduce it I attached a simple SoftRefTest.java to easily reproduce
it. It allocates (only) softly referenced objects that occupy some heap,
occasionally printing counts (instantiated, finalized, free heap).
Thanks for the test. The problem with the test is that you have a
finalize() function on Instance. This means the object will be kept
around until it has been finalized. So even if the SoftReference
referring to it is cleared, the memory will not be freed until the
object also has been finalized. In your test, you're simply creating too
many objects, and the Finalizer thread is simply unable to keep up
finalizing them at the same pace as they are created, so the heap fills up.
I adjusted your test a bit, and removed the use of finalize(). You can
run this test forever without running into OOMEs. By having the same
thread that creates the objects also poll the reference queue, the test
will automatically be throttled. If SoftReferences weren't cleared as
they should, this test would OOME pretty fast.
import java.lang.ref.Reference;
import java.lang.ref.ReferenceQueue;
import java.lang.ref.SoftReference;
import java.util.HashMap;
public class SoftRefTest2 {
public static void main(String[] args) throws Exception {
final ReferenceQueue<Instance> queue = new ReferenceQueue<>();
final HashMap<Object, SoftReference<Instance>> instances = new
HashMap<>();
long createdCount = 0;
long clearedCount = 0;
for (;;) {
SoftReference<Instance> softref = new SoftReference<Instance>(new
Instance(), queue);
instances.put(softref, softref);
if (++createdCount % 100000 == 0) {
System.out.println(createdCount + " instances created; free=" +
Runtime.getRuntime().freeMemory() / 1024 / 1024 + "M");
}
// Drain queue
Reference<?> ref;
while ((ref = queue.poll()) != null) {
instances.remove(ref);
if (++clearedCount % 100000 == 0) {
System.out.println(clearedCount + " instances cleared");
}
}
}
}
static class Instance {
static int finalizedCount = 0;
String[] occupySomeHeap = new String[50];
}
}
cheers,
Per
Usage:
javac SoftRefTest.java
java -Xms256m -Xmx256m SoftRefTest
Output:
100000 instances created; free=212M
200000 instances created; free=181M
300000 instances created; free=152M
400000 instances created; free=121M
500000 instances created; free=93M
600000 instances created; free=61M
700000 instances created; free=33M
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at Instance.<init>(SoftRefTest.java:27)
at SoftRefTest.main(SoftRefTest.java:12)
Interpretation:
It doesn't free any of the only softly referenced objects, resulting in
an OutOfMemoryError, contradicting the 'guarantee' in the javadoc.
Workaround: if you additionally configure
`-XX:SoftRefLRUPolicyMSPerMB=0`, it finalizes them and doesn't run out
of memory.
Tested with:
openjdk version "1.8.0_212"
java version "1.8.0_144" (oracle)
openjdk version "9.0.4.1" (zulu build)
openjdk version "10.0.2" 2018-07-17 (zulu build)
Cheers
Michael
On 16/04/19 6:27 pm, Per Liden wrote:
Hi Michael,
On 4/16/19 4:19 AM, David Holmes wrote:
Hi Michael,
Re-directing to core-libs-dev and hotspot-gc-dev.
Thanks,
David
On 16/04/2019 12:14 pm, Michael Pollmeier wrote:
Quoting
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/ref/SoftReference.html
All soft references to softly-reachable objects are guaranteed to have
been cleared before the virtual machine throws an OutOfMemoryError
That statement was true when soft references were first introduced in
java 1.2, but from java 1.3.1 the jvm property
`-XX:SoftRefLRUPolicyMSPerMB` was introduced.
That statement is still true. When the GC gets into a situation where it
is having trouble satisfying an allocation, then SoftRefLRUPolicyMSPerMB
will be ignored and all soft references will be cleared, before the GC
gives up and throws an OOME.
It defaults to 1000 (milliseconds), meaning that if there’s only 10MB
available heap, the garbage collector will free references that have
been used more than 10s ago. I.e. everything else (including young
softly reachable objects) will *not* be freed, leading to an
OutOfMemoryError, contradicting the above quoted 'guarantee'.
That's also the behaviour I observed on various JREs. Would you agree,
i.e. should I propose an updated doc?
Could you elaborate on what kind of test you did to come to this
conclusion? Preferably provide a re-producer. In OOME situations, if a
SoftReference isn't cleared, it is typically because you have
unknowingly made it strongly reachable.
cheers,
Per
