In order to get a better understanding, I made two configurable changes
in ClassInfo, in a branch from the GROOVY_2_4_6 tag (ClassInfo is still
practically the same in the GROOVY_2_4_X branch):
- -Dgctest.classreftype=(hard|soft|weak|phantom), where hard=as today,
soft=SoftReference
- -Dgctest.cacheclassvalue=(true|false), if true and using ClassValue,
then do not cache it
See here:
https://github.com/jexler/groovy/compare/GROOVY_2_4_6...jexler:f92c2866653208ad68db5580b5bf9febc347fe1d
Compiled Groovy JAR:
https://www.jexler.net/groovy-2.4.6-gctest.jar
First thing I learned was that you cannot get the value of a
PhantomReference, it always returns null, by design. From its Javadoc:
"In order to ensure that a reclaimable object remains so, the referent
of a phantom reference may not be retrieved: The get method of a phantom
reference always returns null."
(By the way, this very probably means that the already existing
PhantomReference myThread in ClassInfo makes no sense.)
Then I ran a full matrix of tests:
------------------------------------------------------------------------
same loader | use class value | cache class value | hard | soft | weak
------------------------------------------------------------------------
YES | YES | YES | FAIL | FAIL | FAIL
YES | YES | NO | FAIL | FAIL | FAIL
YES | NO | -- | OK | OK* | OK*
------------------------------------------------------------------------
NO | YES | YES | OK | OK* | OK*
NO | YES | NO | OK | OK* | OK*
NO | NO | -- | FAIL | OK* | OK*
------------------------------------------------------------------------
- "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m -cp .
ClassGCTester -cp groovy-2.4.6-gctest.jar:filling/ -parent null -classes
GroovyFilling
- not "same loader" <=> java [opts] -XX:MaxMetaspaceSize=64m -Xmx512m
-cp .:groovy-2.4.6-gctest.jar ClassGCTester -cp filling/ -parent tester
-classes GroovyFilling
- "use class value" <=> -Dgroovy.use.classvalue=<true|false>
- "cache class value" <=> -Dgctest.cacheclassvalue=<true|false>
- "hard"|"soft"|"weak" <=> -Dgctest.classreftype=<hard|soft|weak>
* Garbage collection in all cases still only when the limit on Metaspace
or Heap is reached.
So:
- Caching ClassValue or not made no difference.
- Using weak oder soft references did not help when using ClassValue.
- When not using ClassValue, using weak or soft references helped. :)
Actually the latter is also reflected (as I noticed in retrospect) by
the pull request by John Wagenleitner for "GROOVY-7683 - Memory leak
when using Groovy as JSR-223 scripting language":
https://github.com/apache/groovy/pull/219/files
There a WeakReference is used.
Which brings my mind back to my question regarding whether it is "good
architecture" to have a reference to the class in ClassInfo (or any
other metadata associated with a class) - again, I mean fundamentally,
independently of whether this is an option for a Groovy 2.4.7 or even
anything before a Groovy 3, because I fear it would likely require to
change several Groovy APIs and internals.
If using now a WeakReference or SoftReferencefor the class reference in
ClassInfo instead of a hard reference, you now have to handle the case
where the class is already null because it has been garbage collected.
(Actually this is in principle more likely with a WeakReference than
with a SoftReference, so I would rather tend to favor SoftReference
because class GC so far only kicks in when a memory limit is reached
anyway, but likely it makes no difference in practice exactly for the
same reason. Actually, this may even save the situation, maybe in
practice you never get the Reference to return null because classes and
ClassInfo are only garbage collected together when the memory limit is
reached in a Java VM that does nothing else then, but I am not sure...)
My argument is still the same: ClassInfo (or other assiociated metadata)
only makes sense if you have your hands on a class (or an instance of
it) to apply it to. The one who wants to do something with the
class/instance has it and in principle can pass it down to ClassInfo in
order to extract whatever is needed. If there is no "client" with a
class/instance, there is no need to create ClassInfo (or similar). And
if the class is garbage collected, automatically ClassInfo cannot be
accessed with such queries any more, and then also the JVM bug with
ClassValue would no longer affect Groovy, ClassValue could be used again
by default.
But I don't want to make too much of this.
Using a WeakReference or SoftReference for the class reference in
ClassInfo would already be step forward, at least no better realistic
ideas from my side at the moment...
Alain
On 15.05.16 12:37, Jochen Theodorou wrote:
On 15.05.2016 10:39, Alain Stalder wrote:
Thanks, that clarifies a lot to me, especially SoftReference.
So with Groovy it is only realistic to have GC of classes (and attached
ClassInfo) kick in once a limit on Metaspace/PermGen (or Heap) is
reached - fine with me, no point to try to "outrun the bear"... :)
well... I do think the ClassValue version should not have this
behaviour. But for this I think we would have to ensure not to keep
any references to the ClassValue anywhere in a global strucutre. Not
even as a WeakReference... PhantomReference would probably be ok...
but I find the usages for PhantomReferences quite rare...and not
fitting here I guess
A general question (current implementation and most likely APIs to keep
aside): Why does ClassInfo need a reference to the class? To me the use
case would be that you have an Groovy object or a Groovy class and want
to do something with it (call a static or instance method, for example),
so you only need to find ClassInfo from the class and then maybe pass
the class temporarily just for doing things, but don't need it a
reference back from ClassInfo.
ClassInfo represents a cached reflective information of a Class, plus
some more internal stuff. To create that structure you need the Class.
And if you do not want to do it eager, you need to keep a reference...
at least till after init. Of course that does not have to be a
SoftReference.
[...]
This allows, for example, two produce two of the known
"OutOfMemoryError: Metaspace|PermGen" issues with Groovy 2.4.6, as
follows.
[...]
good job
bye Jochen
