Hi all,
while investigating a Java heap space issue in an Ignite 2.18 cluster, I
noticed something that I cannot explain and would like to ask whether
this is expected behaviour.
In a heap dump, I found a large number of
|ThreadLocalAllocator$Chunk| instances retained by Tomcat request
threads. Several of these chunks had grown to very large backing byte
arrays (up to 128 MB), and they appeared to stay at that size even after
the requests that allocated them had finished.
From reading the code, it looks as if the |ThreadLocalAllocator| keeps
the largest allocated chunk in a |ThreadLocal| for reuse. I could not
find any mechanism that shrinks the backing array again. It looks like
#release(byte[] data, int maxMsgSize)can only make the maximum message
size grow. However, in order to shrink the data array maxMsgSize field
must account to less than half of the array's length.
My questions are:
* Is this behaviour expected by design?
* Is there any configuration or cleanup mechanism that allows these
buffers to shrink again?
* Has anyone observed similar memory retention under workloads that
occasionally serialize very large objects?
For context, the large allocations occurred while processing unusually
large session objects. After those requests completed, the corresponding
Tomcat worker threads still retained the large
|ThreadLocalAllocator$Chunk| instances.
I realize that retaining the buffer may be a deliberate performance
optimization. However, in environments with long-lived request threads,
this can permanently increase the retained heap after a few large requests.
Has anyone investigated this before, or is this something that should be
considered for improvement?
Thanks in advance!
Best regards,
Mathias
