A. Sophie Blee-Goldman created KAFKA-12500:
----------------------------------------------
Summary: Cache memory is leaked after removing/replacing a
StreamThread
Key: KAFKA-12500
URL: https://issues.apache.org/jira/browse/KAFKA-12500
Project: Kafka
Issue Type: Bug
Components: streams
Reporter: A. Sophie Blee-Goldman
Fix For: 2.8.0
We currently leak the memory given to a StreamThread for its cache after it
gets shutdown due to being removed or replaced.
Removal:
If the StreamThread is removing itself, or if a thread is being removed by an
external caller but fails to shutdown within the allotted time, it won't be
removed from the `threads` list to avoid freeing up it's thread id while still
running. If the thread hasn't reached the DEAD state when we resize the cache,
the new cache size per thread is computed based on the number of live threads,
which includes the removed thread at this point.
Replacement:
When a thread is replaced, we first shutdown that thread but hold off on
removing it from the threads list until it's DEAD. Immediately after the
shutdown we call addStreamThread to start up a new thread, which resizes the
cache according to num-live-threads + 1.
In both of these cases, the cache memory of the shutting down thread won't be
made available to the remaining threads.
Note: in general this leak may not be permanent, as a subsequent event once the
thread has reached DEAD and been removed from the threads list will cause it's
memory to be released. OF course if the subsequent event is another thread
removal or replacement, then we just have a new memory leak.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
