[
https://issues.apache.org/jira/browse/HDFS-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036195#comment-16036195
]
Weiwei Yang commented on HDFS-11887:
------------------------------------
Hi [~msingh]
bq. There is a StackOverFlow which happens when the client is re-enqueued in
case there is reference on the client.
I see. If add the entry back to cache in {{onRemoval}} function gives
StackOverFlow exception, so this is not an option. Thanks for pointing this out.
bq. 1) as explained earlier, if there are more references on the client, it
will be closed when all the references drop down to 0.
This works only if clients are released before eviction is triggered.
Let me explain my concern with an example, to simplify the problem, lets assume
the client cache with only size 1
# A client calls {{acquireClient}} to get an instance for container *A*, lets
say it gets *clientA*. As now the cache is empty, *clientA* will be added into
the cache as first entry. And now the *referenceCount* is 1.
# This client did not call {{releaseClient}} for the time being.
# Another client calls {{acquireClient}} to get an instance for container *B*,
as the cache size is 1, *clientA* will need to be evicted from cache. This
triggers {{onRemoval}} code, which calls {{clientA.setEvicted()}}. This sets
the flag {{isEvicted}} to true but the *referenceCount* is still 1 (as client
is not released). In this case, in {{cleanup}} code {{xceiverClient.close()}}
will not be called. But *clientA* is removed and now cache has 1 entry
*clientB*.
Maybe we should think another approach. E.g create a regular cache maintenance
thread with ScheduledExecutorService.
Let me know your thoughts, thank you.
> XceiverClientManager should close XceiverClient on eviction from cache
> ----------------------------------------------------------------------
>
> Key: HDFS-11887
> URL: https://issues.apache.org/jira/browse/HDFS-11887
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Mukul Kumar Singh
> Assignee: Mukul Kumar Singh
> Attachments: HDFS-11887-HDFS-7240.001.patch,
> HDFS-11887-HDFS-7240.002.patch
>
>
> XceiverClientManager doesn't close client on eviction which can leak
> resources.
> {code}
> public XceiverClientManager(Configuration conf) {
> .
> .
> .
> public void onRemoval(
> RemovalNotification<String, XceiverClientWithAccessInfo>
> removalNotification) {
> // If the reference count is not 0, this xceiver client should
> not
> // be evicted, add it back to the cache.
> WithAccessInfo info = removalNotification.getValue();
> if (info.hasRefence()) {
> synchronized (XceiverClientManager.this.openClient) {
> XceiverClientManager.this
> .openClient.put(removalNotification.getKey(), info);
> }
> }
> {code}
> Also a stack overflow can be triggered because of putting the element back in
> the cache on eviction.
> {code}
> synchronized (XceiverClientManager.this.openClient) {
> XceiverClientManager.this
> .openClient.put(removalNotification.getKey(), info);
> }
> {code}
> This bug will try to fix both of these cases.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
