[
https://issues.apache.org/jira/browse/HADOOP-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637832#comment-17637832
]
ASF GitHub Bot commented on HADOOP-18526:
-----------------------------------------
steveloughran commented on PR #5144:
URL: https://github.com/apache/hadoop/pull/5144#issuecomment-1325255012
latest test run s3 london, params ` -Dparallel-tests -DtestsThreadCount=10
-Dscale`; all good.
new test to verify that instrumentation.close() unregisters on the first
call and is no-op on the second...both unit test for standalone and itest for
lifecycle through s3afs.
that showed that instrumentation is being unregistered on close(), so if
leakage is reported, it's from something creating fs instances but not closing
them. the weak refs will ensure that the instrumentation isn't being held on
-but thread pools are still there to consume resources.
> Leak of S3AInstrumentation instances via hadoop Metrics references
> ------------------------------------------------------------------
>
> Key: HADOOP-18526
> URL: https://issues.apache.org/jira/browse/HADOOP-18526
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.4
> Reporter: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> A heap dump of a process running OOM shows that if a process creates then
> destroys lots of S3AFS instances, you seem to run out of heap due to
> references to S3AInstrumentation and the IOStatisticsStore kept via the
> hadoop metrics registry
> It doesn't look like S3AInstrumentation.close() is being invoked in
> S3AFS.close(). it should -with the IOStats being snapshotted to a local
> reference before this happens. This allows for stats of a closed fs to be
> examined.
> If you look at org.apache.hadoop.ipc.DecayRpcScheduler.MetricsProxy it uses a
> WeakReference to refer back to the larger object. we should do the same for
> abfs/s3a bindings. ideally do some template proxy class in hadoop common they
> can both use.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
