[
https://issues.apache.org/jira/browse/HADOOP-19640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18048592#comment-18048592
]
Steve Loughran commented on HADOOP-19640:
-----------------------------------------
Any PR for this?
> Resource leak in AssumedRoleCredentialProvider
> ----------------------------------------------
>
> Key: HADOOP-19640
> URL: https://issues.apache.org/jira/browse/HADOOP-19640
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.4.1
> Reporter: Antoni Reus
> Priority: Minor
>
> When `org.apache.hadoop.fs.s3a.auth.AssumedRoleCredentialProvider` is used in
> a Hadoop Configuration object, it will attempt to `resolveCredentials()`
> inside the constructor.
> (lines 165-167)
> {code:java}
> // and force in a fail-fast check just to keep the stack traces less
> // convoluted
> resolveCredentials();{code}
> If this method fails, because the current identity is not able to assume
> role, the constructor will throw an exception, and fail to close the
> `stsClient`, `stsProvider` and any other resources that are created in the
> constructor, leaking threads and other resources.
>
> In a long running application, that handles Hadoop S3 file systems, where the
> user can dynamically change to configured role to assume, and external id,
> this will lead to eventually the system running out of resources due to the
> leaked threads created by the AWS SDK clients that are not closed when a
> wrong role or external id is used.
>
> There are two potential fixes for this problem:
> - Don't attempt to `resolveCredentials()` inside the constructor
> - Wrap the `resolveCredentials()` in the constructor in a try/catch block,
> that cleans the resources and rethrows the exception in the catch block.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
