[
https://issues.apache.org/jira/browse/HDFS-17148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17756432#comment-17756432
]
ASF GitHub Bot commented on HDFS-17148:
---------------------------------------
hchaverri opened a new pull request, #5966:
URL: https://github.com/apache/hadoop/pull/5966
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
This is a backport to branch-3.3 of commit:
https://github.com/apache/hadoop/commit/ad2f45c64f01a520a01f6876d8493140b9f89b03
Patch applied cleanly except for checkstyle warnings on test class
(addressed on commit 34faeee)
JIRA: [HDFS-17148](https://issues.apache.org/jira/browse/HDFS-17148). RBF:
SQLDelegationTokenSecretManager must cleanup expired tokens in SQL
These changes update the SQLDelegationTokenSecretManager to cleanup expired
tokens found in SQL. Currently, AbstractDelegationTokenSecretManagers only
cleanup tokens in its memory cache. The SQLDelegationTokenSecretManager was
recently updated to use a LoadingCache with a short TTL, so most expired tokens
won't be present in memory.
During token cleanup, the SQLDelegationTokenSecretManager will query SQL for
a list of tokens that have not been updated recently, based on the modifiedTime
column. We will limit the amount of results returned to prevent performance
impact on SQL. Once the list is returned, the ExpiredTokenRemover will evaluate
if the tokens are actually expired and delete them from SQL if so.
### How was this patch tested?
Added unit test for different token cleanup scenarios:
Having an expired token in SQL. which should be deleted
Having a token with a long renewal time, which should not be deleted
Having a token recently renewed, which should not be deleted
### For code changes:
- [Y] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [Y] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [Y] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [Y] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> RBF: SQLDelegationTokenSecretManager must cleanup expired tokens in SQL
> -----------------------------------------------------------------------
>
> Key: HDFS-17148
> URL: https://issues.apache.org/jira/browse/HDFS-17148
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rbf
> Reporter: Hector Sandoval Chaverri
> Assignee: Hector Sandoval Chaverri
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> The SQLDelegationTokenSecretManager fetches tokens from SQL and stores them
> temporarily in a memory cache with a short TTL. The ExpiredTokenRemover in
> AbstractDelegationTokenSecretManager runs periodically to cleanup any expired
> tokens from the cache, but most tokens have been evicted automatically per
> the TTL configuration. This leads to many expired tokens in the SQL database
> that should be cleaned up.
> The SQLDelegationTokenSecretManager should find expired tokens in SQL instead
> of in the memory cache when running the periodic cleanup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
