[
https://issues.apache.org/jira/browse/HADOOP-19239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HADOOP-19239:
------------------------------
Description:
We have an online service which uses Hadoop FileSystem to load files from
Clould storage.
The current cache in FileSystem is a
[HashMap|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3635C1-L3635C62],
and its key honors scheme, authority (like
[user@host:port|https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#Syntax]),
ugi and a unique long for its [hash
code|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3891C1-L3894C8].
And among those 4 fields, only "scheme" and "authority" could be controlled
externally.
That results in a wrong case like: A FileSystem entry in the cache was created
with schemeA + authorityA, and with read + write access, and an expiration.
Later, an API to get FileSystem comes still using schemeA + authorityA, but
with less access (maybe read only), or it already expires, that FileSystem
entry in the cache is honored by mistake, while no new FilleSystem is created.
It does not lead to a security issue, but subsequent calls (may to read the
file) will be rejected with 403 by the remote stoage.
was:
We have an online service which uses Hadoop FileSystem to load files from
Clould storage.
The current cache in FileSystem is a
[HashMap|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3635C1-L3635C62],
and its key honors scheme, authority (like
[user@host:port|https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#Syntax]),
ugi and a unique long for its [hash
code|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3891C1-L3894C8].
And among those 4 fields, only "scheme" and "authority" could be controlled
externally.
That results in a wrong case like: A FileSystem entry in the cache was created
with schemeA + authorityA, and with read + write access, and an expiration.
Later, an API to get FileSystem comes still using schemeA + authorityA, but
with less access (maybe read only) comes, or it already expires, that
FileSystem entry in the cache is honored by mistake, while no new FilleSystem
is created. It does not lead to a security issue, but subsequent calls (may
read the file) will be rejected with 403 by the remote stoage.
> Enhance FileSystem to honor security token and expiration in its cache
> ----------------------------------------------------------------------
>
> Key: HADOOP-19239
> URL: https://issues.apache.org/jira/browse/HADOOP-19239
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 3.3.4
> Reporter: Xiang Li
> Priority: Critical
> Fix For: 3.3.4
>
>
> We have an online service which uses Hadoop FileSystem to load files from
> Clould storage.
> The current cache in FileSystem is a
> [HashMap|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3635C1-L3635C62],
> and its key honors scheme, authority (like
> [user@host:port|https://en.wikipedia.org/wiki/Uniform_Resource_Identifier#Syntax]),
> ugi and a unique long for its [hash
> code|https://github.com/apache/hadoop/blob/4525c7e35ea22d7a6350b8af10eb8d2ff68376e7/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L3891C1-L3894C8].
> And among those 4 fields, only "scheme" and "authority" could be controlled
> externally.
> That results in a wrong case like: A FileSystem entry in the cache was
> created with schemeA + authorityA, and with read + write access, and an
> expiration. Later, an API to get FileSystem comes still using schemeA +
> authorityA, but with less access (maybe read only), or it already expires,
> that FileSystem entry in the cache is honored by mistake, while no new
> FilleSystem is created. It does not lead to a security issue, but subsequent
> calls (may to read the file) will be rejected with 403 by the remote stoage.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
