Uma Maheswara Rao G created HDFS-15289:
------------------------------------------
Summary: Allow viewfs mounts with hdfs scheme and centralized
mount table
Key: HDFS-15289
URL: https://issues.apache.org/jira/browse/HDFS-15289
Project: Hadoop HDFS
Issue Type: New Feature
Components: fs
Affects Versions: 3.2.0
Reporter: Uma Maheswara Rao G
Assignee: Uma Maheswara Rao G
Fix For: 3.4.0
ViewFS provides flexibility to mount different filesystem types with mount
points configuration table. Additionally viewFS provides flexibility to
configure any fs (not only HDFS) scheme in mount table mapping. This approach
is solving the scalability problems, but users need to reconfigure the
filesystem to ViewFS and to its scheme. This will be problematic in the case
of paths persisted in meta stores, ex: Hive. In systems like Hive, it will
store uris in meta store. So, changing the file system scheme will create a
burden to upgrade/recreate meta stores. In our experience many users are not
ready to change that.
Router based federation is another implementation to provide coordinated mount
points for HDFS federation clusters. Even though this provides flexibility to
handle mount points easily, this will not allow other(non-HDFS) file systems to
mount. So, this does not solve the purpose when users want to mount
external(non-HDFS) filesystems.
So, the problem here is: Even though many users want to adapt to the scalable
fs options available, technical challenges of changing schemes (ex: in meta
stores) in deployments are obstructing them.
So, we propose to allow hdfs scheme in ViewFS like client side mount system and
provision user to create mount links without changing URI paths.
I will upload detailed design doc shortly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
