Yuanbo Liu created HDFS-17147:
---------------------------------
Summary: RBF: RouterRpcServer getListing become extremely slow
when the children of the dir are mounted in the same ns.
Key: HDFS-17147
URL: https://issues.apache.org/jira/browse/HDFS-17147
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Yuanbo Liu
Suppose we mount table as below:
/dir -> ns0 -> /target/dir
/dir/child1 -> ns0 -> /target/dir/child1
/dir/child2 -> ns0 -> /target/dir/child2
......
/dir/child200 -> ns0 -> /target/dir/child200
when listing /dir with RBF, it's getting extremely slow as getListing has two
parts:
1. list all children of /target/dir
2. append the rest 200 mount points to the result.
The second part invoke getFileInfo concurrently to make sure mount points are
accessed under rightful permission. But in this case, the first part includes
the result of the second part, and there is no need to append second part
repeatly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
