Hi Fengnan, Not sure if I am catching you right, but did you explore: HDFS-16943
There are other impl I believe as well as mentioned over here [1], can explore that. or did I catch the question wrong -Ayush [1] https://github.com/apache/hadoop/blob/700b3e4800b074c94c2d48ed61cb0880fe3b408d/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/resources/hdfs-rbf-default.xml#L358-L371 On Sat, 11 May 2024 at 03:10, Fengnan Li <fengna...@apache.org> wrote: > > Hi community, > > Router uses Zookeeper to its MountTable, which is the key dataset to make > the component function. Over time more and more entries are added into the > Mount table. The current Router in memory cache from ZK often takes a lot > of time to sync. (10-15 mins to download 1Million entries in our scenario). > > On another hand, Router mounts are intended to be modified by admin only, > which naturally limits the number of writes to it and the write is often > done either via cmd/scripting. We also see more use cases where customers > might want to create their own dirs and truly uses Router as one endpoint > to access the whole Storage service. This goes against Zookeeper usage > pattern and we also saw some perf and consistency issues. > > Both of the above behaviors fall into Key-Value Store use cases. Has > someone considered or implemented certain features/components using > Key-Value Store? Would love to hear more from the community. > > Thanks, > Fengnan --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
