[ https://issues.apache.org/jira/browse/HDFS-12555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16201505#comment-16201505 ]
luoge123 commented on HDFS-12555: --------------------------------- In current federation function, if the /user directory has been configured as follows : <property> <name>fs.viewfs.mounttable.nsX.link./user</name> <value>hdfs://nn1:8020/user</value> </property> then, we can't add the follow /user/hive configuration to core-site.xml any more. <property> <name>fs.viewfs.mounttable.nsX.link./user/hive</name> <value>hdfs://nn2:8020/user/hive</value> </property> My improvement is that the two configuration items all exist in core-site.xml at same time. After support this configuration, we need copy data from hdfs://nn1:8020/user/hive to hdfs://nn2:8020/user/hive, fastcp or distcp tools can help us. And then we can delete hdfs://nn1:8020/user/hive. After we done this, the meta data in /user directory except /user/hive all manager by nn1, and read and write requests are send to nn1. The meta data for /user/hive manager by nn2, read and write requests for /user/hive are send to nn2. I'm not sure this explanation could answer your question or not, if you have any questions, please fell free to contact me. > HDFS federation should support configure secondary directory > ------------------------------------------------------------- > > Key: HDFS-12555 > URL: https://issues.apache.org/jira/browse/HDFS-12555 > Project: Hadoop HDFS > Issue Type: Improvement > Components: federation > Environment: 2.6.0-cdh5.10.0 > Reporter: luoge123 > Fix For: 2.6.0 > > Attachments: HDFS-12555.001.patch > > > HDFS federation support multiple namenodes horizontally scales the file > system namespace. As the amount of data grows, using a single group of > namenodes to manage a single directory, namenode still achieves performance > bottlenecks. In order to reduce the pressure of namenode, we can split out > the secondary directory, and manager it by a new namenode. This is > transparent for users. > For example, nn1 only manager the /user directory, when nn1 achieve > performance bottlenecks, we can split out /user/hive directory, and ues nn2 > to manager it. > That means core-site.xml should support as follows configuration. > <property> > <name>fs.viewfs.mounttable.nsX.link./user</name> > <value>hdfs://nn1:8020/user</value> > </property> > <property> > <name>fs.viewfs.mounttable.nsX.link./user/hive</name> > <value>hdfs://nn2:8020/user/hive</value> > </property> -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org