yangzhanling created HDFS-16416: ----------------------------------- Summary: Namenode was unavailable when high concurrency Key: HDFS-16416 URL: https://issues.apache.org/jira/browse/HDFS-16416 Project: Hadoop HDFS Issue Type: Improvement Components: namanode Affects Versions: 2.7.7 Environment: org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager#outputBufferCapacity private int outputBufferCapacity = 512 * 1024;
so we can update sample: //first add key and value in DFSConfigKeys //org.apache.hadoop.hdfs.DFSConfigKeys //hdfs-site.xml public static final String DFS_NAMENODE_BUFFER_KEY = "dfs.namenode.buffer.key"; public static final int DFS_NAMENODE_BUFFER_VALUE_DEFAULT = 512 * 1024; //second,add param from configurature //org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager this.outputBufferCapacity = conf.getInt( DFSConfigKeys.DFS_NAMENODE_BUFFER_KEY, DFSConfigKeys.DFS_NAMENODE_BUFFER_VALUE_DEFAULT ); //third ,add config in hdfs-site.xml <property> <name>dfs.namenode.buffer.key"</name> <value>1024</value> </property> Reporter: yangzhanling Namenode was unavailable when high concurrency when meta data wrote to DoubleBuffer,and the currentBuff was full (512k) ,the other cursyncBuffer was happend sync to dist or journalnode,so we request to namenode was unavailable. Because the currentBuffer size is 512k,it is smaller in some bigger request cluster.but it is not updated by configuring with *hdfs-site.xml*.it is hardcoding in Java source code and unreasonable. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org