[ https://issues.apache.org/jira/browse/HDFS-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142884#comment-17142884 ]
Akira Ajisaka edited comment on HDFS-15421 at 6/23/20, 1:33 PM: ---------------------------------------------------------------- {quote}I think we need to update genstamp when rolling {{OP_APPEND}}. In {{OP_TRUNCATE}}, it is the same. {quote} This change does not fix the problem for append. When appending a block without {{CreateFlag.NEW_BLOCK}}, the edit log becomes as follows: * {{OP_APPEND}}: prepare for append * {{OP_SET_GENSTAMP_V2}}: update pipeline * (edited) {{OP_UPDATE_BLOCKS}}: update blocks That way SNN will tail {{OP_SET_GENSTAMP_V2}} after {{OP_APPEND}}, so apply impending genstamp in {{OP_APPEND}} does not fix this problem. I'll attach a patch with some regression tests. was (Author: ajisakaa): {quote}I think we need to update genstamp when rolling {{OP_APPEND}}. In {{OP_TRUNCATE}}, it is the same. {quote} This change does not fix the problem for append. When appending a block without {{CreateFlag.NEW_BLOCK}}, the edit log becomes as follows: * {{OP_APPEND}}: prepare for append * {{OP_SET_GENSTAMP_V2}}: update pipeline That way SNN will tail {{OP_SET_GENSTAMP_V2}} after {{OP_APPEND}}, so apply impending genstamp in {{OP_APPEND}} does not fix this problem. I'll attach a patch with some regression tests. > IBR leak causes standby NN to be stuck in safe mode > --------------------------------------------------- > > Key: HDFS-15421 > URL: https://issues.apache.org/jira/browse/HDFS-15421 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Reporter: Kihwal Lee > Assignee: Akira Ajisaka > Priority: Blocker > Labels: release-blocker > Attachments: HDFS-15421-000.patch, HDFS-15421-001.patch > > > After HDFS-14941, update of the global gen stamp is delayed in certain > situations. This makes the last set of incremental block reports from append > "from future", which causes it to be simply re-queued to the pending DN > message queue, rather than processed to complete the block. The last set of > IBRs will leak and never cleaned until it transitions to active. The size of > {{pendingDNMessages}} constantly grows until then. > If a leak happens while in a startup safe mode, the namenode will never be > able to come out of safe mode on its own. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org