[ https://issues.apache.org/jira/browse/HDFS-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502070#comment-17502070 ]
liutongwei commented on HDFS-16493: ----------------------------------- Thanks [~xkrogen] for replying. {quote}Thanks for reporting liutongwei! I guess this is a continuation of your comment on HDFS-13150, is that correct? {quote} Yes. When I reviewing the journalnode source code to track the issue [HDFS-16490|https://issues.apache.org/jira/browse/HDFS-16490]. The doubt of fast path tail resurface. After add test case [^example.patch], it seems that fast path tail not return edit logs same as the original tail process. [~shv] , what's your opinion about this issue? > [SBN Read]When fast path tail enabled, standby or observer namenode may read > uncommitted data > --------------------------------------------------------------------------------------------- > > Key: HDFS-16493 > URL: https://issues.apache.org/jira/browse/HDFS-16493 > Project: Hadoop HDFS > Issue Type: Bug > Components: journal-node, namanode > Reporter: liutongwei > Priority: Critical > Attachments: example.patch > > > Although fast path tail use quorum read to pull edit log, it seem like can > read uncommitted data in some corner case. > Here is an example. Suppose we have three JN, their init state is: > > {code:java} > epoch 1 > JN1 [1-3](in-progress) > JN2 [1-3](in-progress) > JN3 [1-4](in-progress) > Note that, in epoch 1 txid 1-3 was committed, and txid 4 not. > {code} > When a failover occur, if a new writer cannot contact to JN3 for network > partition, and finish the recovery stage, and write a new txid 4 in epoch 2, > which value not equal to JN3's. > > {code:java} > epcho 2 > JN1 [1-3](finalized) [4-4](inprogress) > JN2 [1-3](finalized) [4-4](inprogress) > JN3 [1-4](inprogress) > Note that, in JN3 txid4's value not equal to other JN. > {code} > > Now there is a read namenode to pull edits, and it contact to JN3 and JN2, it > got majority response. But it got logs of same length but different > content.And no more information to choose which log is right. If we choose > JN3, we got meta data corruption. > There is a test example patch [^example.patch] for running and debug. > For fix it i think we should add finalized state to > {{{}GetJournaledEditsResponseProto{}}}, so we can discard the fault log. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org