[ https://issues.apache.org/jira/browse/HDFS-13689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802444#comment-17802444 ]
Shilun Fan commented on HDFS-13689: ----------------------------------- updated the target version for preparing 3.4.0 release. > NameNodeRpcServer getEditsFromTxid assumes it is run on active NameNode > ----------------------------------------------------------------------- > > Key: HDFS-13689 > URL: https://issues.apache.org/jira/browse/HDFS-13689 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: hdfs, namenode > Reporter: Erik Krogen > Priority: Major > > {{NameNodeRpcServer#getEditsFromTxid}} currently decides which transactions > are able to be served, i.e. which transactions are durable, using the > following logic: > {code} > long syncTxid = log.getSyncTxId(); > // If we haven't synced anything yet, we can only read finalized > // segments since we can't reliably determine which txns in in-progress > // segments have actually been committed (e.g. written to a quorum of > JNs). > // If we have synced txns, we can definitely read up to syncTxid since > // syncTxid is only updated after a transaction is committed to all > // journals. (In-progress segments written by old writers are already > // discarded for us, so if we read any in-progress segments they are > // guaranteed to have been written by this NameNode.) > boolean readInProgress = syncTxid > 0; > {code} > This assumes that the NameNode serving this request is the current > writer/active NameNode, which may not be true in the ObserverNode situation. > Since {{selectInputStreams}} now has a {{onlyDurableTxns}} flag, which, if > enabled, will only return durable/committed transactions, we can instead > leverage this to provide the same functionality. We should utilize this to > avoid consistency issues when serving this request from the ObserverNode. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org