[jira] [Commented] (HDFS-17527) RBF: Routers should not allow observer reads when namenode stateId context is disabled
[ https://issues.apache.org/jira/browse/HDFS-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866808#comment-17866808 ] Simbarashe Dzinamarira commented on HDFS-17527: --- In the RouterStateIdContext, we should also filter namespaces with stateId <= 0 from the federatedStates header field. [https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterStateIdContext.java#L103] > RBF: Routers should not allow observer reads when namenode stateId context is > disabled > -- > > Key: HDFS-17527 > URL: https://issues.apache.org/jira/browse/HDFS-17527 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Simbarashe Dzinamarira >Assignee: Jian Zhang >Priority: Major > > HDFS-17514 addressed the case when state ID context is first enabled and then > disabled. However, if state Id is never enabled at all, there should be no > observer reads. > Tests in TestNoNamenodesAvailableLongTime do not enable the namenode state Id > context but there are still observer reads. > The solution to this is to not advance the shareGlobalStateID in > PoolAlignmentContext when the namenode returns a values of zero in the > RpcResponseHeader. Zero indicates that stateIdContext is disabled and should > not be treated as a valid state ID value. Note, fixing this will require > adjusting the unit tests as well. > A further optimization related to HDFS-17514 is that when sharedGlobalStateId > and poolLocalStateId have been reset, we also should not allow > poolLocalStateId to be advanced by clients until the sharedGlobalStateId has > been advanced. This will protect existing clients from using a stale ID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17527) RBF: Routers should not allow observer reads when namenode stateId context is disabled
[ https://issues.apache.org/jira/browse/HDFS-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846381#comment-17846381 ] Simbarashe Dzinamarira commented on HDFS-17527: --- Great, I'll assign it to you. Thanks for taking it up. > RBF: Routers should not allow observer reads when namenode stateId context is > disabled > -- > > Key: HDFS-17527 > URL: https://issues.apache.org/jira/browse/HDFS-17527 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Simbarashe Dzinamarira >Priority: Major > > HDFS-17514 addressed the case when state ID context is first enabled and then > disabled. However, if state Id is never enabled at all, there should be no > observer reads. > Tests in TestNoNamenodesAvailableLongTime do not enable the namenode state Id > context but there are still observer reads. > The solution to this is to not advance the shareGlobalStateID in > PoolAlignmentContext when the namenode returns a values of zero in the > RpcResponseHeader. Zero indicates that stateIdContext is disabled and should > not be treated as a valid state ID value. Note, fixing this will require > adjusting the unit tests as well. > A further optimization related to HDFS-17514 is that when sharedGlobalStateId > and poolLocalStateId have been reset, we also should not allow > poolLocalStateId to be advanced by clients until the sharedGlobalStateId has > been advanced. This will protect existing clients from using a stale ID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17527) RBF: Routers should not allow observer reads when namenode stateId context is disabled
[ https://issues.apache.org/jira/browse/HDFS-17527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846378#comment-17846378 ] Jian Zhang commented on HDFS-17527: --- [~simbadzina] hi,I can work on the issues, if no one is currently working on it, you can assign it to me. > RBF: Routers should not allow observer reads when namenode stateId context is > disabled > -- > > Key: HDFS-17527 > URL: https://issues.apache.org/jira/browse/HDFS-17527 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Simbarashe Dzinamarira >Priority: Major > > HDFS-17514 addressed the case when state ID context is first enabled and then > disabled. However, if state Id is never enabled at all, there should be no > observer reads. > Tests in TestNoNamenodesAvailableLongTime do not enable the namenode state Id > context but there are still observer reads. > The solution to this is to not advance the shareGlobalStateID in > PoolAlignmentContext when the namenode returns a values of zero in the > RpcResponseHeader. Zero indicates that stateIdContext is disabled and should > not be treated as a valid state ID value. Note, fixing this will require > adjusting the unit tests as well. > A further optimization related to HDFS-17514 is that when sharedGlobalStateId > and poolLocalStateId have been reset, we also should not allow > poolLocalStateId to be advanced by clients until the sharedGlobalStateId has > been advanced. This will protect existing clients from using a stale ID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
