[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752243#comment-17752243 ] farmmamba commented on HDFS-17149: -- [~hexiaoqiao] [~ayushsaxena] [~tomscut] [~zhangshuyan] Sir, sorry for disturbing you here. Please have a look at this issue when you have free time and please correct me if i understand wrongly. Thanks all. > getBlockLocations RPC should use actual client ip to compute network distance > when using RBF. > - > > Key: HDFS-17149 > URL: https://issues.apache.org/jira/browse/HDFS-17149 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namanode >Affects Versions: 3.4.0 >Reporter: farmmamba >Priority: Major > > Please correct me if i understand wrongly. Thanks. > Currently, when a getBlockLocations RPC forwards to namenode via router. > NameNode will use router ip address as client machine to compute network > distance against block's locations. See FSNamesystem#sortLocatedBlocksMore > method for more detailed information. > I think this compute method is not correct and should use actual client ip. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752284#comment-17752284 ] Xiaoqiao He commented on HDFS-17149: Hi [~zhanghaobo], Please check if HDFS-15079 can solve this issue. > getBlockLocations RPC should use actual client ip to compute network distance > when using RBF. > - > > Key: HDFS-17149 > URL: https://issues.apache.org/jira/browse/HDFS-17149 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namanode >Affects Versions: 3.4.0 >Reporter: farmmamba >Assignee: farmmamba >Priority: Major > > Please correct me if i understand wrongly. Thanks. > Currently, when a getBlockLocations RPC forwards to namenode via router. > NameNode will use router ip address as client machine to compute network > distance against block's locations. See FSNamesystem#sortLocatedBlocksMore > method for more detailed information. > I think this compute method is not correct and should use actual client ip. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17752303#comment-17752303 ] farmmamba commented on HDFS-17149: -- [~hexiaoqiao] Sir, thanks a lot for pointing that. I have read the comments in HDFS-15079 and i think current issue is different from HDFS-15079. The parameter clientMachine in FSNamesystem#sortLocatedBlocks method is router's ip address which should be actual client ip i think. > getBlockLocations RPC should use actual client ip to compute network distance > when using RBF. > - > > Key: HDFS-17149 > URL: https://issues.apache.org/jira/browse/HDFS-17149 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namanode >Affects Versions: 3.4.0 >Reporter: farmmamba >Assignee: farmmamba >Priority: Major > > Please correct me if i understand wrongly. Thanks. > Currently, when a getBlockLocations RPC forwards to namenode via router. > NameNode will use router ip address as client machine to compute network > distance against block's locations. See FSNamesystem#sortLocatedBlocks method > for more detailed information. > I think this compute method is not correct and should use actual client ip. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17753895#comment-17753895 ] Xiaoqiao He commented on HDFS-17149: [~zhanghaobo], Thanks, IIRC, there are many and many times we have discussed about locality for RBF, and from now on, still not agreement for some cases, such as #getBlockLocations here. I think we should push it forwards again thus enhance RBF for prod cluster. cc [~elgoiri], [~ayushtkn] what do you think about? > getBlockLocations RPC should use actual client ip to compute network distance > when using RBF. > - > > Key: HDFS-17149 > URL: https://issues.apache.org/jira/browse/HDFS-17149 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namanode >Affects Versions: 3.4.0 >Reporter: farmmamba >Assignee: farmmamba >Priority: Major > > Please correct me if i understand wrongly. Thanks. > Currently, when a getBlockLocations RPC forwards to namenode via router. > NameNode will use router ip address as client machine to compute network > distance against block's locations. See FSNamesystem#sortLocatedBlocks method > for more detailed information. > I think this compute method is not correct and should use actual client ip. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17149) getBlockLocations RPC should use actual client ip to compute network distance when using RBF.
[ https://issues.apache.org/jira/browse/HDFS-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802588#comment-17802588 ] Shilun Fan commented on HDFS-17149: --- Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > getBlockLocations RPC should use actual client ip to compute network distance > when using RBF. > - > > Key: HDFS-17149 > URL: https://issues.apache.org/jira/browse/HDFS-17149 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namanode >Affects Versions: 3.4.0 >Reporter: farmmamba >Assignee: farmmamba >Priority: Major > > Please correct me if i understand wrongly. Thanks. > Currently, when a getBlockLocations RPC forwards to namenode via router. > NameNode will use router ip address as client machine to compute network > distance against block's locations. See FSNamesystem#sortLocatedBlocks method > for more detailed information. > I think this compute method is not correct and should use actual client ip. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
