[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802633#comment-17802633 ] Shilun Fan commented on HDFS-16093: --- Bulk update: moved all 3.4.0 non-blocker issues, please move back if it is a blocker. Retarget 3.5.0. > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Assignee: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17529026#comment-17529026 ] Stephen O'Donnell commented on HDFS-16093: -- I think HDFS-16076 addressed the sorting, to put the out of service nodes last in the list. > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Assignee: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528889#comment-17528889 ] Renukaprasad C commented on HDFS-16093: --- [~Daniel Ma] Thanks for reporting the issue. Thanks [~hexiaoqiao] [~sodonnell] [~tomscut] for review & feedback. I do agree with [~hexiaoqiao] / [~tomscut] / [~sodonnell] , instead of excluding the Decommissioning (Decommissioned), this can be placed last. And read will be success with other normal DNs. Are you still working on this solution? > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Assignee: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372116#comment-17372116 ] Xiaoqiao He commented on HDFS-16093: -1 to remove DECOMMISSIONING state node from `LocatedBlocks` directly. A. It is possible that all replicas of one block on DECOMMISSIONING node, and client will meet BlockMissing Exception if we do that but actually not as [~sodonnell] mentioned above. B. We would sort DECOMMISSIONING or DECOMMISSIONED to the end of `LocatedBlocks` but not remove directly. > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371306#comment-17371306 ] tomscut commented on HDFS-16093: IMO, abnormal nodes cannot be removed directly. Maybe sorting is a better choice, because it makes it easier to degrade read when there are not enough normal nodes. Just like the example [~sodonnell] gave. > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371298#comment-17371298 ] Stephen O'Donnell commented on HDFS-16093: -- I'm not sure it we can simply removed them. There is also a distinction between DECOMMISSIONING and DECOMMISSIONED. It is possible for all 3 replicas of a file to be on DECOMMISSIONING host, and therefore it can only be read if those hosts are returned. For DECOMMISSIONED hosts which are alive and not stale, I think they can be used for reads in some circumstances. I recall seeing some comments in the code suggesting DECOMMISSIONED replicas can be used as a "last resort". > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371015#comment-17371015 ] Daniel Ma commented on HDFS-16093: -- Hi [~tomscut], thanks for your quick reply. The Jira you mentioned can relieve such issue to some extent, but I think only the DataNode in service should be returned to the client. All the abnornal state DataNode like DECOMMISSION or MAINTANENCE should be removed in the return list. > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16093) DataNodes under decommission will still be returned to the client via getLocatedBlocks, so the client may request decommissioning datanodes to read which will cause bad
[ https://issues.apache.org/jira/browse/HDFS-16093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17371008#comment-17371008 ] tomscut commented on HDFS-16093: Hi [~Daniel Ma], may I ask if [HDFS-16076|https://issues.apache.org/jira/browse/HDFS-16076] can solve your problem? After sorting locations, the order will be: live -> slow -> stale -> staleAndSlow -> entering_maintenance -> decommissioned. Do you mean that we need to consider decommissioning state as well? > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > -- > > Key: HDFS-16093 > URL: https://issues.apache.org/jira/browse/HDFS-16093 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.1 >Reporter: Daniel Ma >Priority: Critical > > DataNodes under decommission will still be returned to the client via > getLocatedBlocks, so the client may request decommissioning datanodes to read > which will cause badly competation on disk IO. > Therefore, datanodes under decommission should be removed from the return > list of getLocatedBlocks api. > !image-2021-06-29-10-50-44-739.png! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
