[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765-Implementation-Proposal.pdf > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G >Priority: Major > Fix For: 2.0.0-alpha, 0.23.7 > > Attachments: HDFS-1765-Implementation-Proposal.pdf, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, > underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-1765: - Target Version/s: 0.23.3, 0.24.0 (was: 0.24.0, 0.23.3) Fix Version/s: 0.23.7 Committed to branch-0.23. > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 2.0.0-alpha, 0.23.7 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Resolution: Fixed Target Version/s: 0.24.0, 0.23.3 (was: 0.23.3, 0.24.0) Status: Resolved (was: Patch Available) > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0, 0.23.3 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-1765: -- Target Version/s: 0.24.0, 0.23.3 (was: 0.23.3, 0.24.0) Fix Version/s: 0.23.3 > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0, 0.23.3 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Target Version/s: 0.24.0, 0.23.3 > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-1765: -- Target Version/s: (was: 0.24.0) Fix Version/s: (was: 0.23.1) 0.24.0 I've committed this. Thanks Uma! I think this would be a good fix for 23, however w/o 1st merging HDFS-2362 this is a non-trivial merge so leaving as 0.24 for now. > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-1765: -- Fix Version/s: (was: 0.24.0) 0.23.1 Issue Type: Bug (was: Improvement) Hadoop Flags: Reviewed +1 looks great > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.23.1 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.patch > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Status: Patch Available (was: Open) > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.patch > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: (was: HDFS-1765.patch) > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, > underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Status: Open (was: Patch Available) > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.patch > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Uma Maheswara Rao G > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.patch, > HDFS-1765.pdf, underReplicatedQueue.pdf > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Target Version/s: 0.24.0 > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, > underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.patch > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.patch, HDFS-1765.pdf, > underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Status: Patch Available (was: Open) HDFS-1765.patch :1st Patch. Updated with the propose solution. @Nicholas, Addressed your comment for moving the chooseUnderReplicatedBlocks to UnderReplicatedBlocks class. Thanks a lot for the document Reviews. > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.patch > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.24.0 > > Attachments: HDFS-1765.patch, HDFS-1765.pdf, underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-1765: -- Attachment: HDFS-1765.pdf > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.24.0 > > Attachments: HDFS-1765.pdf, underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1765) Block Replication should respect under-replication block priority
[ https://issues.apache.org/jira/browse/HDFS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haryadi Gunawi updated HDFS-1765: - Attachment: underReplicatedQueue.pdf > Block Replication should respect under-replication block priority > - > > Key: HDFS-1765 > URL: https://issues.apache.org/jira/browse/HDFS-1765 > Project: Hadoop HDFS > Issue Type: Improvement > Components: name-node >Affects Versions: 0.23.0 >Reporter: Hairong Kuang >Assignee: Hairong Kuang > Fix For: 0.23.0 > > Attachments: underReplicatedQueue.pdf > > > Currently under-replicated blocks are assigned different priorities depending > on how many replicas a block has. However the replication monitor works on > blocks in a round-robin fashion. So the newly added high priority blocks > won't get replicated until all low-priority blocks are done. One example is > that on decommissioning datanode WebUI we often observe that "blocks with > only decommissioning replicas" do not get scheduled to replicate before other > blocks, so risking data availability if the node is shutdown for repair > before decommission completes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
