[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-10987: --- Fix Version/s: 2.7.4 > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987-branch-2.7.patch, > HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Labels: (was: release-blocker) > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987-branch-2.7.patch, > HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Attachment: HDFS-10987-branch-2.7.patch Uploading the branch-2.7 patch.Kindly Review. > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Labels: release-blocker > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987-branch-2.7.patch, > HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-10987: --- Labels: release-blocker (was: ) Target Version/s: 2.7.4 > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Labels: release-blocker > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10987: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Thanks everyone for reviews and thanks for working on this [~brahmareddy]. Committed this to trunk, branch-2 and branch-2.8. branch-2* had a couple of minor conflicts that was easily resolved. > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10987: -- Fix Version/s: 3.0.0-alpha2 > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Fix For: 3.0.0-alpha2 > > Attachments: HDFS-10987-002.patch, HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Attachment: HDFS-10987-002.patch bq. Yes. Along with that, IMO should also add 'namesystem.isRunning()' to while loop condition in 'check()' to end execution fast Sounds good to me.. Updated the patch..Addressed kihwal and vinay comments.. > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Attachments: HDFS-10987-002.patch, HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Priority: Critical (was: Major) > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula >Priority: Critical > Attachments: HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Attachment: HDFS-10987.patch Yielding is required in case of single Node have huge number of blocks to process.Introduced same and uploading the patch. > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10987) Make Decommission less expensive when lot of blocks present.
[ https://issues.apache.org/jira/browse/HDFS-10987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-10987: Status: Patch Available (was: Open) > Make Decommission less expensive when lot of blocks present. > > > Key: HDFS-10987 > URL: https://issues.apache.org/jira/browse/HDFS-10987 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: HDFS-10987.patch > > > When user want to decommission a node which having 50M blocks +,it could hold > the namesystem lock for long time.We've seen it is taking 36 sec+. > As we knew during this time, Namenode will not available... As this > decommission will continuosly run till all the blocks got replicated,hence > Namenode will unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
