[jira] [Commented] (HDFS-15120) Refresh BlockPlacementPolicy at runtime.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039770#comment-17039770 ] Jinglun commented on HDFS-15120: Upload v05 fix unit tests > Refresh BlockPlacementPolic

[jira] [Updated] (HDFS-15120) Refresh BlockPlacementPolicy at runtime.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15120: --- Attachment: HDFS-15120.005.patch > Refresh BlockPlacementPolicy at runtime. > ---

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039753#comment-17039753 ] Hadoop QA commented on HDFS-15176: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Updated] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15180: - Component/s: datanode > DataNode FsDatasetImpl Fine-Grained Locking via BlockPool. > ---

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039723#comment-17039723 ] zhuqi commented on HDFS-15177: -- Hi [~sodonnell] Thanks for your reply. Next, i will change

[jira] [Comment Edited] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039720#comment-17039720 ] Siddharth Wagle edited comment on HDFS-15154 at 2/19/20 6:03 AM: --

[jira] [Commented] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039720#comment-17039720 ] Siddharth Wagle commented on HDFS-15154: Looking into the test failure will updat

[jira] [Comment Edited] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039720#comment-17039720 ] Siddharth Wagle edited comment on HDFS-15154 at 2/19/20 5:59 AM: --

[jira] [Commented] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039712#comment-17039712 ] Hadoop QA commented on HDFS-15154: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039710#comment-17039710 ] Masatake Iwasaki commented on HDFS-15181: - [~kihwal] I will appreciate if you tak

[jira] [Commented] (HDFS-15165) In Du missed calling getAttributesProvider

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039709#comment-17039709 ] Hadoop QA commented on HDFS-15165: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039693#comment-17039693 ] Hadoop QA commented on HDFS-15182: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-13739) Add option to disable rack local write preference

2020-02-18 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039675#comment-17039675 ] Hudson commented on HDFS-13739: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #

[jira] [Commented] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039673#comment-17039673 ] Ayush Saxena commented on HDFS-15182: - can we do the init in {{@Before}}? > TestBloc

[jira] [Assigned] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri reassigned HDFS-15182: -- Assignee: Lisheng Sun > TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk > ---

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039669#comment-17039669 ] Íñigo Goiri commented on HDFS-15176: Let's do a few cosmetic changes: * Values like {

[jira] [Updated] (HDFS-13739) Add option to disable rack local write preference

2020-02-18 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-13739: Fix Version/s: 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolve

[jira] [Commented] (HDFS-13739) Add option to disable rack local write preference

2020-02-18 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039667#comment-17039667 ] Ayush Saxena commented on HDFS-13739: - Committed to trunk. Thanx [~vinayakumarb] for

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039666#comment-17039666 ] Jinglun commented on HDFS-15176: Hi [~elgoiri], thanks your comments !  Upload v03 using

[jira] [Updated] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15176: --- Attachment: HDFS-15176.003.patch > Enable GcTimePercentage Metric in NameNode's JvmMetrics. > ---

[jira] [Comment Edited] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039645#comment-17039645 ] Siddharth Wagle edited comment on HDFS-15154 at 2/19/20 2:15 AM: --

[jira] [Commented] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039645#comment-17039645 ] Siddharth Wagle commented on HDFS-15154: Patch version 03: Addressed review comme

[jira] [Updated] (HDFS-15154) Allow only hdfs superusers the ability to assign HDFS storage policies

2020-02-18 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle updated HDFS-15154: --- Attachment: HDFS-15154.03.patch > Allow only hdfs superusers the ability to assign HDFS stora

[jira] [Updated] (HDFS-15165) In Du missed calling getAttributesProvider

2020-02-18 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-15165: -- Attachment: HDFS-15165.01.patch > In Du missed calling getAttributesProvider >

[jira] [Commented] (HDFS-15165) In Du missed calling getAttributesProvider

2020-02-18 Thread Bharat Viswanadham (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039643#comment-17039643 ] Bharat Viswanadham commented on HDFS-15165: --- Thank You [~sodonnell] for the rev

[jira] [Updated] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Lisheng Sun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lisheng Sun updated HDFS-15182: --- Attachment: HDFS-15182-001.patch Status: Patch Available (was: Open) > TestBlockManager#test

[jira] [Created] (HDFS-15182) TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk

2020-02-18 Thread Lisheng Sun (Jira)
Lisheng Sun created HDFS-15182: -- Summary: TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk Key: HDFS-15182 URL: https://issues.apache.org/jira/browse/HDFS-15182 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039586#comment-17039586 ] Hadoop QA commented on HDFS-15181: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039532#comment-17039532 ] Kihwal Lee commented on HDFS-15181: --- [~weichiu] My patch here works, but want to get ri

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039493#comment-17039493 ] Wei-Chiu Chuang commented on HDFS-15181: Incidentally, we just hit this issue tod

[jira] [Updated] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15181: -- Target Version/s: 3.3.0, 3.1.4, 3.2.2, 2.10.1 (was: 2.10.1) Status: Patch Available (was

[jira] [Updated] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15181: -- Attachment: HDFS-15181.trunk.patch > Webhdfs getTrashRoot() causes internal AccessControlException > --

[jira] [Assigned] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-15181: - Assignee: Kihwal Lee > Webhdfs getTrashRoot() causes internal AccessControlException > -

[jira] [Commented] (HDFS-15115) Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug

2020-02-18 Thread Yuval Degani (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039463#comment-17039463 ] Yuval Degani commented on HDFS-15115: - [~ayushtkn], [~wzx513] thanks for the fix. The

[jira] [Commented] (HDFS-14861) Reset LowRedundancyBlocks Iterator periodically

2020-02-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039407#comment-17039407 ] Wei-Chiu Chuang commented on HDFS-14861: +1 The patch looks good and it's sit for

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039367#comment-17039367 ] Kihwal Lee commented on HDFS-15181: --- Discussed with [~daryn]. {{getTrashRoot()}} is a u

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039354#comment-17039354 ] Kihwal Lee commented on HDFS-15181: --- Also, even if it somehow pass the security, the na

[jira] [Commented] (HDFS-10756) Expose getTrashRoot to HTTPFS and WebHDFS

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039317#comment-17039317 ] Kihwal Lee commented on HDFS-10756: --- Filed HDFS-15181. > Expose getTrashRoot to HTTPFS

[jira] [Commented] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039314#comment-17039314 ] Kihwal Lee commented on HDFS-15181: --- This is what is logged in the namenode log. {nofor

[jira] [Updated] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15181: -- Description: HDFS-10756 added the {{getTrashRoot()}} support for WebHdfs. However, it was done by crea

[jira] [Updated] (HDFS-15181) Webhdfs getTrashRoot() causes internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-15181: -- Summary: Webhdfs getTrashRoot() causes internal AccessControlException (was: Webhdfs getTrashRoot() ca

[jira] [Created] (HDFS-15181) Webhdfs getTrashRoot() caused internal AccessControlException

2020-02-18 Thread Kihwal Lee (Jira)
Kihwal Lee created HDFS-15181: - Summary: Webhdfs getTrashRoot() caused internal AccessControlException Key: HDFS-15181 URL: https://issues.apache.org/jira/browse/HDFS-15181 Project: Hadoop HDFS

[jira] [Commented] (HDFS-10756) Expose getTrashRoot to HTTPFS and WebHDFS

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039289#comment-17039289 ] Kihwal Lee commented on HDFS-10756: --- To be clear, this is not a performance bug, but a

[jira] [Commented] (HDFS-10756) Expose getTrashRoot to HTTPFS and WebHDFS

2020-02-18 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039263#comment-17039263 ] Kihwal Lee commented on HDFS-10756: --- As Daryn said, this needs to be reverted and redon

[jira] [Commented] (HDFS-15167) Block Report Interval shouldn't be reset apart from first Block Report

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039251#comment-17039251 ] Hadoop QA commented on HDFS-15167: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Updated] (HDFS-13739) Add option to disable rack local write preference

2020-02-18 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-13739: Summary: Add option to disable rack local write preference (was: Option to disable Rack Local Writ

[jira] [Commented] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2020-02-18 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039209#comment-17039209 ] Stephen O'Donnell commented on HDFS-15180: -- We have some work on going related t

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039207#comment-17039207 ] Hadoop QA commented on HDFS-15176: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-15167) Block Report Interval shouldn't be reset apart from first Block Report

2020-02-18 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039204#comment-17039204 ] Íñigo Goiri commented on HDFS-15167: +1 on [^HDFS-15167-08.patch]. > Block Report

[jira] [Commented] (HDFS-15172) Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes()

2020-02-18 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039202#comment-17039202 ] Íñigo Goiri commented on HDFS-15172: OK, let's do this first then. +1 on [^HDFS-1517

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039198#comment-17039198 ] Íñigo Goiri commented on HDFS-15176: For maintainability, let's use conf.getTimeDurat

[jira] [Commented] (HDFS-13739) Option to disable Rack Local Write Preference to avoid 2 issues - 1. Rack-by-Rack Maintenance leaves last data replica at risk, 2. avoid Major Storage Imbalance across

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039173#comment-17039173 ] Hadoop QA commented on HDFS-13739: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039169#comment-17039169 ] Stephen O'Donnell commented on HDFS-15177: -- You posted just about the same time

[jira] [Commented] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039167#comment-17039167 ] zhuqi commented on HDFS-15180: -- cc [~sodonnell] ,  [~linyiqun], [~weichiu] , [~hexiaoqiao] 

[jira] [Updated] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15180: - Description: Now the FsDatasetImpl datasetLock is heavy, when their are many namespaces in big cluster. If we can

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039165#comment-17039165 ] zhuqi #ommented on HDFS-15177: -- cc [~sodonnell] ,   Not the trunk version, my version is ha

[jira] [Created] (HDFS-15180) DataNode FsDatasetImpl Fine-Grained Locking via BlockPool.

2020-02-18 Thread zhuqi (Jira)
zhuqi created HDFS-15180: Summary: DataNode FsDatasetImpl Fine-Grained Locking via BlockPool. Key: HDFS-15180 URL: https://issues.apache.org/jira/browse/HDFS-15180 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039156#comment-17039156 ] Stephen O'Donnell commented on HDFS-15177: -- In the first screen shot you shared,

[jira] [Comment Edited] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039146#comment-17039146 ] zhuqi edited comment on HDFS-15177 at 2/18/20 2:56 PM: --- cc [~sodonn

[jira] [Commented] (HDFS-15120) Refresh BlockPlacementPolicy at runtime.

2020-02-18 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039147#comment-17039147 ] Hadoop QA commented on HDFS-15120: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039146#comment-17039146 ] zhuqi commented on HDFS-15177: -- cc [~sodonnell] ,  Thanks for your reply. Our cluster' dele

[jira] [Updated] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15177: - Attachment: image-2020-02-18-22-52-59-202.png > Split datanode invalide block deletion, to avoid the FsDatasetImp

[jira] [Updated] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15177: - Attachment: image-2020-02-18-22-51-28-624.png > Split datanode invalide block deletion, to avoid the FsDatasetImp

[jira] [Updated] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15177: - Attachment: image-2020-02-18-22-39-00-642.png > Split datanode invalide block deletion, to avoid the FsDatasetImp

[jira] [Commented] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039051#comment-17039051 ] Jinglun commented on HDFS-15176: Upload v02 fixing unit test failure > Enable GcTimePerc

[jira] [Updated] (HDFS-15176) Enable GcTimePercentage Metric in NameNode's JvmMetrics.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15176: --- Attachment: HDFS-15176.002.patch > Enable GcTimePercentage Metric in NameNode's JvmMetrics. > ---

[jira] [Commented] (HDFS-15171) Add a thread to call saveDfsUsed periodically, to prevent datanode too long restart time.

2020-02-18 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039018#comment-17039018 ] Stephen O'Donnell commented on HDFS-15171: -- [~zhuqi] There are a few parts to th

[jira] [Updated] (HDFS-15120) Refresh BlockPlacementPolicy at runtime.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15120: --- Attachment: HDFS-15120.004.patch > Refresh BlockPlacementPolicy at runtime. > ---

[jira] [Commented] (HDFS-15120) Refresh BlockPlacementPolicy at runtime.

2020-02-18 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039006#comment-17039006 ] Jinglun commented on HDFS-15120: Upload v04 fix checkstyle. > Refresh BlockPlacementPoli

[jira] [Commented] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038993#comment-17038993 ] Stephen O'Donnell commented on HDFS-15177: -- Hi [~zhuqi], do you have any jstack

[jira] [Commented] (HDFS-13739) Option to disable Rack Local Write Preference to avoid 2 issues - 1. Rack-by-Rack Maintenance leaves last data replica at risk, 2. avoid Major Storage Imbalance across

2020-02-18 Thread Vinayakumar B (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038990#comment-17038990 ] Vinayakumar B commented on HDFS-13739: -- +1 > Option to disable Rack Local Write Pre

[jira] [Created] (HDFS-15179) [Federation] Add missing REST APIs for Router

2020-02-18 Thread Bilwa S T (Jira)
Bilwa S T created HDFS-15179: Summary: [Federation] Add missing REST APIs for Router Key: HDFS-15179 URL: https://issues.apache.org/jira/browse/HDFS-15179 Project: Hadoop HDFS Issue Type: Improve

[jira] [Updated] (HDFS-15177) Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too much time.

2020-02-18 Thread zhuqi (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuqi updated HDFS-15177: - Component/s: datanode > Split datanode invalide block deletion, to avoid the FsDatasetImpl lock too > much time.

[jira] [Updated] (HDFS-12766) Ozone: Ensure usage of parameterized slf4j log syntax for ozone

2020-02-18 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDFS-12766: --- Parent: (was: HDFS-12761) Issue Type: Improvement (was: Sub-task) > Ozone: Ensure usage