[jira] [Commented] (HDFS-15346) DistCpFedBalance implementation

2020-06-15 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136267#comment-17136267 ] Jinglun commented on HDFS-15346: Hi [~linyiqun], thanks your great comments ! Address the comments in

[jira] [Updated] (HDFS-15346) DistCpFedBalance implementation

2020-06-15 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15346: --- Attachment: HDFS-15346.012.patch > DistCpFedBalance implementation > --- > >

[jira] [Commented] (HDFS-15294) Federation balance tool

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136263#comment-17136263 ] Yiqun Lin commented on HDFS-15294: -- As this feature tool is designed as a common tool like distcp, I

[jira] [Updated] (HDFS-15410) Add separated config file fedbalance-default.xml for fedbalance tool

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15410: - Summary: Add separated config file fedbalance-default.xml for fedbalance tool (was: RBF: Add separated

[jira] [Updated] (HDFS-15374) Add documentation for fedbalance tool

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15374: - Summary: Add documentation for fedbalance tool (was: RBF: Add documentation for fedbalance tool) > Add

[jira] [Updated] (HDFS-15346) DistCpFedBalance implementation

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15346: - Summary: DistCpFedBalance implementation (was: RBF: DistCpFedBalance implementation) >

[jira] [Updated] (HDFS-15294) Federation balance tool

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15294: - Summary: Federation balance tool (was: RBF: Balance data across federation namespaces with DistCp and

[jira] [Updated] (HDFS-15414) java.net.SocketException: Original Exception : java.io.IOException: Broken pipe

2020-06-15 Thread YCozy (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YCozy updated HDFS-15414: - Description: We observed this exception in a DataNode's log while we are not shutting down any nodes in the

[jira] [Updated] (HDFS-15414) java.net.SocketException: Original Exception : java.io.IOException: Broken pipe

2020-06-15 Thread YCozy (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YCozy updated HDFS-15414: - Description: We observed this exception in a DataNode's log while we are not shutting down any nodes in the

[jira] [Created] (HDFS-15414) java.net.SocketException: Original Exception : java.io.IOException: Broken pipe

2020-06-15 Thread YCozy (Jira)
YCozy created HDFS-15414: Summary: java.net.SocketException: Original Exception : java.io.IOException: Broken pipe Key: HDFS-15414 URL: https://issues.apache.org/jira/browse/HDFS-15414 Project: Hadoop HDFS

[jira] [Created] (HDFS-15413) DFSStripedInputStream throws exception when datanodes close idle connections

2020-06-15 Thread Andrey Elenskiy (Jira)
Andrey Elenskiy created HDFS-15413: -- Summary: DFSStripedInputStream throws exception when datanodes close idle connections Key: HDFS-15413 URL: https://issues.apache.org/jira/browse/HDFS-15413

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-15 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136060#comment-17136060 ] Hadoop QA commented on HDFS-15406: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-15 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-15406: - Attachment: HDFS-15406.001.patch Status: Patch Available (was: Open) > Improve the

[jira] [Commented] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-15 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135895#comment-17135895 ] Hadoop QA commented on HDFS-15412: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-15346) RBF: DistCpFedBalance implementation

2020-06-15 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135841#comment-17135841 ] Hadoop QA commented on HDFS-15346: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135832#comment-17135832 ] Hadoop QA commented on HDFS-15175: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-15410) RBF: Add separated config file fedbalance-default.xml for fedbalance tool

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15410: - Summary: RBF: Add separated config file fedbalance-default.xml for fedbalance tool (was: Add separated

[jira] [Commented] (HDFS-15346) RBF: DistCpFedBalance implementation

2020-06-15 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135819#comment-17135819 ] Yiqun Lin commented on HDFS-15346: -- [~LiJinglun], the refactor looks great. I find you decrease the

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-15 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135767#comment-17135767 ] hemanthboyina commented on HDFS-15406: -- thanks [~brahmareddy] for the comment {quote}Not sure,

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-15 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135756#comment-17135756 ] hemanthboyina commented on HDFS-15406: -- discussed with [~pilchard] offline  , the major drawback in

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-15 Thread Brahma Reddy Battula (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135742#comment-17135742 ] Brahma Reddy Battula commented on HDFS-15406: - {quote}we get the datanode jstack, with 11M

[jira] [Commented] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135722#comment-17135722 ] Xiaoqiao He commented on HDFS-15175: Thanks [~wanchang] for the patch. It is almost LGTM, adding ut

[jira] [Commented] (HDFS-15391) Standby NameNode due loads the corruption edit log, the service exits and cannot be restarted

2020-06-15 Thread huhaiyang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135712#comment-17135712 ] huhaiyang commented on HDFS-15391: -- Thanks [~hexiaoqiao] To help solve. > Standby NameNode due loads

[jira] [Commented] (HDFS-15391) Standby NameNode due loads the corruption edit log, the service exits and cannot be restarted

2020-06-15 Thread huhaiyang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135698#comment-17135698 ] huhaiyang commented on HDFS-15391: -- [~liuml07] Thank you for reply! The current issue is the same as

[jira] [Updated] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Wan Chang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wan Chang updated HDFS-15175: - Attachment: HDFS-15175-trunk.1.patch Labels: NameNode (was: ) Status: Patch Available

[jira] [Comment Edited] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135675#comment-17135675 ] Xiaoqiao He edited comment on HDFS-15175 at 6/15/20, 9:43 AM: -- Add

[jira] [Comment Edited] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135675#comment-17135675 ] Xiaoqiao He edited comment on HDFS-15175 at 6/15/20, 9:42 AM: -- Add

[jira] [Resolved] (HDFS-15391) Standby NameNode due loads the corruption edit log, the service exits and cannot be restarted

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He resolved HDFS-15391. Resolution: Duplicate > Standby NameNode due loads the corruption edit log, the service exits and

[jira] [Commented] (HDFS-15391) Standby NameNode due loads the corruption edit log, the service exits and cannot be restarted

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135683#comment-17135683 ] Xiaoqiao He commented on HDFS-15391: This issue is duplicated by HDFS-15175, will close this one.

[jira] [Commented] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135675#comment-17135675 ] Xiaoqiao He commented on HDFS-15175: Add [~wanchang] as contributor and assign this issue to him. >

[jira] [Assigned] (HDFS-15175) Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog

2020-06-15 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HDFS-15175: -- Assignee: Wan Chang (was: Yicong Cai) > Multiple CloseOp shared block instance causes the

[jira] [Updated] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-15 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Summary: Add options to set different block scan period for diffrent StorageType (was: Add options to set

[jira] [Updated] (HDFS-15412) Add options to set different scan period for diffrent StorageType

2020-06-15 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Attachment: HDFS-15412.001.patch Status: Patch Available (was: Open) > Add options to set

[jira] [Created] (HDFS-15412) Add options to set different scan period for diffrent StorageType

2020-06-15 Thread Yang Yun (Jira)
Yang Yun created HDFS-15412: --- Summary: Add options to set different scan period for diffrent StorageType Key: HDFS-15412 URL: https://issues.apache.org/jira/browse/HDFS-15412 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15346) RBF: DistCpFedBalance implementation

2020-06-15 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135615#comment-17135615 ] Jinglun commented on HDFS-15346: Your are genius [~linyiqun] !  Thanks your brilliant comments, the 

[jira] [Updated] (HDFS-15346) RBF: DistCpFedBalance implementation

2020-06-15 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15346: --- Attachment: HDFS-15346.011.patch > RBF: DistCpFedBalance implementation >

[jira] [Commented] (HDFS-15411) TestNameNodeMXBean.testDecommissioningNodes fails

2020-06-15 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135610#comment-17135610 ] Ayush Saxena commented on HDFS-15411: - Thanx [~aajisaka] for reporting. I tried to reproduce this by

[jira] [Commented] (HDFS-15411) TestNameNodeMXBean.testDecommissioningNodes fails

2020-06-15 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135448#comment-17135448 ] Akira Ajisaka commented on HDFS-15411: -- It seems that the value of lastBlockReport has been updated

[jira] [Created] (HDFS-15411) TestNameNodeMXBean.testDecommissioningNodes fails

2020-06-15 Thread Akira Ajisaka (Jira)
Akira Ajisaka created HDFS-15411: Summary: TestNameNodeMXBean.testDecommissioningNodes fails Key: HDFS-15411 URL: https://issues.apache.org/jira/browse/HDFS-15411 Project: Hadoop HDFS Issue