[jira] [Updated] (HDFS-15374) Add documentation for fedbalance tool

2020-06-17 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15374: --- Attachment: BalanceProcedureScheduler.png > Add documentation for fedbalance tool > -

[jira] [Updated] (HDFS-15374) Add documentation for fedbalance tool

2020-06-17 Thread Jinglun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinglun updated HDFS-15374: --- Attachment: HDFS-15374.002.patch > Add documentation for fedbalance tool > --

[jira] [Updated] (HDFS-15419) Router should retry communicate with NN when cluster is unavailable using configurable time interval

2020-06-17 Thread bhji123 (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bhji123 updated HDFS-15419: --- Description: When cluster is unavailable, router -> namenode communication will only retry once without any

[jira] [Updated] (HDFS-15419) Router should retry communicate with NN when cluster is unavailable using configurable time interval

2020-06-17 Thread bhji123 (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bhji123 updated HDFS-15419: --- Summary: Router should retry communicate with NN when cluster is unavailable using configurable time interval

[jira] [Work started] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15416 started by jianghua zhu. --- > The addStorageLocations() method in the DataStorage class is not perfect. >

[jira] [Work stopped] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15416 stopped by jianghua zhu. --- > The addStorageLocations() method in the DataStorage class is not perfect. >

[jira] [Work started] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15416 started by jianghua zhu. --- > The addStorageLocations() method in the DataStorage class is not perfect. >

[jira] [Commented] (HDFS-15346) FedBalance tool implementation

2020-06-17 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139109#comment-17139109 ] Hudson commented on HDFS-15346: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #

[jira] [Commented] (HDFS-15374) Add documentation for fedbalance tool

2020-06-17 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139105#comment-17139105 ] Yiqun Lin commented on HDFS-15374: -- Hi [~LiJinglun], can you attach the latest patch her

[jira] [Updated] (HDFS-15346) FedBalance tool implementation

2020-06-17 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15346: - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (wa

[jira] [Updated] (HDFS-15346) FedBalance tool implementation

2020-06-17 Thread Yiqun Lin (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-15346: - Summary: FedBalance tool implementation (was: DistCpFedBalance implementation) > FedBalance tool implem

[jira] [Commented] (HDFS-15098) Add SM4 encryption method for HDFS

2020-06-17 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139100#comment-17139100 ] Hadoop QA commented on HDFS-15098: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15419) router retry with configurable time interval when cluster is unavailable

2020-06-17 Thread bhji123 (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17139085#comment-17139085 ] bhji123 commented on HDFS-15419: [https://github.com/apache/hadoop/pull/2082] Here is th

[jira] [Created] (HDFS-15419) router retry with configurable time interval when cluster is unavailable

2020-06-17 Thread bhji123 (Jira)
bhji123 created HDFS-15419: -- Summary: router retry with configurable time interval when cluster is unavailable Key: HDFS-15419 URL: https://issues.apache.org/jira/browse/HDFS-15419 Project: Hadoop HDFS

[jira] [Created] (HDFS-15418) ViewFileSystemOverloadScheme should represent mount links as non symlinks

2020-06-17 Thread Uma Maheswara Rao G (Jira)
Uma Maheswara Rao G created HDFS-15418: -- Summary: ViewFileSystemOverloadScheme should represent mount links as non symlinks Key: HDFS-15418 URL: https://issues.apache.org/jira/browse/HDFS-15418 P

[jira] [Commented] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138966#comment-17138966 ] jianghua zhu commented on HDFS-15416: - [~hemanthboyina] Thank you so much for your s

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Attachment: HDFS-15416.patch > The addStorageLocations() method in the DataStorage class is not per

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Attachment: (was: HDFS-15416.patch) > The addStorageLocations() method in the DataStorage class

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Attachment: HDFS-15416.patch > The addStorageLocations() method in the DataStorage class is not per

[jira] [Updated] (HDFS-15098) Add SM4 encryption method for HDFS

2020-06-17 Thread liusheng (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated HDFS-15098: Attachment: (was: HDFS-15098.007.patch) > Add SM4 encryption method for HDFS >

[jira] [Updated] (HDFS-15098) Add SM4 encryption method for HDFS

2020-06-17 Thread liusheng (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated HDFS-15098: Status: Open (was: Patch Available) > Add SM4 encryption method for HDFS > ---

[jira] [Updated] (HDFS-15098) Add SM4 encryption method for HDFS

2020-06-17 Thread liusheng (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liusheng updated HDFS-15098: Attachment: HDFS-15098.007.patch Status: Patch Available (was: Open) > Add SM4 encryption method f

[jira] [Updated] (HDFS-15417) Lazy get the datanode report for federation WebHDFS operations

2020-06-17 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15417: - Description: *Why* For WebHDFS CREATE, OPEN, APPEND and GETFILECHECKSUM operations, router or namenode needs to

[jira] [Commented] (HDFS-15417) Lazy get the datanode report for federation WebHDFS operations

2020-06-17 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138869#comment-17138869 ] Íñigo Goiri commented on HDFS-15417: I think it makes sense to restrict when we actua

[jira] [Updated] (HDFS-15417) Lazy get the datanode report for federation WebHDFS operations

2020-06-17 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-15417: --- Status: Patch Available (was: Open) > Lazy get the datanode report for federation WebHDFS operations

[jira] [Assigned] (HDFS-15417) Lazy get the datanode report for federation WebHDFS operations

2020-06-17 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni reassigned HDFS-15417: Assignee: Ye Ni [https://github.com/apache/hadoop/pull/2080] > Lazy get the datanode report for federatio

[jira] [Created] (HDFS-15417) Lazy get the datanode report for federation WebHDFS operations

2020-06-17 Thread Ye Ni (Jira)
Ye Ni created HDFS-15417: Summary: Lazy get the datanode report for federation WebHDFS operations Key: HDFS-15417 URL: https://issues.apache.org/jira/browse/HDFS-15417 Project: Hadoop HDFS Issue Typ

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138749#comment-17138749 ] Hadoop QA commented on HDFS-15406: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Assigned] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-15416: -- Assignee: jianghua zhu > The addStorageLocations() method in the DataStorage class is

[jira] [Commented] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138634#comment-17138634 ] hemanthboyina commented on HDFS-15416: -- thanks for filing the issue [~jianghuazhu]

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138627#comment-17138627 ] hemanthboyina commented on HDFS-15406: -- thanks [~sodonnell] , the test failure was r

[jira] [Updated] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-15406: - Attachment: HDFS-15406.002.patch > Improve the speed of Datanode Block Scan > ---

[jira] [Commented] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138484#comment-17138484 ] Hadoop QA commented on HDFS-15412: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138428#comment-17138428 ] Stephen O'Donnell commented on HDFS-15406: -- I looked into the test failures and

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Description: SuccessLocations content is an array, when the number is 0, do not need to be execute

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Description: SuccessLocations content is an array, when the number is 0, do not need to be execute

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Description: SuccessLocations content is an array, when the number is 0, do not need to be execute

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Description: SuccessLocations content is an array, when the number is 0, do not need to be execute

[jira] [Issue Comment Deleted] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Comment: was deleted (was: [link title|[https://github.com/apache/hadoop/pull/2079]]) > The addSto

[jira] [Commented] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138380#comment-17138380 ] jianghua zhu commented on HDFS-15416: - [link title|[https://github.com/apache/hadoop/

[jira] [Commented] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138378#comment-17138378 ] jianghua zhu commented on HDFS-15416: - solve: [https://github.com/apache/hadoop/pull

[jira] [Commented] (HDFS-15098) Add SM4 encryption method for HDFS

2020-06-17 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138360#comment-17138360 ] Hadoop QA commented on HDFS-15098: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Created] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
jianghua zhu created HDFS-15416: --- Summary: The addStorageLocations() method in the DataStorage class is not perfect. Key: HDFS-15416 URL: https://issues.apache.org/jira/browse/HDFS-15416 Project: Hadoop

[jira] [Updated] (HDFS-15416) The addStorageLocations() method in the DataStorage class is not perfect.

2020-06-17 Thread jianghua zhu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianghua zhu updated HDFS-15416: Affects Version/s: 3.1.0 3.1.1 > The addStorageLocations() method in the Dat

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138322#comment-17138322 ] Stephen O'Donnell commented on HDFS-15406: -- Created HDFS-15415 for further inves

[jira] [Created] (HDFS-15415) Reduce locking in Datanode DirectoryScanner

2020-06-17 Thread Stephen O'Donnell (Jira)
Stephen O'Donnell created HDFS-15415: Summary: Reduce locking in Datanode DirectoryScanner Key: HDFS-15415 URL: https://issues.apache.org/jira/browse/HDFS-15415 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138318#comment-17138318 ] Stephen O'Donnell commented on HDFS-15406: -- [~hemanthboyina] Looks like the test

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138282#comment-17138282 ] Stephen O'Donnell commented on HDFS-15406: -- Sounds good. I will try to create th

[jira] [Updated] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Attachment: HDFS-15412.001.patch Status: Patch Available (was: Open) > Add options to set differen

[jira] [Updated] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Status: Open (was: Patch Available) > Add options to set different block scan period for diffrent StorageT

[jira] [Commented] (HDFS-15406) Improve the speed of Datanode Block Scan

2020-06-17 Thread hemanthboyina (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138275#comment-17138275 ] hemanthboyina commented on HDFS-15406: -- thanks [~sodonnell] for the comment {quote}I

[jira] [Commented] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138273#comment-17138273 ] Hadoop QA commented on HDFS-15412: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-15289) Allow viewfs mounts with HDFS/HCFS scheme and centralized mount table

2020-06-17 Thread Uma Maheswara Rao G (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17138178#comment-17138178 ] Uma Maheswara Rao G commented on HDFS-15289: [View FS OverloadScheme User Gui

[jira] [Updated] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Status: Open (was: Patch Available) > Add options to set different block scan period for diffrent StorageT

[jira] [Updated] (HDFS-15412) Add options to set different block scan period for diffrent StorageType

2020-06-17 Thread Yang Yun (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15412: Attachment: HDFS-15412.001.patch Status: Patch Available (was: Open) > Add options to set differen