[GitHub] [hadoop] brahmareddybattula commented on pull request #3200: HDFS-15160. branch-3.2. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlo

2021-09-05 Thread GitBox
brahmareddybattula commented on pull request #3200: URL: https://github.com/apache/hadoop/pull/3200#issuecomment-913357847 @amahussein thanks for working on this.. At first glance,it's better to have seperate PR for Each jira backport.It can be easy to review and maintain in future. --

[GitHub] [hadoop] jianghuazhu commented on pull request #3368: HDFS-16204.Improve FSDirEncryptionZoneOp related parameter comments.

2021-09-05 Thread GitBox
jianghuazhu commented on pull request #3368: URL: https://github.com/apache/hadoop/pull/3368#issuecomment-913343592 @ferhui , can you help me check the code? If there is something to be corrected, I will continue to work. Thank you very much. -- This is an automated message

[jira] [Commented] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410305#comment-17410305 ] Max Xie commented on HADOOP-17893: --- [~aajisaka] Thank you for your review.  HADOOP-17804 it looks

[jira] [Work logged] (HADOOP-17804) Prometheus metrics only include the last set of labels

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17804?focusedWorklogId=646732=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646732 ] ASF GitHub Bot logged work on HADOOP-17804: --- Author: ASF GitHub Bot

[GitHub] [hadoop] Neilxzn commented on pull request #3369: HADOOP-17804. Expose prometheus metrics only after a flush and dedupe with tag values

2021-09-05 Thread GitBox
Neilxzn commented on pull request #3369: URL: https://github.com/apache/hadoop/pull/3369#issuecomment-913296041 LGTM + 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] tomscut commented on pull request #3378: HDFS-16209. Set dfs.namenode.caching.enabled to false as default

2021-09-05 Thread GitBox
tomscut commented on pull request #3378: URL: https://github.com/apache/hadoop/pull/3378#issuecomment-913278183 @ayushtkn @ferhui @virajjasani Thank you very much for your comments and suggestions. I think what you are saying is reasonable, we should not change the default value of this

[jira] [Commented] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410265#comment-17410265 ] Akira Ajisaka commented on HADOOP-17893: Thank you [~Max Xie] for the report and the patch. It

[jira] [Work logged] (HADOOP-17804) Prometheus metrics only include the last set of labels

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17804?focusedWorklogId=646726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646726 ] ASF GitHub Bot logged work on HADOOP-17804: --- Author: ASF GitHub Bot

[GitHub] [hadoop] aajisaka commented on pull request #3369: HADOOP-17804. Expose prometheus metrics only after a flush and dedupe with tag values

2021-09-05 Thread GitBox
aajisaka commented on pull request #3369: URL: https://github.com/apache/hadoop/pull/3369#issuecomment-913268547 @Kimahriman Thank you for your contribution! The patch looks good to me. Would you fix the check style issues?

[GitHub] [hadoop] GauthamBanasandra opened a new pull request #3388: HDFS-16205. Make hdfs_allowSnapshot tool cross platform

2021-09-05 Thread GitBox
GauthamBanasandra opened a new pull request #3388: URL: https://github.com/apache/hadoop/pull/3388 ### Description of PR * The source files for hdfs_allowSnapshot uses getopt for parsing the command line arguments. * getopt is available only on Linux and thus,

[GitHub] [hadoop] virajjasani commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
virajjasani commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913189856 @aajisaka @ayushtkn @jojochuang @tasanuma could you please take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread Renukaprasad C (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410204#comment-17410204 ] Renukaprasad C commented on HADOOP-17892: - Thanks [~vjasani] for the issue. Let's wait for

[GitHub] [hadoop] hadoop-yetus commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913187495 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Work logged] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?focusedWorklogId=646705=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646705 ] ASF GitHub Bot logged work on HADOOP-17892: --- Author: ASF GitHub Bot

[GitHub] [hadoop] prasad-acit commented on pull request #3387: HADOOP-17892. Add Hadoop code formatter in dev-support

2021-09-05 Thread GitBox
prasad-acit commented on pull request #3387: URL: https://github.com/apache/hadoop/pull/3387#issuecomment-913186322 This is good to have and easy to maintain the formatter changes with in the code base. @steveloughran/ @surendralilhore Is it ok to maintain here or any alternative you

[GitHub] [hadoop] hadoop-yetus commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913179678 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913179175 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913163730 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] hadoop-yetus commented on pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3386: URL: https://github.com/apache/hadoop/pull/3386#issuecomment-913148446 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[jira] [Updated] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Xie updated HADOOP-17893: -- Fix Version/s: 3.4.0 > Improve PrometheusSink for Namenode and ResourceManager Metrics >

[jira] [Updated] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Xie updated HADOOP-17893: -- Attachment: HADOOP-17893.01.patch Status: Patch Available (was: Open) add patch for 

[jira] [Updated] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Xie updated HADOOP-17893: -- Description: HADOOP-16398 added exporter for hadoop metrics to prometheus. But some of metrics

[jira] [Updated] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Xie updated HADOOP-17893: -- Description: HADOOP-16398 added exporter for hadoop metrics to prometheus. But some of metrics

[GitHub] [hadoop] virajjasani commented on pull request #3378: HDFS-16209. Set dfs.namenode.caching.enabled to false as default

2021-09-05 Thread GitBox
virajjasani commented on pull request #3378: URL: https://github.com/apache/hadoop/pull/3378#issuecomment-913134902 Thanks for the reference and find @tomscut ! I agree to the general opinion of not making incompatible change until absolutely required, and on the other hand, this change

[jira] [Updated] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Xie updated HADOOP-17893: -- Description: HADOOP-16398 added exporter for hadoop metrics to prometheus. But some of metrics

[jira] [Created] (HADOOP-17893) Improve PrometheusSink for Namenode and ResourceManager Metrics

2021-09-05 Thread Max Xie (Jira)
Max Xie created HADOOP-17893: - Summary: Improve PrometheusSink for Namenode and ResourceManager Metrics Key: HADOOP-17893 URL: https://issues.apache.org/jira/browse/HADOOP-17893 Project: Hadoop Common

[GitHub] [hadoop] hadoop-yetus commented on pull request #3357: HDFS-16197.Simplify getting NNStorage in FSNamesystem.

2021-09-05 Thread GitBox
hadoop-yetus commented on pull request #3357: URL: https://github.com/apache/hadoop/pull/3357#issuecomment-913133618 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #3359: HDFS-16198. Short circuit read leaks Slot objects when InvalidToken exception is thrown

2021-09-05 Thread GitBox
Hexiaoqiao commented on a change in pull request #3359: URL: https://github.com/apache/hadoop/pull/3359#discussion_r702405022 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithShortCircuitRead.java ##

[GitHub] [hadoop] Hexiaoqiao merged pull request #3244: HDFS-16138. BlockReportProcessingThread exit doesnt print the acutal stack

2021-09-05 Thread GitBox
Hexiaoqiao merged pull request #3244: URL: https://github.com/apache/hadoop/pull/3244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] Hexiaoqiao commented on pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-09-05 Thread GitBox
Hexiaoqiao commented on pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#issuecomment-913114162 @whbing This feature is useful for parse fsimage. It is near to check in when fix the comment. Any plan to push this forward? Thanks. -- This is an automated message from

[jira] [Commented] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17410117#comment-17410117 ] Viraj Jasani commented on HADOOP-17892: --- FYI [~prasad-acit]  Thanks > Add Hadoop code formatter

[jira] [Work logged] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?focusedWorklogId=646683=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646683 ] ASF GitHub Bot logged work on HADOOP-17892: --- Author: ASF GitHub Bot

[jira] [Updated] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17892: Labels: pull-request-available (was: ) > Add Hadoop code formatter in dev-support >

[GitHub] [hadoop] virajjasani opened a new pull request #3387: HADOOP-17892. Add Hadoop code formatter in dev-support

2021-09-05 Thread GitBox
virajjasani opened a new pull request #3387: URL: https://github.com/apache/hadoop/pull/3387 ### Description of PR Adding Hadoop code formatter in dev-support. Once done, we should also update our Wiki to refer to this link directly. -- This is an automated message from the Apache

[jira] [Created] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-17892: - Summary: Add Hadoop code formatter in dev-support Key: HADOOP-17892 URL: https://issues.apache.org/jira/browse/HADOOP-17892 Project: Hadoop Common Issue

[jira] [Assigned] (HADOOP-17892) Add Hadoop code formatter in dev-support

2021-09-05 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-17892: - Assignee: Viraj Jasani > Add Hadoop code formatter in dev-support >

[GitHub] [hadoop] virajjasani opened a new pull request #3386: HDFS-16213. De-flake TestFsDatasetImpl#testDnRestartWithHardLink

2021-09-05 Thread GitBox
virajjasani opened a new pull request #3386: URL: https://github.com/apache/hadoop/pull/3386 ### Description of PR TestFsDatasetImpl#testDnRestartWithHardLink is flapper: ``` [ERROR] testDnRestartWithHardLink(org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl)

[jira] [Work logged] (HADOOP-17891) Lz4 should be excluded from relocation in shaded Hadoop libraries

2021-09-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17891?focusedWorklogId=646681=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-646681 ] ASF GitHub Bot logged work on HADOOP-17891: --- Author: ASF GitHub Bot

[GitHub] [hadoop] viirya commented on pull request #3385: HADOOP-17891. Exclude lz4 from relocation in shaded hadoop client libraries

2021-09-05 Thread GitBox
viirya commented on pull request #3385: URL: https://github.com/apache/hadoop/pull/3385#issuecomment-913100179 @sunchao Thanks. Do you mean a test for codec? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] Hexiaoqiao merged pull request #3354: HDFS-16194. Simplify the code with DatanodeID#getXferAddrWithHostname

2021-09-05 Thread GitBox
Hexiaoqiao merged pull request #3354: URL: https://github.com/apache/hadoop/pull/3354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] jianghuazhu edited a comment on pull request #3380: HDFS-16211.Complete some descriptions related to AuthToken.

2021-09-05 Thread GitBox
jianghuazhu edited a comment on pull request #3380: URL: https://github.com/apache/hadoop/pull/3380#issuecomment-913092071 From the compilation results, some exceptions occur, such as: ERROR: Failed to write github status. Token expired or missing repo:status write? I'm not sure

[GitHub] [hadoop] jianghuazhu edited a comment on pull request #3380: HDFS-16211.Complete some descriptions related to AuthToken.

2021-09-05 Thread GitBox
jianghuazhu edited a comment on pull request #3380: URL: https://github.com/apache/hadoop/pull/3380#issuecomment-913092071 From the compilation results, some exceptions occur, such as: ERROR: Failed to write github status. Token expired or missing repo:status write? I'm not sure if