[GitHub] [hadoop] yuw1 commented on a diff in pull request #6110: HDFS-17209. Correct comments to align with the code

2023-09-29 Thread via GitHub
yuw1 commented on code in PR #6110: URL: https://github.com/apache/hadoop/pull/6110#discussion_r1341913353 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -2569,7 +2569,7 @@ public void shutdown() { if

[GitHub] [hadoop] hadoop-yetus commented on pull request #6127: YARN-11580. YARN Router Web supports displaying information for Non-Federation.

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6127: URL: https://github.com/apache/hadoop/pull/6127#issuecomment-1741642495 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
ayushtkn commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741637747 This fix was fixing those tests, so when we reverted that, they started failing again. for the fix, @zhengchenyu I think we should update the PR reverting the source fix and

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6120: YARN-11578. Cache fs supports chmod in LogAggregationFileController.

2023-09-29 Thread via GitHub
slfan1989 commented on code in PR #6120: URL: https://github.com/apache/hadoop/pull/6120#discussion_r1341880623 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/LogAggregationFileController.java: ## @@

[GitHub] [hadoop] zhengchenyu commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
zhengchenyu commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741619676 > Another observation, the day this PR was reverted, some tests which were broken since eternity broke again.

[GitHub] [hadoop] zhengchenyu commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
zhengchenyu commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741613920 @ayushtkn @virajjasani In my opinion, I think CLA should use FileAppender, and CRLA should use RollingFileAppender. FileAppender will disable the cap. Then CRLA will

[GitHub] [hadoop] zhengchenyu commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
zhengchenyu commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741612979 > ``` > 0 disables the cap. > ``` @ayushtkn Thanks for you reply! log4j.appender.CLA change s from 'org.apache.hadoop.yarn.ContainerLogAppender' to

[GitHub] [hadoop] slfan1989 commented on pull request #6127: YARN-11580. YARN Router Web supports displaying information for Non-Federation.

2023-09-29 Thread via GitHub
slfan1989 commented on PR #6127: URL: https://github.com/apache/hadoop/pull/6127#issuecomment-1741603421 - Router We don't need to make changes to Router's configuration. By configuring a specific YARN cluster in Router's yarn-site.xml, Router can automatically route client

[GitHub] [hadoop-site] simbadzina merged pull request #45: Adds Simbarashe Dzinamarira to Who page.

2023-09-29 Thread via GitHub
simbadzina merged PR #45: URL: https://github.com/apache/hadoop-site/pull/45 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop-site] simbadzina opened a new pull request, #45: Adds Simbarashe Dzinamarira to Who page.

2023-09-29 Thread via GitHub
simbadzina opened a new pull request, #45: URL: https://github.com/apache/hadoop-site/pull/45 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [hadoop] ayushtkn commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
ayushtkn commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741258883 Another observation, the day this PR was reverted, some tests which were broken since eternity broke again. https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1355/

[GitHub] [hadoop] ayushtkn commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
ayushtkn commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741254621 Thanx @zhengchenyu exploring a bit more. you removed this line ``` 0 disables the cap. ``` Now how does this change restores that behaviour? You did set a cap, now if I

[jira] [Commented] (HADOOP-18857) AWS v2 SDK: fail meaningfully if legacy v2 signing requested

2023-09-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770526#comment-17770526 ] Steve Loughran commented on HADOOP-18857: - the test is added. now InstantiationIOException is

[GitHub] [hadoop] virajjasani commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
virajjasani commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741207214 On the other hand, as a second option, since this is now going to be an incompatible change (change of default value) and mostly we will not have much progress on log4j2 migration,

[GitHub] [hadoop] hadoop-yetus commented on pull request #6129: YARN-11581. Fix compilation error in hadoop-yarn-applications-catalog-webapp due to unresolved dependency on hamcrest.

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6129: URL: https://github.com/apache/hadoop/pull/6129#issuecomment-1741203641 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] sodonnel commented on pull request #6126: HDFS-16966. Add threshold to avoid hot disk in AvailableSpaceVolumeCh…

2023-09-29 Thread via GitHub
sodonnel commented on PR #6126: URL: https://github.com/apache/hadoop/pull/6126#issuecomment-1741169561 Does `dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction` not allow this to be controlled? The default is 75% of new writes go to disks with more

[GitHub] [hadoop] hadoop-yetus commented on pull request #5988: HADOOP-18867. Upgrade ZooKeeper to 3.6.4.

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1741135337 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770496#comment-17770496 ] ASF GitHub Bot commented on HADOOP-18867: - hadoop-yetus commented on PR #5988: URL:

[GitHub] [hadoop] virajjasani commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
virajjasani commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741114265 Thank you for the explanation @zhengchenyu The changes look good IMO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] zhengchenyu commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-29 Thread via GitHub
zhengchenyu commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1741028712 @virajjasani Thanks for your reply. For CLA before HADOOP-18649: CLA is org.apache.hadoop.yarn.ContainerLogAppender which is extend from FileAppender. In default,

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770470#comment-17770470 ] ASF GitHub Bot commented on HADOOP-18867: - iwasakims commented on PR #5988: URL:

[GitHub] [hadoop] iwasakims commented on pull request #5988: HADOOP-18867. Upgrade ZooKeeper to 3.6.4.

2023-09-29 Thread via GitHub
iwasakims commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1741014142 I rebased this on current trunk given YARN-11558 is merged. I ran tests related to hbase with `-Dhbase.profile=2.0`. ``` $ mvn clean install -DskipTests -DskipShade

[GitHub] [hadoop] iwasakims opened a new pull request, #6129: YARN-11581. Fix compilation error in hadoop-yarn-applications-catalog-webapp due to unresolved dependency on hamcrest.

2023-09-29 Thread via GitHub
iwasakims opened a new pull request, #6129: URL: https://github.com/apache/hadoop/pull/6129 https://issues.apache.org/jira/browse/YARN-11581 ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:testCompile (default-testCompile) on project

[jira] [Commented] (HADOOP-16062) Add ability to disable Configuration reload registry

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770425#comment-17770425 ] ASF GitHub Bot commented on HADOOP-16062: - hadoop-yetus commented on PR #6128: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6128: HADOOP-16062. Add ability to disable Configuration reload registry

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6128: URL: https://github.com/apache/hadoop/pull/6128#issuecomment-1740855133 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18890) remove okhttp usage

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770417#comment-17770417 ] ASF GitHub Bot commented on HADOOP-18890: - pjfanning commented on PR #6101: URL:

[GitHub] [hadoop] pjfanning commented on pull request #6101: HADOOP-18890. Remove use of okhttp in runtime code (#6057)

2023-09-29 Thread via GitHub
pjfanning commented on PR #6101: URL: https://github.com/apache/hadoop/pull/6101#issuecomment-1740825126 @steveloughran this PR now seems to just have failing tests - test failures that I think are not related to the PR changes -- This is an automated message from the Apache Git Service.

[jira] [Commented] (HADOOP-18890) remove okhttp usage

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770402#comment-17770402 ] ASF GitHub Bot commented on HADOOP-18890: - hadoop-yetus commented on PR #6101: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6101: HADOOP-18890. Remove use of okhttp in runtime code (#6057)

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6101: URL: https://github.com/apache/hadoop/pull/6101#issuecomment-1740783324 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6127: YARN-11580. YARN Router Web supports displaying information for Non-Federation.

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6127: URL: https://github.com/apache/hadoop/pull/6127#issuecomment-1740757062 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-16062) Add ability to disable Configuration reload registry

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-16062: Labels: pull-request-available (was: ) > Add ability to disable Configuration reload

[jira] [Commented] (HADOOP-16062) Add ability to disable Configuration reload registry

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770391#comment-17770391 ] ASF GitHub Bot commented on HADOOP-16062: - kgyrtkirk opened a new pull request, #6128: URL:

[GitHub] [hadoop] kgyrtkirk opened a new pull request, #6128: HADOOP-16062. Add ability to disable Configuration reload registry

2023-09-29 Thread via GitHub
kgyrtkirk opened a new pull request, #6128: URL: https://github.com/apache/hadoop/pull/6128 ### Description of PR see https://issues.apache.org/jira/browse/HADOOP-16062 ### How was this patch tested? unit test ### For code changes: - [ ] Does the

[jira] [Commented] (HADOOP-18885) Add rpcRejectedByObserverCalls metric to quantify the number of rejected RPCs by Observer NameNode

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770378#comment-17770378 ] ASF GitHub Bot commented on HADOOP-18885: - haiyang1987 commented on PR #6043: URL:

[GitHub] [hadoop] haiyang1987 commented on pull request #6043: HADOOP-18885. Add rpcCallsRejectedByObserver metric to quantify the number of rejected RPCs by Observer NameNode

2023-09-29 Thread via GitHub
haiyang1987 commented on PR #6043: URL: https://github.com/apache/hadoop/pull/6043#issuecomment-1740714667 @goiri Hi Sir, Could you please help me push this modification forward when you have free time ? Thanks a lot~ -- This is an automated message from the Apache Git Service.

[jira] [Commented] (HADOOP-18868) Optimize the configuration and use of callqueue overflow trigger failover

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770376#comment-17770376 ] ASF GitHub Bot commented on HADOOP-18868: - haiyang1987 commented on PR #5998: URL:

[GitHub] [hadoop] haiyang1987 commented on pull request #5998: HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover

2023-09-29 Thread via GitHub
haiyang1987 commented on PR #5998: URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1740711739 Hi Sirs @ayushtkn can you help me take a look at this PR when you have time? Thank you very much~ -- This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HADOOP-18915) HTTP timeouts are not set correctly

2023-09-29 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail updated HADOOP-18915: -- Parent Issue: HADOOP-18886 (was: HADOOP-18073) > HTTP timeouts are not set correctly >

[jira] [Created] (HADOOP-18915) HTTP timeouts are not set correctly

2023-09-29 Thread Ahmar Suhail (Jira)
Ahmar Suhail created HADOOP-18915: - Summary: HTTP timeouts are not set correctly Key: HADOOP-18915 URL: https://issues.apache.org/jira/browse/HADOOP-18915 Project: Hadoop Common Issue Type:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6126: HDFS-16966. Add threshold to avoid hot disk in AvailableSpaceVolumeCh…

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6126: URL: https://github.com/apache/hadoop/pull/6126#issuecomment-1740618431 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #6121: YARN-11537. [Addendum][Federation] Router CLI Supports List SubClusterPolicyConfiguration Of Queues.

2023-09-29 Thread via GitHub
slfan1989 commented on PR #6121: URL: https://github.com/apache/hadoop/pull/6121#issuecomment-1740550117 @goiri Thank you very much for your help in reviewing the code! I will continue to improve the function of router command. I will merge this PR into the trunk branch. -- This is an

[GitHub] [hadoop] slfan1989 merged pull request #6121: YARN-11537. [Addendum][Federation] Router CLI Supports List SubClusterPolicyConfiguration Of Queues.

2023-09-29 Thread via GitHub
slfan1989 merged PR #6121: URL: https://github.com/apache/hadoop/pull/6121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5996: HDFS-17171. CONGESTION_RATIO should be configurable

2023-09-29 Thread via GitHub
ayushtkn commented on code in PR #5996: URL: https://github.com/apache/hadoop/pull/5996#discussion_r1340969835 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java: ## @@ -614,6 +616,8 @@ public Map load(String key) {

[GitHub] [hadoop] ayushtkn commented on pull request #5825: HDFS-17072. DFSAdmin: add a triggerDirectoryScanner command.

2023-09-29 Thread via GitHub
ayushtkn commented on PR #5825: URL: https://github.com/apache/hadoop/pull/5825#issuecomment-1740387388 There is a test failure for directory scanner itself, can you check? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] ayushtkn merged pull request #6079: HDFS-17133. TestFsDatasetImpl missing null check when cleaning up

2023-09-29 Thread via GitHub
ayushtkn merged PR #6079: URL: https://github.com/apache/hadoop/pull/6079 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] ayushtkn merged pull request #6124: HDFS-17211. RBF: The comments in the RemoteParam class are incorrect.

2023-09-29 Thread via GitHub
ayushtkn merged PR #6124: URL: https://github.com/apache/hadoop/pull/6124 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs

2023-09-29 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770306#comment-17770306 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on code in PR #6000: URL:

[GitHub] [hadoop] mehakmeet commented on a diff in pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-29 Thread via GitHub
mehakmeet commented on code in PR #6000: URL: https://github.com/apache/hadoop/pull/6000#discussion_r1340942956 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java: ## @@ -0,0 +1,248 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hadoop] hadoop-yetus commented on pull request #6127: YARN-11580. YARN Router Web supports displaying information for Non-Federation.

2023-09-29 Thread via GitHub
hadoop-yetus commented on PR #6127: URL: https://github.com/apache/hadoop/pull/6127#issuecomment-1740368803 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |