[GitHub] [hadoop] hadoop-yetus commented on pull request #6101: HADOOP-18890. Remove use of okhttp in runtime code (#6057)

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6101: URL: https://github.com/apache/hadoop/pull/6101#issuecomment-1727004213 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726987006 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1726974509 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726949536 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726912503 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726909124 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
saxenapranav commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1330950502 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java: ## @@ -119,6 +119,7 @@ public final class FileSyst

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1330948781 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java: ## @@ -119,6 +119,7 @@ public final class FileSyst

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1330947774 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1427,28 +1428,45 @@ private void appendIfNotEmpty(StringBu

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1330947631 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1427,28 +1428,45 @@ private void appendIfNotEmpty(StringBu

[GitHub] [hadoop] hadoop-yetus commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726884617 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] Hexiaoqiao commented on pull request #6036: HDFS-17105. mistakenly purge editLogs even after it is empty in NNStorageRetentionManager

2023-09-19 Thread via GitHub
Hexiaoqiao commented on PR #6036: URL: https://github.com/apache/hadoop/pull/6036#issuecomment-1726847843 Committed to trunk. Thanks @teamconfx for your contributions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [hadoop] Hexiaoqiao merged pull request #6036: HDFS-17105. mistakenly purge editLogs even after it is empty in NNStorageRetentionManager

2023-09-19 Thread via GitHub
Hexiaoqiao merged PR #6036: URL: https://github.com/apache/hadoop/pull/6036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@ha

[GitHub] [hadoop] liubin101 commented on pull request #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
liubin101 commented on PR #6100: URL: https://github.com/apache/hadoop/pull/6100#issuecomment-1726832136 @slfan1989 I deployed a router(http://router1:8089) and a subcluster(http://sc1:8088), the config as below: ``` hadoop.http.authentication.type=simple yarn.router.webapp.interce

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #6094: HDFS-17194. Enhance the log message for striped block recovery

2023-09-19 Thread via GitHub
haiyang1987 commented on code in PR #6094: URL: https://github.com/apache/hadoop/pull/6094#discussion_r1330927906 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java: ## @@ -427,26 +426,41 @@ protected void recover() t

[jira] [Assigned] (HADOOP-18208) Remove all the log4j reference in modules other than hadoop-logging

2023-09-19 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-18208: - Assignee: (was: Viraj Jasani) > Remove all the log4j reference in modules other th

[jira] [Assigned] (HADOOP-16206) Migrate from Log4j1 to Log4j2

2023-09-19 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-16206: - Assignee: (was: Viraj Jasani) > Migrate from Log4j1 to Log4j2 > --

[jira] [Assigned] (HADOOP-18207) Introduce hadoop-logging module

2023-09-19 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-18207: - Assignee: (was: Viraj Jasani) > Introduce hadoop-logging module >

[jira] [Assigned] (HADOOP-15984) Update jersey from 1.19 to 2.x

2023-09-19 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-15984: - Assignee: (was: Viraj Jasani) > Update jersey from 1.19 to 2.x > -

[GitHub] [hadoop] KeeProMise closed pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
KeeProMise closed pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified URL: https://github.com/apache/hadoop/pull/6096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HADOOP-15984) Update jersey from 1.19 to 2.x

2023-09-19 Thread H. Vetinari (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766962#comment-17766962 ] H. Vetinari commented on HADOOP-15984: -- As of today, we can add Java 21 to the lis

[GitHub] [hadoop] ChaosJu closed pull request #6104: Branch 3.3.6 dev

2023-09-19 Thread via GitHub
ChaosJu closed pull request #6104: Branch 3.3.6 dev URL: https://github.com/apache/hadoop/pull/6104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: comm

[GitHub] [hadoop] ChaosJu commented on pull request #6104: Branch 3.3.6 dev

2023-09-19 Thread via GitHub
ChaosJu commented on PR #6104: URL: https://github.com/apache/hadoop/pull/6104#issuecomment-1726782231 w -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-m

[GitHub] [hadoop] ChaosJu closed pull request #6104: Branch 3.3.6 dev

2023-09-19 Thread via GitHub
ChaosJu closed pull request #6104: Branch 3.3.6 dev URL: https://github.com/apache/hadoop/pull/6104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: comm

[GitHub] [hadoop] ChaosJu opened a new pull request, #6104: Branch 3.3.6 dev

2023-09-19 Thread via GitHub
ChaosJu opened a new pull request, #6104: URL: https://github.com/apache/hadoop/pull/6104 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-

[GitHub] [hadoop] zhengchenyu closed pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
zhengchenyu closed pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0. URL: https://github.com/apache/hadoop/pull/6102 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] zhengchenyu commented on pull request #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
zhengchenyu commented on PR #6102: URL: https://github.com/apache/hadoop/pull/6102#issuecomment-1726712035 @ayushtkn https://github.com/apache/hadoop/pull/6042 was reverted. Can you please review this PR? And I still think we should change the default value. I found that before HA

[GitHub] [hadoop] hadoop-yetus commented on pull request #6103: Revert MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0 (#6042).

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6103: URL: https://github.com/apache/hadoop/pull/6103#issuecomment-1726701994 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] zhengchenyu closed pull request #6102: MAPREDUCE-7453. Addendum: Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
zhengchenyu closed pull request #6102: MAPREDUCE-7453. Addendum: Container logs are missing when yarn.app.container.log.filesize is set to default value 0. URL: https://github.com/apache/hadoop/pull/6102 -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hadoop] slfan1989 merged pull request #6103: Revert MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0 (#6042).

2023-09-19 Thread via GitHub
slfan1989 merged PR #6103: URL: https://github.com/apache/hadoop/pull/6103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] slfan1989 opened a new pull request, #6103: Revert MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0 (#6042).

2023-09-19 Thread via GitHub
slfan1989 opened a new pull request, #6103: URL: https://github.com/apache/hadoop/pull/6103 This reverts commit ab2bc90e090564b7466ab1b759fbb849612ae82b. ### Description of PR JIRA: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to de

[GitHub] [hadoop] slfan1989 commented on pull request #6042: MAPREDUCE-7453. Container logs are missing when yarn.app.container.lo…

2023-09-19 Thread via GitHub
slfan1989 commented on PR #6042: URL: https://github.com/apache/hadoop/pull/6042#issuecomment-1726696355 @ayushtkn Thank you very much for pointing out the issue! I will revert this PR and I will continue to follow up on this PR. -- This is an automated message from the Apache Git Service

[GitHub] [hadoop] zhengchenyu opened a new pull request, #6102: MAPREDUCE-7453. Container logs are missing when yarn.app.container.log.filesize is set to default value 0.

2023-09-19 Thread via GitHub
zhengchenyu opened a new pull request, #6102: URL: https://github.com/apache/hadoop/pull/6102 ### Description of PR Fix unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [hadoop] zhengchenyu commented on pull request #6042: MAPREDUCE-7453. Container logs are missing when yarn.app.container.lo…

2023-09-19 Thread via GitHub
zhengchenyu commented on PR #6042: URL: https://github.com/apache/hadoop/pull/6042#issuecomment-1726668134 @ayushtkn Sorry this is my fault. I thought that changing the default value is not a serious behavior. I will fix the UT. BTW, can we trigger the test4tests when show 'The patch doe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6006: URL: https://github.com/apache/hadoop/pull/6006#issuecomment-1726402089 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #6042: MAPREDUCE-7453. Container logs are missing when yarn.app.container.lo…

2023-09-19 Thread via GitHub
ayushtkn commented on code in PR #6042: URL: https://github.com/apache/hadoop/pull/6042#discussion_r1330552980 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml: ## @@ -823,16 +823,15 @@ mapreduce.task.u

[GitHub] [hadoop] hadoop-yetus commented on pull request #6095: HDFS-17197. Show file replication when listing corrupt files.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6095: URL: https://github.com/apache/hadoop/pull/6095#issuecomment-1726243027 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] shameersss1 commented on pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on PR #6006: URL: https://github.com/apache/hadoop/pull/6006#issuecomment-1726215230 @steveloughran - I have addressed your comments with the lastest commit. Could you please re-review? -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [hadoop] shameersss1 commented on pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on PR #6006: URL: https://github.com/apache/hadoop/pull/6006#issuecomment-1726211609 > looks good, only a few minor changes left. > > when you submit these changes, don't bother rebasing and force pushing unless changes in trunk have broken the code. it's a lot e

[GitHub] [hadoop] shameersss1 commented on a diff in pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on code in PR #6006: URL: https://github.com/apache/hadoop/pull/6006#discussion_r1330497585 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitUtilsWithMR.java: ## @@ -49,10 +50,11 @@ private CommitUtilsWithMR() { /**

[GitHub] [hadoop] shameersss1 commented on a diff in pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on code in PR #6006: URL: https://github.com/apache/hadoop/pull/6006#discussion_r1330496445 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committer_architecture.md: ## @@ -1694,7 +1694,7 @@ must be used, which means: the V2 classes. **Mag

[GitHub] [hadoop] shameersss1 commented on a diff in pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on code in PR #6006: URL: https://github.com/apache/hadoop/pull/6006#discussion_r1330495768 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committer_architecture.md: ## @@ -1694,7 +1694,7 @@ must be used, which means: the V2 classes. **Mag

[GitHub] [hadoop] goiri commented on a diff in pull request #6094: HDFS-17194. Enhance the log message for striped block recovery

2023-09-19 Thread via GitHub
goiri commented on code in PR #6094: URL: https://github.com/apache/hadoop/pull/6094#discussion_r1330440303 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java: ## @@ -427,26 +426,41 @@ protected void recover() throws

[GitHub] [hadoop] goiri commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-19 Thread via GitHub
goiri commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1330435400 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.jav

[GitHub] [hadoop] goiri commented on a diff in pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-19 Thread via GitHub
goiri commented on code in PR #6016: URL: https://github.com/apache/hadoop/pull/6016#discussion_r1330434643 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/ZookeeperFederationStateStore.jav

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766859#comment-17766859 ] Steve Loughran commented on HADOOP-18889: - and a few more to skip. one finding

[GitHub] [hadoop] KeeProMise commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
KeeProMise commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1725925618 @goiri The last "root in trunk failed." may not be a problem with my code, the penultimate one with the same commit will pass, if you have time, please help review again -- This is a

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1725791963 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] Hexiaoqiao commented on pull request #6036: HDFS-17105. mistakenly purge editLogs even after it is empty in NNStorageRetentionManager

2023-09-19 Thread via GitHub
Hexiaoqiao commented on PR #6036: URL: https://github.com/apache/hadoop/pull/6036#issuecomment-1725654563 LGTM. +1 from my side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #6035: HDFS-17098. DatanodeManager does not handle null storage type properly

2023-09-19 Thread via GitHub
Hexiaoqiao commented on code in PR #6035: URL: https://github.com/apache/hadoop/pull/6035#discussion_r1330184496 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java: ## @@ -666,7 +666,15 @@ private Consumer> create

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1725628985 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] qcastel commented on pull request #5082: HADOOP-18510. Support confidential client in Azure refresh token gran…

2023-09-19 Thread via GitHub
qcastel commented on PR #5082: URL: https://github.com/apache/hadoop/pull/5082#issuecomment-1725626763 Soon it will be the anniversary of this PR @snvijaya @ashutoshcipher should we do a money pot? -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1725554738 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6069: URL: https://github.com/apache/hadoop/pull/6069#issuecomment-1725513045 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
saxenapranav commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1330030917 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemChecksum.java: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #6095: HDFS-17197. Show file replication when listing corrupt files.

2023-09-19 Thread via GitHub
zhangshuyan0 commented on code in PR #6095: URL: https://github.com/apache/hadoop/pull/6095#discussion_r1330009497 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java: ## @@ -6195,7 +6197,11 @@ Collection listCorruptFileBloc

[GitHub] [hadoop] pjfanning commented on pull request #6057: HADOOP-18890: remove use of okhttp in runtime code

2023-09-19 Thread via GitHub
pjfanning commented on PR #6057: URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1725372522 I created https://github.com/apache/hadoop/pull/6101 for backporting this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [hadoop] pjfanning opened a new pull request, #6101: HADOOP-18890. Remove use of okhttp in runtime code (#6057)

2023-09-19 Thread via GitHub
pjfanning opened a new pull request, #6101: URL: https://github.com/apache/hadoop/pull/6101 backport of #6057 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
KeeProMise commented on code in PR #6096: URL: https://github.com/apache/hadoop/pull/6096#discussion_r1330001570 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/impl/MembershipStoreImpl.java: ## @@ -279,30 +279,30 @@ private Me

[GitHub] [hadoop] slfan1989 commented on pull request #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
slfan1989 commented on PR #6100: URL: https://github.com/apache/hadoop/pull/6100#issuecomment-1725358508 @liubin101 I verified in the test environment and the issue you described did not occur. Can you describe your test steps? Kerberos is not enabled in my test environment. cu

[jira] [Updated] (HADOOP-18576) Java 11 JavaDoc fails due to missing package comments

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18576: Fix Version/s: 3.3.9 > Java 11 JavaDoc fails due to missing package comments > --

[GitHub] [hadoop] steveloughran merged pull request #5362: HADOOP-18576. Java 11 JavaDoc fails due to missing package comments

2023-09-19 Thread via GitHub
steveloughran merged PR #5362: URL: https://github.com/apache/hadoop/pull/5362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4835: HDFS-16740. Mini cluster test flakiness

2023-09-19 Thread via GitHub
steveloughran commented on code in PR #4835: URL: https://github.com/apache/hadoop/pull/4835#discussion_r1329984828 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/MiniJournalCluster.java: ## @@ -289,7 +293,8 @@ public void waitActive() throws IO

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Fix Version/s: 3.4.0 > remove okhttp usage > --- > > Key:

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Issue Type: Improvement (was: Bug) > remove okhttp usage > --- > >

[jira] [Updated] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18890: Component/s: build > remove okhttp usage > --- > > Key: H

[jira] [Assigned] (HADOOP-18890) remove okhttp usage

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18890: --- Assignee: PJ Fanning > remove okhttp usage > --- > >

[GitHub] [hadoop] steveloughran commented on pull request #6057: HADOOP-18890: remove use of okhttp in runtime code

2023-09-19 Thread via GitHub
steveloughran commented on PR #6057: URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1725334975 +1 merged to trunk. presumably it needs a backport? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [hadoop] steveloughran merged pull request #6057: HADOOP-18890: remove use of okhttp in runtime code

2023-09-19 Thread via GitHub
steveloughran merged PR #6057: URL: https://github.com/apache/hadoop/pull/6057 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[GitHub] [hadoop] steveloughran commented on pull request #6057: HADOOP-18890: remove use of okhttp in runtime code

2023-09-19 Thread via GitHub
steveloughran commented on PR #6057: URL: https://github.com/apache/hadoop/pull/6057#issuecomment-1725332719 test failures are unrelated; the timeline one has been seen before YARN-11546 -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329976899 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1427,28 +1428,45 @@ private void appendIfNotEmpty(String

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anmolanmol1234 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329976407 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1427,28 +1428,45 @@ private void appendIfNotEmpty(String

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

2023-09-19 Thread via GitHub
anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1329975077 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java: ## @@ -1858,7 +1895,25 @@ public void storeEmptyLinkFile(St

[GitHub] [hadoop] steveloughran commented on a diff in pull request #6057: HADOOP-18890: remove use of okhttp in runtime code

2023-09-19 Thread via GitHub
steveloughran commented on code in PR #6057: URL: https://github.com/apache/hadoop/pull/6057#discussion_r1329974227 ## hadoop-project/pom.xml: ## @@ -220,62 +218,6 @@ - Review Comment: thanks -- This is an automated message from the Apache Git Service.

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

2023-09-19 Thread via GitHub
anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1329972478 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java: ## @@ -1858,7 +1895,25 @@ public void storeEmptyLinkFile(St

[GitHub] [hadoop] anmolanmol1234 commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

2023-09-19 Thread via GitHub
anmolanmol1234 commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1329972478 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java: ## @@ -1858,7 +1895,25 @@ public void storeEmptyLinkFile(St

[GitHub] [hadoop] steveloughran commented on a diff in pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
steveloughran commented on code in PR #6006: URL: https://github.com/apache/hadoop/pull/6006#discussion_r1329969310 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/impl/CommitUtilsWithMR.java: ## @@ -49,10 +50,11 @@ private CommitUtilsWithMR() { /**

[jira] [Resolved] (HADOOP-18895) upgrade to commons-compress 1.24.0 due to CVE

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18895. - Fix Version/s: 3.3.9 Resolution: Fixed > upgrade to commons-compress 1.24.0 due

[GitHub] [hadoop] steveloughran merged pull request #6073: HADOOP-18895. Upgrade to commons-compress 1.24.0

2023-09-19 Thread via GitHub
steveloughran merged PR #6073: URL: https://github.com/apache/hadoop/pull/6073 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[GitHub] [hadoop] steveloughran commented on pull request #6073: HADOOP-18895. Upgrade to commons-compress 1.24.0

2023-09-19 Thread via GitHub
steveloughran commented on PR #6073: URL: https://github.com/apache/hadoop/pull/6073#issuecomment-1725298689 test failures look unrelated; created HDFS-17202 to track the TestDfsAdmin failure...that's been a brittle test as it looks for the output for messages. -- This is an automated mes

[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766751#comment-17766751 ] Steve Loughran commented on HADOOP-18889: - HADOOP-18908 to cover region stuff,

[jira] [Assigned] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18908: --- Assignee: Ahmar Suhail > Improve s3a region handling, including determining from e

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329938677 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringBui

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329935952 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringBui

[GitHub] [hadoop] alkis commented on a diff in pull request #6025: Pass down eTag as part of accessCondition to SDK

2023-09-19 Thread via GitHub
alkis commented on code in PR #6025: URL: https://github.com/apache/hadoop/pull/6025#discussion_r1329934424 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java: ## @@ -1858,7 +1895,25 @@ public void storeEmptyLinkFile(String key,

[GitHub] [hadoop] liubin101 commented on pull request #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
liubin101 commented on PR #6100: URL: https://github.com/apache/hadoop/pull/6100#issuecomment-1725207929 > @liubin101 Thanks for your contribution! But we can't make this change, hsr should be empty if we don't want to authenticate the user. @slfan1989 Thanks for review this pr! Wheth

[GitHub] [hadoop] slfan1989 commented on pull request #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
slfan1989 commented on PR #6100: URL: https://github.com/apache/hadoop/pull/6100#issuecomment-1725131527 @liubin101 Thanks for your contribution! But we can't make this change, hsr should be empty if we don't want to authenticate the user. -- This is an automated message from the Apache G

[GitHub] [hadoop] hadoop-yetus commented on pull request #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6100: URL: https://github.com/apache/hadoop/pull/6100#issuecomment-1725106436 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] shameersss1 commented on pull request #6006: HADOOP-18797: Support Concurrent Writes With S3A Magic Committer

2023-09-19 Thread via GitHub
shameersss1 commented on PR #6006: URL: https://github.com/apache/hadoop/pull/6006#issuecomment-1725079623 @steveloughran , I have addressed your comments. Could you please re-review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [hadoop] hadoop-yetus commented on pull request #6098: YARN-11573. Add config option to make container allocation prefer nodes without reserved containers

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6098: URL: https://github.com/apache/hadoop/pull/6098#issuecomment-1725064167 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6096: HDFS-17198. RBF: fix bug of getRepresentativeQuorum when records have same dateModified

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6096: URL: https://github.com/apache/hadoop/pull/6096#issuecomment-1725056426 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6083: URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1725045802 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #6025: Pass down eTag as part of accessCondition to SDK

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6025: URL: https://github.com/apache/hadoop/pull/6025#issuecomment-1725042789 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6099: HDFS-17200. Add some datanode related metrics to Metrics.md.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6099: URL: https://github.com/apache/hadoop/pull/6099#issuecomment-1725033612 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6085: YARN-9586. Need more doc for yarn.federation.policy-manager-params when LoadBasedRouterPolicy is used.

2023-09-19 Thread via GitHub
hadoop-yetus commented on PR #6085: URL: https://github.com/apache/hadoop/pull/6085#issuecomment-1724999282 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
saxenapranav commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329706865 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringBui

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6069: [ABFS] Supporting Checksum Validation for Read and Write

2023-09-19 Thread via GitHub
saxenapranav commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1329704618 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1412,6 +1421,54 @@ private void appendIfNotEmpty(StringBui

[GitHub] [hadoop] liubin101 opened a new pull request, #6100: YARN-11574. Make yarn router webservice forward auth type configurable.

2023-09-19 Thread via GitHub
liubin101 opened a new pull request, #6100: URL: https://github.com/apache/hadoop/pull/6100 ### Description of PR Now if we request yarn router webservice to get apps without authentication, will return null and print error log "Unable to obtain user name, user not authenticated", b

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5996: HDFS-17171. CONGESTION_RATIO should be configurable

2023-09-19 Thread via GitHub
hfutatzhanghb commented on PR #5996: URL: https://github.com/apache/hadoop/pull/5996#issuecomment-1724962530 @Hexiaoqiao @ayushtkn @tomscut Hi, Sir. Could you please help me review this PR when you have free time? Thanks a lot! -- This is an automated message from the Apache Git Service.