[jira] [Commented] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846173#comment-17846173 ] ASF GitHub Bot commented on HADOOP-19172: - hadoop-yetus commented on PR #6823: URL:

Re: [PR] HADOOP-19172. S3A: upgrade AWS v1 sdk to 1.12.720 [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6823: URL: https://github.com/apache/hadoop/pull/6823#issuecomment-2109293902 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17329. DiskBalancerCluster#nodesToProcess should better use ArrayList. [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #6423: URL: https://github.com/apache/hadoop/pull/6423#issuecomment-2109258078 When we want to improve it, the following patch should be better, because it also involve `list.add` where LinkedList is less cost. ``` ---

Re: [PR] HDFS-17476. fix: False positive "Observer Node is too far behind" due to long overflow. [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #6747: URL: https://github.com/apache/hadoop/pull/6747#issuecomment-2109219912 Hi @haiyang1987 @ZanderXu would you mind to take review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HADOOP-19156) ZooKeeper based state stores use different ZK address configs

2024-05-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846162#comment-17846162 ] Xiaoqiao He commented on HADOOP-19156: -- Add [~inVisible] to contributor list and assign this

[jira] [Assigned] (HADOOP-19156) ZooKeeper based state stores use different ZK address configs

2024-05-13 Thread Xiaoqiao He (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He reassigned HADOOP-19156: Assignee: liu bin > ZooKeeper based state stores use different ZK address configs >

[jira] [Commented] (HADOOP-18851) Performance improvement for DelegationTokenSecretManager.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846160#comment-17846160 ] ASF GitHub Bot commented on HADOOP-18851: - Hexiaoqiao commented on PR #6803: URL:

Re: [PR] HADOOP-18851. DT performance improvement latest review incorporation [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #6803: URL: https://github.com/apache/hadoop/pull/6803#issuecomment-2109216164 > the mvnsite failure is, looks like not relevant, but other merged MR seems doesn't have this failure. Hi @ChenSammi, @vikaskr22 `mvnsite` failure is not related to this PR, it

Re: [PR] HDFS-17099. Fix Null Pointer Exception when stop namesystem in HDFS. [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #6034: URL: https://github.com/apache/hadoop/pull/6034#issuecomment-2109197470 Sorry for too late to push forward this PR. Committed to trunk. Thanks @teamconfx and @ayushtkn -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] HDFS-17099. Fix Null Pointer Exception when stop namesystem in HDFS. [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao merged PR #6034: URL: https://github.com/apache/hadoop/pull/6034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience (#6716) [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6825: URL: https://github.com/apache/hadoop/pull/6825#issuecomment-2109190422 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Resolved] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HADOOP-19170. -- Fix Version/s: 3.5.0 Resolution: Fixed > Fixes compilation issues on Mac >

[jira] [Commented] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846145#comment-17846145 ] ASF GitHub Bot commented on HADOOP-19170: - jojochuang merged PR #6822: URL:

Re: [PR] HADOOP-19170. Fixes compilation issues on Mac [hadoop]

2024-05-13 Thread via GitHub
jojochuang merged PR #6822: URL: https://github.com/apache/hadoop/pull/6822 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17518: Sync the editslog if a file is closed in the lease monitor [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on code in PR #6809: URL: https://github.com/apache/hadoop/pull/6809#discussion_r1599314271 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java: ## @@ -626,7 +626,8 @@ private synchronized boolean

Re: [PR] HDFS-17438. RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. [hadoop]

2024-05-13 Thread via GitHub
KeeProMise commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2109177969 @goiri The failed ut should not be caused by my modifications. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Chenyu Zheng (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846134#comment-17846134 ] Chenyu Zheng commented on HADOOP-19170: --- [~weichiu] I have add the environment. Any OS that does

[jira] [Updated] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Chenyu Zheng (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenyu Zheng updated HADOOP-19170: -- Environment: OS:  macOS Catalina 10.15.7 compiler: clang 12.0.0 cmake: 3.24.0 > Fixes

Re: [PR] HDFS-17497. The number of bytes of the last committed block should be calculated into the file length [hadoop]

2024-05-13 Thread via GitHub
haiyang1987 commented on code in PR #6765: URL: https://github.com/apache/hadoop/pull/6765#discussion_r1599287673 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java: ## @@ -387,6 +387,19 @@ public boolean

[PR] HDFS-17526. getMetadataInputStream should use getShareDeleteFileInputStream for windows [hadoop]

2024-05-13 Thread via GitHub
dannytbecker opened a new pull request, #6826: URL: https://github.com/apache/hadoop/pull/6826 ### Description of PR In HDFS-10636, the getDataInputStream method uses the getShareDeleteFileInputStream for windows, but the getMetaDataInputStream does not use this. The following error can

[jira] [Commented] (HADOOP-19167) Change of Codec configuration does not work

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846126#comment-17846126 ] ASF GitHub Bot commented on HADOOP-19167: - skyskyhu commented on PR #6807: URL:

Re: [PR] HADOOP-19167 Bug Fix: Change of Codec configuration does not work [hadoop]

2024-05-13 Thread via GitHub
skyskyhu commented on PR #6807: URL: https://github.com/apache/hadoop/pull/6807#issuecomment-2109118290 @ferhui @adoroszlai @goiri Can you help review and merge the commit when you have free time~ Thanks a lot. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience (#6716) [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6824: URL: https://github.com/apache/hadoop/pull/6824#issuecomment-2109077510 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6804: URL: https://github.com/apache/hadoop/pull/6804#issuecomment-2109071666 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] YARN-11692. Support mixed cgroup v1/v2 controller structure [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6821: URL: https://github.com/apache/hadoop/pull/6821#issuecomment-2109020769 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6780: URL: https://github.com/apache/hadoop/pull/6780#issuecomment-2109018378 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6804: URL: https://github.com/apache/hadoop/pull/6804#issuecomment-2109013347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846110#comment-17846110 ] ASF GitHub Bot commented on HADOOP-19073: - xuzifu666 commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-05-13 Thread via GitHub
xuzifu666 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-2109009003 > @xuzifu666 before I merge, what name do you want to be credited with in the patch...your github account doesn't have one. Thanks for your review, do you mean my real name? My

Re: [PR] YARN-11692. Support mixed cgroup v1/v2 controller structure [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6821: URL: https://github.com/apache/hadoop/pull/6821#issuecomment-2108904702 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19013) fs.getXattrs(path) for S3FS doesn't have x-amz-server-side-encryption-aws-kms-key-id header.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846102#comment-17846102 ] ASF GitHub Bot commented on HADOOP-19013: - hadoop-yetus commented on PR #6646: URL:

Re: [PR] HADOOP-19013. Adding x-amz-server-side-encryption-aws-kms-key-id in the get file attributes for S3A. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6646: URL: https://github.com/apache/hadoop/pull/6646#issuecomment-2108903980 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6780: URL: https://github.com/apache/hadoop/pull/6780#issuecomment-2108868944 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

[jira] [Commented] (HADOOP-19152) Do not hard code security providers.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846074#comment-17846074 ] ASF GitHub Bot commented on HADOOP-19152: - hadoop-yetus commented on PR #6739: URL:

Re: [PR] HADOOP-19152. Do not hard code security providers. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6739: URL: https://github.com/apache/hadoop/pull/6739#issuecomment-2108804028 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17438. RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6655: URL: https://github.com/apache/hadoop/pull/6655#issuecomment-2108803506 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[PR] MAPREDUCE-7474. Improve Manifest committer resilience (#6716) [hadoop]

2024-05-13 Thread via GitHub
steveloughran opened a new pull request, #6825: URL: https://github.com/apache/hadoop/pull/6825 Improve task commit resilience everywhere and add an option to reduce delete IO requests on job cleanup (relevant for ABFS and HDFS). Task Commit Resilience --

[PR] MAPREDUCE-7474. Improve Manifest committer resilience (#6716) [hadoop]

2024-05-13 Thread via GitHub
steveloughran opened a new pull request, #6824: URL: https://github.com/apache/hadoop/pull/6824 backport of #6716 Improve task commit resilience everywhere and add an option to reduce delete IO requests on job cleanup (relevant for ABFS and HDFS). Task Commit

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-05-13 Thread via GitHub
steveloughran merged PR #6716: URL: https://github.com/apache/hadoop/pull/6716 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17504. DN process should exit when BPServiceActor exit [hadoop]

2024-05-13 Thread via GitHub
vinayakumarb commented on PR #6792: URL: https://github.com/apache/hadoop/pull/6792#issuecomment-2108716673 > @zhuzilong2013 Thanks for your report and contribution! IMO, they are independent between different BPServiceActor, if exit DN process due to one BPServiceActor issue, it will

[jira] [Commented] (HADOOP-19154) upgrade bouncy castle to 1.78.1 due to CVEs

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846055#comment-17846055 ] ASF GitHub Bot commented on HADOOP-19154: - vinayakumarb commented on PR #6755: URL:

Re: [PR] MAPREDUCE-7475. Fixed non-idempotent unit tests [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6785: URL: https://github.com/apache/hadoop/pull/6785#issuecomment-2108690394 @ayushtkn the issue with @after is doesn't clean up reliably on job interrupt. but that's more of an issue with cloud stuff, isn't it? as here a `mvn clean` will do the cleanup.

Re: [PR] HADOOP-19154. Upgrade bouncycastle to 1.78.1 due to CVEs [hadoop]

2024-05-13 Thread via GitHub
vinayakumarb commented on PR #6755: URL: https://github.com/apache/hadoop/pull/6755#issuecomment-2108689940 > This seems to have gone in as #6811. > > I'm going to propose rolling back #6811 and merging this one instead as it has a jira ID, goes to a later version and updates the

[jira] [Commented] (HADOOP-18679) Add API for bulk/paged object deletion

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846052#comment-17846052 ] ASF GitHub Bot commented on HADOOP-18679: - steveloughran commented on PR #6726: URL:

Re: [PR] HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. [hadoop]

2024-05-13 Thread via GitHub
ctrezzo commented on code in PR #6804: URL: https://github.com/apache/hadoop/pull/6804#discussion_r1599008233 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/PoolAlignmentContext.java: ## @@ -64,7 +64,11 @@ public void

Re: [PR] HADOOP-18679. Add API for bulk/paged object deletion [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6726: URL: https://github.com/apache/hadoop/pull/6726#issuecomment-2108677588 mukund, if you can do those naming changes then I'm +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] HDFS-17518: Sync the editslog if a file is closed in the lease monitor [hadoop]

2024-05-13 Thread via GitHub
vinayakumarb commented on code in PR #6809: URL: https://github.com/apache/hadoop/pull/6809#discussion_r1599005240 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java: ## @@ -626,7 +626,8 @@ private synchronized boolean

[jira] [Commented] (HADOOP-18786) Hadoop build depends on archives.apache.org

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846044#comment-17846044 ] ASF GitHub Bot commented on HADOOP-18786: - ctubbsii commented on PR #5789: URL:

Re: [PR] HADOOP-18786. Use CDN instead of ASF archive [hadoop]

2024-05-13 Thread via GitHub
ctubbsii commented on PR #5789: URL: https://github.com/apache/hadoop/pull/5789#issuecomment-2108580048 > The main thing I want to be sure is from this build, what gets into the distro? only stuff from the maven repo, right? that is: this PR MUST NOT force updates in the binaries we ship.

[jira] [Commented] (HADOOP-19152) Do not hard code security providers.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846039#comment-17846039 ] ASF GitHub Bot commented on HADOOP-19152: - szetszwo commented on code in PR #6739: URL:

Re: [PR] HADOOP-19152. Do not hard code security providers. [hadoop]

2024-05-13 Thread via GitHub
szetszwo commented on code in PR #6739: URL: https://github.com/apache/hadoop/pull/6739#discussion_r1598897001 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/crypto/TestCryptoUtils.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software

Re: [PR] YARN-11692. Support mixed cgroup v1/v2 controller structure [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6821: URL: https://github.com/apache/hadoop/pull/6821#issuecomment-2108554141 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6780: URL: https://github.com/apache/hadoop/pull/6780#issuecomment-2108463869 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17509. RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6784: URL: https://github.com/apache/hadoop/pull/6784#issuecomment-2108370233 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846026#comment-17846026 ] ASF GitHub Bot commented on HADOOP-19073: - steveloughran commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-2108366767 @xuzifu666 before I merge, what name do you want to be credited with in the patch...your github account doesn't have one. -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19073) WASB: Fix connection leak in FolderRenamePending

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846025#comment-17846025 ] ASF GitHub Bot commented on HADOOP-19073: - steveloughran commented on PR #6534: URL:

Re: [PR] HADOOP-19073 WASB: Fix connection leak in FolderRenamePending [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-2108355675 I've been away for a week and am catching up, so not tested yet, sorry... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HADOOP-18786) Hadoop build depends on archives.apache.org

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846024#comment-17846024 ] ASF GitHub Bot commented on HADOOP-18786: - steveloughran commented on PR #5789: URL:

Re: [PR] HADOOP-18786. Use CDN instead of ASF archive [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #5789: URL: https://github.com/apache/hadoop/pull/5789#issuecomment-2108352558 The main thing I want to be sure is from this build, what gets into the distro? only stuff from the maven repo, right? that is: this PR MUST NOT force updates in the binaries we

[jira] [Commented] (HADOOP-19161) S3A: option "fs.s3a.performance.flags" to take list of performance flags

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846023#comment-17846023 ] ASF GitHub Bot commented on HADOOP-19161: - steveloughran commented on PR #6789: URL:

Re: [PR] HADOOP-19161. S3A: option "fs.s3a.performance.flags" to take list of performance flags [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6789: URL: https://github.com/apache/hadoop/pull/6789#issuecomment-2108338924 I have a better design for this. changign this to draft. Proposed: we have a `Configuration.getEnumOptions(Enum x, boolean failIfUnknown)` which returns an EnumSet of all

[jira] [Comment Edited] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846022#comment-17846022 ] Wei-Chiu Chuang edited comment on HADOOP-19170 at 5/13/24 5:25 PM: ---

[jira] [Comment Edited] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846022#comment-17846022 ] Wei-Chiu Chuang edited comment on HADOOP-19170 at 5/13/24 5:23 PM: ---

[jira] [Commented] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846022#comment-17846022 ] Wei-Chiu Chuang commented on HADOOP-19170: -- [~zhengchenyu] thanks for reporting the issue. Can

[jira] [Updated] (HADOOP-19170) Fixes compilation issues on Mac

2024-05-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-19170: - Summary: Fixes compilation issues on Mac (was: Fixes compilation issues on non-Linux

[jira] [Commented] (HADOOP-19165) Explore dropping protobuf 2.5.0 from the distro

2024-05-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846021#comment-17846021 ] Steve Loughran commented on HADOOP-19165: - relates to HADOOP-18487, where I tried to do most of

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6780: URL: https://github.com/apache/hadoop/pull/6780#issuecomment-2108294132 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19013) fs.getXattrs(path) for S3FS doesn't have x-amz-server-side-encryption-aws-kms-key-id header.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846016#comment-17846016 ] ASF GitHub Bot commented on HADOOP-19013: - steveloughran commented on code in PR #6646: URL:

Re: [PR] HADOOP-19013. Adding x-amz-server-side-encryption-aws-kms-key-id in the get file attributes for S3A. [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on code in PR #6646: URL: https://github.com/apache/hadoop/pull/6646#discussion_r1598785291 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/EncryptionTestUtils.java: ## @@ -19,7 +19,13 @@ package org.apache.hadoop.fs.s3a; import

Re: [PR] HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. [hadoop]

2024-05-13 Thread via GitHub
simbadzina commented on code in PR #6804: URL: https://github.com/apache/hadoop/pull/6804#discussion_r1598782895 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/PoolAlignmentContext.java: ## @@ -64,7 +64,11 @@ public void

Re: [PR] HDFS-17514: RBF: Routers should unset cached stateID when namenode does not set stateID in RPC response header. [hadoop]

2024-05-13 Thread via GitHub
simbadzina commented on PR #6804: URL: https://github.com/apache/hadoop/pull/6804#issuecomment-2108218576 > @simbadzina Should we also add a test case to

Re: [PR] YARN-11692. Support mixed cgroup v1/v2 controller structure [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6821: URL: https://github.com/apache/hadoop/pull/6821#issuecomment-2108203238 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17438. RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. [hadoop]

2024-05-13 Thread via GitHub
KeeProMise commented on code in PR #6655: URL: https://github.com/apache/hadoop/pull/6655#discussion_r1598727611 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java: ## @@ -404,6 +404,28 @@ public static void waitFor(final Supplier

Re: [PR] YARN-11471. [Federation] FederationStateStoreFacade Cache Support Caffeine. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6795: URL: https://github.com/apache/hadoop/pull/6795#issuecomment-2108107635 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | -1 :x: | patch

Re: [PR] HDFS-17524. OIV: add Transformed processor which reconstructs an fsimage from another fsimage file [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6820: URL: https://github.com/apache/hadoop/pull/6820#issuecomment-2108085535 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

[jira] [Commented] (HADOOP-18851) Performance improvement for DelegationTokenSecretManager.

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845987#comment-17845987 ] ASF GitHub Bot commented on HADOOP-18851: - hadoop-yetus commented on PR #6803: URL:

Re: [PR] HADOOP-18851. DT performance improvement latest review incorporation [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6803: URL: https://github.com/apache/hadoop/pull/6803#issuecomment-2108044944 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] YARN-11471. [Federation] FederationStateStoreFacade Cache Support Caffeine. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6795: URL: https://github.com/apache/hadoop/pull/6795#issuecomment-2107954804 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17509. RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6784: URL: https://github.com/apache/hadoop/pull/6784#issuecomment-2107945602 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11471. [Federation] FederationStateStoreFacade Cache Support Caffeine. [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6795: URL: https://github.com/apache/hadoop/pull/6795#issuecomment-2107894913 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Created] (HADOOP-19173) Upgrade org.apache.derby:derby to 10.17.1.0

2024-05-13 Thread Shilun Fan (Jira)
Shilun Fan created HADOOP-19173: --- Summary: Upgrade org.apache.derby:derby to 10.17.1.0 Key: HADOOP-19173 URL: https://issues.apache.org/jira/browse/HADOOP-19173 Project: Hadoop Common Issue

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
brumi1024 commented on code in PR #6780: URL: https://github.com/apache/hadoop/pull/6780#discussion_r1598604638 ##

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
brumi1024 commented on code in PR #6780: URL: https://github.com/apache/hadoop/pull/6780#discussion_r1598585675 ##

[jira] [Commented] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845964#comment-17845964 ] ASF GitHub Bot commented on HADOOP-19172: - steveloughran commented on PR #6823: URL:

Re: [PR] HADOOP-19172. S3A: upgrade AWS v1 sdk to 1.12.720 [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6823: URL: https://github.com/apache/hadoop/pull/6823#issuecomment-2107752565 oh, it's just the STS builder issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845962#comment-17845962 ] ASF GitHub Bot commented on HADOOP-19172: - steveloughran commented on PR #6823: URL:

Re: [PR] HADOOP-19172. S3A: upgrade AWS v1 sdk to 1.12.720 [hadoop]

2024-05-13 Thread via GitHub
steveloughran commented on PR #6823: URL: https://github.com/apache/hadoop/pull/6823#issuecomment-2107746813 test run, two failures ``` [ERROR] ITestS3ACommitterFactory.testEverything:115->testInvalidFileBinding:165 Expected a org.apache.hadoop.fs.s3a.commit.PathCommitException to

[jira] [Updated] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19172: Labels: pull-request-available (was: ) > Upgrade aws-java-sdk to 1.12.720 >

[jira] [Commented] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845958#comment-17845958 ] ASF GitHub Bot commented on HADOOP-19172: - steveloughran opened a new pull request, #6823: URL:

[PR] HADOOP-19172. S3A: upgrade AWS v1 sdk to 1.12.720 [hadoop]

2024-05-13 Thread via GitHub
steveloughran opened a new pull request, #6823: URL: https://github.com/apache/hadoop/pull/6823 +remove reference in LICENSE-binary as it is no longer shipped tests in progress; for v1 update in trunk, as it isn't used except in tests for reflection-based support of v1

[jira] [Assigned] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-19172: --- Assignee: Steve Loughran > Upgrade aws-java-sdk to 1.12.720 >

[jira] [Created] (HADOOP-19172) Upgrade aws-java-sdk to 1.12.720

2024-05-13 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19172: --- Summary: Upgrade aws-java-sdk to 1.12.720 Key: HADOOP-19172 URL: https://issues.apache.org/jira/browse/HADOOP-19172 Project: Hadoop Common Issue Type:

Re: [PR] HDFS-17522. JournalNode web interfaces lack configs for X-FRAME-OPTIONS protection [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #6814: URL: https://github.com/apache/hadoop/pull/6814#issuecomment-2107680415 Committed to trunk. Thanks @hiwangzhihui and @vinayakumarb . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] HDFS-17522. JournalNode web interfaces lack configs for X-FRAME-OPTIONS protection [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao merged PR #6814: URL: https://github.com/apache/hadoop/pull/6814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17518: Sync the editslog if a file is closed in the lease monitor [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on code in PR #6809: URL: https://github.com/apache/hadoop/pull/6809#discussion_r1598536353 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java: ## @@ -626,7 +626,8 @@ private synchronized boolean

Re: [PR] YARN-11687. Update CGroupsResourceCalculator to track usages using cgroupv2 [hadoop]

2024-05-13 Thread via GitHub
K0K0V0K commented on code in PR #6780: URL: https://github.com/apache/hadoop/pull/6780#discussion_r1598513673 ##

Re: [PR] HDFS-16368. DFSadmin supports refresh topology info without restarting namenode [hadoop]

2024-05-13 Thread via GitHub
Hexiaoqiao commented on PR #3743: URL: https://github.com/apache/hadoop/pull/3743#issuecomment-2107611900 Not sure why this PR has been pick up again. IIRC we have discussed add tools to support refresh topology online many times. I would like to share my concern again. This is one

Re: [PR] Bump ch.qos.logback:logback-classic from 1.1.2 to 1.2.13 in /hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle [hadoop]

2024-05-13 Thread via GitHub
hadoop-yetus commented on PR #6819: URL: https://github.com/apache/hadoop/pull/6819#issuecomment-2107602896 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| _ Prechecks

Re: [PR] HDFS-17509. RBF: Fix ClientProtocol.concat will throw NPE if tgr is a empty file. [hadoop]

2024-05-13 Thread via GitHub
LiuGuH commented on code in PR #6784: URL: https://github.com/apache/hadoop/pull/6784#discussion_r1598472326 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpc.java: ## @@ -1224,6 +1239,26 @@ public void

  1   2   >