[jira] [Commented] (HADOOP-18981) Move oncrpc/portmap from hadoop-nfs to hadoop-common

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802436#comment-17802436 ] ASF GitHub Bot commented on HADOOP-18981: - xinglin commented on PR #6280: URL:

Re: [PR] HADOOP-18981. moved oncrpc/portmap from hadoop-common-project/hadoop-nfs to hadoop-common-project/hadoop-common [hadoop]

2024-01-03 Thread via GitHub
xinglin commented on PR #6280: URL: https://github.com/apache/hadoop/pull/6280#issuecomment-1876538794 > Note that as hadoop-common is a provided dependency of hadoop-nfs, with explicit use of org.apache.hadoop.util classes in MountdBase, is there any deployment/use of the hadoop-nfs module

[jira] [Commented] (HADOOP-18981) Move oncrpc/portmap from hadoop-nfs to hadoop-common

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802413#comment-17802413 ] ASF GitHub Bot commented on HADOOP-18981: - xinglin commented on PR #6280: URL:

Re: [PR] HADOOP-18981. moved oncrpc/portmap from hadoop-common-project/hadoop-nfs to hadoop-common-project/hadoop-common [hadoop]

2024-01-03 Thread via GitHub
xinglin commented on PR #6280: URL: https://github.com/apache/hadoop/pull/6280#issuecomment-1876492612 @steveloughran, Thanks for taking a look at this change. I added a couple of package-info.java files. First time doing this and not sure whether I am adding these files correctly. Could yo

Re: [PR] HDFS-17322. RetryCache#MAX_CAPACITY seems to be MIN_CAPACITY. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6405: URL: https://github.com/apache/hadoop/pull/6405#issuecomment-1876480172 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Created] (HADOOP-19023) ITestS3AConcurrentOps#testParallelRename intermittent timeout failure

2024-01-03 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-19023: - Summary: ITestS3AConcurrentOps#testParallelRename intermittent timeout failure Key: HADOOP-19023 URL: https://issues.apache.org/jira/browse/HADOOP-19023 Project: Ha

[jira] [Commented] (HADOOP-18980) S3A credential provider remapping: make extensible

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802396#comment-17802396 ] ASF GitHub Bot commented on HADOOP-18980: - virajjasani commented on PR #6406: U

Re: [PR] HADOOP-18980. S3A credential provider remapping: make extensible [hadoop]

2024-01-03 Thread via GitHub
virajjasani commented on PR #6406: URL: https://github.com/apache/hadoop/pull/6406#issuecomment-1876317165 Tested against `us-west-2`: 1. ``` [ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 256.281 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.scale.IT

[jira] [Commented] (HADOOP-19022) ITestS3AConfiguration#testRequestTimeout failure

2024-01-03 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802395#comment-17802395 ] Viraj Jasani commented on HADOOP-19022: --- It's small test, but perhaps good to cov

[jira] [Created] (HADOOP-19022) ITestS3AConfiguration#testRequestTimeout failure

2024-01-03 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-19022: - Summary: ITestS3AConfiguration#testRequestTimeout failure Key: HADOOP-19022 URL: https://issues.apache.org/jira/browse/HADOOP-19022 Project: Hadoop Common

[PR] HADOOP-18980. S3A credential provider remapping: make extensible [hadoop]

2024-01-03 Thread via GitHub
virajjasani opened a new pull request, #6406: URL: https://github.com/apache/hadoop/pull/6406 Jira: HADOOP-18980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
KeeProMise commented on code in PR #6403: URL: https://github.com/apache/hadoop/pull/6403#discussion_r1441264235 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1647,16 +1661,8 @@ public synchronized boolean seekToNewSou

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
hfutatzhanghb commented on code in PR #6403: URL: https://github.com/apache/hadoop/pull/6403#discussion_r1441251384 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1647,16 +1661,8 @@ public synchronized boolean seekToNew

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
KeeProMise commented on code in PR #6403: URL: https://github.com/apache/hadoop/pull/6403#discussion_r1441243319 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1647,16 +1661,8 @@ public synchronized boolean seekToNewSou

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
KeeProMise commented on code in PR #6403: URL: https://github.com/apache/hadoop/pull/6403#discussion_r1441243319 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1647,16 +1661,8 @@ public synchronized boolean seekToNewSou

[PR] HDFS-17322. RetryCache#MAX_CAPACITY seems to be MIN_CAPACITY. [hadoop]

2024-01-03 Thread via GitHub
hfutatzhanghb opened a new pull request, #6405: URL: https://github.com/apache/hadoop/pull/6405 ### Description of PR From the code logic, we can infer that RetryCache#MAX_CAPACITY should  better be  MIN_CAPACITY. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
LiuGuH commented on PR #6404: URL: https://github.com/apache/hadoop/pull/6404#issuecomment-1876223751 > I'm wondering if we can instead fix the existing mechanism such that only a single read is sent to the active, vs. adding a new mechanism. Yes, it can. But for only a single read is sen

Re: [PR] HDFS-17306. RBF: Router should not return nameservices that does not enable observer nodes in RpcResponseHeaderProto [hadoop]

2024-01-03 Thread via GitHub
LiuGuH commented on PR #6385: URL: https://github.com/apache/hadoop/pull/6385#issuecomment-1876216428 > I'm okay with saving this optimization for a separate pull request though. Thanks for review . And I will be add a separate pull request for DFS_ROUTER_OBSERVER_READ_DEFAULT_KEY a

Re: [PR] HDFS-17310. DiskBalancer: Enhance the log message for submitPlan [hadoop]

2024-01-03 Thread via GitHub
haiyang1987 commented on PR #6391: URL: https://github.com/apache/hadoop/pull/6391#issuecomment-1876205914 Thanks @slfan1989 @tasanuma @ashutoshcipher for your review and merge it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] HDFS-17304. Update fsck -blockId to display slownode status of blocks. [hadoop]

2024-01-03 Thread via GitHub
huangzhaobo99 commented on PR #6384: URL: https://github.com/apache/hadoop/pull/6384#issuecomment-1876204046 @slfan1989 Can this change be pushed forward when you have free time ? Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
xuzifu666 commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1876194790 @ayushtkn https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6402/7/ UT seems all success,but the result is not all passed -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-19019) Parallel Maven Build Support for Apache Hadoop

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802353#comment-17802353 ] ASF GitHub Bot commented on HADOOP-19019: - JiaLiangC commented on PR #6373: URL

Re: [PR] HADOOP-19019: Parallel Maven Build Support for Apache Hadoop [hadoop]

2024-01-03 Thread via GitHub
JiaLiangC commented on PR #6373: URL: https://github.com/apache/hadoop/pull/6373#issuecomment-1876166196 @Hexiaoqiao Test environment: CentOS 8 x86_64, 16GB RAM, SSD. Tested on Hadoop 3.3.6. The initial serial compilation took almost 3 hours due to slow dependency downloads. With p

[jira] [Commented] (HADOOP-19015) Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802350#comment-17802350 ] ASF GitHub Bot commented on HADOOP-19015: - mukund-thakur commented on PR #6372:

Re: [PR] HADOOP-19015. Increase fs.s3a.connection.maximum to 500 to minimize risk of Timeout waiting for connection from pool. [hadoop]

2024-01-03 Thread via GitHub
mukund-thakur commented on PR #6372: URL: https://github.com/apache/hadoop/pull/6372#issuecomment-1876126039 Yetus failure because of no tests added. Re-ran all the tests using us-west-1 and all is good. -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] YARN-11115. Add configuration to globally disable AM preemption for capacity scheduler [hadoop]

2024-01-03 Thread via GitHub
ashutoshcipher closed pull request #4377: YARN-5. Add configuration to globally disable AM preemption for capacity scheduler URL: https://github.com/apache/hadoop/pull/4377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
simbadzina commented on PR #6404: URL: https://github.com/apache/hadoop/pull/6404#issuecomment-1876005260 @LiuGuH can you expand more on the following > And [HDFS-16890](https://issues.apache.org/jira/browse/HDFS-16890) maybe lead to many read requests into active NN at the same time.

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875821498 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17314. Add a metrics to record congestion backoff counts. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6398: URL: https://github.com/apache/hadoop/pull/6398#issuecomment-1875821500 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] YARN-7953. [BackPort] [GQ] Data structures for federation global queues calculations. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6361: URL: https://github.com/apache/hadoop/pull/6361#issuecomment-1875785768 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] YARN-11631. [GPG] Add GPGWebServices. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6354: URL: https://github.com/apache/hadoop/pull/6354#issuecomment-1875771769 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875756070 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875697807 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875697478 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] YARN-11638. [GPG] GPG Support CLI. [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6396: URL: https://github.com/apache/hadoop/pull/6396#issuecomment-1875628978 @goiri Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDFS-17182. DataSetLockManager.lockLeakCheck() is not thread-safe. [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6029: URL: https://github.com/apache/hadoop/pull/6029#issuecomment-1875626968 @LiuGuH Thank you for your contribution! When we submit a PR, we need to write the JIRA number into the commit message. ![image](https://github.com/apache/hadoop/assets/55643692/b

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875625521 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17310. DiskBalancer: Enhance the log message for submitPlan [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6391: URL: https://github.com/apache/hadoop/pull/6391#issuecomment-1875613663 @haiyang1987 Thank you for your contribution! Merged Into trunk. @ashutoshcipher @tasanuma Thanks for reviewing the code! -- This is an automated message from the Apache Git Service. T

Re: [PR] HDFS-17310. DiskBalancer: Enhance the log message for submitPlan [hadoop]

2024-01-03 Thread via GitHub
slfan1989 merged PR #6391: URL: https://github.com/apache/hadoop/pull/6391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

Re: [PR] YARN-7953. [BackPort] [GQ] Data structures for federation global queues calculations. [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6361: URL: https://github.com/apache/hadoop/pull/6361#issuecomment-1875590456 @goiri Can you help review this PR? Thank you very much! I will continue to follow up on [YARN-7402](https://issues.apache.org/jira/browse/YARN-7402). -- This is an automated message f

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
hfutatzhanghb commented on code in PR #6403: URL: https://github.com/apache/hadoop/pull/6403#discussion_r1440611536 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1647,16 +1661,8 @@ public synchronized boolean seekToNew

Re: [PR] YARN-11631. [GPG] Add GPGWebServices. [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6354: URL: https://github.com/apache/hadoop/pull/6354#issuecomment-1875589582 @goiri Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on code in PR #6404: URL: https://github.com/apache/hadoop/pull/6404#discussion_r1440611314 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterAutoMsyncService.java: ## @@ -0,0 +1,86 @@ +/** + * Lice

Re: [PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6404: URL: https://github.com/apache/hadoop/pull/6404#issuecomment-1875585566 @simbadzina Can you help review this PR? Thank you very much! I see that `HDFS-16890` and `HDFS-17027` were both completed by you. -- This is an automated message from the Apache Git

Re: [PR] YARN-11638. [GPG] GPG Support CLI. [hadoop]

2024-01-03 Thread via GitHub
slfan1989 commented on PR #6396: URL: https://github.com/apache/hadoop/pull/6396#issuecomment-1875561860 @goiri Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6404: URL: https://github.com/apache/hadoop/pull/6404#issuecomment-1875515543 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HADOOP-18184. S3A Prefetching unbuffer. [hadoop]

2024-01-03 Thread via GitHub
steveloughran commented on PR #5832: URL: https://github.com/apache/hadoop/pull/5832#issuecomment-1875498186 ...just catching up on this; not ready for merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[jira] [Commented] (HADOOP-18184) s3a prefetching stream to support unbuffer()

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802207#comment-17802207 ] ASF GitHub Bot commented on HADOOP-18184: - steveloughran commented on PR #5832:

Re: [PR] HDFS-17320. seekToNewSource uses ignoredNodes to get a new node other than the current node. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6403: URL: https://github.com/apache/hadoop/pull/6403#issuecomment-1875485613 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] YARN-11638. [GPG] GPG Support CLI. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6396: URL: https://github.com/apache/hadoop/pull/6396#issuecomment-1875410714 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HADOOP-19019: Parallel Maven Build Support for Apache Hadoop [hadoop]

2024-01-03 Thread via GitHub
Hexiaoqiao commented on PR #6373: URL: https://github.com/apache/hadoop/pull/6373#issuecomment-1875368262 @JiaLiangC Thanks for your work and involve me here. It is very interesting improvement. I want to know if any time cost save when change to parallel build. Another side, beside hadoop-

[jira] [Commented] (HADOOP-19019) Parallel Maven Build Support for Apache Hadoop

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802173#comment-17802173 ] ASF GitHub Bot commented on HADOOP-19019: - Hexiaoqiao commented on PR #6373: UR

[PR] HDFS-17321. RBF: Add RouterAutoMsyncService for auto msync in Router [hadoop]

2024-01-03 Thread via GitHub
LiuGuH opened a new pull request, #6404: URL: https://github.com/apache/hadoop/pull/6404 ### Description of PR Router should have the ability to to auto msync to a nameservice. And it can ensure router periodically refreshes its record of a namespace's state.

Re: [PR] HDFS-17182. DataSetLockManager.lockLeakCheck() is not thread-safe. [hadoop]

2024-01-03 Thread via GitHub
Hexiaoqiao commented on PR #6029: URL: https://github.com/apache/hadoop/pull/6029#issuecomment-1875355078 Committed to trunk. Thanks @LiuGuH for your contribution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] HDFS-17182. DataSetLockManager.lockLeakCheck() is not thread-safe. [hadoop]

2024-01-03 Thread via GitHub
Hexiaoqiao merged PR #6029: URL: https://github.com/apache/hadoop/pull/6029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@ha

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802168#comment-17802168 ] ASF GitHub Bot commented on HADOOP-18910: - anujmodi2021 commented on code in PR

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-01-03 Thread via GitHub
anujmodi2021 commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1440430936 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1074,11 +1079,14 @@ public AbfsRestOperation read(final St

Re: [PR] Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout. [hadoop]

2024-01-03 Thread via GitHub
anujmodi2021 commented on PR #5881: URL: https://github.com/apache/hadoop/pull/5881#issuecomment-1875338541 Resolved Merged Conflicts and ran the test suite. @steveloughran kindly request you to merge this. Thanks for all the efforts. -- This is an automated message from the Apac

[jira] [Updated] (HADOOP-17912) ABFS: Support for Encryption Context

2024-01-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17912: Fix Version/s: 3.3.9 > ABFS: Support for Encryption Context > ---

[jira] [Commented] (HADOOP-17912) ABFS: Support for Encryption Context

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802156#comment-17802156 ] ASF GitHub Bot commented on HADOOP-17912: - steveloughran merged PR #6401: URL:

Re: [PR] HADOOP-17912. ABFS: Support for Encryption Context (#6221) [hadoop]

2024-01-03 Thread via GitHub
steveloughran merged PR #6401: URL: https://github.com/apache/hadoop/pull/6401 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

Re: [PR] HADOOP-18971: [ABFS] Enable Footer Read Optimizations with Appropriate Footer Read Buffer Size [hadoop]

2024-01-03 Thread via GitHub
steveloughran merged PR #6270: URL: https://github.com/apache/hadoop/pull/6270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802152#comment-17802152 ] ASF GitHub Bot commented on HADOOP-18910: - steveloughran commented on code in P

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-01-03 Thread via GitHub
steveloughran commented on code in PR #6069: URL: https://github.com/apache/hadoop/pull/6069#discussion_r1440414581 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java: ## @@ -1074,11 +1079,14 @@ public AbfsRestOperation read(final S

Re: [PR] HDFS-17314. Add a metrics to record congestion backoff counts. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6398: URL: https://github.com/apache/hadoop/pull/6398#issuecomment-1875296933 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[PR] use ignoredNodes [hadoop]

2024-01-03 Thread via GitHub
KeeProMise opened a new pull request, #6403: URL: https://github.com/apache/hadoop/pull/6403 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADO

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
xuzifu666 commented on code in PR #6402: URL: https://github.com/apache/hadoop/pull/6402#discussion_r1440364986 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java: ## @@ -306,7 +306,7 @@ int run(List args) throws IOException {

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
ayushtkn commented on code in PR #6402: URL: https://github.com/apache/hadoop/pull/6402#discussion_r1440357368 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java: ## @@ -306,7 +306,7 @@ int run(List args) throws IOException {

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
xuzifu666 commented on code in PR #6402: URL: https://github.com/apache/hadoop/pull/6402#discussion_r1440354333 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java: ## @@ -306,7 +306,7 @@ int run(List args) throws IOException {

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
ayushtkn commented on code in PR #6402: URL: https://github.com/apache/hadoop/pull/6402#discussion_r1440351242 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java: ## @@ -306,7 +306,7 @@ int run(List args) throws IOException {

Re: [PR] Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #5881: URL: https://github.com/apache/hadoop/pull/5881#issuecomment-1875228322 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
xuzifu666 commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875222042 @ayushtkn Here need flush first,had change it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] HDFS-17300. [SBN READ] Observer should throw ObserverRetryOnActiveException if stateid is always delayed with Active Namenode for a configured time [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6383: URL: https://github.com/apache/hadoop/pull/6383#issuecomment-1875203426 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17317. DebugAdmin metaOut not need multiple close [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6402: URL: https://github.com/apache/hadoop/pull/6402#issuecomment-1875202329 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17314. Add a metrics to record congestion backoff counts. [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6398: URL: https://github.com/apache/hadoop/pull/6398#issuecomment-1875184321 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Assigned] (HADOOP-18656) ABFS: Support for Pagination in Recursive Directory Delete

2024-01-03 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi reassigned HADOOP-18656: -- Assignee: Anuj Modi (was: Sree Bhattacharyya) > ABFS: Support for Pagination in Recursive D

Re: [PR] Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout. [hadoop]

2024-01-03 Thread via GitHub
anujmodi2021 commented on PR #5881: URL: https://github.com/apache/hadoop/pull/5881#issuecomment-1875151334 AGGREGATED TEST RESULT HNS-OAuth [INFO] Results: [INFO] [WARNING] Tests run: 137, Failures: 0, Errors:

[jira] [Commented] (HADOOP-18971) ABFS: Enable Footer Read Optimizations with Appropriate Footer Read Buffer Size

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802097#comment-17802097 ] ASF GitHub Bot commented on HADOOP-18971: - anujmodi2021 commented on PR #6270:

Re: [PR] HADOOP-18971: [ABFS] Enable Footer Read Optimizations with Appropriate Footer Read Buffer Size [hadoop]

2024-01-03 Thread via GitHub
anujmodi2021 commented on PR #6270: URL: https://github.com/apache/hadoop/pull/6270#issuecomment-1875108638 > > I think not only footer reads but this can be expanded to other prefetches as well. > > Especially small files that are read fully can be cached such that multiple streams can

[jira] [Commented] (HADOOP-18910) ABFS: Adding Support for MD5 Hash based integrity verification of the request content during transport

2024-01-03 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17802096#comment-17802096 ] ASF GitHub Bot commented on HADOOP-18910: - anujmodi2021 commented on PR #6069:

Re: [PR] HADOOP-18910: [ABFS] Adding Support for MD5 Hash based integrity verification of the request content during transport [hadoop]

2024-01-03 Thread via GitHub
anujmodi2021 commented on PR #6069: URL: https://github.com/apache/hadoop/pull/6069#issuecomment-1875106413 Thanks for the review @steveloughran If it looks good, please get it merged to trunk -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] HDFS-17309. RBF: Fix Router Safemode check condition error [hadoop]

2024-01-03 Thread via GitHub
hadoop-yetus commented on PR #6390: URL: https://github.com/apache/hadoop/pull/6390#issuecomment-1875019152 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe