[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011362#comment-18011362 ] ASF GitHub Bot commented on HADOOP-19645: - manika137 commented on code in PR #7

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
manika137 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2247066421 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/TracingHeaderVersion.java: ## @@ -0,0 +1,50 @@ +/** + * Licensed to the Apache Software

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011354#comment-18011354 ] ASF GitHub Bot commented on HADOOP-19645: - manika137 commented on code in PR #7

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
manika137 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2247014906 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/TracingContext.java: ## @@ -265,6 +286,34 @@ private String addFailureReasons(final Stri

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011351#comment-18011351 ] ASF GitHub Bot commented on HADOOP-19645: - manika137 commented on code in PR #7

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
manika137 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2246993787 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/TracingContext.java: ## @@ -193,32 +213,33 @@ public void setListener(Listener listener)

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011349#comment-18011349 ] ASF GitHub Bot commented on HADOOP-19645: - manika137 commented on code in PR #7

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
manika137 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2246929766 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java: ## @@ -442,6 +451,7 @@ private int optimisedRead(final byte[] b

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011345#comment-18011345 ] ASF GitHub Bot commented on HADOOP-19645: - manika137 commented on code in PR #7

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
manika137 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2246885435 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java: ## @@ -544,7 +555,9 @@ private int readInternal(final long posi

Re: [PR] HDFS-17815. Fix upload fsimage failure when checkpoint takes a long time [hadoop]

2025-07-31 Thread via GitHub
wangxin97-02 commented on PR #7845: URL: https://github.com/apache/hadoop/pull/7845#issuecomment-3142008829 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] HDFS-12431. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs Part10. [hadoop]

2025-07-31 Thread via GitHub
zhtttylz commented on code in PR #7841: URL: https://github.com/apache/hadoop/pull/7841#discussion_r2246768594 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java: ## @@ -172,8 +172,8 @@ public DatanodeRegistrati

Re: [PR] YARN-11838: YARN ConcurrentModificationException When Refreshing Node Attributes [hadoop]

2025-07-31 Thread via GitHub
shameersss1 commented on PR #7828: URL: https://github.com/apache/hadoop/pull/7828#issuecomment-3141897374 @slfan1989 - Gentle reminder for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (HADOOP-19612) Add Support for Propagating Access Token via RPC Header in HDFS

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011327#comment-18011327 ] ASF GitHub Bot commented on HADOOP-19612: - hadoop-yetus commented on PR #7844:

Re: [PR] HADOOP-19612 Rpc auth header [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7844: URL: https://github.com/apache/hadoop/pull/7844#issuecomment-3141824285 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19617) [JDK17] Remove JUnit4 Dependency

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011326#comment-18011326 ] ASF GitHub Bot commented on HADOOP-19617: - slfan1989 commented on PR #7798: URL

Re: [PR] HADOOP-19617. [JDK17] Remove JUnit4 Dependency (YARN) . [hadoop]

2025-07-31 Thread via GitHub
slfan1989 commented on PR #7798: URL: https://github.com/apache/hadoop/pull/7798#issuecomment-3141780541 The unit tests pass successfully locally, but there are issues on `Yetus`. I tried adjusting the `-Xss` parameter in hopes of mitigating the timeout issue. > hadoop-yarn-server-res

Re: [PR] HDFS-12431. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs Part10. [hadoop]

2025-07-31 Thread via GitHub
slfan1989 commented on code in PR #7841: URL: https://github.com/apache/hadoop/pull/7841#discussion_r2246635377 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/BlockReportTestBase.java: ## @@ -910,7 +927,7 @@ public void run() { s

Re: [PR] HDFS-12431. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs Part10. [hadoop]

2025-07-31 Thread via GitHub
slfan1989 commented on code in PR #7841: URL: https://github.com/apache/hadoop/pull/7841#discussion_r2246635173 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/BlockReportTestBase.java: ## @@ -696,7 +715,7 @@ public Void call() throws IOEx

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011310#comment-18011310 ] ASF GitHub Bot commented on HADOOP-18296: - hadoop-yetus commented on PR #7846:

Re: [PR] HADOOP-18296. Memory fragmentation in ChecksumFileSystem readVectored() (#7732) [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7846: URL: https://github.com/apache/hadoop/pull/7846#issuecomment-3141608204 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17815. Fix upload fsimage failure when checkpoint takes a long time [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7845: URL: https://github.com/apache/hadoop/pull/7845#issuecomment-3141534703 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-12431. [JDK17] Upgrade JUnit from 4 to 5 in hadoop-hdfs Part10. [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7841: URL: https://github.com/apache/hadoop/pull/7841#issuecomment-3141290981 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011281#comment-18011281 ] ASF GitHub Bot commented on HADOOP-18296: - steveloughran commented on PR #7846:

Re: [PR] HADOOP-18296. Memory fragmentation in ChecksumFileSystem readVectored() (#7732) [hadoop]

2025-07-31 Thread via GitHub
steveloughran commented on PR #7846: URL: https://github.com/apache/hadoop/pull/7846#issuecomment-3141237145 tested s3 london with ` -Dparallel-tests -DtestsThreadCount=8 -Dscale` all good except those needing a yarn cluster. -- This is an automated message from the Apache Git Servi

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011268#comment-18011268 ] ASF GitHub Bot commented on HADOOP-18296: - steveloughran opened a new pull requ

[PR] HADOOP-18296. Memory fragmentation in ChecksumFileSystem readVectored() (#7732) [hadoop]

2025-07-31 Thread via GitHub
steveloughran opened a new pull request, #7846: URL: https://github.com/apache/hadoop/pull/7846 Option "fs.file.checksum.verify" disables checksum verification in local FS, so sliced subsets of larger buffers are never returned. Stream capability "fs.capability.vectoredio.sliced"

Re: [PR] HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #6517: URL: https://github.com/apache/hadoop/pull/6517#issuecomment-3140878552 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[PR] HDFS-17815. Fix upload fsimage failure when checkpoint takes a long time [hadoop]

2025-07-31 Thread via GitHub
lfxy opened a new pull request, #7845: URL: https://github.com/apache/hadoop/pull/7845 The capacity of Our hdfs federation cluster are more then 500 PB, with one NS containing over 600 million files. Once checkpoint takes nearly two hours. We discover checkpoint frequently failures du

[jira] [Updated] (HADOOP-19628) Managed Identity Token Provider is not implemented

2025-07-31 Thread Lotte Felius (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lotte Felius updated HADOOP-19628: -- Description: Managed Identity Token Provider is not implemented in the hadoop-azure jar. Now,

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011212#comment-18011212 ] ASF GitHub Bot commented on HADOOP-19645: - hadoop-yetus commented on PR #7837:

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7837: URL: https://github.com/apache/hadoop/pull/7837#issuecomment-3140309047 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-19612) Add Support for Propagating Access Token via RPC Header in HDFS

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011201#comment-18011201 ] ASF GitHub Bot commented on HADOOP-19612: - mccormickt12 commented on PR #7844:

Re: [PR] HADOOP-19612 Rpc auth header [hadoop]

2025-07-31 Thread via GitHub
mccormickt12 commented on PR #7844: URL: https://github.com/apache/hadoop/pull/7844#issuecomment-3140156165 failures seem to be flaky tests, unrelated to my changes ``` [INFO] Results: [INFO] [ERROR] Errors: [ERROR] TestBlockListAsLongs.testFuzz:139->checkReport:156 » NoSu

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
bhattmanish98 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2245456486 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/AbfsHttpConstants.java: ## @@ -128,6 +128,7 @@ public final class AbfsHttpConsta

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011196#comment-18011196 ] ASF GitHub Bot commented on HADOOP-19645: - bhattmanish98 commented on code in P

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011190#comment-18011190 ] ASF GitHub Bot commented on HADOOP-19645: - anujmodi2021 commented on code in PR

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
anujmodi2021 commented on code in PR #7837: URL: https://github.com/apache/hadoop/pull/7837#discussion_r2245426192 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/utils/TracingContext.java: ## @@ -265,6 +289,34 @@ private String addFailureReasons(final S

[jira] [Commented] (HADOOP-19617) [JDK17] Remove JUnit4 Dependency

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011183#comment-18011183 ] ASF GitHub Bot commented on HADOOP-19617: - hadoop-yetus commented on PR #7798:

Re: [PR] HADOOP-19617. [JDK17] Remove JUnit4 Dependency (YARN) . [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7798: URL: https://github.com/apache/hadoop/pull/7798#issuecomment-3139907980 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19635) ABFS: [FNS Over Blob] Marker creation fail exception should not be propagated

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011145#comment-18011145 ] ASF GitHub Bot commented on HADOOP-19635: - anujmodi2021 merged PR #7825: URL: h

Re: [PR] HADOOP-19635. ABFS: Marker creation failure should not be propagated [hadoop]

2025-07-31 Thread via GitHub
anujmodi2021 merged PR #7825: URL: https://github.com/apache/hadoop/pull/7825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@

Re: [PR] Hadoop 19641: [ABFS][ReadAheadV2] First Read should bypass ReadBufferManager [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7835: URL: https://github.com/apache/hadoop/pull/7835#issuecomment-3139479470 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17814. Add missing LevelDB dependencies to hadoop-hdfs module [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7842: URL: https://github.com/apache/hadoop/pull/7842#issuecomment-3139457968 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19645) ABFS: [ReadAheadV2] Improve Metrics for Read Calls to identify type of read done.

2025-07-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18011105#comment-18011105 ] ASF GitHub Bot commented on HADOOP-19645: - hadoop-yetus commented on PR #7837:

Re: [PR] HADOOP-19645. [ABFS][ReadAheadV2] Improve Metrics for Read Calls to identify type of read done. [hadoop]

2025-07-31 Thread via GitHub
hadoop-yetus commented on PR #7837: URL: https://github.com/apache/hadoop/pull/7837#issuecomment-3138813312 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe