[PR] Add API for bulk/paged object deletion [hadoop]

2024-04-12 Thread via GitHub
mukund-thakur opened a new pull request, #6726: URL: https://github.com/apache/hadoop/pull/6726 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g.

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6716: URL: https://github.com/apache/hadoop/pull/6716#issuecomment-2052490254 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] MAPREDUCE-7474. Improve Manifest committer resilience [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6716: URL: https://github.com/apache/hadoop/pull/6716#issuecomment-2052380306 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17464. Improve some logs output in class FsDatasetImpl [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6724: URL: https://github.com/apache/hadoop/pull/6724#issuecomment-2052146135 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836699#comment-17836699 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2052142063 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836697#comment-17836697 ] Dongjoon Hyun commented on HADOOP-18296: Thank you for sharing the direction. > Memory

[jira] [Commented] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836691#comment-17836691 ] ASF GitHub Bot commented on HADOOP-19129: - steveloughran merged PR #6676: URL:

Re: [PR] HADOOP-19129: [ABFS] Test Fixes and Test Script Bug Fixes [hadoop]

2024-04-12 Thread via GitHub
steveloughran merged PR #6676: URL: https://github.com/apache/hadoop/pull/6676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836690#comment-17836690 ] ASF GitHub Bot commented on HADOOP-19129: - steveloughran commented on code in PR #6676: URL:

Re: [PR] HADOOP-19129: [ABFS] Test Fixes and Test Script Bug Fixes [hadoop]

2024-04-12 Thread via GitHub
steveloughran commented on code in PR #6676: URL: https://github.com/apache/hadoop/pull/6676#discussion_r1562837609 ## hadoop-tools/hadoop-azure/src/site/markdown/index.md: ## @@ -18,8 +18,8 @@ See also: -* [ABFS](./abfs.html) -* [Testing](./testing_azure.html) +*

[jira] [Commented] (HADOOP-19082) S3A: Update AWS SDK V2 to 2.24.6

2024-04-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836675#comment-17836675 ] Steve Loughran commented on HADOOP-19082: - FYI this SDK has an unshaded copy of

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1783#comment-1783 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2052026204 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] YARN-11684. Fix general contract violation in PriorityQueueComparator. [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6725: URL: https://github.com/apache/hadoop/pull/6725#issuecomment-2051972158 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836612#comment-17836612 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051807908 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836595#comment-17836595 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-04-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1562502863 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java: ## @@ -20,370 +20,515 @@ import java.io.IOException;

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836577#comment-17836577 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-04-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1562462047 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientThrottlingIntercept.java: ## @@ -124,23 +125,24 @@ static

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
Hexiaoqiao commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051614898 Committed to trunk. Thanks @haiyang1987 , @ZanderXu and @ayushtkn ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
Hexiaoqiao merged PR #6721: URL: https://github.com/apache/hadoop/pull/6721 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
Hexiaoqiao commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051611333 @ayushtkn Got it. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2051603077 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836565#comment-17836565 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051564903 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
ayushtkn commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051554589 @Hexiaoqiao If you check the previous Jenkins result, it has two runs for the spotbugs, one without patch, just on trunk & then with patch. Without patch, on trunk it is red, with the

[PR] YARN-11684. Fix general contract violation in PriorityQueueComparator. [hadoop]

2024-04-12 Thread via GitHub
tomicooler opened a new pull request, #6725: URL: https://github.com/apache/hadoop/pull/6725 Change-Id: I0ca72134b3328a250445ad41525d76d6cbd50a98 ### Description of PR Jira: [YARN-11684](https://issues.apache.org/jira/browse/YARN-11684) Previous attempt for the fix:

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
Hexiaoqiao commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051526835 Hi @ayushtkn , 'With the patch it shows fixed' where is this report link, hadoop-yetus comment here is not the correct one? Thanks. > That spotbug warning is for the trunk,

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051501737 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19120) [ABFS]: ApacheHttpClient adaptation as network library

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836550#comment-17836550 ] ASF GitHub Bot commented on HADOOP-19120: - anmolanmol1234 commented on code in PR #6633: URL:

Re: [PR] HADOOP-19120. ApacheHttpClient adaptation in ABFS. [hadoop]

2024-04-12 Thread via GitHub
anmolanmol1234 commented on code in PR #6633: URL: https://github.com/apache/hadoop/pull/6633#discussion_r1562362652 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsAHCHttpOperation.java: ## @@ -0,0 +1,422 @@ +/** + * Licensed to the Apache

[PR] HDFS-17464. Improve some logs output in class FsDatasetImpl [hadoop]

2024-04-12 Thread via GitHub
hfutatzhanghb opened a new pull request, #6724: URL: https://github.com/apache/hadoop/pull/6724 ### Description of PR Improve some logs output in class FsDatasetImpl, add block info. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836528#comment-17836528 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051406699 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2051294433 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
ayushtkn commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051257011 That spotbug warning is for the trunk, without the patch https://github.com/apache/hadoop/assets/25608848/b74a66a1-e0e8-4412-bd3b-737616063795;> With the patch it shows fixed

Re: [PR] HDFS-17461. Fix spotbugs in PeerCache#getInternal [hadoop]

2024-04-12 Thread via GitHub
Hexiaoqiao commented on PR #6721: URL: https://github.com/apache/hadoop/pull/6721#issuecomment-2051238063 Still report spotbugs at latest. Try to trigger CI again, let's wait what it will say. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836445#comment-17836445 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051187441 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

Re: [PR] Hadoop 18325: ABFS: Add correlated metric support for ABFS operations [hadoop]

2024-04-12 Thread via GitHub
anmolanmol1234 commented on PR #6314: URL: https://github.com/apache/hadoop/pull/6314#issuecomment-2051161677 Hi @steveloughran @mukund-thakur @mehakmeet, requesting you to kindly review this PR. -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836430#comment-17836430 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051090442 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19129) ABFS: Fixing Test Script Bug and Some Known test Failures in ABFS Test Suite

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836429#comment-17836429 ] ASF GitHub Bot commented on HADOOP-19129: - hadoop-yetus commented on PR #6676: URL:

Re: [PR] HADOOP-19129: [ABFS] Test Fixes and Test Script Bug Fixes [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6676: URL: https://github.com/apache/hadoop/pull/6676#issuecomment-2051088177 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-19139) [ABFS]: No GetPathStatus call for opening AbfsInputStream

2024-04-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836423#comment-17836423 ] ASF GitHub Bot commented on HADOOP-19139: - hadoop-yetus commented on PR #6699: URL:

Re: [PR] HADOOP-19139.No GetPathStatus for opening AbfsInputStream [hadoop]

2024-04-12 Thread via GitHub
hadoop-yetus commented on PR #6699: URL: https://github.com/apache/hadoop/pull/6699#issuecomment-2051059398 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |