[jira] [Updated] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19066: Labels: pull-request-available (was: ) > AWS SDK V2 - Enabling FIPS should be allowed wi

[jira] [Commented] (HADOOP-19066) AWS SDK V2 - Enabling FIPS should be allowed with central endpoint

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815551#comment-17815551 ] ASF GitHub Bot commented on HADOOP-19066: - virajjasani opened a new pull reques

[PR] HADOOP-19066. S3A: AWS SDK V2 - Enabling FIPS should be allowed with central endpoint [hadoop]

2024-02-07 Thread via GitHub
virajjasani opened a new pull request, #6539: URL: https://github.com/apache/hadoop/pull/6539 Jira: HADOOP-19066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

Re: [PR] HDFS-17362. RBF: Implement RouterObserverReadConfiguredFailoverProxyProvider [hadoop]

2024-02-07 Thread via GitHub
tasanuma commented on PR #6510: URL: https://github.com/apache/hadoop/pull/6510#issuecomment-1933469403 @simbadzina Thanks for your review! I will merge it in a couple of days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815546#comment-17815546 ] ASF GitHub Bot commented on HADOOP-19071: - hadoop-yetus commented on PR #6537:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933467016 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815535#comment-17815535 ] ASF GitHub Bot commented on HADOOP-14837: - bpahuja commented on PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
bpahuja commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1933450335 Hi @steveloughran, Have addressed the checkstyle violations in the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[jira] [Commented] (HADOOP-18342) Upgrade to Avro 1.11.1

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815534#comment-17815534 ] ASF GitHub Bot commented on HADOOP-18342: - hadoop-yetus commented on PR #4854:

Re: [PR] HADOOP-18342: shaded avro jar [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #4854: URL: https://github.com/apache/hadoop/pull/4854#issuecomment-1933446188 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17299. Adding rack failure tolerance when creating a new file [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6513: URL: https://github.com/apache/hadoop/pull/6513#issuecomment-1933421772 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Comment Edited] (HADOOP-19070) Java 17 compile support

2024-02-07 Thread Benjamin Teke (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815521#comment-17815521 ] Benjamin Teke edited comment on HADOOP-19070 at 2/8/24 6:00 AM: -

[jira] [Commented] (HADOOP-19070) Java 17 compile support

2024-02-07 Thread Benjamin Teke (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815521#comment-17815521 ] Benjamin Teke commented on HADOOP-19070: [~slfan1989] thanks for reporting this

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933404637 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815507#comment-17815507 ] ASF GitHub Bot commented on HADOOP-19071: - hadoop-yetus commented on PR #6537:

Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6537: URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933340199 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815506#comment-17815506 ] ASF GitHub Bot commented on HADOOP-19059: - adnanhemani commented on PR #6538: U

Re: [PR] HADOOP-19059. Update AWS Java SDK to 2.23.19 [hadoop]

2024-02-07 Thread via GitHub
adnanhemani commented on PR #6538: URL: https://github.com/apache/hadoop/pull/6538#issuecomment-1933326099 Integration and Unit Test results: ``` 88665a0afd7e:hadoop-aws ahemani$ mvn verify -Dparallel-tests -Dprefetch -DtestsThreadCount=8 Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.fo

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815505#comment-17815505 ] ASF GitHub Bot commented on HADOOP-19059: - adnanhemani opened a new pull reques

[PR] HADOOP-19059. Update AWS Java SDK to 2.23.19 [hadoop]

2024-02-07 Thread via GitHub
adnanhemani opened a new pull request, #6538: URL: https://github.com/apache/hadoop/pull/6538 ### Description of PR This upgrades the AWS Java SDK version to 2.23.19, which brings the S3 Access Grants plugin into the bundle.jar. We can then use that plugin to support S3 Access Gr

[jira] [Commented] (HADOOP-19059) update AWS SDK to support S3 Access Grants in S3A

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815503#comment-17815503 ] ASF GitHub Bot commented on HADOOP-19059: - adnanhemani commented on PR #6506: U

Re: [PR] HADOOP-19059. Update AWS SDK to v2.23.7 [hadoop]

2024-02-07 Thread via GitHub
adnanhemani commented on PR #6506: URL: https://github.com/apache/hadoop/pull/6506#issuecomment-1933319453 As per https://github.com/apache/hadoop/pull/6507#issuecomment-1933245232, I'm marking this PR as deprecated and will be opening a new one with a larger AWS Java SDK upgrade. -- Thi

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
xuzifu666 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1933310950 > > Well, this may not belong to the HDFS module. > > @zhangshuyan0 Thank you for your review! but I think this PR needs Steve’s help to review it. > > @xuzifu666 Thanks for

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
slfan1989 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1933302730 > Well, this may not belong to the HDFS module. @zhangshuyan0 Thank you for your review! but I think this PR needs Steve’s help to review it. @xuzifu666 Thanks for the contribut

[jira] [Updated] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-19071: Labels: pull-request-available (was: ) > Update maven-surefire-plugin from 3.0.0 to 3.2.

[PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]

2024-02-07 Thread via GitHub
slfan1989 opened a new pull request, #6537: URL: https://github.com/apache/hadoop/pull/6537 ### Description of PR JIRA: HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. ### How was this patch tested? ### For code changes: - [ ] Does the t

[jira] [Commented] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815497#comment-17815497 ] ASF GitHub Bot commented on HADOOP-19071: - slfan1989 opened a new pull request,

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
zhangshuyan0 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1933299546 Well, this may not belong to the HDFS module. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Commented] (HADOOP-18980) S3A credential provider remapping: make extensible

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815493#comment-17815493 ] ASF GitHub Bot commented on HADOOP-18980: - hadoop-yetus commented on PR #6525:

Re: [PR] HADOOP-18980. S3A credential provider remapping: make extensible (#6406) [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6525: URL: https://github.com/apache/hadoop/pull/6525#issuecomment-1933297866 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Created] (HADOOP-19071) Update maven-surefire-plugin from 3.0.0 to 3.2.2

2024-02-07 Thread Shilun Fan (Jira)
Shilun Fan created HADOOP-19071: --- Summary: Update maven-surefire-plugin from 3.0.0 to 3.2.2 Key: HADOOP-19071 URL: https://issues.apache.org/jira/browse/HADOOP-19071 Project: Hadoop Common Is

[jira] [Created] (HADOOP-19070) Java 17 compile support

2024-02-07 Thread Shilun Fan (Jira)
Shilun Fan created HADOOP-19070: --- Summary: Java 17 compile support Key: HADOOP-19070 URL: https://issues.apache.org/jira/browse/HADOOP-19070 Project: Hadoop Common Issue Type: Task

[jira] [Commented] (HADOOP-19058) [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815483#comment-17815483 ] ASF GitHub Bot commented on HADOOP-19058: - slfan1989 commented on PR #6531: URL

Re: [PR] HADOOP-19058. [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn [hadoop]

2024-02-07 Thread via GitHub
slfan1989 commented on PR #6531: URL: https://github.com/apache/hadoop/pull/6531#issuecomment-1933275327 > commented. I looked at surefire and it is at 3.2.5; probably time to upgrade. I propose it is done before doing this one, splitting them in two. > > and we need java8 and java17

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
xuzifu666 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1933246860 @steveloughran PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] HADOOP-19050, Add Support for AWS S3 Access Grants [hadoop]

2024-02-07 Thread via GitHub
adnanhemani commented on PR #6507: URL: https://github.com/apache/hadoop/pull/6507#issuecomment-1933245232 Update: I've successfully influenced the S3 Access Grants team and the AWS Java SDK team to include the S3 Access Grants Plugin within the AWS Java SDK bundle. For that, we will requir

[jira] [Commented] (HADOOP-19050) Add S3 Access Grants Support in S3A

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815467#comment-17815467 ] ASF GitHub Bot commented on HADOOP-19050: - adnanhemani commented on PR #6507: U

Re: [PR] HDFS-17316 Add hadoop-compat-bench [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6535: URL: https://github.com/apache/hadoop/pull/6535#issuecomment-1933237018 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] YARN-11649. [Federation] Create Override for Maximum Resource Capability in getNewApplication [hadoop]

2024-02-07 Thread via GitHub
jchanggg commented on PR #6481: URL: https://github.com/apache/hadoop/pull/6481#issuecomment-1933105891 Fixed all the lint issues. Could you help review this change @slfan1989 ? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. [hadoop]

2024-02-07 Thread via GitHub
slfan1989 commented on PR #6533: URL: https://github.com/apache/hadoop/pull/6533#issuecomment-1933081495 @Hexiaoqiao Can you help review this PR? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] HDFS-17146.Use the dfsadmin -reconfig command to initiate reconfiguration on all decommissioning datanodes. [hadoop]

2024-02-07 Thread via GitHub
slfan1989 commented on PR #6504: URL: https://github.com/apache/hadoop/pull/6504#issuecomment-1933082242 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815451#comment-17815451 ] ASF GitHub Bot commented on HADOOP-19069: - slfan1989 commented on PR #6533: URL

Re: [PR] [NOT FOR MERGE] test with shaded protobuf-java 3.21 (snapshot) [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6350: URL: https://github.com/apache/hadoop/pull/6350#issuecomment-1933069853 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6536: URL: https://github.com/apache/hadoop/pull/6536#issuecomment-1932775446 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815398#comment-17815398 ] ASF GitHub Bot commented on HADOOP-19057: - hadoop-yetus commented on PR #6515:

Re: [PR] HADOOP-19057. S3A: Landsat bucket used in tests no longer accessible [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6515: URL: https://github.com/apache/hadoop/pull/6515#issuecomment-1932653106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17361. DiskBalancer: Query command support with multiple nodes [hadoop]

2024-02-07 Thread via GitHub
tasanuma commented on code in PR #6508: URL: https://github.com/apache/hadoop/pull/6508#discussion_r1481790761 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/diskbalancer/command/QueryCommand.java: ## @@ -56,52 +66,72 @@ public QueryCommand(Config

[jira] [Commented] (HADOOP-18830) S3A: Cut S3 Select

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815359#comment-17815359 ] ASF GitHub Bot commented on HADOOP-18830: - hadoop-yetus commented on PR #6523:

Re: [PR] HADOOP-18830. Cut S3 Select (#6144) [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6523: URL: https://github.com/apache/hadoop/pull/6523#issuecomment-1932467536 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Updated] (HADOOP-19045) S3A: pass request timeouts down to sdk clients

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19045: Description: s3a client timeout settings are getting down to http client, but not sdk ti

[jira] [Resolved] (HADOOP-19045) S3A: pass request timeouts down to sdk clients

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19045. - Resolution: Fixed > S3A: pass request timeouts down to sdk clients > --

Re: [PR] HADOOP-18980. S3A credential provider remapping: make extensible (#6406) [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6525: URL: https://github.com/apache/hadoop/pull/6525#issuecomment-1932273541 afraid another change has broken your merge. rebase and force push -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815330#comment-17815330 ] ASF GitHub Bot commented on HADOOP-14837: - steveloughran commented on PR #6407:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1932206238 @bpahuja can you not do rebase and force push unless some other change has broken the merge? breaks the "changes since your last review" feature. If someone has stepped on the code,

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815328#comment-17815328 ] ASF GitHub Bot commented on HADOOP-14837: - steveloughran commented on PR #6407:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1932200850 checkstyle failures, we all get those. and sometimes we have to say "better if broken" ``` ./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/api/RequestFactory.j

[jira] [Commented] (HADOOP-19043) S3A: Regression: ITestS3AOpenCost fails on prefetch test runs

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815327#comment-17815327 ] ASF GitHub Bot commented on HADOOP-19043: - steveloughran commented on PR #6465:

Re: [PR] HADOOP-19043. S3A: Regression: ITestS3AOpenCost fails on prefetch test runs [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6465: URL: https://github.com/apache/hadoop/pull/6465#issuecomment-1932193942 Ready to go in; reviews welcome. @ahmarsuhail @mukund-thakur @HarshitGupta11 -- This is an automated message from the Apache Git Service. To respond to the message, please l

[jira] [Updated] (HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18993: Description: In HADOOP-17372 the S3AFileSystem forces the configuration classloader to be

[jira] [Updated] (HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18993: Component/s: fs/s3 (was: hadoop-thirdparty) > S3A: Add option fs.s3a

[jira] [Updated] (HADOOP-18993) S3A: Add option fs.s3a.classloader.isolation (#6301)

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18993: Summary: S3A: Add option fs.s3a.classloader.isolation (#6301) (was: Allow to not isolate

[jira] [Updated] (HADOOP-18993) Allow to not isolate S3AFileSystem classloader when needed

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18993: Fix Version/s: 3.4.1 > Allow to not isolate S3AFileSystem classloader when needed > -

Re: [PR] Hadoop 18993 - target 3.4 [hadoop]

2024-02-07 Thread via GitHub
steveloughran merged PR #6529: URL: https://github.com/apache/hadoop/pull/6529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-18993) Allow to not isolate S3AFileSystem classloader when needed

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815322#comment-17815322 ] ASF GitHub Bot commented on HADOOP-18993: - steveloughran commented on PR #6301:

Re: [PR] HADOOP-18993 Allow to not isolate S3AFileSystem classloader when needed [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6301: URL: https://github.com/apache/hadoop/pull/6301#issuecomment-1932165399 thanks. low priority issue as it'll just be fs leakage somewhere. simplest fix would be to use closeAllForUGI() in setup to destroy any old ones around -- This is an automated mess

[PR] HDFS-17374. EC: StripedBlockReader#newConnectedPeer should set SO_TIMEOUT and SO_KEEPALIVE [hadoop]

2024-02-07 Thread via GitHub
hfutatzhanghb opened a new pull request, #6536: URL: https://github.com/apache/hadoop/pull/6536 ### Description of PR Refer to HDFS-17374. more details need time to finish. ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR star

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1932003350 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1931980610 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19045) S3A: pass request timeouts down to sdk clients

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815271#comment-17815271 ] ASF GitHub Bot commented on HADOOP-19045: - steveloughran merged PR #6532: URL:

Re: [PR] HADOOP-19045. CreateSession Timeout - followup [hadoop]

2024-02-07 Thread via GitHub
steveloughran merged PR #6532: URL: https://github.com/apache/hadoop/pull/6532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-19069) Use hadoop-thirdparty 1.2.0

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815270#comment-17815270 ] ASF GitHub Bot commented on HADOOP-19069: - hadoop-yetus commented on PR #6533:

Re: [PR] HADOOP-19069. Use hadoop-thirdparty 1.2.0. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6533: URL: https://github.com/apache/hadoop/pull/6533#issuecomment-1931908407 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Updated] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-19057: Description: The s3 test bucket used in hadoop-aws tests of S3 select and large file read

Re: [PR] Hadoop 18993 - target 3.4 [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6529: URL: https://github.com/apache/hadoop/pull/6529#issuecomment-1931901281 yeah, landsat bucket failures. see #6523. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Commented] (HADOOP-19058) [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815264#comment-17815264 ] ASF GitHub Bot commented on HADOOP-19058: - steveloughran commented on PR #6531:

Re: [PR] HADOOP-19058. [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6531: URL: https://github.com/apache/hadoop/pull/6531#issuecomment-1931896429 commented. I looked at surefire and it is at 3.2.5; probably time to upgrade. I propose it is done before doing this one, splitting them in two. and we need java8 and java17 pr

[PR] HDFS-17316 Add hadoop-compat-bench [hadoop]

2024-02-07 Thread via GitHub
HanFreedom opened a new pull request, #6535: URL: https://github.com/apache/hadoop/pull/6535 ### Description of PR A new hadoop-compat-bench module which is directly related to HDFS-17316. ### How was this patch tested? Unit test ### For code changes: - [ ]

[jira] [Commented] (HADOOP-19058) [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815261#comment-17815261 ] ASF GitHub Bot commented on HADOOP-19058: - steveloughran commented on code in P

Re: [PR] HADOOP-19058. [JDK-17] Fix UT Failures in hadoop common, hdfs, yarn [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on code in PR #6531: URL: https://github.com/apache/hadoop/pull/6531#discussion_r1481358429 ## hadoop-project/pom.xml: ## @@ -168,7 +168,18 @@ [3.3.0,) --Xmx2048m -XX:+HeapDumpOnOutOfMemoryError +-Xmx2048m -XX:+HeapDumpOnOutOfMemoryE

[jira] [Commented] (HADOOP-19068) ITestS3AClosedFS.testClosedInstrumentation fails on trunk on Mac OS Silicon

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815260#comment-17815260 ] Steve Loughran commented on HADOOP-19068: - do a parallel run to see if that mak

Re: [PR] YARN-11387. [GPG] YARN GPG mistakenly deleted applicationid. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6473: URL: https://github.com/apache/hadoop/pull/6473#issuecomment-1931867203 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up [hadoop]

2024-02-07 Thread via GitHub
ThinkerLei commented on PR #6498: URL: https://github.com/apache/hadoop/pull/6498#issuecomment-1931846902 @zhangshuyan0 @simbadzina Add a test case, please help to review again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6498: URL: https://github.com/apache/hadoop/pull/6498#issuecomment-1931819876 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1931817110 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815249#comment-17815249 ] ASF GitHub Bot commented on HADOOP-14837: - hadoop-yetus commented on PR #6407:

Re: [PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
xuzifu666 commented on PR #6534: URL: https://github.com/apache/hadoop/pull/6534#issuecomment-1931780456 @ayushtkn @slfan1989 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[PR] HDFS-17373. Fix connection leak in FolderRenamePending [hadoop]

2024-02-07 Thread via GitHub
xuzifu666 opened a new pull request, #6534: URL: https://github.com/apache/hadoop/pull/6534 ### Description of PR Currently connection leak in FolderRenamePending in getting bytes. the pr aim to fix it ### How was this patch tested? ### For code changes:

[jira] [Commented] (HADOOP-19044) AWS SDK V2 - Update S3A region logic

2024-02-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815233#comment-17815233 ] Steve Loughran commented on HADOOP-19044: - as well as the fips stuff, we've hit

Re: [PR] YARN-11387. [GPG] YARN GPG mistakenly deleted applicationid. [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6473: URL: https://github.com/apache/hadoop/pull/6473#issuecomment-1931745994 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-19057) S3 public test bucket landsat-pds unreadable -needs replacement

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815225#comment-17815225 ] ASF GitHub Bot commented on HADOOP-19057: - steveloughran commented on PR #6515:

Re: [PR] HADOOP-19057. Landsat bucket deleted [hadoop]

2024-02-07 Thread via GitHub
steveloughran commented on PR #6515: URL: https://github.com/apache/hadoop/pull/6515#issuecomment-1931710763 file s3a://noaa-cors-pds/raw/2023/017/ohfh/OHFH017d.23_.gz is longer; should stop eof regressions on the older branches. -- This is an automated message from the Apache Git Service

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815211#comment-17815211 ] ASF GitHub Bot commented on HADOOP-14837: - bpahuja commented on PR #6407: URL:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
bpahuja commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1931661739 Build failed due to an OOM Error, build passing on local ``` [INFO] [INFO] BUILD FAILURE [I

Re: [PR] HDFS-17365. EC: Add extra redunency configuration in checkStreamerFailures to prevent data loss. [hadoop]

2024-02-07 Thread via GitHub
hfutatzhanghb commented on PR #6517: URL: https://github.com/apache/hadoop/pull/6517#issuecomment-1931654235 > This is a good idea for me. I think we should allow different EC strategies to use different sizes of `extraStreamerRedunency`. May be a structure like `ReplaceDatanodeOnFailure`?

[jira] [Commented] (HADOOP-14837) Handle S3A "glacier" data

2024-02-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815153#comment-17815153 ] ASF GitHub Bot commented on HADOOP-14837: - hadoop-yetus commented on PR #6407:

Re: [PR] HADOOP-14837 : Support Read Restored Glacier Objects [hadoop]

2024-02-07 Thread via GitHub
hadoop-yetus commented on PR #6407: URL: https://github.com/apache/hadoop/pull/6407#issuecomment-1931517312 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe