[GitHub] [hadoop] MingXiangLi commented on a diff in pull request #4558: HDFS-16657 Changing pool-level lock to volume-level lock for invalida…

2022-07-16 Thread GitBox
MingXiangLi commented on code in PR #4558: URL: https://github.com/apache/hadoop/pull/4558#discussion_r922773966 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java: ## @@ -2312,23 +2312,23 @@ private void

[GitHub] [hadoop] hadoop-yetus commented on pull request #4568: HDFS-16664. Use correct GenerationStamp when invalidating corrupt block replicas

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4568: URL: https://github.com/apache/hadoop/pull/4568#issuecomment-1186359471 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18330) S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18330?focusedWorklogId=791702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791702 ] ASF GitHub Bot logged work on HADOOP-18330: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4572: HADOOP-18330-S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4572: URL: https://github.com/apache/hadoop/pull/4572#issuecomment-1186311338 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri commented on a diff in pull request #4540: YARN-11160. Support getResourceProfiles, getResourceProfile API's for Federation

2022-07-16 Thread GitBox
goiri commented on code in PR #4540: URL: https://github.com/apache/hadoop/pull/4540#discussion_r922725216 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java: ## @@ -917,4 +919,17 @@ private static void

[GitHub] [hadoop] goiri commented on a diff in pull request #4543: YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs

2022-07-16 Thread GitBox
goiri commented on code in PR #4543: URL: https://github.com/apache/hadoop/pull/4543#discussion_r922724973 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4567: HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4567: URL: https://github.com/apache/hadoop/pull/4567#issuecomment-1186292751 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18330) S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18330?focusedWorklogId=791690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791690 ] ASF GitHub Bot logged work on HADOOP-18330: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ashutoshpant commented on pull request #4557: HADOOP-18330 S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread GitBox
ashutoshpant commented on PR #4557: URL: https://github.com/apache/hadoop/pull/4557#issuecomment-1186290248 > aah, I'd merged this and only then noticed this was against 3.3.3. reverted. > > Can you create a PR with the final patch applied to trunk? and test it (just tell us the

[jira] [Work logged] (HADOOP-18330) S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18330?focusedWorklogId=791689=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791689 ] ASF GitHub Bot logged work on HADOOP-18330: --- Author: ASF GitHub Bot

[GitHub] [hadoop] ashutoshpant opened a new pull request, #4572: HADOOP-18330-S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread GitBox
ashutoshpant opened a new pull request, #4572: URL: https://github.com/apache/hadoop/pull/4572 ### Description of PR added new parameter object in s3ClientCreationParameters that returns full s3a path ### How was this patch tested? Tested against the following endpoint ->

[GitHub] [hadoop] hadoop-yetus commented on pull request #4571: HDFS-16464. Create only libhdfspp static libraries for Windows

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4571: URL: https://github.com/apache/hadoop/pull/4571#issuecomment-1186284772 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18074) Partial/Incomplete groups list can be returned in LDAP groups lookup

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18074?focusedWorklogId=791684=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791684 ] ASF GitHub Bot logged work on HADOOP-18074: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4570: HADOOP-18074 - Partial/Incomplete groups list can be returned in LDAP. (#4503)

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4570: URL: https://github.com/apache/hadoop/pull/4570#issuecomment-1186283078 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18215) Enhance WritableName to be able to return aliases for classes that use serializers

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18215?focusedWorklogId=791683=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791683 ] ASF GitHub Bot logged work on HADOOP-18215: --- Author: ASF GitHub Bot

[GitHub] [hadoop] bbeaudreault commented on pull request #4215: HADOOP-18215. Enhance WritableName to be able to return aliases for classes that use serializers

2022-07-16 Thread GitBox
bbeaudreault commented on PR #4215: URL: https://github.com/apache/hadoop/pull/4215#issuecomment-1186281286 @jojochuang thabka again for looking at this. Do you think you could merge given the above test results? -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2022-07-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567536#comment-17567536 ] Steve Loughran commented on HADOOP-18296: - when you run spark standalone locally, checksumfs is

[GitHub] [hadoop] hadoop-yetus commented on pull request #4514: YARN-11200. Backport Numa patch to 2.10

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4514: URL: https://github.com/apache/hadoop/pull/4514#issuecomment-1186263947 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] GauthamBanasandra opened a new pull request, #4571: HDFS-16464. Create only libhdfspp static libraries for Windows

2022-07-16 Thread GitBox
GauthamBanasandra opened a new pull request, #4571: URL: https://github.com/apache/hadoop/pull/4571 ### Description of PR While building dynamic libraries (.dll) on Windows, there's a constraint that all the dependent libraries be linked dynamically. This poses an issue since

[jira] [Commented] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567527#comment-17567527 ] Steve Loughran commented on HADOOP-18340: - This is vaguely familiar. I think we hit it with

[jira] [Updated] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18340: Priority: Minor (was: Major) > deleteOnExit does not work with S3AFileSystem >

[jira] [Assigned] (HADOOP-18340) deleteOnExit does not work with S3AFileSystem

2022-07-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18340: --- Component/s: fs/s3 (was: fs/s3)

[jira] [Work logged] (HADOOP-18074) Partial/Incomplete groups list can be returned in LDAP groups lookup

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18074?focusedWorklogId=791676=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791676 ] ASF GitHub Bot logged work on HADOOP-18074: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18074) Partial/Incomplete groups list can be returned in LDAP groups lookup

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18074?focusedWorklogId=791675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791675 ] ASF GitHub Bot logged work on HADOOP-18074: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4570: HADOOP-18074 - Partial/Incomplete groups list can be returned in LDAP. (#4503)

2022-07-16 Thread GitBox
steveloughran commented on PR #4570: URL: https://github.com/apache/hadoop/pull/4570#issuecomment-1186247878 #4503 cherrypicked to 3.3.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] steveloughran opened a new pull request, #4570: HADOOP-18074 - Partial/Incomplete groups list can be returned in LDAP. (#4503)

2022-07-16 Thread GitBox
steveloughran opened a new pull request, #4570: URL: https://github.com/apache/hadoop/pull/4570 Partial/Incomplete groups list can be returned in LDAP groups lookup. Backported in #4550; minor tuning of parameters needed. Contributed by larry mccay -- This is an

[jira] [Work logged] (HADOOP-18332) remove rs-api dependency by downgrading jackson to 2.12.7

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18332?focusedWorklogId=791674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791674 ] ASF GitHub Bot logged work on HADOOP-18332: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4569: HADOOP-18332. Remove rs-api dependency by downgrading jackson to 2.12.7. (#4552)

2022-07-16 Thread GitBox
steveloughran commented on PR #4569: URL: https://github.com/apache/hadoop/pull/4569#issuecomment-1186245945 pushing through yetus before merging -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Work logged] (HADOOP-18332) remove rs-api dependency by downgrading jackson to 2.12.7

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18332?focusedWorklogId=791673=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791673 ] ASF GitHub Bot logged work on HADOOP-18332: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran opened a new pull request, #4569: HADOOP-18332. Remove rs-api dependency by downgrading jackson to 2.12.7. (#4552)

2022-07-16 Thread GitBox
steveloughran opened a new pull request, #4569: URL: https://github.com/apache/hadoop/pull/4569 This downgrades jackson from the version switched to in HADOOP-18033 (2.13.0), to Jackson 2.12.7. This removes the dependency on javax.ws.rs-api, so avoiding runtime problems with

[GitHub] [hadoop] steveloughran merged pull request #4552: HADOOP-18332. remove rs-api dependency (3.3 branch)

2022-07-16 Thread GitBox
steveloughran merged PR #4552: URL: https://github.com/apache/hadoop/pull/4552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Work logged] (HADOOP-18332) remove rs-api dependency by downgrading jackson to 2.12.7

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18332?focusedWorklogId=791672=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791672 ] ASF GitHub Bot logged work on HADOOP-18332: --- Author: ASF GitHub Bot

[jira] [Work logged] (HADOOP-18332) remove rs-api dependency by downgrading jackson to 2.12.7

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18332?focusedWorklogId=791671=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791671 ] ASF GitHub Bot logged work on HADOOP-18332: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4552: HADOOP-18332. remove rs-api dependency (3.3 branch)

2022-07-16 Thread GitBox
steveloughran commented on PR #4552: URL: https://github.com/apache/hadoop/pull/4552#issuecomment-1186243757 +1, merging to 3.3 with a detailed commit message, then i will submit as a 3.3.4 change & merge in there -- This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HADOOP-18332) remove rs-api dependency by downgrading jackson to 2.12.7

2022-07-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18332: Summary: remove rs-api dependency by downgrading jackson to 2.12.7 (was: remove rs-api

[jira] [Work logged] (HADOOP-18330) S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18330?focusedWorklogId=791669=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791669 ] ASF GitHub Bot logged work on HADOOP-18330: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran commented on pull request #4557: HADOOP-18330 S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread GitBox
steveloughran commented on PR #4557: URL: https://github.com/apache/hadoop/pull/4557#issuecomment-1186239135 aah, I'd merged this and only then noticed this was against 3.3.3. reverted. Can you create a PR with the final patch applied to trunk? and test it (just tell us the endpoint,

[jira] [Work logged] (HADOOP-18330) S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18330?focusedWorklogId=791668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791668 ] ASF GitHub Bot logged work on HADOOP-18330: --- Author: ASF GitHub Bot

[GitHub] [hadoop] steveloughran merged pull request #4557: HADOOP-18330 S3AFileSystem removes Path when calling createS3Client

2022-07-16 Thread GitBox
steveloughran merged PR #4557: URL: https://github.com/apache/hadoop/pull/4557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] KevinWikant opened a new pull request, #4568: HDFS-16664. Use correct GenerationStamp when invalidating corrupt replicas

2022-07-16 Thread GitBox
KevinWikant opened a new pull request, #4568: URL: https://github.com/apache/hadoop/pull/4568 ### Description of PR Under certain conditions the Namenode can send the incorrect generationStamp to a datanode when invalidating a corrupt block replica. - the generationStamp sent

[GitHub] [hadoop] hadoop-yetus commented on pull request #4567: HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4567: URL: https://github.com/apache/hadoop/pull/4567#issuecomment-1186216211 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4311: HDFS-13522: IPC changes to support observer reads through routers.

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4311: URL: https://github.com/apache/hadoop/pull/4311#issuecomment-1186182868 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18302) Remove WhiteBox in hadoop-common module.

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18302?focusedWorklogId=791655=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791655 ] ASF GitHub Bot logged work on HADOOP-18302: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4457: URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1186180917 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4463: YARN-11187. Remove WhiteBox in yarn module.

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4463: URL: https://github.com/apache/hadoop/pull/4463#issuecomment-1186163408 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] GauthamBanasandra commented on pull request #4538: HDFS-16654. Link OpenSSL lib for CMake deps check

2022-07-16 Thread GitBox
GauthamBanasandra commented on PR #4538: URL: https://github.com/apache/hadoop/pull/4538#issuecomment-1186148893 @iwasakims I'm able to verify it on my Windows system - ``` Performing C SOURCE FILE Test HAS_NEW_ENOUGH_OPENSSL succeeded with the following output: Change Dir:

[GitHub] [hadoop] lfxy opened a new pull request, #4567: HDFS-16663. Allow block reconstruction pending timeout refreshable to increase decommission performance

2022-07-16 Thread GitBox
lfxy opened a new pull request, #4567: URL: https://github.com/apache/hadoop/pull/4567 In [HDFS-16613](https://issues.apache.org/jira/browse/HDFS-16613), increase the value of dfs.namenode.replication.max-streams-hard-limit would maximize the IO performance of the decommissioning DN, witch

[GitHub] [hadoop] hadoop-yetus commented on pull request #4462: MAPREDUCE-7390 Remove WhiteBox in mapreduce module.

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4462: URL: https://github.com/apache/hadoop/pull/4462#issuecomment-1186117924 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Work logged] (HADOOP-18302) Remove WhiteBox in hadoop-common module.

2022-07-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18302?focusedWorklogId=791637=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-791637 ] ASF GitHub Bot logged work on HADOOP-18302: --- Author: ASF GitHub Bot

[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4457: URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1186108416 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4543: YARN-8900. [Router] Federation: routing getContainers REST invocations transparently to multiple RMs

2022-07-16 Thread GitBox
hadoop-yetus commented on PR #4543: URL: https://github.com/apache/hadoop/pull/4543#issuecomment-1186106709 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |