[GitHub] [hadoop] xinglin closed pull request #5089: HDFS-15654/15674: Fix TestBPOfferService#testMissBlocksWhenReregister

2023-06-09 Thread via GitHub
xinglin closed pull request #5089: HDFS-15654/15674: Fix TestBPOfferService#testMissBlocksWhenReregister URL: https://github.com/apache/hadoop/pull/5089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] xinglin commented on pull request #5089: HDFS-15654/15674: Fix TestBPOfferService#testMissBlocksWhenReregister

2023-06-09 Thread via GitHub
xinglin commented on PR #5089: URL: https://github.com/apache/hadoop/pull/5089#issuecomment-1585495645 close this PR as two of these commits have been backported to branch-3.3 by others. ``` Author: Masatake Iwasaki AuthorDate: Wed Nov 18 16:11:09 2020 +0900 Commit:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5696: HDFS-16946. Fix getTopTokenRealOwners to return String

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5696: URL: https://github.com/apache/hadoop/pull/5696#issuecomment-1585452857 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18761) Remove mysql-connector-java

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731145#comment-17731145 ] Wei-Chiu Chuang commented on HADOOP-18761: -- Ok. I took a different approach. I

[jira] [Commented] (HADOOP-18761) Remove mysql-connector-java

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731143#comment-17731143 ] ASF GitHub Bot commented on HADOOP-18761: - hadoop-yetus commented on PR #5731:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5731: HADOOP-18761. Remove mysql-connector-java

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5731: URL: https://github.com/apache/hadoop/pull/5731#issuecomment-1585415842 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18763) Upgrade aws-java-sdk to 1.12.367+

2023-06-09 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731132#comment-17731132 ] Viraj Jasani commented on HADOOP-18763: --- we were excluding netty from aws-sdk? {c

[jira] [Commented] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731128#comment-17731128 ] ASF GitHub Bot commented on HADOOP-18762: - hadoop-yetus commented on PR #5725:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5725: HADOOP-18762. Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5725: URL: https://github.com/apache/hadoop/pull/5725#issuecomment-1585297527 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731125#comment-17731125 ] ASF GitHub Bot commented on HADOOP-18762: - hadoop-yetus commented on PR #5725:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5725: HADOOP-18762. Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5725: URL: https://github.com/apache/hadoop/pull/5725#issuecomment-1585292157 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18655) Upgrade Kerby to 2.0.3 due to CVE-2023-25613

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731115#comment-17731115 ] ASF GitHub Bot commented on HADOOP-18655: - jojochuang commented on PR #5484: UR

[GitHub] [hadoop] jojochuang commented on pull request #5484: HADOOP-18655. Upgrade Kerby to 2.0.3

2023-06-09 Thread via GitHub
jojochuang commented on PR #5484: URL: https://github.com/apache/hadoop/pull/5484#issuecomment-1585230825 The solr version is the same in branch-3.3 and trunk. So we would need to do the same for trunk. We could possible update Solr to 9.x. Looks like Solr 9 does not depend on Kerby.

[GitHub] [hadoop] hadoop-yetus commented on pull request #5730: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5730: URL: https://github.com/apache/hadoop/pull/5730#issuecomment-1585215169 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5730: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5730: URL: https://github.com/apache/hadoop/pull/5730#issuecomment-1585203549 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5730: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5730: URL: https://github.com/apache/hadoop/pull/5730#issuecomment-1585202549 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18761) Remove mysql-connector-java

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731105#comment-17731105 ] ASF GitHub Bot commented on HADOOP-18761: - jojochuang opened a new pull request

[GitHub] [hadoop] jojochuang opened a new pull request, #5731: HADOOP-18761. Remove mysql-connector-java

2023-06-09 Thread via GitHub
jojochuang opened a new pull request, #5731: URL: https://github.com/apache/hadoop/pull/5731 ### Description of PR mysql-connector-java is GPL licensed. It was included as an optional feature in HDFS RBF. Fortunately, it looks like we do not need to take compile dependency on it at al

[jira] [Assigned] (HADOOP-18761) Remove mysql-connector-java

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-18761: Assignee: Wei-Chiu Chuang > Remove mysql-connector-java > --

[jira] [Updated] (HADOOP-18761) Remove mysql-connector-java

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18761: - Summary: Remove mysql-connector-java (was: Revert HADOOP-18535 because mysql-connector

[GitHub] [hadoop] xinglin closed pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router

2023-06-09 Thread via GitHub
xinglin closed pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router URL: https://github.com/apache/hadoop/pull/5071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] xinglin commented on pull request #5071: HDFS-16816 RBF: auto-create user home dir for trash paths by router

2023-06-09 Thread via GitHub
xinglin commented on PR #5071: URL: https://github.com/apache/hadoop/pull/5071#issuecomment-1585083991 this PR is no longer pursued. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] xinglin commented on pull request #5730: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread via GitHub
xinglin commented on PR #5730: URL: https://github.com/apache/hadoop/pull/5730#issuecomment-1585081937 Hi @goiri, Please kindly review this PR as well if you have bandwidth. thanks, -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [hadoop] xinglin commented on pull request #5700: HDFS-17030. Limit wait time for getHAServiceState in ObserverReaderProxy

2023-06-09 Thread via GitHub
xinglin commented on PR #5700: URL: https://github.com/apache/hadoop/pull/5700#issuecomment-1585078365 Hi @goiri, We are not making further code changes to this PR and there is a clean build without any unit test failures (the second to the last build). Could you take a look now? tha

[GitHub] [hadoop] hadoop-yetus commented on pull request #5728: MAPREDUCE-7435. Manifest Committer OOM on abfs (#5519)

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5728: URL: https://github.com/apache/hadoop/pull/5728#issuecomment-1585066047 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18646) Upgrade Netty to 4.1.89.Final

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731078#comment-17731078 ] ASF GitHub Bot commented on HADOOP-18646: - hadoop-yetus commented on PR #5729:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5729: HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 (#5435)

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5729: URL: https://github.com/apache/hadoop/pull/5729#issuecomment-1585039246 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] xinglin opened a new pull request, #5730: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode

2023-06-09 Thread via GitHub
xinglin opened a new pull request, #5730: URL: https://github.com/apache/hadoop/pull/5730 ### Description of PR Add two new types of metrics to the existing NN RpcMetrics/RpcDetailedMetrics. These two metrics can then be used as part of SLA/SLO for the HDFS service. - RpcCallSucce

[GitHub] [hadoop] ayushtkn commented on pull request #4990: HDFS-13507. RBF RouterAdmin disable update functionality in add cmd and supports add/update one mount table with different destination

2023-06-09 Thread via GitHub
ayushtkn commented on PR #4990: URL: https://github.com/apache/hadoop/pull/4990#issuecomment-1584996396 Builds are clean, nobody has additional feedback either nor objections, looks good then. If no sounds would merge by early next week. @simbadzina in case this interests you and you

[jira] [Commented] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731066#comment-17731066 ] ASF GitHub Bot commented on HADOOP-18762: - hadoop-yetus commented on PR #5725:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5725: HADOOP-18762. Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5725: URL: https://github.com/apache/hadoop/pull/5725#issuecomment-1584994059 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] ayushtkn merged pull request #5683: HDFS-17023. RBF: Record proxy time when call invokeConcurrent method.

2023-06-09 Thread via GitHub
ayushtkn merged PR #5683: URL: https://github.com/apache/hadoop/pull/5683 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hado

[GitHub] [hadoop] hadoop-yetus commented on pull request #5683: HDFS-17023. RBF: Record proxy time when call invokeConcurrent method.

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5683: URL: https://github.com/apache/hadoop/pull/5683#issuecomment-1584972089 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Updated] (HADOOP-18766) 3.3.6 Dependency vulnerability check

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18766: - Attachment: dependency-check-report.html > 3.3.6 Dependency vulnerability check > -

[jira] [Assigned] (HADOOP-18766) 3.3.6 Dependency vulnerability check

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-18766: Assignee: Wei-Chiu Chuang > 3.3.6 Dependency vulnerability check > -

[jira] [Created] (HADOOP-18766) 3.3.6 Dependency vulnerability check

2023-06-09 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HADOOP-18766: Summary: 3.3.6 Dependency vulnerability check Key: HADOOP-18766 URL: https://issues.apache.org/jira/browse/HADOOP-18766 Project: Hadoop Common Issue

[jira] [Updated] (HADOOP-18646) Upgrade Netty to 4.1.89.Final

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18646: - Target Version/s: 3.3.6 > Upgrade Netty to 4.1.89.Final > -

[jira] [Commented] (HADOOP-18646) Upgrade Netty to 4.1.89.Final

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731050#comment-17731050 ] ASF GitHub Bot commented on HADOOP-18646: - jojochuang opened a new pull request

[GitHub] [hadoop] jojochuang opened a new pull request, #5729: HADOOP-18646. Upgrade Netty to 4.1.89.Final to fix CVE-2022-41881 (#5435)

2023-06-09 Thread via GitHub
jojochuang opened a new pull request, #5729: URL: https://github.com/apache/hadoop/pull/5729 This fixes CVE-2022-41881. This also upgrades io.opencensus dependencies to 0.12.3 Contributed by Aleksandr Nikolaev (cherry picked from commit 734f7abfb8b84a4c20dbae5073c

[jira] [Commented] (HADOOP-18760) 3.3.6 Release NOTICE and LICENSE file update

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731046#comment-17731046 ] Wei-Chiu Chuang commented on HADOOP-18760: -- branch-3.3 is on netty 4.1.77 (tru

[jira] [Updated] (HADOOP-18765) Release 3.3.6

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18765: - Description: * Move out all incomplete jiras * Branching * Unit test and verifications

[jira] [Created] (HADOOP-18765) Release 3.3.6

2023-06-09 Thread Wei-Chiu Chuang (Jira)
Wei-Chiu Chuang created HADOOP-18765: Summary: Release 3.3.6 Key: HADOOP-18765 URL: https://issues.apache.org/jira/browse/HADOOP-18765 Project: Hadoop Common Issue Type: Task

[jira] [Updated] (HADOOP-18740) s3a prefetch cache blocks should be accessed by RW locks

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18740: - Resolution: Fixed Status: Resolved (was: Patch Available) Looks like it's done

[jira] [Updated] (HADOOP-18740) s3a prefetch cache blocks should be accessed by RW locks

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-18740: - Fix Version/s: 3.4.0 3.3.6 > s3a prefetch cache blocks should be acc

[jira] [Commented] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731023#comment-17731023 ] Wei-Chiu Chuang commented on HADOOP-18762: -- +1 what everything Steve said. >

[GitHub] [hadoop] steveloughran opened a new pull request, #5728: MAPREDUCE-7435. Manifest Committer OOM on abfs (#5519)

2023-06-09 Thread via GitHub
steveloughran opened a new pull request, #5728: URL: https://github.com/apache/hadoop/pull/5728 This modifies the manifest committer so that the list of files to rename is passed between stages as a file of writeable entries on the local filesystem. The map of directories to creat

[jira] [Commented] (HADOOP-18756) CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731014#comment-17731014 ] ASF GitHub Bot commented on HADOOP-18756: - virajjasani commented on PR #5718: U

[GitHub] [hadoop] virajjasani commented on pull request #5718: HADOOP-18756. S3A prefetch - CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread via GitHub
virajjasani commented on PR #5718: URL: https://github.com/apache/hadoop/pull/5718#issuecomment-1584823011 test run is complete against `us-west-2`. results look good for `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` -- This is an automated message from th

[GitHub] [hadoop] steveloughran commented on pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-06-09 Thread via GitHub
steveloughran commented on PR #5519: URL: https://github.com/apache/hadoop/pull/5519#issuecomment-1584817582 merged to trunk, now backporting to 3.3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop] steveloughran merged pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-06-09 Thread via GitHub
steveloughran merged PR #5519: URL: https://github.com/apache/hadoop/pull/5519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...

[jira] [Commented] (HADOOP-18756) CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731007#comment-17731007 ] ASF GitHub Bot commented on HADOOP-18756: - virajjasani commented on PR #5718: U

[GitHub] [hadoop] virajjasani commented on pull request #5718: HADOOP-18756. S3A prefetch - CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread via GitHub
virajjasani commented on PR #5718: URL: https://github.com/apache/hadoop/pull/5718#issuecomment-1584775948 > `us-west-2` > > `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` re-running this with the addendum change right now. will post here as soon as

[jira] [Commented] (HADOOP-18756) CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731005#comment-17731005 ] ASF GitHub Bot commented on HADOOP-18756: - virajjasani commented on PR #5718: U

[GitHub] [hadoop] virajjasani commented on pull request #5718: HADOOP-18756. S3A prefetch - CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread via GitHub
virajjasani commented on PR #5718: URL: https://github.com/apache/hadoop/pull/5718#issuecomment-1584771885 Oops, my sincere apologies, i thought the addendum was small enough so didn't re-test. Let me trigger the test run right away against `us-west-2`. -- This is an automated message

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs

2023-06-09 Thread via GitHub
steveloughran commented on code in PR #5519: URL: https://github.com/apache/hadoop/pull/5519#discussion_r1224453999 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/EntryFileIO.j

[GitHub] [hadoop] steveloughran commented on pull request #5718: HADOOP-18756. S3A prefetch - CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread via GitHub
steveloughran commented on PR #5718: URL: https://github.com/apache/hadoop/pull/5718#issuecomment-1584765644 ok. looks good. final check: which s3 store did you run the final build against and what were the settings -- This is an automated message from the Apache Git Service. To respond t

[jira] [Commented] (HADOOP-18756) CachingBlockManager to use AtomicBoolean for closed flag

2023-06-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731003#comment-17731003 ] ASF GitHub Bot commented on HADOOP-18756: - steveloughran commented on PR #5718:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5727: YARN-11509. The FederationInterceptor#launchUAM Added retry logic.

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5727: URL: https://github.com/apache/hadoop/pull/5727#issuecomment-1584720885 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5726: YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5726: URL: https://github.com/apache/hadoop/pull/5726#issuecomment-1584677082 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Updated] (HADOOP-17386) fs.s3a.buffer.dir to be under Yarn container path on yarn applications

2023-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17386: Fix Version/s: 3.3.9 > fs.s3a.buffer.dir to be under Yarn container path on yarn applicat

[jira] [Created] (HADOOP-18764) fs.azure.buffer.dir to be under Yarn container path on yarn applications

2023-06-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18764: --- Summary: fs.azure.buffer.dir to be under Yarn container path on yarn applications Key: HADOOP-18764 URL: https://issues.apache.org/jira/browse/HADOOP-18764 Proj

[GitHub] [hadoop] slfan1989 opened a new pull request, #5727: YARN-11509. The FederationInterceptor#launchUAM Added retry logic.

2023-06-09 Thread via GitHub
slfan1989 opened a new pull request, #5727: URL: https://github.com/apache/hadoop/pull/5727 ### Description of PR JIRA: YARN-11509. The FederationInterceptor#launchUAM Added retry logic. There is a "todo" in the FederationInterceptor#registerAndAllocateWithNewSubClusters me

[GitHub] [hadoop] hadoop-yetus commented on pull request #5716: YARN-11506.The formatted yarn queue list is displayed on CLI

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5716: URL: https://github.com/apache/hadoop/pull/5716#issuecomment-1584414427 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5696: HDFS-16946. Fix getTopTokenRealOwners to return String

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5696: URL: https://github.com/apache/hadoop/pull/5696#issuecomment-1584412839 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18760) 3.3.6 Release NOTICE and LICENSE file update

2023-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730920#comment-17730920 ] Steve Loughran commented on HADOOP-18760: - we need to get an aws sdk update in

[jira] [Created] (HADOOP-18763) Upgrade aws-java-sdk to 1.12.367+

2023-06-09 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18763: --- Summary: Upgrade aws-java-sdk to 1.12.367+ Key: HADOOP-18763 URL: https://issues.apache.org/jira/browse/HADOOP-18763 Project: Hadoop Common Issue Type:

[jira] [Commented] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730918#comment-17730918 ] Steve Loughran commented on HADOOP-18762: - Hello and welcome I'd like to start

[jira] [Updated] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18762: Component/s: fs > Incorporate Qiniu Cloud Kodo File System Implementation > -

[jira] [Updated] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18762: Affects Version/s: 3.3.9 > Incorporate Qiniu Cloud Kodo File System Implementation >

[jira] [Updated] (HADOOP-18762) Incorporate Qiniu Cloud Kodo File System Implementation

2023-06-09 Thread Zhiqiang Zhang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhiqiang Zhang updated HADOOP-18762: Summary: Incorporate Qiniu Cloud Kodo File System Implementation (was: Add Qiniu Cloud K

[GitHub] [hadoop] hadoop-yetus commented on pull request #5716: YARN-11506.The formatted yarn queue list is displayed on CLI

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5716: URL: https://github.com/apache/hadoop/pull/5716#issuecomment-1584345452 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5716: YARN-11506.The formatted yarn queue list is displayed on CLI

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5716: URL: https://github.com/apache/hadoop/pull/5716#issuecomment-1584307324 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5716: YARN-11506.The formatted yarn queue list is displayed on CLI

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5716: URL: https://github.com/apache/hadoop/pull/5716#issuecomment-1584303518 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5722: YARN-11504. [Federation] YARN Federation Supports Non-HA mode.

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5722: URL: https://github.com/apache/hadoop/pull/5722#issuecomment-1584282253 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #5726: YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5726: URL: https://github.com/apache/hadoop/pull/5726#issuecomment-1584261974 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Comment Edited] (HADOOP-18655) Upgrade Kerby to 2.0.3 due to CVE-2023-25613

2023-06-09 Thread Rohit Kumar (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730891#comment-17730891 ] Rohit Kumar edited comment on HADOOP-18655 at 6/9/23 9:23 AM: ---

[jira] [Comment Edited] (HADOOP-18655) Upgrade Kerby to 2.0.3 due to CVE-2023-25613

2023-06-09 Thread Rohit Kumar (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730891#comment-17730891 ] Rohit Kumar edited comment on HADOOP-18655 at 6/9/23 9:19 AM: ---

[jira] [Commented] (HADOOP-18655) Upgrade Kerby to 2.0.3 due to CVE-2023-25613

2023-06-09 Thread Rohit Kumar (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17730891#comment-17730891 ] Rohit Kumar commented on HADOOP-18655: -- Hi [~weichiu] , as per Steve's comment, it

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5696: HDFS-16946. Fix getTopTokenRealOwners to return String

2023-06-09 Thread via GitHub
ayushtkn commented on code in PR #5696: URL: https://github.com/apache/hadoop/pull/5696#discussion_r1224038596 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/security/TestRouterSecurityManager.java: ## @@ -259,6 +272,39 @@ private s

[GitHub] [hadoop] hadoop-yetus commented on pull request #5726: YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5726: URL: https://github.com/apache/hadoop/pull/5726#issuecomment-1584217880 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] NishthaShah commented on a diff in pull request #5696: HDFS-16946. Fix getTopTokenRealOwners to return String

2023-06-09 Thread via GitHub
NishthaShah commented on code in PR #5696: URL: https://github.com/apache/hadoop/pull/5696#discussion_r1224032808 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/security/TestRouterSecurityManager.java: ## @@ -259,6 +272,39 @@ privat

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5715: HDFS-17037. Consider nonDfsUsed when running balancer.

2023-06-09 Thread via GitHub
Hexiaoqiao commented on PR #5715: URL: https://github.com/apache/hadoop/pull/5715#issuecomment-1584168251 Committed to trunk. Thanks @zhangshuyan0 for your contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] Hexiaoqiao merged pull request #5715: HDFS-17037. Consider nonDfsUsed when running balancer.

2023-06-09 Thread via GitHub
Hexiaoqiao merged PR #5715: URL: https://github.com/apache/hadoop/pull/5715 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@ha

[GitHub] [hadoop] hadoop-yetus commented on pull request #5505: HDFS-16963. Remove the unnecessary use of Optional from DistributedFileSystem

2023-06-09 Thread via GitHub
hadoop-yetus commented on PR #5505: URL: https://github.com/apache/hadoop/pull/5505#issuecomment-1584139210 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 opened a new pull request, #5726: YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code

2023-06-09 Thread via GitHub
slfan1989 opened a new pull request, #5726: URL: https://github.com/apache/hadoop/pull/5726 ### Description of PR JIRA: YARN-11508. [Minor] Improve UnmanagedAMPoolManager/UnmanagedApplicationManager Code ### How was this patch tested? ### For code changes: