[GitHub] [hadoop] hadoop-yetus commented on pull request #4687: YARN-10846. Add dispatcher metrics to NM.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4687: URL: https://github.com/apache/hadoop/pull/4687#issuecomment-1229128165 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4769: YARN-8482. [Router] Add cache for fast answers to getApps.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4769: URL: https://github.com/apache/hadoop/pull/4769#issuecomment-1229113907 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585650#comment-17585650 ] ASF GitHub Bot commented on HADOOP-18426: - zhangshuyan0 commented on PR #4811:

[GitHub] [hadoop] zhangshuyan0 commented on pull request #4811: HADOOP-18426. Improve the accuracy of MutableStat mean.

2022-08-26 Thread GitBox
zhangshuyan0 commented on PR #4811: URL: https://github.com/apache/hadoop/pull/4811#issuecomment-1229102067 @Hexiaoqiao @xkrogen Would you mind reviewing this? I noticed that [https://issues.apache.org/jira/browse/HADOOP-13804](url) is a related JIRA. -- This is an automated message from

[GitHub] [hadoop] slfan1989 commented on pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4780: URL: https://github.com/apache/hadoop/pull/4780#issuecomment-1229060959 @zhangshuyan0 fix checkstyle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] slfan1989 commented on pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4757: URL: https://github.com/apache/hadoop/pull/4757#issuecomment-1229048816 @goiri Thank you very much for helping to review the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [hadoop] goiri merged pull request #4763: HDFS-16734. RBF: fix some bugs when handling getContentSummary RPC

2022-08-26 Thread GitBox
goiri merged PR #4763: URL: https://github.com/apache/hadoop/pull/4763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] goiri merged pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.

2022-08-26 Thread GitBox
goiri merged PR #4757: URL: https://github.com/apache/hadoop/pull/4757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4764: YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation.

2022-08-26 Thread GitBox
slfan1989 commented on code in PR #4764: URL: https://github.com/apache/hadoop/pull/4764#discussion_r956480233 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java: ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4764: YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation.

2022-08-26 Thread GitBox
slfan1989 commented on code in PR #4764: URL: https://github.com/apache/hadoop/pull/4764#discussion_r956480015 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java: ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4764: YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation.

2022-08-26 Thread GitBox
slfan1989 commented on code in PR #4764: URL: https://github.com/apache/hadoop/pull/4764#discussion_r956479938 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java: ##

[GitHub] [hadoop] goiri commented on a diff in pull request #4764: YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation.

2022-08-26 Thread GitBox
goiri commented on code in PR #4764: URL: https://github.com/apache/hadoop/pull/4764#discussion_r956442103 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/clientrm/FederationClientInterceptor.java: ##

[jira] [Commented] (HADOOP-18133) Add Dockerfile for Windows 10

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585616#comment-17585616 ] ASF GitHub Bot commented on HADOOP-18133: - goiri commented on code in PR #4765:

[GitHub] [hadoop] goiri commented on a diff in pull request #4765: HADOOP-18133. Add Dockerfile for Windows 10

2022-08-26 Thread GitBox
goiri commented on code in PR #4765: URL: https://github.com/apache/hadoop/pull/4765#discussion_r956441305 ## dev-support/jenkins.sh: ## @@ -224,6 +224,18 @@ function github_status_recovery() { /usr/bin/env bash "${TESTPATCHBIN}" "${YETUS_ARGS[@]}" "${EXTRA_ARGS}" || true }

[jira] [Commented] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585597#comment-17585597 ] ASF GitHub Bot commented on HADOOP-18426: - hadoop-yetus commented on PR #4811:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4811: HADOOP-18426. Improve the accuracy of MutableStat mean.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4811: URL: https://github.com/apache/hadoop/pull/4811#issuecomment-1228842817 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] ashutoshcipher commented on pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call

2022-08-26 Thread GitBox
ashutoshcipher commented on PR #4248: URL: https://github.com/apache/hadoop/pull/4248#issuecomment-1228778495 Thanks @cnauroth for reviewing the PR again and sharing your comments and discussion. I will address last your comments in my next commit. -- This is an automated message from the

[GitHub] [hadoop] cnauroth commented on a diff in pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call

2022-08-26 Thread GitBox
cnauroth commented on code in PR #4248: URL: https://github.com/apache/hadoop/pull/4248#discussion_r956291996 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/MultipleOutputs.java: ## @@ -527,9 +558,41 @@

[GitHub] [hadoop] cnauroth commented on a diff in pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call

2022-08-26 Thread GitBox
cnauroth commented on code in PR #4248: URL: https://github.com/apache/hadoop/pull/4248#discussion_r955418817 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/MultipleOutputs.java: ## @@ -528,8 +545,33 @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #4810: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4810: URL: https://github.com/apache/hadoop/pull/4810#issuecomment-1228763458 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18133) Add Dockerfile for Windows 10

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585538#comment-17585538 ] ASF GitHub Bot commented on HADOOP-18133: - hadoop-yetus commented on PR #4765:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4765: HADOOP-18133. Add Dockerfile for Windows 10

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4765: URL: https://github.com/apache/hadoop/pull/4765#issuecomment-1228712949 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4766: URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1228692815 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585526#comment-17585526 ] ASF GitHub Bot commented on HADOOP-18410: - hadoop-yetus commented on PR #4766:

[GitHub] [hadoop] goiri merged pull request #4782: HDFS-16736. Link to Boost library in libhdfspp

2022-08-26 Thread GitBox
goiri merged PR #4782: URL: https://github.com/apache/hadoop/pull/4782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.

[GitHub] [hadoop] slfan1989 commented on pull request #4746: YARN-9708. Yarn Federation Router Support DelegationToken.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4746: URL: https://github.com/apache/hadoop/pull/4746#issuecomment-1228667429 @goiri Please help to review the code again, Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on pull request #4764: YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4764: URL: https://github.com/apache/hadoop/pull/4764#issuecomment-1228667060 @goiri Please help to review the code again, Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on pull request #4751: YARN-11253. Add Configuration to delegationToken RemoverScanInterval.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4751: URL: https://github.com/apache/hadoop/pull/4751#issuecomment-122878 @goiri Please help to review the code again, Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on pull request #4792: YARN-11275. [Federation] Add batchFinishApplicationMaster in UAMPoolManager.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4792: URL: https://github.com/apache/hadoop/pull/4792#issuecomment-1228666558 @goiri Please help to review the code again, Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Assigned] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread Shuyan Zhang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuyan Zhang reassigned HADOOP-18426: - Assignee: Shuyan Zhang > Improve the accuracy of MutableStat mean > --

[jira] [Commented] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585462#comment-17585462 ] ASF GitHub Bot commented on HADOOP-18426: - zhangshuyan0 opened a new pull reque

[jira] [Updated] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18426: Labels: pull-request-available (was: ) > Improve the accuracy of MutableStat mean >

[GitHub] [hadoop] zhangshuyan0 opened a new pull request, #4811: HADOOP-18426. Improve the accuracy of MutableStat mean.

2022-08-26 Thread GitBox
zhangshuyan0 opened a new pull request, #4811: URL: https://github.com/apache/hadoop/pull/4811 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HA

[jira] [Created] (HADOOP-18426) Improve the accuracy of MutableStat mean

2022-08-26 Thread Shuyan Zhang (Jira)
Shuyan Zhang created HADOOP-18426: - Summary: Improve the accuracy of MutableStat mean Key: HADOOP-18426 URL: https://issues.apache.org/jira/browse/HADOOP-18426 Project: Hadoop Common Issue Ty

[GitHub] [hadoop] hadoop-yetus commented on pull request #4810: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4810: URL: https://github.com/apache/hadoop/pull/4810#issuecomment-1228645189 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4810: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4810: URL: https://github.com/apache/hadoop/pull/4810#issuecomment-1228645254 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4314: YARN-11153. Make proxy server support YARN federation.

2022-08-26 Thread GitBox
slfan1989 commented on code in PR #4314: URL: https://github.com/apache/hadoop/pull/4314#discussion_r956166577 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java: ## @@ -

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4314: YARN-11153. Make proxy server support YARN federation.

2022-08-26 Thread GitBox
slfan1989 commented on code in PR #4314: URL: https://github.com/apache/hadoop/pull/4314#discussion_r956166342 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java: ## @@ -

[GitHub] [hadoop] slfan1989 commented on pull request #4314: YARN-11153. Make proxy server support YARN federation.

2022-08-26 Thread GitBox
slfan1989 commented on PR #4314: URL: https://github.com/apache/hadoop/pull/4314#issuecomment-1228639768 Thank you very much for your contribution, overall it looks ok, but some code is better to extract. -- This is an automated message from the Apache Git Service. To respond to the messa

[jira] [Commented] (HADOOP-18408) [ABFS]: Set correct value of requireRenameResilience for NonHNS-SharedKey configuration

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585447#comment-17585447 ] ASF GitHub Bot commented on HADOOP-18408: - hadoop-yetus commented on PR #4758:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4758: HADOOP-18408. ABFS: Setting correct value of requireRenameResilience for nonHNS configuration

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4758: URL: https://github.com/apache/hadoop/pull/4758#issuecomment-1228632579 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4780: URL: https://github.com/apache/hadoop/pull/4780#issuecomment-1228610031 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Commented] (HADOOP-18417) Upgrade Maven Surefire plugin to 3.0.0-M7

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585415#comment-17585415 ] ASF GitHub Bot commented on HADOOP-18417: - steveloughran commented on PR #4807:

[GitHub] [hadoop] steveloughran commented on pull request #4807: HADOOP-18417. Upgrade to M7 of surefire plugin

2022-08-26 Thread GitBox
steveloughran commented on PR #4807: URL: https://github.com/apache/hadoop/pull/4807#issuecomment-1228566883 lgtm; let's see what yetus says -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[jira] [Commented] (HADOOP-18408) [ABFS]: Set correct value of requireRenameResilience for NonHNS-SharedKey configuration

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585413#comment-17585413 ] ASF GitHub Bot commented on HADOOP-18408: - steveloughran commented on PR #4758:

[GitHub] [hadoop] steveloughran commented on pull request #4758: HADOOP-18408. ABFS: Setting correct value of requireRenameResilience for nonHNS configuration

2022-08-26 Thread GitBox
steveloughran commented on PR #4758: URL: https://github.com/apache/hadoop/pull/4758#issuecomment-1228565026 looks good. have you tested it with a hierachical fs to make sure all is good there? -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585410#comment-17585410 ] ASF GitHub Bot commented on HADOOP-18410: - hadoop-yetus commented on PR #4766:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4766: URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1228560368 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585408#comment-17585408 ] ASF GitHub Bot commented on HADOOP-18410: - hadoop-yetus commented on PR #4766:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4766: URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1228554053 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585405#comment-17585405 ] ASF GitHub Bot commented on HADOOP-18410: - hadoop-yetus commented on PR #4766:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4766: URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1228548408 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4810: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor.

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4810: URL: https://github.com/apache/hadoop/pull/4810#issuecomment-1228531678 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18425) [ABFS]: RenameFilePath Source File Not Found (404) error in retry loop

2022-08-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585392#comment-17585392 ] Steve Loughran commented on HADOOP-18425: - we've hit this in job commit at scal

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call

2022-08-26 Thread GitBox
ashutoshcipher commented on code in PR #4248: URL: https://github.com/apache/hadoop/pull/4248#discussion_r956070828 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/lib/TestMultipleOutputs.java: ## @@ -7

[GitHub] [hadoop] ashutoshcipher commented on a diff in pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call

2022-08-26 Thread GitBox
ashutoshcipher commented on code in PR #4248: URL: https://github.com/apache/hadoop/pull/4248#discussion_r956070096 ## hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/MultipleOutputs.java: ## @@ -527,9 +558

[jira] [Created] (HADOOP-18425) [ABFS]: RenameFilePath Source File Not Found (404) error in retry loop

2022-08-26 Thread Sree Bhattacharyya (Jira)
Sree Bhattacharyya created HADOOP-18425: --- Summary: [ABFS]: RenameFilePath Source File Not Found (404) error in retry loop Key: HADOOP-18425 URL: https://issues.apache.org/jira/browse/HADOOP-18425

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585343#comment-17585343 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on PR #4766:

[GitHub] [hadoop] steveloughran commented on pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on PR #4766: URL: https://github.com/apache/hadoop/pull/4766#issuecomment-1228433169 tested, s3 london -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585342#comment-17585342 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955995293 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/performance/ITestUnbufferDraining.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Softwa

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955993053 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/performance/ITestUnbufferDraining.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Softwa

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585340#comment-17585340 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4782: HDFS-16736. Link to Boost library in libhdfspp

2022-08-26 Thread GitBox
GauthamBanasandra commented on code in PR #4782: URL: https://github.com/apache/hadoop/pull/4782#discussion_r955971687 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/CMakeLists.txt: ## @@ -28,7 +28,7 @@ project (libhdfspp) cmake_minimum_required(VE

[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4782: HDFS-16736. Link to Boost library in libhdfspp

2022-08-26 Thread GitBox
GauthamBanasandra commented on code in PR #4782: URL: https://github.com/apache/hadoop/pull/4782#discussion_r955971687 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/CMakeLists.txt: ## @@ -28,7 +28,7 @@ project (libhdfspp) cmake_minimum_required(VE

[GitHub] [hadoop] slfan1989 opened a new pull request, #4810: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor.

2022-08-26 Thread GitBox
slfan1989 opened a new pull request, #4810: URL: https://github.com/apache/hadoop/pull/4810 JIRA: YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Commented] (HADOOP-18408) [ABFS]: Set correct value of requireRenameResilience for NonHNS-SharedKey configuration

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585310#comment-17585310 ] ASF GitHub Bot commented on HADOOP-18408: - sreeb-msft commented on code in PR #

[jira] [Commented] (HADOOP-18408) [ABFS]: Set correct value of requireRenameResilience for NonHNS-SharedKey configuration

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585311#comment-17585311 ] ASF GitHub Bot commented on HADOOP-18408: - sreeb-msft commented on code in PR #

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #4758: HADOOP-18408. ABFS: Setting correct value of requireRenameResilience for nonHNS configuration

2022-08-26 Thread GitBox
sreeb-msft commented on code in PR #4758: URL: https://github.com/apache/hadoop/pull/4758#discussion_r955941638 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/commit/ITestAbfsRenameStageFailure.java: ## @@ -59,7 +66,10 @@ protected AbstractFSContract cr

[GitHub] [hadoop] sreeb-msft commented on a diff in pull request #4758: HADOOP-18408. ABFS: Setting correct value of requireRenameResilience for nonHNS configuration

2022-08-26 Thread GitBox
sreeb-msft commented on code in PR #4758: URL: https://github.com/apache/hadoop/pull/4758#discussion_r955941443 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/commit/ITestAbfsRenameStageFailure.java: ## @@ -41,6 +42,12 @@ public ITestAbfsRenameStageFail

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585303#comment-17585303 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955917752 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/SDKStreamDrainer.java: ## @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software Foundatio

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585298#comment-17585298 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955909390 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/performance/ITestUnbufferDraining.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Softwa

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585295#comment-17585295 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955908427 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java: ## @@ -1345,6 +1238,10 @@ public synchronized void unbuffer() { closeS

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585290#comment-17585290 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955898795 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestSDKStreamDrainer.java: ## @@ -0,0 +1,342 @@ +/* + * Licensed to the Apache Software Found

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585289#comment-17585289 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585288#comment-17585288 ] ASF GitHub Bot commented on HADOOP-18410: - steveloughran commented on code in P

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955898631 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/SDKStreamDrainer.java: ## @@ -0,0 +1,325 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [hadoop] steveloughran commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
steveloughran commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955898158 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/impl/TestSDKStreamDrainer.java: ## @@ -0,0 +1,342 @@ +/* + * Licensed to the Apache Software Found

[jira] [Commented] (HADOOP-18416) ITestS3AIOStatisticsContext failure

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585286#comment-17585286 ] ASF GitHub Bot commented on HADOOP-18416: - steveloughran commented on PR #4806:

[GitHub] [hadoop] steveloughran commented on pull request #4806: HADOOP-18416. fix ITestS3AIOStatisticsContext test failure

2022-08-26 Thread GitBox
steveloughran commented on PR #4806: URL: https://github.com/apache/hadoop/pull/4806#issuecomment-1228321153 remove that setting from the bucket so it always runs. maybe add a test utils method to do this ``` Configuration disablePrefetching(Configuration c) { removeBaseAndBuc

[jira] [Commented] (HADOOP-18416) ITestS3AIOStatisticsContext failure

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585282#comment-17585282 ] ASF GitHub Bot commented on HADOOP-18416: - hadoop-yetus commented on PR #4806:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4806: HADOOP-18416. fix ITestS3AIOStatisticsContext test failure

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4806: URL: https://github.com/apache/hadoop/pull/4806#issuecomment-1228302129 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18424) Fix org.apache.hadoop.io.ArrayWritable

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585281#comment-17585281 ] ASF GitHub Bot commented on HADOOP-18424: - hadoop-yetus commented on PR #4809:

[GitHub] [hadoop] hadoop-yetus commented on pull request #4809: HADOOP-18424. Fix org.apache.hadoop.io.ArrayWritable

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4809: URL: https://github.com/apache/hadoop/pull/4809#issuecomment-1228301064 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #4560: HDFS-16659. JournalNode should throw NewerTxnIdException when SinceTxId is bigger than HighestWrittenTxId

2022-08-26 Thread GitBox
hadoop-yetus commented on PR #4560: URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1228293430 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[jira] [Commented] (HADOOP-18410) S3AInputStream.unbuffer() async drain not releasing http connections

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585274#comment-17585274 ] ASF GitHub Bot commented on HADOOP-18410: - mehakmeet commented on code in PR #4

[GitHub] [hadoop] mehakmeet commented on a diff in pull request #4766: HADOOP-18410. S3AInputStream.unbuffer() not releasing http connections

2022-08-26 Thread GitBox
mehakmeet commented on code in PR #4766: URL: https://github.com/apache/hadoop/pull/4766#discussion_r955790641 ## hadoop-tools/hadoop-aws/src/test/resources/core-site.xml: ## @@ -184,12 +184,17 @@ true + Review Comment: nit: extra blank line ## hadoop-t

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4598: HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager

2022-08-26 Thread GitBox
Hexiaoqiao commented on PR #4598: URL: https://github.com/apache/hadoop/pull/4598#issuecomment-1228269394 OK, Got it, Thanks. LGTM. +1 from me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] zhangshuyan0 commented on pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
zhangshuyan0 commented on PR #4780: URL: https://github.com/apache/hadoop/pull/4780#issuecomment-1228259955 > LGTM. > > Should we add `dfs.namenode.remove.bad.batch.num` to the xml configuration file? Would it be better to add some unit tests? Thanks @slfan1989 , I have added t

[GitHub] [hadoop] zhangshuyan0 commented on pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
zhangshuyan0 commented on PR #4780: URL: https://github.com/apache/hadoop/pull/4780#issuecomment-1228258366 > LGTM. > > Should we add `dfs.namenode.remove.bad.batch.num` to the xml configuration file? Would it be better to add some unit tests? Thanks @slfan1989 , I have added t

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
zhangshuyan0 commented on code in PR #4780: URL: https://github.com/apache/hadoop/pull/4780#discussion_r955836358 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java: ## @@ -96,6 +97,9 @@ class HeartbeatManager imp

[GitHub] [hadoop] cxzl25 commented on pull request #4598: HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager

2022-08-26 Thread GitBox
cxzl25 commented on PR #4598: URL: https://github.com/apache/hadoop/pull/4598#issuecomment-1228251220 > Sorry for the late response. But I did not get why System.nanoTime() may return a negative number. Do you mind to offer any jdk information? Thanks. > > > System.nanoTime()may retur

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #4780: HDFS-16735. Reduce the number of HeartbeatManager loops.

2022-08-26 Thread GitBox
zhangshuyan0 commented on code in PR #4780: URL: https://github.com/apache/hadoop/pull/4780#discussion_r955820370 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java: ## @@ -71,6 +71,7 @@ class HeartbeatManager imp

[GitHub] [hadoop] Hexiaoqiao commented on pull request #4598: HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager

2022-08-26 Thread GitBox
Hexiaoqiao commented on PR #4598: URL: https://github.com/apache/hadoop/pull/4598#issuecomment-1228234743 Sorry for the late response. But I did not get why System.nanoTime() may return a negative number. Do you mind to offer any jdk information? Thanks. > System.nanoTime()may return

[jira] [Commented] (HADOOP-18416) ITestS3AIOStatisticsContext failure

2022-08-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585227#comment-17585227 ] ASF GitHub Bot commented on HADOOP-18416: - mehakmeet commented on PR #4806: URL

[GitHub] [hadoop] mehakmeet commented on pull request #4806: HADOOP-18416. fix ITestS3AIOStatisticsContext test failure

2022-08-26 Thread GitBox
mehakmeet commented on PR #4806: URL: https://github.com/apache/hadoop/pull/4806#issuecomment-1228205573 Thoughts on how to handle Prefetch streams for this test(it doesn't have stream capability for IOContext), Asserting in all tests to break before checking statistics values, or skipping

  1   2   >