[jira] [Commented] (HADOOP-18813) Fix typo: form -> from

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744892#comment-17744892 ] ASF GitHub Bot commented on HADOOP-18813: - tomscut commented on PR #5861: URL:

[GitHub] [hadoop] tomscut commented on pull request #5861: HADOOP-18813. Fix typo: form -> from

2023-07-19 Thread via GitHub
tomscut commented on PR #5861: URL: https://github.com/apache/hadoop/pull/5861#issuecomment-1643286032 Maybe we should create the JIRA under `YARN`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] Tre2878 commented on pull request #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block rep

2023-07-19 Thread via GitHub
Tre2878 commented on PR #5856: URL: https://github.com/apache/hadoop/pull/5856#issuecomment-1643253493 @Hexiaoqiao Ok, I've committed in the original branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Commented] (HADOOP-18813) Fix typo: form -> from

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744879#comment-17744879 ] ASF GitHub Bot commented on HADOOP-18813: - yoonsch217 opened a new pull request, #5861: URL:

[jira] [Updated] (HADOOP-18813) Fix typo: form -> from

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18813: Labels: pull-request-available (was: ) > Fix typo: form -> from >

[GitHub] [hadoop] yoonsch217 opened a new pull request, #5861: HADOOP-18813. Fix typo: form -> from

2023-07-19 Thread via GitHub
yoonsch217 opened a new pull request, #5861: URL: https://github.com/apache/hadoop/pull/5861 ### Description of PR There is a typo in a log message that NodeManager prints: > Retrieved credentials form RM for {}: {} ### How was this patch tested? This is

[jira] [Created] (HADOOP-18813) Fix typo: form -> from

2023-07-19 Thread Seokchan Yoon (Jira)
Seokchan Yoon created HADOOP-18813: -- Summary: Fix typo: form -> from Key: HADOOP-18813 URL: https://issues.apache.org/jira/browse/HADOOP-18813 Project: Hadoop Common Issue Type: Improvement

[GitHub] [hadoop] slfan1989 closed pull request #5747: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-19 Thread via GitHub
slfan1989 closed pull request #5747: YARN-7707. BackPort [GPG] Policy generator framework. URL: https://github.com/apache/hadoop/pull/5747 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] slfan1989 commented on pull request #5747: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5747: URL: https://github.com/apache/hadoop/pull/5747#issuecomment-1643179394 @goiri Thank you very much for reviewing the code! I will follow up on #5810 and #5747 will be closed. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] brumi1024 merged pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-19 Thread via GitHub
brumi1024 merged PR #5784: URL: https://github.com/apache/hadoop/pull/5784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] brumi1024 merged pull request #5852: YARN-11533. CapacityScheduler CapacityConfigType changed in legacy queue allocation mode

2023-07-19 Thread via GitHub
brumi1024 merged PR #5852: URL: https://github.com/apache/hadoop/pull/5852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] brumi1024 commented on pull request #5852: YARN-11533. CapacityScheduler CapacityConfigType changed in legacy queue allocation mode

2023-07-19 Thread via GitHub
brumi1024 commented on PR #5852: URL: https://github.com/apache/hadoop/pull/5852#issuecomment-1643131100 Thanks for the reviews @tomicooler @sunilgovind! Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] tomscut commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
tomscut commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1643035843 > @tomscut This PR can cherry pick to branch-3.3 smoothly, Please cherry-pick directly if you evaluate it also need to fix for branch-3.3 rather than submit another PR. Thanks.

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1643028418 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1643006475 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1643001437 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] slfan1989 commented on pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5841: URL: https://github.com/apache/hadoop/pull/5841#issuecomment-1643001164 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744841#comment-17744841 ] ASF GitHub Bot commented on HADOOP-18805: - hadoop-yetus commented on PR #5851: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1643000810 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
Hexiaoqiao commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642997064 @tomscut This PR can cherry pick to branch-3.3 smoothly, Please cherry-pick directly if you evaluate it also need to fix for branch-3.3 rather than submit another PR. Thanks. --

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
zhangshuyan0 commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642992778 > @zhangshuyan0 Could you please backport this to branch-3.3? Thanks! Ok, I'll do this later. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] tomscut commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
tomscut commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642968772 @zhangshuyan0 Could you please backport this to branch-3.3? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] tomscut commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
tomscut commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642967557 Thanks @zhangshuyan0 for your contribution! Thanks @Hexiaoqiao for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] tomscut merged pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
tomscut merged PR #5854: URL: https://github.com/apache/hadoop/pull/5854 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744831#comment-17744831 ] ASF GitHub Bot commented on HADOOP-18073: - virajjasani commented on PR #5857: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-19 Thread via GitHub
virajjasani commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-1642940300 Nice, i will be happy to perform some testing when/if needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744830#comment-17744830 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on PR #5851: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5851: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-19 Thread via GitHub
virajjasani commented on PR #5851: URL: https://github.com/apache/hadoop/pull/5851#issuecomment-1642927647 `us-west-2`: ``` $ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch [INFO]

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744825#comment-17744825 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on code in PR #5851: URL:

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5851: HADOOP-18805. S3A prefetch LRU test to work with small files

2023-07-19 Thread via GitHub
virajjasani commented on code in PR #5851: URL: https://github.com/apache/hadoop/pull/5851#discussion_r1268796371 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingInputStream.java: ## @@ -94,17 +97,16 @@ public void teardown() throws

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744822#comment-17744822 ] ASF GitHub Bot commented on HADOOP-18805: - virajjasani commented on code in PR #5851: URL:

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5851: HADOOP-18805. S3A prefetch LRU test to work with small files

2023-07-19 Thread via GitHub
virajjasani commented on code in PR #5851: URL: https://github.com/apache/hadoop/pull/5851#discussion_r1268791996 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingInputStream.java: ## @@ -64,26 +63,30 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1642893412 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18207) Introduce hadoop-logging module

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744819#comment-17744819 ] ASF GitHub Bot commented on HADOOP-18207: - virajjasani commented on PR #5717: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5717: HADOOP-18207. Introduce hadoop-logging module

2023-07-19 Thread via GitHub
virajjasani commented on PR #5717: URL: https://github.com/apache/hadoop/pull/5717#issuecomment-1642891564 TestObserverNode failure is not relevant, verified the same -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1642886100 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642880725 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1642870547 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744799#comment-17744799 ] ASF GitHub Bot commented on HADOOP-18073: - hadoop-yetus commented on PR #5858: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5858: URL: https://github.com/apache/hadoop/pull/5858#issuecomment-1642806923 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744798#comment-17744798 ] ASF GitHub Bot commented on HADOOP-18073: - hadoop-yetus commented on PR #5857: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-1642800789 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1642792350 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744779#comment-17744779 ] ASF GitHub Bot commented on HADOOP-18752: - hadoop-yetus commented on PR #5859: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5859: URL: https://github.com/apache/hadoop/pull/5859#issuecomment-1642724285 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744777#comment-17744777 ] ASF GitHub Bot commented on HADOOP-18752: - hadoop-yetus commented on PR #5859: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5859: URL: https://github.com/apache/hadoop/pull/5859#issuecomment-1642694187 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete bloc

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5856: URL: https://github.com/apache/hadoop/pull/5856#issuecomment-1642638647 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri merged pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-19 Thread via GitHub
goiri merged PR #5816: URL: https://github.com/apache/hadoop/pull/5816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] goiri merged pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-19 Thread via GitHub
goiri merged PR #5841: URL: https://github.com/apache/hadoop/pull/5841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] goiri merged pull request #5804: HDFS-17042 Add rpcCallSuccesses and OverallRpcProcessingTime to RpcMetrics for Namenode (#5730)

2023-07-19 Thread via GitHub
goiri merged PR #5804: URL: https://github.com/apache/hadoop/pull/5804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] simbadzina opened a new pull request, #5860: HDFS-17111. RBF: Optimize msync to only call nameservices with observer namenodes.

2023-07-19 Thread via GitHub
simbadzina opened a new pull request, #5860: URL: https://github.com/apache/hadoop/pull/5860 HDFS-17111. RBF: Optimize msync to only call nameservices with observer namenodes. ### Description of PR Routers only need to msync to nameservices that have CRS configured.

[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #1747: HDFS-15042 Add more tests for ByteBufferPositionedReadable.

2023-07-19 Thread via GitHub
mukund-thakur commented on code in PR #1747: URL: https://github.com/apache/hadoop/pull/1747#discussion_r956411939 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1684,6 +1685,9 @@ public int read(long position, final

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744739#comment-17744739 ] ASF GitHub Bot commented on HADOOP-18752: - steveloughran commented on PR #5859: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-19 Thread via GitHub
steveloughran commented on PR #5859: URL: https://github.com/apache/hadoop/pull/5859#issuecomment-1642515852 testing: s3 london. this is a backport and as it doesn't include the contentious issue "actually changing the switch" then I'm happy to cherrypick as is. lets see what the tests say

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744737#comment-17744737 ] ASF GitHub Bot commented on HADOOP-18073: - mukund-thakur commented on PR #5858: URL:

[GitHub] [hadoop] mukund-thakur commented on pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-19 Thread via GitHub
mukund-thakur commented on PR #5858: URL: https://github.com/apache/hadoop/pull/5858#issuecomment-1642514483 Shouldn't there be one for trunk first? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744736#comment-17744736 ] ASF GitHub Bot commented on HADOOP-18752: - steveloughran commented on PR #5689: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5689: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread via GitHub
steveloughran commented on PR #5689: URL: https://github.com/apache/hadoop/pull/5689#issuecomment-1642513305 Update: I have a pr of this for branch-3.3 which does everything but changing the default/documenting this change #5859 this is so those releases stop warning about

[jira] [Commented] (HADOOP-18752) Change fs.s3a.directory.marker.retention to "keep"

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744735#comment-17744735 ] ASF GitHub Bot commented on HADOOP-18752: - steveloughran opened a new pull request, #5859: URL:

[GitHub] [hadoop] steveloughran opened a new pull request, #5859: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep" -everything but the switch

2023-07-19 Thread via GitHub
steveloughran opened a new pull request, #5859: URL: https://github.com/apache/hadoop/pull/5859 This change has all of PR #5689 *except* for changing the default value of marker retention from keep to delete. 1. leaves the default value of fs.s3a.directory.marker.retention at

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744729#comment-17744729 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran opened a new pull request, #5858: URL:

[GitHub] [hadoop] steveloughran opened a new pull request, #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-19 Thread via GitHub
steveloughran opened a new pull request, #5858: URL: https://github.com/apache/hadoop/pull/5858 ### Description of PR rebase of feature-HADOOP-18073-s3a-sdk-upgrade-3.3-rebase onto branch-3.3 ### How was this patch tested? ### For code changes: - [ ]

[jira] [Created] (HADOOP-18812) list aws SDK v2 libraries in LICENSE.binary

2023-07-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-18812: --- Summary: list aws SDK v2 libraries in LICENSE.binary Key: HADOOP-18812 URL: https://issues.apache.org/jira/browse/HADOOP-18812 Project: Hadoop Common

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744724#comment-17744724 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran commented on PR #5857: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-19 Thread via GitHub
steveloughran commented on PR #5857: URL: https://github.com/apache/hadoop/pull/5857#issuecomment-1642466300 just noticed LICENSE.binary is out of date; added as a new item -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744723#comment-17744723 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran opened a new pull request, #5857: URL:

[GitHub] [hadoop] steveloughran opened a new pull request, #5857: HADOOP-18073. Upgrade AWS SDK to v2 in S3A [work in progress]

2023-07-19 Thread via GitHub
steveloughran opened a new pull request, #5857: URL: https://github.com/apache/hadoop/pull/5857 ### Description of PR this is the feature branch feature-HADOOP-18073-s3a-sdk-upgrade rebased onto trunk commit #b6b259066fd ### How was this patch tested? ### For code

[jira] [Commented] (HADOOP-18673) AWS SDK V2 - Refactor getS3Region & other follow up items

2023-07-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744716#comment-17744716 ] Steve Loughran commented on HADOOP-18673: - what's up with getXattr? > AWS SDK V2 - Refactor

[jira] [Commented] (HADOOP-18758) build/release a shaded version of the AWS SDK

2023-07-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744714#comment-17744714 ] Steve Loughran commented on HADOOP-18758: - aws do provide a bundle.jar file, it is just called

[GitHub] [hadoop] hadoop-yetus commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1642442293 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18811) Buggy ZKFCRpcServer constructor creates null object and crashes the rpcServer

2023-07-19 Thread ConfX (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HADOOP-18811: --- Attachment: reproduce.sh > Buggy ZKFCRpcServer constructor creates null object and crashes the rpcServer >

[jira] [Updated] (HADOOP-18811) Buggy ZKFCRpcServer constructor creates null object and crashes the rpcServer

2023-07-19 Thread ConfX (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ConfX updated HADOOP-18811: --- Description: h2. What happened: In ZKFailoverController.java, initRPC() function gets ZKFC RpcServer

[jira] [Created] (HADOOP-18811) Buggy ZKFCRpcServer constructor creates null object and crashes the rpcServer

2023-07-19 Thread ConfX (Jira)
ConfX created HADOOP-18811: -- Summary: Buggy ZKFCRpcServer constructor creates null object and crashes the rpcServer Key: HADOOP-18811 URL: https://issues.apache.org/jira/browse/HADOOP-18811 Project: Hadoop

[GitHub] [hadoop] steveloughran commented on a diff in pull request #1747: HDFS-15042 Add more tests for ByteBufferPositionedReadable.

2023-07-19 Thread via GitHub
steveloughran commented on code in PR #1747: URL: https://github.com/apache/hadoop/pull/1747#discussion_r1268335189 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestByteBufferPread.java: ## @@ -161,130 +229,264 @@ private void

[GitHub] [hadoop] steveloughran commented on a diff in pull request #1747: HDFS-15042 Add more tests for ByteBufferPositionedReadable.

2023-07-19 Thread via GitHub
steveloughran commented on code in PR #1747: URL: https://github.com/apache/hadoop/pull/1747#discussion_r1268334625 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java: ## @@ -1684,6 +1685,9 @@ public int read(long position, final

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread via GitHub
steveloughran commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1268331301 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java: ## @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744706#comment-17744706 ] ASF GitHub Bot commented on HADOOP-18807: - steveloughran commented on code in PR #5847: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete bloc

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5855: URL: https://github.com/apache/hadoop/pull/5855#issuecomment-1642377590 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-19 Thread via GitHub
hadoop-yetus commented on PR #5784: URL: https://github.com/apache/hadoop/pull/5784#issuecomment-1642271707 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-17802) S3A Xattr/getXAttr to handle directories without markers

2023-07-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744653#comment-17744653 ] Steve Loughran commented on HADOOP-17802: - revisting this. think we should fall back to an

[GitHub] [hadoop] slfan1989 commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1642225287 @goiri Can you help to merge this pr merge into the trunk branch? Thank you very much! I will continue to follow up the batch save federation queue policies. -- This is an automated

[GitHub] [hadoop] slfan1989 commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-164629 @goiri Can you help to review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] slfan1989 commented on pull request #5841: YARN-11531. [Federation] Code cleanup for NodeManager#amrmproxy.

2023-07-19 Thread via GitHub
slfan1989 commented on PR #5841: URL: https://github.com/apache/hadoop/pull/5841#issuecomment-1642221435 @goiri Can you help to review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744626#comment-17744626 ] ASF GitHub Bot commented on HADOOP-18807: - Hexiaoqiao commented on code in PR #5847: URL:

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread via GitHub
Hexiaoqiao commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1268071401 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java: ## @@ -0,0 +1,64 @@ +/** + * Licensed to the Apache

[GitHub] [hadoop] Hexiaoqiao closed pull request #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block report

2023-07-19 Thread via GitHub
Hexiaoqiao closed pull request #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting URL: https://github.com/apache/hadoop/pull/5856 -- This is an automated message from the Apache Git

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block

2023-07-19 Thread via GitHub
Hexiaoqiao commented on PR #5856: URL: https://github.com/apache/hadoop/pull/5856#issuecomment-1642036068 Please push update to your same original branch (for this case which is Tre2878:HDFS-17093), DO NOT pull repeat request for same issue. -- This is an automated message from the

[GitHub] [hadoop] Tre2878 commented on a diff in pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete

2023-07-19 Thread via GitHub
Tre2878 commented on code in PR #5855: URL: https://github.com/apache/hadoop/pull/5855#discussion_r1268012644 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -2873,7 +2873,9 @@ public boolean

[GitHub] [hadoop] Tre2878 opened a new pull request, #5856: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block re

2023-07-19 Thread via GitHub
Tre2878 opened a new pull request, #5856: URL: https://github.com/apache/hadoop/pull/5856 In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block reporting -- This is an automated message from the Apache Git Service.

[jira] [Resolved] (HADOOP-18757) S3A Committer only finalizes the commits in a single thread

2023-07-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18757. - Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed > S3A Committer

[jira] [Assigned] (HADOOP-18757) S3A Committer only finalizes the commits in a single thread

2023-07-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18757: --- Assignee: Moditha Hewasinghage > S3A Committer only finalizes the commits in a

[jira] [Commented] (HADOOP-18757) S3A Committer only finalizes the commits in a single thread

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744561#comment-17744561 ] ASF GitHub Bot commented on HADOOP-18757: - steveloughran commented on PR #5706: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5706: HADOOP-18757: Bump corePoolSize of HadoopThreadPoolExecutor in s3a committer

2023-07-19 Thread via GitHub
steveloughran commented on PR #5706: URL: https://github.com/apache/hadoop/pull/5706#issuecomment-1641917047 merged to trunk; cherrypicked to branch-3.3 and ran the scale tests against s3 london; all good. thanks! -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] K0K0V0K commented on a diff in pull request #5784: YARN-9877 - Intermittent TIME_OUT of LogAggregationReport

2023-07-19 Thread via GitHub
K0K0V0K commented on code in PR #5784: URL: https://github.com/apache/hadoop/pull/5784#discussion_r1267902588 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java:

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incompl

2023-07-19 Thread via GitHub
Hexiaoqiao commented on code in PR #5855: URL: https://github.com/apache/hadoop/pull/5855#discussion_r1267889826 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java: ## @@ -2873,7 +2873,9 @@ public boolean

[GitHub] [hadoop] zhangshuyan0 commented on pull request #5854: HDFS-17094. EC: Fix bug in block recovery when there are stale datanodes.

2023-07-19 Thread via GitHub
zhangshuyan0 commented on PR #5854: URL: https://github.com/apache/hadoop/pull/5854#issuecomment-1641844035 @Hexiaoqiao @tomscut Thanks for your review. I've update this PR according to the suggestions. Please take a look, thanks again. -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744533#comment-17744533 ] ASF GitHub Bot commented on HADOOP-18807: - zhangshuyan0 commented on code in PR #5847: URL:

[GitHub] [hadoop] zhangshuyan0 commented on a diff in pull request #5847: HADOOP-18807. Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread via GitHub
zhangshuyan0 commented on code in PR #5847: URL: https://github.com/apache/hadoop/pull/5847#discussion_r1267885350 ## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFileSystemClose.java: ## @@ -17,26 +17,26 @@ */ package

[jira] [Commented] (HADOOP-18807) Close child file systems in ViewFileSystem when cache is disabled.

2023-07-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744532#comment-17744532 ] ASF GitHub Bot commented on HADOOP-18807: - zhangshuyan0 commented on code in PR #5847: URL:

  1   2   >