[GitHub] [hadoop] hadoop-yetus commented on pull request #5890: YARN-11538. CS UI: queue filter do not work as expected when submitti…

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5890: URL: https://github.com/apache/hadoop/pull/5890#issuecomment-1652954806 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Assigned] (HADOOP-18826) abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400

2023-07-26 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi reassigned HADOOP-18826: -- Assignee: Anuj Modi > abfs getFileStatus(/) fails with "Value for one of the query

[jira] [Updated] (HADOOP-18829) s3a prefetch LRU cache eviction metric

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18829: Labels: pull-request-available (was: ) > s3a prefetch LRU cache eviction metric >

[jira] [Commented] (HADOOP-18829) s3a prefetch LRU cache eviction metric

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747759#comment-17747759 ] ASF GitHub Bot commented on HADOOP-18829: - virajjasani opened a new pull request, #5893: URL:

[GitHub] [hadoop] virajjasani opened a new pull request, #5893: HADOOP-18829. S3A prefetch LRU cache eviction metric

2023-07-26 Thread via GitHub
virajjasani opened a new pull request, #5893: URL: https://github.com/apache/hadoop/pull/5893 Jira: HADOOP-18829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Created] (HADOOP-18829) s3a prefetch LRU cache eviction metric

2023-07-26 Thread Viraj Jasani (Jira)
Viraj Jasani created HADOOP-18829: - Summary: s3a prefetch LRU cache eviction metric Key: HADOOP-18829 URL: https://issues.apache.org/jira/browse/HADOOP-18829 Project: Hadoop Common Issue

[GitHub] [hadoop] zhiubok commented on pull request #5879: HDFS-17130. Blocks on IN_MAINTENANCE DNs should be sorted properly in LocatedBlocks.

2023-07-26 Thread via GitHub
zhiubok commented on PR #5879: URL: https://github.com/apache/hadoop/pull/5879#issuecomment-1652923993 Thanks @Hexiaoqiao @ayushtkn, I have applied for a Jira account and modified the commit information for this PR. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [hadoop] virajjasani commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
virajjasani commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652921250 > Send FBR asynchronously. (Not include IBR, because we didn't meet some performance issue here). got it, thank you @Hexiaoqiao! -- This is an automated message from the

[GitHub] [hadoop] haiyang1987 commented on pull request #5877: HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread via GitHub
haiyang1987 commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1652912054 Thanks Sir @Hexiaoqiao @tomscut help me review and merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652883289 > If true, is it enough to make IBR and FBR asynchronously together and use only one thread? Variable fullBlockReportLeaseId updates from heartbeat and used by fbr and then update to

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5877: HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1652857021 Committed to trunk. Thanks @haiyang1987 for your contribution and @tomscut review. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hadoop] Hexiaoqiao merged pull request #5877: HDFS-17117. Print reconstructionQueuesInitProgress periodically when BlockManager processMisReplicatesAsync.

2023-07-26 Thread via GitHub
Hexiaoqiao merged PR #5877: URL: https://github.com/apache/hadoop/pull/5877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18393) Hadoop 3.3.2 has CVEs coming from dependencies

2023-07-26 Thread Xiping Zhang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747729#comment-17747729 ] Xiping Zhang commented on HADOOP-18393: --- [~ste...@apache.org]  Ok, thank you very much. > Hadoop

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652852914 About this PR, I didn't get the original purpose (include [HDFS-16016](https://issues.apache.org/jira/browse/HDFS-16016)). IIUC, we want to keep lightweight channel to send heartbeat

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652843223 > just one question: do you mean you already have IBR and FBR being processed asynchronously? or you mean you have [HDFS-16016](https://issues.apache.org/jira/browse/HDFS-16016)

[GitHub] [hadoop] virajjasani commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
virajjasani commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652822084 @LiuGuH sorry i got bit confused, if we want to solve the ordering problem by using the same lock for both IBR and FBR, do we really need HDFS-17129 as separate Jira? -- This is an

[GitHub] [hadoop] LiuGuH commented on pull request #5892: HDFS-17129. mis-order of ibr and fbr on datanode

2023-07-26 Thread via GitHub
LiuGuH commented on PR #5892: URL: https://github.com/apache/hadoop/pull/5892#issuecomment-1652817803 @Hexiaoqiao , Please review of this, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] virajjasani commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
virajjasani commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652816966 > actually we have deploy this feature for years, however we should be safe to land it. @Hexiaoqiao just one question: do you mean you already have IBR and FBR being processed

[GitHub] [hadoop] LiuGuH opened a new pull request, #5892: HDFS-17129. mis-order of ibr and fbr on datanode

2023-07-26 Thread via GitHub
LiuGuH opened a new pull request, #5892: URL: https://github.com/apache/hadoop/pull/5892 As described of HDFS-17129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1652806518 @Hexiaoqiao , well, I user original sendIBRLock to make right order for ibr and fbr. The FBR will execute ibr first, so the sendIBRLock will be successful for it. Please review for it,

[GitHub] [hadoop] Tre2878 commented on pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block rep

2023-07-26 Thread via GitHub
Tre2878 commented on PR #5855: URL: https://github.com/apache/hadoop/pull/5855#issuecomment-1652792009 @zhangshuyan0 We rewrote the unit test to see if we explained the bug -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] Tre2878 commented on pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete block rep

2023-07-26 Thread via GitHub
Tre2878 commented on PR #5855: URL: https://github.com/apache/hadoop/pull/5855#issuecomment-1652782384 Don't know why the hadoop. HDFS. TestFileChecksum unit test failed, I didn't modify the code -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] zhiubok commented on pull request #5879: Blocks on IN_MAINTENANCE DNs should be sorted properly in LocatedBlocks.

2023-07-26 Thread via GitHub
zhiubok commented on PR #5879: URL: https://github.com/apache/hadoop/pull/5879#issuecomment-1652776058 > It isn’t letting me approve either: ![62B3DA92-358D-45EB-B829-D1C6E49639B5](https://user-images.githubusercontent.com/25608848/256363450-320f738a-2b1d-4543-b99c-d183935f1c9d.jpeg) >

[GitHub] [hadoop] yangjiandan commented on pull request #5890: YARN-11538. CS UI: queue filter do not work as expected when submitti…

2023-07-26 Thread via GitHub
yangjiandan commented on PR #5890: URL: https://github.com/apache/hadoop/pull/5890#issuecomment-1652751177 failed ut is not related on this change. @slfan1989 can you trigger ut again? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] hadoop-yetus commented on pull request #5878: HDFS-17030 Limit wait time for getHAServiceState in ObserverReadProxyProvider

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5878: URL: https://github.com/apache/hadoop/pull/5878#issuecomment-1652674935 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5891: HDFS-17123. Sort datanodeStorages when generating StorageBlockReport[] in method BPServiceActor#blockReport for future convenience.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5891: URL: https://github.com/apache/hadoop/pull/5891#issuecomment-1652626812 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18207) Introduce hadoop-logging module

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747673#comment-17747673 ] ASF GitHub Bot commented on HADOOP-18207: - jojochuang commented on PR #5717: URL:

[GitHub] [hadoop] jojochuang commented on pull request #5717: HADOOP-18207. Introduce hadoop-logging module

2023-07-26 Thread via GitHub
jojochuang commented on PR #5717: URL: https://github.com/apache/hadoop/pull/5717#issuecomment-1652568988 LGTM. I'm just going to trigger the test again. If it comes back fine we can proceed with the merge. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] ayushtkn commented on pull request #5879: Blocks on IN_MAINTENANCE DNs should be sorted properly in LocatedBlocks.

2023-07-26 Thread via GitHub
ayushtkn commented on PR #5879: URL: https://github.com/apache/hadoop/pull/5879#issuecomment-1652493272 It isn’t letting me approve either: ![62B3DA92-358D-45EB-B829-D1C6E49639B5](https://github.com/apache/hadoop/assets/25608848/320f738a-2b1d-4543-b99c-d183935f1c9d) Submit a

[GitHub] [hadoop] hadoop-yetus commented on pull request #5871: YARN-11520. Support capacity vector for AQCv2 dynamic templates.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5871: URL: https://github.com/apache/hadoop/pull/5871#issuecomment-1652479553 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747641#comment-17747641 ] ASF GitHub Bot commented on HADOOP-18820: - hadoop-yetus commented on PR #5872: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5872: URL: https://github.com/apache/hadoop/pull/5872#issuecomment-1652352720 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5862: YARN-11536. [Federation] Router CLI Supports Batch Save the SubClusterPolicyConfiguration Of Queues.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5862: URL: https://github.com/apache/hadoop/pull/5862#issuecomment-1652335571 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] xinglin commented on pull request #5878: HDFS-17030 Limit wait time for getHAServiceState in ObserverReadProxyProvider

2023-07-26 Thread via GitHub
xinglin commented on PR #5878: URL: https://github.com/apache/hadoop/pull/5878#issuecomment-1652294268 @goiri, @Hexiaoqiao, do you know why only a single yetus build was triggered for multiple empty commits? Each empty commit would trigger a build in the past. 4 days have been passed

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747618#comment-17747618 ] ASF GitHub Bot commented on HADOOP-18820: - hadoop-yetus commented on PR #5872: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5872: URL: https://github.com/apache/hadoop/pull/5872#issuecomment-1652276683 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5877: HDFS-17117. BlockManager#processMisReplicatesAsync can periodically print reconstructionQueuesInitProgress

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1652265193 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5877: HDFS-17117. BlockManager#processMisReplicatesAsync can periodically print reconstructionQueuesInitProgress

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1652264541 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747608#comment-17747608 ] ASF GitHub Bot commented on HADOOP-18820: - steveloughran commented on PR #5872: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-07-26 Thread via GitHub
steveloughran commented on PR #5872: URL: https://github.com/apache/hadoop/pull/5872#issuecomment-1652219827 hi, needs a bit more review/upvote. with this patch, we still support v1 providers if people set up the classpath, but we map the standard ones to v2 equivalents. -- This is an

[GitHub] [hadoop] hadoop-yetus commented on pull request #5855: HDFS-17093. In the case of all datanodes sending FBR when the namenode restarts (large clusters), there is an issue with incomplete bloc

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5855: URL: https://github.com/apache/hadoop/pull/5855#issuecomment-1652201535 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] umamaheswararao commented on pull request #5885: HDFS-17120. Support snapshot diff based copylisting for flat paths.

2023-07-26 Thread via GitHub
umamaheswararao commented on PR #5885: URL: https://github.com/apache/hadoop/pull/5885#issuecomment-1652194090 Latest changes looks good to me. Thanks @sadanand48 for working on this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Commented] (HADOOP-18342) Upgrade to Avro 1.11.1

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747600#comment-17747600 ] Steve Loughran commented on HADOOP-18342: - [~brahmareddy] time for a new release of that? >

[jira] [Commented] (HADOOP-18778) Test failures with CSE enabled

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747596#comment-17747596 ] ASF GitHub Bot commented on HADOOP-18778: - mukund-thakur commented on PR #5763: URL:

[GitHub] [hadoop] mukund-thakur commented on pull request #5763: HADOOP-18778. Fixes failing tests when CSE is enabled.

2023-07-26 Thread via GitHub
mukund-thakur commented on PR #5763: URL: https://github.com/apache/hadoop/pull/5763#issuecomment-1652159854 Yet to be done for branch-3.3. Waiting for Steve's V2 Sdk merge. I don't see the latest commits after pulling. -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747595#comment-17747595 ] ASF GitHub Bot commented on HADOOP-18820: - steveloughran commented on PR #5872: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-07-26 Thread via GitHub
steveloughran commented on PR #5872: URL: https://github.com/apache/hadoop/pull/5872#issuecomment-1652156127 current dependency setup ``` [INFO] +- com.amazonaws:aws-java-sdk-core:jar:1.12.367:provided [INFO] | +- commons-logging:commons-logging:jar:1.1.3:provided [INFO] | +-

[jira] [Commented] (HADOOP-18742) AWS v2 SDK: stabilise dependencies with rest of hadoop libraries

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747592#comment-17747592 ] ASF GitHub Bot commented on HADOOP-18742: - steveloughran closed pull request #5739:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5885: HDFS-17120. Support snapshot diff based copylisting for flat paths.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5885: URL: https://github.com/apache/hadoop/pull/5885#issuecomment-1652149377 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] steveloughran closed pull request #5739: HADOOP-18742. AWS v2 SDK: stabilise dependencies with rest of hadoop libraries

2023-07-26 Thread via GitHub
steveloughran closed pull request #5739: HADOOP-18742. AWS v2 SDK: stabilise dependencies with rest of hadoop libraries URL: https://github.com/apache/hadoop/pull/5739 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (HADOOP-18742) AWS v2 SDK: stabilise dependencies with rest of hadoop libraries

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747590#comment-17747590 ] ASF GitHub Bot commented on HADOOP-18742: - steveloughran commented on PR #5739: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5739: HADOOP-18742. AWS v2 SDK: stabilise dependencies with rest of hadoop libraries

2023-07-26 Thread via GitHub
steveloughran commented on PR #5739: URL: https://github.com/apache/hadoop/pull/5739#issuecomment-1652143052 pulling in to #5872 which downgrades the core lib to provided; we gain consistency by excluding unshaded stuff -- This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-18778) Test failures with CSE enabled

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747589#comment-17747589 ] ASF GitHub Bot commented on HADOOP-18778: - mukund-thakur merged PR #5763: URL:

[GitHub] [hadoop] mukund-thakur merged pull request #5763: HADOOP-18778. Fixes failing tests when CSE is enabled.

2023-07-26 Thread via GitHub
mukund-thakur merged PR #5763: URL: https://github.com/apache/hadoop/pull/5763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] simbadzina commented on pull request #5860: HDFS-17111. RBF: Optimize msync to only call nameservices that have observer reads enabled.

2023-07-26 Thread via GitHub
simbadzina commented on PR #5860: URL: https://github.com/apache/hadoop/pull/5860#issuecomment-1652123930 Thanks @ayushtkn. Just noticed the commit message is missing `Reviewed-by` and `Signed-off-by`. Probably too late to fix now though. Thank you all for the reviews and merging. --

[GitHub] [hadoop] hadoop-yetus commented on pull request #5877: HDFS-17117. BlockManager#processMisReplicatesAsync can periodically print reconstructionQueuesInitProgress

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1652074130 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18820) AWS SDK v2: make the v1 bridging support optional

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747577#comment-17747577 ] ASF GitHub Bot commented on HADOOP-18820: - hadoop-yetus commented on PR #5872: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5872: HADOOP-18820. Cut AWS v1 support

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5872: URL: https://github.com/apache/hadoop/pull/5872#issuecomment-1652069129 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5891: HDFS-17123. Sort datanodeStorages when generating StorageBlockReport[] in method BPServiceActor#blockReport for future convenience.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5891: URL: https://github.com/apache/hadoop/pull/5891#issuecomment-1652023344 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-26 Thread via GitHub
slfan1989 commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-1651988217 @goiri Can you help to merge this pr into the trunk branch? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] sadanand48 commented on a diff in pull request #5885: HDFS-17120. Support snapshot diff based copylisting for flat paths.

2023-07-26 Thread via GitHub
sadanand48 commented on code in PR #5885: URL: https://github.com/apache/hadoop/pull/5885#discussion_r1275096942 ## hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java: ## @@ -167,6 +169,77 @@ public void testDuplicates() { } } +

[jira] [Commented] (HADOOP-18747) AWS SDK V2 - sigv2 support

2023-07-26 Thread Ahmar Suhail (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747555#comment-17747555 ] Ahmar Suhail commented on HADOOP-18747: --- * Yeah, the NPE isn't ideal, we could update with

[GitHub] [hadoop] hadoop-yetus commented on pull request #5890: YARN-11538. CS UI: queue filter do not work as expected when submitti…

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5890: URL: https://github.com/apache/hadoop/pull/5890#issuecomment-1651940447 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18747) AWS SDK V2 - sigv2 support

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747543#comment-17747543 ] Steve Loughran commented on HADOOP-18747: - also looking at the signer story in general *

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747542#comment-17747542 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran closed pull request #5858:

[jira] [Commented] (HADOOP-18073) Upgrade AWS SDK to v2

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747541#comment-17747541 ] ASF GitHub Bot commented on HADOOP-18073: - steveloughran commented on PR #5858: URL:

[GitHub] [hadoop] steveloughran commented on pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-26 Thread via GitHub
steveloughran commented on PR #5858: URL: https://github.com/apache/hadoop/pull/5858#issuecomment-1651886669 ok, done the forced push of this; closing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hadoop] steveloughran closed pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707)

2023-07-26 Thread via GitHub
steveloughran closed pull request #5858: HADOOP-18073. Upgrade S3A in 3.3 branch to AWS SDK V2.(#5707) URL: https://github.com/apache/hadoop/pull/5858 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] zhiubok commented on pull request #5879: Blocks on IN_MAINTENANCE DNs should be sorted properly in LocatedBlocks.

2023-07-26 Thread via GitHub
zhiubok commented on PR #5879: URL: https://github.com/apache/hadoop/pull/5879#issuecomment-1651882967 > > Public signup for this instance is disabled, After submitting the application, it was rejected, so that I did not apply for a jira account, So much so that it's impossible

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747535#comment-17747535 ] ASF GitHub Bot commented on HADOOP-18805: - steveloughran closed pull request #5887:

[jira] [Commented] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747534#comment-17747534 ] ASF GitHub Bot commented on HADOOP-18805: - steveloughran commented on PR #5887: URL:

[GitHub] [hadoop] steveloughran closed pull request #5887: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-26 Thread via GitHub
steveloughran closed pull request #5887: HADOOP-18805. S3A prefetch tests to work with small files URL: https://github.com/apache/hadoop/pull/5887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] steveloughran commented on pull request #5887: HADOOP-18805. S3A prefetch tests to work with small files

2023-07-26 Thread via GitHub
steveloughran commented on PR #5887: URL: https://github.com/apache/hadoop/pull/5887#issuecomment-1651870211 i did the merge by hand after cherrypicking prerequisites; closing this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] steveloughran commented on a diff in pull request #5885: HDFS-17120. Support snapshot diff based copylisting for flat paths.

2023-07-26 Thread via GitHub
steveloughran commented on code in PR #5885: URL: https://github.com/apache/hadoop/pull/5885#discussion_r1274999246 ## hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestCopyListing.java: ## @@ -167,6 +169,77 @@ public void testDuplicates() { } } +

[GitHub] [hadoop] hadoop-yetus commented on pull request #5862: YARN-11536. [Federation] Router CLI Supports Batch Save the SubClusterPolicyConfiguration Of Queues.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5862: URL: https://github.com/apache/hadoop/pull/5862#issuecomment-1651852626 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18393) Hadoop 3.3.2 has CVEs coming from dependencies

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747519#comment-17747519 ] Steve Loughran commented on HADOOP-18393: - no jira's for those, they are in the category of

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5879: Blocks on IN_MAINTENANCE DNs should be sorted properly in LocatedBlocks.

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5879: URL: https://github.com/apache/hadoop/pull/5879#issuecomment-1651806823 > Public signup for this instance is disabled, After submitting the application, it was rejected, so that I did not apply for a jira account, So much so that it's impossible to create

[GitHub] [hadoop] LiuGuH commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
LiuGuH commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651805316 Yes,I know it. It is a problem with mis-order. a. heartbeat got a delete cmd on replication named r b. generate fbr c. datanode delete repliction r and send ibr d. send fbr

[jira] [Commented] (HADOOP-18826) abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747512#comment-17747512 ] Steve Loughran commented on HADOOP-18826: - but looking into getRelativePath() is a likely

[jira] [Commented] (HADOOP-18826) abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747509#comment-17747509 ] Steve Loughran commented on HADOOP-18826: - based on your analysis, HADOOP-16612 is the cause >

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #5876: HDFS-17116. Reset startupTime and enterSafeModeTime if check time interval is negative during router safe mode exit check

2023-07-26 Thread via GitHub
Hexiaoqiao commented on code in PR #5876: URL: https://github.com/apache/hadoop/pull/5876#discussion_r1274951221 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterSafemodeService.java: ## @@ -161,11 +161,17 @@ protected

[jira] [Updated] (HADOOP-18826) abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18826: Component/s: fs/azure (was: hadoop-thirdparty) > abfs

[jira] [Updated] (HADOOP-18826) abfs getFileStatus(/) fails with "Value for one of the query parameters specified in the request URI is invalid.", 400

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18826: Summary: abfs getFileStatus(/) fails with "Value for one of the query parameters

[jira] [Updated] (HADOOP-16169) ABFS: Bug fix for getPathProperties

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16169: Fix Version/s: 3.4.0 3.2.1 > ABFS: Bug fix for getPathProperties >

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5877: HDFS-17117. BlockManager#processMisReplicatesAsync can periodically print reconstructionQueuesInitProgress

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5877: URL: https://github.com/apache/hadoop/pull/5877#issuecomment-1651792028 +1 from my side. Let's wait what will Yetus say. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (HADOOP-16169) ABFS: Bug fix for getPathProperties

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16169: Fix Version/s: 3.3.1 (was: 3.2.1) > ABFS: Bug fix for

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5888: HDFS-17121. BPServiceActor to provide new thread to handle FBR

2023-07-26 Thread via GitHub
Hexiaoqiao commented on PR #5888: URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1651784829 > And maybe it not a problem. I am sure IT IS (As @tomscut mentioned above.). IMO, if there is issue that confirmed, the proper way is fix it first then improve it. I believe

[jira] [Commented] (HADOOP-18828) Hadoop UGI doesn't work with KCM based kerberos tickets

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747498#comment-17747498 ] Steve Loughran commented on HADOOP-18828: - can you tag with version affected and components,

[GitHub] [hadoop] hadoop-yetus commented on pull request #5871: YARN-11520. Support capacity vector for AQCv2 dynamic templates.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5871: URL: https://github.com/apache/hadoop/pull/5871#issuecomment-1651736152 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18778) Test failures with CSE enabled

2023-07-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747490#comment-17747490 ] ASF GitHub Bot commented on HADOOP-18778: - ahmarsuhail commented on PR #5763: URL:

[GitHub] [hadoop] ahmarsuhail commented on pull request #5763: HADOOP-18778. Fixes failing tests when CSE is enabled.

2023-07-26 Thread via GitHub
ahmarsuhail commented on PR #5763: URL: https://github.com/apache/hadoop/pull/5763#issuecomment-1651721471 thanks @mukund-thakur, could you please merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop] hfutatzhanghb commented on pull request #5597: HDFS-16993. Datanode supports configure TopN DatanodeNetworkCounts

2023-07-26 Thread via GitHub
hfutatzhanghb commented on PR #5597: URL: https://github.com/apache/hadoop/pull/5597#issuecomment-1651690017 @ayushtkn Sir, could you please help me review this PR again when have free time? Thanks. Have updated unit tests. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] hadoop-yetus commented on pull request #5885: HDFS-17120. Support snapshot diff based copylisting for flat paths.

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5885: URL: https://github.com/apache/hadoop/pull/5885#issuecomment-1651686956 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn merged pull request #5875: HDFS-17115. HttpFS Add Support getErasureCodeCodecs API

2023-07-26 Thread via GitHub
ayushtkn merged PR #5875: URL: https://github.com/apache/hadoop/pull/5875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] haiyang1987 commented on pull request #5415: HDFS-16926. Fix Typo of ImageServlet#Put()

2023-07-26 Thread via GitHub
haiyang1987 commented on PR #5415: URL: https://github.com/apache/hadoop/pull/5415#issuecomment-1651603433 Hi Sir @Hexiaoqiao @ayushtkn @tomscut Could you please help me review this minor changes when you have free time ? Thanks a lot~ -- This is an automated message from the Apache Git

[GitHub] [hadoop] hadoop-yetus commented on pull request #5878: HDFS-17030 Limit wait time for getHAServiceState in ObserverReadProxyProvider

2023-07-26 Thread via GitHub
hadoop-yetus commented on PR #5878: URL: https://github.com/apache/hadoop/pull/5878#issuecomment-1651558999 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] yangjiandan commented on pull request #5890: YARN-11538. CS UI: queue filter do not work as expected when submitti…

2023-07-26 Thread via GitHub
yangjiandan commented on PR #5890: URL: https://github.com/apache/hadoop/pull/5890#issuecomment-1651527262 @slfan1989 Thanks for your quick reply. I have added a UT about this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Assigned] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18805: --- Assignee: Viraj Jasani > s3a large file prefetch tests are too slow, don't

[jira] [Updated] (HADOOP-18246) Remove lower limit on s3a prefetching/caching block size

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18246: Fix Version/s: 3.3.9 > Remove lower limit on s3a prefetching/caching block size >

[jira] [Resolved] (HADOOP-18805) s3a large file prefetch tests are too slow, don't validate data

2023-07-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18805. - Fix Version/s: 3.3.9 Resolution: Fixed > s3a large file prefetch tests are too

  1   2   >