[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17743000#comment-17743000 ] ASF GitHub Bot commented on HADOOP-18291: - virajjasani commented on PR #5754: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
virajjasani commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1635203629 `us-west-2`: ``` $ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch $ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale ```

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5807: HDFS-17080. fix ec connection leak

2023-07-13 Thread via GitHub
Hexiaoqiao commented on PR #5807: URL: https://github.com/apache/hadoop/pull/5807#issuecomment-1635171125 Approved. Please following the guide of email from ASF. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hfutatzhanghb closed pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method.

2023-07-13 Thread via GitHub
hfutatzhanghb closed pull request #5594: HDFS-16992. Change log level of some logs in FSNamesystem#internalReleaseLease method. URL: https://github.com/apache/hadoop/pull/5594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] harris233 commented on pull request #5807: HDFS-17080. fix ec connection leak

2023-07-13 Thread via GitHub
harris233 commented on PR #5807: URL: https://github.com/apache/hadoop/pull/5807#issuecomment-1635161725 > @jojochuang Thanks WeiChiu for your help. @harris233 Would you mind to re-apply, and attach this PR link to the application form? I will try to trace and approve. OK, I have

[GitHub] [hadoop] harris233 commented on pull request #5807: HDFS-17080. fix ec connection leak

2023-07-13 Thread via GitHub
harris233 commented on PR #5807: URL: https://github.com/apache/hadoop/pull/5807#issuecomment-1635159912 > Opened [HDFS-17080](https://issues.apache.org/jira/browse/HDFS-17080) to track this PR Thanks -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742974#comment-17742974 ] ASF GitHub Bot commented on HADOOP-18291: - hadoop-yetus commented on PR #5754: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1635020426 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5835: HDFS-17084. Utilize StringTable for numerable XAttributes

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5835: URL: https://github.com/apache/hadoop/pull/5835#issuecomment-1634979649 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn merged pull request #5834: HDFS-17082. Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md"

2023-07-13 Thread via GitHub
ayushtkn merged PR #5834: URL: https://github.com/apache/hadoop/pull/5834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742919#comment-17742919 ] ASF GitHub Bot commented on HADOOP-18291: - virajjasani commented on PR #5754: URL:

[GitHub] [hadoop] virajjasani commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
virajjasani commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1634715531 re-running the whole test suite with -prefetch and -scale combinations -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hadoop] hadoop-yetus commented on pull request #5834: HDFS-17082. Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md"

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5834: URL: https://github.com/apache/hadoop/pull/5834#issuecomment-1634606037 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18776) Add OptimizedS3AMagicCommitter For Zero Rename Commits to S3 Endpoints

2023-07-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742881#comment-17742881 ] Steve Loughran commented on HADOOP-18776: - you know, one way this could be made to work would

[GitHub] [hadoop] hadoop-yetus commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1634561803 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1634508657 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on pull request #5816: YARN-11525. [Federation] Router CLI Supports Save the SubClusterPolicyConfiguration Of Queues.

2023-07-13 Thread via GitHub
slfan1989 commented on PR #5816: URL: https://github.com/apache/hadoop/pull/5816#issuecomment-1634424471 @goiri Can you help review the code again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-1634386815 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-1634371201 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5837: fix NullPointerException on invalid remote-app-log-dir

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5837: URL: https://github.com/apache/hadoop/pull/5837#issuecomment-1634296139 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5809: HDFS-17068. Datanode should record last directory scan time.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5809: URL: https://github.com/apache/hadoop/pull/5809#issuecomment-1634142332 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18776) Add OptimizedS3AMagicCommitter For Zero Rename Commits to S3 Endpoints

2023-07-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742785#comment-17742785 ] Steve Loughran commented on HADOOP-18776: - thanks -sorry for being so strict here. will take

[GitHub] [hadoop] bucko909 commented on pull request #5837: fix NullPointerException on invalid remote-app-log-dir

2023-07-13 Thread via GitHub
bucko909 commented on PR #5837: URL: https://github.com/apache/hadoop/pull/5837#issuecomment-1634095336 (I haven't included a Jira ticket as there isn't one; I'm not sure what the object storage requirements are. Feel free to reject if this is unacceptable; I don't really want to go to

[GitHub] [hadoop] bucko909 opened a new pull request, #5837: fix NullPointerException on invalid remote-app-log-dir

2023-07-13 Thread via GitHub
bucko909 opened a new pull request, #5837: URL: https://github.com/apache/hadoop/pull/5837 I had an invalid URL for this property. The NodeManager output: ``` 2023-07-13 10:14:58,397 ERROR logaggregation.AppLogAggregatorImpl: Error occurred while aggregating the log for the

[GitHub] [hadoop] haiyang1987 commented on pull request #5833: HDFS-17081. Append ec file check if a block is replicated to at least the minimum replication need consider striped block

2023-07-13 Thread via GitHub
haiyang1987 commented on PR #5833: URL: https://github.com/apache/hadoop/pull/5833#issuecomment-1634065274 Hi sir @Hexiaoqiao @ayushtkn @ZanderXu could you please help me review this changes when you have free time ? Thanks a lot~ -- This is an automated message from the Apache Git

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742753#comment-17742753 ] ASF GitHub Bot commented on HADOOP-18291: - hadoop-yetus commented on PR #5754: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1634009019 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742751#comment-17742751 ] ASF GitHub Bot commented on HADOOP-18291: - hadoop-yetus commented on PR #5754: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1633997580 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742748#comment-17742748 ] ASF GitHub Bot commented on HADOOP-18291: - hadoop-yetus commented on PR #5754: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5754: URL: https://github.com/apache/hadoop/pull/5754#issuecomment-1633984369 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5834: HDFS-17082. Add documentation for provisionSnapshotTrash command to HDFSCommands.md and HdfsSnapshots.md"

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5834: URL: https://github.com/apache/hadoop/pull/5834#issuecomment-1633792237 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742676#comment-17742676 ] ASF GitHub Bot commented on HADOOP-18291: - virajjasani commented on code in PR #5754: URL:

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
virajjasani commented on code in PR #5754: URL: https://github.com/apache/hadoop/pull/5754#discussion_r1262051712 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java: ## @@ -1276,4 +1276,16 @@ private Constants() { public static final String

[jira] [Commented] (HADOOP-18291) S3A prefetch - Implement LRU cache for SingleFilePerBlockCache

2023-07-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742674#comment-17742674 ] ASF GitHub Bot commented on HADOOP-18291: - virajjasani commented on code in PR #5754: URL:

[GitHub] [hadoop] virajjasani commented on a diff in pull request #5754: HADOOP-18291. S3A prefetch - Implement thread-safe LRU cache for SingleFilePerBlockCache

2023-07-13 Thread via GitHub
virajjasani commented on code in PR #5754: URL: https://github.com/apache/hadoop/pull/5754#discussion_r1262044107 ## hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3APrefetchingLruEviction.java: ## @@ -0,0 +1,241 @@ +/* + * Licensed to the Apache Software

[GitHub] [hadoop] hadoop-yetus commented on pull request #5810: YARN-7707. BackPort [GPG] Policy generator framework.

2023-07-13 Thread via GitHub
hadoop-yetus commented on PR #5810: URL: https://github.com/apache/hadoop/pull/5810#issuecomment-1633605106 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |