[GitHub] [hadoop] hadoop-yetus commented on pull request #6116: YARN-11548. [Federation] Router Supports Format FederationStateStore.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6116: URL: https://github.com/apache/hadoop/pull/6116#issuecomment-1734796810 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6117: YARN-11576. Improve FederationInterceptorREST AuditLog.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6117: URL: https://github.com/apache/hadoop/pull/6117#issuecomment-1734796210 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6083: URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1734775757 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 opened a new pull request, #6117: YARN-11576. Improve FederationInterceptorREST AuditLog.

2023-09-25 Thread via GitHub
slfan1989 opened a new pull request, #6117: URL: https://github.com/apache/hadoop/pull/6117 ### Description of PR JIRA: YARN-11576. Improve FederationInterceptorREST AuditLog. ### How was this patch tested? ### For code changes: - [ ] Does the title or

[GitHub] [hadoop] slfan1989 opened a new pull request, #6116: YARN-11548. [Federation] Router Supports Format FederationStateStore.

2023-09-25 Thread via GitHub
slfan1989 opened a new pull request, #6116: URL: https://github.com/apache/hadoop/pull/6116 ### Description of PR JIRA: YARN-11548. [Federation] Router Supports Format FederationStateStore. ### How was this patch tested? Add Junit Test. ### For code

[GitHub] [hadoop] slfan1989 commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-25 Thread via GitHub
slfan1989 commented on PR #6055: URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1734671947 @goiri Thank you very much for your help in reviewing the code! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6114: URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734625684 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6114: URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734624458 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] goiri commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
goiri commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336380046 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:

[GitHub] [hadoop] goiri merged pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-25 Thread via GitHub
goiri merged PR #6055: URL: https://github.com/apache/hadoop/pull/6055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18868) Optimize the configuration and use of callqueue overflow trigger failover

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1776#comment-1776 ] ASF GitHub Bot commented on HADOOP-18868: - hadoop-yetus commented on PR #5998: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5998: HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #5998: URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1734364197 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6113: HDFS-17210.Optimize AvailableSpaceBlockPlacementPolicy

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6113: URL: https://github.com/apache/hadoop/pull/6113#issuecomment-1734337567 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6083: URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1734285313 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768785#comment-17768785 ] ASF GitHub Bot commented on HADOOP-18912: - pjfanning opened a new pull request, #6115: URL:

[jira] [Updated] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18912: Labels: pull-request-available (was: ) > upgrade snappy-java to 1.1.10.4 due to CVE >

[GitHub] [hadoop] hadoop-yetus commented on pull request #6114: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6114: URL: https://github.com/apache/hadoop/pull/6114#issuecomment-1734138046 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] pjfanning opened a new pull request, #6115: HADOOP-18912. upgrade snappy-java to 1.1.10.4

2023-09-25 Thread via GitHub
pjfanning opened a new pull request, #6115: URL: https://github.com/apache/hadoop/pull/6115 ### Description of PR HADOOP-18912 ### How was this patch tested? CI build ### For code changes: - [x] Does the title or this PR starts with the corresponding

[jira] [Created] (HADOOP-18912) upgrade snappy-java to 1.1.10.4 due to CVE

2023-09-25 Thread PJ Fanning (Jira)
PJ Fanning created HADOOP-18912: --- Summary: upgrade snappy-java to 1.1.10.4 due to CVE Key: HADOOP-18912 URL: https://issues.apache.org/jira/browse/HADOOP-18912 Project: Hadoop Common Issue

[GitHub] [hadoop] hadoop-yetus commented on pull request #6112: HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6112: URL: https://github.com/apache/hadoop/pull/6112#issuecomment-1734090460 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
slfan1989 commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336093087 ##

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
slfan1989 commented on code in PR #6083: URL: https://github.com/apache/hadoop/pull/6083#discussion_r1336092755 ##

[GitHub] [hadoop] hadoop-yetus commented on pull request #6027: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6027: URL: https://github.com/apache/hadoop/pull/6027#issuecomment-1733954497 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768740#comment-17768740 ] ASF GitHub Bot commented on HADOOP-18257: - hadoop-yetus commented on PR #6000: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733914140 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] brumi1024 commented on pull request #5989: YARN-11514. Extend SchedulerResponse with capacityVector

2023-09-25 Thread via GitHub
brumi1024 commented on PR #5989: URL: https://github.com/apache/hadoop/pull/5989#issuecomment-1733817064 Thanks @tomicooler for the review, merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] brumi1024 merged pull request #5989: YARN-11514. Extend SchedulerResponse with capacityVector

2023-09-25 Thread via GitHub
brumi1024 merged PR #5989: URL: https://github.com/apache/hadoop/pull/5989 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768698#comment-17768698 ] ASF GitHub Bot commented on HADOOP-18908: - hadoop-yetus commented on PR #6106: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6106: HADOOP-18908. Improve S3A region handling.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6106: URL: https://github.com/apache/hadoop/pull/6106#issuecomment-1733731771 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ferdelyi commented on pull request #6027: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
ferdelyi commented on PR #6027: URL: https://github.com/apache/hadoop/pull/6027#issuecomment-1733714660 A new pull request was created for the same change, as broke the chain: https://github.com/apache/hadoop/pull/6114 -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] ferdelyi opened a new pull request, #6114: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
ferdelyi opened a new pull request, #6114: URL: https://github.com/apache/hadoop/pull/6114 ### Description of PR New parameter introduced to enable SSL/TLS for ZK Client for YARN HA, which takes effect when Curator is used (when

[jira] [Updated] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself

2023-09-25 Thread konwu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] konwu updated HADOOP-18911: --- Environment: hadoop 3.3.1 yarn 3.3.1 spark 3.1.3   was: hadoop 3.3.1 yarn 3.3.1   >

[jira] [Updated] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself

2023-09-25 Thread konwu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] konwu updated HADOOP-18911: --- Description: Spark task Driver Thread hang on awaitUninterruptibly thread stack like below: {code:java}

[jira] [Created] (HADOOP-18911) DfsClientShmManager allocSlot hang on awaitUninterruptibly itself

2023-09-25 Thread konwu (Jira)
konwu created HADOOP-18911: -- Summary: DfsClientShmManager allocSlot hang on awaitUninterruptibly itself Key: HADOOP-18911 URL: https://issues.apache.org/jira/browse/HADOOP-18911 Project: Hadoop Common

[GitHub] [hadoop] ferdelyi closed pull request #6027: YARN-11468. Zookeeper SSL/TLS support

2023-09-25 Thread via GitHub
ferdelyi closed pull request #6027: YARN-11468. Zookeeper SSL/TLS support URL: https://github.com/apache/hadoop/pull/6027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768659#comment-17768659 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on PR #6000: URL:

[GitHub] [hadoop] mehakmeet commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-25 Thread via GitHub
mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1733516788 > Have you tested this on actual files? And specifically so many files... total size in GB's kind of scale testing? Have tested this but not at scale. Will do that. Example

[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768655#comment-17768655 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on code in PR #6000: URL:

[GitHub] [hadoop] mehakmeet commented on a diff in pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-25 Thread via GitHub
mehakmeet commented on code in PR #6000: URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335767883 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (HADOOP-18868) Optimize the configuration and use of callqueue overflow trigger failover

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768653#comment-17768653 ] ASF GitHub Bot commented on HADOOP-18868: - haiyang1987 commented on PR #5998: URL:

[GitHub] [hadoop] haiyang1987 commented on pull request #5998: HADOOP-18868. Optimize the configuration and use of callqueue overflow trigger failover

2023-09-25 Thread via GitHub
haiyang1987 commented on PR #5998: URL: https://github.com/apache/hadoop/pull/5998#issuecomment-1733494535 Hi Sirs @Hexiaoqiao @goiri can you help me take a look at this PR when you have time? Thank you very much~ -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hadoop] haiyang1987 opened a new pull request, #6112: HDFS-17205. HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

2023-09-25 Thread via GitHub
haiyang1987 opened a new pull request, #6112: URL: https://github.com/apache/hadoop/pull/6112 ### Description of PR https://issues.apache.org/jira/browse/HDFS-17205. Current allocate new block , the NameNode will choose datanode and choose a good storage of given storage type

[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768647#comment-17768647 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on code in PR #6000: URL:

[GitHub] [hadoop] mehakmeet commented on a diff in pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-25 Thread via GitHub
mehakmeet commented on code in PR #6000: URL: https://github.com/apache/hadoop/pull/6000#discussion_r1335750686 ## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/audit/AuditTool.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1733325918 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18829) s3a prefetch LRU cache eviction metric

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768605#comment-17768605 ] ASF GitHub Bot commented on HADOOP-18829: - hadoop-yetus commented on PR #6111: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6111: HADOOP-18829. S3A prefetch LRU cache eviction metrics (#5893)

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6111: URL: https://github.com/apache/hadoop/pull/6111#issuecomment-1733240527 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18873) ABFS: AbfsOutputStream doesnt close DataBlocks object.

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768594#comment-17768594 ] ASF GitHub Bot commented on HADOOP-18873: - mehakmeet merged PR #6105: URL:

[GitHub] [hadoop] mehakmeet merged pull request #6105: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object. [branch-3.3]

2023-09-25 Thread via GitHub
mehakmeet merged PR #6105: URL: https://github.com/apache/hadoop/pull/6105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #6083: YARN-8862. [BackPort] [GPG] Add Yarn Registry cleanup in ApplicationCleaner.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6083: URL: https://github.com/apache/hadoop/pull/6083#issuecomment-1733147498 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] lfrancke commented on pull request #6108: HDFS-17181 WebHDFS: Route all CREATE requests to the BlockManager

2023-09-25 Thread via GitHub
lfrancke commented on PR #6108: URL: https://github.com/apache/hadoop/pull/6108#issuecomment-1733108376 All test failures seem unrelated and I didn't add any new tests because the code seems to be covered by tests already. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] hadoop-yetus commented on pull request #6055: YARN-11547. [Federation] Router Supports Remove individual application records from FederationStateStore.

2023-09-25 Thread via GitHub
hadoop-yetus commented on PR #6055: URL: https://github.com/apache/hadoop/pull/6055#issuecomment-1733030176 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18883) Expect-100 JDK bug resolution: prevent multiple server calls

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768542#comment-17768542 ] ASF GitHub Bot commented on HADOOP-18883: - saxenapranav commented on PR #6022: URL:

[jira] [Updated] (HADOOP-18883) Expect-100 JDK bug resolution: prevent multiple server calls

2023-09-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18883: Labels: pull-request-available (was: ) > Expect-100 JDK bug resolution: prevent

[GitHub] [hadoop] saxenapranav commented on pull request #6022: HADOOP-18883. [ABFS]: Expect-100 JDK bug resolution: prevent multiple server calls

2023-09-25 Thread via GitHub
saxenapranav commented on PR #6022: URL: https://github.com/apache/hadoop/pull/6022#issuecomment-1732987230 @steveloughran @mehakmeet , requesting your review please. Thank you so much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on