[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637062#comment-17637062 ] xinqiu.hu commented on HADOOP-18536: Thanks for your patience! > RPC Client Improvement >

[GitHub] [hadoop] howzi commented on pull request #5147: HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance

2022-11-21 Thread GitBox
howzi commented on PR #5147: URL: https://github.com/apache/hadoop/pull/5147#issuecomment-1323243368 > > Actually it is an obvious performance problem, it takes over 3 mins to refresh the state store cache in our enviroment. Different deployment of ZK may cause a diffrent choice. For

[jira] [Comment Edited] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637046#comment-17637046 ] Shilun Fan edited comment on HADOOP-18536 at 11/22/22 7:42 AM: --- Thank you

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637046#comment-17637046 ] Shilun Fan commented on HADOOP-18536: - Thank you very much for your contribution, let us wait for

[GitHub] [hadoop] howzi commented on pull request #5147: HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance

2022-11-21 Thread GitBox
howzi commented on PR #5147: URL: https://github.com/apache/hadoop/pull/5147#issuecomment-1323233078 > @howzi Thanks for your report and this change makes sense. > > 1. How about keeping the sync mode and adding a new async mode? > 2. Can you add one UT to verify the performance

[jira] [Comment Edited] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637039#comment-17637039 ] Shilun Fan edited comment on HADOOP-18536 at 11/22/22 7:35 AM: --- Thank you

[jira] [Comment Edited] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637039#comment-17637039 ] Shilun Fan edited comment on HADOOP-18536 at 11/22/22 7:34 AM: --- Thank you

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637039#comment-17637039 ] Shilun Fan commented on HADOOP-18536: - Thank you very much for your feedback. Because it is widely

[jira] [Comment Edited] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637036#comment-17637036 ] xinqiu.hu edited comment on HADOOP-18536 at 11/22/22 7:25 AM: --   Thank you

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637036#comment-17637036 ] xinqiu.hu commented on HADOOP-18536:   Thank you very much for reviewing the code, I agree with the

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Description:   In the RPC Client, before a request (including RpcRequestHeaderProto,

[jira] [Comment Edited] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637013#comment-17637013 ] Shilun Fan edited comment on HADOOP-18536 at 11/22/22 6:04 AM: --- Thank you

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637013#comment-17637013 ] Shilun Fan commented on HADOOP-18536: - >From my personal point of view, I think the risk of this

[jira] [Commented] (HADOOP-18533) RPC Client performance improvement

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637012#comment-17637012 ] ASF GitHub Bot commented on HADOOP-18533: - huxinqiu closed pull request #5151: HADOOP-18533.

[GitHub] [hadoop] huxinqiu closed pull request #5151: HADOOP-18533. RPC Client performance improvement

2022-11-21 Thread GitBox
huxinqiu closed pull request #5151: HADOOP-18533. RPC Client performance improvement URL: https://github.com/apache/hadoop/pull/5151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Description:   In the RPC Client, before a request (including RpcRequestHeaderProto,

[jira] [Commented] (HADOOP-18533) RPC Client performance improvement

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637011#comment-17637011 ] ASF GitHub Bot commented on HADOOP-18533: - hadoop-yetus commented on PR #5151: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5151: HADOOP-18533. RPC Client performance improvement

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5151: URL: https://github.com/apache/hadoop/pull/5151#issuecomment-1323121288 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Description:   In the RPC Client, before a request (including RpcRequestHeaderProto,

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637010#comment-17637010 ] xinqiu.hu commented on HADOOP-18536: HADOOP-18536 is a better approach than HADOOP-18533, with the

[jira] [Commented] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread Shilun Fan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637009#comment-17637009 ] Shilun Fan commented on HADOOP-18536: - Is there any relationship between the three jiras

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Description:   In the RPC Client, before a request (including RpcRequestHeaderProto,

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Target Version/s: 3.4.0 Priority: Minor (was: Major) > RPC Client Improvement >

[jira] [Updated] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinqiu.hu updated HADOOP-18536: --- Description:   In the RPC Client, before a request (including RpcRequestHeaderProto,

[jira] [Commented] (HADOOP-18533) RPC Client performance improvement

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637008#comment-17637008 ] ASF GitHub Bot commented on HADOOP-18533: - huxinqiu commented on PR #5151: URL:

[GitHub] [hadoop] huxinqiu commented on pull request #5151: HADOOP-18533. RPC Client performance improvement

2022-11-21 Thread GitBox
huxinqiu commented on PR #5151: URL: https://github.com/apache/hadoop/pull/5151#issuecomment-1323110323 [HADOOP-18536](https://github.com/apache/hadoop/pull/5156) may be more suitable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hadoop] huxinqiu opened a new pull request, #5156: RPC Client Improvement

2022-11-21 Thread GitBox
huxinqiu opened a new pull request, #5156: URL: https://github.com/apache/hadoop/pull/5156 In the RPC Client, before a request (including RpcRequestHeaderProto, RequestHeaderProto, Message Payload) is sent, they will be copied to the three CodedOutputStream internal byte arrays, and then

[jira] [Created] (HADOOP-18536) RPC Client Improvement

2022-11-21 Thread xinqiu.hu (Jira)
xinqiu.hu created HADOOP-18536: -- Summary: RPC Client Improvement Key: HADOOP-18536 URL: https://issues.apache.org/jira/browse/HADOOP-18536 Project: Hadoop Common Issue Type: Improvement

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1323041154 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1323037298 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1323033683 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] GuoPhilipse commented on pull request #4602: HDFS-16673. Fix usage of chown

2022-11-21 Thread GitBox
GuoPhilipse commented on PR #4602: URL: https://github.com/apache/hadoop/pull/4602#issuecomment-1322966990 > Did you test it? This behavior is consistent with the local file system. Thanks for your review @tomscut , actually `chown ` command can be used for change owner or group

[GitHub] [hadoop] hadoop-yetus commented on pull request #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5155: URL: https://github.com/apache/hadoop/pull/5155#issuecomment-1322938837 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] haiyang1987 commented on pull request #5129: HDFS-16840. Enhance the usage description about oiv in HDFSCommands.md and OfflineImageViewerPB

2022-11-21 Thread GitBox
haiyang1987 commented on PR #5129: URL: https://github.com/apache/hadoop/pull/5129#issuecomment-1322920504 Update PR @ZanderXu @tomscut @tasanuma please help me to reivew it again, Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #5129: HDFS-16840. Enhance the usage description about oiv in HDFSCommands.md and OfflineImageViewerPB

2022-11-21 Thread GitBox
haiyang1987 commented on code in PR #5129: URL: https://github.com/apache/hadoop/pull/5129#discussion_r1028719428 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java: ## @@ -81,6 +81,8 @@ public class

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #5129: HDFS-16840. Enhance the usage description about oiv in HDFSCommands.md and OfflineImageViewerPB

2022-11-21 Thread GitBox
haiyang1987 commented on code in PR #5129: URL: https://github.com/apache/hadoop/pull/5129#discussion_r1028717101 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java: ## @@ -81,6 +81,8 @@ public class

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
simbadzina commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028715143 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -348,25 +357,28 @@

[GitHub] [hadoop] hadoop-yetus commented on pull request #4602: HDFS-16673. Fix usage of chown

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #4602: URL: https://github.com/apache/hadoop/pull/4602#issuecomment-1322882225 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] tomscut commented on pull request #4209: HDFS-16550. Improper cache-size for journal node may cause cluster crash

2022-11-21 Thread GitBox
tomscut commented on PR #4209: URL: https://github.com/apache/hadoop/pull/4209#issuecomment-1322873539 > I am -1 on the PR as-is. We have publicly exposed the current config `dfs.journalnode.edit-cache-size.bytes`; we can't just rename it and change the behavior now. I also think there is

[jira] [Created] (HADOOP-18535) Implement token storage solution based on MySQL

2022-11-21 Thread Hector Sandoval Chaverri (Jira)
Hector Sandoval Chaverri created HADOOP-18535: - Summary: Implement token storage solution based on MySQL Key: HADOOP-18535 URL: https://issues.apache.org/jira/browse/HADOOP-18535 Project:

[jira] [Commented] (HADOOP-18399) SingleFilePerBlockCache to use LocalDirAllocator for file allocatoin

2022-11-21 Thread Viraj Jasani (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636945#comment-17636945 ] Viraj Jasani commented on HADOOP-18399: --- [~ste...@apache.org], just checking here if you got some

[GitHub] [hadoop] omalley commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
omalley commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028642292 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java: ## @@ -349,6 +349,13 @@ public static ClientProtocol

[GitHub] [hadoop] omalley commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
omalley commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028642292 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java: ## @@ -349,6 +349,13 @@ public static ClientProtocol

[GitHub] [hadoop] hadoop-yetus commented on pull request #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5155: URL: https://github.com/apache/hadoop/pull/5155#issuecomment-1322844291 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] omalley commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
omalley commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028638870 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -88,6 +88,15 @@

[GitHub] [hadoop] slfan1989 commented on pull request #5153: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

2022-11-21 Thread GitBox
slfan1989 commented on PR #5153: URL: https://github.com/apache/hadoop/pull/5153#issuecomment-1322823190 @ayushtkn Can you help review this PR? Thank you very much! The issue of checkstyle is not caused by this pr, this issue also existed before. -- This is an automated message from the

[GitHub] [hadoop] slfan1989 commented on pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
slfan1989 commented on PR #5152: URL: https://github.com/apache/hadoop/pull/5152#issuecomment-1322821848 @ayushtkn Can you help to review this PR again? Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] hadoop-yetus commented on pull request #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5155: URL: https://github.com/apache/hadoop/pull/5155#issuecomment-1322720740 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] omalley commented on a diff in pull request #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
omalley commented on code in PR #5155: URL: https://github.com/apache/hadoop/pull/5155#discussion_r1028560332 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java: ## @@ -97,6 +104,7 @@ public class RouterAdmin extends

[GitHub] [hadoop] hadoop-yetus commented on pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5142: URL: https://github.com/apache/hadoop/pull/5142#issuecomment-1322676213 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #4949: YARN-8262. get_executable in container-executor should provide meaningful error codes

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #4949: URL: https://github.com/apache/hadoop/pull/4949#issuecomment-1322675838 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1322665295 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
ayushtkn commented on code in PR #5155: URL: https://github.com/apache/hadoop/pull/5155#discussion_r1028506972 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java: ## @@ -97,6 +104,7 @@ public class RouterAdmin extends

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
simbadzina commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028476373 ## hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java: ## @@ -349,6 +349,13 @@ public static ClientProtocol

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
simbadzina commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028474832 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestObserverWithRouter.java: ## @@ -439,4 +440,60 @@ public void

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1322583721 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1322571575 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] omalley opened a new pull request, #5155: HDFS-16851: RBF: Add a utility to dump the StateStore.

2022-11-21 Thread GitBox
omalley opened a new pull request, #5155: URL: https://github.com/apache/hadoop/pull/5155 ### Description of PR Adds a utility to dump the RBF StateStore. ### How was this patch tested? It was tested manually against our RBF cluster -- This is an automated

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
simbadzina commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028419083 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -88,6 +88,15 @@

[GitHub] [hadoop] simbadzina commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
simbadzina commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028394821 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -366,7 +375,7 @@

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider

2022-11-21 Thread GitBox
mkuchenbecker commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028371062 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestObserverWithRouter.java: ## @@ -122,7 +123,9 @@ public

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
mkuchenbecker commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028385376 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/store/driver/TestStateStoreDriverBase.java: ## @@ -234,6 +234,25 @@

[GitHub] [hadoop] xkrogen merged pull request #4201: HDFS-16547. [SBN read] Namenode in safe mode should not be transfered to observer state

2022-11-21 Thread GitBox
xkrogen merged PR #4201: URL: https://github.com/apache/hadoop/pull/4201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
mkuchenbecker commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028382408 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -366,7 +375,7

[GitHub] [hadoop] xkrogen commented on pull request #4201: HDFS-16547. [SBN read] Namenode in safe mode should not be transfered to observer state

2022-11-21 Thread GitBox
xkrogen commented on PR #4201: URL: https://github.com/apache/hadoop/pull/4201#issuecomment-1322464405 `TestLeaseRecovery2` indeed seems to be broken; I confirmed that the behavior is the same before/after applying this PR. Merging to trunk. Thanks for the contribution @tomscut ! --

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
mkuchenbecker commented on code in PR #5145: URL: https://github.com/apache/hadoop/pull/5145#discussion_r1028378298 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreFileBaseImpl.java: ## @@ -88,6 +88,15 @@

[GitHub] [hadoop] mkuchenbecker commented on a diff in pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider

2022-11-21 Thread GitBox
mkuchenbecker commented on code in PR #5142: URL: https://github.com/apache/hadoop/pull/5142#discussion_r1028372645 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestObserverWithRouter.java: ## @@ -439,4 +440,60 @@ public

[GitHub] [hadoop] mkuchenbecker commented on pull request #5142: HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider.

2022-11-21 Thread GitBox
mkuchenbecker commented on PR #5142: URL: https://github.com/apache/hadoop/pull/5142#issuecomment-1322457075 Mostly mechanical review, with some questions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hadoop] simbadzina commented on pull request #5145: HDFS-16847: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException.

2022-11-21 Thread GitBox
simbadzina commented on PR #5145: URL: https://github.com/apache/hadoop/pull/5145#issuecomment-1322414231 I've added a unit test. Without my patch. We write a zero byte file and end up with the following error when trying to read the state store. > Caused by: java.io.IOException:

[jira] [Commented] (HADOOP-16761) KMSClientProvider does not work with client using ticket logged in externally

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636783#comment-17636783 ] ASF GitHub Bot commented on HADOOP-16761: - hadoop-yetus commented on PR #1769: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #1769: HADOOP-16761. KMSClientProvider does not work with client using ticke…

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #1769: URL: https://github.com/apache/hadoop/pull/1769#issuecomment-132241 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] xkrogen commented on pull request #4209: HDFS-16550. Improper cache-size for journal node may cause cluster crash

2022-11-21 Thread GitBox
xkrogen commented on PR #4209: URL: https://github.com/apache/hadoop/pull/4209#issuecomment-1322361104 I am -1 on the PR as-is. We have publicly exposed the current config `dfs.journalnode.edit-cache-size.bytes`; we can't just rename it and change the behavior now. I also think there is a

[GitHub] [hadoop] xkrogen commented on a diff in pull request #4744: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer

2022-11-21 Thread GitBox
xkrogen commented on code in PR #4744: URL: https://github.com/apache/hadoop/pull/4744#discussion_r1028281649 ## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java: ## @@ -174,6 +175,11 @@ protected FSImage(Configuration conf,

[GitHub] [hadoop] xkrogen merged pull request #5099: HDFS-16832. [SBN READ] Fix NPE when check the block location of empty…

2022-11-21 Thread GitBox
xkrogen merged PR #5099: URL: https://github.com/apache/hadoop/pull/5099 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] xkrogen commented on pull request #5099: HDFS-16832. [SBN READ] Fix NPE when check the block location of empty…

2022-11-21 Thread GitBox
xkrogen commented on PR #5099: URL: https://github.com/apache/hadoop/pull/5099#issuecomment-1322317231 `TestLeaseRecovery2` has been flaky; I am not worried about the failure. Merging to trunk. Thank you for the contribution @zhengchenyu ! -- This is an automated message from the

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #4655: YARN-11216. Avoid unnecessary reconstruction of ConfigurationProperties

2022-11-21 Thread GitBox
szilard-nemeth commented on code in PR #4655: URL: https://github.com/apache/hadoop/pull/4655#discussion_r1028179146 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java: ## @@ -242,6 +244,10 @@ public class Configuration implements

[jira] [Commented] (HADOOP-18533) RPC Client performance improvement

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636744#comment-17636744 ] ASF GitHub Bot commented on HADOOP-18533: - hadoop-yetus commented on PR #5151: URL:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5151: HADOOP-18533. RPC Client performance improvement

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5151: URL: https://github.com/apache/hadoop/pull/5151#issuecomment-1322263878 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5119: YARN-5607. Document TestContainerResourceUsage#waitForContainerCompletion

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5119: URL: https://github.com/apache/hadoop/pull/5119#issuecomment-1322241513 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] szilard-nemeth commented on a diff in pull request #4949: YARN-8262. get_executable in container-executor should provide meaningful error codes

2022-11-21 Thread GitBox
szilard-nemeth commented on code in PR #4949: URL: https://github.com/apache/hadoop/pull/4949#discussion_r1028166041 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java:

[GitHub] [hadoop] slfan1989 commented on pull request #5146: YARN-11373. [Federation] Support refreshQueues、refreshNodes API's for Federation.

2022-11-21 Thread GitBox
slfan1989 commented on PR #5146: URL: https://github.com/apache/hadoop/pull/5146#issuecomment-1322185823 @goiri Can you help review this PR? Thank you very much! The java-doc error is not caused by this pr code, I submitted 2 prs for repair. YARN-11380. Fix hadoop-yarn-api module

[GitHub] [hadoop] hadoop-yetus commented on pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5152: URL: https://github.com/apache/hadoop/pull/5152#issuecomment-1322078791 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] hadoop-yetus commented on pull request #5153: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5153: URL: https://github.com/apache/hadoop/pull/5153#issuecomment-1321975381 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
ayushtkn commented on code in PR #5152: URL: https://github.com/apache/hadoop/pull/5152#discussion_r1027903125 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/package-info.java: ## @@ -1,4 +1,4 @@ -/* +/** Review

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
slfan1989 commented on code in PR #5152: URL: https://github.com/apache/hadoop/pull/5152#discussion_r1027897718 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/package-info.java: ## @@ -1,4 +1,4 @@ -/* +/** Review

[GitHub] [hadoop] hadoop-yetus commented on pull request #5131: YARN-11350. [Federation] Router Support DelegationToken With ZK.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5131: URL: https://github.com/apache/hadoop/pull/5131#issuecomment-1321858433 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18523) Allow to retrieve an object from MinIO (S3 API) with a very restrictive policy

2022-11-21 Thread Jira
[ https://issues.apache.org/jira/browse/HADOOP-18523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636624#comment-17636624 ] Sébastien Burton commented on HADOOP-18523: --- Hello [~ste...@apache.org], I cannot take the

[GitHub] [hadoop] ahmarsuhail closed pull request #5154: Hadoop 18073 sdk upgrade delete select mpu

2022-11-21 Thread GitBox
ahmarsuhail closed pull request #5154: Hadoop 18073 sdk upgrade delete select mpu URL: https://github.com/apache/hadoop/pull/5154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] hadoop-yetus commented on pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5152: URL: https://github.com/apache/hadoop/pull/5152#issuecomment-1321831548 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] ahmarsuhail opened a new pull request, #5154: Hadoop 18073 sdk upgrade delete select mpu

2022-11-21 Thread GitBox
ahmarsuhail opened a new pull request, #5154: URL: https://github.com/apache/hadoop/pull/5154 WIP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hadoop] hadoop-yetus commented on pull request #5146: YARN-11373. [Federation] Support refreshQueues、refreshNodes API's for Federation.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5146: URL: https://github.com/apache/hadoop/pull/5146#issuecomment-1321787296 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18533) RPC Client performance improvement

2022-11-21 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636570#comment-17636570 ] ASF GitHub Bot commented on HADOOP-18533: - huxinqiu commented on PR #5151: URL:

[GitHub] [hadoop] huxinqiu commented on pull request #5151: HADOOP-18533. RPC Client performance improvement

2022-11-21 Thread GitBox
huxinqiu commented on PR #5151: URL: https://github.com/apache/hadoop/pull/5151#issuecomment-1321736995 > @huxinqiu Thank you very much for your contribution! > > We need to discuss something: > > 1. It seems that the benefit is to avoid declaring this variable

[GitHub] [hadoop] hadoop-yetus commented on pull request #5153: YARN-11381. Fix hadoop-yarn-common module Java Doc Errors.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5153: URL: https://github.com/apache/hadoop/pull/5153#issuecomment-1321705500 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
slfan1989 commented on code in PR #5152: URL: https://github.com/apache/hadoop/pull/5152#discussion_r1027713838 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/package-info.java: ## @@ -1,4 +1,4 @@ -/* +/** Review

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #5152: YARN-11380. Fix hadoop-yarn-api module Java Doc Errors.

2022-11-21 Thread GitBox
slfan1989 commented on code in PR #5152: URL: https://github.com/apache/hadoop/pull/5152#discussion_r1027713838 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/package-info.java: ## @@ -1,4 +1,4 @@ -/* +/** Review

[GitHub] [hadoop] hadoop-yetus commented on pull request #5104: YARN-11158. Support (Create/Renew/Cancel) DelegationToken API's for Federation.

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #5104: URL: https://github.com/apache/hadoop/pull/5104#issuecomment-1321684954 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |

[jira] [Commented] (HADOOP-18534) Propose a mechanism to free the direct memory occupied by RPC Connections

2022-11-21 Thread xinqiu.hu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636516#comment-17636516 ] xinqiu.hu commented on HADOOP-18534: [~slfan1989]    If a connection continues to process requests

[GitHub] [hadoop] hadoop-yetus commented on pull request #4717: YARN-6946. Upgrade JUnit from 4 to 5 in hadoop-yarn-common

2022-11-21 Thread GitBox
hadoop-yetus commented on PR #4717: URL: https://github.com/apache/hadoop/pull/4717#issuecomment-1321649048 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: |