[GitHub] [hadoop] hadoop-yetus commented on pull request #6019: HADOOP-18872: [ABFS] [BugFix] Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6019: URL: https://github.com/apache/hadoop/pull/6019#issuecomment-1704696088 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704650805 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] anujmodi2021 commented on a diff in pull request #5881: Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout.

2023-09-03 Thread via GitHub
anujmodi2021 commented on code in PR #5881: URL: https://github.com/apache/hadoop/pull/5881#discussion_r1308427326 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/StaticRetryPolicy.java: ## @@ -0,0 +1,68 @@ +/** + * Licensed to the Apache Softwa

[GitHub] [hadoop] mehakmeet commented on pull request #6000: HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis.

2023-09-03 Thread via GitHub
mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1704605738 CC: @steveloughran @mukund-thakur -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[jira] [Updated] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-09-03 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18872: --- Description: There was a bug identified where retry count in the client correlation id was wrongly

[jira] [Updated] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-09-03 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18872: --- Labels: Bug (was: ) > ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations >

[jira] [Updated] (HADOOP-18872) ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations

2023-09-03 Thread Anuj Modi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anuj Modi updated HADOOP-18872: --- Summary: ABFS: Misreporting Retry Count for Sub-sequential and Parallel Operations (was: ABFS: Fix

[GitHub] [hadoop] Aakansha99 opened a new pull request, #6020: Oss/hadoop java17 runtime feature branch prerequisite

2023-09-03 Thread via GitHub
Aakansha99 opened a new pull request, #6020: URL: https://github.com/apache/hadoop/pull/6020 ### Description of PR The latest Hadoop version 3.3.3 which supports Java 8 compile and [Java 8, Java 11](https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions) runtime

[GitHub] [hadoop] anujmodi2021 opened a new pull request, #6019: Hadoop 18872 tracing fix

2023-09-03 Thread via GitHub
anujmodi2021 opened a new pull request, #6019: URL: https://github.com/apache/hadoop/pull/6019 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HA

[GitHub] [hadoop] saxenapranav commented on a diff in pull request #6010: HADOOP-18873. ABFS: AbfsOutputStream doesnt close DataBlocks object.

2023-09-03 Thread via GitHub
saxenapranav commented on code in PR #6010: URL: https://github.com/apache/hadoop/pull/6010#discussion_r1314415846 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsOutputStream.java: ## @@ -345,6 +345,7 @@ private void uploadBlockAsync(DataBl

[GitHub] [hadoop] hadoop-yetus commented on pull request #6018: HDFS-17178: BootstrapStandby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6018: URL: https://github.com/apache/hadoop/pull/6018#issuecomment-1704557206 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398962 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/FederationInterceptor.

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314400717 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestWorkPreservingUnmana

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398962 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/FederationInterceptor.

[GitHub] [hadoop] zhengchenyu commented on a diff in pull request #5975: YARN-8980. Mapreduce application container start fail after AM restart.

2023-09-03 Thread via GitHub
zhengchenyu commented on code in PR #5975: URL: https://github.com/apache/hadoop/pull/5975#discussion_r1314398549 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/amrmproxy/FederationInterceptor.

[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4

2023-09-03 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761668#comment-17761668 ] Masatake Iwasaki commented on HADOOP-18867: --- YARN-11558 should be fixed first

[GitHub] [hadoop] virajjasani commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
virajjasani commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704478561 `us-west-2`: `mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch` Checkstyle complains about 8th param added to CachingBlockManager c'tor. Shall we mark

[GitHub] [hadoop] dannytbecker commented on pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-03 Thread via GitHub
dannytbecker commented on PR #6013: URL: https://github.com/apache/hadoop/pull/6013#issuecomment-1704453488 @simbadzina @goiri Looks like the test failure is unrelated. The test I added, “TestObserverNode#testConfigStartup”, was fixed. -- This is an automated message from the Apache Git S

[GitHub] [hadoop] hadoop-yetus commented on pull request #6013: HDFS-17167. Add config to startup NameNode as Observer

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6013: URL: https://github.com/apache/hadoop/pull/6013#issuecomment-1704445299 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metrics

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704428844 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6018: HDFS-17178: BootstrapStandby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6018: URL: https://github.com/apache/hadoop/pull/6018#issuecomment-1704397100 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] dannytbecker opened a new pull request, #6018: HDFS-17178: Bootstrap Standby needs to handle RollingUpgrade

2023-09-03 Thread via GitHub
dannytbecker opened a new pull request, #6018: URL: https://github.com/apache/hadoop/pull/6018 ### Description of PR During rollingUpgrade, bootstrapStandby will fail with an exception due to different NameNodeLayoutVersions. We can ignore this safely during RollingUpgrade because differ

[GitHub] [hadoop] virajjasani commented on pull request #5893: HADOOP-18829. S3A prefetch LRU cache eviction metric

2023-09-03 Thread via GitHub
virajjasani commented on PR #5893: URL: https://github.com/apache/hadoop/pull/5893#issuecomment-1704373448 > Duration tracker for the removal of a block? done, thank you for this suggestion! -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [hadoop] hadoop-yetus commented on pull request #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6017: URL: https://github.com/apache/hadoop/pull/6017#issuecomment-1704349722 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[jira] [Created] (HADOOP-18881) ZKDTSM could be stuck when meet znode version overflow

2023-09-03 Thread Xiaoqiao He (Jira)
Xiaoqiao He created HADOOP-18881: Summary: ZKDTSM could be stuck when meet znode version overflow Key: HADOOP-18881 URL: https://issues.apache.org/jira/browse/HADOOP-18881 Project: Hadoop Common

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #5990: URL: https://github.com/apache/hadoop/pull/5990#issuecomment-1704342943 Hi @goiri @simbadzina The latest CI report green here and you have approved this PR. If no more comments here, Please help to check in. Thanks. -- This is an automated message from th

[GitHub] [hadoop] Hexiaoqiao commented on a diff in pull request #6001: HADOOP-18851: Perfm improvement for ZKDT management

2023-09-03 Thread via GitHub
Hexiaoqiao commented on code in PR #6001: URL: https://github.com/apache/hadoop/pull/6001#discussion_r1314280986 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java: ## @@ -520,24 +525,32 @@ protect

[GitHub] [hadoop] Hexiaoqiao commented on pull request #6002: HDFS-17172. Support FSNamesystemLock Parameters reconfigurable

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #6002: URL: https://github.com/apache/hadoop/pull/6002#issuecomment-1704336062 @haiyang1987 Thanks for your works. I am confused why we do not enable it by default and remove to config, any performance impact? Thanks. -- This is an automated message from the Apa

[GitHub] [hadoop] Hexiaoqiao commented on pull request #5430: HDFS-16933. A race in SerialNumberMap will cause wrong owner, group and XATTR

2023-09-03 Thread via GitHub
Hexiaoqiao commented on PR #5430: URL: https://github.com/apache/hadoop/pull/5430#issuecomment-1704333917 @ZanderXu Thanks for your works. Great catch here! Do you take benchmark if any possible performance decrease here when parallel fsimage loading? -- This is an automated message from

[GitHub] [hadoop] hadoop-yetus commented on pull request #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6016: URL: https://github.com/apache/hadoop/pull/6016#issuecomment-1704323280 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] hadoop-yetus commented on pull request #5920: YARN-6476. Advanced Federation UI based on YARN UI v2.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5920: URL: https://github.com/apache/hadoop/pull/5920#issuecomment-1704309490 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] iwasakims commented on pull request #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
iwasakims commented on PR #6017: URL: https://github.com/apache/hadoop/pull/6017#issuecomment-1704304520 I ran tests of hadoop-yarn-server-timelineservice-hbase and hadoop-yarn-server-timelineservice-hbase-tests with `-Dhbase.profile=2.0`. ``` $ hadoop-yarn-project/hadoop-yarn/had

[GitHub] [hadoop] iwasakims opened a new pull request, #6017: YARN-11558. Fix dependency convergence error on hbase2 profile.

2023-09-03 Thread via GitHub
iwasakims opened a new pull request, #6017: URL: https://github.com/apache/hadoop/pull/6017 https://issues.apache.org/jira/browse/YARN-11558 Building with `-Dhbase.profile=2.0` fails due to dependency convergence error. ``` $ mvn clean install -DskipTests -DskipShade -Dhbase

[GitHub] [hadoop] KeeProMise commented on a diff in pull request #5990: HDFS-17166. RBF: Throwing NoNamenodesAvailableException for a long time, when failover

2023-09-03 Thread via GitHub
KeeProMise commented on code in PR #5990: URL: https://github.com/apache/hadoop/pull/5990#discussion_r1313860811 ## hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/MockResolver.java: ## @@ -397,6 +397,11 @@ public List getMountPoints(St

[GitHub] [hadoop] haiyang1987 commented on a diff in pull request #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-03 Thread via GitHub
haiyang1987 commented on code in PR #6015: URL: https://github.com/apache/hadoop/pull/6015#discussion_r1314247707 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcMetrics.java: ## @@ -141,7 +141,7 @@ public static RpcMetrics create(Server ser

[GitHub] [hadoop] slfan1989 commented on a diff in pull request #6015: HADOOP-18880. Add some rpc related metrics to Metrics.md

2023-09-03 Thread via GitHub
slfan1989 commented on code in PR #6015: URL: https://github.com/apache/hadoop/pull/6015#discussion_r1314246416 ## hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/metrics/RpcMetrics.java: ## @@ -141,7 +141,7 @@ public static RpcMetrics create(Server serve

[GitHub] [hadoop] slfan1989 opened a new pull request, #6016: YARN-9048. Add znode hierarchy in Federation ZK State Store.

2023-09-03 Thread via GitHub
slfan1989 opened a new pull request, #6016: URL: https://github.com/apache/hadoop/pull/6016 ### Description of PR JIRA: YARN-9048. Add znode hierarchy in Federation ZK State Store. ### How was this patch tested? ### For code changes: - [ ] Does the titl

[GitHub] [hadoop] slfan1989 commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
slfan1989 commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1704289235 @goiri Thank you very much for your help in reviewing the code! I will continue to follow up YARN-11433 And YARN-11434. -- This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] slfan1989 merged pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
slfan1989 merged PR #5967: URL: https://github.com/apache/hadoop/pull/5967 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had

[GitHub] [hadoop] hadoop-yetus commented on pull request #5967: YARN-11435. [Router] FederationStateStoreFacade is not reinitialized with Router conf.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #5967: URL: https://github.com/apache/hadoop/pull/5967#issuecomment-1704186867 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6004: HADOOP-18874: [ABFS] Adding Server returned request id in Exception Message thrown to caller.

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6004: URL: https://github.com/apache/hadoop/pull/6004#issuecomment-1704063040 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reex

[GitHub] [hadoop] hadoop-yetus commented on pull request #6003: HADOOP-18869: [ABFS] Fixing Behavior of a File System APIs on root path

2023-09-03 Thread via GitHub
hadoop-yetus commented on PR #6003: URL: https://github.com/apache/hadoop/pull/6003#issuecomment-1704057809 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexe

[GitHub] [hadoop] slfan1989 commented on pull request #5957: YARN-6537. Running RM tests against the Router.

2023-09-03 Thread via GitHub
slfan1989 commented on PR #5957: URL: https://github.com/apache/hadoop/pull/5957#issuecomment-1704038067 @goiri Thank you very much for your help in reviewing the code! I will continue to follow up other pr. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [hadoop] slfan1989 merged pull request #5957: YARN-6537. Running RM tests against the Router.

2023-09-03 Thread via GitHub
slfan1989 merged PR #5957: URL: https://github.com/apache/hadoop/pull/5957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@had