[GitHub] [hadoop] hadoop-yetus commented on pull request #3118: [Do not commit] CI for Debian 10

hadoop-yetus commented on pull request #3118: URL: https://github.com/apache/hadoop/pull/3118#issuecomment-864362707 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] hadoop-yetus commented on pull request #2738: HDFS-15842. HDFS mover to emit metrics.

hadoop-yetus commented on pull request #2738: URL: https://github.com/apache/hadoop/pull/2738#issuecomment-864360570 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomscut edited a comment on pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

tomscut edited a comment on pull request #3119: URL: https://github.com/apache/hadoop/pull/3119#issuecomment-864350412 > Looks good to me Thanks @jojochuang for your review. Could you also help to review those PRs([PR#3120](https://github.com/apache/hadoop/pull/3120) [PR#3117](http

[GitHub] [hadoop] tomscut commented on pull request #3117: HDFS-16076. Avoid using slow DataNodes for reading by sorting locations

tomscut commented on pull request #3117: URL: https://github.com/apache/hadoop/pull/3117#issuecomment-864352035 Rebased to the latest commit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] tomscut commented on pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

tomscut commented on pull request #3119: URL: https://github.com/apache/hadoop/pull/3119#issuecomment-864350412 > Looks good to me Thanks @jojochuang for your review. Could you also help to review those PRs([PR#3120](https://github.com/apache/hadoop/pull/3120) [PR#3117](https://git

[jira] [Created] (HADOOP-17766) CI for Debian 10

Gautham Banasandra created HADOOP-17766: --- Summary: CI for Debian 10 Key: HADOOP-17766 URL: https://issues.apache.org/jira/browse/HADOOP-17766 Project: Hadoop Common Issue Type: Improvem

[GitHub] [hadoop] hadoop-yetus commented on pull request #3118: [Do not commit] CI for Debian 10

hadoop-yetus commented on pull request #3118: URL: https://github.com/apache/hadoop/pull/3118#issuecomment-864344828 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomscut commented on pull request #3120: HDFS-16079. Improve the block state change log

tomscut commented on pull request #3120: URL: https://github.com/apache/hadoop/pull/3120#issuecomment-864336913 Those failed UTs are unrelated to the change and work fine locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [hadoop] tomscut commented on pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

tomscut commented on pull request #3119: URL: https://github.com/apache/hadoop/pull/3119#issuecomment-864336517 Hi @ayushtkn , could you also help to review it? Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [hadoop] tomscut commented on pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

tomscut commented on pull request #3119: URL: https://github.com/apache/hadoop/pull/3119#issuecomment-864335737 Hi @tasanuma @jojochuang , could you please take a look at this little change? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2738: HDFS-15842. HDFS mover to emit metrics.

LeonGao91 commented on a change in pull request #2738: URL: https://github.com/apache/hadoop/pull/2738#discussion_r654722499 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/MoverMetrics.java ## @@ -0,0 +1,84 @@ +/** + * Licensed

[GitHub] [hadoop] hadoop-yetus commented on pull request #3118: [Do not commit] CI for Debian 10

hadoop-yetus commented on pull request #3118: URL: https://github.com/apache/hadoop/pull/3118#issuecomment-864322983 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] sunchao commented on pull request #3005: HDFS-13522. RBF: Support observer node from Router-Based Federation

sunchao commented on pull request #3005: URL: https://github.com/apache/hadoop/pull/3005#issuecomment-864302452 @zhengzhuobinzzb to help reviewing, could you describe the approach you're taking in this PR in the description? cc @fengnanli too. -- This is an automated message from the Apa

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3005: HDFS-13522. RBF: Support observer node from Router-Based Federation

hadoop-yetus removed a comment on pull request #3005: URL: https://github.com/apache/hadoop/pull/3005#issuecomment-840106802 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3005: HDFS-13522. RBF: Support observer node from Router-Based Federation

hadoop-yetus removed a comment on pull request #3005: URL: https://github.com/apache/hadoop/pull/3005#issuecomment-840089955 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:|

[GitHub] [hadoop] Jing9 commented on a change in pull request #2738: HDFS-15842. HDFS mover to emit metrics.

Jing9 commented on a change in pull request #2738: URL: https://github.com/apache/hadoop/pull/2738#discussion_r654685245 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/mover/MoverMetrics.java ## @@ -0,0 +1,84 @@ +/** + * Licensed to t

[GitHub] [hadoop] ferhui merged pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

ferhui merged pull request #3114: URL: https://github.com/apache/hadoop/pull/3114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [hadoop] virajjasani edited a comment on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

virajjasani edited a comment on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-864127746 > @virajjasani Thanks. > It seems that the following source files have the same problem. > > > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapredu

[GitHub] [hadoop] ferhui commented on pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

ferhui commented on pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#issuecomment-863660083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries a

[GitHub] [hadoop] ferhui commented on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

ferhui commented on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-864111057 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries a

[GitHub] [hadoop] tomscut commented on pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

tomscut commented on pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#issuecomment-863816478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [hadoop] goiri commented on a change in pull request #3100: HDFS-16065. RBF: Add metrics to record Router's operations

goiri commented on a change in pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#discussion_r653838543 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientMetrics.java ## @@ -0,0 +1,646 @

[GitHub] [hadoop] virajjasani opened a new pull request #3121: HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result

virajjasani opened a new pull request #3121: URL: https://github.com/apache/hadoop/pull/3121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [hadoop] AlphaGouGe commented on pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

AlphaGouGe commented on pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#issuecomment-863810415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queri

[GitHub] [hadoop] hadoop-yetus commented on pull request #3005: HDFS-13522. RBF: Support observer node from Router-Based Federation

hadoop-yetus commented on pull request #3005: URL: https://github.com/apache/hadoop/pull/3005#issuecomment-864078875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

hadoop-yetus commented on pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#issuecomment-863779456 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[GitHub] [hadoop] hadoop-yetus commented on pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

hadoop-yetus commented on pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#issuecomment-863817574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[GitHub] [hadoop] tomscut opened a new pull request #3120: HDFS-16079. Improve the block state change log

tomscut opened a new pull request #3120: URL: https://github.com/apache/hadoop/pull/3120 JIRA: [HDFS-16079](https://issues.apache.org/jira/browse/HDFS-16079) Improve the block state change log. Add readOnlyReplicas and replicasOnStaleNodes. -- This is an automated mes

[GitHub] [hadoop] ferhui commented on a change in pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

ferhui commented on a change in pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#discussion_r654226534 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java ## @@ -4120,6 +4077,12 @@ priva

[GitHub] [hadoop] hadoop-yetus commented on pull request #3120: HDFS-16079. Improve the block state change log

hadoop-yetus commented on pull request #3120: URL: https://github.com/apache/hadoop/pull/3120#issuecomment-864262029 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] whbing commented on pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

whbing commented on pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#issuecomment-863810193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries a

[jira] [Work logged] (HADOOP-16528) Update document for web authentication kerberos principal configuration

[ https://issues.apache.org/jira/browse/HADOOP-16528?focusedWorklogId=612005&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-612005 ] ASF GitHub Bot logged work on HADOOP-16528: --- Author: ASF GitHub B

[jira] [Updated] (HADOOP-16528) Update document for web authentication kerberos principal configuration

[ https://issues.apache.org/jira/browse/HADOOP-16528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-16528: Labels: pull-request-available (was: ) > Update document for web authentication kerberos

[GitHub] [hadoop] hadoop-yetus commented on pull request #1953: HADOOP-16528. Update document for web authentication kerberos principal configuration.

hadoop-yetus commented on pull request #1953: URL: https://github.com/apache/hadoop/pull/1953#issuecomment-864064259 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] LeonGao91 commented on a change in pull request #2639: HDFS-15785. Datanode to support using DNS to resolve nameservices to IP addresses to get list of namenodes.

LeonGao91 commented on a change in pull request #2639: URL: https://github.com/apache/hadoop/pull/2639#discussion_r654073694 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java ## @@ -647,6 +634,58 @@ public static String addKeySuffi

[GitHub] [hadoop] virajjasani commented on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

virajjasani commented on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-863768777 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For quer

[GitHub] [hadoop] ferhui merged pull request #3113: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

ferhui merged pull request #3113: URL: https://github.com/apache/hadoop/pull/3113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [hadoop] whbing commented on a change in pull request #3114: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

whbing commented on a change in pull request #3114: URL: https://github.com/apache/hadoop/pull/3114#discussion_r654193188 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java ## @@ -4120,6 +4077,12 @@ priva

[GitHub] [hadoop] ferhui commented on pull request #3113: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

ferhui commented on pull request #3113: URL: https://github.com/apache/hadoop/pull/3113#issuecomment-863797134 @AlphaGouGe Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [hadoop] tasanuma commented on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

tasanuma commented on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-863700707 It makes sense to me. A finalized empty array is immutable. @virajjasani Could you fix the new checkstyle warning? -- This is an automated message from the Apache Git S

[GitHub] [hadoop] AlphaGouGe commented on pull request #3113: HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

AlphaGouGe commented on pull request #3113: URL: https://github.com/apache/hadoop/pull/3113#issuecomment-863796508 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [hadoop] symious commented on a change in pull request #3100: HDFS-16065. RBF: Add metrics to record Router's operations

symious commented on a change in pull request #3100: URL: https://github.com/apache/hadoop/pull/3100#discussion_r654378797 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterClientMetrics.java ## @@ -0,0 +1,646

[GitHub] [hadoop] hadoop-yetus commented on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

hadoop-yetus commented on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-863434460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[GitHub] [hadoop] ayushtkn commented on a change in pull request #3121: HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result

ayushtkn commented on a change in pull request #3121: URL: https://github.com/apache/hadoop/pull/3121#discussion_r654568584 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java ## @@ -1129,25 +1129

[GitHub] [hadoop] ferhui edited a comment on pull request #3115: HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects

ferhui edited a comment on pull request #3115: URL: https://github.com/apache/hadoop/pull/3115#issuecomment-864141691 @virajjasani Thanks, we don't need to replace anything with ResourceEstimatorUtil and CachedBlock, I just grep EMPTY_ARRAY in source files. > Is it good to track Tas

[GitHub] [hadoop] bogthe commented on a change in pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt

bogthe commented on a change in pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#discussion_r653931489 ## File path: hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java ## @@ -396,6 +396,41 @@ private void incrementBytesRead(lo

[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt

[ https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=611897&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611897 ] ASF GitHub Bot logged work on HADOOP-17764: --- Author: ASF GitHub B

[GitHub] [hadoop] hadoop-yetus commented on pull request #3118: [Do not commit] CI for Debian 10

hadoop-yetus commented on pull request #3118: URL: https://github.com/apache/hadoop/pull/3118#issuecomment-863450554 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[GitHub] [hadoop] hadoop-yetus commented on pull request #3117: HDFS-16076. Avoid using slow DataNodes for reading by sorting locations

hadoop-yetus commented on pull request #3117: URL: https://github.com/apache/hadoop/pull/3117#issuecomment-863612534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For que

[GitHub] [hadoop] tomscut opened a new pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

tomscut opened a new pull request #3119: URL: https://github.com/apache/hadoop/pull/3119 JIRA: [HDFS-16078](https://issues.apache.org/jira/browse/HDFS-16078) Remove unused parameters (blockPoolId, maxTransfers) for DatanodeManager.handleLifeline(). -- This is an automa

[GitHub] [hadoop] hadoop-yetus commented on pull request #3121: HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result

hadoop-yetus commented on pull request #3121: URL: https://github.com/apache/hadoop/pull/3121#issuecomment-864214409 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] GauthamBanasandra opened a new pull request #3118: [Do not commit] CI for Debian 10

GauthamBanasandra opened a new pull request #3118: URL: https://github.com/apache/hadoop/pull/3118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [hadoop] virajjasani commented on a change in pull request #3121: HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result

virajjasani commented on a change in pull request #3121: URL: https://github.com/apache/hadoop/pull/3121#discussion_r654572527 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterRpcClient.java ## @@ -1129,25 +1

[GitHub] [hadoop] hadoop-yetus commented on pull request #3119: HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeli…

hadoop-yetus commented on pull request #3119: URL: https://github.com/apache/hadoop/pull/3119#issuecomment-864176054 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[jira] [Resolved] (HADOOP-17114) Replace Guava initialization of Lists.newArrayList

[ https://issues.apache.org/jira/browse/HADOOP-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HADOOP-17114. --- Resolution: Duplicate > Replace Guava initialization of Lists.newArrayList > --

[jira] [Assigned] (HADOOP-17114) Replace Guava initialization of Lists.newArrayList

[ https://issues.apache.org/jira/browse/HADOOP-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani reassigned HADOOP-17114: - Assignee: (was: Viraj Jasani) > Replace Guava initialization of Lists.newArrayList

[jira] [Commented] (HADOOP-17114) Replace Guava initialization of Lists.newArrayList

[ https://issues.apache.org/jira/browse/HADOOP-17114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365612#comment-17365612 ] Viraj Jasani commented on HADOOP-17114: --- With HADOOP-17152 and it's sub-tasks res

[jira] [Commented] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout

[ https://issues.apache.org/jira/browse/HADOOP-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17365311#comment-17365311 ] Xuesen Liang commented on HADOOP-17749: --- ping [~omalley] , could you kindly revie