[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=666238=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666238 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 18/Oct/21 11:00 Start Date: 18/Oct/21 11:00 Worklog Time Spent: 10m Work Description: aajisaka commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-945649327 Thank you @symious for your contribution and thanks @goiri for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 666238) Time Spent: 3h (was: 2h 50m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Fix For: 2.10.2 > > Time Spent: 3h > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=666236=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666236 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 18/Oct/21 10:58 Start Date: 18/Oct/21 10:58 Worklog Time Spent: 10m Work Description: aajisaka merged pull request #3524: URL: https://github.com/apache/hadoop/pull/3524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 666236) Time Spent: 2h 50m (was: 2h 40m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=662215=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-662215 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 23:26 Start Date: 07/Oct/21 23:26 Worklog Time Spent: 10m Work Description: symious commented on a change in pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#discussion_r72379 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MountTableResolver.java ## @@ -138,6 +138,8 @@ public MountTableResolver(Configuration conf, Router routerService, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE_DEFAULT); this.locationCache = CacheBuilder.newBuilder() + // To warkaround guava bug https://github.com/google/guava/issues/1055 Review comment: Updated, please help to check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 662215) Time Spent: 2h 40m (was: 2.5h) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=662047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-662047 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 23:11 Start Date: 07/Oct/21 23:11 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937409811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 662047) Time Spent: 2.5h (was: 2h 20m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661631=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661631 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 14:07 Start Date: 07/Oct/21 14:07 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937827956 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 14m 32s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 35s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 30s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 21s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 47s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 51s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 37s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 48s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 3s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | mvnsite | 0m 29s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 0s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 45m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux 023779c2cd06 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/8/testReport/ | | Max. process+thread count | 1295 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/8/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661626=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661626 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 14:01 Start Date: 07/Oct/21 14:01 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937822584 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 14m 33s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 35s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 31s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 20s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 37s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 48s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 33s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 34s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 7s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 29s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 26s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 46m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux e47a6fe08c90 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/7/testReport/ | | Max. process+thread count | 1389 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/7/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661362=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661362 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 07:25 Start Date: 07/Oct/21 07:25 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937526316 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 14m 56s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 35s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 29s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 21s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 35s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 49s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 36s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 33s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 5s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 27s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 6s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 46m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux 2c1fa729faf1 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/6/testReport/ | | Max. process+thread count | 1434 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/6/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661348=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661348 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 06:56 Start Date: 07/Oct/21 06:56 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937508312 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 15m 28s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 34s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 29s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 22s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 36s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 48s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 33s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 38s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 9s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 30s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 13s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 22s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 47m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux f3f4eeabd861 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/5/testReport/ | | Max. process+thread count | 1395 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/5/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661297=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661297 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 04:05 Start Date: 07/Oct/21 04:05 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937425783 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 15m 25s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 38s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 29s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 20s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 37s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 50s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 33s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 36s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 9s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 29s | the patch passed | | +1 :green_heart: | compile | 0m 29s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 29s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | mvnsite | 0m 30s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 10s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 55m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux bca894c0206c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/4/testReport/ | | Max. process+thread count | 1228 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/4/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661293=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661293 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 03:22 Start Date: 07/Oct/21 03:22 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937412397 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 15m 16s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 33s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 27s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 20s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 39s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 49s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 33s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 41s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 12s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 15s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 31s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 15s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 47m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux 3d82906a0f67 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/3/testReport/ | | Max. process+thread count | 1445 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/3/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661290=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661290 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 03:15 Start Date: 07/Oct/21 03:15 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-937409811 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 15m 11s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 35s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 28s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 21s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 37s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 49s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 34s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 37s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 7s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 31s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 31s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 29s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 10s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. | | | | 56m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux 356e4376ce9f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/2/testReport/ | | Max. process+thread count | 1092 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/2/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661279=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661279 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 07/Oct/21 02:19 Start Date: 07/Oct/21 02:19 Worklog Time Spent: 10m Work Description: symious commented on a change in pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#discussion_r72379 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MountTableResolver.java ## @@ -138,6 +138,8 @@ public MountTableResolver(Configuration conf, Router routerService, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE_DEFAULT); this.locationCache = CacheBuilder.newBuilder() + // To warkaround guava bug https://github.com/google/guava/issues/1055 Review comment: Updated, please help to check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 661279) Time Spent: 1h 10m (was: 1h) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661139=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661139 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 19:09 Start Date: 06/Oct/21 19:09 Worklog Time Spent: 10m Work Description: goiri commented on a change in pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#discussion_r723600325 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MountTableResolver.java ## @@ -138,6 +138,8 @@ public MountTableResolver(Configuration conf, Router routerService, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE_DEFAULT); this.locationCache = CacheBuilder.newBuilder() + // To warkaround guava bug https://github.com/google/guava/issues/1055 Review comment: mention in what version this is fixed ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/MountTableResolver.java ## @@ -138,6 +138,8 @@ public MountTableResolver(Configuration conf, Router routerService, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE, FEDERATION_MOUNT_TABLE_MAX_CACHE_SIZE_DEFAULT); this.locationCache = CacheBuilder.newBuilder() + // To warkaround guava bug https://github.com/google/guava/issues/1055 Review comment: work around -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 661139) Time Spent: 1h (was: 50m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=661001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661001 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 15:30 Start Date: 06/Oct/21 15:30 Worklog Time Spent: 10m Work Description: symious commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-936503908 @ayushtkn Thanks for the review. Trunk uses guava from org.apache.hadoop.thirdparty.hadoop-shaded-guava:1.1.1, the guava cache in it should have fixed the bug. Tested with the above dependency, the overhead is eliminated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 661001) Time Spent: 50m (was: 40m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=660970=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660970 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 14:50 Start Date: 06/Oct/21 14:50 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-936419231 Can we have this change merged to trunk as well, and then backport it till 2.10? Any pointers, can this have some adverse affects on trunk version of Guava? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 660970) Time Spent: 40m (was: 0.5h) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=660781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660781 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 08:57 Start Date: 06/Oct/21 08:57 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-935785881 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 9m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.10 Compile Tests _ | | +1 :green_heart: | mvninstall | 14m 34s | branch-2.10 passed | | +1 :green_heart: | compile | 0m 35s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | compile | 0m 29s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | checkstyle | 0m 21s | branch-2.10 passed | | +1 :green_heart: | mvnsite | 0m 35s | branch-2.10 passed | | +1 :green_heart: | javadoc | 0m 49s | branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 36s | branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +0 :ok: | spotbugs | 3m 32s | Both FindBugs and SpotBugs are enabled, using SpotBugs. | | +1 :green_heart: | spotbugs | 1m 2s | branch-2.10 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 28s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | checkstyle | 0m 15s | the patch passed | | +1 :green_heart: | mvnsite | 0m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | +1 :green_heart: | spotbugs | 1m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 17m 3s | hadoop-hdfs-rbf in the patch passed. | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 54m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/3524 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle | | uname | Linux ebc2d0291df7 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | branch-2.10 / dc03afc | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_262-b10 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/1/testReport/ | | Max. process+thread count | 1454 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3524/1/console | | versions | git=2.7.4 maven=3.3.9 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=660756=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660756 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 08:02 Start Date: 06/Oct/21 08:02 Worklog Time Spent: 10m Work Description: symious commented on pull request #3524: URL: https://github.com/apache/hadoop/pull/3524#issuecomment-935700482 @goiri Could you help to review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 660756) Time Spent: 20m (was: 10m) > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16257) [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver
[ https://issues.apache.org/jira/browse/HDFS-16257?focusedWorklogId=660754=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-660754 ] ASF GitHub Bot logged work on HDFS-16257: - Author: ASF GitHub Bot Created on: 06/Oct/21 08:02 Start Date: 06/Oct/21 08:02 Worklog Time Spent: 10m Work Description: symious opened a new pull request #3524: URL: https://github.com/apache/hadoop/pull/3524 ### Description of PR Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the performance of cache, which was mentioned in HDFS-13821. Since upgrading guava version seems affecting too much, this ticket is to add a configuration setting when initializing cache to walk around this issue. ### How was this patch tested? Locally tested. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 660754) Remaining Estimate: 0h Time Spent: 10m > [HDFS] [RBF] Guava cache performance issue in Router MountTableResolver > --- > > Key: HDFS-16257 > URL: https://issues.apache.org/jira/browse/HDFS-16257 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 2.10.1 >Reporter: Janus Chow >Assignee: Janus Chow >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Branch 2.10.1 uses guava version of 11.0.2, which has a bug which affects the > performance of cache, which was mentioned in HDFS-13821. > Since upgrading guava version seems affecting too much, this ticket is to add > a configuration setting when initializing cache to walk around this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org