[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=752022=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-752022 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 03/Apr/22 02:28 Start Date: 03/Apr/22 02:28 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1086762980 Thanks @ayushtkn . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 752022) Time Spent: 7h 50m (was: 7h 40m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751985 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 02/Apr/22 20:06 Start Date: 02/Apr/22 20:06 Worklog Time Spent: 10m Work Description: ayushtkn merged pull request #4009: URL: https://github.com/apache/hadoop/pull/4009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751985) Time Spent: 7h 40m (was: 7.5h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 7h 40m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751984=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751984 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 02/Apr/22 20:04 Start Date: 02/Apr/22 20:04 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1086713413 Anyway it says `` generated 2 new + 321 unchanged - 2 fixed = 323 total (was 323) `` 2 new and 2 fixed. some issue with the yetus. The change doesn't touch mapreduce anyway. Proceeding with the commit considering this as a false alarm. If by chance it isn't will revert -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751984) Time Spent: 7.5h (was: 7h 20m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 7.5h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751900=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751900 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 02/Apr/22 07:46 Start Date: 02/Apr/22 07:46 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1086579086 Hi @ayushtkn , I add timeout for the failed unit test in this PR #4129. The cc warning is still here and I think it is not related to this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751900) Time Spent: 7h 20m (was: 7h 10m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 7h 20m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751777=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751777
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 01/Apr/22 20:00
Start Date: 01/Apr/22 20:00
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1086274323
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 16s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 20s | | trunk passed |
| +1 :green_heart: | compile | 24m 41s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 10s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 4s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 26s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 54s | | the patch passed |
| +1 :green_heart: | compile | 24m 0s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 24m 0s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/11/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 36 new + 287 unchanged - 36
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 24m 0s | | the patch passed |
| +1 :green_heart: | compile | 21m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 21m 7s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/11/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 321
unchanged - 2 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 21m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 49s | | root: The patch generated
0 new + 237 unchanged - 1 fixed = 237 total (was 238) |
| +1 :green_heart: | mvnsite | 4m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 2s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 50s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 333m 49s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 35m 53s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
|
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751412=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751412 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 01/Apr/22 06:43 Start Date: 01/Apr/22 06:43 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1085494617 Hi @aajisaka , do you know the cause of these cc warnings? This PR does not change the c++ code associated with mapreduce-client. Could you please take a look? Thank you very much. [/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/10/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751412) Time Spent: 7h (was: 6h 50m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 7h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751346 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 01/Apr/22 01:58 Start Date: 01/Apr/22 01:58 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1085316647 Hi @ayushtkn , the jenkins log seems compiled failed. But I can compiled successfully locally. ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hadoop-hdfs-rbf: Compilation failure [ERROR] /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-4009/ubuntu-focal/src/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/NamenodeBeanMetrics.java:[877,2] error: method does not override or implement a method from a supertype ``` And the cc warings seem related to mapreduce-client, but not the changes. Do we need to retrigger the build? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751346) Time Spent: 6h 50m (was: 6h 40m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 6h 50m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751340=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751340
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 01/Apr/22 01:42
Start Date: 01/Apr/22 01:42
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1085304848
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 14s | | trunk passed |
| +1 :green_heart: | compile | 24m 43s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 6s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 9s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 55s | | the patch passed |
| +1 :green_heart: | compile | 24m 6s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 24m 6s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/10/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 27 new + 296 unchanged - 27
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 24m 6s | | the patch passed |
| +1 :green_heart: | compile | 21m 7s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 21m 7s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/10/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 24 new + 299
unchanged - 24 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 21m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 54s | | root: The patch generated
0 new + 237 unchanged - 1 fixed = 237 total (was 238) |
| +1 :green_heart: | mvnsite | 4m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 3s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 33s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 361m 52s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 1m 17s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
|
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=751032=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-751032 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 15:22 Start Date: 31/Mar/22 15:22 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1084741403 The cc warnings seem related to `mapreduce-client`. I'll rebase and retrigger the build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 751032) Time Spent: 6.5h (was: 6h 20m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 6.5h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750987=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750987
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 31/Mar/22 13:06
Start Date: 31/Mar/22 13:06
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1084552554
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 35m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 11s | | trunk passed |
| +1 :green_heart: | compile | 21m 0s | | trunk passed |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 34s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 4s | | trunk passed |
| +1 :green_heart: | spotbugs | 7m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 52s | | the patch passed |
| +1 :green_heart: | compile | 21m 21s | | the patch passed |
| -1 :x: | cc | 21m 21s |
[/results-compile-cc-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/9/artifact/out/results-compile-cc-root.txt)
| root generated 10 new + 195 unchanged - 10 fixed = 205 total (was 205) |
| +1 :green_heart: | javac | 21m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 47s | | root: The patch generated
0 new + 238 unchanged - 1 fixed = 238 total (was 239) |
| +1 :green_heart: | mvnsite | 4m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 51s | | the patch passed |
| +1 :green_heart: | spotbugs | 8m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 18m 3s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/9/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | unit | 230m 25s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 21m 47s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 475m 9s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec |
| | hadoop.crypto.TestCryptoCodec |
| | hadoop.service.launcher.TestServiceInterruptHandling |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4009 |
| Optional Tests | dupname asflicense mvnsite codespell markdownlint compile
javac javadoc mvninstall unit shadedclient spotbugs checkstyle cc buflint
bufcompat |
| uname | Linux 5b6a171da275 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 075b0cab50eec18cbcff666a5cd030f19507602f |
| Default Java | Red Hat, Inc.-1.8.0_322-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/9/testReport/ |
| Max. process+thread count | 3678 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750768=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750768 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 06:02 Start Date: 31/Mar/22 06:02 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1084117125 > I have retriggered the build. Lets wait. If all clean will push this post that Thanks @ayushtkn for your help. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750768) Time Spent: 6h 10m (was: 6h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 6h 10m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750762=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750762 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 05:38 Start Date: 31/Mar/22 05:38 Worklog Time Spent: 10m Work Description: ayushtkn commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1084096825 I have retriggered the build. Lets wait. If all clean will push this post that -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750762) Time Spent: 6h (was: 5h 50m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 6h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750663=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750663 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 01:08 Start Date: 31/Mar/22 01:08 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1083908208 The cc warnings seem unrelated to the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750663) Time Spent: 5h 40m (was: 5.5h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 5h 40m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750664 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 01:08 Start Date: 31/Mar/22 01:08 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1083911072 > Some minor stuff with the test else the prod changes LGTM. Give a check to the Jenkins warning. The cc warnings if they aren't related. > > TestRouterDistCpProcedure seems to be failing with timeout. Doesn't look related. If you have bandwidth can track & increase the timeout for it in a separate jira Hi @ayushtkn , the cc warnings seem unrelated to the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750664) Time Spent: 5h 50m (was: 5h 40m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 5h 50m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750661=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750661 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 31/Mar/22 01:06 Start Date: 31/Mar/22 01:06 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1083908208 The cc warnings seem unrelated to the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 750661) Time Spent: 5.5h (was: 5h 20m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 5.5h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=750156=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-750156
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 30/Mar/22 14:27
Start Date: 30/Mar/22 14:27
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1083209351
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 12s | | trunk passed |
| +1 :green_heart: | compile | 22m 55s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 20m 8s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 30s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 48s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 25s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 47s | | the patch passed |
| +1 :green_heart: | compile | 22m 9s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 22m 9s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/8/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 33 new + 290 unchanged - 33
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 22m 9s | | the patch passed |
| +1 :green_heart: | compile | 19m 59s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 19m 59s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/8/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 35 new + 288
unchanged - 35 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 19m 59s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 38s | | root: The patch generated
0 new + 238 unchanged - 1 fixed = 238 total (was 239) |
| +1 :green_heart: | mvnsite | 4m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 22s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 4s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 38s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 50s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 231m 44s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 21m 56s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 492m 21s | | |
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=749862=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749862 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 30/Mar/22 06:16 Start Date: 30/Mar/22 06:16 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1082664700 > TestRouterDistCpProcedure seems to be failing with timeout. Doesn't look related. If you have bandwidth can track & increase the timeout for it in a separate jira Thank you very much, I will open a new JIRA to do this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 749862) Time Spent: 5h 10m (was: 5h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 5h 10m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=749859=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749859
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 30/Mar/22 06:12
Start Date: 30/Mar/22 06:12
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r838152418
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -432,7 +443,6 @@ public void testTraverseWhenParentDeleted() throws
Exception {
public void testTraverseWhenRootParentDeleted() throws Exception {
}
-
Review comment:
OK. Let me roll back it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749859)
Time Spent: 5h (was: 4h 50m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=749858=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749858
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 30/Mar/22 06:10
Start Date: 30/Mar/22 06:10
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r838151716
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -482,6 +492,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
Review comment:
Thanks for your comments. I'm sorry I left that out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749858)
Time Spent: 4h 50m (was: 4h 40m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=749853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-749853
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 30/Mar/22 05:55
Start Date: 30/Mar/22 05:55
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r838142566
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -482,6 +492,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
Review comment:
it is still there, you added a new test as well to get rid of this only
right?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -432,7 +443,6 @@ public void testTraverseWhenParentDeleted() throws
Exception {
public void testTraverseWhenRootParentDeleted() throws Exception {
}
-
Review comment:
nit:
avoid this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 749853)
Time Spent: 4h 40m (was: 4.5h)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748375 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 28/Mar/22 00:05 Start Date: 28/Mar/22 00:05 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1080050362 Hi @ayushtkn , the failed unit test is unrelated to the change. Please have a look again. Thank you very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 748375) Time Spent: 4.5h (was: 4h 20m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748355=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748355
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 27/Mar/22 17:58
Start Date: 27/Mar/22 17:58
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1079984552
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 33s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 29s | | trunk passed |
| +1 :green_heart: | compile | 24m 44s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 14s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 6s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 56s | | the patch passed |
| +1 :green_heart: | compile | 24m 7s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 24m 7s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/7/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 32 new + 291 unchanged - 32
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 24m 7s | | the patch passed |
| +1 :green_heart: | compile | 21m 12s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 21m 12s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/7/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 16 new + 307
unchanged - 16 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 21m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 52s | | root: The patch generated
0 new + 237 unchanged - 1 fixed = 237 total (was 238) |
| +1 :green_heart: | mvnsite | 4m 4s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 3s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 22s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 54s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 333m 49s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 32m 32s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
|
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748308=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748308
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 27/Mar/22 07:36
Start Date: 27/Mar/22 07:36
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835865303
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
> ```
> // Wait till namenode notified about the block location details
> DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3,
35000,
> dfs);
> ```
>
> Here you are waiting for SPS to process the path and move the blocks to
the correct place, once this is done, whether `getPendingSPSPaths` will still
return 1? I suppose no, right? the path got processed so the count should
reduce to 0.
>
> So, my take is you don't have a control on `
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);`, if by chance SPS process that path before your assertion then the test
will fail.
>
> I haven't gone through the code, but that is what I felt in my initial
pass, if it doesn't work this way do lemme know
Thank you @ayushtkn for your detailed explanation. I made a mistake and you
are right. I'll add a new unit test to assert metrics without running SPS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 748308)
Time Spent: 4h 10m (was: 4h)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748249=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748249
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 26/Mar/22 18:55
Start Date: 26/Mar/22 18:55
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835797579
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
```
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
```
Here you are waiting for SPS to process the path and move the blocks to the
correct place, once this is done, whether ``getPendingSPSPaths`` will still
return 1? I suppose no, right? the path got processed so the count should
reduce to 0.
So, my take is you don't have a control on ``
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);``, if by chance SPS process that path before your assertion
then the test will fail.
I haven't gone through the code, but that is what I felt in my initial
pass, if it doesn't work this way do lemme know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 748249)
Time Spent: 4h (was: 3h 50m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748187=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748187
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 26/Mar/22 10:33
Start Date: 26/Mar/22 10:33
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835748939
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
> Is there a race condition possible, if the storage policy gets
satisfied before we get the pending sps paths? in that case the assertion shall
fail I suppose?
Thanks for your review. The method doTestWhenStoragePolicySetToCOLD is only
called in one place. I think there should be no race condition? What do you
think of this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 748187)
Time Spent: 3h 50m (was: 3h 40m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748186=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748186
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 26/Mar/22 10:33
Start Date: 26/Mar/22 10:33
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835748863
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
Thanks for your review. The method `doTestWhenStoragePolicySetToCOLD` is
only called in one place. I think there should be no race condition? What do
you think of this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 748186)
Time Spent: 3h 40m (was: 3.5h)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748182=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748182 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 26/Mar/22 10:02 Start Date: 26/Mar/22 10:02 Worklog Time Spent: 10m Work Description: tomscut commented on a change in pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#discussion_r835746197 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/NamenodeStatusReport.java ## @@ -382,7 +384,8 @@ public void setNamesystemInfo(long available, long total, this.numOfBlocksPendingDeletion = numBlocksPendingDeletion; this.numOfFiles = numFiles; this.statsValid = true; -this.providedSpace = providedSpace; +this.providedSpace = providedStorageSpace; Review comment: This is to fix [checkstyles](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/5/artifact/out/results-checkstyle-root.txt). I don't know if I need to update the checkstyles of the old code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 748182) Time Spent: 3.5h (was: 3h 20m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=748177=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-748177
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 26/Mar/22 09:48
Start Date: 26/Mar/22 09:48
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r835744910
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/sps/TestExternalStoragePolicySatisfier.java
##
@@ -441,6 +442,8 @@ private void doTestWhenStoragePolicySetToCOLD() throws
Exception {
hdfsCluster.triggerHeartbeats();
dfs.satisfyStoragePolicy(new Path(FILE));
+// Assert metrics.
+assertEquals(1, hdfsCluster.getNamesystem().getPendingSPSPaths());
// Wait till namenode notified about the block location details
DFSTestUtil.waitExpectedStorageType(FILE, StorageType.ARCHIVE, 3, 35000,
dfs);
Review comment:
Is there a race condition possible, if the storage policy gets satisfied
before we get the pending sps paths? in that case the assertion shall fail I
suppose?
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/resolver/NamenodeStatusReport.java
##
@@ -382,7 +384,8 @@ public void setNamesystemInfo(long available, long total,
this.numOfBlocksPendingDeletion = numBlocksPendingDeletion;
this.numOfFiles = numFiles;
this.statsValid = true;
-this.providedSpace = providedSpace;
+this.providedSpace = providedStorageSpace;
Review comment:
if this is just formatting change, please avoid this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 748177)
Time Spent: 3h 20m (was: 3h 10m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=747199=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-747199 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 24/Mar/22 12:50 Start Date: 24/Mar/22 12:50 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1073427612 Hi @ayushtkn @Hexiaoqiao @ferhui , could you please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 747199) Time Spent: 3h 10m (was: 3h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=744809=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744809 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 21/Mar/22 02:37 Start Date: 21/Mar/22 02:37 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1073427612 Hi @ayushtkn @Hexiaoqiao @ferhui , could you please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744809) Time Spent: 3h (was: 2h 50m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=744808=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744808 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 21/Mar/22 02:36 Start Date: 21/Mar/22 02:36 Worklog Time Spent: 10m Work Description: tomscut removed a comment on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1061337598 Hi @ayushtkn , the failed unit tests seem unrelated to the change. Could you please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744808) Time Spent: 2h 50m (was: 2h 40m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=741517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-741517
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 15/Mar/22 09:56
Start Date: 15/Mar/22 09:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-106892
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 35s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 9s | | trunk passed |
| +1 :green_heart: | compile | 22m 44s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 56s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 23s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 50s | | the patch passed |
| +1 :green_heart: | compile | 21m 54s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 21m 54s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/6/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 35 new + 288 unchanged - 35
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 21m 54s | | the patch passed |
| +1 :green_heart: | compile | 19m 54s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 19m 54s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/6/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 31 new + 292
unchanged - 31 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 19m 54s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 36s | | root: The patch generated
0 new + 238 unchanged - 1 fixed = 238 total (was 239) |
| +1 :green_heart: | mvnsite | 4m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 20s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 47s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 232m 24s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 21m 18s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 491m 40s | | |
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=741387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-741387
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 15/Mar/22 01:29
Start Date: 15/Mar/22 01:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1067468033
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 4s | | trunk passed |
| +1 :green_heart: | compile | 24m 20s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 20m 59s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 6s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 24s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 53s | | the patch passed |
| +1 :green_heart: | compile | 23m 54s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 23m 54s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/5/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 20 new + 303 unchanged - 20
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 23m 54s | | the patch passed |
| +1 :green_heart: | compile | 20m 57s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 20m 57s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/5/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 35 new + 288
unchanged - 35 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 20m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 47s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/5/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 237 unchanged - 1 fixed = 239 total (was
238) |
| +1 :green_heart: | mvnsite | 4m 3s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 3s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 45s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 397m 2s |
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=738031=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-738031 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 08/Mar/22 08:57 Start Date: 08/Mar/22 08:57 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1061548454 Hi @tasanuma @virajjasani , could you please also take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 738031) Time Spent: 2h 20m (was: 2h 10m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=737905=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-737905 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 08/Mar/22 02:19 Start Date: 08/Mar/22 02:19 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1061337598 Hi @ayushtkn , the failed unit tests seem unrelated to the change. Could you please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 737905) Time Spent: 2h 10m (was: 2h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=737899=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-737899
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 08/Mar/22 01:46
Start Date: 08/Mar/22 01:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1061321311
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 17m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 45s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 43s | | trunk passed |
| +1 :green_heart: | compile | 24m 8s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 20m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 54s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 0s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 3s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 29s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 54s | | the patch passed |
| +1 :green_heart: | compile | 23m 38s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | cc | 23m 38s |
[/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/4/artifact/out/results-compile-cc-root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 32 new + 291 unchanged - 32
fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 23m 38s | | the patch passed |
| +1 :green_heart: | compile | 20m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | cc | 20m 49s |
[/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/4/artifact/out/results-compile-cc-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 22 new + 301
unchanged - 22 fixed = 323 total (was 323) |
| +1 :green_heart: | javac | 20m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/4/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 237 unchanged - 1 fixed = 239 total (was
238) |
| +1 :green_heart: | mvnsite | 4m 1s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 0s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 4m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 45s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 330m 21s |
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=737522=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-737522
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 07/Mar/22 14:08
Start Date: 07/Mar/22 14:08
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r820737511
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/sps/StoragePolicySatisfyManager.java
##
@@ -292,4 +292,13 @@ public boolean isEnabled() {
public StoragePolicySatisfierMode getMode() {
return mode;
}
+
+ /**
+ * @return the number of paths to be processed by storage policy satisfier.
+ */
+ public int getPendingSPSPaths() {
+synchronized (pathsToBeTraversed) {
+ return pathsToBeTraversed.size();
+}
Review comment:
Thanks for the comment. As metric, synchronised can be omitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 737522)
Time Spent: 1h 50m (was: 1h 40m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=737332=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-737332 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 07/Mar/22 02:24 Start Date: 07/Mar/22 02:24 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1060124641 > We need to have this metric is Router as well, as we do have for other FSN related metrics. The value there would be the sum of values from all other namespaces. Thank you @ayushtkn very much for the review. I will update it ASAP. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 737332) Time Spent: 1h 40m (was: 1.5h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=736786=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736786
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 04/Mar/22 16:59
Start Date: 04/Mar/22 16:59
Worklog Time Spent: 10m
Work Description: ayushtkn commented on a change in pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#discussion_r819740564
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/sps/StoragePolicySatisfyManager.java
##
@@ -292,4 +292,13 @@ public boolean isEnabled() {
public StoragePolicySatisfierMode getMode() {
return mode;
}
+
+ /**
+ * @return the number of paths to be processed by storage policy satisfier.
+ */
+ public int getPendingSPSPaths() {
+synchronized (pathsToBeTraversed) {
+ return pathsToBeTraversed.size();
+}
Review comment:
This is synchronised and will be called a bunch of times. What impact is
it going to have on normal SPS performance
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
##
@@ -5524,6 +5524,11 @@ public int getBlockCapacity() {
return blockManager.getCapacity();
}
+ @Metric({"PendingSPSPaths", "The number of paths to be processed by storage
policy satisfier"})
+ public int getPendingSPSPaths() {
Review comment:
Do you not plan to add this method to ``FSNamesystemMBean``?
if no specific reason, we should add it there, this is quite specific to FSN
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 736786)
Time Spent: 1.5h (was: 1h 20m)
> [SPS]: Add metric PendingSPSPaths for getting the number of paths to be
> processed by SPS
>
>
> Key: HDFS-16477
> URL: https://issues.apache.org/jira/browse/HDFS-16477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: tomscut
>Assignee: tomscut
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Currently we have no idea how many paths are waiting to be processed when
> using the SPS feature. We should add metric PendingSPSPaths for getting the
> number of paths to be processed by SPS in NameNode.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=736546=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736546 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 04/Mar/22 09:37 Start Date: 04/Mar/22 09:37 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1058998161 Unit tests failed because of OOM, regardless of changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 736546) Time Spent: 1h 20m (was: 1h 10m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=736544=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736544
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 04/Mar/22 09:28
Start Date: 04/Mar/22 09:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1058991178
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 14s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 11m 28s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 46s | | trunk passed |
| +1 :green_heart: | compile | 22m 25s | | trunk passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 19m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 27s | | trunk passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 26s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 23m 43s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 23m 43s | | the patch passed |
| +1 :green_heart: | compile | 23m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 23m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 25s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 17m 2s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/3/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| -1 :x: | unit | 0m 34s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| +0 :ok: | asflicense | 0m 43s | | ASF License check generated no
output? |
| | | 246m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| | hadoop.fs.viewfs.TestViewFSOverloadSchemeCentralMountTableConfig |
| | hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4009 |
| Optional Tests | dupname asflicense mvnsite codespell markdownlint compile
javac javadoc mvninstall unit shadedclient spotbugs checkstyle |
| uname | Linux 754b3ee36d57 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=736485=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736485 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 04/Mar/22 05:42 Start Date: 04/Mar/22 05:42 Worklog Time Spent: 10m Work Description: tomscut edited a comment on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1058847572 Hi @tasanuma @ayushtkn @virajith @virajjasani , could you please also take a look at this? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 736485) Time Spent: 1h (was: 50m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=736484=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736484 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 04/Mar/22 05:41 Start Date: 04/Mar/22 05:41 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1058847572 Hi @tasanuma @ayushtkn @virajith , could you please also take a look at this? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 736484) Time Spent: 50m (was: 40m) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=731431=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731431 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 23/Feb/22 08:37 Start Date: 23/Feb/22 08:37 Worklog Time Spent: 10m Work Description: tomscut commented on pull request #4009: URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1048551207 Hi @jojochuang @Hexiaoqiao , PTAL. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 731431) Time Spent: 40m (was: 0.5h) > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=731430=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731430
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 23/Feb/22 08:34
Start Date: 23/Feb/22 08:34
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1048549230
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 31s | | trunk passed |
| +1 :green_heart: | compile | 22m 24s | | trunk passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 19m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 3m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 21m 49s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 21m 49s | | the patch passed |
| +1 :green_heart: | compile | 19m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 19m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 23s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 25m 1s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 248m 42s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 9s | | The patch does not
generate ASF License warnings. |
| | | 480m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4009 |
| Optional Tests | dupname asflicense mvnsite codespell markdownlint compile
javac javadoc mvninstall unit shadedclient spotbugs checkstyle |
| uname | Linux 188ae4e762b2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f9b315a8898895e1f0841ea8d282922bae909c2b |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/2/testReport/ |
| Max. process+thread count | 3156 (vs. ulimit of
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[
https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=731047=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-731047
]
ASF GitHub Bot logged work on HDFS-16477:
-
Author: ASF GitHub Bot
Created on: 22/Feb/22 16:07
Start Date: 22/Feb/22 16:07
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #4009:
URL: https://github.com/apache/hadoop/pull/4009#issuecomment-1047953347
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 4s | | Maven dependency ordering for branch |
| -1 :x: | mvninstall | 21m 27s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/1/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| -1 :x: | compile | 16m 14s |
[/branch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/1/artifact/out/branch-compile-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt)
| root in trunk failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04. |
| -1 :x: | compile | 13m 43s |
[/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/1/artifact/out/branch-compile-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root in trunk failed with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07. |
| +1 :green_heart: | checkstyle | 3m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 20s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 28s | | trunk passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 5m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 21m 59s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 21m 59s |
[/results-compile-javac-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/1/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 188 new + 1624 unchanged - 0
fixed = 1812 total (was 1624) |
| +1 :green_heart: | compile | 19m 48s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 19m 48s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4009/1/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 172 new + 1515
unchanged - 0 fixed = 1687 total (was 1515) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 39s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 24s | | the patch passed with JDK
Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 31m 58s | |
[jira] [Work logged] (HDFS-16477) [SPS]: Add metric PendingSPSPaths for getting the number of paths to be processed by SPS
[ https://issues.apache.org/jira/browse/HDFS-16477?focusedWorklogId=730686=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-730686 ] ASF GitHub Bot logged work on HDFS-16477: - Author: ASF GitHub Bot Created on: 22/Feb/22 08:16 Start Date: 22/Feb/22 08:16 Worklog Time Spent: 10m Work Description: tomscut opened a new pull request #4009: URL: https://github.com/apache/hadoop/pull/4009 JIRA: [HDFS-16477](https://issues.apache.org/jira/browse/HDFS-16477). Currently we have no idea how many paths are waiting to be processed when using the SPS feature. We should add metric `PendingSPSPaths` for getting the number of paths to be processed by SPS in NameNode. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 730686) Remaining Estimate: 0h Time Spent: 10m > [SPS]: Add metric PendingSPSPaths for getting the number of paths to be > processed by SPS > > > Key: HDFS-16477 > URL: https://issues.apache.org/jira/browse/HDFS-16477 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: tomscut >Assignee: tomscut >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently we have no idea how many paths are waiting to be processed when > using the SPS feature. We should add metric PendingSPSPaths for getting the > number of paths to be processed by SPS in NameNode. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
