[jira] [Commented] (HDFS-15157) Fix compilation failure for branch-3.1
[
https://issues.apache.org/jira/browse/HDFS-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032844#comment-17032844
]
Hadoop QA commented on HDFS-15157:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 18m
42s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.1 Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 9m
25s{color} | {color:red} root in branch-3.1 failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
55s{color} | {color:red} hadoop-hdfs in branch-3.1 failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} branch-3.1 passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
56s{color} | {color:red} hadoop-hdfs in branch-3.1 failed. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 4m
44s{color} | {color:red} branch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
28s{color} | {color:red} hadoop-hdfs in branch-3.1 failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} branch-3.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new +
392 unchanged - 2 fixed = 392 total (was 394) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 52s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 18s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}153m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.server.diskbalancer.TestDiskBalancer |
| | hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA |
| | hadoop.hdfs.TestRollingUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:70a0ef5d4a6 |
| JIRA Issue | HDFS-15157 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992945/HDFS-15157-branch-3.1-01.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 609a999bcd5a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-3.1 / 5dfd1dc |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| mvninstall |
https://builds.apache.org/job/PreCommit-HDFS-Build/28753/artifact/out/branch-mvninstall-root.txt
|
| compile |
https://builds.apache.org/job/PreCommit-HDFS-Build/28753/artifact/out/branch-compile-hadoop-hdfs-project_hadoop-h
[jira] [Created] (HDFS-15158) The number of failed volumes is mismatch with Datanode metrics
Yang Yun created HDFS-15158: --- Summary: The number of failed volumes is mismatch with Datanode metrics Key: HDFS-15158 URL: https://issues.apache.org/jira/browse/HDFS-15158 Project: Hadoop HDFS Issue Type: Bug Components: datanode Reporter: Yang Yun Assignee: Yang Yun The metrics of Datanode only increment 1, even If more than one volume fails during a disk check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15158) The number of failed volumes is mismatch with Datanode metrics
[ https://issues.apache.org/jira/browse/HDFS-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15158: Attachment: HDFS-15158.patch Status: Patch Available (was: Open) > The number of failed volumes is mismatch with Datanode metrics > --- > > Key: HDFS-15158 > URL: https://issues.apache.org/jira/browse/HDFS-15158 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Yang Yun >Assignee: Yang Yun >Priority: Minor > Attachments: HDFS-15158.patch > > > The metrics of Datanode only increment 1, even If more than one volume fails > during a disk check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15158) The number of failed volumes mismatch with volumeFailures of Datanode metrics
[ https://issues.apache.org/jira/browse/HDFS-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yun updated HDFS-15158: Summary: The number of failed volumes mismatch with volumeFailures of Datanode metrics (was: The number of failed volumes is mismatch with Datanode metrics) > The number of failed volumes mismatch with volumeFailures of Datanode > metrics > --- > > Key: HDFS-15158 > URL: https://issues.apache.org/jira/browse/HDFS-15158 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Yang Yun >Assignee: Yang Yun >Priority: Minor > Attachments: HDFS-15158.patch > > > The metrics of Datanode only increment 1, even If more than one volume fails > during a disk check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15158) The number of failed volumes mismatch with volumeFailures of Datanode metrics
[ https://issues.apache.org/jira/browse/HDFS-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032853#comment-17032853 ] Ayush Saxena commented on HDFS-15158: - Thanx [~hadoop_yangyun] for the report. Please extend a UT for the issue. > The number of failed volumes mismatch with volumeFailures of Datanode > metrics > --- > > Key: HDFS-15158 > URL: https://issues.apache.org/jira/browse/HDFS-15158 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode >Reporter: Yang Yun >Assignee: Yang Yun >Priority: Minor > Attachments: HDFS-15158.patch > > > The metrics of Datanode only increment 1, even If more than one volume fails > during a disk check. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15157) Fix compilation failure for branch-3.1
[
https://issues.apache.org/jira/browse/HDFS-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032854#comment-17032854
]
Ayush Saxena commented on HDFS-15157:
-
Test Failures Unrelated
> Fix compilation failure for branch-3.1
> --
>
> Key: HDFS-15157
> URL: https://issues.apache.org/jira/browse/HDFS-15157
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.1.4
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Attachments: HDFS-15157-branch-3.1-01.patch
>
>
> {noformat}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/ayush/hadoop/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestLazyPersistReplicaRecovery.java:[102,60]
> incompatible types:
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeDescriptor cannot be
> converted to java.lang.Throwable
> [INFO] 1 error
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15158) The number of failed volumes mismatch with volumeFailures of Datanode metrics
[
https://issues.apache.org/jira/browse/HDFS-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032864#comment-17032864
]
Hadoop QA commented on HDFS-15158:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
20m 31s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 13s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 85m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15158 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992949/HDFS-15158.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 000e5375bcea 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d23317b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/28754/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28754/testReport/ |
| Max. process+thread count | 345 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/28754/console |
| Powered by | Apache Yetus 0.8.0
[jira] [Commented] (HDFS-15127) RBF: Do not allow writes when a subcluster is unavailable for HASH_ALL mount points.
[
https://issues.apache.org/jira/browse/HDFS-15127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032872#comment-17032872
]
Hadoop QA commented on HDFS-15127:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 7m 25s{color}
| {color:red} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterFaultTolerant |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15127 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992924/HDFS-15127.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 108f6f7712f7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d23317b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/28755/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28755/testReport/ |
| Max. process+thread count | 3317 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/28755/console |
| Powered by | Apache Yetus
[jira] [Commented] (HDFS-15127) RBF: Do not allow writes when a subcluster is unavailable for HASH_ALL mount points.
[
https://issues.apache.org/jira/browse/HDFS-15127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032892#comment-17032892
]
Hadoop QA commented on HDFS-15127:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
1s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 7m
33s{color} | {color:green} hadoop-hdfs-rbf in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15127 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992924/HDFS-15127.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 435278a1ea9a 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d23317b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_232 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28756/testReport/ |
| Max. process+thread count | 3212 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/28756/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> RBF: Do not allow writes when a subcluster is unavailable for HASH_ALL mount
> points.
>
[jira] [Commented] (HDFS-15157) Fix compilation failure for branch-3.1
[
https://issues.apache.org/jira/browse/HDFS-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032933#comment-17032933
]
Íñigo Goiri commented on HDFS-15157:
+1 on [^HDFS-15157-branch-3.1-01.patch].
> Fix compilation failure for branch-3.1
> --
>
> Key: HDFS-15157
> URL: https://issues.apache.org/jira/browse/HDFS-15157
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.1.4
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Attachments: HDFS-15157-branch-3.1-01.patch
>
>
> {noformat}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/ayush/hadoop/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestLazyPersistReplicaRecovery.java:[102,60]
> incompatible types:
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeDescriptor cannot be
> converted to java.lang.Throwable
> [INFO] 1 error
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15159) Prevent adding same DN multiple times in PendingReconstructionBlocks
hemanthboyina created HDFS-15159: Summary: Prevent adding same DN multiple times in PendingReconstructionBlocks Key: HDFS-15159 URL: https://issues.apache.org/jira/browse/HDFS-15159 Project: Hadoop HDFS Issue Type: Improvement Reporter: hemanthboyina Assignee: hemanthboyina -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15086) Block scheduled counter never get decremet if the block got deleted before replication.
[ https://issues.apache.org/jira/browse/HDFS-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-15086: - Attachment: HDFS-15086.003.patch > Block scheduled counter never get decremet if the block got deleted before > replication. > --- > > Key: HDFS-15086 > URL: https://issues.apache.org/jira/browse/HDFS-15086 > Project: Hadoop HDFS > Issue Type: Improvement > Components: 3.1.1 >Reporter: Surendra Singh Lilhore >Assignee: hemanthboyina >Priority: Major > Attachments: HDFS-15086.001.patch, HDFS-15086.002.patch, > HDFS-15086.003.patch > > > If the block is scheduled for replication and same file get deleted then this > type of block will be reported as a bad block from DN. > For this failed replication work scheduled block counter never get decrement. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15086) Block scheduled counter never get decremet if the block got deleted before replication.
[
https://issues.apache.org/jira/browse/HDFS-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032967#comment-17032967
]
hemanthboyina commented on HDFS-15086:
--
thanks [~surendrasingh] for the review
have updated the patch with changes , please review
{quote}One more thing, can you create new Jira for this. It is not related to
this jira.
{quote}
have created HDFS-15159 for this
> Block scheduled counter never get decremet if the block got deleted before
> replication.
> ---
>
> Key: HDFS-15086
> URL: https://issues.apache.org/jira/browse/HDFS-15086
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: 3.1.1
>Reporter: Surendra Singh Lilhore
>Assignee: hemanthboyina
>Priority: Major
> Attachments: HDFS-15086.001.patch, HDFS-15086.002.patch,
> HDFS-15086.003.patch
>
>
> If the block is scheduled for replication and same file get deleted then this
> type of block will be reported as a bad block from DN.
> For this failed replication work scheduled block counter never get decrement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15133) Use rocksdb to store NameNode inode and blockInfo
[
https://issues.apache.org/jira/browse/HDFS-15133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032990#comment-17032990
]
hemanthboyina commented on HDFS-15133:
--
thanks for the work [~maobaolong]
{quote}We propose to implement the HDFS namespace on top of a key-value (KV)
store
{quote}
IMO Storing the INode into RocksDB doesn't satisfy our problem scenario , as in
the existing HDFS code we have a tree structure of INode's in Heap which will
be put in INodeMap and used as reference
To solve our problem scenario we are proposing to have a light weight INode
which will be having variables ID and name only , so the size of light weight
INode object will be very less comparing to existing INode
After this modification , light weight INode object will be in Heap and
Complete INode object in RocksDB
I have created HDFS-15152 to implement this
> Use rocksdb to store NameNode inode and blockInfo
> -
>
> Key: HDFS-15133
> URL: https://issues.apache.org/jira/browse/HDFS-15133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.0
>Reporter: maobaolong
>Priority: Major
> Attachments: image-2020-01-28-12-30-33-015.png
>
>
> Maybe we don't need checkpoint to a fsimage file, the rocksdb checkpoint can
> achieve the same request.
> This is ozone and alluxio way to manage meta data of master node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15086) Block scheduled counter never get decremet if the block got deleted before replication.
[
https://issues.apache.org/jira/browse/HDFS-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033000#comment-17033000
]
Hadoop QA commented on HDFS-15086:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 169 unchanged - 2 fixed = 169 total (was 171) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 5m
4s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
26s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
29s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 28s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue} 0m
29s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 |
| JIRA Issue | HDFS-15086 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12992963/HDFS-15086.003.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8e17b586d414 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d23317b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_242 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-HDFS-Build/28757/artifact/out/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
|
| javadoc |
https://builds.apache.org/job/PreCommit-HDFS-Build/28757/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/28757/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/28757/testReport/ |
| Max. process+thread count | 413 (
[jira] [Commented] (HDFS-15133) Use rocksdb to store NameNode inode and blockInfo
[
https://issues.apache.org/jira/browse/HDFS-15133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033008#comment-17033008
]
Ayush Saxena commented on HDFS-15133:
-
bq. IMO Storing the INode into RocksDB doesn't satisfy our problem scenario ,
as in the existing HDFS code we have a tree structure of INode's in Heap which
will be put in INodeMap and used as reference
Are you talking about {{INodeDirectory rootDir;}} in FSDirectory? If so,
Presently it stores root as {{InodeDirectory}} we can change to just the inode
Id of the root and whenever the root is required we can fetch it using the
root's inode Id from the inode map. The rootDir is also put into InodeMap.
[~maobaolong] regarding the whole of Inodes in the DB, I have a doubt on this
if you put all of the nodes into DB, if you fetch a entry from DB and then do
modifications on it. I think we need to replace the updated Inode in the DB,
which was not done when the Inode object stayed in heap.
Each inode contains reference to parent and its children, When in heap, its all
references, if you add one child it can be chained from the top most Inode, now
to add a INode, do we need to update all the parent entries?
> Use rocksdb to store NameNode inode and blockInfo
> -
>
> Key: HDFS-15133
> URL: https://issues.apache.org/jira/browse/HDFS-15133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.0
>Reporter: maobaolong
>Priority: Major
> Attachments: image-2020-01-28-12-30-33-015.png
>
>
> Maybe we don't need checkpoint to a fsimage file, the rocksdb checkpoint can
> achieve the same request.
> This is ozone and alluxio way to manage meta data of master node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15133) Use rocksdb to store NameNode inode and blockInfo
[
https://issues.apache.org/jira/browse/HDFS-15133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033022#comment-17033022
]
hemanthboyina commented on HDFS-15133:
--
{quote}as in the existing HDFS code we have a tree structure of INode's in Heap
which will be put in INodeMap and used as reference
{quote}
Each inode contains reference to parent and its children. So putting the INode
into DB doesn't solve our issue , as inode contains its children inodes as a
referrence ,if you add a child its parent needs to be updated . this doesn't
happen if we put inode in rocksdb
> Use rocksdb to store NameNode inode and blockInfo
> -
>
> Key: HDFS-15133
> URL: https://issues.apache.org/jira/browse/HDFS-15133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.0
>Reporter: maobaolong
>Priority: Major
> Attachments: image-2020-01-28-12-30-33-015.png
>
>
> Maybe we don't need checkpoint to a fsimage file, the rocksdb checkpoint can
> achieve the same request.
> This is ozone and alluxio way to manage meta data of master node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15157) Fix compilation failure for branch-3.1
[
https://issues.apache.org/jira/browse/HDFS-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033107#comment-17033107
]
Ayush Saxena commented on HDFS-15157:
-
Committed to branch-3.1
Thanx [~elgoiri] for the review!!!
> Fix compilation failure for branch-3.1
> --
>
> Key: HDFS-15157
> URL: https://issues.apache.org/jira/browse/HDFS-15157
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.1.4
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Attachments: HDFS-15157-branch-3.1-01.patch
>
>
> {noformat}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/ayush/hadoop/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestLazyPersistReplicaRecovery.java:[102,60]
> incompatible types:
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeDescriptor cannot be
> converted to java.lang.Throwable
> [INFO] 1 error
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15157) Fix compilation failure for branch-3.1
[
https://issues.apache.org/jira/browse/HDFS-15157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated HDFS-15157:
Fix Version/s: 3.1.4
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Fix compilation failure for branch-3.1
> --
>
> Key: HDFS-15157
> URL: https://issues.apache.org/jira/browse/HDFS-15157
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 3.1.4
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Fix For: 3.1.4
>
> Attachments: HDFS-15157-branch-3.1-01.patch
>
>
> {noformat}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR]
> /home/ayush/hadoop/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestLazyPersistReplicaRecovery.java:[102,60]
> incompatible types:
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeDescriptor cannot be
> converted to java.lang.Throwable
> [INFO] 1 error
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
