[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551264#comment-16551264 ] Wei-Chiu Chuang commented on HDFS-12886: I stand corrected: the data corruption bug caused by concurrent recoverLease() and close() is *NOT* fixed by HDFS-12886. In fact, it is accidentally fixed by HDFS-8909. Basically, HDFS-8909 added an assertion that if a block recovery command is being asked, it checks if the block is actually under recovery. So after HDFS-8909, the recoverLease() call is ignored. However, note that under a more delicate condition, concurrent recoverLease() + close() could still corrupt block. Ignoring block recovery if the block is not under recovery is not the ultimate solution. Instead, the close() should be ignored/rejected when block is being recovered. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.3 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548717#comment-16548717 ] Wei-Chiu Chuang commented on HDFS-12886: Very good work! Incidentally, it looks like this jira also fixed HDFS-10240 – close while recoverLease() could cause data corruption previously, and now it won't. Close would instead throw "AssertionError: Negative replicas!" instead of corrupt data. It is still not ideal as client code would need to handle this case, but it's better than corrupt data. I'll attach a test case at HDFS-10240, and will also think about how to handle the AssertError. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.3 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402612#comment-16402612 ] Íñigo Goiri commented on HDFS-12886: Thanks [~lukmajercak] for [^HDFS-12886_branch-2.000.patch]. As Yetus for branch-2 is messed up, no point of on waiting. The build works locally. Thanks [~ywskycn] for catching this and sorry for the inconvenience. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402603#comment-16402603 ] Lukas Majercak commented on HDFS-12886: --- Added patch for branch-2 > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402563#comment-16402563 ] Íñigo Goiri commented on HDFS-12886: branch-2 and branch-2.9 break the build. We need patches for those. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402289#comment-16402289 ] Hudson commented on HDFS-12886: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13849 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13849/]) HDFS-12886. Ignore minReplication for block recovery. Contributed by (inigoiri: rev 08ff1586d5d3e39f546200f9e696f62ea4cf000d) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402236#comment-16402236 ] Íñigo Goiri commented on HDFS-12886: Committed to trunk, branch-3.1, branch-3.0, branch-2, and branch-2.9. Thanks [~lukmajercak] for the patch and [~giovanni.fumarola] for the review. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399398#comment-16399398 ] Íñigo Goiri commented on HDFS-12886: Thanks [~giovanni.fumarola]. We had this working in production for almost a year (April 2017). I will commit this at the end of the day. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399122#comment-16399122 ] Giovanni Matteo Fumarola commented on HDFS-12886: - Thanks [~lukmajercak] for the patch. LGTM +1. If [~daryn] has no concern about the patch itself, you can commit it. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[
https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396527#comment-16396527
]
genericqa commented on HDFS-12886:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 21s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}119m 43s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12886 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12900580/HDFS-12886.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 249b4c51d5af 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 91c82c9 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23439/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23439/testReport/ |
| Max. process+thread count | 3039 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/23439/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396352#comment-16396352 ] Íñigo Goiri commented on HDFS-12886: [~daryn], did you have a chance to looks at this? I'd like to commit this soon. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16283101#comment-16283101 ] Daryn Sharp commented on HDFS-12886: Thanks for the ping, will make time to review in the next few days. Just based on cursory read, agree recovery should be able to succeed regardless of min replication. If the number of recovered/available replicas is less than desired then it’s just under replicated. You can’t wait for min replicas when there physically aren’t that many - think rack loss. Waiting only increases odds of data loss. That said, rarely is anything easy to fix as it seems with block management... > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279233#comment-16279233 ] Lukas Majercak commented on HDFS-12886: --- One problem that we should create a JIRA for is having the client get the minReplication from NN, because currently it just ignores it, and can happily write to just 1 DN. But that still would not solve this issue, and it needs to be decided whether minReplication should be ignored in this block recovery case. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279065#comment-16279065 ] Íñigo Goiri commented on HDFS-12886: Agree with [~lukmajercak], the truth is that this situation is currently allowed and we should handle it. I'm open to file a new JIRA to prevent this though. +1 on [^HDFS-12886.002.patch]. [~daryn], any thoughts on this? > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279055#comment-16279055 ] Lukas Majercak commented on HDFS-12886: --- [~goiri], the checkstyle warnings are gone and the failing test seem to be unrelated. For your question, I'm divided as well, but I lean more towards ignoring minReplication for block recovery to allow the block to be sealed and replicated, mostly because of cases such as the one that the test simulates. Thoughts? [~chris.douglas]? > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277942#comment-16277942 ] Íñigo Goiri commented on HDFS-12886: Let's see what [^HDFS-12886.002.patch] looks like but that should fine. Then there is a more philosophical question regarding minReplication, should this situation we are fixing even be allowed? This is the correct fix but not sure we should even be getting to this situation. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[
https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277792#comment-16277792
]
genericqa commented on HDFS-12886:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 52s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 37s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 2 new + 168 unchanged - 0 fixed = 170 total (was 168) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 38s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}165m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | HDFS-12886 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12900552/HDFS-12886.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux a349e7f3cf72 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d8863fc |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/22273/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[
https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16277764#comment-16277764
]
Íñigo Goiri commented on HDFS-12886:
Thanks [~lukmajercak] for working on this.
Can we avoid changing {{BLOCK_RECOVERY_TIMEOUT_MULTIPLIER}}?
> Ignore minReplication for block recovery
>
>
> Key: HDFS-12886
> URL: https://issues.apache.org/jira/browse/HDFS-12886
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs, namenode
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
> Attachments: HDFS-12886.001.patch
>
>
> Ignore minReplication for blocks that went through recovery, and allow NN to
> complete them and replicate.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
