[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-14 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057036#comment-15057036
 ] 

Hudson commented on HDFS-9535:
--

ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #692 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/692/])
HDFS-9535. Newly completed blocks in IBR should not be considered (jing9: rev 
e53456981474d6e16e3c134e3777b3588dc6fedf)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Fix For: 2.8.0
>
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.
> This test failure reveals a scenario that HDFS-1172 missed: if a block is 
> first committed by the client, and then the first IBR comes to the NN, as 
> proposed by HDFS-1172, we should still put the remaining expected replicas 
> into the pending queue, instead of the under-replicated queue. Please see 
> [~liuml07]'s comment 
> [here|https://issues.apache.org/jira/browse/HDFS-9535?focusedCommentId=15052397&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15052397]
>  for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-14 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056523#comment-15056523
 ] 

Hudson commented on HDFS-9535:
--

FAILURE: Integrated in Hadoop-trunk-Commit #8962 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/8962/])
HDFS-9535. Newly completed blocks in IBR should not be considered (jing9: rev 
e53456981474d6e16e3c134e3777b3588dc6fedf)
* 
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt


> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.
> This test failure reveals a scenario that HDFS-1172 missed. Please see 
> [~liuml07]'s comment [here|



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-14 Thread Mingliang Liu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056476#comment-15056476
 ] 

Mingliang Liu commented on HDFS-9535:
-

Thanks to [~jingzhao] for cutting this jira, reviewing and committing the 
patch. Thanks to [~iwasakims] for useful comments.

> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-14 Thread Jing Zhao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056469#comment-15056469
 ] 

Jing Zhao commented on HDFS-9535:
-

The fix looks good to me. The test failures should be unrelated. +1. I will 
commit the patch shortly.

> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Mingliang Liu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15054079#comment-15054079
 ] 

Mingliang Liu commented on HDFS-9535:
-

Failing tests can not be re-produced locally. Seem unrelated?

> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15054026#comment-15054026
 ] 

Hadoop QA commented on HDFS-9535:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 49s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
51s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 45s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s 
{color} | {color:red} Patch generated 1 new checkstyle issues in 
hadoop-hdfs-project/hadoop-hdfs (total was 160, now 160). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 8s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 49s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 51m 56s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 53s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 19s 
{color} | {color:red} Patch generated 58 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 128m 34s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | 
hadoop.hdfs.server.datanode.TestBlockScanner |
|   | hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA |
|   | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| JDK v1.7.0_91 Failed junit tests | hadoop.hdfs.TestRecoverStripedFile |
|   | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
|   | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15054023#comment-15054023
 ] 

Hadoop QA commented on HDFS-9535:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m 
11s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 55s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 10s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
27s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 25s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
4s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 56s 
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 20s 
{color} | {color:red} Patch generated 1 new checkstyle issues in 
hadoop-hdfs-project/hadoop-hdfs (total was 161, now 161). {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 32s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 21s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 13s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 70m 56s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 26s 
{color} | {color:red} Patch generated 56 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 187m 58s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | 
hadoop.hdfs.security.TestDelegationTokenForProxyUser |
|   | hadoop.hdfs.TestDFSStorageStateRecovery |
|   | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
|   | hadoop.hdfs.TestRecoverStripedFile |
|   | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery 
|
| JDK v1.7.0_91 Failed junit tests | hadoop.hdfs.TestBlockReaderLocal |
|   | hadoop.hdfs.shortcircuit.TestShortCircuitCache |
|   | hadoop.hdfs.Test

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053972#comment-15053972
 ] 

Hadoop QA commented on HDFS-9535:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 18s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
22s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 21s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 45s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 16s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 36s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 46s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 48s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 82m 21s {color} 
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 31s 
{color} | {color:red} Patch generated 57 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 202m 7s {color} 
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | 
hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 |
|   | hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes |
|   | hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes |
|   | hadoop.hdfs.TestFileAppend |
|   | hadoop.hdfs.security.TestDelegationTokenForProxyUser |
|   | hadoop.hdfs.server.namenode.TestCacheDirectives |
|   | hadoop.hdfs.TestDataTransferKeepalive |
|   | hadoop.hdfs.shortcircuit.TestShortCircuitCache |
|   | hadoop.hdfs.server.namenode.ha.TestHAAppend |
| JDK v1.8.0_66 Timed out junit tests | 
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
|   | org.apache.hadoop.hdfs.server.balan

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Mingliang Liu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053940#comment-15053940
 ] 

Mingliang Liu commented on HDFS-9535:
-

Thank you both for the comment. The v2 patch is to address this.

> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch, 
> HDFS-9535.002.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Masatake Iwasaki (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053897#comment-15053897
 ] 

Masatake Iwasaki commented on HDFS-9535:


bq. The guarantee Jing Zhao proposed in addStoredBlock makes sense to me.

I too agree with the fix. Should we have checking {{if (!bc.isStriped())}} 
around {{addExpectedReplicasToPending}}?


> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-9535) Newly completed blocks in IBR should not be considered under-replicated too quickly

2015-12-11 Thread Jing Zhao (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-9535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053893#comment-15053893
 ] 

Jing Zhao commented on HDFS-9535:
-

Thanks for the patch, Mingliang! One comment is that for now we should skip 
this logic for a striped block, since we cannot guarantee that the remaining 
expected internal blocks are healthy even if the client commits the block. 
Other than this the patch looks good to me.

> Newly completed blocks in IBR should not be considered under-replicated too 
> quickly
> ---
>
> Key: HDFS-9535
> URL: https://issues.apache.org/jira/browse/HDFS-9535
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.8.0
>Reporter: Jing Zhao
>Assignee: Mingliang Liu
> Attachments: HDFS-9535.000.patch, HDFS-9535.001.patch
>
>
> TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate failed in 
> several Jenkins run (e.g., 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13818/testReport/). The 
> failure is on the last {{assertNoReplicationWasPerformed}} check.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)