[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954790#comment-14954790
]
Hadoop QA commented on HDFS-7264:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 26m 59s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 10m 50s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 13m 30s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 37s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 39s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 2 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 2m 13s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 44s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 6m 36s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 4m 19s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 214m 55s | Tests failed in hadoop-hdfs. |
| {color:green}+1{color} | hdfs tests | 0m 43s | Tests passed in
hadoop-hdfs-client. |
| | | 284m 10s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | hadoop.hdfs.server.namenode.ha.TestHAFsck |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| | hadoop.hdfs.server.namenode.ha.TestEditLogTailer |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12766265/h7264_20151012.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 69b025d |
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12948/artifact/patchprocess/whitespace.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12948/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| hadoop-hdfs-client test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12948/artifact/patchprocess/testrun_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12948/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12948/console |
This message was automatically generated.
> Tha last datanode in a pipeline should send a heartbeat when there is no
> traffic
>
>
> Key: HDFS-7264
> URL: https://issues.apache.org/jira/browse/HDFS-7264
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Labels: BB2015-05-TBR
> Attachments: h7264_20141017.patch, h7264_20141020.patch,
> h7264_20151012.patch
>
>
> When the client is writing slowly, the client will send a heartbeat to signal
> that the connection is still alive. This case works fine.
> However, when a client is writing fast but some of the datanodes in the
> pipeline are busy, a PacketResponder may get a timeout since no ack is sent
> from the upstream datanode. We suggest that the last datanode in a pipeline
> should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[ https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14950927#comment-14950927 ] Yongjun Zhang commented on HDFS-7264: - HI [~szetszwo], thanks for reporting and working on this issue earlier. Any concern not to push this forward? thanks. > Tha last datanode in a pipeline should send a heartbeat when there is no > traffic > > > Key: HDFS-7264 > URL: https://issues.apache.org/jira/browse/HDFS-7264 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze > Labels: BB2015-05-TBR > Attachments: h7264_20141017.patch, h7264_20141020.patch > > > When the client is writing slowly, the client will send a heartbeat to signal > that the connection is still alive. This case works fine. > However, when a client is writing fast but some of the datanodes in the > pipeline are busy, a PacketResponder may get a timeout since no ack is sent > from the upstream datanode. We suggest that the last datanode in a pipeline > should send a heartbeat when there is no traffic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14951093#comment-14951093
]
Hadoop QA commented on HDFS-7264:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12675891/h7264_20141020.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c32614f |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12895/console |
This message was automatically generated.
> Tha last datanode in a pipeline should send a heartbeat when there is no
> traffic
>
>
> Key: HDFS-7264
> URL: https://issues.apache.org/jira/browse/HDFS-7264
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
> Labels: BB2015-05-TBR
> Attachments: h7264_20141017.patch, h7264_20141020.patch
>
>
> When the client is writing slowly, the client will send a heartbeat to signal
> that the connection is still alive. This case works fine.
> However, when a client is writing fast but some of the datanodes in the
> pipeline are busy, a PacketResponder may get a timeout since no ack is sent
> from the upstream datanode. We suggest that the last datanode in a pipeline
> should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524974#comment-14524974
]
Hadoop QA commented on HDFS-7264:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12675891/h7264_20141020.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10679/console |
This message was automatically generated.
Tha last datanode in a pipeline should send a heartbeat when there is no
traffic
Key: HDFS-7264
URL: https://issues.apache.org/jira/browse/HDFS-7264
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h7264_20141017.patch, h7264_20141020.patch
When the client is writing slowly, the client will send a heartbeat to signal
that the connection is still alive. This case works fine.
However, when a client is writing fast but some of the datanodes in the
pipeline are busy, a PacketResponder may get a timeout since no ack is sent
from the upstream datanode. We suggest that the last datanode in a pipeline
should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525036#comment-14525036
]
Hadoop QA commented on HDFS-7264:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12675891/h7264_20141020.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10702/console |
This message was automatically generated.
Tha last datanode in a pipeline should send a heartbeat when there is no
traffic
Key: HDFS-7264
URL: https://issues.apache.org/jira/browse/HDFS-7264
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h7264_20141017.patch, h7264_20141020.patch
When the client is writing slowly, the client will send a heartbeat to signal
that the connection is still alive. This case works fine.
However, when a client is writing fast but some of the datanodes in the
pipeline are busy, a PacketResponder may get a timeout since no ack is sent
from the upstream datanode. We suggest that the last datanode in a pipeline
should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176778#comment-14176778
]
Vinayakumar B commented on HDFS-7264:
-
Thanks [~szetszwo],
One nit,
1. Typo in {{DFS_DATANODE_WRITE_HEARTBEAT_EANBLE_KEY}} and
{{DFS_DATANODE_WRITE_HEARTBEAT_EANBLE_DEFAULT}}
Tha last datanode in a pipeline should send a heartbeat when there is no
traffic
Key: HDFS-7264
URL: https://issues.apache.org/jira/browse/HDFS-7264
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h7264_20141017.patch
When the client is writing slowly, the client will send a heartbeat to signal
that the connection is still alive. This case works fine.
However, when a client is writing fast but some of the datanodes in the
pipeline are busy, a PacketResponder may get a timeout since no ack is sent
from the upstream datanode. We suggest that the last datanode in a pipeline
should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[ https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176805#comment-14176805 ] Vinayakumar B commented on HDFS-7264: - One doubt,, Why can't heartbeat be enabled always.. without configuration flag, which is disabled by default. ? Tha last datanode in a pipeline should send a heartbeat when there is no traffic Key: HDFS-7264 URL: https://issues.apache.org/jira/browse/HDFS-7264 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h7264_20141017.patch When the client is writing slowly, the client will send a heartbeat to signal that the connection is still alive. This case works fine. However, when a client is writing fast but some of the datanodes in the pipeline are busy, a PacketResponder may get a timeout since no ack is sent from the upstream datanode. We suggest that the last datanode in a pipeline should send a heartbeat when there is no traffic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[ https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177248#comment-14177248 ] Tsz Wo Nicholas Sze commented on HDFS-7264: --- Hi Vinay, thanks for reviewing the patch. Why can't heartbeat be enabled always.. without configuration flag, which is disabled by default. ? It is for rolling upgrade. We have to disable the feature first, upgrade, and then enable the feature. Otherwise, the old software cannot handle the new heartbeat. Tha last datanode in a pipeline should send a heartbeat when there is no traffic Key: HDFS-7264 URL: https://issues.apache.org/jira/browse/HDFS-7264 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h7264_20141017.patch When the client is writing slowly, the client will send a heartbeat to signal that the connection is still alive. This case works fine. However, when a client is writing fast but some of the datanodes in the pipeline are busy, a PacketResponder may get a timeout since no ack is sent from the upstream datanode. We suggest that the last datanode in a pipeline should send a heartbeat when there is no traffic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177386#comment-14177386
]
Hadoop QA commented on HDFS-7264:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12675891/h7264_20141020.patch
against trunk revision d5084b9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8455//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8455//console
This message is automatically generated.
Tha last datanode in a pipeline should send a heartbeat when there is no
traffic
Key: HDFS-7264
URL: https://issues.apache.org/jira/browse/HDFS-7264
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h7264_20141017.patch, h7264_20141020.patch
When the client is writing slowly, the client will send a heartbeat to signal
that the connection is still alive. This case works fine.
However, when a client is writing fast but some of the datanodes in the
pipeline are busy, a PacketResponder may get a timeout since no ack is sent
from the upstream datanode. We suggest that the last datanode in a pipeline
should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[ https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14176190#comment-14176190 ] Tsz Wo Nicholas Sze commented on HDFS-7264: --- Shouldn't we add the heartbeat packet to the front of the queue? The queue is empty. So it is the same. org.apache.hadoop.hdfs.TestLeaseRecovery2 I cannot reproduce the failure of TestLeaseRecovery2. It seems not related. Tha last datanode in a pipeline should send a heartbeat when there is no traffic Key: HDFS-7264 URL: https://issues.apache.org/jira/browse/HDFS-7264 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h7264_20141017.patch When the client is writing slowly, the client will send a heartbeat to signal that the connection is still alive. This case works fine. However, when a client is writing fast but some of the datanodes in the pipeline are busy, a PacketResponder may get a timeout since no ack is sent from the upstream datanode. We suggest that the last datanode in a pipeline should send a heartbeat when there is no traffic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[ https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175832#comment-14175832 ] Jitendra Nath Pandey commented on HDFS-7264: bq. ackQueue.addLast(Packet.newHeartbeat()); Shouldn't we add the heartbeat packet to the front of the queue? Tha last datanode in a pipeline should send a heartbeat when there is no traffic Key: HDFS-7264 URL: https://issues.apache.org/jira/browse/HDFS-7264 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Reporter: Tsz Wo Nicholas Sze Assignee: Tsz Wo Nicholas Sze Attachments: h7264_20141017.patch When the client is writing slowly, the client will send a heartbeat to signal that the connection is still alive. This case works fine. However, when a client is writing fast but some of the datanodes in the pipeline are busy, a PacketResponder may get a timeout since no ack is sent from the upstream datanode. We suggest that the last datanode in a pipeline should send a heartbeat when there is no traffic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7264) Tha last datanode in a pipeline should send a heartbeat when there is no traffic
[
https://issues.apache.org/jira/browse/HDFS-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14175844#comment-14175844
]
Hadoop QA commented on HDFS-7264:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12675634/h7264_20141017.patch
against trunk revision c3de241.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestLeaseRecovery2
org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8449//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8449//console
This message is automatically generated.
Tha last datanode in a pipeline should send a heartbeat when there is no
traffic
Key: HDFS-7264
URL: https://issues.apache.org/jira/browse/HDFS-7264
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Reporter: Tsz Wo Nicholas Sze
Assignee: Tsz Wo Nicholas Sze
Attachments: h7264_20141017.patch
When the client is writing slowly, the client will send a heartbeat to signal
that the connection is still alive. This case works fine.
However, when a client is writing fast but some of the datanodes in the
pipeline are busy, a PacketResponder may get a timeout since no ack is sent
from the upstream datanode. We suggest that the last datanode in a pipeline
should send a heartbeat when there is no traffic.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
