[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271169#comment-14271169 ] Hudson commented on HDFS-7579: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2019 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2019/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270927#comment-14270927 ] Hudson commented on HDFS-7579: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #802 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/802/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271067#comment-14271067 ] Hudson commented on HDFS-7579: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2000 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2000/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271059#comment-14271059 ] Hudson commented on HDFS-7579: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #65 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/65/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270903#comment-14270903 ] Hudson commented on HDFS-7579: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #68 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/68/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14271093#comment-14271093 ] Hudson commented on HDFS-7579: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #69 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/69/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270233#comment-14270233 ] Hudson commented on HDFS-7579: -- FAILURE: Integrated in Hadoop-trunk-Commit #6834 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6834/]) HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb. (cnauroth: rev 7e2d9a32426d04b5f08c2835f61882b053612a20) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/ServerCommand.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Fix For: 2.7.0 Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269774#comment-14269774 ] Arpit Agarwal commented on HDFS-7579: - You are right - it cannot be null. I confused it with cmd. +1 for the .001 patch. I will hold off committing until the evening just in case Chris has comments. Thanks! Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269791#comment-14269791 ] Chris Nauroth commented on HDFS-7579: - +1 from me too, pending Jenkins run on the .001 patch. Thanks for responding to the code review feedback, Charles. Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14269795#comment-14269795 ] Charles Lamb commented on HDFS-7579: Thanks for the review Arpit and Chris. We'll see what Mr. Jenkins has to say. Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[
https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270012#comment-14270012
]
Hadoop QA commented on HDFS-7579:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12690884/HDFS-7579.001.patch
against trunk revision a6ed489.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDecommission
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9155//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9155//console
This message is automatically generated.
Improve log reporting during block report rpc failure
-
Key: HDFS-7579
URL: https://issues.apache.org/jira/browse/HDFS-7579
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 2.7.0
Reporter: Charles Lamb
Assignee: Charles Lamb
Priority: Minor
Labels: supportability
Attachments: HDFS-7579.000.patch, HDFS-7579.001.patch
During block reporting, if the block report RPC fails, for example because it
exceeded the max rpc len, we should still produce some sort of LOG.info
output to help with debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[
https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268329#comment-14268329
]
Chris Nauroth commented on HDFS-7579:
-
Hi [~clamb]. This is a good idea. Thanks for the patch.
One difference I see in this patch is that the logged value of
{{numReportsSent}} is set to the length of the per-volume report list on both
the single message and split RPC per report code paths. Previously, this
would have been 1 for the single message case. I think the intent of this
counter was to give an indication of the actual number of RPCs, so we'd want to
keep 1 for the single message case. Cc'ing [~arpitagarwal] for a second
opinion.
Improve log reporting during block report rpc failure
-
Key: HDFS-7579
URL: https://issues.apache.org/jira/browse/HDFS-7579
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 2.7.0
Reporter: Charles Lamb
Assignee: Charles Lamb
Priority: Minor
Labels: supportability
Attachments: HDFS-7579.000.patch
During block reporting, if the block report RPC fails, for example because it
exceeded the max rpc len, we should still produce some sort of LOG.info
output to help with debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[
https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268364#comment-14268364
]
Arpit Agarwal commented on HDFS-7579:
-
Thank you for the heads up Chris.
I agree that numReportsSent should be the number of RPCs and not
reports.length. We could rename it to be clearer.
Couple of additional comments:
# _final int nCmds = cmds.size();_ should check for {{cmds}} being null.
# The logged message should give some indication whether or not all RPCs
succeeded. Perhaps we can log something like _successfully completed x of y RPC
calls_. For the usual case (no split), y would be 1. For the split case it
would be {{reports.length}}.
Improve log reporting during block report rpc failure
-
Key: HDFS-7579
URL: https://issues.apache.org/jira/browse/HDFS-7579
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 2.7.0
Reporter: Charles Lamb
Assignee: Charles Lamb
Priority: Minor
Labels: supportability
Attachments: HDFS-7579.000.patch
During block reporting, if the block report RPC fails, for example because it
exceeded the max rpc len, we should still produce some sort of LOG.info
output to help with debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266709#comment-14266709 ] Charles Lamb commented on HDFS-7579: The test failure is a timeout and appears to be unrelated. I reran it myself on my local machine with the patch applied and it passed. Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Labels: supportability Attachments: HDFS-7579.000.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[
https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266641#comment-14266641
]
Hadoop QA commented on HDFS-7579:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12690360/HDFS-7579.000.patch
against trunk revision 4cd66f7.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.qjournal.TestNNWithQJM
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9148//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9148//console
This message is automatically generated.
Improve log reporting during block report rpc failure
-
Key: HDFS-7579
URL: https://issues.apache.org/jira/browse/HDFS-7579
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode
Affects Versions: 2.7.0
Reporter: Charles Lamb
Assignee: Charles Lamb
Priority: Minor
Labels: supportability
Attachments: HDFS-7579.000.patch
During block reporting, if the block report RPC fails, for example because it
exceeded the max rpc len, we should still produce some sort of LOG.info
output to help with debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7579) Improve log reporting during block report rpc failure
[ https://issues.apache.org/jira/browse/HDFS-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14266331#comment-14266331 ] Charles Lamb commented on HDFS-7579: Also, since this is just a remodularization of the LOG.info call, I did not add a unit test. Improve log reporting during block report rpc failure - Key: HDFS-7579 URL: https://issues.apache.org/jira/browse/HDFS-7579 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.0 Reporter: Charles Lamb Assignee: Charles Lamb Priority: Minor Attachments: HDFS-7579.000.patch During block reporting, if the block report RPC fails, for example because it exceeded the max rpc len, we should still produce some sort of LOG.info output to help with debugging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
