[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224372#comment-14224372
]
Hudson commented on HDFS-7412:
--
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #16 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/16/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224362#comment-14224362
]
Hudson commented on HDFS-7412:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #754 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/754/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224591#comment-14224591
]
Hudson commented on HDFS-7412:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1944 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1944/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224600#comment-14224600
]
Hudson commented on HDFS-7412:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #16 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/16/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224692#comment-14224692
]
Hudson commented on HDFS-7412:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #16 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/16/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224705#comment-14224705
]
Hudson commented on HDFS-7412:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1968 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1968/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223270#comment-14223270
]
Jing Zhao commented on HDFS-7412:
-
The latest patch looks good to me. +1
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223348#comment-14223348
]
Hudson commented on HDFS-7412:
--
FAILURE: Integrated in Hadoop-trunk-Commit #6594 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/6594/])
HDFS-7412. Move RetryCache to NameNodeRpcServer. Contributed by Haohui Mai.
(wheat9: rev 8e253cb93030642f5a7324bad0f161cd0ad33206)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDefaultBlockPlacementPolicy.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsLimits.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Fix For: 2.7.0
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221062#comment-14221062
]
Charles Lamb commented on HDFS-7412:
Hi [~wheat9],
Thanks for working on this. I think it's a good change. As [~yzhangal] said, it
looks like a pretty faithful refactoring. I have a few comments for your
consideration:
I believe you'll need to rebase this patch. One of the FSN changes in rename
didn't apply cleanly.
FSN:
* In FSN there's a funny indentation for the #appendFile() decl.
* In FSN#saveNamespace, there are two blank lines before the readLock() call.
Ditto in #deleteSnapshot. In #renameSnapshot and #deleteSnapshot where you
removed code you left a blank line which can be removed for eye candy.
NNRpcServer:
The import statics that your patch introduces appear to be unused.
TestDefaultBlockPlacementPolicy:
The change in setup() overflows the 80 char limit.
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221340#comment-14221340
]
Jing Zhao commented on HDFS-7412:
-
Looks like in our current implementation we have two patterns: 1) doing
OperationCategory check before retry cache check, or 2) checking retry cache
before the OperationCategory check. The difference here is that for 1) if the
client side's retry happens during the NN failover, the client has to failover
to the other NN to get the result stored in retry cache, which may cause issue
if the client has already hit the max failover times. The current patch makes
sure all the calls follow pattern 2), which should be a good change.
After the change the FSNamesystem calls have to take an extra argument to
indicate whether the retry cache is available. Since this is only used to
determine whether to record call/client id in the edits, maybe we can let
fseditlog check a globally visible marker or even a threadlocal context, so
that we can hide this logic away from the FSNamesystem APIs. But this can be
future work addressed by a separate jira.
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221681#comment-14221681
]
Hadoop QA commented on HDFS-7412:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682963/HDFS-7412.001.patch
against trunk revision 23dacb3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8806//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8806//console
This message is automatically generated.
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch, HDFS-7412.001.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219938#comment-14219938
]
Suresh Srinivas commented on HDFS-7412:
---
+1 for the idea. However it may not be straight forward. I look forward to the
patch to see if I am wrong :)
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220071#comment-14220071
]
Hadoop QA commented on HDFS-7412:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12682703/HDFS-7412.000.patch
against trunk revision acf1e03.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8790//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8790//console
This message is automatically generated.
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7412) Move RetryCache to NameNodeRpcServer
[
https://issues.apache.org/jira/browse/HDFS-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220536#comment-14220536
]
Yongjun Zhang commented on HDFS-7412:
-
Hi Haohui,
I happen to see this jira. It's nice to do this refactoring, thanks for working
on that!
I reviewed the changes you made, it looks a pretty faithful refactoring to me.
I noticed only one thing here that I'd like to bring to your attention:
according the coding guideline, when splitting a long line into multiple, the
second line need to be indented for 4 spaces. I noticed that some of the
changes involved in this refactoring only has 2 spaces. Though the original
code is like that too, since the changes appear to new code in
NameNodeRpcServer.java, it's an opportunity to make the change in my opinion.
Wonder whether you would consider addressing this.
Thanks.
Move RetryCache to NameNodeRpcServer
Key: HDFS-7412
URL: https://issues.apache.org/jira/browse/HDFS-7412
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Haohui Mai
Assignee: Haohui Mai
Attachments: HDFS-7412.000.patch
The concept of RetryCache belongs to the RPC layer.It would be nice to
separate it from the implementation of {{FSNameSystem}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
