[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435041#comment-13435041 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Hdfs-trunk #1135 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1135/]) Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk. (Revision 1372605) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1372605 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435070#comment-13435070 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Mapreduce-trunk #1167 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1167/]) Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk. (Revision 1372605) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1372605 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433612#comment-13433612 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Hdfs-trunk-Commit #2639 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2639/]) Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk. (Revision 1372605) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1372605 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433623#comment-13433623 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Common-trunk-Commit #2574 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2574/]) Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk. (Revision 1372605) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1372605 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433628#comment-13433628 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2596 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2596/]) Move HDFS-2330 and HDFS-3190 to branch-2 section, since they have been backported from trunk. (Revision 1372605) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1372605 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0, 2.2.0-alpha Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13407008#comment-13407008 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Hdfs-trunk #1094 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1094/]) HDFS-3190. Simple refactors in existing NN code to assist QuorumJournalManager extension. Contributed by Todd Lipcon. (Revision 1356525) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1356525 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageErrorReporter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/AtomicFileOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/PersistentLongFile.java Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0 Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13407131#comment-13407131 ] Hudson commented on HDFS-3190: -- Integrated in Hadoop-Mapreduce-trunk #1127 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1127/]) HDFS-3190. Simple refactors in existing NN code to assist QuorumJournalManager extension. Contributed by Todd Lipcon. (Revision 1356525) Result = SUCCESS todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1356525 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageErrorReporter.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FileJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/TransferFsImage.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/AtomicFileOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/PersistentLongFile.java Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0 Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405382#comment-13405382 ] Suresh Srinivas commented on HDFS-3190: --- +1 for the patch. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405446#comment-13405446 ] Hadoop QA commented on HDFS-3190: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12534302/hdfs-3190.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2730//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2730//console This message is automatically generated. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405451#comment-13405451 ] Todd Lipcon commented on HDFS-3190: --- Thanks for the review, Suresh. I'll commit this momentarily. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13405482#comment-13405482 ] Hadoop QA commented on HDFS-3190: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12534302/hdfs-3190.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2732//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2732//console This message is automatically generated. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Fix For: 3.0.0 Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403590#comment-13403590 ] Aaron T. Myers commented on HDFS-3190: -- +1, the latest patch looks good to me. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403597#comment-13403597 ] Suresh Srinivas commented on HDFS-3190: --- Todd, can you please hold off committing it. I want to take a quick look. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403616#comment-13403616 ] Suresh Srinivas commented on HDFS-3190: --- Nice refactoring. +1 for the patch with some minor comments: # Some lines are over 80 chars. # StorageErrorReporter.java #* StorageErrorReporter javadoc - can you please add link to JournalManager. I am not very clear on what you mean by central manager of storage directory state. #* Also can you please add javadoc description to the methods # PersistentLong - PersistentLongFile or AtomicLongFile better name? Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401904#comment-13401904 ] Hadoop QA commented on HDFS-3190: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533572/hdfs-3190.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2704//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2704//console This message is automatically generated. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291412#comment-13291412 ] Hadoop QA commented on HDFS-3190: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531319/hdfs-3190.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2616//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2616//console This message is automatically generated. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0-alpha Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt, hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[
https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291479#comment-13291479
]
Colin Patrick McCabe commented on HDFS-3190:
It's good to see some cleanup.
Hmm. LongContainingFile just seems like a containing file that is long.
How does SingleValueFile seem as a class name? Then you could have
SingleValueFile#readLong and SingleValueFile#writeLong.
{code}
+ public static long read(File file, long defaultVal) throws IOException {
+long txid = defaultVal;
+// TODO: canRead below seems wrong
+if (file.exists() file.canRead()) {
+ BufferedReader br = new BufferedReader(new FileReader(file));
+ try {
+txid = Long.valueOf(br.readLine());
+br.close();
+br = null;
+ } finally {
+IOUtils.cleanup(LOG, br);
+ }
+}
+return txid;
+ }
{code}
Yeah, I agree that the canRead() seems unecessary.
It's unfortunate that the BufferedReader constructor always throws a
FileNotFound exception, even if the file *is* found but simply is not readable.
However, despite the misnamed exception, if you get rid of the canRead(), you
should have the semantic that file not found = default, file not readable =
IOException, which I think is what you want.
Simple refactors in existing NN code to assist QuorumJournalManager extension
-
Key: HDFS-3190
URL: https://issues.apache.org/jira/browse/HDFS-3190
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: 2.0.0-alpha
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
Attachments: hdfs-3190.txt, hdfs-3190.txt
This JIRA is for some simple refactors in the NN:
- refactor the code which writes the seen_txid file in NNStorage into a new
LongContainingFile utility class. This is useful for the JournalNode to
atomically/durably record its last promised epoch
- refactor the interface from FileJournalManager back to StorageDirectory to
use a StorageErrorReport interface. This allows FileJournalManager to be used
in isolation of a full StorageDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245765#comment-13245765 ] Bikas Saha commented on HDFS-3190: -- +1 lgtm. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[
https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245778#comment-13245778
]
Todd Lipcon commented on HDFS-3190:
---
thanks Bikas. Quick question for reviewers: when I moved this code, I noticed
the {{canRead()}} check. Currently if the file exists but can't be read, it
returns the default value. I thought this was a little suspicious. Anyone
adverse to removing that check, so that we throw an exception if it exists but
we can't read it? Or better to keep this as a straight refactor and file a
follow-up to think about that?
Simple refactors in existing NN code to assist QuorumJournalManager extension
-
Key: HDFS-3190
URL: https://issues.apache.org/jira/browse/HDFS-3190
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: 2.0.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
Attachments: hdfs-3190.txt
This JIRA is for some simple refactors in the NN:
- refactor the code which writes the seen_txid file in NNStorage into a new
LongContainingFile utility class. This is useful for the JournalNode to
atomically/durably record its last promised epoch
- refactor the interface from FileJournalManager back to StorageDirectory to
use a StorageErrorReport interface. This allows FileJournalManager to be used
in isolation of a full StorageDirectory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3190) Simple refactors in existing NN code to assist QuorumJournalManager extension
[ https://issues.apache.org/jira/browse/HDFS-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13245871#comment-13245871 ] Hadoop QA commented on HDFS-3190: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521220/hdfs-3190.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2173//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2173//console This message is automatically generated. Simple refactors in existing NN code to assist QuorumJournalManager extension - Key: HDFS-3190 URL: https://issues.apache.org/jira/browse/HDFS-3190 Project: Hadoop HDFS Issue Type: Sub-task Components: name-node Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Attachments: hdfs-3190.txt This JIRA is for some simple refactors in the NN: - refactor the code which writes the seen_txid file in NNStorage into a new LongContainingFile utility class. This is useful for the JournalNode to atomically/durably record its last promised epoch - refactor the interface from FileJournalManager back to StorageDirectory to use a StorageErrorReport interface. This allows FileJournalManager to be used in isolation of a full StorageDirectory. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
