[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462716#comment-13462716 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk #1176 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1176/]) Moving HDFS-234 from trunk to Release 2.0.0-alpha section (Revision 1389494) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1389494 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 2.0.0-alpha, 3.0.0 Attachments: create.png, HADOOP-5189.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13462748#comment-13462748 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk #1207 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1207/]) Moving HDFS-234 from trunk to Release 2.0.0-alpha section (Revision 1389494) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1389494 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 2.0.0-alpha, 3.0.0 Attachments: create.png, HADOOP-5189.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461945#comment-13461945 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Common-trunk-Commit #2761 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2761/]) Moving HDFS-234 from trunk to Release 2.0.0-alpha section (Revision 1389494) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1389494 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 2.0.0-alpha, 3.0.0 Attachments: create.png, HADOOP-5189.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461946#comment-13461946 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk-Commit #2824 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2824/]) Moving HDFS-234 from trunk to Release 2.0.0-alpha section (Revision 1389494) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1389494 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 2.0.0-alpha, 3.0.0 Attachments: create.png, HADOOP-5189.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461987#comment-13461987 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk-Commit #2783 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2783/]) Moving HDFS-234 from trunk to Release 2.0.0-alpha section (Revision 1389494) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1389494 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 2.0.0-alpha, 3.0.0 Attachments: create.png, HADOOP-5189.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270658#comment-13270658 ] Todd Lipcon commented on HDFS-234: -- Uma: I'm a little concerned about having this as a main part of our 2.0 release, to be honest. There hasn't been activity and the only activity I've seen around it has been from Ivan. Is anyone committed to actually QAing and running this code? A lot of the other features around HA don't integrate with it, it isn't documented, etc. Maybe we should start a discussion on the list. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270686#comment-13270686 ] Uma Maheswara Rao G commented on HDFS-234: -- Hi Todd, As this is also one option for HA, we are working on this patch merged to Hadoop-2 internally and we are testing with BookKeeper. Also this is a contrib part in HDFS. As this is a pluggable option, why can't we keep this in contrib? If we have other options ready and performing well, then we can choose one among, as you mentioned in other discussions. I have seen the comment from Dhruba for continue invest on BK based approach also. https://issues.apache.org/jira/browse/HDFS-3092?focusedCommentId=13265110page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13265110 Since the basic thing ready from BKJM side, it will help some users for evaluating it as this is an alpha quality release. Identified some issues like HDFS-3058 in our testing also. Infact we have to push this also in. Regarding document: I agree, we may have to prepare well documentation for using this option, currently it is available only in BKJM class. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270691#comment-13270691 ] dhruba borthakur commented on HDFS-234: --- Hi Uma, while it is true that we are dedicating resources to evaluate HDFS-HA with BK, it will possibly be for trunk and not for 2.0. are you folks planning to use this in hadoop 2.0 in production? Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270705#comment-13270705 ] Eli Collins commented on HDFS-234: -- Uma, Given all the bugs you found recently, and haven't been triaged/assigned, there are no docs, etc I'd keep this slated for 3.0. If it gets tested/doc'd and stabilizes on trunk then we can consider merging. We're actually trying to get away from using contrib. See this thread for history: http://search-hadoop.com/m/EPdTjoLpuh2. I removed most of the hdfs contrib components. Thanks, Eli Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270721#comment-13270721 ] Suresh Srinivas commented on HDFS-234: -- Eli, given that is this an option that Uma is trying out and 2.0-alpha is being release, I feel it is fine for HDFS-234 to be merged to 2.0. As regards to contrib components, lets have a separate discussion on how BK related code should be made available, if not contrib. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[
https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270729#comment-13270729
]
Uma Maheswara Rao G commented on HDFS-234:
--
Eli, HDFS-2717 and HDFS-3058 are the main issues identified at BKJM side with
testing and discussions with Ivan. Also we have been filing some in BK project
also.
{quote}
given that is this an option that Uma is trying out and 2.0-alpha is being
release, I feel it is fine for HDFS-234 to be merged to 2.0.
{quote}
Yes, Suresh, this is what my opinion.
Integration with BookKeeper logging system
--
Key: HDFS-234
URL: https://issues.apache.org/jira/browse/HDFS-234
Project: Hadoop HDFS
Issue Type: New Feature
Reporter: Luca Telloli
Assignee: Ivan Kelly
Fix For: 3.0.0
Attachments: HADOOP-5189-trunk-preview.patch,
HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch,
HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch,
HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff,
HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar,
zookeeper-dev.jar
BookKeeper is a system to reliably log streams of records
(https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a
natural target for such a system for being the metadata repository of the
entire file system for HDFS.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270790#comment-13270790 ] Uma Maheswara Rao G commented on HDFS-234: -- Others, Agreeing with Suresh? Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13270874#comment-13270874 ] Nathan Roberts commented on HDFS-234: - I agree it would be good to have it in 2.0-alpha for folks to experiment with and compare with other alternatives. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: 3.0.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234-branch-2.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13239791#comment-13239791 ] Uma Maheswara Rao G commented on HDFS-234: -- Looks this particular issue, did not merged to HA branch ( but marked fix versions to HA branch also). It was committed only to trunk. So, 23 branch missing this code. We have to merge this 23 branch for HA release right.( as this is also one option in shared storage).? @Aaron, are we planning this issue merge to 23? Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13173145#comment-13173145 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk #900 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/900/]) Amend HDFS-234 (bkjournal contrib). Fix findbugs build - Fixes pom parent to point to hadoop-project instead of hadoop-project-dist - Fix relative path in pom to have the correct number of ../s todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1220940 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13173190#comment-13173190 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk #933 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/933/]) Amend HDFS-234 (bkjournal contrib). Fix findbugs build - Fixes pom parent to point to hadoop-project instead of hadoop-project-dist - Fix relative path in pom to have the correct number of ../s todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1220940 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172626#comment-13172626 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Common-trunk-Commit #1453 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1453/]) Amend HDFS-234 (bkjournal contrib). Fix findbugs build - Fixes pom parent to point to hadoop-project instead of hadoop-project-dist - Fix relative path in pom to have the correct number of ../s todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1220940 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172630#comment-13172630 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk-Commit #1526 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1526/]) Amend HDFS-234 (bkjournal contrib). Fix findbugs build - Fixes pom parent to point to hadoop-project instead of hadoop-project-dist - Fix relative path in pom to have the correct number of ../s todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1220940 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13172634#comment-13172634 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk-Commit #1476 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1476/]) Amend HDFS-234 (bkjournal contrib). Fix findbugs build - Fixes pom parent to point to hadoop-project instead of hadoop-project-dist - Fix relative path in pom to have the correct number of ../s todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1220940 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13169212#comment-13169212 ] Ivan Kelly commented on HDFS-234: - Awesome :D Thanks for your help Jitendra. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13169451#comment-13169451 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk #927 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/927/]) HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1213983 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/WriteLock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties * /hadoop/common/trunk/hadoop-hdfs-project/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13169471#comment-13169471 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk #894 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/894/]) HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1213983 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/WriteLock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties * /hadoop/common/trunk/hadoop-hdfs-project/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168691#comment-13168691 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507213/HDFS-234.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 90 warning messages. -1 javac. The applied patch generated 39 javac compiler warnings (more than the trunk's current 37 warnings). -1 eclipse:eclipse. The patch failed to build with eclipse:eclipse. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hdfs.TestFileAppend org.apache.hadoop.hdfs.TestDFSRemove +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1704//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1704//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168764#comment-13168764 ] Jitendra Nath Pandey commented on HDFS-234: --- +1 for the patch. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168869#comment-13168869 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Hdfs-trunk-Commit #1493 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1493/]) HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1213983 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/WriteLock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties * /hadoop/common/trunk/hadoop-hdfs-project/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168870#comment-13168870 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Common-trunk-Commit #1419 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1419/]) HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1213983 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/WriteLock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties * /hadoop/common/trunk/hadoop-hdfs-project/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar,
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168879#comment-13168879 ] Hudson commented on HDFS-234: - Integrated in Hadoop-Mapreduce-trunk-Commit #1443 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1443/]) HDFS-234. Integration with BookKeeper logging system. Contributed by Ivan Kelly. jitendra : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1213983 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/WriteLock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties * /hadoop/common/trunk/hadoop-hdfs-project/pom.xml Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar,
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167586#comment-13167586 ] Ivan Kelly commented on HDFS-234: - I think we should commit it to hdfs contrib. Please move the bkjournal package to contrib. In principle I have no problem with this. However, contrib doesn't seem to be set up to run with maven at the moment. This needs to be compiled and tested as part of the normal build cycle. Otherwise, changes to the NN interfaces will silently break this. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13167639#comment-13167639 ] Jakob Homan commented on HDFS-234: -- bq. I think we should commit it to hdfs contrib. Please move the bkjournal package to contrib. +1 Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166616#comment-13166616 ] Jitendra Nath Pandey commented on HDFS-234: --- I think we should commit it to hdfs contrib. Please move the bkjournal package to contrib. For secure hadoop setup, it will be important to have authenticated connection to book keeper servers. That should be addressed in follow up jiras. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13164697#comment-13164697 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506511/HDFS-234.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1658//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1658//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13164849#comment-13164849 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506535/HDFS-234.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 37 javac compiler warnings (more than the trunk's current 35 warnings). -1 findbugs. The patch appears to introduce 9 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1659//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1659//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1659//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13163545#comment-13163545 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12506247/HDFS-234.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1646//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1646//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13150586#comment-13150586 ] jirapos...@reviews.apache.org commented on HDFS-234: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2835/ --- Review request for hadoop-hdfs. Summary --- BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. This addresses bug HDFS-234. http://issues.apache.org/jira/browse/HDFS-234 Diffs - hadoop-hdfs-project/hadoop-hdfs/pom.xml 1bcc372 hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/BookKeeperEditLogInputStream.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/TestBookKeeperJournalManager.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/EditLogLedgerMetadata.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/MaxTxId.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/WriteLock.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/BookKeeperEditLogOutputStream.java PRE-CREATION hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/bkjournal/BookKeeperJournalManager.java PRE-CREATION Diff: https://reviews.apache.org/r/2835/diff Testing --- Thanks, Ivan Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.patch, create.png, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13150589#comment-13150589 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503761/HDFS-234.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The patch appears to cause tar ant target to fail. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed the unit tests build +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1559//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1559//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13150598#comment-13150598 ] Hadoop QA commented on HDFS-234: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12503763/hdfs_tpt_lat.pdf against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1560//console This message is automatically generated. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Ivan Kelly Fix For: HA branch (HDFS-1623), 0.24.0 Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.diff, HDFS-234.patch, create.png, hdfs_tpt_lat.pdf, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13104471#comment-13104471 ] Ivan Kelly commented on HDFS-234: - Reviving this JIRA as the namenode journalling refactor is almost complete. This JIRA blocked by HDFS-2188, HDFS-2334 HDFS-1580. Once these are in trunk I'll be able to cut a patch for this. Until then, I've got a beta of HDFS+PluggableJournal+BK on github. https://github.com/ivankelly/hadoop-common/tree/BKJournal Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Luca Telloli Attachments: HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.patch, create.png, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-234) Integration with BookKeeper logging system
[ https://issues.apache.org/jira/browse/HDFS-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12730481#action_12730481 ] Flavio Paiva Junqueira commented on HDFS-234: - When you read from a BookKeeper ledger, it seems that you are reading the whole ledger before processing it. I can see two potential problems with it: # If the ledger is large, then the process reading the ledger might end up consuming too much memory; # It does not overlap reading the ledger with processing. It might be best to try to overlap reading from the ledger with processing the edits. I don't know what changes this suggestion implies, though. Integration with BookKeeper logging system -- Key: HDFS-234 URL: https://issues.apache.org/jira/browse/HDFS-234 Project: Hadoop HDFS Issue Type: New Feature Reporter: Luca Telloli Assignee: Luca Telloli Attachments: create.png, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-trunk-preview.patch, HADOOP-5189-v.19.patch, HADOOP-5189.patch, HDFS-234.patch, zookeeper-dev-bookkeeper.jar, zookeeper-dev.jar BookKeeper is a system to reliably log streams of records (https://issues.apache.org/jira/browse/ZOOKEEPER-276). The NameNode is a natural target for such a system for being the metadata repository of the entire file system for HDFS. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
