[
https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242806#comment-13242806
]
Todd Lipcon commented on HDFS-3077:
-----------------------------------
bq. BK also has a notion of Ledger to support multiple clients - we are
re-inventing a fair part of BK. One of the arguments was that BK is a general
system while thre JD solution has only one client - the NN.
True, though BK takes effort to interleave all of the ledgers into a single
sequential stream, while writing an index file to allow de-interleaving upon
read. This is to support hundreds or thousands of concurrent WALs. In contrast,
I think even large HDFS installations would only run a few federated NNs with
the current design. So it's a much simpler problem, IMO.
bq. One our design is fleshed out, we should compare with BK in an objective
way.
Absolutely.
> Quorum-based protocol for reading and writing edit logs
> -------------------------------------------------------
>
> Key: HDFS-3077
> URL: https://issues.apache.org/jira/browse/HDFS-3077
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: ha, name-node
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
> Currently, one of the weak points of the HA design is that it relies on
> shared storage such as an NFS filer for the shared edit log. One alternative
> that has been proposed is to depend on BookKeeper, a ZooKeeper subproject
> which provides a highly available replicated edit log on commodity hardware.
> This JIRA is to implement another alternative, based on a quorum commit
> protocol, integrated more tightly in HDFS and with the requirements driven
> only by HDFS's needs rather than more generic use cases. More details to
> follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
