[
https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245656#comment-13245656
]
Todd Lipcon commented on HDFS-3077:
-----------------------------------
bq. So currently, state is epoch number is 2 on all the journals and J1, J2 and
J3 are at 153. We have a problem since it is not possible to distinguish
between log entries in J1 vs J2 and J3.
Hey Hari. Thanks for taking a look in such good detail.
I think the doc is currently unclear about the proposed solution described in
2.5.6 -- the idea is not to use just the "lastPromisedEpoch" here to
distinguish the JNs, but rather to attach the epoch number to each log segment,
based on the epoch in which that segment was started. So, even though in your
scenario NN1 sets J1.lastPromisedEpoch=2, the log segment will retain e=1. Once
a segment's epoch is set, it is never changed (unless the segment is removed by
a synchronization)
Does that make sense? If so I will try to clarify the document.
> Quorum-based protocol for reading and writing edit logs
> -------------------------------------------------------
>
> Key: HDFS-3077
> URL: https://issues.apache.org/jira/browse/HDFS-3077
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: ha, name-node
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3077-partial.txt, qjournal-design.pdf
>
>
> Currently, one of the weak points of the HA design is that it relies on
> shared storage such as an NFS filer for the shared edit log. One alternative
> that has been proposed is to depend on BookKeeper, a ZooKeeper subproject
> which provides a highly available replicated edit log on commodity hardware.
> This JIRA is to implement another alternative, based on a quorum commit
> protocol, integrated more tightly in HDFS and with the requirements driven
> only by HDFS's needs rather than more generic use cases. More details to
> follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
