[
https://issues.apache.org/jira/browse/HDFS-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396431#comment-13396431
]
Todd Lipcon commented on HDFS-3425:
-----------------------------------
bq. I am not sure I got the idea about using selectInputStreams() instead of
if(usesJournalService()). Is your branch currently visible, so we can see an
example?
That's what I'm working on right now. But basically, the idea is the following:
- the NN runs with just a QuorumJournalManager, which maintains the list of
remote JournalNodes to talk to. I believe your design doc calls this a
JournalList instead, but same thing.
- when the NN (or the Standby NN) wants to read edits, it uses
editLog.selectInputStreams() exactly as it does today. But the
QuorumJournalManager implementation of it services the call as follows:
-- RPC to each of the remote journals
-- get back a list of finalized edit log segments
-- for each, creates an object which stores the HTTP URL at which that segment
is accessible, along with the txid info
-- creates an EditLogFileInputStream for each such object, but when opened,
instead of creating FileInputStream, it opens the URL connection and calls
getInputStream() on the resulting URL
So, the NN code remains identical, and all the magic is encapsulated in the
journal manager implementation. This is basically the same as how the
BookKeeper implementation works today -- the journal manager just has to
implement selectInputStreams and the rest of the code _should_ work the same
without special cases.
> Enable standby namenode to tail edits from journal service
> ----------------------------------------------------------
>
> Key: HDFS-3425
> URL: https://issues.apache.org/jira/browse/HDFS-3425
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Brandon Li
> Assignee: Brandon Li
> Attachments: HDFS-3425.HDFS-3092.patch, HDFS-3425.HDFS-3092.patch.2
>
>
> In an HA cluster with the journal service configured, namenode should get
> edit logs from journal service instead of from local or shared storage
> directories.
> As one of the first steps towards this goal, this JIRA is to enable standby
> namenode to tail edits from journal services.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
