[
https://issues.apache.org/jira/browse/HDFS-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488246#comment-13488246
]
Colin Patrick McCabe commented on HDFS-4045:
--------------------------------------------
I guess we have to think about whether the overhead of avoiding an extra
"deserialize + serialize" pair on the JN is worth violating the
EditLogInputStream abstraction. Unfortunately, I can't think of a good way to
get the benefits of {{RedundantEditLogInputStream}} without using its intended
interface-- {{readOp}}.
If you do choose to violate the abstraction, it would be best to do so by
creating FileInputStream objects for the underlying edit log files, when
appropriate. Then just read directly from the files. You'll still have a hard
time dealing with IOExceptions in the middle of reading the file. It wouldn't
be *too* hard to do that failover yourself, but you run into sticky situations.
What if you get an IOExecption, fail over to the next file, and then discover
that it's actually fairly different from the first one? You might have sent
bytes over the wire that completely fail the edit log operation checksums.
I think it's best to punt on the issue of zero-copy {{serveEdits}} for now.
Maybe open up a follow-up JIRA for that. Just do the simple thing of streaming
it for now. I also wonder whether serveEdits should serve a range of edits,
rather than discrete edit log segments? The API feels very segment-oriented,
and I thought we were trying to get away from that when possible.
> SecondaryNameNode cannot read from QuorumJournal URI
> ----------------------------------------------------
>
> Key: HDFS-4045
> URL: https://issues.apache.org/jira/browse/HDFS-4045
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 3.0.0
> Reporter: Vinithra Varadharajan
> Assignee: Andy Isaacson
> Attachments: hdfs-4045-2.txt, hdfs4045-3.txt, hdfs-4045.txt
>
>
> If HDFS is set up in basic mode (non-HA) with QuorumJournal, and the
> dfs.namenode.edits.dir is set to only the QuorumJournal URI and no local dir,
> the SecondaryNameNode is unable to do a checkpoint.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
