[
https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-2018:
------------------------------
Attachment: hdfs-2018-otherapi.txt
Here's a rough cut of the way I'm thinking about this. I didn't update
TestFileJournalManager, but the other tests compile and pass (TestBackupNode,
TestCheckpoint, TestEditLog).
I think it's a little easier to understand these APIs, and it's ~100 fewer
lines of code.
Some of the advantages IMO:
- no need for the more complicated caching in FileJournalManager, since we only
scan the directory once
- treats log recovery as an explicit step at startup - it's good to make it
explicit since we need to _not_ do recovery when a NN starts up in standby
mode, for example.
- the EditLogReference interface will also make it easier to allow other types
of journal managers to participate in edits-transfer, I think.
> 1073: Move all journal stream management code into one place
> ------------------------------------------------------------
>
> Key: HDFS-2018
> URL: https://issues.apache.org/jira/browse/HDFS-2018
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: 0.23.0
>
> Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
> HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
> HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
> HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
> HDFS-2018.diff, hdfs-2018-otherapi.txt
>
>
> Currently in the HDFS-1073 branch, the code for creating output streams is in
> FileJournalManager and the code for input streams is in the inspectors. This
> change does a number of things.
> - Input and Output streams are now created by the JournalManager.
> - FSImageStorageInspectors now deals with URIs when referring to edit logs
> - Recovery of inprogress logs is performed by counting the number of
> transactions instead of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
