[ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090474#comment-13090474 ]
Todd Lipcon commented on HDFS-2018: ----------------------------------- Having the 2NN use an EditLogInputStream interface to fetch edits seems a little "off" since EditLogInputStream only passes along logical operations and not bytes. So, the 2NN will be deserializing all the transactions. I guess in some ways it is more elegant, though. Do you foresee the extra deserialize/serialize step during edits transfer to be expensive? If this has been implemented, could you point me to the branch where it is? As is, in the patches I could find, edits transfer is left alone and it breaks a lot of the abstractions that are being pushed for. > 1073: Move all journal stream management code into one place > ------------------------------------------------------------ > > Key: HDFS-2018 > URL: https://issues.apache.org/jira/browse/HDFS-2018 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Ivan Kelly > Assignee: Ivan Kelly > Fix For: 0.23.0 > > Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, > HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, > HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, > HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, > HDFS-2018.diff, hdfs-2018-otherapi.txt, hdfs-2018.txt > > > Currently in the HDFS-1073 branch, the code for creating output streams is in > FileJournalManager and the code for input streams is in the inspectors. This > change does a number of things. > - Input and Output streams are now created by the JournalManager. > - FSImageStorageInspectors now deals with URIs when referring to edit logs > - Recovery of inprogress logs is performed by counting the number of > transactions instead of looking at the length of the file. > The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira