[
https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990760#comment-12990760
]
Jitendra Nath Pandey commented on HDFS-1580:
--------------------------------------------
> > ShouldWALStreamCustodians control rolling themselves?
As I understand, book-keeper doesn't allow reading logs from open ledger, is
that correct? If that is the case it would be better to first roll and then do
the checkpoint to capture most latest edits.
>WALStreamCustodianNotifier
It seems to me this will be implemented only by fsedit log. If that is the
case we could handle errors just by exceptions i.e. if an operation on
WALStreamCustodian interface fails an exception is thrown and fsedit log can
decide to remove the custodian depending on the kind of exception.
> namenode crash
If the namenode comes back before the znode disappears from zookeeper, the
ledger will be open. In that case, will the FSEditLog.load be able to load
transactions from the open ledger as well?
> interface JournalStream
The document defines this interface but doesn't describe its purpose or
use-case.
> List<URI> getLogs(long sinceTransactionId);
The list returned must be ordered w.r.t the transactions contained. It might
be a good idea to encode the ordering attribute in the url itself, so that the
caller of this method can also verify that order is correct. The uri naming
convention could mimic this aspect from the convention proposed in 1073.
> void startRoll()
> void endRoll()
I can only imagine a single roll method, that cuts a log, and starts a new
one. I believe the naming convention or the ordering attribute for the logs
should be controlled by the application and not the storage, therefore the roll
method should take a parameter which becomes part of the log metadata and is
used to order the logs. Again this also depends on how 1073 does it for file
logs.
> Add interface for generic Write Ahead Logging mechanisms
> --------------------------------------------------------
>
> Key: HDFS-1580
> URL: https://issues.apache.org/jira/browse/HDFS-1580
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Ivan Kelly
> Attachments: generic_wal_iface.pdf, generic_wal_iface.pdf,
> generic_wal_iface.txt
>
>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
