[
https://issues.apache.org/jira/browse/ZOOKEEPER-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012892#comment-13012892
]
Ivan Kelly commented on ZOOKEEPER-1016:
---------------------------------------
*Pros*
[1] Provide a common interface for developers writing journalling code.
[2] Journals can be inspected using hadoop fs calls.
[3] file:// implementation already exists.
[4] You could also store images in BK.
*Cons*
[5] Developer must know that "lock" is a special file. Implementation must also
know that lock is a special file. This is putting application logic into the
FileSystem logic. It would be better to have a lock() type call on FileSystem.
Also, locking in general won't work with all implementations of FileSystem.
[6] No notifications of newly available ledgers. I know the HDFS guys have said
this isn't important to them, but we want to use this on another project also.
[7] NN is not currently using FileSystem to write it's file:// based files.
This is the biggest issue here. NN uses StorageDirectories and a Storage
implementation. The Storage interface is shared by namenode and datanode. You
could replace this with FileSystem, but you would need to add locking and
VERSION management to the API. Also, it's an enormous amount of work.
Otherwise you could just use FileSystem in namenode, wrapping it around
StorageDirectory. You still need StorageDirectory for managing VERSIONS files
and such. Again though, this is a massive amount of work. FSImage and FSEditLog
would need to be gutted.
It would be simpler to create a BK/ZK implementation of StorageDirectory, but
this would be HDFS specific, so doesn't belong in BookKeeper.
[8] Someone will undoubtedly start logging to hdfs:// for extra super high
availability.
[9] Creates a dependency from bk to hadoop.
*Conclusion*
Given the cons, especially [7], I dont think the effort required for this
solution is justified by the payoff. The changes to namenode would be too
fundamental, getting a patch of that size committed would be a nightmare.
-1 from me.
> TeaKeeper: Hot standby support using bookkeeper
> -----------------------------------------------
>
> Key: ZOOKEEPER-1016
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1016
> Project: ZooKeeper
> Issue Type: Improvement
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Priority: Minor
> Attachments: tledger.pdf, tledger.pdf
>
>
> Currently Bookkeeper provides functionality for cold backups. If the entity
> logging to bookkeeper fails, its replacement must recover the ledgers which
> had been used for backup before becoming available. This is acceptable in
> some cases, such as HBase Wals where a small delay in recovery only results
> in a small percentage of data being unavailable.
> However, systems such as the HDFS namenode, this delay can be unacceptable,
> such as cases where data is being served to customers. Secondary namenodes
> should be ready to go the instant the primary goes down.
> TeaKeeper proposes a wrapper library around Bookkeeper providing T-Junction
> like functionality for logging. It also provides for primary/secondary
> election and automated hot failover.
> HDFS namenode is primary target of this work.
> The attached design doc contains more details.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
