Thanks for your advice Ivan. It's very helpful. Actually, I am experimenting write a In Memory Batch Store for write-behind to reduce the load of DB and found that BookKeeper seems exactly what I need for crash recovery. Any suggestions about the write-behind approach ?
Best Regards, Stone On Wed, May 30, 2012 at 7:13 PM, Ivan Kelly <[email protected]> wrote: > Hi Stone, > > Currently the bookkeeper API only provides the primatives, and > developers need to add their own glue code on top of this. > > It may be worth your while to take a look at the WAL implementation we > did for HDFS[1]. This has "named" ledgers, and allow supports the > checkpointing. > > A "editlog" is a sequence of ledgers, which the journal manager > maintains in zookeeper. When a checkpoint is made, the current ledger > is closed, and a new one is created. HDFS maintains it's own > transaction ids so that it can removed old ledgers which are no longer > needed after checkpointing. > > We have been thinking about pulling out this abstract to make it > easier for general use, but not much progress has been made on this > yet. > > Regards > Ivan > > > [1] > https://github.com/apache/hadoop-common/tree/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal > >
