[
https://issues.apache.org/jira/browse/ZOOKEEPER-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011540#comment-13011540
]
Mahadev konar commented on ZOOKEEPER-1016:
------------------------------------------
ok, I have been thinking abt this a little more and had a discussion with
Suresh on what there use case might look like at the NN.
So here is something concrete:
The problem: Provide an api for resource locking which can store the
transacition log files for NN, allowing the resource to be fenced and allowing
some others to read the resource.
The filesystem would look like bk://someimportantstuff:port/resource_path
resource_path is the path to the resource you want to lock.
Lets say you want to use /nn1/transactions/ as the resource.
The user would do:
FileSystem bkfs = new Filesystem(bk://someimportantstuff:port/nn1/transactions);
you can do bkfs.create(lock) to lock the resource.
This would create a znode /nn1/transactions/lock) (owner is ephemeral) and who
ever is able to create owner node would own the resouce /nn1/transactions.
now you can do bkfs.create(txnlog1),
that would check to see if who is the owner for resource /nn1/transactions/ and
would only allow the owner to create files in the directory /nn1/transacitons.
This allows fencing of resource /nn1/transactions/ and disallows backup NN from
grabbing the resource while NN is running.
Reads are allowed to the resource even if you are not the owner of resource.
This allows back up namenode to read the files that are being written to. All
of the above might look a little jumbled up but Ill put up a clean design for
this.
The important thing is we need to define the problem statement we are solving.
In the design I described, it is a txn logging resource that can be fenced
across different NN's and can change ownership.
Ill update with better problem statement and design doc.
> TeaKeeper: Hot standby support using bookkeeper
> -----------------------------------------------
>
> Key: ZOOKEEPER-1016
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1016
> Project: ZooKeeper
> Issue Type: Improvement
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Priority: Minor
> Attachments: tledger.pdf, tledger.pdf
>
>
> Currently Bookkeeper provides functionality for cold backups. If the entity
> logging to bookkeeper fails, its replacement must recover the ledgers which
> had been used for backup before becoming available. This is acceptable in
> some cases, such as HBase Wals where a small delay in recovery only results
> in a small percentage of data being unavailable.
> However, systems such as the HDFS namenode, this delay can be unacceptable,
> such as cases where data is being served to customers. Secondary namenodes
> should be ready to go the instant the primary goes down.
> TeaKeeper proposes a wrapper library around Bookkeeper providing T-Junction
> like functionality for logging. It also provides for primary/secondary
> election and automated hot failover.
> HDFS namenode is primary target of this work.
> The attached design doc contains more details.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
