Re: [Discussion] efficient metadata accesses for the applications using bookkeeper

2013-01-21 Thread Ivan Kelly
On Fri, Jan 18, 2013 at 11:09:39PM -0800, Sijie Guo wrote: 1) performance concern: too many metadata accesses, hitting the bottleneck of the system. Part of this is the price you pay for having an API. We could have a completely integrated system, which would reduce everything to the absolute

Re: [Discussion] efficient metadata accesses for the applications using bookkeeper

2013-01-19 Thread Jiannan Wang
I can't agree more with Sijie that issues this discussion from an application's perspective. I roughly know your idea but I have a question: so you mean there will be a implicit mapping from application metadata to the ledger id, right? then if there are many ledger id, is it the bookie server's

Re: [Discussion] efficient metadata accesses for the applications using bookkeeper

2013-01-19 Thread Sijie Guo
if application runs in rolling ledgers, one application entity (e.g a Hedwig topic) will use several ledgers. It is a one-to-multiple mapping. for a one-to-multiple mapping, since application doesn't know how the ledger id is generated, application has to record this mapping as its metadata as

[Discussion] efficient metadata accesses for the applications using bookkeeper

2013-01-18 Thread Sijie Guo
*** this thread is splited from previous thread, focusing on the metadata accesses discussion for applications using bookkeeper. I will try to make everything clearly and add the details (although I assumed you should have knew the details before). If I don't make it clearly, please forgive my