Hi Mudit, Thanks Flavio for showing BookKeeper. Yeah this is another option you can explore in your free time:)
I'd like to introduce it briefly to you, please have a look if you are interested. Documentation available at http://zookeeper.apache.org/bookkeeper/docs/r4.2.2/ Basic terminologies:- - servers called as"bookies", - log are "ledgers", - and each unit of a log (aka record) is a "ledger entry" Bookie server uses filesystem to store the ledgers and their entries(Since it uses filesystem, message size won't be a constraint I guess). Also, it uses ZK for storing the ledger metadata information. Basic operations:- 1) Open a bookkeeper client. 2) Create a ledger - Here it will internally generate id for this ledger and it will be unique. Like ZooKeeper sequential znodes it internally maintains sequence to generate the ids. Upon creating a ledger, a BookKeeper client writes metadata about the ledger to ZK. 3) Write to the ledger - User can add entries(user data) to the ledger. BK guarantees single writer. 4) After write close the ledger. Assume user has created four ledgers, now ledger id looks like L00001, L00002, L00003, L00004. When user tries to create a new ledger, then id will be incremented L00005. Ledger metadata in ZK: /ledgers/L00001 /ledgers/L00002 /ledgers/L00003 /ledgers/L00004 /ledgers/L00005 Now using this sequential ledger znodes present in ZK, one can write the logic of distributed queue. Any queries feel free to ping us happy to help you:) You can reach us bookkeeper-...@zookeeper.apache.org or user mailing id. Regards, Rakesh On Sat, Jun 7, 2014 at 2:32 AM, Flavio Junqueira < fpjunque...@yahoo.com.invalid> wrote: > You may want to have a look at BookKeeper. > > -Flavio > > On 05 Jun 2014, at 16:32, Mudit Verma <mudit.f2004...@gmail.com> wrote: > > > Hi Zookeeper Users, > > > > Lately, I have been working on a research project where I want to use > zookeeper as a distributed logging service. > > > > I want to build a queue on top of zookeeper (also provided in recipes). > > > > What for: > > Intention is to insert some operations performed by different clients in > a distributed queue, and process them lazily at some later point of time. > And I want some ordering between these operations. > > > > Setup: > > 5 physical zookeeper servers > > > > The problem is: > > In my current setup, I am observing a latency of about 13 ms per enqueue > operation (using synchronous create APIs with sequential flags). I want to > significantly reduce this time. The other way could be to use asynchronous > zookeeper calls but I am not sure what can be the side effects. Would it > still be monotonous when used with SEQUENTIAL flag? > > > > For example, a client X created a SEQUENTIAL node Z1 at time t1 using > async create, same client created another SEQUENTIAL node Z2 at time t2 > where t2 > t1. Would the monotonic number associated with Z1 be lesser than > that of Z2? > > > > Your help is much appreciated. > > > > Thanks > > Mudit > > > >
