Multiple zookeeper clusters with data partitioning (sharding) ? Regards, Dima Gutzeit.
Sent from my iPhone On 3 באוק 2011, at 20:25, Ted Dunning <[email protected]> wrote: > It definitely can be done, but what if your application grows by 10x? > > On Mon, Oct 3, 2011 at 10:56 AM, Dima Gutzeit > <[email protected]>wrote: > >> Very nice analysis at the link, thanks. >> >> Assuming my nodes are around the same size ~100 bytes, ensemble of 5 >> servers and 10 clients can easily support the required ~20,000 >> operations per second, no ? >> >> I will run a smoke-test to prove it ofcourse ... >> >> Regards, >> Dima Gutzeit. >> >> Sent from my iPhone >> >> On 3 באוק 2011, at 19:41, "Fournier, Camille F." >> <[email protected]> wrote: >> >>> It's pretty easy to set up a zk-smoketest to simulate what you are doing. >> We can't answer this question without knowing how big the data you're >> writing, etc etc. I would recommend testing it out yourself on realistic >> data sizes. >>> >>> >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ServiceLatencyOverview >>> >>> -----Original Message----- >>> From: Dima Gutzeit [mailto:[email protected]] >>> Sent: Monday, October 03, 2011 1:36 PM >>> To: [email protected] >>> Subject: Re: ZooKeeper performance >>> >>> Inline. >>> >>> Regards, >>> Dima Gutzeit. >>> >>> Sent from my iPhone >>> >>> On 3 באוק 2011, at 19:31, Ted Dunning <[email protected]> wrote: >>> >>>> Questions in-line >>>> >>>> On Mon, Oct 3, 2011 at 10:19 AM, Dima Gutzeit >>>> <[email protected]>wrote: >>>> >>>>> Hi, >>>>> >>>>> I have some performance related question. >>>>> >>>>> I am running a cluster of 5 zookeeper machines, each one has dual quad >>>>> core Xeon 2.5 Ghz, 8 gb RAM. >>>>> >>>>> I want achieve the following numbers: >>>>> >>>>> 10 clients producing in total (at peaks): >>>>> >>>> >>>> Each producing this? >>> >>> Total. >>>> >>>> >>>>> 3K add nodes, 3K delete nodes and 10K watches. Per second. >>>>> >>>> >>>> What do you mean by 10K watches? 10,000 watch notifications? >>> >>> Yes. >>>> >>>> Does those numbers make any sense ? >>>>> >>>> >>>> This is a bit high. It sounds like you are trying to use ZK as a >> message >>>> bus rather than a coordination service. You can do this, but the >> throughput >>>> you can achieve is limited. If you want higher throughput, it is better >> to >>>> have ZK coordinate a higher performance messaging system such as Kafka. >>
