Hi Suma, I think ZooKeeper can handle thousands of updates, so thousands of jobs can be "launched" at the same time. More jobs can be running at the same time since the number of updates against ZooKeeper is less than the number of jobs. Please free to ask us if you face the scalability or performance issue when you test. We can tackle the issue.
Thanks, - Tsuyoshi On Wed, Feb 11, 2015 at 6:08 PM, Suma Shivaprasad <sumasai.shivapra...@gmail.com> wrote: > Can ZKStateStore scale for large clusters. Any idea on the number of > concurrent jobs that can be supported on top of these ? > > Thanks > Suma > > On Wed, Feb 11, 2015 at 1:45 PM, Karthik Kambatla <ka...@cloudera.com> > wrote: >> >> We recommend ZK-store, particularly if you plan to deploy multiple >> ResourceManagers with failover. ZK-store ensures a single RM has write >> access and thus is better protected against split-brain cases where both RMs >> think they are active. >> >> On Tue, Feb 10, 2015 at 9:59 PM, Suma Shivaprasad >> <sumasai.shivapra...@gmail.com> wrote: >>> >>> We are planning to deploy Hadoop 2.6.0 with a default configuration to >>> cache 10000 entries in the state store. With a workload of 150-250 >>> concurrent applications at any time , which state store is better to use >>> and for what reasons ? >>> >>> Thanks >>> Suma >> >> >> >> >> -- >> Karthik Kambatla >> Software Engineer, Cloudera Inc. >> -------------------------------------------- >> http://five.sentenc.es >> >
