Whether or not you can use local sessions is a critical aspect: https://issues.apache.org/jira/browse/ZOOKEEPER-1147
Patrick On Fri, Sep 27, 2019 at 2:27 PM Michael Han <[email protected]> wrote: > >> can launch tens of thousands of calls > > Is it possible for you to quantify this in a form of (read and write) > request per second, and the average request payload if it's OK to disclose? > This information is critical on shaping the best scaling solution. > > Without knowing any of ballpark numbers of your system and workload > characteristics, one immediate experiment you could do is to set up > Observer servers, remove quorum servers from serving client traffic by > redirecting all of your client traffic to observer servers. This will at > least scale your concurrent connections linearly with the number of > observers. This will also scale concurrent requests processing capabilities > for read requests (and to a limited extend, for write requests as well), > but request processing scaling is harder and depends on your workload > characteristics. > > > > On Fri, Sep 27, 2019 at 12:07 PM Yue Shen <[email protected]> wrote: > > > Thank you, Jorn. > > > > We don't use Solr. We inherited this architecture from another team, and > we > > don't have time to redesign a new system to scale in 2 months. > > > > As you said, if I were to design it, I would definitely put a queue in > > front of Lambda service, our new design is actually on the way with Kafka > > upfront. However we need to scale it out with the coming holiday > > season before we can roll out the new system, which is just kicked off a > > couple of weeks ago. > > > > At this point, we want to tune ZooKeeper so it can handle 10K concurrent > > calls. Any suggestions? > > > > Thank you, > > Yue > > > > On Fri, Sep 27, 2019 at 10:39 AM Jörn Franke <[email protected]> > wrote: > > > > > Put the Solr request on a SQS queue using your 10k instances and have > 10 > > > or so worker working on the queue to put it in Solr. Having 10k > > connections > > > just because lambda creates that many instances does not make sense for > > no > > > database service. > > > > > > > Am 27.09.2019 um 19:01 schrieb Yue Shen <[email protected]>: > > > > > > > > Dear ZooKeeper users, > > > > > > > > I have a special use case, in which I use AWS lambda service. > > > > > > > > Inside the lambda service logic, it goes to ZooKeeper to check the > > worker > > > > for the data, if exists, connect to the worker endpoint and send the > > > data. > > > > If the worker isn't assigned, the logic will post a new assignment, > and > > > > wait for it to be assigned to a worker. There is a coordinator to > watch > > > the > > > > new assignment and assign tasks. > > > > > > > > My problem comes with AWS Lambda service, which can launch tens of > > > > thousands of calls. When this happens, I found many calls get > timeout. > > > The > > > > active connections to ZooKeeper plateau around 6500. > > > > > > > > BTW, I run ZooKeeper as 3 node ensemble, run on Quorum. > > > > > > > > How can I scale ZooKeeper to support more concurrent connections? > > > > > > > > Thank you, > > > > Yue > > > > > >
