Hi Dmitry, ;) Yes, thank you. I'm also thinking about this approach. I just wanted to know how people use poolboy for that (It seems they do). But even considering your case, imagine that you have quite a big cluster and quite a big amount of rps. In this case you may want to have more than one load balancer to spread the load and be more tolerant to LB failures. Exactly the same situation arises. It's not a problem to implement a pooler in this case on my own. I'm just asking about using poolboy for that.
On Mon, Sep 24, 2012 at 12:08 PM, Dmitry Demeshchuk <[email protected]> wrote: > Why not use load balancer on top of Riak cluster, independently from > clients? If load balancing is sophisticated enough, it can even do a greater > job than just uniformly spreading requests between machines. > > Consider the following situation. You happen to send several requests for a > huge piece of data (each can be just a simple get, or even a complicated > map-reduce query), but the rest of the nodes, or just some of them, receive > much smaller queries. > > What an external load balancer can do is gathering some basic stats from > each machine (like load average, memory and IO consumption, number of rps to > Riak and so on) and balancing the requests according to this data. Another > benefit from this approach is that you can easily add any other Riak client > (Python, Ruby, whatever else) on top of it, and load will be still nicely > distributed between the machines. > > On Mon, Sep 24, 2012 at 11:38 AM, Yuri Lukyanov <[email protected]> wrote: >> >> Hi, >> >> I'm trying to use poolboy to organize a pool of connections to riak >> nodes with protobuf client. >> >> This has come from the suggestion here: >> >> http://lists.basho.com/pipermail/riak-users_lists.basho.com/2012-September/009346.html >> >> The quote: >> "Using Poolboy is convenient because it comes as a dependency of >> riak_core. >> >> If you use Poolboy, you'll have to modify riakc_pb_socket slightly to >> account for the way poolboy initializes connections (add a start_link/1), >> or create a simple module to pass the initialization from poolboy to >> riakc_pb_socket." >> >> What I'm pondering about is what would be a convenient way to start >> riakc_pb_socket clients in a pool. >> For example, if I have 10 riak nodes in a cluster and want to open a >> single connection to each of them (using riakc_pb_socket), I need one >> poolboy worker per one riakc_pb_socket instance. In this case, I need >> a host/port of a single riak node for each >> riakc_pb_socket:start_link(). So I use WorkerArgs in >> poolboy:start_link/3 to pass a list of riak nodes. And here comes a >> confusion. poolboy passes WorkerArgs to each of its workers, so each >> worker gets the list of riak nodes. Now I need to somehow choose a >> node, maybe like proposed here: >> https://github.com/basho/riak-erlang-client/pull/45/files >> But I don't think it's a good solution. >> >> Maybe my idea of using riakc_pb_socket together with poolboy is not >> convenient? Maybe I should start a pool per each riak node? Maybe >> poolboy is not that convenient in this case as it seems? >> >> Would you share your experiences and thoughts on the subject? >> >> Thank you. >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > -- > Best regards, > Dmitry Demeshchuk _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
