In the diagram, I think you meant for the REQ/REP workers to all be connected to the Worker, not the PUSH/PULL.
If so then I understand the idea. I'll have to think about that a bit more. It sounds similar to the thing the guy was talking about last week on this list. On Fri, Sep 13, 2013 at 8:45 PM, Jason Miller <[email protected]> wrote: > Hi Brian, > > There are a couple of ways of managing this. One way is to do something > like the load-balancing pattern in the ZMQ guide. You will have a > single push/pull socket to get requests from m2 and then will need to do > req/rep from the workers. The workers can reply directly to mongrel2, > so you'll want to do a dummy REP once you get a request: > > +--REQ/REP-->Worker--PUB/SUB--> Mongrel2 > | > Mongrel2 --PUSH/PULL--> Broker +--REQ/REP-->Worker--PUB/SUB--> Mongrel2 > | > +--REQ/REP-->Worker--PUB/SUB--> Mongrel2 > > As a disclaimer, I haven't actually tried the above, but it should work > fine, and is what the ZMQ people suggest for load balancing. > > What I *have* done is this: > > A single process with one thread that reads on the push/pull; then I use > language native queues to distribute to worker threads. You can then > either have a the replies go back directly from the worker threads, or > have another thread that sends all the responses back. Just remmeber > that zmq sockets can't be shared between threads safely, so each worker > will need it's own PUB socket if you want them to send directly to > Mongrel2 -- I didn't know that the first time I treid this, and it didn't > end well :). > > -Jason > > On 16:11 Fri 13 Sep , Brian McQueen wrote: > > I've got handlers running slow long running backend jobs.  They start > > up and return a response and close the connection, and then they waits > > for their slow jobs to finish before they goes back to conn.recv. > > While a slow job is running other handlers are connected to the queues > > and receiving.  I'm seeing that the distribution of the requests > still > > seems to round robin into the busy handler which is not even at > > conn.recv.  The other two handlers are sitting there waiting.  They > > are fast and not busy and waiting, and they will each take a request, > > and then mongrel2 stops like its trying to direct the next request in > a > > round robin fashion to the busy handler, which is not even at > > conn.recv, but instead is busy doing a big backend job. > > What's going on here?  How should I debug this? > > I need the requests to go the the ready handlers and not return to the > > slow handler until its back at conn.recv. > -- the news wire of the 21st century - twitchy.com
