On Mon, Feb 25, 2013 at 4:23 PM, Trevor Bernard <trevor.bern...@gmail.com> wrote:
> All I want to do is send work upstream reliably and have the > ventilator be notified by the sink through an acknowledgement if the > batch was successful or not so I can take appropriate action. > > How would you suggest I go about this? The ventilator can tell the sink directly how many jobs to expect. If there's any missing after a timeout, that means a worker died. The sink would then tell the client "Failed" and the client could decide to retry, or cancel. This handles the probably most common failure, which is crashed workers. If you want to handle workers that get into endless loops but appear to be alive, you would have to add heartbeating and it all gets more complex. I'd not go there in a first design. Trick is to make the simplest possible design you can, break it with simulated failures, and solve those. The common trap is to over-design your first architecture. A reliable pipeline was one of the patterns I'd meant to build but didn't get around to. Sorry about that. -Pieter _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
