[zeromq-dev] design question

2018-10-26 Thread SIMON BABY
Hi, I have a design requirement. Can anyone please help. I have multiple connection from client to server and my client sends a very large file, say 4GB size to server. The file is send by splitting into chunks and sending each chunk to the different channels (sockets created through multiple in

[zeromq-dev] Design Question

2013-01-18 Thread Nishant Mittal
I have 3 workers (REP) connected to a DLR socket. each worker takes a msg, processes and responds... however, as the DLR socket fair-queues.. requests are sent like this... 1st -> 1st worker 2nd -> 2nd worker 3rd -> 3rd worker 4th -> 1st worker problem is if the 1st worker is still busy with the 1

Re: [zeromq-dev] design question

2018-10-26 Thread SIMON BABY
corrected mail: Hi, I have a design requirement. Can anyone please help. I have multiple connection from client to server and my client sends a very large file, say 4GB size to server. The file is send by splitting into chunks and sending each chunk to the different channels (sockets created th

Re: [zeromq-dev] design question

2018-10-27 Thread James Gatannah
On Sat, Oct 27, 2018 at 5:04 SIMON BABY wrote: > Hi, > > I have a design requirement. Can anyone please help. > > I have multiple connection from client to server and my client sends a very > large file, say 4GB size to server. The file is send by splitting into > chunks and sending each chunk t

Re: [zeromq-dev] Design Question

2013-01-18 Thread Andy Ballingall TF
On 18 January 2013 15:14, Nishant Mittal wrote: > I have 3 workers (REP) connected to a DLR socket. each worker takes a msg, > processes and responds... however, as the DLR socket fair-queues.. requests > are sent like this... > 1st -> 1st worker > 2nd -> 2nd worker > 3rd -> 3rd worker > 4th -> 1

Re: [zeromq-dev] Design Question

2013-01-18 Thread Nishant Mittal
aah, i remember that now.. i'll look into it again. my hope was DLR would "try" to fair queue but if the worker was not accepting any more msgs.. DLR would try the other worker.. but its not doing that. thanks On Fri, Jan 18, 2013 at 10:49 AM, Andy Ballingall TF < balling...@thefoundry.co.uk> wro

Re: [zeromq-dev] Design Question

2013-01-18 Thread Stuart Brandt
Your DLR has a send_hwm too, so it's going fair queue against its outbound queues (which might not be full despite receiver not accepting more). The mentioned load-balancing-message-broker or one of the reliable request-response patterns in section 4 will probably serve you better than trying

Re: [zeromq-dev] Design Question

2013-01-18 Thread Nishant Mittal
i see.. that might explain why DLR is blocking.. so do you think if i set the SND buffer to 0 on the dealer.. this will work? effectively giving me a load balanced broker? On Fri, Jan 18, 2013 at 11:10 AM, Stuart Brandt wrote: > Your DLR has a send_hwm too, so it's going fair queue against its

Re: [zeromq-dev] Design Question

2013-01-18 Thread Harsh Doshi
0 would mean no limit. And it would either block or drop messages (I think). You'll need a router socket On Fri, Jan 18, 2013 at 9:25 AM, Nishant Mittal wrote: > i see.. that might explain why DLR is blocking.. so do you think if i set > the SND buffer to 0 on the dealer.. this will work? effec

Re: [zeromq-dev] Design Question

2013-01-18 Thread Harsh Doshi
Btw I meant, it would block or drop messages if you use a HWM that is low (but not 0). 0 should give you unlimited limit, but you will eventually block or drop messages (tcp buffers filling up, etc). Better load balancing will be achieved by using a router socket On Fri, Jan 18, 2013 at 10:53 AM,

Re: [zeromq-dev] Design Question

2013-01-18 Thread Nishant Mittal
thanks Harsh On Fri, Jan 18, 2013 at 2:42 PM, Harsh Doshi wrote: > > Btw I meant, it would block or drop messages if you use a HWM that is low > (but not 0). 0 should give you unlimited limit, but you will eventually > block or drop messages (tcp buffers filling up, etc). > Better load balancin

Re: [zeromq-dev] Design Question

2013-01-18 Thread Nishant Mittal
so to implement this the broker would keep a list of workers.. what if a worker crashes and the router tries to send to it.. the guide mentions that the msg would be silently dropped. can I set option SET_ROUTER_MANDATORY so the send command returns an error and I try another connected worker? On

[zeromq-dev] Design question: python interface for zmq_poll

2010-02-16 Thread Brian Granger
Hi, My Python bindings have support for zmq_poll now: http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 This handles both 0MQ sockets as well as any Python object that has a fileno() function (sockets, files, etc.) Currently, the interface is similar to the underlying zmq_poll fun

[zeromq-dev] Design question for multithreaded XREP server

2010-12-19 Thread Dr Tune
Hi all, Firstly, ZMQ + protocol buffers = awesomeness, thanks! Now I can mix Java and c++ and it all Just Works. Secondly; I have a server which has a single thread listening on an XREP socket and distributing incoming requests to a pool of worker threads which process msgs and return a result. My

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-16 Thread Martin Sustrik
Brian, > My Python bindings have support for zmq_poll now: > > http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 > > This handles both 0MQ sockets as well as any Python object that has a fileno() > function (sockets, files, etc.) Nice! AFAIK this is the first binding (aside of C a

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-16 Thread Brian Granger
Martin, >> This handles both 0MQ sockets as well as any Python object that has a >> fileno() >> function (sockets, files, etc.) > > Nice! AFAIK this is the first binding (aside of C and C++) that's able to > combine polling for BSD and 0MQ sockets. Cool, wasn't too difficult once I understood wha

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-21 Thread Carlos A. Rocha
> My Python bindings have support for zmq_poll now: > > http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 > > This handles both 0MQ sockets as well as any Python object that has a fileno() > function (sockets, files, etc.) Awesome! Great work. Just what I was looking for. > I am thi

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-21 Thread Brian Granger
Carlos, I went ahead and implemented the select.poll interface as well as select.select. The implementation was dead simple, so I didn't see any issue supporting both. But I agree, the select.poll interface is probably the nicer of the two. We are still doing some testing and tweaking of the AP

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-21 Thread ntupel
On Tue, Feb 16, 2010 at 10:08 PM, Martin Sustrik wrote: > Brian, > >> My Python bindings have support for zmq_poll now: >> >> http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 >> >> This handles both 0MQ sockets as well as any Python object that has a >> fileno() >> function (socket

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-21 Thread Martin Sustrik
ntu...@googlemail.com wrote: > On Tue, Feb 16, 2010 at 10:08 PM, Martin Sustrik wrote: >> Brian, >> >>> My Python bindings have support for zmq_poll now: >>> >>> http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 >>> >>> This handles both 0MQ sockets as well as any Python object that

Re: [zeromq-dev] Design question: python interface for zmq_poll

2010-02-21 Thread Martin Sustrik
Martin Sustrik wrote: > ntu...@googlemail.com wrote: >> On Tue, Feb 16, 2010 at 10:08 PM, Martin Sustrik wrote: >>> Brian, >>> My Python bindings have support for zmq_poll now: http://github.com/ellisonbg/pyzmq/blob/master/zmq/_zmq.pyx#L436 This handles both 0MQ sockets as

Re: [zeromq-dev] Design question for multithreaded XREP server

2010-12-19 Thread kasicass
client1/client2/... ==> xreply/xrequest(zmq_device) ==> multi-worker(threads) I think zmq_device(ZMQ_QUQUE, xreply, xrequest) is what your need. On 2010-12-19 9:10, Dr Tune wrote: Hi all, Firstly, ZMQ + protocol buffers = awesomeness, thanks! Now I can mix Java and c++ and it all Just Works