Re: [zeromq-dev] Feedback on new PATCH socket

2011-05-08 Thread Martin Sustrik
On 05/08/2011 07:25 AM, Fabien Ninoles wrote: I'm not sure about how 190 can be fixed by separating the semantic of the ROUTER from the req/rep socket; propagating a disconnection from one edge to another could be quite hard and probably hardly possible if you have any fairqueuing in-between.

[zeromq-dev] HWM default

2011-05-08 Thread Martin Sustrik
Hi all, I though of changing the default value for HWM to say 1000 in 0MQ/3.0. The rationale being that unlimited buffer is a stability threat. In other words, default socket when overloaded or DoS'ed will run out of memory and crash the application rather than behave decently. So, I think,

[zeromq-dev] Outstanding issues on zeromq

2011-05-08 Thread Pieter Hintjens
Hi Martin, I noticed that the github issue tracker now allows assignment of issues to individuals. If it's any help, I'm willing to take on investigation reproduction of issues, so feel free to assign issues to me if you think it makes sense. -Pieter

Re: [zeromq-dev] Outstanding issues on zeromq

2011-05-08 Thread Martin Sustrik
On 05/08/2011 09:53 AM, Pieter Hintjens wrote: Hi Martin, I noticed that the github issue tracker now allows assignment of issues to individuals. If it's any help, I'm willing to take on investigation reproduction of issues, so feel free to assign issues to me if you think it makes sense.

Re: [zeromq-dev] HWM default

2011-05-08 Thread Tobias Hintze
I though of changing the default value for HWM to say 1000 in 0MQ/3.0. The rationale being that unlimited buffer is a stability threat. In other words, default socket when overloaded or DoS'ed will run out of memory and crash the application rather than behave decently. a HWM measured in

Re: [zeromq-dev] HWM default

2011-05-08 Thread Martin Sustrik
On 05/08/2011 10:15 AM, Tobias Hintze wrote: I though of changing the default value for HWM to say 1000 in 0MQ/3.0. The rationale being that unlimited buffer is a stability threat. In other words, default socket when overloaded or DoS'ed will run out of memory and crash the application rather

Re: [zeromq-dev] HWM default

2011-05-08 Thread Pieter Hintjens
On Sun, May 8, 2011 at 8:15 AM, Martin Sustrik sust...@250bpm.com wrote: I though of changing the default value for HWM to say 1000 in 0MQ/3.0. It's a good idea but IMO it should be done as a context configuration option. In czmq I've added the ability to configure a context (e.g. to specify

Re: [zeromq-dev] HWM default

2011-05-08 Thread Tobias Hintze
It's a good idea but IMO it should be done as a context configuration option. do you mean as a _default_ when no limit for the socket is specified? th -- Tobias Hintze ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org

Re: [zeromq-dev] HWM default

2011-05-08 Thread Pieter Hintjens
On Sun, May 8, 2011 at 10:34 AM, Tobias Hintze th-0mq-j...@thzn.de wrote: It's a good idea but IMO it should be done as a context configuration option. do you mean as a _default_ when no limit for the socket is specified? Yes. -Pieter ___

Re: [zeromq-dev] Feedback on new PATCH socket

2011-05-08 Thread Fabien Ninoles
This won't work. The peer can be a device rather than an endpoint. In such case you should expect to get arbitrary number of responses from a single connection. My bad... The code should loop on the RCVMORE condition and send a full message. Also, take note that it only discard a socket

Re: [zeromq-dev] HWM default

2011-05-08 Thread Andrew Hume
having fixed HWM also consititutes a stability threat, namely that of deadlocks. and these are often far harder to diagnose than gradually increasing memory footprints. if dos is your threat model, then i think you might need more stuff anyway. (such as autoshed when the HWM is reached) or

Re: [zeromq-dev] Feedback on new PATCH socket

2011-05-08 Thread Fabien Ninoles
However, one way to be more transparent is to mark a frame as being added to the original message instead of using the empty delimiter. This will allow REQ/REP to work directly without change.  The flag need to be pass between sockets however (just like the MORE flag) and so this is

Re: [zeromq-dev] HWM default

2011-05-08 Thread Joshua Foster
I would prefer that the HWM stay as the default. The application needs to be built with the HWM in mind. If they don't explicitly set the value, they are probably not considering it. There is also the challenge of knowing the proper size that the HWM needs to be set based on the size of the

Re: [zeromq-dev] Feedback on new PATCH socket

2011-05-08 Thread Fabien Ninoles
OK, I give a second look to the collector pattern I wrote and build a basic prototype using a variation on the LRU queue. I would send the proto once I get access to a computer where I can test it (my N900 doesn't have a pyzmq package and the debian package depends on a more recent toolchain)

Re: [zeromq-dev] HWM default

2011-05-08 Thread Paul Colomiets
I should second Andrew Hume. The HWM is really easy way to get deadlock for novice. And if you are proficient with zeromq, you know when to set HWM and which value suites your usage. Also should remind that deadlock probability was the reason why my shutdown proposal was rejected. On Sun, May 8,

Re: [zeromq-dev] Feedback on new PATCH socket

2011-05-08 Thread Martin Sustrik
Hi Fabien, This won't work. The peer can be a device rather than an endpoint. In such case you should expect to get arbitrary number of responses from a single connection. My bad... The code should loop on the RCVMORE condition and send a full message. Also, take note that it only discard

Re: [zeromq-dev] HWM default

2011-05-08 Thread Martin Sustrik
On 05/09/2011 03:49 AM, Paul Colomiets wrote: I should second Andrew Hume. The HWM is really easy way to get deadlock for novice. And if you are proficient with zeromq, you know when to set HWM and which value suites your usage. Also should remind that deadlock probability was the reason why