Re: [zeromq-dev] Python bindings strings and bytes

2015-11-25 Thread André Caron
> From the binding perspective this would be easiest. From a user perspective it isn't If you mean that this makes it harder for the application developer using the binding because they have to figure out which encoding name to pass to .decode, then I disagree. Nothing is worse than a binding tha

Re: [zeromq-dev] Are bindings supposed to set linger on implicit close/destroy?

2015-03-19 Thread André Caron
FWIW, I've been bitten several times by a close without linger and I now explicitly set the linger on each socket. It's quite annoying to have to set `.close(linger=1)` everywhere. It also means I can't use the "contextlib.closing" context manager to automatically close the socket and I have to w

Re: [zeromq-dev] adding other file descriptors to zmq::pollitem_t array

2015-03-03 Thread André Caron
AFAICT, zmq_poll supports any native fd by design. What fds are accepted is system dependant. For example, on Windows, it only accepts sockets (but not regular files like stdin). On other platforms, YMMV I guess. Cheers, André On Tue, Mar 3, 2015 at 4:51 PM, alex. wrote: > I am building a t

Re: [zeromq-dev] STREAM sockets merge two chunks of data on a single recv

2015-03-03 Thread André Caron
Stream sockets are basically glorified TCP sockets. They don't have any framing, which means that there is no way to guarantee the receiving peer will receive the same number of chunks you send. Some things that can happen: - peer receives chunks as sent (likely if chunks are small AND there is e

Re: [zeromq-dev] "thread safe" sockets and ZMQ_FD

2015-02-21 Thread André Caron
Hi all, Be careful with requiring C++11 constructs. Some environments may need to use older compilers. For example, building PyZMQ from source for Python 2.7 (and some early Python 3 releases) requires building with VS2008, which does not support C++11. Cheers, André On Sat, Feb 21, 2015 at 1

Re: [zeromq-dev] Notes from a hackathon

2015-02-07 Thread André Caron
Hi all, Even if the topic seems rather closed, I'd like to contribute a few things :-) There is a lot of discussion about thread safety of sockets from the sender's perspective. The reader's perspective is even more interesting IMO. For example, if I zmq_poll() to wait for input, then zmq_recv(

Re: [zeromq-dev] About CURVE and ROUTER sockets

2015-01-19 Thread André Caron
authority is the Internet Corporation (or other central servers). > >> Usually when people talk about cryptography they mean this kind of > >> cryptography. > >> > >> An alternative framework is web-of-trust. Alice knows Bob and Bob > >> knows Charlie

Re: [zeromq-dev] About CURVE and ROUTER sockets

2015-01-18 Thread André Caron
Guide. > Router-to-router topologies tend to be nasty. I don't use them, and > don't recommend them. > > On Sun, Jan 18, 2015 at 5:57 PM, André Caron > wrote: > > Hi all, > > > > I've been building an example of a router-router setup in which a

[zeromq-dev] About CURVE and ROUTER sockets

2015-01-18 Thread André Caron
Hi all, I've been building an example of a router-router setup in which all nodes dynamically discover each other. This currently works like a charm. Now, I'm trying to add curve support for this application, which is proving a bit confusing despite the API's apparent simplicity. Basically, all

[zeromq-dev] Purpose of empty frame in MDP 0.1

2015-01-13 Thread André Caron
Hi all, In the MDP 0.1 (RFC 7) spec, all messages begin with a 0-length frame. In the MDP 0.2 spec (RFC 18), this empty frame is no longer accepted so I started wondering why it was there in the original spec and if removing it breaks anything. I always thought this empty frame was used as a del

Re: [zeromq-dev] Router socket reconnection failure

2014-12-16 Thread André Caron
Hi Justin, Thanks for the info :-) Just read that thread, but the case seems slightly different: all my nodes use a persistent identity, which I set immediately after creating the socket and thus before any bind or connect operation. However, I just tried having P2 restart with a new identity an

Re: [zeromq-dev] using majordomo broker with asynchronous clients

2014-12-11 Thread André Caron
14 at 8:05 AM, Vishal Ahuja wrote: > Sure, but when I have 3 parallel clients (300k messages), I am observing > packet loss. > > Sincerely, > > Vishal > > On Thu, Dec 11, 2014 at 5:43 PM, André Caron > wrote: > >> Throughput is bound by multiple factors. In your ca

Re: [zeromq-dev] using majordomo broker with asynchronous clients

2014-12-11 Thread André Caron
Throughput is bound by multiple factors. In your case, my guess is that your bottleneck is the throughput of a single TCP socket on your system. I doubt that you can reach millions of messages per second with only 3 clients. If you add more clients, you should see the server's throughput increas

Re: [zeromq-dev] EOF on file transfer

2014-12-10 Thread André Caron
Hi there, Maybe it's just me, but you code seems quite complicated for the task at hand. I don't understand why you need to seek at all since successive reads on the same file handle remember the file pointer. I got this to work locally with exactlythe following code: def sendfile(path, chu

Re: [zeromq-dev] PUB/SUB and TLS/SSL

2014-11-19 Thread André Caron
I prefer Curve to SSL/TLS because it's simpler to use and administrate overall: no need to purchase certificates or, worse, become your own CA. However, if you really must use SSL/TLS because some higher power is imposing it on you, you may want to look at SSLTunnel[1], which looks designed for thi

Re: [zeromq-dev] Closed dealer socket may cause context termination to hang

2014-11-07 Thread André Caron
Thanks for the quick replies and getting this included in the docs! Cheers, André On Thu, Nov 6, 2014 at 9:38 AM, Michi Henning wrote: > > On 7 Nov 2014, at 0:33 , Pieter Hintjens wrote: > > > Well, I used the term "insane" in the man page. > > I was thinking of "wrong", but "insane" will do

[zeromq-dev] Closed dealer socket may cause context termination to hang

2014-11-05 Thread André Caron
Hi there, I've been bit by this nasty little bugger and I thought I'd share. In this little snippet, terminating the context hangs unless the connection was established, even though we're closing the socket properly before terminating the context. import zmq c = zmq.Context() s = c.s

Re: [zeromq-dev] PyZMQ with Libsodium on Windows?

2014-10-11 Thread André Caron
43 PM, André Caron wrote: > OK. I got it working. > > The entire procedure seems to be the following: > 1) add a stub in libsodium and change use of "static inline" > for "static" to prevent VS 2008's compiler from choking; > 2) change the "DLL_

Re: [zeromq-dev] PyZMQ with Libsodium on Windows?

2014-10-11 Thread André Caron
ing to accept a pull request to patch libzmq (#2) and then another to patch PyZMQ (#3)? Thanks, André On Sat, Oct 11, 2014 at 11:14 PM, André Caron wrote: > Thanks! > > I've now modified the "setup.py" file so that it builds libsodium on > Windows. However, I'm

Re: [zeromq-dev] PyZMQ with Libsodium on Windows?

2014-10-11 Thread André Caron
at, Oct 11, 2014 at 8:58 PM, MinRK wrote: > PyZMQ builds libzmq (and libsodium) as Python extensions, using the > compiler associated with Python (hence the need for VC9, the compiler > associated with Python.org 2.7). They are built with `setup.py build_ext`. > > -MinRK > > O

Re: [zeromq-dev] PyZMQ with Libsodium on Windows?

2014-10-11 Thread André Caron
9 October 2014 17:47, MinRK wrote: > >>> It's not bundled simply because I couldn't build it on Windows. If you > can come up with a simple fix for building bundled libsodium, then I would > bundle libsodium on Windows. > >>> > >>>

[zeromq-dev] PyZMQ with Libsodium on Windows?

2014-10-09 Thread André Caron
Hi there! I'm trying to secure some PyZMQ communications with the curve security features. The code seems straightforward, but when I call `zmq.auth.create_certificates()` to generate some keys, I get the following exception: zmq.error.ZMQError: Not supported This seems to be due to the abs

Re: [zeromq-dev] First draft for PPPP rfc (paranoid pirate publish protocol)

2014-08-28 Thread André Caron
Interesting spec :-) The one thing I might be concerned about is the high volume of bandwidth required by ACK messages. The're a section in the guide that does a quick computation of the bandwith requirements for heartbeating and the conclusion is to treat any message from a peer as a heartbeat b

Re: [zeromq-dev] zeromq with protobuf segmentation fault while parsing in c++

2014-07-01 Thread André Caron
Hi Nayab, This snippet looks suspect in your example code. It seems like you are using an uninitialized pointer: > *Scan *pb_laser_msg_rcv; // No allocation.* *> pb_laser_msg_rcv->ParseFromString(ls_msg_str); // Using pointer here is undefined behavior.* Try allocating a "Scan" object using ne

Re: [zeromq-dev] On the performance of ZMQ_STREAM sockets

2014-01-21 Thread André Caron
BTW, I'm going to experiment with a ZeroMQ-based backend infrastructure that supports credit-based flow control (I'm not only interested in the stream sockets). I'll look at that once I've had all my fun with the HTTP stuff :-) André On Tue, Jan 21, 2014 at 10:30 PM, André

[zeromq-dev] On the performance of ZMQ_STREAM sockets

2014-01-21 Thread André Caron
FYI, I just wrote a small HTTP server based on ZMQ_STREAM sockets and an outdated version of http-parser[1] wrapped in some C++ objects with naïve buffering. I profiled it with "ab" and got some interesting results: - ~6K req/sec without HTTP keep alive; and - ~19.5K req/sec with HTTP keep alive.

Re: [zeromq-dev] How should test cases be written?

2014-01-21 Thread André Caron
Hi Goswing, The test suite in libzmq uses "assert" in to check results, which tends to crash the entire test suite that's running in the same program. In the worst case, you get a failure on the first test and you have no idea if the rest of the tests pass. In my experience, getting a partial su

Re: [zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-14 Thread André Caron
you can reopen them on libzmq and > we'll merge them. We then need a test case (or you can add to the > existing tests/test_stream.cpp test case). > > Thanks > Pieter > > On Tue, Jan 14, 2014 at 2:32 AM, André Caron > wrote: > > Provided you accept this patch, how s

Re: [zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-13 Thread André Caron
Provided you accept this patch, how should I proceed in order to get this fix included in the next pyzmq release? Thanks, André On Mon, Jan 13, 2014 at 8:30 PM, André Caron wrote: > Hi Pieter, > > Created issue #48 (https://github.com/zeromq/zeromq4-x/issues/48) and > sent pull

Re: [zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-13 Thread André Caron
really contradictory with what > you're doing, and worth keeping in mind. > > On Mon, Jan 13, 2014 at 4:36 AM, André Caron > wrote: > > Thanks Pieter, > > > > I've spent some time fiddling around with the code in order to submit a > > patch. > > >

Re: [zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-12 Thread André Caron
someone to help > make the patch, or learn enough to make it yourself. > > -Pieter > > On Thu, Jan 9, 2014 at 6:05 AM, André Caron > wrote: > > Hi there! > > > > First and foremost, kudos for all your awesome work on this excellent > > library :-) > > &

Re: [zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-10 Thread André Caron
I've looked into this, but haven't tried it because the ZMQ_EVENT_DISCONNECTED event doesn't provide file descriptor and (I assume IP) address. I have no idea how to map this back to the socket identities... Cheers, André On Fri, Jan 10, 2014 at 2:25 AM, Chris Laws wrote: > In CZMQ and PYZMQ

[zeromq-dev] Notification of peer disconnection for ZMQ_STREAM sockets.

2014-01-08 Thread André Caron
Hi there! First and foremost, kudos for all your awesome work on this excellent library :-) I'm experimenting with ZMQ_STREAM sockets and I'm not sure how to handle disconnection of peers. The man page is pretty clear on how to forcibly disconnect a peer (send 0-length message), but there is no