Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-22 Thread Doron Somech
Eventually the idea is to create epoll like API (zpoll?) to be used when polling on multiple sockets (which will create one file descriptor for the zpoll object) and to stop using the zmq_poll (only relevant for thread safe sockets). Example of the API: void* zmq_zpoll(); int

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-22 Thread Joe McIlvain
I thought one of the major motivations for sharing a socket among threads was to avoid the (anti-)pattern of creating a socket for each request in a multithreaded environment and worrying about running out of file descriptors. It sounds like this new strategy would avoid allocations of more zmq

[zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
It looks like the recently added code for supporting thread safe sockets does not support returning the signalling FD for a socket that is marked thread safe. How would these socket types be used in a polling reactor then? ___ zeromq-dev mailing list

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
Specifically, something like epoll or kqueues? On Sat, Feb 21, 2015 at 9:27 AM, Thomas Rodgers rodg...@twrodgers.com wrote: It looks like the recently added code for supporting thread safe sockets does not support returning the signalling FD for a socket that is marked thread safe. How would

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Doron Somech
Currently the thread safe sockets doesn't support polling on multiple sockets as stated in the PR: https://github.com/zeromq/libzmq/pull/1349 I'm planning to solve that by epoll like API where you create a poll and then add sockets to it. The poll type will create a file descriptor that will be

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Doron Somech
You will be able to create a signaler which is a file descriptor, associate it with the socket and add it to the epoll or kqueue On Sat, Feb 21, 2015 at 5:32 PM, Thomas Rodgers rodg...@twrodgers.com wrote: Specifically, something like epoll or kqueues? On Sat, Feb 21, 2015 at 9:27 AM, Thomas

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
I will hold off until you've fleshed out those details, but it sounds like the API for getting the signaling FD will be different from what everything else uses currently, which means there will be a class of socket types that I will need to special case to support in my bindings. Is there anyway

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
s/comdition/condition/ On Sat, Feb 21, 2015 at 10:45 AM, Thomas Rodgers rodg...@twrodgers.com wrote: Which all brings us to my next question - Can we just move on to -std=c++11 for future libzmq versions? The big 3 compilers (well mostly, Microsoft still presents a few issues) support C++11

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Doron Somech
Destroying a socket is still not thread safe, so the user still need to synchronize that On Feb 21, 2015 6:36 PM, Bjorn Reese bre...@mail1.stofanet.dk wrote: On 02/21/2015 04:44 PM, Doron Somech wrote: (https://github.com/zeromq/libzmq/blob/master/src/mailbox_safe.hpp) have I had a quick

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Pieter Hintjens
On Sat, Feb 21, 2015 at 5:45 PM, Thomas Rodgers rodg...@twrodgers.com wrote: Can we just move on to -std=c++11 for future libzmq versions? The big 3 compilers (well mostly, Microsoft still presents a few issues) support C++11 at this point. I'd guess Yes, though it's worth asking the list.

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Doron Somech
The one need to dispose it. My plan was to support zmq_poll by create signaler on each call. On Feb 21, 2015 6:21 PM, Thomas Rodgers rodg...@twrodgers.com wrote: I will hold off until you've fleshed out those details, but it sounds like the API for getting the signaling FD will be different

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Bjorn Reese
On 02/21/2015 04:44 PM, Doron Somech wrote: (https://github.com/zeromq/libzmq/blob/master/src/mailbox_safe.hpp) have I had a quick look at this class... The workaround in the destructor is not thread-safe. Another thread may enter and wait between the sync-unlock() and the end of the

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
Which all brings us to my next question - Can we just move on to -std=c++11 for future libzmq versions? The big 3 compilers (well mostly, Microsoft still presents a few issues) support C++11 at this point. Many of the issues below would just 'go away' with the use of std::mutex,

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
in addition to not being exception safe, all of the explicit entry and early exit from the mutex is a fully loaded footgun for some future developer working on this code. RAII is a core idiom of C++ and scoped locks/guards are the only sensible and safe way to write this sort of code so that will

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread André Caron
Hi all, Be careful with requiring C++11 constructs. Some environments may need to use older compilers. For example, building PyZMQ from source for Python 2.7 (and some early Python 3 releases) requires building with VS2008, which does not support C++11. Cheers, André On Sat, Feb 21, 2015 at

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Thomas Rodgers
Why would these be not compile-able with newer versions of VC (I believe VS2008 Goes EOL in July of this year)? Note, changing the libzmq to build with -std=c++11 would not change the C89 compatibility of it's public API. On Saturday, February 21, 2015, André Caron andre.l.ca...@gmail.com wrote:

Re: [zeromq-dev] thread safe sockets and ZMQ_FD

2015-02-21 Thread Doron Somech
You are right. I will change it to use scoped lock. On Sat, Feb 21, 2015 at 6:36 PM, Bjorn Reese bre...@mail1.stofanet.dk wrote: On 02/21/2015 04:44 PM, Doron Somech wrote: (https://github.com/zeromq/libzmq/blob/master/src/mailbox_safe.hpp) have I had a quick look at this class... The