Hello, On 12/10/2014 6:54 PM, Arnaud Kapp wrote: > Hello, > > Sorry it took me a while, but I finally go to test your patch. > My setup that use POLLPRI seems to work properly with your patch applied :). Good. Do you see a performance improvement ? How long have you been using it ? > > Did you submit a PR to get it merged into libzmq master yet? No, not yet. I was waiting for feedback before. And I had another issue with a memory hog in libzmq. Cheers, Francis
> > On Fri, Nov 28, 2014 at 11:35 AM, Francis Le Bourse > <[email protected]> wrote: >> On 11/24/2014 8:08 PM, Arnaud Kapp wrote: >>>> Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq >>>> master >> The first patch for 3.2.4 had an issue in zmq_poll(), I had tried a little >> too aggressive optimization by bypassing the "first_pass" processing. It is >> fixed in the current one. >> The patch for the current head is clean. >> Cheers, >> Francis >> >> >>> Oh okay. This is the commit that added the flag: >>> >>> https://github.com/zeromq/libzmq/commit/779c37abc433cb6595ddeedaf86b280317656bdd >>> >>> libzmq was 4.1 at the time I believe. >>> >>> I'll probably look at it this week-end then :) >>> >>> On Mon, Nov 24, 2014 at 12:10 PM, Francis Le Bourse >>> <[email protected]> wrote: >>>> Hi, >>>> On 11/24/2014 11:35 AM, Arnaud Kapp wrote: >>>>> Hello, >>>>> >>>>> I recently added support for POLLPRI flag. >>>>> It looks like it's not handled in your patch >>>> No, it isn't handled. In which version do you have added this flag ? >>>> Currently, the patch is written for 3.2.4. I'll wait to put it in libzmq >>>> master. >>>>> and that it needs custom >>>>> support. Since there is no test related to this flags you wouldn't >>>>> notice. >>>>> >>>>> I can give it a look if you want. >>>> That would be nice. >>>> >>>> Cheers, >>>> Francis >>>> >>>> >>>>> On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <[email protected]> wrote: >>>>>> I suggest you send the patch to libzmq master, and ensure all test >>>>>> cases pass. Then we can get this into the next version. >>>>>> >>>>>> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse >>>>>> <[email protected]> wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote: >>>>>>>> Oh, ok. Sounds like you have a good candidate for some before/after >>>>>>>> measurement and optimization. Are you going to try to make a patch >>>>>>>> for >>>>>>>> this? >>>>>>> I have a patch candidate for this optimization, the performance >>>>>>> improvement >>>>>>> is very good and it doesn't seem to introduce any new instability. >>>>>>> What is modified: >>>>>>> - zmq_poll(), there is only one poll() now, >>>>>>> - and epoll() from epoll.cpp >>>>>>> Other calls to poll() and select() are left unmodified. >>>>>>> >>>>>>> I woulld be happy to have any feedback. >>>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> Francis >>>>>>>> >>>>>>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse >>>>>>>> <[email protected]> wrote: >>>>>>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote: >>>>>>>>>> A simple optimization is, when you are polling sockets for input, >>>>>>>>>> to >>>>>>>>>> continue reading from an active socket using a non-blocking read. >>>>>>>>>> So >>>>>>>>>> you process all waiting messages on a socket and then only switch >>>>>>>>>> back >>>>>>>>>> to poll when needed. >>>>>>>>> Thank you for you quick reply. >>>>>>>>> >>>>>>>>> Yes, but the question was more about the zmq_poll() internals. >>>>>>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number of >>>>>>>>> times >>>>>>>>> for only a few that will trigger a POLLIN and the relevant >>>>>>>>> information >>>>>>>>> is >>>>>>>>> already known / present in the pollfds array. The performance hit is >>>>>>>>> there. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Francis >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I am looking at a performance issue in zmq, when the number of >>>>>>>>>>> zsockets >>>>>>>>>>> / >>>>>>>>>>> file descriptors becomes large. >>>>>>>>>>> The relevant calls are: >>>>>>>>>>> poll+0x57 >>>>>>>>>>> zmq_poll+0x2e3 >>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>> main+0xb40 >>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>> immediately followed by a loop of >>>>>>>>>>> poll+0x57 >>>>>>>>>>> zmq::signaler_t::wait(int)+0x33 >>>>>>>>>>> zmq::mailbox_t::recv(zmq::command_t*, int)+0x78 >>>>>>>>>>> zmq::socket_base_t::process_commands(int, bool)+0xbe >>>>>>>>>>> zmq::socket_base_t::getsockopt(int, void*, unsigned >>>>>>>>>>> long*)+0x135 >>>>>>>>>>> zmq_getsockopt+0x75 >>>>>>>>>>> zmq_poll+0x3da >>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>> main+0xb40 >>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>> >>>>>>>>>>> The code in the loop is executed once per file descriptor in the >>>>>>>>>>> initial >>>>>>>>>>> pollarray, redoing a poll() system call each time. >>>>>>>>>>> Is there a reason to proceed that way ? >>>>>>>>>>> Would be possible to reuse the results of the first poll() in >>>>>>>>>>> order >>>>>>>>>>> to >>>>>>>>>>> bypass the second set of system calls ? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Francis >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> zeromq-dev mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>>>>> >>>>>> _______________________________________________ >>>>>> zeromq-dev mailing list >>>>>> [email protected] >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>>> >>> > > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
