Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
W dniu 2014-09-25 13:54, Goswin von Brederlow pisze: It might help if you would post a testcase that: a) actualy compiles It actualy compiles on Windows. I think I should state it more clearly in first post. b) doesn't involve undefined behaviour: (ii) select() may update the timeout argument to indicate how much time was left. pselect() does not change this argument. On Linux, select() modifies timeout to reflect the amount of time not slept; most other implementations do not do this. (POSIX.1-2001 per- mits either behavior.) This causes problems both when Linux code which reads timeout is ported to other operating systems, and when code is ported to Linux that reuses a struct timeval for multiple select()s in a loop without reinitializing it. Consider timeout to be undefined after select() returns. Again, it's not undefined on Windows. However, your point is worth remembering in case I'll need to port my code in the future. Resetting timeval is cheap enough to do it on regular basis on every attempt to use select() on any platform. I set nfds (which is ignored on Windows) but omitted timeval. I'll remember that. All I see is that you use select with 0 timeout, which makes the select basically pointless. You also wrongly select the FD for writability (which is always possible, so a NOP) and then break when a message was received. Since messages arrive asynchronously you eventually hit the case where a message gets received just at that time. Nothing wrong there. Of course 0 timeout is pointless. The same as using select() on just one descriptor. This was naive method to simulate event without actualy introducing external library. Anyway, both of you gave me some valuable input to reconsider differences between 0mq and BSD sockets. I still have to figure out some elegant way to deal with REQ-REP sockets but it should be easier now. With regards, Jarek ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
W dniu 2014-09-26 02:05, KIU Shueng Chuan pisze: Your code could be stripped down a lot: 1) PUB socket is only for sending. There's no need to test it for ZMQ_POLLIN. 2) PUB socket never blocks when sending. There's no need to test it for ZMQ_POLLOUT. 3) SUB socket is only for receiving. There's no need to test it for ZMQ_POLLOUT. 4) The ZMQ_FDs are only to be tested for readability. They are NOT to be tested for writability (and in fact should always test positive for writability.) You are right. My case isn't really minimal but allows to test on different types of sockets with just minimal changes. I still feel not right about receiving POLLIN on write but it might just be my habit from old times of low level socket programming. You both made me start to think about 0mq in different way. Thanks, Jarek ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
On Fri, Sep 26, 2014 at 09:51:40AM +0200, Dorvin wrote: W dniu 2014-09-25 13:54, Goswin von Brederlow pisze: It might help if you would post a testcase that: a) actualy compiles It actualy compiles on Windows. I think I should state it more clearly in first post. With the includes for FD_SET missing and Sleep() undefined? b) doesn't involve undefined behaviour: (ii) select() may update the timeout argument to indicate how much time was left. pselect() does not change this argument. On Linux, select() modifies timeout to reflect the amount of time not slept; most other implementations do not do this. (POSIX.1-2001 per- mits either behavior.) This causes problems both when Linux code which reads timeout is ported to other operating systems, and when code is ported to Linux that reuses a struct timeval for multiple select()s in a loop without reinitializing it. Consider timeout to be undefined after select() returns. Again, it's not undefined on Windows. However, your point is worth remembering in case I'll need to port my code in the future. Resetting timeval is cheap enough to do it on regular basis on every attempt to use select() on any platform. I set nfds (which is ignored on Windows) but omitted timeval. I'll remember that. All I see is that you use select with 0 timeout, which makes the select basically pointless. You also wrongly select the FD for writability (which is always possible, so a NOP) and then break when a message was received. Since messages arrive asynchronously you eventually hit the case where a message gets received just at that time. Nothing wrong there. Of course 0 timeout is pointless. The same as using select() on just one descriptor. This was naive method to simulate event without actualy introducing external library. Anyway, both of you gave me some valuable input to reconsider differences between 0mq and BSD sockets. I still have to figure out some elegant way to deal with REQ-REP sockets but it should be easier now. With regards, Jarek MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
On Fri, Sep 26, 2014 at 09:55:59AM +0200, Dorvin wrote: W dniu 2014-09-26 02:05, KIU Shueng Chuan pisze: Your code could be stripped down a lot: 1) PUB socket is only for sending. There's no need to test it for ZMQ_POLLIN. 2) PUB socket never blocks when sending. There's no need to test it for ZMQ_POLLOUT. 3) SUB socket is only for receiving. There's no need to test it for ZMQ_POLLOUT. 4) The ZMQ_FDs are only to be tested for readability. They are NOT to be tested for writability (and in fact should always test positive for writability.) You are right. My case isn't really minimal but allows to test on different types of sockets with just minimal changes. I still feel not right about receiving POLLIN on write but it might just be my habit from old times of low level socket programming. You both made me start to think about 0mq in different way. Thanks, Jarek You aren't receiving POLLIN on write. You receive POLLIN on event, whatever that may be. That's just how an eventfd works. There is no way to make a FD emit a POLLOUT event like you can with a POLLIN. Reading from an FD doesn't generate the event due to buffering. Remember that a zmq socket has a fan in/out structure and the FD has to signal all events for all lowlevel sockets: / socket 1 |/--- socket 2 FD ---* socket 3 |\--- socket 4 \ socket 5 MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
W dniu 2014-09-26 10:39, Goswin von Brederlow pisze: a) actualy compiles It actualy compiles on Windows. I think I should state it more clearly in first post. With the includes for FD_SET missing and Sleep() undefined? I don't know what Windows version and which build environment you are using but zmq.h includes winsock2.h (which defines fd_set). Winsock2.h includes windows.h (which includes winbase.h). Winbase.h defines Sleep(). I just created empty MSVC project and it worked like a charm. This is interesting, so if you would like we could investigate your situation but I would not like to spam this group so let's continue in private. With regards, Jarek ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
On Fri, Sep 26, 2014 at 11:25:37AM +0200, Dorvin wrote: W dniu 2014-09-26 10:39, Goswin von Brederlow pisze: a) actualy compiles It actualy compiles on Windows. I think I should state it more clearly in first post. With the includes for FD_SET missing and Sleep() undefined? I don't know what Windows version and which build environment you are using but zmq.h includes winsock2.h (which defines fd_set). Winsock2.h includes windows.h (which includes winbase.h). Winbase.h defines Sleep(). I just created empty MSVC project and it worked like a charm. This is interesting, so if you would like we could investigate your situation but I would not like to spam this group so let's continue in private. With regards, Jarek Ok, problem solved then. Under linux the zmq.h includes far less and yoiu have to actually include the proper headers for the stuff you use. I assume Sleep() is another windows thing. Portability is hard. MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
Thank you for your replies so far. I'll clarify if it does matter that
I'm using Windows and libzmq 4.0.4.
W dniu 2014-09-25 04:38, Goswin von Brederlow pisze:
the ØMQ library
shall signal any pending events on the socket in an edge-triggered
fashion by making the file descriptor become ready for reading.
This is exactly what bothers me. According to documentation I'm supposed
to get most socket events when FD gets ready for reading.
But when I do:
sel_ret = select(sub_fd + 1, sub_set, NULL, NULL, time );
if (sel_ret == 1)
{
zmq_getsockopt(subscriber, ZMQ_EVENTS, zevents, zevents_len);
if(zevents ZMQ_POLLIN)
{
//retrieve and do something with message
}
}
I receive only some or none messages at all. You may see it in testcase.
For now I found 3 options to work this out:
1. Use zmq_poll() periodically - this seems to get read and write
notifications at the same time and therefore not lose events. This does
not look like event-driven way of doing things, though.
2. Use getsockopt with ZMQ_EVENTS after each send or receive - this is
like small version of zmq_poll invoked almost constantly.
3. Keeping write notifier active all time which is not a good idea
because it fires on every pass of event loop. In first reply I already
was discouraged from this option.
If you would be so kind I would ask you to execute my code, see results
and tell me if it's expected behavior (some kind of optimization or
something), I miss something in how 0mq sockets should work, or if I
should file bug report.
You may remove line if (occurences 0) {break;}; to see that what I
describe is not single event but it repeats.
With regards,
Jarek
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
On Thu, Sep 25, 2014 at 11:09:15AM +0200, Dorvin wrote:
Thank you for your replies so far. I'll clarify if it does matter that
I'm using Windows and libzmq 4.0.4.
W dniu 2014-09-25 04:38, Goswin von Brederlow pisze:
the ØMQ library
shall signal any pending events on the socket in an edge-triggered
fashion by making the file descriptor become ready for reading.
This is exactly what bothers me. According to documentation I'm supposed
to get most socket events when FD gets ready for reading.
But when I do:
sel_ret = select(sub_fd + 1, sub_set, NULL, NULL, time );
if (sel_ret == 1)
{
zmq_getsockopt(subscriber, ZMQ_EVENTS, zevents, zevents_len);
if(zevents ZMQ_POLLIN)
{
//retrieve and do something with message
}
}
I receive only some or none messages at all. You may see it in testcase.
For now I found 3 options to work this out:
1. Use zmq_poll() periodically - this seems to get read and write
notifications at the same time and therefore not lose events. This does
not look like event-driven way of doing things, though.
2. Use getsockopt with ZMQ_EVENTS after each send or receive - this is
like small version of zmq_poll invoked almost constantly.
3. Keeping write notifier active all time which is not a good idea
because it fires on every pass of event loop. In first reply I already
was discouraged from this option.
If you would be so kind I would ask you to execute my code, see results
and tell me if it's expected behavior (some kind of optimization or
something), I miss something in how 0mq sockets should work, or if I
should file bug report.
You may remove line if (occurences 0) {break;}; to see that what I
describe is not single event but it repeats.
With regards,
Jarek
It might help if you would post a testcase that:
a) actualy compiles
b) doesn't involve undefined behaviour:
(ii) select() may update the timeout argument to indicate how much
time was left. pselect() does not change this argument.
On Linux, select() modifies timeout to reflect the amount of time not
slept; most other implementations do not do this. (POSIX.1-2001 per-
mits either behavior.) This causes problems both when Linux code which
reads timeout is ported to other operating systems, and when code is
ported to Linux that reuses a struct timeval for multiple select()s in
a loop without reinitializing it. Consider timeout to be undefined
after select() returns.
c) gives an error where the problem occurs or describe what the
expected outcome should be and what it acutally is
All I see is that you use select with 0 timeout, which makes the
select basically pointless. You also wrongly select the FD for
writability (which is always possible, so a NOP) and then break when
a message was received. Since messages arrive asynchronously you
eventually hit the case where a message gets received just at that
time. Nothing wrong there.
MfG
Goswin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
Your code could be stripped down a lot:
1) PUB socket is only for sending. There's no need to test it for
ZMQ_POLLIN.
2) PUB socket never blocks when sending. There's no need to test it for
ZMQ_POLLOUT.
3) SUB socket is only for receiving. There's no need to test it for
ZMQ_POLLOUT.
4) The ZMQ_FDs are only to be tested for readability. They are NOT to be
tested for writability (and in fact should always test positive for
writability.)
On Thu, Sep 25, 2014 at 5:09 PM, Dorvin dor...@tlen.pl wrote:
Thank you for your replies so far. I'll clarify if it does matter that
I'm using Windows and libzmq 4.0.4.
W dniu 2014-09-25 04:38, Goswin von Brederlow pisze:
the ØMQ library
shall signal any pending events on the socket in an edge-triggered
fashion by making the file descriptor become ready for reading.
This is exactly what bothers me. According to documentation I'm supposed
to get most socket events when FD gets ready for reading.
But when I do:
sel_ret = select(sub_fd + 1, sub_set, NULL, NULL, time );
if (sel_ret == 1)
{
zmq_getsockopt(subscriber, ZMQ_EVENTS, zevents, zevents_len);
if(zevents ZMQ_POLLIN)
{
//retrieve and do something with message
}
}
I receive only some or none messages at all. You may see it in testcase.
For now I found 3 options to work this out:
1. Use zmq_poll() periodically - this seems to get read and write
notifications at the same time and therefore not lose events. This does
not look like event-driven way of doing things, though.
2. Use getsockopt with ZMQ_EVENTS after each send or receive - this is
like small version of zmq_poll invoked almost constantly.
3. Keeping write notifier active all time which is not a good idea
because it fires on every pass of event loop. In first reply I already
was discouraged from this option.
If you would be so kind I would ask you to execute my code, see results
and tell me if it's expected behavior (some kind of optimization or
something), I miss something in how 0mq sockets should work, or if I
should file bug report.
You may remove line if (occurences 0) {break;}; to see that what I
describe is not single event but it repeats.
With regards,
Jarek
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
Don't check the FD for writability. You are only meant to check for its readability. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZMQ_POLLIN received on WRITE event
On Wed, Sep 24, 2014 at 07:51:39PM +0200, Dorvin wrote: Hi, I started using ZeroMQ not long ago and tried to use it in evented app. When I try to use FD with socket notifier it appears I'm receiving ZMQ_POLLIN events when FD is signalling write event. Is there any rationale behind such behavior? Naive testcase to show this behavior is at: http://pastebin.com/iXV2GVVv Thanks in advance for you reply, Jarek From http://api.zeromq.org/4-1:zmq-getsockopt: ZMQ_FD: Retrieve file descriptor associated with the socket The ZMQ_FD option shall retrieve the file descriptor associated with the specified socket. The returned file descriptor can be used to integrate the socket into an existing event loop; the ØMQ library shall signal any pending events on the socket in an edge-triggered fashion by making the file descriptor become ready for reading. The FD isn't the actual lowlevel socket. Instead it is a signalfd or eventfd or something like it. The IO threads write to it when something happens on any of the low-level socket attached to the zmq socket and the event handler reads from it to clear it again. The ability to read from the returned file descriptor does not necessarily indicate that messages are available to be read from, or can be written to, the underlying socket; applications must retrieve the actual event state with a subsequent retrieval of the ZMQ_EVENTS option. MfG Goswin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
