Re: [zeromq-dev] ZeroMQ STREAM Socket Type
It seems what you are doing won't work as currently it is hard to use STREAM as the first sender, as you don't know the identity if the peer. I think someone added notify option, worth to check it out. Are you using STREAM at both sides? On May 24, 2016 00:04, "Peter Witkowski" wrote: > Sorry to spam everyone, but just to clarify, when I say that PUSH-PULL > doesn't work because of handshaking, I'm speaking about the ZeroMQ > handshake (greetings, etc.). The data link handles the TCP connection > stuff. > > On Mon, May 23, 2016 at 4:58 PM, Peter Witkowski > wrote: > >> Hello, >> >> Long story short, I have code that works that I now need to refactor. >> The networking for my application has changed and I need to push my ZeroMQ >> messages over a half-duplex (i.e., one way) wireless data link. The data >> link is looking for raw TCP on either end and specifies what side is the >> client and server. You can think of this data link as a one way bridge, >> with a TCP endpoint on either end that I need to talk to. The data link >> then forwards the TCP traffic in some weird protocol to the other side. >> >> The code I have written is a PUSH-PULL pattern, which doesn't work due to >> the handshaking involved at the socket's start-up (I'm assuming there's no >> way to disable this). The code sends and receives a three-part message >> (the third message part is large, about 65KB). I need to refactor this >> into two STREAM sockets, but I'm having problems (conceptually) with >> sending and receiving. >> >> Here's some pseudo code for the sender (which is the client per the data >> link): >> >>- Set-up context, open socket, call connect. >>- Get ZMQ_IDENTITY using zmq_getsockopt() >>- Send identity >>- Send message part 1 (assuming that this is an atomic send and won't >>only send N bytes) >>- Send identity >>- Send message part 2 (see previous assumption about atomic) >>- Send identity >>- Send message part 3 (see previous assumption about atomic) >>- When I'm ready to close, I send the identity followed by a zero >>length packet >> >> The receiver (TCP server per the data link) is basically the opposite, >> but here I'm a bit confused. Namely, is each receive call atomic, or is >> there a chance that the data gets chunked up into multiple messages? Also, >> do I need to read off the identifier each time or does the library only >> forward the message parts? >> >> Also, in general, are there any issues with using ZeroMQ (even STREAM >> sockets) with this set-up? Note that my assumption is that the receiver >> never needs to send data and the sender never needs to receive data. >> >> Thanks in advance for the help. >> -- >> Peter Witkowski >> pwitkow...@gmail.com >> > > > > -- > Peter Witkowski > pwitkow...@gmail.com > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?
Only if you are using thread safe sockets. You can save a call by with just
calling with ZMQ_NOBLOCK (no need for ZMQ_EVENTS). Then you don't need to
worry about race condition.
On May 24, 2016 08:22, "Huttunen, Kalle (GE Healthcare)" <
kalle.huttu...@ge.com> wrote:
> > after your call to zmq_recv(), but before your call to select() / poll()
>
>
>
> Are you talking about the way to receive messages when the ZMQ_FD signals?
> Basically I currently have the following code called when the ZMQ_FD
> becomes readable (in pseudocode):
>
>
>
> while ZMQ_EVENTS has ZMQ_POLLIN bit set
>
> {
>
> receive message from socket;
>
> process message;
>
> }
>
>
>
> Is there a potential race condition here?
>
>
>
> Thanks,
>
> --
>
> Kalle Huttunen
>
>
>
> *From:* zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] *On
> Behalf Of *Dave Lambley
> *Sent:* 23. toukokuuta 2016 19:08
> *To:* ZeroMQ development list
> *Subject:* EXT: Re: [zeromq-dev] What are the “serious caveats” with
> ZMQ_FD?
>
>
>
> On 23 May 2016 at 10:23, Huttunen, Kalle (GE Healthcare) <
> kalle.huttu...@ge.com> wrote:
>
> It seems that sending on the socket makes the ZMQ_FD readable. That in
> turn triggers the calling of the code where I check ZMQ_EVENTS and receive
> everything from the socket. This way I end up checking ZMQ_EVENTS after
> each send.
>
> Is the ZMQ_FD becoming readable when sending on the socket something that
> can be relied on?
>
>
>
> I believe you have a race condition. If a message arrives after your call
> to zmq_recv(), but before your call to select() / poll(), it will block
> despite there being a message ready.
>
> If you can tolerate messages being delayed, there is an simple workaround
> in which you call zmq_recv() periodically. We have code doing this by
> specifying a timeout in the select() call.
>
> Dave
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
Great, thanks. All the rest makes sense. On 24 May 2016 at 06:34, Pieter Hintjens wrote: > OK, done here: https://github.com/zeromq/rfc/pull/95 > > On Mon, May 23, 2016 at 11:11 PM, Elliot Crosby-McCullough < > elliot...@gmail.com> wrote: > >> 👍🏻 >> >> On 23 May 2016 at 21:35, Pieter Hintjens wrote: >> >>> Yes, good catch. I need to double-check that, it may be that with NULL >>> we can strictly define the server as "binding peer" and client as >>> "connecting peer" since there's no security relationship. >>> >>> On Mon, May 23, 2016 at 7:34 PM, Elliot Crosby-McCullough >>> wrote: >>> > OK, cool. Does this line need updating then, as the NULL mechanism now >>> > describes client and server behaviours with regard to the sequencing of >>> > READY commands? >>> > >>> > https://github.com/zeromq/rfc/blob/master/spec_37.txt#L227 >>> > >>> > On 23 May 2016 at 18:20, Pieter Hintjens wrote: >>> >> >>> >> The as_server property lets you reverse the usual bind/connect >>> >> direction so that the client binds and the server connects. Without >>> >> this, it's per the usual default. >>> >> >>> >> On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough >>> >> wrote: >>> >> > Great, thanks. Since we're talking about a security mechanism that >>> >> > doesn't >>> >> > set `as_server` for either peer should I assume that the binding >>> peer is >>> >> > the >>> >> > "server" and the connecting peer is the "client", i.e. standard TCP >>> >> > terminology? >>> >> > >>> >> > On 23 May 2016 at 13:15, Pieter Hintjens wrote: >>> >> >> >>> >> >> I've made changes and sent a PR: >>> https://github.com/zeromq/rfc/pull/93 >>> >> >> >>> >> >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens >>> wrote: >>> >> >> > OK, there's an error in the explanation here. It's not >>> symmetric. The >>> >> >> > client is the one that needs to send its READY upfront. The >>> server is >>> >> >> > the one that checks the READY properties before sending its own >>> >> >> > READY. >>> >> >> > The example is correct; the text is not. Thanks for pointing this >>> >> >> > out. >>> >> >> > >>> >> >> > >>> >> >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough >>> >> >> > wrote: >>> >> >> >> Hi Pieter, >>> >> >> >> >>> >> >> >> You're right, I noticed the more detailed description of the >>> partial >>> >> >> >> signature read shortly after posting, but I'm still not sure >>> about >>> >> >> >> the >>> >> >> >> worked example. As I mentioned there's also the conflict >>> between >>> >> >> >> waiting >>> >> >> >> for a READY, validating it, *then* sending your own READY as >>> per the >>> >> >> >> worked >>> >> >> >> example, while the spec says to send READY without waiting for >>> the >>> >> >> >> other >>> >> >> >> side to do so. >>> >> >> >> >>> >> >> >> All the best, >>> >> >> >> Elliot >>> >> >> >> >>> >> >> >> >>> >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: >>> >> >> >>> >>> >> >> >>> The partial read is to deal with older versions of ZMTP. Peers >>> that >>> >> >> >>> do >>> >> >> >>> this won't deadlock since as soon as they've read the first 11 >>> >> >> >>> bytes >>> >> >> >>> they can send the rest of their own greeting, then they can >>> >> >> >>> validate >>> >> >> >>> the socket type, send READY, then wait for a READY. Peers that >>> >> >> >>> don't >>> >> >> >>> do backwards compatibility just send the whole greeting, wait >>> for >>> >> >> >>> the >>> >> >> >>> greeting, validate it, etc. >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >>> >> >> >>> wrote: >>> >> >> >>> > Hi there! >>> >> >> >>> > >>> >> >> >>> > I'm working my way through an implementation of 37/ZMTP >>> >> >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few >>> inconsistencies >>> >> >> >>> > in >>> >> >> >>> > the >>> >> >> >>> > spec, mostly the timing of message exchanges, and how it >>> differs >>> >> >> >>> > in >>> >> >> >>> > the >>> >> >> >>> > spec >>> >> >> >>> > proper from the worked example. >>> >> >> >>> > >>> >> >> >>> > Here's a couple of quotes where anti-deadlock approaches are >>> >> >> >>> > discussed: >>> >> >> >>> >> >>> >> >> >>> >> A peer that reads a full greeting, including mechanism, MUST >>> >> >> >>> >> also >>> >> >> >>> >> send >>> >> >> >>> >> a >>> >> >> >>> >> full greeting including mechanism. This avoids deadlocks in >>> >> >> >>> >> which >>> >> >> >>> >> two >>> >> >> >>> >> peers >>> >> >> >>> >> each wait for the other to divulge the remainder of their >>> >> >> >>> >> greeting. >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY >>> >> >> >>> >> command >>> >> >> >>> >> before >>> >> >> >>> >> attempting to receive a READY from the other peer. In the >>> NULL >>> >> >> >>> >> mechanism, >>> >> >> >>> >> peers are symmetric. >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> > However, the worked example doesn't do this: >>> >> >> >>> >> >>> >> >> >>> >> The client sends a partia
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
OK, done here: https://github.com/zeromq/rfc/pull/95 On Mon, May 23, 2016 at 11:11 PM, Elliot Crosby-McCullough < elliot...@gmail.com> wrote: > 👍🏻 > > On 23 May 2016 at 21:35, Pieter Hintjens wrote: > >> Yes, good catch. I need to double-check that, it may be that with NULL >> we can strictly define the server as "binding peer" and client as >> "connecting peer" since there's no security relationship. >> >> On Mon, May 23, 2016 at 7:34 PM, Elliot Crosby-McCullough >> wrote: >> > OK, cool. Does this line need updating then, as the NULL mechanism now >> > describes client and server behaviours with regard to the sequencing of >> > READY commands? >> > >> > https://github.com/zeromq/rfc/blob/master/spec_37.txt#L227 >> > >> > On 23 May 2016 at 18:20, Pieter Hintjens wrote: >> >> >> >> The as_server property lets you reverse the usual bind/connect >> >> direction so that the client binds and the server connects. Without >> >> this, it's per the usual default. >> >> >> >> On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough >> >> wrote: >> >> > Great, thanks. Since we're talking about a security mechanism that >> >> > doesn't >> >> > set `as_server` for either peer should I assume that the binding >> peer is >> >> > the >> >> > "server" and the connecting peer is the "client", i.e. standard TCP >> >> > terminology? >> >> > >> >> > On 23 May 2016 at 13:15, Pieter Hintjens wrote: >> >> >> >> >> >> I've made changes and sent a PR: >> https://github.com/zeromq/rfc/pull/93 >> >> >> >> >> >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens >> wrote: >> >> >> > OK, there's an error in the explanation here. It's not symmetric. >> The >> >> >> > client is the one that needs to send its READY upfront. The >> server is >> >> >> > the one that checks the READY properties before sending its own >> >> >> > READY. >> >> >> > The example is correct; the text is not. Thanks for pointing this >> >> >> > out. >> >> >> > >> >> >> > >> >> >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough >> >> >> > wrote: >> >> >> >> Hi Pieter, >> >> >> >> >> >> >> >> You're right, I noticed the more detailed description of the >> partial >> >> >> >> signature read shortly after posting, but I'm still not sure >> about >> >> >> >> the >> >> >> >> worked example. As I mentioned there's also the conflict between >> >> >> >> waiting >> >> >> >> for a READY, validating it, *then* sending your own READY as per >> the >> >> >> >> worked >> >> >> >> example, while the spec says to send READY without waiting for >> the >> >> >> >> other >> >> >> >> side to do so. >> >> >> >> >> >> >> >> All the best, >> >> >> >> Elliot >> >> >> >> >> >> >> >> >> >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: >> >> >> >>> >> >> >> >>> The partial read is to deal with older versions of ZMTP. Peers >> that >> >> >> >>> do >> >> >> >>> this won't deadlock since as soon as they've read the first 11 >> >> >> >>> bytes >> >> >> >>> they can send the rest of their own greeting, then they can >> >> >> >>> validate >> >> >> >>> the socket type, send READY, then wait for a READY. Peers that >> >> >> >>> don't >> >> >> >>> do backwards compatibility just send the whole greeting, wait >> for >> >> >> >>> the >> >> >> >>> greeting, validate it, etc. >> >> >> >>> >> >> >> >>> >> >> >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >> >> >> >>> wrote: >> >> >> >>> > Hi there! >> >> >> >>> > >> >> >> >>> > I'm working my way through an implementation of 37/ZMTP >> >> >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few >> inconsistencies >> >> >> >>> > in >> >> >> >>> > the >> >> >> >>> > spec, mostly the timing of message exchanges, and how it >> differs >> >> >> >>> > in >> >> >> >>> > the >> >> >> >>> > spec >> >> >> >>> > proper from the worked example. >> >> >> >>> > >> >> >> >>> > Here's a couple of quotes where anti-deadlock approaches are >> >> >> >>> > discussed: >> >> >> >>> >> >> >> >> >>> >> A peer that reads a full greeting, including mechanism, MUST >> >> >> >>> >> also >> >> >> >>> >> send >> >> >> >>> >> a >> >> >> >>> >> full greeting including mechanism. This avoids deadlocks in >> >> >> >>> >> which >> >> >> >>> >> two >> >> >> >>> >> peers >> >> >> >>> >> each wait for the other to divulge the remainder of their >> >> >> >>> >> greeting. >> >> >> >>> > >> >> >> >>> > >> >> >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY >> >> >> >>> >> command >> >> >> >>> >> before >> >> >> >>> >> attempting to receive a READY from the other peer. In the >> NULL >> >> >> >>> >> mechanism, >> >> >> >>> >> peers are symmetric. >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > However, the worked example doesn't do this: >> >> >> >>> >> >> >> >> >>> >> The client sends a partial greeting (11 octets) greeting to >> the >> >> >> >>> >> server, >> >> >> >>> >> and at the same time (before receiving anything from the >> >> >> >>> >> client), >> >> >> >>> >> the >> >> >> >>> >> server >> >> >> >>> >> also sends a p
[zeromq-dev] What are the “serious caveats” with ZMQ_FD?
> after your call to zmq_recv(), but before your call to select() / poll()
Are you talking about the way to receive messages when the ZMQ_FD signals?
Basically I currently have the following code called when the ZMQ_FD becomes
readable (in pseudocode):
while ZMQ_EVENTS has ZMQ_POLLIN bit set
{
receive message from socket;
process message;
}
Is there a potential race condition here?
Thanks,
--
Kalle Huttunen
From: zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Dave
Lambley
Sent: 23. toukokuuta 2016 19:08
To: ZeroMQ development list
Subject: EXT: Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?
On 23 May 2016 at 10:23, Huttunen, Kalle (GE Healthcare)
mailto:kalle.huttu...@ge.com>> wrote:
It seems that sending on the socket makes the ZMQ_FD readable. That in turn
triggers the calling of the code where I check ZMQ_EVENTS and receive
everything from the socket. This way I end up checking ZMQ_EVENTS after each
send.
Is the ZMQ_FD becoming readable when sending on the socket something that can
be relied on?
I believe you have a race condition. If a message arrives after your call to
zmq_recv(), but before your call to select() / poll(), it will block despite
there being a message ready.
If you can tolerate messages being delayed, there is an simple workaround in
which you call zmq_recv() periodically. We have code doing this by specifying a
timeout in the select() call.
Dave
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ STREAM Socket Type
Sorry to spam everyone, but just to clarify, when I say that PUSH-PULL doesn't work because of handshaking, I'm speaking about the ZeroMQ handshake (greetings, etc.). The data link handles the TCP connection stuff. On Mon, May 23, 2016 at 4:58 PM, Peter Witkowski wrote: > Hello, > > Long story short, I have code that works that I now need to refactor. The > networking for my application has changed and I need to push my ZeroMQ > messages over a half-duplex (i.e., one way) wireless data link. The data > link is looking for raw TCP on either end and specifies what side is the > client and server. You can think of this data link as a one way bridge, > with a TCP endpoint on either end that I need to talk to. The data link > then forwards the TCP traffic in some weird protocol to the other side. > > The code I have written is a PUSH-PULL pattern, which doesn't work due to > the handshaking involved at the socket's start-up (I'm assuming there's no > way to disable this). The code sends and receives a three-part message > (the third message part is large, about 65KB). I need to refactor this > into two STREAM sockets, but I'm having problems (conceptually) with > sending and receiving. > > Here's some pseudo code for the sender (which is the client per the data > link): > >- Set-up context, open socket, call connect. >- Get ZMQ_IDENTITY using zmq_getsockopt() >- Send identity >- Send message part 1 (assuming that this is an atomic send and won't >only send N bytes) >- Send identity >- Send message part 2 (see previous assumption about atomic) >- Send identity >- Send message part 3 (see previous assumption about atomic) >- When I'm ready to close, I send the identity followed by a zero >length packet > > The receiver (TCP server per the data link) is basically the opposite, but > here I'm a bit confused. Namely, is each receive call atomic, or is there > a chance that the data gets chunked up into multiple messages? Also, do I > need to read off the identifier each time or does the library only forward > the message parts? > > Also, in general, are there any issues with using ZeroMQ (even STREAM > sockets) with this set-up? Note that my assumption is that the receiver > never needs to send data and the sender never needs to receive data. > > Thanks in advance for the help. > -- > Peter Witkowski > pwitkow...@gmail.com > -- Peter Witkowski pwitkow...@gmail.com ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
👍🏻 On 23 May 2016 at 21:35, Pieter Hintjens wrote: > Yes, good catch. I need to double-check that, it may be that with NULL > we can strictly define the server as "binding peer" and client as > "connecting peer" since there's no security relationship. > > On Mon, May 23, 2016 at 7:34 PM, Elliot Crosby-McCullough > wrote: > > OK, cool. Does this line need updating then, as the NULL mechanism now > > describes client and server behaviours with regard to the sequencing of > > READY commands? > > > > https://github.com/zeromq/rfc/blob/master/spec_37.txt#L227 > > > > On 23 May 2016 at 18:20, Pieter Hintjens wrote: > >> > >> The as_server property lets you reverse the usual bind/connect > >> direction so that the client binds and the server connects. Without > >> this, it's per the usual default. > >> > >> On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough > >> wrote: > >> > Great, thanks. Since we're talking about a security mechanism that > >> > doesn't > >> > set `as_server` for either peer should I assume that the binding peer > is > >> > the > >> > "server" and the connecting peer is the "client", i.e. standard TCP > >> > terminology? > >> > > >> > On 23 May 2016 at 13:15, Pieter Hintjens wrote: > >> >> > >> >> I've made changes and sent a PR: > https://github.com/zeromq/rfc/pull/93 > >> >> > >> >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens > wrote: > >> >> > OK, there's an error in the explanation here. It's not symmetric. > The > >> >> > client is the one that needs to send its READY upfront. The server > is > >> >> > the one that checks the READY properties before sending its own > >> >> > READY. > >> >> > The example is correct; the text is not. Thanks for pointing this > >> >> > out. > >> >> > > >> >> > > >> >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough > >> >> > wrote: > >> >> >> Hi Pieter, > >> >> >> > >> >> >> You're right, I noticed the more detailed description of the > partial > >> >> >> signature read shortly after posting, but I'm still not sure about > >> >> >> the > >> >> >> worked example. As I mentioned there's also the conflict between > >> >> >> waiting > >> >> >> for a READY, validating it, *then* sending your own READY as per > the > >> >> >> worked > >> >> >> example, while the spec says to send READY without waiting for the > >> >> >> other > >> >> >> side to do so. > >> >> >> > >> >> >> All the best, > >> >> >> Elliot > >> >> >> > >> >> >> > >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: > >> >> >>> > >> >> >>> The partial read is to deal with older versions of ZMTP. Peers > that > >> >> >>> do > >> >> >>> this won't deadlock since as soon as they've read the first 11 > >> >> >>> bytes > >> >> >>> they can send the rest of their own greeting, then they can > >> >> >>> validate > >> >> >>> the socket type, send READY, then wait for a READY. Peers that > >> >> >>> don't > >> >> >>> do backwards compatibility just send the whole greeting, wait for > >> >> >>> the > >> >> >>> greeting, validate it, etc. > >> >> >>> > >> >> >>> > >> >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough > >> >> >>> wrote: > >> >> >>> > Hi there! > >> >> >>> > > >> >> >>> > I'm working my way through an implementation of 37/ZMTP > >> >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few > inconsistencies > >> >> >>> > in > >> >> >>> > the > >> >> >>> > spec, mostly the timing of message exchanges, and how it > differs > >> >> >>> > in > >> >> >>> > the > >> >> >>> > spec > >> >> >>> > proper from the worked example. > >> >> >>> > > >> >> >>> > Here's a couple of quotes where anti-deadlock approaches are > >> >> >>> > discussed: > >> >> >>> >> > >> >> >>> >> A peer that reads a full greeting, including mechanism, MUST > >> >> >>> >> also > >> >> >>> >> send > >> >> >>> >> a > >> >> >>> >> full greeting including mechanism. This avoids deadlocks in > >> >> >>> >> which > >> >> >>> >> two > >> >> >>> >> peers > >> >> >>> >> each wait for the other to divulge the remainder of their > >> >> >>> >> greeting. > >> >> >>> > > >> >> >>> > > >> >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY > >> >> >>> >> command > >> >> >>> >> before > >> >> >>> >> attempting to receive a READY from the other peer. In the NULL > >> >> >>> >> mechanism, > >> >> >>> >> peers are symmetric. > >> >> >>> > > >> >> >>> > > >> >> >>> > However, the worked example doesn't do this: > >> >> >>> >> > >> >> >>> >> The client sends a partial greeting (11 octets) greeting to > the > >> >> >>> >> server, > >> >> >>> >> and at the same time (before receiving anything from the > >> >> >>> >> client), > >> >> >>> >> the > >> >> >>> >> server > >> >> >>> >> also sends a partial greeting > >> >> >>> > > >> >> >>> > > >> >> >>> >> The server validates the socket type, accepts it, and replies > >> >> >>> >> with > >> >> >>> >> a > >> >> >>> >> READY > >> >> >>> >> command > >> >> >>> > > >> >> >>> > > >> >> >>> > The partial greeting isn't quite against the word
[zeromq-dev] ZeroMQ STREAM Socket Type
Hello, Long story short, I have code that works that I now need to refactor. The networking for my application has changed and I need to push my ZeroMQ messages over a half-duplex (i.e., one way) wireless data link. The data link is looking for raw TCP on either end and specifies what side is the client and server. You can think of this data link as a one way bridge, with a TCP endpoint on either end that I need to talk to. The data link then forwards the TCP traffic in some weird protocol to the other side. The code I have written is a PUSH-PULL pattern, which doesn't work due to the handshaking involved at the socket's start-up (I'm assuming there's no way to disable this). The code sends and receives a three-part message (the third message part is large, about 65KB). I need to refactor this into two STREAM sockets, but I'm having problems (conceptually) with sending and receiving. Here's some pseudo code for the sender (which is the client per the data link): - Set-up context, open socket, call connect. - Get ZMQ_IDENTITY using zmq_getsockopt() - Send identity - Send message part 1 (assuming that this is an atomic send and won't only send N bytes) - Send identity - Send message part 2 (see previous assumption about atomic) - Send identity - Send message part 3 (see previous assumption about atomic) - When I'm ready to close, I send the identity followed by a zero length packet The receiver (TCP server per the data link) is basically the opposite, but here I'm a bit confused. Namely, is each receive call atomic, or is there a chance that the data gets chunked up into multiple messages? Also, do I need to read off the identifier each time or does the library only forward the message parts? Also, in general, are there any issues with using ZeroMQ (even STREAM sockets) with this set-up? Note that my assumption is that the receiver never needs to send data and the sender never needs to receive data. Thanks in advance for the help. -- Peter Witkowski pwitkow...@gmail.com ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Meetup in Brussels, June 4th
Awesome, it will be great seeing all of you! I'll be there around 11 depending on traffic. Am 19.05.2016 7:33 nachm. schrieb "Greg Young" : > May come a day early! > > On Thu, May 19, 2016 at 5:55 PM, Pieter Hintjens wrote: > > Hi All, > > > > I'd like to invite anyone who's in the neighborhood to join an all-day > > meetup/patch party on June 4th in Brussels. The address is Rue des > > Ateliers 15. No registration necessary. There will be wifi, seating, > > coffee, drinks, snacks. > > > > -Pieter > > ___ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > -- > Studying for the Turing test > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
Yes, good catch. I need to double-check that, it may be that with NULL we can strictly define the server as "binding peer" and client as "connecting peer" since there's no security relationship. On Mon, May 23, 2016 at 7:34 PM, Elliot Crosby-McCullough wrote: > OK, cool. Does this line need updating then, as the NULL mechanism now > describes client and server behaviours with regard to the sequencing of > READY commands? > > https://github.com/zeromq/rfc/blob/master/spec_37.txt#L227 > > On 23 May 2016 at 18:20, Pieter Hintjens wrote: >> >> The as_server property lets you reverse the usual bind/connect >> direction so that the client binds and the server connects. Without >> this, it's per the usual default. >> >> On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough >> wrote: >> > Great, thanks. Since we're talking about a security mechanism that >> > doesn't >> > set `as_server` for either peer should I assume that the binding peer is >> > the >> > "server" and the connecting peer is the "client", i.e. standard TCP >> > terminology? >> > >> > On 23 May 2016 at 13:15, Pieter Hintjens wrote: >> >> >> >> I've made changes and sent a PR: https://github.com/zeromq/rfc/pull/93 >> >> >> >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens wrote: >> >> > OK, there's an error in the explanation here. It's not symmetric. The >> >> > client is the one that needs to send its READY upfront. The server is >> >> > the one that checks the READY properties before sending its own >> >> > READY. >> >> > The example is correct; the text is not. Thanks for pointing this >> >> > out. >> >> > >> >> > >> >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough >> >> > wrote: >> >> >> Hi Pieter, >> >> >> >> >> >> You're right, I noticed the more detailed description of the partial >> >> >> signature read shortly after posting, but I'm still not sure about >> >> >> the >> >> >> worked example. As I mentioned there's also the conflict between >> >> >> waiting >> >> >> for a READY, validating it, *then* sending your own READY as per the >> >> >> worked >> >> >> example, while the spec says to send READY without waiting for the >> >> >> other >> >> >> side to do so. >> >> >> >> >> >> All the best, >> >> >> Elliot >> >> >> >> >> >> >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: >> >> >>> >> >> >>> The partial read is to deal with older versions of ZMTP. Peers that >> >> >>> do >> >> >>> this won't deadlock since as soon as they've read the first 11 >> >> >>> bytes >> >> >>> they can send the rest of their own greeting, then they can >> >> >>> validate >> >> >>> the socket type, send READY, then wait for a READY. Peers that >> >> >>> don't >> >> >>> do backwards compatibility just send the whole greeting, wait for >> >> >>> the >> >> >>> greeting, validate it, etc. >> >> >>> >> >> >>> >> >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >> >> >>> wrote: >> >> >>> > Hi there! >> >> >>> > >> >> >>> > I'm working my way through an implementation of 37/ZMTP >> >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies >> >> >>> > in >> >> >>> > the >> >> >>> > spec, mostly the timing of message exchanges, and how it differs >> >> >>> > in >> >> >>> > the >> >> >>> > spec >> >> >>> > proper from the worked example. >> >> >>> > >> >> >>> > Here's a couple of quotes where anti-deadlock approaches are >> >> >>> > discussed: >> >> >>> >> >> >> >>> >> A peer that reads a full greeting, including mechanism, MUST >> >> >>> >> also >> >> >>> >> send >> >> >>> >> a >> >> >>> >> full greeting including mechanism. This avoids deadlocks in >> >> >>> >> which >> >> >>> >> two >> >> >>> >> peers >> >> >>> >> each wait for the other to divulge the remainder of their >> >> >>> >> greeting. >> >> >>> > >> >> >>> > >> >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY >> >> >>> >> command >> >> >>> >> before >> >> >>> >> attempting to receive a READY from the other peer. In the NULL >> >> >>> >> mechanism, >> >> >>> >> peers are symmetric. >> >> >>> > >> >> >>> > >> >> >>> > However, the worked example doesn't do this: >> >> >>> >> >> >> >>> >> The client sends a partial greeting (11 octets) greeting to the >> >> >>> >> server, >> >> >>> >> and at the same time (before receiving anything from the >> >> >>> >> client), >> >> >>> >> the >> >> >>> >> server >> >> >>> >> also sends a partial greeting >> >> >>> > >> >> >>> > >> >> >>> >> The server validates the socket type, accepts it, and replies >> >> >>> >> with >> >> >>> >> a >> >> >>> >> READY >> >> >>> >> command >> >> >>> > >> >> >>> > >> >> >>> > The partial greeting isn't quite against the word of the spec >> >> >>> > though >> >> >>> > it >> >> >>> > would be good if it was described more clearly in the spec >> >> >>> > itself, >> >> >>> > however >> >> >>> > waiting to send the READY until after validating the client's >> >> >>> > READY >> >> >>> > goes >> >> >>> > directly against the MUST requirement. >> >> >>> > >> >>
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
OK, cool. Does this line need updating then, as the NULL mechanism now describes client and server behaviours with regard to the sequencing of READY commands? https://github.com/zeromq/rfc/blob/master/spec_37.txt#L227 On 23 May 2016 at 18:20, Pieter Hintjens wrote: > The as_server property lets you reverse the usual bind/connect > direction so that the client binds and the server connects. Without > this, it's per the usual default. > > On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough > wrote: > > Great, thanks. Since we're talking about a security mechanism that > doesn't > > set `as_server` for either peer should I assume that the binding peer is > the > > "server" and the connecting peer is the "client", i.e. standard TCP > > terminology? > > > > On 23 May 2016 at 13:15, Pieter Hintjens wrote: > >> > >> I've made changes and sent a PR: https://github.com/zeromq/rfc/pull/93 > >> > >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens wrote: > >> > OK, there's an error in the explanation here. It's not symmetric. The > >> > client is the one that needs to send its READY upfront. The server is > >> > the one that checks the READY properties before sending its own READY. > >> > The example is correct; the text is not. Thanks for pointing this out. > >> > > >> > > >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough > >> > wrote: > >> >> Hi Pieter, > >> >> > >> >> You're right, I noticed the more detailed description of the partial > >> >> signature read shortly after posting, but I'm still not sure about > the > >> >> worked example. As I mentioned there's also the conflict between > >> >> waiting > >> >> for a READY, validating it, *then* sending your own READY as per the > >> >> worked > >> >> example, while the spec says to send READY without waiting for the > >> >> other > >> >> side to do so. > >> >> > >> >> All the best, > >> >> Elliot > >> >> > >> >> > >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: > >> >>> > >> >>> The partial read is to deal with older versions of ZMTP. Peers that > do > >> >>> this won't deadlock since as soon as they've read the first 11 bytes > >> >>> they can send the rest of their own greeting, then they can validate > >> >>> the socket type, send READY, then wait for a READY. Peers that don't > >> >>> do backwards compatibility just send the whole greeting, wait for > the > >> >>> greeting, validate it, etc. > >> >>> > >> >>> > >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough > >> >>> wrote: > >> >>> > Hi there! > >> >>> > > >> >>> > I'm working my way through an implementation of 37/ZMTP > >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few > inconsistencies in > >> >>> > the > >> >>> > spec, mostly the timing of message exchanges, and how it differs > in > >> >>> > the > >> >>> > spec > >> >>> > proper from the worked example. > >> >>> > > >> >>> > Here's a couple of quotes where anti-deadlock approaches are > >> >>> > discussed: > >> >>> >> > >> >>> >> A peer that reads a full greeting, including mechanism, MUST also > >> >>> >> send > >> >>> >> a > >> >>> >> full greeting including mechanism. This avoids deadlocks in which > >> >>> >> two > >> >>> >> peers > >> >>> >> each wait for the other to divulge the remainder of their > greeting. > >> >>> > > >> >>> > > >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY > command > >> >>> >> before > >> >>> >> attempting to receive a READY from the other peer. In the NULL > >> >>> >> mechanism, > >> >>> >> peers are symmetric. > >> >>> > > >> >>> > > >> >>> > However, the worked example doesn't do this: > >> >>> >> > >> >>> >> The client sends a partial greeting (11 octets) greeting to the > >> >>> >> server, > >> >>> >> and at the same time (before receiving anything from the client), > >> >>> >> the > >> >>> >> server > >> >>> >> also sends a partial greeting > >> >>> > > >> >>> > > >> >>> >> The server validates the socket type, accepts it, and replies > with > >> >>> >> a > >> >>> >> READY > >> >>> >> command > >> >>> > > >> >>> > > >> >>> > The partial greeting isn't quite against the word of the spec > though > >> >>> > it > >> >>> > would be good if it was described more clearly in the spec itself, > >> >>> > however > >> >>> > waiting to send the READY until after validating the client's > READY > >> >>> > goes > >> >>> > directly against the MUST requirement. > >> >>> > > >> >>> > Am I missing something or does the spec and/or worked example need > >> >>> > changing? > >> >>> > > >> >>> > Regards, > >> >>> > Elliot > >> >>> > > >> >>> > > >> >>> > ___ > >> >>> > zeromq-dev mailing list > >> >>> > zeromq-dev@lists.zeromq.org > >> >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> >>> ___ > >> >>> zeromq-dev mailing list > >> >>> zeromq-dev@lists.zeromq.org > >> >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> >> > >> >> > >> >> > >> >>
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
The as_server property lets you reverse the usual bind/connect direction so that the client binds and the server connects. Without this, it's per the usual default. On Mon, May 23, 2016 at 2:17 PM, Elliot Crosby-McCullough wrote: > Great, thanks. Since we're talking about a security mechanism that doesn't > set `as_server` for either peer should I assume that the binding peer is the > "server" and the connecting peer is the "client", i.e. standard TCP > terminology? > > On 23 May 2016 at 13:15, Pieter Hintjens wrote: >> >> I've made changes and sent a PR: https://github.com/zeromq/rfc/pull/93 >> >> On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens wrote: >> > OK, there's an error in the explanation here. It's not symmetric. The >> > client is the one that needs to send its READY upfront. The server is >> > the one that checks the READY properties before sending its own READY. >> > The example is correct; the text is not. Thanks for pointing this out. >> > >> > >> > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough >> > wrote: >> >> Hi Pieter, >> >> >> >> You're right, I noticed the more detailed description of the partial >> >> signature read shortly after posting, but I'm still not sure about the >> >> worked example. As I mentioned there's also the conflict between >> >> waiting >> >> for a READY, validating it, *then* sending your own READY as per the >> >> worked >> >> example, while the spec says to send READY without waiting for the >> >> other >> >> side to do so. >> >> >> >> All the best, >> >> Elliot >> >> >> >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: >> >>> >> >>> The partial read is to deal with older versions of ZMTP. Peers that do >> >>> this won't deadlock since as soon as they've read the first 11 bytes >> >>> they can send the rest of their own greeting, then they can validate >> >>> the socket type, send READY, then wait for a READY. Peers that don't >> >>> do backwards compatibility just send the whole greeting, wait for the >> >>> greeting, validate it, etc. >> >>> >> >>> >> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >> >>> wrote: >> >>> > Hi there! >> >>> > >> >>> > I'm working my way through an implementation of 37/ZMTP >> >>> > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies in >> >>> > the >> >>> > spec, mostly the timing of message exchanges, and how it differs in >> >>> > the >> >>> > spec >> >>> > proper from the worked example. >> >>> > >> >>> > Here's a couple of quotes where anti-deadlock approaches are >> >>> > discussed: >> >>> >> >> >>> >> A peer that reads a full greeting, including mechanism, MUST also >> >>> >> send >> >>> >> a >> >>> >> full greeting including mechanism. This avoids deadlocks in which >> >>> >> two >> >>> >> peers >> >>> >> each wait for the other to divulge the remainder of their greeting. >> >>> > >> >>> > >> >>> >> Note that to avoid deadlocks, each peer MUST send its READY command >> >>> >> before >> >>> >> attempting to receive a READY from the other peer. In the NULL >> >>> >> mechanism, >> >>> >> peers are symmetric. >> >>> > >> >>> > >> >>> > However, the worked example doesn't do this: >> >>> >> >> >>> >> The client sends a partial greeting (11 octets) greeting to the >> >>> >> server, >> >>> >> and at the same time (before receiving anything from the client), >> >>> >> the >> >>> >> server >> >>> >> also sends a partial greeting >> >>> > >> >>> > >> >>> >> The server validates the socket type, accepts it, and replies with >> >>> >> a >> >>> >> READY >> >>> >> command >> >>> > >> >>> > >> >>> > The partial greeting isn't quite against the word of the spec though >> >>> > it >> >>> > would be good if it was described more clearly in the spec itself, >> >>> > however >> >>> > waiting to send the READY until after validating the client's READY >> >>> > goes >> >>> > directly against the MUST requirement. >> >>> > >> >>> > Am I missing something or does the spec and/or worked example need >> >>> > changing? >> >>> > >> >>> > Regards, >> >>> > Elliot >> >>> > >> >>> > >> >>> > ___ >> >>> > zeromq-dev mailing list >> >>> > zeromq-dev@lists.zeromq.org >> >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >>> ___ >> >>> zeromq-dev mailing list >> >>> zeromq-dev@lists.zeromq.org >> >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> >> >> >> >> ___ >> >> zeromq-dev mailing list >> >> zeromq-dev@lists.zeromq.org >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _
Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?
On 23 May 2016 at 10:23, Huttunen, Kalle (GE Healthcare) < kalle.huttu...@ge.com> wrote: > It seems that sending on the socket makes the ZMQ_FD readable. That in > turn triggers the calling of the code where I check ZMQ_EVENTS and receive > everything from the socket. This way I end up checking ZMQ_EVENTS after > each send. > > Is the ZMQ_FD becoming readable when sending on the socket something that > can be relied on? > I believe you have a race condition. If a message arrives after your call to zmq_recv(), but before your call to select() / poll(), it will block despite there being a message ready. If you can tolerate messages being delayed, there is an simple workaround in which you call zmq_recv() periodically. We have code doing this by specifying a timeout in the select() call. Dave ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?
I would not rely on this. Query ZMQ_EVENTS after every send no matter what. On Mon, May 23, 2016, at 02:23 AM, Huttunen, Kalle (GE Healthcare) wrote: > It seems that sending on the socket makes the ZMQ_FD readable. That in > turn triggers the calling of the code where I check ZMQ_EVENTS and > receive everything from the socket. This way I end up checking ZMQ_EVENTS > after each send. > > Is the ZMQ_FD becoming readable when sending on the socket something that > can be relied on? > > -- > Kalle Huttunen > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
Great, thanks. Since we're talking about a security mechanism that doesn't set `as_server` for either peer should I assume that the binding peer is the "server" and the connecting peer is the "client", i.e. standard TCP terminology? On 23 May 2016 at 13:15, Pieter Hintjens wrote: > I've made changes and sent a PR: https://github.com/zeromq/rfc/pull/93 > > On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens wrote: > > OK, there's an error in the explanation here. It's not symmetric. The > > client is the one that needs to send its READY upfront. The server is > > the one that checks the READY properties before sending its own READY. > > The example is correct; the text is not. Thanks for pointing this out. > > > > > > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough > > wrote: > >> Hi Pieter, > >> > >> You're right, I noticed the more detailed description of the partial > >> signature read shortly after posting, but I'm still not sure about the > >> worked example. As I mentioned there's also the conflict between > waiting > >> for a READY, validating it, *then* sending your own READY as per the > worked > >> example, while the spec says to send READY without waiting for the other > >> side to do so. > >> > >> All the best, > >> Elliot > >> > >> > >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: > >>> > >>> The partial read is to deal with older versions of ZMTP. Peers that do > >>> this won't deadlock since as soon as they've read the first 11 bytes > >>> they can send the rest of their own greeting, then they can validate > >>> the socket type, send READY, then wait for a READY. Peers that don't > >>> do backwards compatibility just send the whole greeting, wait for the > >>> greeting, validate it, etc. > >>> > >>> > >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough > >>> wrote: > >>> > Hi there! > >>> > > >>> > I'm working my way through an implementation of 37/ZMTP > >>> > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies > in the > >>> > spec, mostly the timing of message exchanges, and how it differs in > the > >>> > spec > >>> > proper from the worked example. > >>> > > >>> > Here's a couple of quotes where anti-deadlock approaches are > discussed: > >>> >> > >>> >> A peer that reads a full greeting, including mechanism, MUST also > send > >>> >> a > >>> >> full greeting including mechanism. This avoids deadlocks in which > two > >>> >> peers > >>> >> each wait for the other to divulge the remainder of their greeting. > >>> > > >>> > > >>> >> Note that to avoid deadlocks, each peer MUST send its READY command > >>> >> before > >>> >> attempting to receive a READY from the other peer. In the NULL > >>> >> mechanism, > >>> >> peers are symmetric. > >>> > > >>> > > >>> > However, the worked example doesn't do this: > >>> >> > >>> >> The client sends a partial greeting (11 octets) greeting to the > server, > >>> >> and at the same time (before receiving anything from the client), > the > >>> >> server > >>> >> also sends a partial greeting > >>> > > >>> > > >>> >> The server validates the socket type, accepts it, and replies with a > >>> >> READY > >>> >> command > >>> > > >>> > > >>> > The partial greeting isn't quite against the word of the spec though > it > >>> > would be good if it was described more clearly in the spec itself, > >>> > however > >>> > waiting to send the READY until after validating the client's READY > goes > >>> > directly against the MUST requirement. > >>> > > >>> > Am I missing something or does the spec and/or worked example need > >>> > changing? > >>> > > >>> > Regards, > >>> > Elliot > >>> > > >>> > > >>> > ___ > >>> > zeromq-dev mailing list > >>> > zeromq-dev@lists.zeromq.org > >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>> ___ > >>> zeromq-dev mailing list > >>> zeromq-dev@lists.zeromq.org > >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> > >> > >> > >> ___ > >> zeromq-dev mailing list > >> zeromq-dev@lists.zeromq.org > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
I've made changes and sent a PR: https://github.com/zeromq/rfc/pull/93 On Mon, May 23, 2016 at 2:11 PM, Pieter Hintjens wrote: > OK, there's an error in the explanation here. It's not symmetric. The > client is the one that needs to send its READY upfront. The server is > the one that checks the READY properties before sending its own READY. > The example is correct; the text is not. Thanks for pointing this out. > > > On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough > wrote: >> Hi Pieter, >> >> You're right, I noticed the more detailed description of the partial >> signature read shortly after posting, but I'm still not sure about the >> worked example. As I mentioned there's also the conflict between waiting >> for a READY, validating it, *then* sending your own READY as per the worked >> example, while the spec says to send READY without waiting for the other >> side to do so. >> >> All the best, >> Elliot >> >> >> On 22 May 2016 at 19:38, Pieter Hintjens wrote: >>> >>> The partial read is to deal with older versions of ZMTP. Peers that do >>> this won't deadlock since as soon as they've read the first 11 bytes >>> they can send the rest of their own greeting, then they can validate >>> the socket type, send READY, then wait for a READY. Peers that don't >>> do backwards compatibility just send the whole greeting, wait for the >>> greeting, validate it, etc. >>> >>> >>> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >>> wrote: >>> > Hi there! >>> > >>> > I'm working my way through an implementation of 37/ZMTP >>> > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies in the >>> > spec, mostly the timing of message exchanges, and how it differs in the >>> > spec >>> > proper from the worked example. >>> > >>> > Here's a couple of quotes where anti-deadlock approaches are discussed: >>> >> >>> >> A peer that reads a full greeting, including mechanism, MUST also send >>> >> a >>> >> full greeting including mechanism. This avoids deadlocks in which two >>> >> peers >>> >> each wait for the other to divulge the remainder of their greeting. >>> > >>> > >>> >> Note that to avoid deadlocks, each peer MUST send its READY command >>> >> before >>> >> attempting to receive a READY from the other peer. In the NULL >>> >> mechanism, >>> >> peers are symmetric. >>> > >>> > >>> > However, the worked example doesn't do this: >>> >> >>> >> The client sends a partial greeting (11 octets) greeting to the server, >>> >> and at the same time (before receiving anything from the client), the >>> >> server >>> >> also sends a partial greeting >>> > >>> > >>> >> The server validates the socket type, accepts it, and replies with a >>> >> READY >>> >> command >>> > >>> > >>> > The partial greeting isn't quite against the word of the spec though it >>> > would be good if it was described more clearly in the spec itself, >>> > however >>> > waiting to send the READY until after validating the client's READY goes >>> > directly against the MUST requirement. >>> > >>> > Am I missing something or does the spec and/or worked example need >>> > changing? >>> > >>> > Regards, >>> > Elliot >>> > >>> > >>> > ___ >>> > zeromq-dev mailing list >>> > zeromq-dev@lists.zeromq.org >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> ___ >>> zeromq-dev mailing list >>> zeromq-dev@lists.zeromq.org >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
OK, there's an error in the explanation here. It's not symmetric. The client is the one that needs to send its READY upfront. The server is the one that checks the READY properties before sending its own READY. The example is correct; the text is not. Thanks for pointing this out. On Mon, May 23, 2016 at 1:54 PM, Elliot Crosby-McCullough wrote: > Hi Pieter, > > You're right, I noticed the more detailed description of the partial > signature read shortly after posting, but I'm still not sure about the > worked example. As I mentioned there's also the conflict between waiting > for a READY, validating it, *then* sending your own READY as per the worked > example, while the spec says to send READY without waiting for the other > side to do so. > > All the best, > Elliot > > > On 22 May 2016 at 19:38, Pieter Hintjens wrote: >> >> The partial read is to deal with older versions of ZMTP. Peers that do >> this won't deadlock since as soon as they've read the first 11 bytes >> they can send the rest of their own greeting, then they can validate >> the socket type, send READY, then wait for a READY. Peers that don't >> do backwards compatibility just send the whole greeting, wait for the >> greeting, validate it, etc. >> >> >> On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough >> wrote: >> > Hi there! >> > >> > I'm working my way through an implementation of 37/ZMTP >> > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies in the >> > spec, mostly the timing of message exchanges, and how it differs in the >> > spec >> > proper from the worked example. >> > >> > Here's a couple of quotes where anti-deadlock approaches are discussed: >> >> >> >> A peer that reads a full greeting, including mechanism, MUST also send >> >> a >> >> full greeting including mechanism. This avoids deadlocks in which two >> >> peers >> >> each wait for the other to divulge the remainder of their greeting. >> > >> > >> >> Note that to avoid deadlocks, each peer MUST send its READY command >> >> before >> >> attempting to receive a READY from the other peer. In the NULL >> >> mechanism, >> >> peers are symmetric. >> > >> > >> > However, the worked example doesn't do this: >> >> >> >> The client sends a partial greeting (11 octets) greeting to the server, >> >> and at the same time (before receiving anything from the client), the >> >> server >> >> also sends a partial greeting >> > >> > >> >> The server validates the socket type, accepts it, and replies with a >> >> READY >> >> command >> > >> > >> > The partial greeting isn't quite against the word of the spec though it >> > would be good if it was described more clearly in the spec itself, >> > however >> > waiting to send the READY until after validating the client's READY goes >> > directly against the MUST requirement. >> > >> > Am I missing something or does the spec and/or worked example need >> > changing? >> > >> > Regards, >> > Elliot >> > >> > >> > ___ >> > zeromq-dev mailing list >> > zeromq-dev@lists.zeromq.org >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Fwd: 37/ZMTP RFC inconsistencies
Hi Pieter, You're right, I noticed the more detailed description of the partial signature read shortly after posting, but I'm still not sure about the worked example. As I mentioned there's also the conflict between waiting for a READY, validating it, *then* sending your own READY as per the worked example, while the spec says to send READY without waiting for the other side to do so. All the best, Elliot On 22 May 2016 at 19:38, Pieter Hintjens wrote: > The partial read is to deal with older versions of ZMTP. Peers that do > this won't deadlock since as soon as they've read the first 11 bytes > they can send the rest of their own greeting, then they can validate > the socket type, send READY, then wait for a READY. Peers that don't > do backwards compatibility just send the whole greeting, wait for the > greeting, validate it, etc. > > > On Sun, May 22, 2016 at 10:54 AM, Elliot Crosby-McCullough > wrote: > > Hi there! > > > > I'm working my way through an implementation of 37/ZMTP > > (http://rfc.zeromq.org/spec:37) and there's a few inconsistencies in the > > spec, mostly the timing of message exchanges, and how it differs in the > spec > > proper from the worked example. > > > > Here's a couple of quotes where anti-deadlock approaches are discussed: > >> > >> A peer that reads a full greeting, including mechanism, MUST also send a > >> full greeting including mechanism. This avoids deadlocks in which two > peers > >> each wait for the other to divulge the remainder of their greeting. > > > > > >> Note that to avoid deadlocks, each peer MUST send its READY command > before > >> attempting to receive a READY from the other peer. In the NULL > mechanism, > >> peers are symmetric. > > > > > > However, the worked example doesn't do this: > >> > >> The client sends a partial greeting (11 octets) greeting to the server, > >> and at the same time (before receiving anything from the client), the > server > >> also sends a partial greeting > > > > > >> The server validates the socket type, accepts it, and replies with a > READY > >> command > > > > > > The partial greeting isn't quite against the word of the spec though it > > would be good if it was described more clearly in the spec itself, > however > > waiting to send the READY until after validating the client's READY goes > > directly against the MUST requirement. > > > > Am I missing something or does the spec and/or worked example need > changing? > > > > Regards, > > Elliot > > > > > > ___ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] What are the “serious caveats” with ZMQ_FD?
It seems that sending on the socket makes the ZMQ_FD readable. That in turn triggers the calling of the code where I check ZMQ_EVENTS and receive everything from the socket. This way I end up checking ZMQ_EVENTS after each send. Is the ZMQ_FD becoming readable when sending on the socket something that can be relied on? -- Kalle Huttunen ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
