@Doron: Yes I like multi part because its a very simple way build
"multiple-fields message". I once considered using protobuff + raw
socket for a pet project of mine, and I ended using ZMQ framing
because it was simpler.
The problem if we pick an other serialization solution is that it
needs to built-in libzmq. Per-binding serialization would be a
disaster IMO.
> Arnaud - you can do that today, just use synchronization object, however this
> is not really work, what if you call receive and there is no message
> currently? so you block and the socket is still locked, maybe for long time.
> Now you want to send from another thread but socket is locked.
Ah yes that's true, you'd have to use non-blocking read and such. I
reckon this is clearly not as practical as a fully thread safe socket
would be.
> Multi-part does easy serialization, yes. Not in C though. We've
> learned how to do variable list serialization with zproto-compatible
> picture sending in CZMQ. Is this a sufficient alternative? I think so.
> zsock_send() was essential to making actors cheap enough to use
> widely. egrep zsock_send *.c in malamute-core/src.
I'd argue that using zmsg_addstr() / zmsg_add() / zmsg_pop() are fine
for serialization. Agreed, when using the raw libzmq API this is not
as nice.
> If you're doing multipart a lot in an application, you can replace
> that with zproto. You already have contracts then, why not make them
> explicit and documented? That's always a win in my experience.
Yes I'm using multi-part quite a lot. I manually documented those
contracts and they are enforced by assert() (for anything that does'nt
come from the external world, obviously).
I'd argue that using c++ stream operator is easier, but I'd really
have to convert part of my code to zproto and give it an honest try.
> Picture sending solves a cross-language problem, namely how to
> construct zproto-style commands without using dedicated codecs. While
> the API is C-ish, the functionality is not. I'd expect that to be
> wrapped in bindings.
Yes but I believe pictures are sometime not practical. (I think) they
require that the picture be known when the message is being created.
I have some code where I do the following (if it's that weird / bad
please tell me):
Some code create a message, push the identity frame and then some
application-level header.
Control flow is handed to some other function, along with a
reference to the message. This function push a "command" frame and
more data (type and how many are unkown to the previous piece of
code).
Control flow return to initial code, and send the message.
On the other side of the socket, I do something similar:
Pop application-level header. Pop the "command" frame.
Based on those 2 piece of data, call a function (again, along with
a reference to the message).
Function will expect some kind of data layout, will extract it and
perform work.
In this scenario, the first part of the code doesn't know (or care,
really) about the "real" content of the message. In case my
explanation sucked, this is a bit similar to the ZAP rfc: the request
contains some "must be there" frames, and then some "zero or more
opaque frames". In case of ZAP, those opaque frames only make sense to
mechanism specific code. In my example, they only make sense to some
specific message handling code. I do not really see how ZAP could be
that simple and nice to work w/ if it wasn't for multi-part messages.
> The main goal with this is simplicity and accuracy. I'm tired of
> teaching people ZeroMQ constructs that are clearly over-designed and
> disconnected from reality in subtle ways. Each apology for friction
> ("sorry, but don't use REQ-REP, they are kind of weird", "sorry,
> IDENTITY is not really identity", "sorry, message sometimes means
> frame and sometimes message") really signals costs up and down the
> stacks.
This is a valid point. However, a good read through the guide really
help understanding how things work. I wouldn't be that surprised if
a lot of people having trouble understanding frames vs message or
REQ-REP problem didn't really read the zguide.
So *maybe* a documentation effort is what is needed to better explain
those thing. I agree about the IDENTITY tho.
I do not know, I don't have much experience teaching, and none teaching ZMQ.
Anyway, no matter what it's going to be interesting.
On Tue, Feb 3, 2015 at 11:04 PM, Pieter Hintjens <p...@imatix.com> wrote:
> Thanks for the questions. I'd try to flatter by saying how great these
> questions are, except I know you're all way too smart to fall for
> that. :)
>
> Let me answer a bunch of questions in one go:
>
> * Sorry for not posting this here. I've written a more detailed
> analysis of the motivation for this thread (and experiments):
> http://hintjens.com/blog:84.
>
> * The main goal with this is simplicity and accuracy. I'm tired of
> teaching people ZeroMQ constructs that are clearly over-designed and
> disconnected from reality in subtle ways. Each apology for friction
> ("sorry, but don't use REQ-REP, they are kind of weird", "sorry,
> IDENTITY is not really identity", "sorry, message sometimes means
> frame and sometimes message") really signals costs up and down the
> stacks.
>
> * Sessions require protocol commands that are invisible to "normal"
> use. That means either protocol support (e.g. command frames) or some
> layering on top of ZeroMQ blobs.
>
> * Picture sending solves a cross-language problem, namely how to
> construct zproto-style commands without using dedicated codecs. While
> the API is C-ish, the functionality is not. I'd expect that to be
> wrapped in bindings.
>
> * Per-language serialization is an anti-pattern. It's common, and fine
> as such, yet ZeroMQ should IMO always strive to be cross-language.
>
> * REQ-REP between threads may make sense. We certainly use it (ZAP,
> for authentication).
>
> * Is this discussion C4 compatible? Yes, no, this is chatter, not a
> plan or roadmap. Until pull requests hit the road, it's all vapour.
> However the problems I restated in my blog article are real, to many
> people, and should be aired.
>
> * Multipart does easy serialization, yes. Not in C though. We've
> learned how to do variable list serialization with zproto-compatible
> picture sending in CZMQ. Is this a sufficient alternative? I think so.
> zsock_send() was essential to making actors cheap enough to use
> widely. egrep zsock_send *.c in malamute-core/src.
>
> * Locking on socketsL: we did in fact have a pull request for this
> once. It was... nasty. We reverted it. We breathed a sigh of relief.
> However looking back, the problem was valid and has never been solved
> since.
>
> * If you're doing multipart a lot in an application, you can replace
> that with zproto. You already have contracts then, why not make them
> explicit and documented? That's always a win in my experience.
>
> * Multi-hop request reply is a strange beast and I'm keen to
> understand those architectures that use it. My suspicion is that we've
> shifted our vision of what ZeroMQ "is" over the years. My own
> experience seems plausible, yet I distrust it. So, the stories in my
> article are please for falsification, not attempts to convince.
>
> * I think we have learned to not break existing code. It's going to be
> interesting to run multiple models at once. We do this in CZMQ and it
> works OK. There isn't much overhead.
>
> * Authentication and routing information should be (MUST be) two
> totally separate layers. CURVE certainly does not use the routing ID
> (even if it's still misnamed as "IDENTITY"). You can have the same
> client on N connections (obviously). You authenticate a client using a
> mechanism-specific key: PLAIN user id, or CURVE public key.
>
> -Pieter
> _______________________________________________
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
--
Kapp Arnaud - Xaqq
_______________________________________________
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
