Can I ask why people think thread safety around sockets is in any way important? or even beneficial at all if it impacts performance negatively? Most libraries I've found aren't thread safe because it's cheaper to be that way and let the application handle any concurrency in a sensible way for whatever they are trying to solve.
On 2 February 2015 at 20:36, Thomas Rodgers <rodg...@twrodgers.com> wrote: > My eventual suggestion was to look at using metadata for this, it is > already there and is per-connection. But there are (currently) a few > problems with using it this way and I wanted to think those through a bit > before making the suggestion. > > I am not sure why file_desc was placed where it is. > > On Mon, Feb 2, 2015 at 2:25 PM, Doron Somech <somdo...@gmail.com> wrote: > >> Currently the router_id is in the last 4 bytes of the 64 bytes message >> size, could this cause alignment issue? >> >> Regarding the idea to combine it with the file descriptor, I think it can >> cause portability issues in the future, but it might be platform specific. >> >> Why wasn't it originally in end of the union which can solve the >> alignment issue? >> >> Anyway in the future we might not need it, because we can query the >> socket with the routing id to get metadata. >> >> On Mon, Feb 2, 2015 at 10:02 PM, Thomas Rodgers <rodg...@twrodgers.com> >> wrote: >> >>> Some observations on adding a discrete router_id field to the message - >>> >>> Depending on alignment, this reduces the size of a VSM message by 4 or 8 >>> bytes. >>> >>> Does this field really need to be a member of the union? Currently >>> msg_t::file_desc is 64 bit to force alignment, but FDs are 32bit values >>> (even on Windows, where the FD is 64 bits, it will never give values > >>> 2^32). Could we not just pack these two together, and make use of the >>> currently unused bits? It gets rid of the combinatorial expansion of adding >>> fields to each member of the union (at least for this case). >>> >>> On Mon, Feb 2, 2015 at 1:06 PM, Doron Somech <somdo...@gmail.com> wrote: >>> >>>> Brian - >>>> >>>> The issue is that with the new api and to enable thread safety the >>>> routing id is part of the message. So czmq new api should somehow expose >>>> routing id on the message and/or part of the send message. >>>> On Feb 2, 2015 7:39 PM, "Brian Knox" <bk...@digitalocean.com> wrote: >>>> >>>>> Doron - >>>>> >>>>> I did a test implementation around this idea in goczmq today and I'm >>>>> liking the semantics. I wrote an interface that conforms to io.Reader and >>>>> io.Writer where you pass a call a byte buffer and you get the message >>>>> placed into it. >>>>> >>>>> When the socket is a router socket, it silently drops the ID frame >>>>> under the hood, but sets a lastClientID in the socket struct, that if you >>>>> need, you can get with a GetLastClientID call. >>>>> >>>>> For reference: >>>>> >>>>> https://github.com/zeromq/goczmq/blob/master/sock.go#L35 (note >>>>> "strings" in go are just immutable byte arrays) >>>>> >>>>> https://github.com/zeromq/goczmq/blob/master/sock.go#L333 >>>>> >>>>> https://github.com/zeromq/goczmq/blob/master/sock.go#L366 >>>>> >>>>> So if the socket is a router, the Write call just transparently uses >>>>> the lastClientID value. In a one to one request / reply, you then don't >>>>> need to think about it at all, and if you need to do async work, you can >>>>> get and set the id as needed. >>>>> >>>>> I'll just move this to use the new API later - just wanted to try it >>>>> out and see how I liked it, and I say thumbs up to dropping frames on the >>>>> new Server / Client sockets. >>>>> >>>>> >>>>> >>>>> On Mon, Feb 2, 2015 at 4:37 PM, Doron Somech <somdo...@gmail.com> >>>>> wrote: >>>>> >>>>>> So the new sockets are in the github master, you can take a loot at >>>>>> the test to see how to use the new routing id field: >>>>>> >>>>>> >>>>>> https://github.com/zeromq/libzmq/blob/master/tests/test_client_server.cpp >>>>>> >>>>>> Few of the reasons I didn't like multi frames: >>>>>> * I tried in the past to make both zeromq and NetMQ thread safe, >>>>>> think of sending from multiple threads or receiving from multiple >>>>>> threads. >>>>>> we cannot do that with Multipart. >>>>>> * There are a lot of good transport that already implement message >>>>>> framing (UDP, WebSockets, SCTP and even HTTP), but because zeromq >>>>>> required >>>>>> it is own framing it was not easy to add them. >>>>>> * Multipart, router usage (routing id frame) and not being thread >>>>>> safe make the learning curve of zeromq hard to beginners. Without three >>>>>> of >>>>>> them zeromq become much simpler. >>>>>> * At least with NetMQ single part message is much faster than >>>>>> multipart. >>>>>> * New stacks, multipart is complicated to implement, with the new API >>>>>> it will much more easier to implement new stacks (like JSMQ) or any >>>>>> native >>>>>> stack. >>>>>> >>>>>> Pieter I'm looking forward to see how you expose the routing id in >>>>>> the czmq API. >>>>>> I also like the czmq API for sending mutlipart messages (the picture >>>>>> feature) so maybe we can use that to generate single frame which is also >>>>>> compatible with zproto. >>>>>> >>>>>> About the implementation, none of new sockets support any option now. >>>>>> server behave like mandatory router, so when router is not reachable or >>>>>> highwater mark is reached an error will be returned. >>>>>> >>>>>> As ZMTP 3.1 is still in raw status, what do you think of removing the >>>>>> multipart from it? maybe the 3.1 will only support the new socket types. >>>>>> >>>>>> Anyway I really excited about this change. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Feb 2, 2015 at 4:17 PM, Thomas Rodgers <rodg...@twrodgers.com >>>>>> > wrote: >>>>>> >>>>>>> What we really want IMO is per-peer metadata, and an API to get/set >>>>>>>> this. Using messages is a hack. >>>>>>> >>>>>>> >>>>>>> Currently working on that :) >>>>>>> >>>>>>> Having two layers that both >>>>>>>> carry per-message data is... wrong IMO. >>>>>>> >>>>>>> >>>>>>> Protocols supporting 'out of band' data aren't exactly uncommon. >>>>>>> >>>>>>> However the key thing is, what's the problem. Then we can discuss >>>>>>>> solutions to that. >>>>>>> >>>>>>> >>>>>>> I don't have an immediate use-case justifying it. I noted it, mostly >>>>>>> because it has come up a few times since I started paying attention. >>>>>>> >>>>>>> On Mon, Feb 2, 2015 at 7:55 AM, Pieter Hintjens <p...@imatix.com> >>>>>>> wrote: >>>>>>> >>>>>>>> > It seems to me that in order to remove multi-part messages and >>>>>>>> introduce new >>>>>>>> > socket types (e.g. SERVER/CLIENT) that would >>>>>>>> > necessitate a revision of the wire protocol. If we are going to >>>>>>>> do that, it >>>>>>>> > might be worth considering per-message >>>>>>>> > metadata - >>>>>>>> >>>>>>>> We'd have to be very clear about the problem that per-message >>>>>>>> metadata >>>>>>>> is aiming for. There is an elegance to delivering blobs and nothing >>>>>>>> more. Metadata can be added on top using zproto. >>>>>>>> >>>>>>>> What we really want IMO is per-peer metadata, and an API to get/set >>>>>>>> this. Using messages is a hack. If we are sending/receiving data on >>>>>>>> a >>>>>>>> per-message basis, that is the message. Having two layers that both >>>>>>>> carry per-message data is... wrong IMO. >>>>>>>> >>>>>>>> However the key thing is, what's the problem. Then we can discuss >>>>>>>> solutions to that. >>>>>>>> >>>>>>>> -Pieter >>>>>>>> _______________________________________________ >>>>>>>> zeromq-dev mailing list >>>>>>>> zeromq-dev@lists.zeromq.org >>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> zeromq-dev mailing list >>>>>>> zeromq-dev@lists.zeromq.org >>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> zeromq-dev mailing list >>>>>> zeromq-dev@lists.zeromq.org >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> zeromq-dev mailing list >>>>> zeromq-dev@lists.zeromq.org >>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>>> >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> zeromq-dev@lists.zeromq.org >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>>> >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> zeromq-dev@lists.zeromq.org >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >> >> _______________________________________________ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
