On Wed, Feb 4, 2015 at 10:51 AM, Pieter Hintjens <p...@imatix.com> wrote:
> The discussion about thread safety was quite short iirc, though that > contributor did discuss other things... at length. I merged his > "thread safe socket" change rapidly, then we reverted it after a few > days, and he disappeared. It was rather brute force and I suspect did > not work at all, it simply wrapped all accesses to the socket > structure in mutexes. No discussion at the time of multipart data and > atomic send/recv. > My memory of the conversation at the time is pretty dim, I agree the changes were ugly and untested and the contributor was difficult to reason with and seemed to want to make the changes based on no real need at all. > > As for socket safety, I've no strong opinion. I see that many people > expect that to work and hit errors when it doesn't. I see that nanomsg > has threadsafe sockets and no multipart. I see that sharing sockets > across threads would make some actor models simpler, which is nice. > This is the classic problem with thread safe anything. Threads are hard, and there is a balance between the complexity of making a thread safe construct and the skill required of a programmer to use "unsafe" construct in a safe manner. I still think if the concrete problem is very short lived threads causing slow joiner problems, then the simple solution is pools (troupes of actors?). -Michel > > > > > > > On Wed, Feb 4, 2015 at 7:35 PM, Michel Pelletier > <pelletier.mic...@gmail.com> wrote: > > I think Brian has some good points here, there are numerous unrelated > issues > > being discussed in this thread. > > > > A few points that I have: > > > > Multi part messages have also bothered me. However as a Python > programmer i > > see Min's points about the expense of buffer creation. To my knowledge > > zproto does not (yet) have Python generation support either, or maybe > > something like generated cffi or ctypes wrappers around the zproto > generated > > C code. That being said there are a variety of serialization libraries > for > > Python. With some ctypes and mmap magic they can also be done "zero > copy" > > but it's not pretty: > > > > https://gist.github.com/michelp/7522179 > > > > Multi part envelops are also how multi-hop routing is done. I don't see > how > > the new ideas handle that. I don't think we can just say "multi hop > routing > > is bad" and get rid of it. > > > > "Thread safe" sockets do not sound appealing to me. We did that, had a > long > > and contentious discussion with the person championing them, merged it, > then > > reverted it and that person is now no longer in the community. Pieter > was > > the most vocal opponent to them then and now he wants them back. Of > course, > > anyone can change their mind, but the only current argument I hear now > for > > them though is improving the performance of short lived threads, but that > > can be solved, more correctly in my opinion, with thread or connection > > pools. If you creating and tearing down threads that rapidly then you > have > > two problems. > > > > -Michel > > > > On Wed, Feb 4, 2015 at 3:37 AM, Brian Knox <bk...@digitalocean.com> > wrote: > >> > >> After catching up on this thread, I feel like at least three problems > are > >> being conflated into one problem. I'll state what I see being discussed > >> from my perspective: > >> > >> 1. "Using multi part messages as a way to route to clients from a router > >> socket is overly complicated and not how new users expect things to > work" > >> > >> 2. "Using multi part messages for message serialization is costly, and > >> potentially confusing to others." > >> > >> 3. "ZeroMQ sockets are not thread safe." > >> > >> While on an implementation level these three problems may be related, > on a > >> conceptual level I don't see them as related. I may agree with some of > >> these problem statements and not others. > >> > >> For me, my first priority is to always have the ability to get back a > nice > >> agnostic blob of bytes from ZeroMQ. This makes it easy to make ZeroMQ > >> socket use compatible with standard io interfaces in Go. Structure for > what > >> is contained in those bytes is a concern of a different layer. > Sometimes I > >> use zproto for this (which I like), and other times I don't. > >> > >> As a demonstration that the problems are different problems, I solved #1 > >> for myself in goczmq without addressing anything else. > >> > >> I would assert some of the confusion in this discussion is that we're > >> talking about multiple problem statements at the same time. > >> > >> Cheers - and it was great meeting people this week! > >> > >> Brian > >> > >> > >> > >> > >> On Wed, Feb 4, 2015 at 12:50 AM, Pieter Hintjens <p...@imatix.com> wrote: > >>> > >>> Ironically, in my testing of high message rate), allowing multipart > >>> creates significant costs. Multipart is just one way of getting > >>> zero-copy, and even then only works on writing, not reading. > >>> > >>> For high performance brokers like Malamute I'd *really* like to be > >>> moving blobs around instead of lists of blobs. > >>> > >>> > >>> On Wed, Feb 4, 2015 at 12:41 AM, Gregg Irwin <gr...@pointillistic.com> > >>> wrote: > >>> > M> Perhaps it is because I spend my days in a higher level language > >>> > M> like Python, but zproto is not an attractive option. > >>> > > >>> > Same here. I will read in detail about it shortly, but it may not > make > >>> > it into my toolbox as a multipart replacement. Multipart looked very > >>> > cool when I found 0MQ, but I've ended up not using it much. I'm not > >>> > doing high performance stuff though. Simplicity and ease of use are > >>> > tops on my list. > >>> > > >>> > -- Gregg > >>> > > >>> > _______________________________________________ > >>> > zeromq-dev mailing list > >>> > zeromq-dev@lists.zeromq.org > >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >>> _______________________________________________ > >>> zeromq-dev mailing list > >>> zeromq-dev@lists.zeromq.org > >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> > >> > >> > >> _______________________________________________ > >> zeromq-dev mailing list > >> zeromq-dev@lists.zeromq.org > >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >> > > > > > > _______________________________________________ > > zeromq-dev mailing list > > zeromq-dev@lists.zeromq.org > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
