Yes, I didn't mean to imply that there are no blocking operations anywhere, only that nothing in the worker threads (where the put! occurs) will block, and that backpressure isn't exerted by causing an active thread to hang.
As to the second point, I'm not sure why you'd think that. These aren't futures that are being accessed via blocking dereference, so there's no reason any thread would block as a result of an unrealized future/deferred. On Wed, Oct 8, 2014 at 12:22 PM, Jozef Wagner <jozef.wag...@gmail.com> wrote: > Thank you! Using put! callback to control backpressure is a very elegant > solution. BTW there always has to be blocking somewhere. Netty uses > selector to block at [1] and .setAutoRead causes respective channel to > deregister itself from a selector [2], until put! completes. > > Regarding the other way around, it seems to me from the quick look (sorry > if I misinterpreted) that using futures to represent write completion will > cause the eventual backpressure to block the thread, causing analogous > drawback as a blocking put! in the "reading from multiple connection" case. > > [1] > https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/NioEventLoop.java#L625 > [2] > https://github.com/netty/netty/blob/220660e351b2a22112b19c4af45e403eab1f73ab/transport/src/main/java/io/netty/channel/nio/AbstractNioChannel.java#L173 > > Jozef > > On Wednesday, October 8, 2014 7:16:44 PM UTC+2, Zach Tellman wrote: >> >> The documentation for Manifold can explain the API better than I can >> here. The point where that interacts with Netty w.r.t. backpressure is >> here: https://github.com/ztellman/aleph/blob/0.4.0/src/ >> aleph/netty.clj#L109. Here the stream represents data coming off the >> wire, and if the put onto the stream is not immediately successful, >> backpressure is enabled until the put completes. No blocking required >> anywhere. >> >> On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner <jozef....@gmail.com> >> wrote: >> >>> If you want to handle multiple TCP connections and async channels in one >>> thread, you need a way how to block on both connections (wait for new input >>> to arrive) and channels (wait for a free space in a buffer). Blocking only >>> on connections will get you a busy loop if channels are full. If you could >>> point me to the part of Aleph sources that handles this issue, I would be >>> very grateful. I'm not familiar with netty API nor manifold's concepts, so >>> I'm having trouble navigating in the Aleph sources. >>> >>> Thanks, >>> Jozef >>> >>> On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote: >>>> >>>> I wasn't aware of hermod, that's interesting. I would still >>>> characterize its approach to backpressure as "broken", though, since when >>>> the queues get full it silently drops messages on the ground. In fairness, >>>> this is very clearly documented, so it's less pernicious than some of the >>>> other cases out there. >>>> >>>> Both core.async buffers and SelectableChannels have very particular >>>> semantics, and I would be very surprised if they could be combined in that >>>> way. It's perfectly possible to feed one into the other and handle >>>> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using >>>> Netty), but it's a nuanced integration and easy to get wrong. >>>> >>>> On Wed, Oct 8, 2014 at 7:12 AM, <adrian...@mail.yu.edu> wrote: >>>> >>>>> Check out https://github.com/halgari/com.tbaldridge.hermod for an >>>>> interesting take on this. >>>>> >>>>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote: >>>>>> >>>>>> BTW, is there any network based core.async channel available now? >>>>>> >>>>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote: >>>>>> >>>>>> It's not about 'safety' (depending on what that means in this >>>>>> context), but as Zach pointed out, if you aren't careful about >>>>>> backpressure >>>>>> you can run into performance bottlenecks with unrestrained async IO >>>>>> operations because although they let you code as if you could handle an >>>>>> unlimited amount of connections, obviously that isn't true. There is >>>>>> only a >>>>>> finite amount of data that can be buffered in and out of any network >>>>>> according to its hardware. When you don't regulate that, your system will >>>>>> end up spending an inordinate amount of time compensating for this. You >>>>>> don't need to worry about this with "regular io" because the "thread per >>>>>> connection" abstraction effectively bounds your activity within the >>>>>> acceptable physical constraints of the server. >>>>>> >>>>>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote: >>>>>>> >>>>>>> >>>>>>> On Mon, Oct 6, 2014 at 12:10 AM, <adrian...@mail.yu.edu> wrote: >>>>>>> >>>>>>>> Zach makes an excellent point; I've used AsyncSocketChannels and >>>>>>>> its irk (http://docs.oracle.com/javase/8/docs/api/java/nio/ >>>>>>>> channels/AsynchronousServerSocketChannel.html), with core.async in >>>>>>>> the past. Perhaps replacing your direct java.net.Sockets with nio >>>>>>>> classes >>>>>>>> that can be given CompletionHandlers (http://docs.oracle.com/javase >>>>>>>> /7/docs/api/java/nio/channels/CompletionHandler.html) would be a >>>>>>>> better fit. >>>>>>>> >>>>>>> >>>>>>> Once I do some performance instrumentation I'll give that a shot. I >>>>>>> admit that I'm not familiar with all the implications of using the nio >>>>>>> classes; were I to switch, is it safe to continue using go blocks, or >>>>>>> is it >>>>>>> worth explicitly allocating a single thread per socket? >>>>>>> >>>>>>> Brian >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Clojure" group. >>>>>> To post to this group, send email to clo...@googlegroups.com >>>>>> Note that posts from new members are moderated - please be patient >>>>>> with your first post. >>>>>> To unsubscribe from this group, send email to >>>>>> clojure+u...@googlegroups.com >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/clojure?hl=en >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Clojure" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to clojure+u...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Clojure" group. >>>>> To post to this group, send email to clo...@googlegroups.com >>>>> Note that posts from new members are moderated - please be patient >>>>> with your first post. >>>>> To unsubscribe from this group, send email to >>>>> clojure+u...@googlegroups.com >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/clojure?hl=en >>>>> --- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "Clojure" group. >>>>> To unsubscribe from this topic, visit https://groups.google.com/d/to >>>>> pic/clojure/TVMQJwaij1U/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> clojure+u...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Clojure" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>> topic/clojure/TVMQJwaij1U/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/TVMQJwaij1U/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
