The documentation for Manifold can explain the API better than I can here. The point where that interacts with Netty w.r.t. backpressure is here: https://github.com/ztellman/aleph/blob/0.4.0/src/aleph/netty.clj#L109. Here the stream represents data coming off the wire, and if the put onto the stream is not immediately successful, backpressure is enabled until the put completes. No blocking required anywhere.
On Wed, Oct 8, 2014 at 10:10 AM, Jozef Wagner <jozef.wag...@gmail.com> wrote: > If you want to handle multiple TCP connections and async channels in one > thread, you need a way how to block on both connections (wait for new input > to arrive) and channels (wait for a free space in a buffer). Blocking only > on connections will get you a busy loop if channels are full. If you could > point me to the part of Aleph sources that handles this issue, I would be > very grateful. I'm not familiar with netty API nor manifold's concepts, so > I'm having trouble navigating in the Aleph sources. > > Thanks, > Jozef > > On Wednesday, October 8, 2014 6:15:47 PM UTC+2, Zach Tellman wrote: >> >> I wasn't aware of hermod, that's interesting. I would still characterize >> its approach to backpressure as "broken", though, since when the queues get >> full it silently drops messages on the ground. In fairness, this is very >> clearly documented, so it's less pernicious than some of the other cases >> out there. >> >> Both core.async buffers and SelectableChannels have very particular >> semantics, and I would be very surprised if they could be combined in that >> way. It's perfectly possible to feed one into the other and handle >> backpressure properly (again, I'm doing just that with Aleph 0.4.0, using >> Netty), but it's a nuanced integration and easy to get wrong. >> >> On Wed, Oct 8, 2014 at 7:12 AM, <adrian...@mail.yu.edu> wrote: >> >>> Check out https://github.com/halgari/com.tbaldridge.hermod for an >>> interesting take on this. >>> >>> On Wednesday, October 8, 2014 1:17:11 AM UTC-4, Sun Ning wrote: >>>> >>>> BTW, is there any network based core.async channel available now? >>>> >>>> On 10/08/2014 04:36 AM, adrian...@mail.yu.edu wrote: >>>> >>>> It's not about 'safety' (depending on what that means in this >>>> context), but as Zach pointed out, if you aren't careful about backpressure >>>> you can run into performance bottlenecks with unrestrained async IO >>>> operations because although they let you code as if you could handle an >>>> unlimited amount of connections, obviously that isn't true. There is only a >>>> finite amount of data that can be buffered in and out of any network >>>> according to its hardware. When you don't regulate that, your system will >>>> end up spending an inordinate amount of time compensating for this. You >>>> don't need to worry about this with "regular io" because the "thread per >>>> connection" abstraction effectively bounds your activity within the >>>> acceptable physical constraints of the server. >>>> >>>> On Tuesday, October 7, 2014 2:49:30 PM UTC-4, Brian Guthrie wrote: >>>>> >>>>> >>>>> On Mon, Oct 6, 2014 at 12:10 AM, <adrian...@mail.yu.edu> wrote: >>>>> >>>>>> Zach makes an excellent point; I've used AsyncSocketChannels and its >>>>>> irk (http://docs.oracle.com/javase/8/docs/api/java/nio/channels/ >>>>>> AsynchronousServerSocketChannel.html), with core.async in the past. >>>>>> Perhaps replacing your direct java.net.Sockets with nio classes that can >>>>>> be >>>>>> given CompletionHandlers (http://docs.oracle.com/javase >>>>>> /7/docs/api/java/nio/channels/CompletionHandler.html) would be a >>>>>> better fit. >>>>>> >>>>> >>>>> Once I do some performance instrumentation I'll give that a shot. I >>>>> admit that I'm not familiar with all the implications of using the nio >>>>> classes; were I to switch, is it safe to continue using go blocks, or is >>>>> it >>>>> worth explicitly allocating a single thread per socket? >>>>> >>>>> Brian >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Clojure" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>> topic/clojure/TVMQJwaij1U/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/TVMQJwaij1U/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
