Hi Sam, > 27 maj 2015 kl. 12:01 skrev Sam Halliday <sam.halli...@gmail.com>: > > Hi Roland, > > On Wednesday, 27 May 2015 08:18:09 UTC+1, rkuhn wrote: > > For this venture you need no HTTP documentation > > Actually I'd like to go futher than akka-http and see documentation that > allows me to use WebSockets with akka-io directly, bypassing the > Streaming API. > > The historic limitation has always been that the HTTP > ConnectionManager of akka-io was unable to handle the upgrade > request. Wandoulabs use their own actor to handle HTTP and UHTTP, > but it has annoying side effects. I'm interested to know how > akka-http has managed to implement WebSockets with that > constraint in place (or if that constraint has been lifted).
Maybe I misunderstand you still, but ConnectionManager is not something that we have in Akka HTTP, and with akka-io you seem to mean the akka.io package in akka-actor, which is not related to HTTP as far as user API is concerned—Streams have TCP support that abstracts over the low-level mechanism and provides higher-level semantics including back-pressure. Bypassing these mechanisms is neither desirable nor possible. The whole architecture of the Wandoulabs solution is completely unrelated to how Akka HTTP works, so the constraint you mention (which I don’t know more about, never having used that library) is unlikely to exist; but we also did not “lift” it since we completely replaced the mechanism wholesale. When thinking about Streams it is best to first forget Actors, learn how to work with Streams alone, and then as an advanced topic interface with plain Actors. > > > > The statement “there are no promises around back-pressure” > > indicates that you did not, in fact, understand the full extent > > of what Akka Streams are. > > Just because something is a Flow does not make any promises about > how back pressure is actually implemented in that flow: you have > even pointed out how to create "open hoses", or just blow up > internal buffers when downstream doesn't consume fast enough. This is not true: a Flow has one open input and one open output port and all data elements that flow through these ports will do so governed by the Reactive Streams back-pressure semantics. This means that the Flow has the ability to slow down the Source that is connected to it and it also reacts to slow-down requests from the Sink that it will be connected to. It is the Sink.actorRef implementation that presents the proper “hose” interface but then just lets the water fall into the bucket. There is also a Sink.ignore which replaces the bucket with a black hole … > > I'd like to know if the underlying Source of incoming client > messages on a websocket endpoint will respect the backpressure > from the `handleWebsocketMessages: Flow` that is dealing with > it (i.e. not read from the network unless > `handleWebsocketMessages` is pulling), and conversely if the > underlying Sink back to the client is going to backpressure using > akka-io's Ack mechanism so that `handleWebsocketMessages` will > only be pulled when akka-io gives the green lights. Yes, of course, this is what the Flow interface designates as explained above. And you are also right that a few layers down akka.io’s Ack and ResumeReading mechanisms are used to propagate the back-pressure to and from the TCP socket. > > > > We’d love to improve the documentation in this regard, but we’d > > need to first figure out where their deficiency is, hence I’m > > talking with you. > > I think clarity on the above two points would be a good start. In > addition, and more generally, integration between "hoses" > and "hammers" is extremely important --- unless you intentionally > want to limit akka-streams uptake to green field projects only. > The project I work on has 1.2 million lines of Scala code with > legacy components dating from Scala 2.6. There isn't a snowball's > chance in hell of rewriting it to use akka-streams. Right. My responses to you have been deliberately cautious in order to make it clear that for a successful integration between hammers and hoses you will need a solid understanding of each of those in isolation. I think that point got across :-) Now, looking at Integrating With Actors <http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-RC3/scala/stream-integrations.html> you’ll see ActorPublisher and ActorSubscriber. These two helpers handle some of the Reactive Streams protocol primitives but leave the exact use of back-pressure and data generation up to you, I think this is what you are looking for. Be aware, though, that combining both sides into one Actor—a so-called Processor—is more difficult than it seems due to failure and termination handling: if you forget to handle a certain ordering of events then the stream will not properly be torn down when “finished” and your websocket connections turn into Actor leaks. I admit to being a bit dramatic here, but I don’t want to leave the dangers unmentioned. There is another way of integrating arbitrary asynchronous elements into a Stream, but it is not yet documented since we want to first gain more experience with it internally: AsyncStage <https://github.com/akka/akka/blob/akka-stream-and-http-experimental-1.0-RC3/akka-stream/src/main/scala/akka/stream/stage/Stage.scala#L293>. This limited abstraction provides all the stream safety properties automatically and leaves it to you when to request or generate elements. You can see it in action in the implementation of mapAsync <https://github.com/akka/akka/blob/akka-stream-and-http-experimental-1.0-RC3/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala#L382>. My goal is to refine this into the best all-round tool for Stream–Actor integration, your feedback would of course be welcome! (but here you’ll have to take code for documentation for now) > > > >> What I'm missing is the ability to hook an existing actor > >> system into something that expects a Flow, with back pressure > >> preserved. > > > > As I hopefully explained in my other mail about hoses: your > > Actors would need to implement the full spec, they’d need to be > > watertight. > > This is a start. Is there a test framework that can be used to > stress test the implementation of a Stream / Actor bridge? There is the Reactive Streams TCK, see http://www.reactive-streams.org/ <http://www.reactive-streams.org/> . > > > > How we implement Flows internally should be of no consequence > > On the contrary, I feel the implementation is of huge > significance. Firstly, it helps to understand the expected > performance, and second it is critical when writing > integration code. It is extremely bizarre that both projects > should be released under the akka banner, yet be so siloed. Hopefully I was able to provide enough background to show that we have indeed thought of the relation between Streams and Actors, but we still need to find an appropriate way of deflecting or avoiding the initial/intuitive reaction of wanting to have Stream-Actors prematurely. Streams are really a much nicer abstraction as soon as you forget about Actors for a second. > > > > In order to solve this particular problem you’ll need to > > carefully describe the back-pressure protocol spoken by your > > Actor. > > The Actor is expecting to send messages directly to akka-io and > speaks the akka-io Ack protocol using a simple object as the Ack > message. It expects an Ack before it will send a message > upstream (and it greedily consumes data, but that could easily be > changed). As explained above directly interacting with akka.io will not happen—there are multiple protocol layers between the ByteStrings and the websocket data format messages—but you might find inspiration on how to use AsyncStage in the following (feel free to re-read after having digested the Stage abstraction): when the downstream ball comes in (i.e. onPull—demand from downstream [which you seem to call upstream]) dispatch an Ack to your Actor and holdDownstream when message comes back from your Actor (via onAsyncInput()), push() it downstream towards the websocket when the upstream ball comes in with a message from the client (i.e. onPush) dispatch to your Actor and pull() if the Actor has previously signaled demand, or holdUpstream when demand is signaled by your Actor, record that and/or release the ball upstream (if previously held) to read more data from the websocket Regards, Roland > > Best regards, > Sam > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ <http://akka.io/docs/> > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> <http://doc.akka.io/docs/akka/current/additional/faq.html> > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > >>>>>>>>>> <https://groups.google.com/group/akka-user> > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com > <mailto:akka-user+unsubscr...@googlegroups.com>. > To post to this group, send email to akka-user@googlegroups.com > <mailto:akka-user@googlegroups.com>. > Visit this group at http://groups.google.com/group/akka-user > <http://groups.google.com/group/akka-user>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. Dr. Roland Kuhn Akka Tech Lead Typesafe <http://typesafe.com/> – Reactive apps on the JVM. twitter: @rolandkuhn <http://twitter.com/#!/rolandkuhn> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
