I think this is an excellent proposal. I've been out there talking to lots of users that are using (or want to use) Akka Persistence and most of them are struggling with the current complexity of the Processor/Channel/Persistent combo and when to use it and when to rely on EventsourcedProcessor instead. I also find it a lot more natural and intuitive to explain how pure event sourcing works since it makes the concepts and design very clean; what is side-effects, what is state change and what is a fact—and not having to expose persistence to the client. To be honest, EP is the only thing I've been willing to talk about and recommend up to now. I'd be very happy to see P/C go. That would make Akka Persistence really simple actually.
I also agree with Patrik. I've seen PersistentChannel as something very separate from P/C/EP, with very different use-cases. We need to fully understand what we are getting rid of and what the alternatives are if we remove it. On Fri, May 9, 2014 at 1:16 PM, Patrik Nordwall <patrik.nordw...@gmail.com>wrote: > Hi all, > > Interesting discussion. I have no problem removing command sourced > Processor from user API, with the added capabilities of > EventsourcedProcessor. I have thought of P as a low level building block > and only to be used for simple write-ahead-log scenarios. External > side-effects during replay makes me shiver. > > I'm not so sure about removing PersistentChannel. It's a useful > at-least-once delivery tool. It is not that easy to implement this stuff > yourself. For example, how to handle unbounded number of unacknowledged > requests? PersistentChannel does not store everything in memory. How to > avoid flooding slow consumers? > > My canonical use case for PersistentChannel is the safe-handoff between > system/component boundaries. > > Frontend server provides REST endpoint to external system. When it > replies OK 200 it means that it has taken over responsibility for the > request. When it replies ERR 5xx it means that the responsibility is still > at the requesting system, i.e. it has to retry later. The replies must be > done within milliseconds. > > Frontend endpoint sends the request to backend service via a > PersistentChannel, and when it has received the ACK from the > PersistentChannel it can reply. > > Isn't this (and variations of it) something that we should provide a tool > for? > > Cheers, > Patrik > > > > > On Fri, May 9, 2014 at 12:15 PM, ahjohannessen <ahjohannes...@gmail.com>wrote: > >> Hi Roland, >> >> We use Channel in conjunction with Eventsourced Processor (EP) in our >> applications in receiveRecover. >> It would be sad to see it go away without a reasonable alternative. >> >> One scenario in our apps is that we use DDD/ES and have a lot of EPs of >> same type, e.g. 10000 instances, >> that are loaded on demand by a supervisor. >> >> In order to have a single view on all of these, we inject an actor that >> wraps a single channel / aggregator EP combo >> into these instances on creation. This makes it possible to react to >> changes, even in case of JVM crashes, >> in that family of EPs as well as having a global view of that family. >> >> >> On Friday, May 9, 2014 9:13:17 AM UTC+1, rkuhn wrote: >>> >>> >>> 9 maj 2014 kl. 09:58 skrev Martin Krasser <kras...@googlemail.com>: >>> >>> >>> On 09.05.14 09:25, Roland Kuhn wrote: >>> >>> >>> 9 maj 2014 kl. 09:08 skrev Martin Krasser <kras...@googlemail.com>: >>> >>> >>> On 09.05.14 08:41, Roland Kuhn wrote: >>> >>> Hi Martin, >>> >>> 9 maj 2014 kl. 08:05 skrev Martin Krasser <kras...@googlemail.com>: >>> >>> Hi Roland, >>> >>> thanks for starting a discussion on this. Here are some initial thoughts >>> on your proposal: >>> >>> "... very same throughput optimization by applying the state changes >>> before persisting them ..." >>> >>> I think we agree that whatever changes are going to be made in the >>> future, we must keep the throughput optimizations (by batching >>> writes/updates). As you said, with an EP, this can only be achieved by >>> applying events to current state *before* persisting them. Furthermore, to >>> enable batching, an EP must therefore be able to process new commands while >>> (previous) events are about to be persisted. This however has a very >>> important consequence for commands that read current state. If we allow >>> events to be applied to current state *before* persisting them, we allow >>> clients to read state from that EP that may not be re-readable after a >>> crash. For example: >>> >>> - EP receives update command, derives event and applies it immediately >>> to current state >>> - EP (asynchronously) persists event >>> - EP receives a read command (while event persistence is in progress) >>> - EP (successfully) returns read response to requestor >>> - EP JVM crashes before event was successfully persisted >>> - EP state cannot be reconstructed i.e. previous read cannot be repeated. >>> >>> >>> This is only true if the recovery is incomplete: the update command >>> will not have been acknowledged at this point, so if someone cared about it >>> they will send it again during recovery and the EP will eventually end up >>> in a state where the read will return the same value again. If this type of >>> consistency is not good enough, then you can always defer reads within the >>> write model until after persistence is completed, meaning that the read is >>> only performed once a corresponding read “event” has gone through the >>> journal. We could allow events that are only looped through to make this >>> work, just like non-Persistent commands are looped today (and for the same >>> reason). >>> >>> >>> Delaying reads is only an option when reads are made via messages to a >>> (E)P. If my processor manages state via an STM ref where only the processor >>> updates the STM ref but reads go directly to the STM ref, then you cannot >>> delay reads. >>> >>> >>> In this scenario you would delay updating the STM ref until after the >>> persistence loop, which is exactly the same as for a current >>> command-sourced Processor: the read gets delayed until after the writes are >>> processed, in the same way the STM ref update gets delayed by the write >>> having to go through the journal. Effects, consistency and latency are the >>> same in both implementations. >>> >>> >>> That's true. So, to achieve >>> >>> - repeatable reads >>> - low read latency and >>> - high write throughput >>> >>> reads can go to the STM refs directly and EP must update the STM ref >>> only after having persisted the events. If one *additionally* wants to >>> achieve >>> >>> - read-your-own-write consistency (assuming a client issues an update >>> command, immediately followed by a read command) >>> >>> one would need a way to loop read commands through the journal as well >>> before serving them (which probably requires an addition to the API then). >>> Alternatively, a client only issues a read after having received a >>> write-ack (at the cost of an additional roundtrip). >>> >>> >>> This is an interesting remark: normally read-your-writes is only >>> guaranteed for reads submitted after having received the ACK for the write, >>> so what we are providing here is actually a qualitative improvement on that >>> status quo that is only possible in Reactive systems (normally the ACK is >>> signaled by a synchronous non-exceptional method return). >>> >>> Anyway, I think you convinced me, as usual :) Great proposal, Dr Kuhn! >>> >>> >>> And as usual you helped in refining the proposal: the addition of >>> looping non-persistent events through the journal is an important one, >>> thanks for providing the use-case! >>> >>> So, to summarize, we can incorporate all current functionality provided >>> by Processor and Channel into EventsourcedProcessor by adding the following >>> two features: >>> >>> >>> - the ability to opt out of stashing everything while waiting for >>> persist()ing >>> - the ability to loop non-persistent events through the journal >>> >>> >>> Everyone, please consider what this would mean for your code base and >>> comment, now is the right time to speak up! The same goes for opinions on >>> whether PersistentChannel pulls its weight or not (as argued earlier in >>> this thread). >>> >>> Regards, >>> >>> Roland >>> >>> >>> Cheers, >>> Martin >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> *Dr. Roland Kuhn* >>> *Akka Tech Lead* >>> Typesafe <http://typesafe.com/> – Reactive apps on the JVM. >>> twitter: @rolandkuhn >>> <http://twitter.com/#!/rolandkuhn> >>> >>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Patrik Nordwall > Typesafe <http://typesafe.com/> - Reactive apps on the JVM > Twitter: @patriknw > JOIN US. REGISTER TODAY! <http://www.scaladays.org/> > Scala <http://www.scaladays.org/> > Days <http://www.scaladays.org/> > June 16th-18th, <http://www.scaladays.org/> > Berlin <http://www.scaladays.org/> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- *Jonas Bonér*Phone: +46 733 777 123 Home: jonasboner.com Twitter: @jboner <https://twitter.com/jboner> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
