You can relatively easily support process managers on top (event store assures idempotency)
On Wed, Aug 20, 2014 at 5:46 PM, Gary Malouf <malouf.g...@gmail.com> wrote: > Greg - if one uses the current Akka Persistence with eventstore as the > backend, is it possible/what are the challenges in getting safe 'process > managers' to work as one would expect? I would think you'd want event > store feeding a different Akka Persistence processor. > > > On Wed, Aug 20, 2014 at 2:10 PM, Ashley Aitken <amait...@gmail.com> wrote: > >> >> Whilst we are talking about s... process managers I would like to include >> this simple way of understanding them I found on the web: "Process Managers >> produce commands and consume events, whereas Aggregate Roots consume >> commands and produce events." The truth is a bit more complicated I >> believe in that Process Managers can also consume commands (e.g. to stop >> the process). >> >> Further, whilst I would like to accept Roland's view that both commands >> and events can be communicated by sending messages (since, as he suggests, >> it would make things a lot simpler and lighter on the write side), I am >> concerned that there are use-cases for process managers that involve them >> listening for events from ARs they have not sent a command message to. Can >> anyone confirm/deny? >> >> Thanks, >> Ashley. >> >> >> >> On Wednesday, 20 August 2014 23:01:41 UTC+8, Greg Young wrote: >> >>> further explanation http://soa.dzone.com/news/are-sagas-and- >>> workflows-same-t >>> >>> >>> On Wed, Aug 20, 2014 at 10:39 AM, Greg Young <gregor...@gmail.com> >>> wrote: >>> >>> I held the same issue with ms pnp >>> >>> Clarifying the terminology >>> >>> The term saga is commonly used in discussions of CQRS to refer to a >>> piece of code that coordinates and routes messages between bounded contexts >>> and aggregates. However, for the purposes of this guidance we prefer to use >>> the term process manager to refer to this type of code artifact. There are >>> two reasons for this: >>> >>> There is a well-known, pre-existing definition of the term saga that has >>> a different meaning from the one generally understood in relation to CQRS. >>> The term process manager is a better description of the role performed by >>> this type of code artifact. >>> >>> Although the term saga is often used in the context of the CQRS pattern, >>> it has a pre-existing definition. We have chosen to use the term process >>> manager in this guidance to avoid confusion with this pre-existing >>> definition. >>> >>> The term saga, in relation to distributed systems, was originally >>> defined in the paper "Sagas" by Hector Garcia-Molina and Kenneth Salem. >>> This paper proposes a mechanism that it calls a saga as an alternative to >>> using a distributed transaction for managing a long-running business >>> process. The paper recognizes that business processes are often comprised >>> of multiple steps, each of which involves a transaction, and that overall >>> consistency can be achieved by grouping these individual transactions into >>> a distributed transaction. However, in long-running business processes, >>> using distributed transactions can impact on the performance and >>> concurrency of the system because of the locks that must be held for the >>> duration of the distributed transaction. >>> >>> >>> On Wed, Aug 20, 2014 at 10:31 AM, Roland Kuhn <goo...@rkuhn.info> wrote: >>> >>> >>> 20 aug 2014 kl. 16:16 skrev Greg Young <gregor...@gmail.com>: >>> >>> Please stop using the terminology of "saga" and replace usage with >>> "process manager" what people (largely influenced by nservicebus call a >>> saga is actually a process manager and a saga is a different pattern). Its >>> bad enough the .net community does this the last thing we need is for the >>> akka community to start doing the same :) >>> >>> >>> Sure, but please do educate us as to the right use of these two words so >>> we persist the correct definitions in the list archives. My main question >>> is: what is that other pattern that shall be called a Saga? >>> >>> Regards, >>> >>> Roland >>> >>> >>> >>> >>> On Wed, Aug 20, 2014 at 4:16 AM, Roland Kuhn <goo...@rkuhn.info> wrote: >>> >>> >>> 19 aug 2014 kl. 18:59 skrev Ashley Aitken <amai...@gmail.com>: >>> >>> On Tuesday, 19 August 2014 21:14:17 UTC+8, rkuhn wrote: >>> >>> >>> 18 aug 2014 kl. 18:01 skrev Ashley Aitken <amai...@gmail.com>: >>> >>> I believe Akka needs to allow actors to: >>> >>> >>> (i) persist events with as much information as efficiently possible on >>> the write side to allow the store to facilitate the read side extracting >>> them according to what criteria is needed, >>> >>> This is a convoluted way of saying that Events must be self-contained, >>> right? In that case: check! >>> >>> >>> No, I don't think so. As I understand it now, the only thing the event >>> store knows about each event is the persistenceId and a chunk of opaque >>> data. It doesn't know the type of the event, the type of the message, any >>> time information, any causal dependency etc. I guess what I am saying is >>> that the events need to include as much metadata as possible so that the >>> event store can provide the necessary synthetic streams if they are >>> requested by the read side. As I mentioned later, some event stores (like >>> Kafka may replicate the events into separate topics based on this >>> information), others (like Event Store) may use this information later to >>> form streams of links to the original events. >>> >>> >>> The event store has the full event available, which is all the >>> information there is: gathering or duplicating arbitrary parts of the >>> information is likely not going to help, because you will discover later >>> that you missed something initially, and if the mechanism is baked into the >>> Akka Persistence Journal SPI then fixing it will take a very long time >>> (until plugins are migrated and your OPS guys allow you to use it etc.). My >>> recommendation is to use a serialization mechanism that fits the Journal, >>> allowing it to understand the events and provide semantic features on top. >>> Both (Journal and serialization) are configured in the same file, so I >>> submit that coupling them is a valid approach. >>> >>> On causal consistency: I am still unconvinced that it is worth pursuing, >>> and I am certain that you are vastly underestimating the amount of data and >>> effort involved. And it cannot be done without collaboration from the user >>> since a single inter-Actor message outside of the traced system (i.e. not >>> using a PersistenceEnvelope of sorts) would hamper or destroy it. >>> >>> >>> (iii) read from (and replay) streams of events on the read and write >>> side according to a range of criteria supported and defined within the >>> store or via the store API (e.g. using a DSL), and >>> >>> This is the unclear point: who defines the query and when? What are the >>> consistency guarantees for the generated event stream? >>> >>> >>> I suggest the developers of the read side specify the queries directly >>> to the event store but this may be after the events have initially been >>> persisted. The event store produces the query stream (if it can) and a >>> PersistentView can be setup to read from that named query. With regards to >>> consistency guarantees - my understanding is that these streams are used to >>> eventually guarantee that the query model will be consistent with the write >>> model, i.e. all the events will get across. With regards to ordering I >>> think the event store does the best it can to provide consistent ordering, >>> e.g. total ordering if there was no distribution and causal ordering, where >>> possible, if there was ordering. The developer would need to understand >>> the limitations of how the query store is configured and queried. >>> >>> >>> As I answered to Greg already, I think that this should not be a core >>> concern of Akka Persistence; as you note it relies on features provided by >>> the underlying event store, and those features are not necessary to achieve >>> the goal of making actors persistent. >>> >>> >>> >>> >>> (iv) reliably (at least once) deliver information to other read side >>> store(s) and systems above and beyond the store used for persisting the >>> events. >>> >>> This is PersistentView, so “check!” (As argued previously “reliably” >>> translates to “persistent”.) >>> >>> >>> As I asked in another thread (I think) I am not sure how PersistentView >>> can do this when PersistentActor is the one that can mixin >>> AtLeastOnceDelivery? >>> >>> I think we need a PeristentView that can guarantee AtLeastOnceDelivery >>> to an actor representing a query store. This would seem to require a >>> PersistentViewActor ;-) that can read from a persistent query and also >>> persist its state to provide guaranteed delivery. >>> >>> My lack of knowledge of Scala and Akka may be showing here. >>> >>> >>> My current impression is that PersistentView needs to be re-thought: >>> instead of tying it to a persistenceId like we do now we should just >>> provide an API for subscribing to named topics in the Journal—be that >>> persistenceIds of some PersistentActors or synthetic ones. One Actor should >>> be able to subscribe to any number of them, but the onus will be on it to >>> keep track of the positions up to which it has consumed from all of them. >>> >>> This does not preclude the Journal from providing a synthetic topic with >>> proper linearization for all events in it (or whatever you want to >>> specifically configure on the storage back-end, outside of the Journal >>> SPI). And this also does not invalidate my point that normally the >>> consumption of Queries should be done directly from the backing store, >>> making full use of its unique feature set. >>> >>> >>> I believe each of these is readily achievable with Akka but: >>> >>> >>> (i) doesn’t mean explicitly persisting the events to specific topics as >>> you suggest in your (1) (although this may be how some stores implement the >>> required functionality on the read side). Instead it means transparently >>> including information like the actorId, event type, actor type, probably >>> the time and possibly information to help with causal ordering (see my next >>> post). >>> >>> No, again we need to strictly keep Topics and Queries separate, they are >>> very different features. Topics are defined up-front and explicitly written >>> to, Queries are constructed later based on the existing event log contents. >>> Marking events within the store with timestamps of some kind might help >>> achieving a pseudo-deterministic behavior, but it is by no means a >>> guarantee. Causal ordering is out of scope, and it also does not help in >>> achieving the desired ability to replay Queries from some given point in >>> the past. >>> >>> >>> I think we do agree somewhere in there but I don't think as was >>> suggested (by you earlier?) that creating topics up-front whether a fixed >>> set or arbitrary tags will work. I feel in what way the store supports the >>> queries (and how much it can) is up to the store (e.g. creating separate >>> topics or synthetic topics), so I would argue against using topics for >>> CQRS. As I mention below for Pub/Sub to Persistent topics it would be >>> great, but not for CQRS. >>> >>> >>> Yes, indeed, topics are interesting but not strictly related to CQRS. >>> >>> C. CQRS with Event Sourcing >>> >>> >>> And finally, there is CQRS with Event Sourcing, which I believe is much >>> more that (A) and (B) and particularly doesn’t necessarily require (B.) for >>> all event stores. So if Akka were to implement (B), which I think would be >>> very useful for other reasons, it would not specifically be for CQRS. >>> >>> >>> Please consider this diagram overviewing CQRS with Event Sourcing: >>> >>> >>> <https://www.dropbox.com/s/z2iu0xi4ki42sl7/annotated_cqrs_ >>> architecture.jpg >>> <https://www.google.com/url?q=https%3A%2F%2Fwww.dropbox.com%2Fs%2Fz2iu0xi4ki42sl7%2Fannotated_cqrs_architecture.jpg&sa=D&sntz=1&usg=AFQjCNEc6KMEVqIPtBTH05x6rYSiCaxUCw> >>> > >>> >>> >>> For example, a saga could be waiting for an event indicating funds had >>> been withdrawn from a bank account after it had issued a command requesting >>> that be done. The saga could subscribe to events from the bank account >>> before issuing the command and watch for a specific event from that time on. >>> >>> Why would the Sage subscribe to events instead of talking with the >>> domain entities directly? >>> >>> >>> I believe it is for loose coupling and to make the process asynchronous >>> (and event-driven). >>> >>> >>> Actor messaging already has those qualities, no event store necessary. >>> >>> When an AR processes a command it cannot know all the other ARs or >>> sagas wanting to know when the command has been performed. >>> >>> >>> It will be the Saga that has sent the Command, so yes, it will know whom >>> to reply to (unless I misunderstand what a Saga is). Another consideration >>> is that modeling the replies by the Saga subscribing to the events produced >>> by various aggregates (in the example linked to below) involves a large >>> overhead as compared to directly receiving just those events pertaining to >>> its own commands—this is best done within the target aggregate by simple >>> sending them with the tell operator after having received the confirmation >>> from the persistent store. >>> >>> Also in an actor-based system to be reliable(?) I would assume sagas >>> need to read events from a persistent store (or receive messages with >>> at-least-once-delivery guarantee). >>> >>> >>> The Saga persist its own progress and retries commands that are not >>> confirmed within a timeout (or falls back to different strategies or fails >>> the overall request). This makes it reliable in exactly the right way >>> without unnecessary overhead. >>> >>> >>> Please check out the short section on Sagas from the CQRS Journey book: >>> >>> http://msdn.microsoft.com/en-us/library/jj591569.aspx >>> >>> Particularly Figure 2. >>> >>> I believe the black filled arrows are events and the white hollow arrows >>> are commands. Commands I believe are well suited to actor messages but >>> events I believe are best suited to persistent publish/subscribe-like >>> communication (i.e. the journal). To be honest though, I note that the >>> text talks about event messages, although it also talks about the event >>> messages using an event bus, so I am a little unsure. >>> >>> >>> >>> My understanding is that the Saga sits logically on the top left side of >>> the first diagram, acting on behalf of and just like the user (and like the >>> user it should have a memory and be persistent). >>> >>> >>> My understanding is that sagas sit where the Services are in the diagram >>> I provided (near [2]), i.e. they issue commands (not labelled as such in >>> the diagram unfortunately but what else could they be?) and listen for >>> events on the write side (as labelled). >>> >>> <https://www.dropbox.com/s/z2iu0xi4ki42sl7/annotated_cqrs_ >>> architecture.jpg >>> <https://www.google.com/url?q=https%3A%2F%2Fwww.dropbox.com%2Fs%2Fz2iu0xi4ki42sl7%2Fannotated_cqrs_architecture.jpg&sa=D&sntz=1&usg=AFQjCNEc6KMEVqIPtBTH05x6rYSiCaxUCw> >>> > >>> >>> Someone please correct me if I am wrong. >>> >>> >>> I don’t know about being wrong, I’m just making up my mind based on what >>> I find useful, and if my conclusions are wrong I trust people will tell me >>> so eventually ;-) >>> >>> >>> This is the dashed area labelled [3] in the diagram. >>> >>> >>> Akka seems to distribute the event store used for persistence of ARs [1] >>> on the write side to the read side, which is an interesting idea. But I >>> don’t believe this is enough for CQRS. One event store cannot provide all >>> the required read models. >>> >>> >>> This is probably where all (our) misunderstanding originates: >>> PersistentView is a very particular thing and it turns out that it does not >>> actually match up with the Q in CQRS. Perhaps we should indeed just remove >>> it and add a facility which lets Actors query the Journal instead (in case >>> you want to roll your own read model adapter). >>> >>> >>> PersistentViews tied to one PersistentActor we know are too limited. >>> However, I think PersistentViews tied to named query streams (specified to >>> the journal) would be useful (if they could guaranteed at least once >>> delivery to an actor representing a query store). >>> >>> >>> If we change the View feature in the way I suggested above then we have >>> all the pieces already, AtLeastOnceDelivery or persist() could be used to >>> reliably write projections into another store where needed. >>> >>> >>> Thank you for your patience reading through all of this text. >>> >>> >>> Thank you for this discussion, it is very useful and enjoyable! >>> >>> Regards, >>> >>> Roland >>> >>> >>> Cheers, >>> Ashley. >>> >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/ >>> akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups. >>> google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> *Dr. Roland Kuhn* >>> *Akka Tech Lead* >>> Typesafe <http://typesafe.com/> – Reactive apps on the JVM. >>> twitter: @rolandkuhn >>> <http://twitter.com/#!/rolandkuhn> >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/ >>> akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups. >>> google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Akka User List" group. >>> To unsubscribe from this topic, visit https://groups.google. >>> com/d/topic/akka-user/SL5vEVW7aTo/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> -- >>> Studying for the Turing test >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/ >>> akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups. >>> google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> *Dr. Roland Kuhn* >>> *Akka Tech Lead* >>> Typesafe <http://typesafe.com/> – Reactive apps on the JVM. >>> twitter: @rolandkuhn >>> <http://twitter.com/#!/rolandkuhn> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Akka User List" group. >>> To unsubscribe from this topic, visit https://groups.google.com/d/ >>> topic/akka-user/SL5vEVW7aTo/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> akka-user+...@googlegroups.com. >>> To post to this group, send email to akka...@googlegroups.com. >>> Visit this gr >>> >>> ... >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "Akka User List" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/akka-user/SL5vEVW7aTo/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to a topic in the > Google Groups "Akka User List" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/akka-user/SL5vEVW7aTo/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Studying for the Turing test -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
