Am 20.08.2014 13:33 schrieb "Roland Kuhn" <goo...@rkuhn.info>: > > > 20 aug 2014 kl. 10:43 skrev Martin Krasser <krass...@googlemail.com>: > >> On 20.08.14 10:16, Roland Kuhn wrote: >>> >>> >>> 19 aug 2014 kl. 18:59 skrev Ashley Aitken <amait...@gmail.com>: >>> >>>> On Tuesday, 19 August 2014 21:14:17 UTC+8, rkuhn wrote: >>>>> >>>>> >>>>> 18 aug 2014 kl. 18:01 skrev Ashley Aitken <amai...@gmail.com>: >>>>> >>>>>> I believe Akka needs to allow actors to: >>>>>> >>>>>> >>>>>> (i) persist events with as much information as efficiently possible on the write side to allow the store to facilitate the read side extracting them according to what criteria is needed, >>>>> >>>>> This is a convoluted way of saying that Events must be self-contained, right? In that case: check! >>>> >>>> >>>> No, I don't think so. As I understand it now, the only thing the event store knows about each event is the persistenceId and a chunk of opaque data. It doesn't know the type of the event, the type of the message, any time information, any causal dependency etc. I guess what I am saying is that the events need to include as much metadata as possible so that the event store can provide the necessary synthetic streams if they are requested by the read side. As I mentioned later, some event stores (like Kafka may replicate the events into separate topics based on this information), others (like Event Store) may use this information later to form streams of links to the original events. >>> >>> >>> The event store has the full event available, which is all the information there is: gathering or duplicating arbitrary parts of the information is likely not going to help, because you will discover later that you missed something initially, and if the mechanism is baked into the Akka Persistence Journal SPI then fixing it will take a very long time (until plugins are migrated and your OPS guys allow you to use it etc.). My recommendation is to use a serialization mechanism that fits the Journal, allowing it to understand the events and provide semantic features on top. Both (Journal and serialization) are configured in the same file, so I submit that coupling them is a valid approach. >>> >>> On causal consistency: I am still unconvinced that it is worth pursuing, and I am certain that you are vastly underestimating the amount of data and effort involved. And it cannot be done without collaboration from the user since a single inter-Actor message outside of the traced system (i.e. not using a PersistenceEnvelope of sorts) would hamper or destroy it. >> >> >> akka.dispatch.Envelope could carry additional information, so that user-collaboration is not needed > > > Well, actually Envelope is going to go away in Akka Gålbma (a.k.a. Akka 3).
My point was that causal consistency can be achieved without forcing users to do extra work, as tracing dependencies can be done completely within Akka, and there are several possible ways to implement that. > >> >>> >>>>>> >>>>>> (iii) read from (and replay) streams of events on the read and write side according to a range of criteria supported and defined within the store or via the store API (e.g. using a DSL), and >>>>> >>>>> This is the unclear point: who defines the query and when? What are the consistency guarantees for the generated event stream? >>>> >>>> >>>> I suggest the developers of the read side specify the queries directly to the event store but this may be after the events have initially been persisted. The event store produces the query stream (if it can) and a PersistentView can be setup to read from that named query. With regards to consistency guarantees - my understanding is that these streams are used to eventually guarantee that the query model will be consistent with the write model, i.e. all the events will get across. With regards to ordering I think the event store does the best it can to provide consistent ordering, e.g. total ordering if there was no distribution and causal ordering, where possible, if there was ordering. The developer would need to understand the limitations of how the query store is configured and queried. >>> >>> >>> As I answered to Greg already, I think that this should not be a core concern of Akka Persistence; as you note it relies on features provided by the underlying event store, and those features are not necessary to achieve the goal of making actors persistent. >>> >>>> >>>> >>>>>> >>>>>> (iv) reliably (at least once) deliver information to other read side store(s) and systems above and beyond the store used for persisting the events. >>>>> >>>>> This is PersistentView, so “check!” (As argued previously “reliably” translates to “persistent”.) >>>> >>>> >>>> As I asked in another thread (I think) I am not sure how PersistentView can do this when PersistentActor is the one that can mixin AtLeastOnceDelivery? >>>> >>>> I think we need a PeristentView that can guarantee AtLeastOnceDelivery to an actor representing a query store. This would seem to require a PersistentViewActor ;-) that can read from a persistent query and also persist its state to provide guaranteed delivery. >>>> >>>> My lack of knowledge of Scala and Akka may be showing here. >>> >>> >>> My current impression is that PersistentView needs to be re-thought: instead of tying it to a persistenceId like we do now we should just provide an API for subscribing to named topics in the Journal—be that persistenceIds of some PersistentActors or synthetic ones. One Actor should be able to subscribe to any number of them, but the onus will be on it to keep track of the positions up to which it has consumed from all of them. >>> >>> This does not preclude the Journal from providing a synthetic topic with proper linearization for all events in it (or whatever you want to specifically configure on the storage back-end, outside of the Journal SPI). And this also does not invalidate my point that normally the consumption of Queries should be done directly from the backing store, making full use of its unique feature set. >>> >>>> >>>>>> I believe each of these is readily achievable with Akka but: >>>>>> >>>>>> >>>>>> (i) doesn’t mean explicitly persisting the events to specific topics as you suggest in your (1) (although this may be how some stores implement the required functionality on the read side). Instead it means transparently including information like the actorId, event type, actor type, probably the time and possibly information to help with causal ordering (see my next post). >>>>> >>>>> No, again we need to strictly keep Topics and Queries separate, they are very different features. Topics are defined up-front and explicitly written to, Queries are constructed later based on the existing event log contents. Marking events within the store with timestamps of some kind might help achieving a pseudo-deterministic behavior, but it is by no means a guarantee. Causal ordering is out of scope, and it also does not help in achieving the desired ability to replay Queries from some given point in the past. >>>> >>>> >>>> I think we do agree somewhere in there but I don't think as was suggested (by you earlier?) that creating topics up-front whether a fixed set or arbitrary tags will work. I feel in what way the store supports the queries (and how much it can) is up to the store (e.g. creating separate topics or synthetic topics), so I would argue against using topics for CQRS. As I mention below for Pub/Sub to Persistent topics it would be great, but not for CQRS. >>> >>> >>> Yes, indeed, topics are interesting but not strictly related to CQRS. >>> >>>>>> C. CQRS with Event Sourcing >>>>>> >>>>>> >>>>>> And finally, there is CQRS with Event Sourcing, which I believe is much more that (A) and (B) and particularly doesn’t necessarily require (B.) for all event stores. So if Akka were to implement (B), which I think would be very useful for other reasons, it would not specifically be for CQRS. >>>>>> >>>>>> >>>>>> Please consider this diagram overviewing CQRS with Event Sourcing: >>>>>> >>>>>> >>>>>> < https://www.dropbox.com/s/z2iu0xi4ki42sl7/annotated_cqrs_architecture.jpg> >>>>>> >>>>>> >>>>>> For example, a saga could be waiting for an event indicating funds had been withdrawn from a bank account after it had issued a command requesting that be done. The saga could subscribe to events from the bank account before issuing the command and watch for a specific event from that time on. >>>>> >>>>> Why would the Sage subscribe to events instead of talking with the domain entities directly? >>>> >>>> >>>> I believe it is for loose coupling and to make the process asynchronous (and event-driven). >>> >>> >>> Actor messaging already has those qualities, no event store necessary. >>> >>>> When an AR processes a command it cannot know all the other ARs or sagas wanting to know when the command has been performed. >>> >>> >>> It will be the Saga that has sent the Command, so yes, it will know whom to reply to (unless I misunderstand what a Saga is). Another consideration is that modeling the replies by the Saga subscribing to the events produced by various aggregates (in the example linked to below) involves a large overhead as compared to directly receiving just those events pertaining to its own commands—this is best done within the target aggregate by simple sending them with the tell operator after having received the confirmation from the persistent store. >>> >>>> Also in an actor-based system to be reliable(?) I would assume sagas need to read events from a persistent store (or receive messages with at-least-once-delivery guarantee). >>> >>> >>> The Saga persist its own progress and retries commands that are not confirmed within a timeout (or falls back to different strategies or fails the overall request). This makes it reliable in exactly the right way without unnecessary overhead. >>> >>>> >>>> Please check out the short section on Sagas from the CQRS Journey book: >>>> >>>> http://msdn.microsoft.com/en-us/library/jj591569.aspx >>>> >>>> Particularly Figure 2. >>>> >>>> I believe the black filled arrows are events and the white hollow arrows are commands. Commands I believe are well suited to actor messages but events I believe are best suited to persistent publish/subscribe-like communication (i.e. the journal). To be honest though, I note that the text talks about event messages, although it also talks about the event messages using an event bus, so I am a little unsure. >>>> >>>> >>>>> >>>>> My understanding is that the Saga sits logically on the top left side of the first diagram, acting on behalf of and just like the user (and like the user it should have a memory and be persistent). >>>> >>>> >>>> My understanding is that sagas sit where the Services are in the diagram I provided (near [2]), i.e. they issue commands (not labelled as such in the diagram unfortunately but what else could they be?) and listen for events on the write side (as labelled). >>>> >>>> < https://www.dropbox.com/s/z2iu0xi4ki42sl7/annotated_cqrs_architecture.jpg> >>>> >>>> Someone please correct me if I am wrong. >>> >>> >>> I don’t know about being wrong, I’m just making up my mind based on what I find useful, and if my conclusions are wrong I trust people will tell me so eventually ;-) >>> >>>> >>>>>> This is the dashed area labelled [3] in the diagram. >>>>>> >>>>>> >>>>>> Akka seems to distribute the event store used for persistence of ARs [1] on the write side to the read side, which is an interesting idea. But I don’t believe this is enough for CQRS. One event store cannot provide all the required read models. >>>>> >>>>> >>>>> This is probably where all (our) misunderstanding originates: PersistentView is a very particular thing and it turns out that it does not actually match up with the Q in CQRS. Perhaps we should indeed just remove it and add a facility which lets Actors query the Journal instead (in case you want to roll your own read model adapter). >>>> >>>> >>>> PersistentViews tied to one PersistentActor we know are too limited. However, I think PersistentViews tied to named query streams (specified to the journal) would be useful (if they could guaranteed at least once delivery to an actor representing a query store). >>> >>> >>> If we change the View feature in the way I suggested above then we have all the pieces already, AtLeastOnceDelivery or persist() could be used to reliably write projections into another store where needed. >>> >>>> >>>> Thank you for your patience reading through all of this text. >>> >>> >>> Thank you for this discussion, it is very useful and enjoyable! >>> >>> Regards, >>> >>> Roland >>> >>>> >>>> Cheers, >>>> Ashley. >>>> >>>> >>>> >>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html >>>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. >>>> To post to this group, send email to akka-user@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >>> >>> Dr. Roland Kuhn >>> Akka Tech Lead >>> Typesafe – Reactive apps on the JVM. >>> twitter: @rolandkuhn >>> >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. >>> To post to this group, send email to akka-user@googlegroups.com. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> Martin Krasser >> >> blog: http://krasserm.blogspot.com >> code: http://github.com/krasserm >> twitter: http://twitter.com/mrt1nz >> >> >> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. >> To post to this group, send email to akka-user@googlegroups.com. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. > > > > > Dr. Roland Kuhn > Akka Tech Lead > Typesafe – Reactive apps on the JVM. > twitter: @rolandkuhn > > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
