Dear hakkers, unfortunately it took me a long time to catch up with akka-user to this point after the vacation, but on the other hand this made for a very interesting and stimulating read, thanks for this thread!
If I may, here’s what I have understood so far: In order to support not only actor persistence but also full CQRS we need to adjust our terminology: events are published to topics, where each persistenceId is one such topic but others are also allowed. Common use-cases of building projections or denormalized views require the ability to query the union of a possibly large number of topics in such a fashion that no events are lost. This union can be viewed as a synthetic or logical topic, but issues arise in that true topics provide total ordering while these synthetic ones have difficulties doing so. Constructing Sagas is hard. AFAICS 3. is not related to the other two, the mentions in this thread have only alluded to the problems so I assume that the difficulty is primarily to design a process that has the right eventual consistency properties (i.e. rollbacks, retries, …). This is an interesting topic but let’s concentrate on the original question first. The first point is a rather simple one, we just need to expose the necessary API for writing to a given topic instead of the local Actor’s persistenceId; I’d opt for adding variants of the persist() methods that take an additional String argument. Using the resulting event log is then done as for the others (i.e. Views and potentially queries should just work). The only concern is that the Journal needs to be prepared to receive events concurrently from multiple sources instead of just the same Actor, but since each topic needs to be totally ordered this will not be an additional hassle beyond just routing to the same replica, just like for persistenceIds. The second point is the contentious one, since a feature request (consistent iteration over a query) clashes with a design choice (scalability). First it is important to note that this clash is genuine: scalability means that we do not want to limit the size of a topic to always fit one unit of consistency, our default assumption is that everything should be prepared for distribution. We all know that in a distributed system linearizability is not generally achievable, meaning that a distributed (synthetic) topic that receives events from concurrent sources will not be able to provide a global ordering. A non-distributed Journal, OTOH, is a single point of failure which is not desirable for many applications (i.e. your business will go down while the Journal has issues—true replication requires the ability to fail independently and hence is distributed in the CAP sense). As I see it, a query (like “all events of this type” etc.) should be configured for the given Journal and should then be available as a (synthetic) topic for normal consumption—but not for being written to. The Journal is then free to implement this in any way it sees fit, but barring fundamental advances in CS or errors on my part this will always require that the synthetic topic is not scalable in the way we usually define that (i.e. distributable). As Vaughn points out this may not be an issue at all, actual benchmarks would help settle this point. Journal backends that already implement a global order can make use of that, for others the synthetic topic would work just like any other non-PersistentActor topic with manual duplication of those events that match the query (akin to (a) in the first post of this thread); this duplication does not necessarily need to double the memory consumption, it could also only persist the events by reference (depending on the storage engine). When it comes to providing queries in a way that does not have a global ordering, my current opinion is that we should not do this because it would be quite pointless (a.k.a. unusable). A compromise would be to provide eventually linearizable queries based on the premise that the application of events should be idempotent in any case and overlapping replay (i.e. where necessary from the last known-linear point instead of the requested one) must be tolerated. AFAIK this is the topic of ongoing research, though, so I’d place that lower on the priority list. Does this sound like a fair summary? Please let me know in case I misrepresent or misunderstand something, once we reach consensus on what we need we’ll ticket and solve it, as usual ;-) Regards, Roland 12 aug 2014 kl. 18:10 skrev Ashley Aitken <amait...@gmail.com>: > > Thanks for your post Vaughn. > > On Monday, 11 August 2014 05:57:05 UTC+8, Vaughn Vernon wrote: > None of this stuff is easy to do, and even harder to do right. > > I am the first to agree with that. > > Your post gives away the main problem with getting this to work correctly, > because Actor Model and akka-persistence currently supports the first half of > A, but not the second half. In other words, to make the interface rich we not > only need a new set of abstractions, we also need to overcome the direct > messaging nature of actors because it can be limiting in some use cases. > With the messaging library I am building, currently named Number 9, which > includes both persistence and a process manager, this problem is handled as > follows. Any actor that sends a message may: > > 1. send a persistent message to another actor > 2. send a persistent message to a topic > 3. send a persistent message primarily to another actor, but also to a topic > > That is very interesting. > > It seems to me that CQRS commands should be sent as messages (persistent or > not) - your (1.) and changes of state (AR or application) should be published > as events (to topics or more generally) - your (2.) but I can't see a need > for (3.)? > > Further, a process manager for a bank account transfer could be implemented > with a command to the source account (withdrawForTransfer) that would be > acknowledged by a published persistent event (WithdrawnForTransfer). Similar > for deposit into target account. > > Pawel Kaczor in his DDD-Leaven-Akka series (Lesson 3) includes projections > from aggregated streams of events and a process manager / saga using Akka > Persistence by having the ARs persisting their events and also publishing > their events. > > > http://pkaczor.blogspot.com.au/2014/06/reactive-ddd-with-akka-projections.html > > https://github.com/pawelkaczor/ddd-leaven-akka > > The only shortcomings (not his fault or a criticism) seem to be: 1) the use > of two event infrastructures (one for persistence and one for pub/sub), 2) > the limited ability for complex projections (like Greg mentioned and > available in Event Store), and 3) lack of persistence for pub/sub events. > > The latter makes reconstruction of a read model or construction of a new read > model after the events have been published more difficult. > > If you have watched any of my presentations on this subject you have heard > this before. I am presenting most of this to the DDD Denver meetup this > Monday night. The title of the talk is "Building a Reactive Process Manager, > Twice". The twice part is because I will demonstrate this working both in > Scala with Akka and also in C# with Dotsero: > > Thank you I will look out for that (please share the video link if it is > recorded and put on the Web). I have seen (but not watched) some of your > videos because I am unsure as to who is leading here and the videos I saw > seemed to be from a few years ago. > > I've just got your book so I will get on with reading that (for DDD and CQRS > enlightenment). > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to akka-user+unsubscr...@googlegroups.com. > To post to this group, send email to akka-user@googlegroups.com. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. Dr. Roland Kuhn Akka Tech Lead Typesafe – Reactive apps on the JVM. twitter: @rolandkuhn -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
