Poule, how long do you expect to keep the events for your 1 cart or 1 user? or in the words, how large is your event stream per cart or per user. it seems that you want some very lightweight stream/queue solutions - like support very large amount of streams and per stream throughput is not very high.
- Sijie On Sun, Oct 30, 2016 at 8:00 AM, Poule Dodue <pouledo...@hotmail.com> wrote: > i'm not expert but ref. my previous cart example, imho i need a way to > fetch events of 1 cart or 1 user. > the actual physical implementation on DL I don't know the technicalities, > as long I can get 1 cart with the > less cpu cycles possible. > > > > Le 30 oct. 2016 à 09:59, john.lonergan <john.loner...@gmail.com> a > écrit : > > > > As an FYI. > > It's a misrepresentation to suggest that ES/CQRS requires each > actor/aggregate root history to be materialised as a physically distinct > stream. There may be specific implementations that require this, and I know > there are others that do not.So regarding those references given, it's is > not a requirement of this tech that we generate one stream in DL per actor > for ES.It might for instance be necessary for Greg Young's event store to > have one phys stream per aggregate sure to implementation choices. > > -------- Original message --------From: Poule Dodue < > pouledo...@hotmail.com> Date: 28/10/2016 16:24 (GMT+00:00) To: > dev@distributedlog.incubator.apache.org Subject: Re: hundreds of millions > of streams? > > yes aka ES/CQRS > > > > some links: > > > > https://msdn.microsoft.com/en-us/library/jj554200.aspx > > http://williamverdolini.github.io/2014/08/11/cqrses-architecture/ > > http://docs.geteventstore.com/introduction/3.9.0/event-sourcing-basics/ > > > > it needs lot of streams to basically replay events for any entity on a > system. > > > > example: i could replay events for all changes that happened in 1 Cart > of 1 User: > > > > > > (read events from stream "cart-of-user-233293111" ): > > > > 1- added item X > > 2- deleted item X > > 3- added item Y > > .... > > > > by replaying that stream, I can rebuild a user's cart state > > > > > >> Le 28 oct. 2016 à 10:13, Leigh Stewart <lstew...@twitter.com.INVALID> > a écrit : > >> > >> Poule- would you mind sharing some information on Event Sourcing? Are > you > >> referring to something like > >> http://martinfowler.com/eaaDev/EventSourcing.html ? > >> > >> On Fri, Oct 28, 2016 at 7:11 AM, Leigh Stewart <lstew...@twitter.com> > wrote: > >> > >>> DL is not able to handle 100s of millions of streams. 10^5-106 is > probably > >>> ok. > >>> ZK is probably the biggest challenge (we are looking at ways to > eliminate > >>> this as we would like to scale to 10^6-10^7 in the not too distant > future), > >>> but 100s of millions is so far beyond what we've worked with there > would > >>> likely be other scaling challenges on the way to that point. > >>> > >>> On Fri, Oct 28, 2016 at 5:56 AM, Poule Dodue <pouledo...@hotmail.com> > >>> wrote: > >>> > >>>> In Event Sourcing, we need to have 1 stream per entity/aggregate so > for > >>>> a typical prod system it means we need hundreds of millions of > streams. > >>>> > >>>> Is DL able to handle that or it is limited to, say, few hundreds > >>>> thousands of streams? > >>>> > >>>> > >>>> > >>> > > > >
