Thanks Bertrand and Stefan! Some more inputs inline below.
Le mar. 26 mars 2019 à 13:39, Stefan Egli <stefane...@apache.org> a écrit : > +1 to the donation, great to see this move! > > Regarding kafka/aries-journaled-events: I think it's good to have such a > separation that allows a 2nd implementation (like the mongo one > currently) to validate the approach. > +1 The distribution design requires a deliberately small and simple API to favour its portability to any backend persisting sequences natively (any RDB, some document stores, some messaging services, etc.). The main motivation is to be able to deploy it at marginal operational costs on any existing infrastructure. > > I'm actually wondering if it would be possible to make a JMS based one. > At first sight it looks challenging as the offset control doesn't go so > well with JMS. However it might be possible to try to build something on > the QueueBrowser together with some MessageSelector representing the > offset and not actually consuming any messages off that queue to keep it > retained. Perhaps not really the main idea of JMS though and it might > also depend on actual support of these features by actual JMS > implementations. > We considered JMS however we concluded that none of the JMS messaging models were directly suitable to support event sourcing design. JMS publish/subscribe seemed the closest natural distribution model to fit our need. However the specification lacks the ability to replay historical messages which would translate in our case to major consistency issues. It may be possible to built a publish/subscribe replay capability on top of JMS point-to-point messaging model (read-only-queue). I am not sure though how the selectors could act as entry points in the history of messages nor how an equivalent of Kafka offsets would be supported. For the latter, anything that can decide the happened-before relation given two individual messages will do. I suggest to bring this idea to the journaled event Aries sub-project should you want to dig it further. Cheers, Timothee > > Cheers, > Stefan > > On 26.03.19 10:56, Timothee Maret wrote: > > Thanks for your feedbacks so far! > > > > Apache Kafka used as a publish/subscribe messaging system with manual > > partition assignment and offset control has been the prime platform to > > support the design. Going forward we shall keep Kafka as the reference > > journal to support our design. The journal.kafka implementation is > > exercised when running the journal.it module and as such is required in > the > > donation. > > > > As Christian noted, we identified a general need for a journal API and we > > have a side effort towards that goal. It's unclear when a stable event > > journal API or OSGi spec would be available. However, should a > generalised > > solution be available at some point, we could offer a migration path from > > the journal.kafka module by offering adapters or simply pointing to the > new > > implementation. > > > > Regards, > > > > Timothee > > > > > > Le mar. 26 mars 2019 à 10:06, Christian Schneider < > ch...@die-schneider.net> > > a écrit : > > > >> We are actually looking to replace the current kafka adapter with > >> something more general. > >> > >> In aries journaled events we started to build a very slim journaled > >> messaging api and also got a kafka adapter there. > >> See https://github.com/apache/aries-journaled-events > >> > >> We also work together with Data in motion on an OSGi spec for messaging > >> that also includes journaled messaging. For the upcoming spec it is not > yet > >> clear where it will end up. > >> > >> So for now we plan to put the kafka adapter to sling but it will be > >> replaced by either the aries one or the new OSGi spec. > >> > >> Christian > >> > >> > >> Am Mo., 25. März 2019 um 14:41 Uhr schrieb Andrei Dulvac < > >> dul...@apache.org>: > >> > >>> Hi Tim, > >>> > >>> I have a small doubt regarding > org.apache.sling.distribution.journal.kafka > >>> and how well it will be maintained. > >>> But looks super neat. > >>> > >>> +1 > >>> > >>> - Andrei > >>> > >>> On Mon, Mar 25, 2019 at 1:09 AM Timothee Maret <tma...@apache.org> > wrote: > >>> > >>>> Hi, > >>>> > >>>> I, Christian Schneider and Marius Petria, have built at Adobe a > content > >>>> distribution implementation optimised for cloud deployments. > >>>> > >>>> The design implements the Sling Content Distribution APIs and is > >>> portable > >>>> to various messaging system or backends exposing journaling > capability. > >>> The > >>>> design is based on the event sourcing pattern, supports Sling nodes > >>>> horizontal auto-scaling, is resilient to errors and minimise content > >>>> repository resources significantly. > >>>> > >>>> This implementation started inside Adobe as it was initially tied to a > >>>> proprietary journal offering. Today, we are proposing this donation to > >>>> Apache Sling as we think it would be a useful addition to the SCD > >>> framework > >>>> (in the distribution group [0]) and we expect to benefit from the > >>> community > >>>> inputs going further. We will keep maintaining & developing the > >>>> implementation as we are rolling it out in production. The donation is > >>>> functional on Apache Kafka. > >>>> > >>>> Please cast your vote. > >>>> > >>>> Regards, > >>>> > >>>> Timothee > >>>> > >>>> [0] > >>>> > >>>> > >>> > https://github.com/apache/sling-aggregator/blob/master/docs/groups/distribution.md > >>>> > >>> > >> > >> > >> -- > >> -- > >> Christian Schneider > >> http://www.liquid-reality.de > >> > >> Computer Scientist > >> http://www.adobe.com > >> > >> > > >
