> I just don't see why there can't be an ecosystem of independent projects built on Pekko. I don't think Apache Pekko needs to take responsibility for all the jars. I have to reiterate just how much harder it is to release an Apache jar than it is to release a jar independently.
I am aware of these things but I think more nuance needs to be added here onto this point. For example one is consistency, that is why do we have persistence-dynamodb as a core Pekko module but not persistence-mongodb? This then needs to be combined with usage + community, I picked persistence-dynamodb here deliberately because if we use a crude metric of github stars it akka-persistence-dynamodb appears to be less used (86 stars) than akka-persistence-mongo which has (104) stars. When Akka was under Lightbend this might have been more acceptable (i.e. hypothetically Lightbend probably had some paying Akka customers that happened to use persistence-dynamodb) but if we are talking about usage + community which is the whole point of Apache projects than if my hypothesis is correct then there is actually more of an argument including akka-persistence-mongo as a core Pekko module rather than some existing modules being there. I understand the current concerns about maintenance, but I don't think it's useful or constructive to stonewall potential modules being added right now, we should evaluate in the future once the Pekko release storm settles down. > There are lots of Apache projects with a vibrant ecosystem of related but independent libs. Have a look at Apache Spark or Flink or Hadoop. Yes but they have a core set of connectors which is comparable to scope and size of Pekko which is what my core point was (see https://github.com/search?q=org%3Aapache+flink-connector&type=repositories as an example with Flink). And given the prevalence of mongo as a storage solution I would consider it a core connector/persistence storage, just like there is a core Flink Mongo connector (see https://github.com/apache/flink-connector-mongodb). > If there is a lib and the maintainer no longer has time to maintain it, then I would prefer to not see it come into Apache Pekko. If the lib is very widely used, then we would have to consider it regardless. As per my earlier point, you can make a somewhat reasonable claim that its more widely used than other current akka-persistence modules (I mean look at https://github.com/akka/akka-persistence-r2dbc, it has only 20 stars). At least from a cursory glance, it does pass the "at least decently used" benchmark > If a regular Pekko committer comes with a project that they would like to bring into Apache Pekko, I would be much more supportive of that use case. In that case, there would be a better chance that the code could be maintained once it becomes an Apache Pekko lib. To put it another way, get actively involved with Pekko development generally and then convince us to take on your lib. Agreed, I can definitely say if the maintainers of https://github.com/scullxbones/pekko-persistence-mongo helped out in contributing with Pekko in general and/or would help out in the effort of both adding and then maintaining an official Pekko Persistence mongo than there would be much more buy in. On a related note, previously I talked about technical overhead for Pekko repositories which is something that I plan to help out with once the release craziness is over. There is a huge amount of manual technical process which can easily be abstracted over and centralized which would greatly reduce the overhead of adding future Pekko repos in the future. On Mon, Jul 17, 2023 at 10:09 PM PJ Fanning <[email protected]> wrote: > I just don't see why there can't be an ecosystem of independent > projects built on Pekko. I don't think Apache Pekko needs to take > responsibility for all the jars. I have to reiterate just how much > harder it is to release an Apache jar than it is to release a jar > independently. > > There are lots of Apache projects with a vibrant ecosystem of related > but independent libs. Have a look at Apache Spark or Flink or Hadoop. > > If a regular Pekko committer comes with a project that they would like > to bring into Apache Pekko, I would be much more supportive of that > use case. In that case, there would be a better chance that the code > could be maintained once it becomes an Apache Pekko lib. To put it > another way, get actively involved with Pekko development generally > and then convince us to take on your lib. > > If there is a lib and the maintainer no longer has time to maintain > it, then I would prefer to not see it come into Apache Pekko. If the > lib is very widely used, then we would have to consider it regardless. > > > On Mon, 17 Jul 2023 at 20:18, Brendan Doyle <[email protected]> wrote: > > > > Agree with all the points of view here, it’s certainly too soon to even > > really have a meaning discussion on the philosophy. > > > > PJ for your curiosity on the repo I don’t think was public until over the > > weekend when I reached out about the pekko process. The port can be found > > here. I’m not sure if the reactive mongo dependency is stil in there, > but I > > was told they’re removing support for that driver since no pekko support > > and using solely the official mongo scala driver which the Akka version > > also had support for. Supposedly it was built so both drivers could be > used > > interchangeably on the Akka library, they’re just removing the one that > > doesn’t have the pekko support. > > > > https://github.com/scullxbones/pekko-persistence-mongo > > > > On Mon, Jul 17, 2023 at 12:09 PM Matthew de Detrich > > <[email protected]> wrote: > > > > > I agree with PJ fanning when it comes to adding the module right now, > we > > > simply don't have > > > the capacity and there are way too many things on our plate. However > when a > > > release occurs > > > and things stabilize a little more it is a different circumstance > which is > > > where I get to > > > > > > > After that, I would probably start arguing to REDUCE the Apache Pekko > > > code base as opposed to increase it. There are a lot of connectors > > > that have broken tests or tests disabled. It's hard enough to support > > > and test the core Pekko features (like Actors, Clustering, HTTP, > > > etc.), while then also having to maintain all the integrations with a > > > huge variety of 3rd party tools. It's taken us 8 months to release the > > > Pekko core. We simply can't handle the enormous size of the current > > > code base. We need to concentrate on the core tooling. > > > > > > So there are 2 views successful views here, one is that > > > we grow Pekko along with our active contributors, which means that > > > we add in more modules when they make technical sense but we also > > > grow the number of active contributors so that this burden/maintenance > can > > > actually be handled. The other view is that we reduce the > > > scope of what is in Pekko so that we can maintain what we currently > have > > > better. > > > > > > I currently lean for the former, and if we look at other projects like > > > Flink > > > and Airflow (which have more connectors than Pekko does) they do this > > > and they do it successfully. If you boil it down, there isn't > difference in > > > the community that maintains a persistence-mongo thats external to > > > Apache Pekko versus a community within Apache Pekko that maintains > > > persistence-mongo. > > > > > > Which gets to the crux of this statement > > > > > > > Let the people > > > who need the integrations (connectors and persistence implementations) > > > maintain them. > > > > > > I would actually argue that our priority after release is to actually > grow > > > the > > > community so that it can be maintained within Apache Pekko, so those > people > > > who need the integrations that you refer to actually become part of > Pekko. > > > It may have been in Lightbend's interest to restrict what Akka can > accept > > > in their org because their open source governance model didn't allow > > > them to employ the community. > > > > > > Its well known that it was impossible to get maintainer/committer > rights to > > > any of Lightbends Akka's repos aside from Alpakka which actually and > > > ironically got a large amount of support and contributions from the > > > community considering how many connectors it supported. And I think > > > there is a premise here, build some bridges plus a story and people > will > > > come. > > > > > > In summary, in my view it's too soon right now to add this right now. > There > > > is > > > just way too much technical and process overhead when it comes to > adding > > > more > > > repos. When those get solved then we can revisit it, in the meantime > as PJ > > > Fanning stated the akka dependencies of persistence-mongo for obvious > > > reasons > > > need to be moved over to pekko (and that's before we even get to the > core > > > persistence-mongo library) so whatever is decided shouldn't block you > > > either way, > > > all we are saying is that we ourselves don't have capacity to do the > work > > > now. > > > > > > On Mon, Jul 17, 2023 at 8:13 PM PJ Fanning <[email protected]> > wrote: > > > > > > > When I checked akka-persistence-mongo at the weekend, it has a > > > > dependency on reactivemongo-akkastream which has a dependency on > > > > reactivemongo (which also has an akka dependency). > > > > > > > > This might have changed for the pekko equivalent. > > > > > > > > Even if it has, I'm still going to argue against Apache Pekko taking > > > > on any more code until we get the current code set of Pekko modules > > > > released. And that will take months. > > > > > > > > After that, I would probably start arguing to REDUCE the Apache Pekko > > > > code base as opposed to increase it. There are a lot of connectors > > > > that have broken tests or tests disabled. It's hard enough to support > > > > and test the core Pekko features (like Actors, Clustering, HTTP, > > > > etc.), while then also having to maintain all the integrations with a > > > > huge variety of 3rd party tools. It's taken us 8 months to release > the > > > > Pekko core. We simply can't handle the enormous size of the current > > > > code base. We need to concentrate on the core tooling. Let the people > > > > who need the integrations (connectors and persistence > implementations) > > > > maintain them. > > > > > > > > On Mon, 17 Jul 2023 at 18:30, Brendan Doyle <[email protected]> > > > wrote: > > > > > > > > > > That’s fine I expected that to be the outcome for the reasons you > > > listed, > > > > > but wanted to raise the discussion. Though I am curious what you > mean > > > by > > > > > the lib dependencies that don’t support pekko, which library > > > dependencies > > > > > are you referring to within? The maintainers of the akka mongo > > > > persistence > > > > > library have already successfully ported a pekko repo which I > linked , > > > > > they’re just working through some artifact publishing issues > before a > > > > > release happens. > > > > > > > > > > On Mon, Jul 17, 2023 at 10:03 AM PJ Fanning <[email protected]> > > > wrote: > > > > > > > > > > > I'd be a strong -1 on this. > > > > > > > > > > > > Maybe in distant future. > > > > > > > > > > > > There are too many lib dependencies in that module. None of which > > > > support > > > > > > Pekko. > > > > > > > > > > > > Software brought into ASF requires permissions from all the past > > > > > > contributors. Pekko itself is an exception because of the Akka > > > license > > > > > > change. > > > > > > > > > > > > ASF requires us to do loads of process work and read every line > to > > > > check > > > > > > its origin in case of licensing issues. Everything required > multiple > > > > votes. > > > > > > > > > > > > Why don't you start by talking to the existing lib maintainers > about > > > > them > > > > > > releasing Pekko libs? If that doesn't work out, you could fork > them > > > > > > yourself. > > > > > > > > > > > > Pekko team have far too much code and too few active developers > to > > > > become a > > > > > > shelter for all the old Akka libs that no-one wants to maintain. > > > > > > > > > > > > On Mon 17 Jul 2023, 17:31 Brendan Doyle, <[email protected]> > > > wrote: > > > > > > > > > > > > > Whoops forgot to include the link to the repo. > > > > > > > > > > > > > > https://github.com/scullxbones/pekko-persistence-mongo > > > > > > > > > > > > > > > > > > > > > On Mon, Jul 17, 2023 at 9:26 AM Brendan Doyle < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > The skullxbones mongo persistence library was the de facto > > > > connector > > > > > > for > > > > > > > > mongo, never having a dedicated library from Lightbend. The > > > creator > > > > > > and a > > > > > > > > collaborator have already successfully ported the library to > > > pekko > > > > > > 1.0.0 > > > > > > > > and are working on getting a release out. The port has > dropped > > > > support > > > > > > > for > > > > > > > > reactive mongo and only using the official mongo-scala-driver > > > going > > > > > > > > forward. They would like to see the library donated to pekko > if > > > the > > > > > > > > community / team would be interested. I know there’s a couple > > > > official > > > > > > > > persistence libraries for widely used db’s like dynamodb, > > > > personally > > > > > > > though > > > > > > > > biased I would argue mongo falls into that category. I’m not > sure > > > > what > > > > > > > the > > > > > > > > bar or threshold is to take in another repo or if there’s a > long > > > > term > > > > > > > plan > > > > > > > > to combine persistence drivers into a single repo like the > way > > > > stream > > > > > > > > connectors are set up besides kafka. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Brendan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > -- > > > > > > Matthew de Detrich > > > > > > *Aiven Deutschland GmbH* > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > *m:* +491603708037 > > > > > > *w:* aiven.io *e:* [email protected] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Matthew de Detrich *Aiven Deutschland GmbH* Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen *m:* +491603708037 *w:* aiven.io *e:* [email protected]
