Can we have some community projects under the pekko-community organization before we can take in? Currently we have many projects under the apache organization.
何品 Brendan Doyle <[email protected]> 于2023年7月18日周二 06:35写道： > The one thing I was going to bring up which Matthew did was consistency > with the persistence modules. I help manage Apache Openwhisk and it’s a > similar problem of many repos management and keeping the project alive with > active committers that actually fits the scope of what there is to > maintain. > > I will add on to Matthew’s point is many professional organizations > encourage open source contributions to apache versus allowing contribution > to random repos. If the repo is under apache you’re far more likely to > attract new contributors. If the repo is low maintenance since it’s long > matured (in this case it hasn’t needed a release since 2021 and has been > working just fine) and just needs security patches / dependency bumps, > there’s not too much risk to adding it on outside the obvious headache of a > process of adding a new repo initially. And there should also be a valid > proactive deprecation process for something that’s dead if you do bring it > in, find it’s not used and no one is helping give the maintenance it needs > to exist. > > Personally, which again I am biased, I think there needs to be a > delineation between what is a new pekko integration someone makes or a > small pet Akka library made by an individual that wasn’t used (not wanting > to take that on) and what is a de facto integration that was used by Akka > and the only reason an official one didn’t exist was because Lightbend > couldn’t commit to maintaining it. Which is essentially the same argument > here against it, the only difference is I think there is value in having it > under the apache name in both 1. Just having more orgs being willing to use > it and 2. More orgs being able to contribute back to it. That on its own > isn’t a reason to do something here, but just wanted to point that out. > > And of course I’m sympathetic to being on the other end of the drive by > “hey can you consider taking on this additional work” so I don’t think my > opinion holds any value here, just wanted to start a convo on the > philosophy of some of the things I think was debatably borderline official > under Akka but wasn’t; and having a specific example we could do a case > study on how to come to a decision by analyzing it. And thought I could > help start the convo for the fine folks that have put in the effort to port > that module already so quickly, we all understand it’s far too soon to > figure this out and it can exist as is under its existing repo for the time > being while all of this initial massive work for getting pekko off the > ground is still underway. > > P.S. for what it’s worth, I’ve always thought pekko persistence on a > serverless solution like dynamodb that is on demand cost based sounds quite > expensive of a solution versus using a managed db like mongo when the > amount of event writing can be extremely high, if dynamodb is the main > supported persistence module. But depends on the use case. Just my two > cents :) > > Brendan > > On Mon, Jul 17, 2023 at 2:18 PM Matthew de Detrich > <[email protected]> wrote: > > > Oh and one other last thing I would like to mention, at least in my view > > its better to have a decently sized amount of committers that is > > maintaining > > a larger set of modules rather than having almost no committers > maintaining > > a smaller set of modules, the former is a far healthier state for the > > project to > > be in. > > > > So given that, if adding modules also happens to increase the amount of > > active committers that we add to Pekko, that is a net positive because > > most of these modules don't need that much maintenance (once integrated) > > and the committers that have been added as a result of adding modules > > are more incentivized to help out elsewhere since they have more skin > > in the game. > > > > > > On Mon, Jul 17, 2023 at 11:11 PM Matthew de Detrich < > > [email protected]> wrote: > > > > > > I just don't see why there can't be an ecosystem of independent > > > projects built on Pekko. I don't think Apache Pekko needs to take > > > responsibility for all the jars. I have to reiterate just how much > > > harder it is to release an Apache jar than it is to release a jar > > > independently. > > > > > > I am aware of these things but I think more nuance needs to be > > > added here onto this point. For example one is consistency, that > > > is why do we have persistence-dynamodb as a core Pekko module > > > but not persistence-mongodb? This then needs to be combined > > > with usage + community, I picked persistence-dynamodb here > > > deliberately because if we use a crude metric of github stars it > > > akka-persistence-dynamodb appears to be less used (86 stars) > > > than akka-persistence-mongo which has (104) stars. > > > > > > When Akka was under Lightbend this might have been more > > > acceptable (i.e. hypothetically Lightbend probably had some > > > paying Akka customers that happened to use persistence-dynamodb) > > > but if we are talking about usage + community which is the whole point > > > of Apache projects than if my hypothesis is correct then there is > > actually > > > more of an argument including akka-persistence-mongo as a core Pekko > > > module rather than some existing modules being there. > > > > > > I understand the current concerns about maintenance, but I don't > > > think it's useful or constructive to stonewall potential modules being > > > added right now, we should evaluate in the future once the Pekko > > > release storm settles down. > > > > > > > There are lots of Apache projects with a vibrant ecosystem of related > > > but independent libs. Have a look at Apache Spark or Flink or Hadoop. > > > > > > Yes but they have a core set of connectors which is comparable to scope > > > and size of Pekko which is what my core point was (see > > > > > > https://github.com/search?q=org%3Aapache+flink-connector&type=repositories > > > as an example with Flink). And given the prevalence of mongo as a > storage > > > solution I would consider it a core connector/persistence storage, just > > > like > > > there is a core Flink Mongo connector (see > > > https://github.com/apache/flink-connector-mongodb). > > > > > > > If there is a lib and the maintainer no longer has time to maintain > > > it, then I would prefer to not see it come into Apache Pekko. If the > > > lib is very widely used, then we would have to consider it regardless. > > > > > > As per my earlier point, you can make a somewhat reasonable claim that > > its > > > more > > > widely used than other current akka-persistence modules (I mean look at > > > https://github.com/akka/akka-persistence-r2dbc, it has only 20 stars). > > At > > > least > > > from a cursory glance, it does pass the "at least decently used" > > benchmark > > > > > > > If a regular Pekko committer comes with a project that they would > like > > > to bring into Apache Pekko, I would be much more supportive of that > > > use case. In that case, there would be a better chance that the code > > > could be maintained once it becomes an Apache Pekko lib. To put it > > > another way, get actively involved with Pekko development generally > > > and then convince us to take on your lib. > > > > > > Agreed, I can definitely say if the maintainers of > > > https://github.com/scullxbones/pekko-persistence-mongo > > > helped out in contributing with Pekko in general and/or would help out > in > > > the effort > > > of both adding and then maintaining an official Pekko Persistence mongo > > > than there > > > would be much more buy in. > > > > > > On a related note, previously I talked about technical overhead for > Pekko > > > repositories > > > which is something that I plan to help out with once the release > > craziness > > > is over. > > > There is a huge amount of manual technical process which can easily be > > > abstracted > > > over and centralized which would greatly reduce the overhead of adding > > > future Pekko > > > repos in the future. > > > > > > On Mon, Jul 17, 2023 at 10:09 PM PJ Fanning <[email protected]> > wrote: > > > > > >> I just don't see why there can't be an ecosystem of independent > > >> projects built on Pekko. I don't think Apache Pekko needs to take > > >> responsibility for all the jars. I have to reiterate just how much > > >> harder it is to release an Apache jar than it is to release a jar > > >> independently. > > >> > > >> There are lots of Apache projects with a vibrant ecosystem of related > > >> but independent libs. Have a look at Apache Spark or Flink or Hadoop. > > >> > > >> If a regular Pekko committer comes with a project that they would like > > >> to bring into Apache Pekko, I would be much more supportive of that > > >> use case. In that case, there would be a better chance that the code > > >> could be maintained once it becomes an Apache Pekko lib. To put it > > >> another way, get actively involved with Pekko development generally > > >> and then convince us to take on your lib. > > >> > > >> If there is a lib and the maintainer no longer has time to maintain > > >> it, then I would prefer to not see it come into Apache Pekko. If the > > >> lib is very widely used, then we would have to consider it regardless. > > >> > > >> > > >> On Mon, 17 Jul 2023 at 20:18, Brendan Doyle <[email protected]> > > wrote: > > >> > > > >> > Agree with all the points of view here, it’s certainly too soon to > > even > > >> > really have a meaning discussion on the philosophy. > > >> > > > >> > PJ for your curiosity on the repo I don’t think was public until > over > > >> the > > >> > weekend when I reached out about the pekko process. The port can be > > >> found > > >> > here. I’m not sure if the reactive mongo dependency is stil in > there, > > >> but I > > >> > was told they’re removing support for that driver since no pekko > > support > > >> > and using solely the official mongo scala driver which the Akka > > version > > >> > also had support for. Supposedly it was built so both drivers could > be > > >> used > > >> > interchangeably on the Akka library, they’re just removing the one > > that > > >> > doesn’t have the pekko support. > > >> > > > >> > https://github.com/scullxbones/pekko-persistence-mongo > > >> > > > >> > On Mon, Jul 17, 2023 at 12:09 PM Matthew de Detrich > > >> > <[email protected]> wrote: > > >> > > > >> > > I agree with PJ fanning when it comes to adding the module right > > now, > > >> we > > >> > > simply don't have > > >> > > the capacity and there are way too many things on our plate. > However > > >> when a > > >> > > release occurs > > >> > > and things stabilize a little more it is a different circumstance > > >> which is > > >> > > where I get to > > >> > > > > >> > > > After that, I would probably start arguing to REDUCE the Apache > > >> Pekko > > >> > > code base as opposed to increase it. There are a lot of connectors > > >> > > that have broken tests or tests disabled. It's hard enough to > > support > > >> > > and test the core Pekko features (like Actors, Clustering, HTTP, > > >> > > etc.), while then also having to maintain all the integrations > with > > a > > >> > > huge variety of 3rd party tools. It's taken us 8 months to release > > the > > >> > > Pekko core. We simply can't handle the enormous size of the > current > > >> > > code base. We need to concentrate on the core tooling. > > >> > > > > >> > > So there are 2 views successful views here, one is that > > >> > > we grow Pekko along with our active contributors, which means that > > >> > > we add in more modules when they make technical sense but we also > > >> > > grow the number of active contributors so that this > > >> burden/maintenance can > > >> > > actually be handled. The other view is that we reduce the > > >> > > scope of what is in Pekko so that we can maintain what we > currently > > >> have > > >> > > better. > > >> > > > > >> > > I currently lean for the former, and if we look at other projects > > like > > >> > > Flink > > >> > > and Airflow (which have more connectors than Pekko does) they do > > this > > >> > > and they do it successfully. If you boil it down, there isn't > > >> difference in > > >> > > the community that maintains a persistence-mongo thats external to > > >> > > Apache Pekko versus a community within Apache Pekko that maintains > > >> > > persistence-mongo. > > >> > > > > >> > > Which gets to the crux of this statement > > >> > > > > >> > > > Let the people > > >> > > who need the integrations (connectors and persistence > > implementations) > > >> > > maintain them. > > >> > > > > >> > > I would actually argue that our priority after release is to > > actually > > >> grow > > >> > > the > > >> > > community so that it can be maintained within Apache Pekko, so > those > > >> people > > >> > > who need the integrations that you refer to actually become part > of > > >> Pekko. > > >> > > It may have been in Lightbend's interest to restrict what Akka can > > >> accept > > >> > > in their org because their open source governance model didn't > allow > > >> > > them to employ the community. > > >> > > > > >> > > Its well known that it was impossible to get maintainer/committer > > >> rights to > > >> > > any of Lightbends Akka's repos aside from Alpakka which actually > and > > >> > > ironically got a large amount of support and contributions from > the > > >> > > community considering how many connectors it supported. And I > think > > >> > > there is a premise here, build some bridges plus a story and > people > > >> will > > >> > > come. > > >> > > > > >> > > In summary, in my view it's too soon right now to add this right > > now. > > >> There > > >> > > is > > >> > > just way too much technical and process overhead when it comes to > > >> adding > > >> > > more > > >> > > repos. When those get solved then we can revisit it, in the > meantime > > >> as PJ > > >> > > Fanning stated the akka dependencies of persistence-mongo for > > obvious > > >> > > reasons > > >> > > need to be moved over to pekko (and that's before we even get to > the > > >> core > > >> > > persistence-mongo library) so whatever is decided shouldn't block > > you > > >> > > either way, > > >> > > all we are saying is that we ourselves don't have capacity to do > the > > >> work > > >> > > now. > > >> > > > > >> > > On Mon, Jul 17, 2023 at 8:13 PM PJ Fanning <[email protected]> > > >> wrote: > > >> > > > > >> > > > When I checked akka-persistence-mongo at the weekend, it has a > > >> > > > dependency on reactivemongo-akkastream which has a dependency on > > >> > > > reactivemongo (which also has an akka dependency). > > >> > > > > > >> > > > This might have changed for the pekko equivalent. > > >> > > > > > >> > > > Even if it has, I'm still going to argue against Apache Pekko > > taking > > >> > > > on any more code until we get the current code set of Pekko > > modules > > >> > > > released. And that will take months. > > >> > > > > > >> > > > After that, I would probably start arguing to REDUCE the Apache > > >> Pekko > > >> > > > code base as opposed to increase it. There are a lot of > connectors > > >> > > > that have broken tests or tests disabled. It's hard enough to > > >> support > > >> > > > and test the core Pekko features (like Actors, Clustering, HTTP, > > >> > > > etc.), while then also having to maintain all the integrations > > with > > >> a > > >> > > > huge variety of 3rd party tools. It's taken us 8 months to > release > > >> the > > >> > > > Pekko core. We simply can't handle the enormous size of the > > current > > >> > > > code base. We need to concentrate on the core tooling. Let the > > >> people > > >> > > > who need the integrations (connectors and persistence > > >> implementations) > > >> > > > maintain them. > > >> > > > > > >> > > > On Mon, 17 Jul 2023 at 18:30, Brendan Doyle < > [email protected] > > > > > >> > > wrote: > > >> > > > > > > >> > > > > That’s fine I expected that to be the outcome for the reasons > > you > > >> > > listed, > > >> > > > > but wanted to raise the discussion. Though I am curious what > you > > >> mean > > >> > > by > > >> > > > > the lib dependencies that don’t support pekko, which library > > >> > > dependencies > > >> > > > > are you referring to within? The maintainers of the akka mongo > > >> > > > persistence > > >> > > > > library have already successfully ported a pekko repo which I > > >> linked , > > >> > > > > they’re just working through some artifact publishing issues > > >> before a > > >> > > > > release happens. > > >> > > > > > > >> > > > > On Mon, Jul 17, 2023 at 10:03 AM PJ Fanning < > > [email protected]> > > >> > > wrote: > > >> > > > > > > >> > > > > > I'd be a strong -1 on this. > > >> > > > > > > > >> > > > > > Maybe in distant future. > > >> > > > > > > > >> > > > > > There are too many lib dependencies in that module. None of > > >> which > > >> > > > support > > >> > > > > > Pekko. > > >> > > > > > > > >> > > > > > Software brought into ASF requires permissions from all the > > past > > >> > > > > > contributors. Pekko itself is an exception because of the > Akka > > >> > > license > > >> > > > > > change. > > >> > > > > > > > >> > > > > > ASF requires us to do loads of process work and read every > > line > > >> to > > >> > > > check > > >> > > > > > its origin in case of licensing issues. Everything required > > >> multiple > > >> > > > votes. > > >> > > > > > > > >> > > > > > Why don't you start by talking to the existing lib > maintainers > > >> about > > >> > > > them > > >> > > > > > releasing Pekko libs? If that doesn't work out, you could > fork > > >> them > > >> > > > > > yourself. > > >> > > > > > > > >> > > > > > Pekko team have far too much code and too few active > > developers > > >> to > > >> > > > become a > > >> > > > > > shelter for all the old Akka libs that no-one wants to > > maintain. > > >> > > > > > > > >> > > > > > On Mon 17 Jul 2023, 17:31 Brendan Doyle, < > > [email protected]> > > >> > > wrote: > > >> > > > > > > > >> > > > > > > Whoops forgot to include the link to the repo. > > >> > > > > > > > > >> > > > > > > https://github.com/scullxbones/pekko-persistence-mongo > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Mon, Jul 17, 2023 at 9:26 AM Brendan Doyle < > > >> > > [email protected]> > > >> > > > > > > wrote: > > >> > > > > > > > > >> > > > > > > > Hi, > > >> > > > > > > > > > >> > > > > > > > The skullxbones mongo persistence library was the de > facto > > >> > > > connector > > >> > > > > > for > > >> > > > > > > > mongo, never having a dedicated library from Lightbend. > > The > > >> > > creator > > >> > > > > > and a > > >> > > > > > > > collaborator have already successfu > > < > https://www.google.com/maps/search/%3E+collaborator+have+already+successfu?entry=gmail&source=g > >lly > > ported the library to > > >> > > pekko > > >> > > > > > 1.0.0 > > >> > > > > > > > and are working on getting a release out. The port has > > >> dropped > > >> > > > support > > >> > > > > > > for > > >> > > > > > > > reactive mongo and only using the official > > >> mongo-scala-driver > > >> > > going > > >> > > > > > > > forward. They would like to see the library donated to > > >> pekko if > > >> > > the > > >> > > > > > > > community / team would be interested. I know there’s a > > >> couple > > >> > > > official > > >> > > > > > > > persistence libraries for widely used db’s like > dynamodb, > > >> > > > personally > > >> > > > > > > though > > >> > > > > > > > biased I would argue mongo falls into that category. I’m > > >> not sure > > >> > > > what > > >> > > > > > > the > > >> > > > > > > > bar or threshold is to take in another repo or if > there’s > > a > > >> long > > >> > > > term > > >> > > > > > > plan > > >> > > > > > > > to combine persistence drivers into a single repo like > the > > >> way > > >> > > > stream > > >> > > > > > > > connectors are set up besides kafka. > > >> > > > > > > > > > >> > > > > > > > Thanks, > > >> > > > > > > > Brendan > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >> --------------------------------------------------------------------- > > >> > > > To unsubscribe, e-mail: [email protected] > > >> > > > For additional commands, e-mail: [email protected] > > >> > > > > > >> > > > > > >> > > > > >> > > -- > > >> > > > > >> > > Matthew de Detrich > > >> > > > > >> > > *Aiven Deutschland GmbH* > > >> > > > > >> > > Immanuelkirchstraße 26, 10405 Berlin > > >> > > > > >> > > Amtsgericht Charlottenburg, HRB 209739 B > > >> > > > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > >> > > > > >> > > *m:* +491603708037 > > >> > > > > >> > > *w:* aiven.io *e:* [email protected] > > >> > > > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: [email protected] > > >> For additional commands, e-mail: [email protected] > > >> > > >> > > > > > > -- > > > > > > Matthew de Detrich > > > > > > *Aiven Deutschland GmbH* > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > *m:* +491603708037 > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* [email protected] > > >
