> Can we have some community projects under the pekko-community organization before we can take in? Currently we have many projects under the apache organization.
Apache **is** a community organization, there is no separation here. It's not like Akka where you have official repos and then some community repos like Alpakka and because of that I don't think there is any formal concept of such segregation in ASF projects (I mean there is incubator but that is global status to the project, once we are out of incubator every project is out of incubator and I don't think adding new modules should go through an incubation process). With Ligthbend it makes sense, they were the BDFL/arbitror of their projects but If they wanted the community to have control over some of their projects on a case by case basis they would make this distinction. Currently with Pekko, every single module is as community driven as every other. In fact I wouldn't be surprised if such segregation is actively discouraged within ASF. On Tue, Jul 18, 2023 at 4:59 AM kerr <[email protected]> wrote: > Can we have some community projects under the pekko-community organization > before we can take in? Currently we have many projects under the apache > organization. > > 何品 > > > Brendan Doyle <[email protected]> 于2023年7月18日周二 06:35写道： > > > The one thing I was going to bring up which Matthew did was consistency > > with the persistence modules. I help manage Apache Openwhisk and it’s a > > similar problem of many repos management and keeping the project alive > with > > active committers that actually fits the scope of what there is to > > maintain. > > > > I will add on to Matthew’s point is many professional organizations > > encourage open source contributions to apache versus allowing > contribution > > to random repos. If the repo is under apache you’re far more likely to > > attract new contributors. If the repo is low maintenance since it’s long > > matured (in this case it hasn’t needed a release since 2021 and has been > > working just fine) and just needs security patches / dependency bumps, > > there’s not too much risk to adding it on outside the obvious headache > of a > > process of adding a new repo initially. And there should also be a valid > > proactive deprecation process for something that’s dead if you do bring > it > > in, find it’s not used and no one is helping give the maintenance it > needs > > to exist. > > > > Personally, which again I am biased, I think there needs to be a > > delineation between what is a new pekko integration someone makes or a > > small pet Akka library made by an individual that wasn’t used (not > wanting > > to take that on) and what is a de facto integration that was used by Akka > > and the only reason an official one didn’t exist was because Lightbend > > couldn’t commit to maintaining it. Which is essentially the same argument > > here against it, the only difference is I think there is value in having > it > > under the apache name in both 1. Just having more orgs being willing to > use > > it and 2. More orgs being able to contribute back to it. That on its own > > isn’t a reason to do something here, but just wanted to point that out. > > > > And of course I’m sympathetic to being on the other end of the drive by > > “hey can you consider taking on this additional work” so I don’t think my > > opinion holds any value here, just wanted to start a convo on the > > philosophy of some of the things I think was debatably borderline > official > > under Akka but wasn’t; and having a specific example we could do a case > > study on how to come to a decision by analyzing it. And thought I could > > help start the convo for the fine folks that have put in the effort to > port > > that module already so quickly, we all understand it’s far too soon to > > figure this out and it can exist as is under its existing repo for the > time > > being while all of this initial massive work for getting pekko off the > > ground is still underway. > > > > P.S. for what it’s worth, I’ve always thought pekko persistence on a > > serverless solution like dynamodb that is on demand cost based sounds > quite > > expensive of a solution versus using a managed db like mongo when the > > amount of event writing can be extremely high, if dynamodb is the main > > supported persistence module. But depends on the use case. Just my two > > cents :) > > > > Brendan > > > > On Mon, Jul 17, 2023 at 2:18 PM Matthew de Detrich > > <[email protected]> wrote: > > > > > Oh and one other last thing I would like to mention, at least in my > view > > > its better to have a decently sized amount of committers that is > > > maintaining > > > a larger set of modules rather than having almost no committers > > maintaining > > > a smaller set of modules, the former is a far healthier state for the > > > project to > > > be in. > > > > > > So given that, if adding modules also happens to increase the amount of > > > active committers that we add to Pekko, that is a net positive because > > > most of these modules don't need that much maintenance (once > integrated) > > > and the committers that have been added as a result of adding modules > > > are more incentivized to help out elsewhere since they have more skin > > > in the game. > > > > > > > > > On Mon, Jul 17, 2023 at 11:11 PM Matthew de Detrich < > > > [email protected]> wrote: > > > > > > > > I just don't see why there can't be an ecosystem of independent > > > > projects built on Pekko. I don't think Apache Pekko needs to take > > > > responsibility for all the jars. I have to reiterate just how much > > > > harder it is to release an Apache jar than it is to release a jar > > > > independently. > > > > > > > > I am aware of these things but I think more nuance needs to be > > > > added here onto this point. For example one is consistency, that > > > > is why do we have persistence-dynamodb as a core Pekko module > > > > but not persistence-mongodb? This then needs to be combined > > > > with usage + community, I picked persistence-dynamodb here > > > > deliberately because if we use a crude metric of github stars it > > > > akka-persistence-dynamodb appears to be less used (86 stars) > > > > than akka-persistence-mongo which has (104) stars. > > > > > > > > When Akka was under Lightbend this might have been more > > > > acceptable (i.e. hypothetically Lightbend probably had some > > > > paying Akka customers that happened to use persistence-dynamodb) > > > > but if we are talking about usage + community which is the whole > point > > > > of Apache projects than if my hypothesis is correct then there is > > > actually > > > > more of an argument including akka-persistence-mongo as a core Pekko > > > > module rather than some existing modules being there. > > > > > > > > I understand the current concerns about maintenance, but I don't > > > > think it's useful or constructive to stonewall potential modules > being > > > > added right now, we should evaluate in the future once the Pekko > > > > release storm settles down. > > > > > > > > > There are lots of Apache projects with a vibrant ecosystem of > related > > > > but independent libs. Have a look at Apache Spark or Flink or Hadoop. > > > > > > > > Yes but they have a core set of connectors which is comparable to > scope > > > > and size of Pekko which is what my core point was (see > > > > > > > > > > https://github.com/search?q=org%3Aapache+flink-connector&type=repositories > > > > as an example with Flink). And given the prevalence of mongo as a > > storage > > > > solution I would consider it a core connector/persistence storage, > just > > > > like > > > > there is a core Flink Mongo connector (see > > > > https://github.com/apache/flink-connector-mongodb). > > > > > > > > > If there is a lib and the maintainer no longer has time to maintain > > > > it, then I would prefer to not see it come into Apache Pekko. If the > > > > lib is very widely used, then we would have to consider it > regardless. > > > > > > > > As per my earlier point, you can make a somewhat reasonable claim > that > > > its > > > > more > > > > widely used than other current akka-persistence modules (I mean look > at > > > > https://github.com/akka/akka-persistence-r2dbc, it has only 20 > stars). > > > At > > > > least > > > > from a cursory glance, it does pass the "at least decently used" > > > benchmark > > > > > > > > > If a regular Pekko committer comes with a project that they would > > like > > > > to bring into Apache Pekko, I would be much more supportive of that > > > > use case. In that case, there would be a better chance that the code > > > > could be maintained once it becomes an Apache Pekko lib. To put it > > > > another way, get actively involved with Pekko development generally > > > > and then convince us to take on your lib. > > > > > > > > Agreed, I can definitely say if the maintainers of > > > > https://github.com/scullxbones/pekko-persistence-mongo > > > > helped out in contributing with Pekko in general and/or would help > out > > in > > > > the effort > > > > of both adding and then maintaining an official Pekko Persistence > mongo > > > > than there > > > > would be much more buy in. > > > > > > > > On a related note, previously I talked about technical overhead for > > Pekko > > > > repositories > > > > which is something that I plan to help out with once the release > > > craziness > > > > is over. > > > > There is a huge amount of manual technical process which can easily > be > > > > abstracted > > > > over and centralized which would greatly reduce the overhead of > adding > > > > future Pekko > > > > repos in the future. > > > > > > > > On Mon, Jul 17, 2023 at 10:09 PM PJ Fanning <[email protected]> > > wrote: > > > > > > > >> I just don't see why there can't be an ecosystem of independent > > > >> projects built on Pekko. I don't think Apache Pekko needs to take > > > >> responsibility for all the jars. I have to reiterate just how much > > > >> harder it is to release an Apache jar than it is to release a jar > > > >> independently. > > > >> > > > >> There are lots of Apache projects with a vibrant ecosystem of > related > > > >> but independent libs. Have a look at Apache Spark or Flink or > Hadoop. > > > >> > > > >> If a regular Pekko committer comes with a project that they would > like > > > >> to bring into Apache Pekko, I would be much more supportive of that > > > >> use case. In that case, there would be a better chance that the code > > > >> could be maintained once it becomes an Apache Pekko lib. To put it > > > >> another way, get actively involved with Pekko development generally > > > >> and then convince us to take on your lib. > > > >> > > > >> If there is a lib and the maintainer no longer has time to maintain > > > >> it, then I would prefer to not see it come into Apache Pekko. If the > > > >> lib is very widely used, then we would have to consider it > regardless. > > > >> > > > >> > > > >> On Mon, 17 Jul 2023 at 20:18, Brendan Doyle <[email protected]> > > > wrote: > > > >> > > > > >> > Agree with all the points of view here, it’s certainly too soon to > > > even > > > >> > really have a meaning discussion on the philosophy. > > > >> > > > > >> > PJ for your curiosity on the repo I don’t think was public until > > over > > > >> the > > > >> > weekend when I reached out about the pekko process. The port can > be > > > >> found > > > >> > here. I’m not sure if the reactive mongo dependency is stil in > > there, > > > >> but I > > > >> > was told they’re removing support for that driver since no pekko > > > support > > > >> > and using solely the official mongo scala driver which the Akka > > > version > > > >> > also had support for. Supposedly it was built so both drivers > could > > be > > > >> used > > > >> > interchangeably on the Akka library, they’re just removing the one > > > that > > > >> > doesn’t have the pekko support. > > > >> > > > > >> > https://github.com/scullxbones/pekko-persistence-mongo > > > >> > > > > >> > On Mon, Jul 17, 2023 at 12:09 PM Matthew de Detrich > > > >> > <[email protected]> wrote: > > > >> > > > > >> > > I agree with PJ fanning when it comes to adding the module right > > > now, > > > >> we > > > >> > > simply don't have > > > >> > > the capacity and there are way too many things on our plate. > > However > > > >> when a > > > >> > > release occurs > > > >> > > and things stabilize a little more it is a different > circumstance > > > >> which is > > > >> > > where I get to > > > >> > > > > > >> > > > After that, I would probably start arguing to REDUCE the > Apache > > > >> Pekko > > > >> > > code base as opposed to increase it. There are a lot of > connectors > > > >> > > that have broken tests or tests disabled. It's hard enough to > > > support > > > >> > > and test the core Pekko features (like Actors, Clustering, HTTP, > > > >> > > etc.), while then also having to maintain all the integrations > > with > > > a > > > >> > > huge variety of 3rd party tools. It's taken us 8 months to > release > > > the > > > >> > > Pekko core. We simply can't handle the enormous size of the > > current > > > >> > > code base. We need to concentrate on the core tooling. > > > >> > > > > > >> > > So there are 2 views successful views here, one is that > > > >> > > we grow Pekko along with our active contributors, which means > that > > > >> > > we add in more modules when they make technical sense but we > also > > > >> > > grow the number of active contributors so that this > > > >> burden/maintenance can > > > >> > > actually be handled. The other view is that we reduce the > > > >> > > scope of what is in Pekko so that we can maintain what we > > currently > > > >> have > > > >> > > better. > > > >> > > > > > >> > > I currently lean for the former, and if we look at other > projects > > > like > > > >> > > Flink > > > >> > > and Airflow (which have more connectors than Pekko does) they do > > > this > > > >> > > and they do it successfully. If you boil it down, there isn't > > > >> difference in > > > >> > > the community that maintains a persistence-mongo thats external > to > > > >> > > Apache Pekko versus a community within Apache Pekko that > maintains > > > >> > > persistence-mongo. > > > >> > > > > > >> > > Which gets to the crux of this statement > > > >> > > > > > >> > > > Let the people > > > >> > > who need the integrations (connectors and persistence > > > implementations) > > > >> > > maintain them. > > > >> > > > > > >> > > I would actually argue that our priority after release is to > > > actually > > > >> grow > > > >> > > the > > > >> > > community so that it can be maintained within Apache Pekko, so > > those > > > >> people > > > >> > > who need the integrations that you refer to actually become part > > of > > > >> Pekko. > > > >> > > It may have been in Lightbend's interest to restrict what Akka > can > > > >> accept > > > >> > > in their org because their open source governance model didn't > > allow > > > >> > > them to employ the community. > > > >> > > > > > >> > > Its well known that it was impossible to get > maintainer/committer > > > >> rights to > > > >> > > any of Lightbends Akka's repos aside from Alpakka which actually > > and > > > >> > > ironically got a large amount of support and contributions from > > the > > > >> > > community considering how many connectors it supported. And I > > think > > > >> > > there is a premise here, build some bridges plus a story and > > people > > > >> will > > > >> > > come. > > > >> > > > > > >> > > In summary, in my view it's too soon right now to add this right > > > now. > > > >> There > > > >> > > is > > > >> > > just way too much technical and process overhead when it comes > to > > > >> adding > > > >> > > more > > > >> > > repos. When those get solved then we can revisit it, in the > > meantime > > > >> as PJ > > > >> > > Fanning stated the akka dependencies of persistence-mongo for > > > obvious > > > >> > > reasons > > > >> > > need to be moved over to pekko (and that's before we even get to > > the > > > >> core > > > >> > > persistence-mongo library) so whatever is decided shouldn't > block > > > you > > > >> > > either way, > > > >> > > all we are saying is that we ourselves don't have capacity to do > > the > > > >> work > > > >> > > now. > > > >> > > > > > >> > > On Mon, Jul 17, 2023 at 8:13 PM PJ Fanning <[email protected] > > > > > >> wrote: > > > >> > > > > > >> > > > When I checked akka-persistence-mongo at the weekend, it has a > > > >> > > > dependency on reactivemongo-akkastream which has a dependency > on > > > >> > > > reactivemongo (which also has an akka dependency). > > > >> > > > > > > >> > > > This might have changed for the pekko equivalent. > > > >> > > > > > > >> > > > Even if it has, I'm still going to argue against Apache Pekko > > > taking > > > >> > > > on any more code until we get the current code set of Pekko > > > modules > > > >> > > > released. And that will take months. > > > >> > > > > > > >> > > > After that, I would probably start arguing to REDUCE the > Apache > > > >> Pekko > > > >> > > > code base as opposed to increase it. There are a lot of > > connectors > > > >> > > > that have broken tests or tests disabled. It's hard enough to > > > >> support > > > >> > > > and test the core Pekko features (like Actors, Clustering, > HTTP, > > > >> > > > etc.), while then also having to maintain all the integrations > > > with > > > >> a > > > >> > > > huge variety of 3rd party tools. It's taken us 8 months to > > release > > > >> the > > > >> > > > Pekko core. We simply can't handle the enormous size of the > > > current > > > >> > > > code base. We need to concentrate on the core tooling. Let the > > > >> people > > > >> > > > who need the integrations (connectors and persistence > > > >> implementations) > > > >> > > > maintain them. > > > >> > > > > > > >> > > > On Mon, 17 Jul 2023 at 18:30, Brendan Doyle < > > [email protected] > > > > > > > >> > > wrote: > > > >> > > > > > > > >> > > > > That’s fine I expected that to be the outcome for the > reasons > > > you > > > >> > > listed, > > > >> > > > > but wanted to raise the discussion. Though I am curious what > > you > > > >> mean > > > >> > > by > > > >> > > > > the lib dependencies that don’t support pekko, which library > > > >> > > dependencies > > > >> > > > > are you referring to within? The maintainers of the akka > mongo > > > >> > > > persistence > > > >> > > > > library have already successfully ported a pekko repo which > I > > > >> linked , > > > >> > > > > they’re just working through some artifact publishing issues > > > >> before a > > > >> > > > > release happens. > > > >> > > > > > > > >> > > > > On Mon, Jul 17, 2023 at 10:03 AM PJ Fanning < > > > [email protected]> > > > >> > > wrote: > > > >> > > > > > > > >> > > > > > I'd be a strong -1 on this. > > > >> > > > > > > > > >> > > > > > Maybe in distant future. > > > >> > > > > > > > > >> > > > > > There are too many lib dependencies in that module. None > of > > > >> which > > > >> > > > support > > > >> > > > > > Pekko. > > > >> > > > > > > > > >> > > > > > Software brought into ASF requires permissions from all > the > > > past > > > >> > > > > > contributors. Pekko itself is an exception because of the > > Akka > > > >> > > license > > > >> > > > > > change. > > > >> > > > > > > > > >> > > > > > ASF requires us to do loads of process work and read every > > > line > > > >> to > > > >> > > > check > > > >> > > > > > its origin in case of licensing issues. Everything > required > > > >> multiple > > > >> > > > votes. > > > >> > > > > > > > > >> > > > > > Why don't you start by talking to the existing lib > > maintainers > > > >> about > > > >> > > > them > > > >> > > > > > releasing Pekko libs? If that doesn't work out, you could > > fork > > > >> them > > > >> > > > > > yourself. > > > >> > > > > > > > > >> > > > > > Pekko team have far too much code and too few active > > > developers > > > >> to > > > >> > > > become a > > > >> > > > > > shelter for all the old Akka libs that no-one wants to > > > maintain. > > > >> > > > > > > > > >> > > > > > On Mon 17 Jul 2023, 17:31 Brendan Doyle, < > > > [email protected]> > > > >> > > wrote: > > > >> > > > > > > > > >> > > > > > > Whoops forgot to include the link to the repo. > > > >> > > > > > > > > > >> > > > > > > https://github.com/scullxbones/pekko-persistence-mongo > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > On Mon, Jul 17, 2023 at 9:26 AM Brendan Doyle < > > > >> > > [email protected]> > > > >> > > > > > > wrote: > > > >> > > > > > > > > > >> > > > > > > > Hi, > > > >> > > > > > > > > > > >> > > > > > > > The skullxbones mongo persistence library was the de > > facto > > > >> > > > connector > > > >> > > > > > for > > > >> > > > > > > > mongo, never having a dedicated library from > Lightbend. > > > The > > > >> > > creator > > > >> > > > > > and a > > > >> > > > > > > > collaborator have already successfu > > > < > > > https://www.google.com/maps/search/%3E+collaborator+have+already+successfu?entry=gmail&source=g > > >lly > > > ported the library to > > > >> > > pekko > > > >> > > > > > 1.0.0 > > > >> > > > > > > > and are working on getting a release out. The port has > > > >> dropped > > > >> > > > support > > > >> > > > > > > for > > > >> > > > > > > > reactive mongo and only using the official > > > >> mongo-scala-driver > > > >> > > going > > > >> > > > > > > > forward. They would like to see the library donated to > > > >> pekko if > > > >> > > the > > > >> > > > > > > > community / team would be interested. I know there’s a > > > >> couple > > > >> > > > official > > > >> > > > > > > > persistence libraries for widely used db’s like > > dynamodb, > > > >> > > > personally > > > >> > > > > > > though > > > >> > > > > > > > biased I would argue mongo falls into that category. > I’m > > > >> not sure > > > >> > > > what > > > >> > > > > > > the > > > >> > > > > > > > bar or threshold is to take in another repo or if > > there’s > > > a > > > >> long > > > >> > > > term > > > >> > > > > > > plan > > > >> > > > > > > > to combine persistence drivers into a single repo like > > the > > > >> way > > > >> > > > stream > > > >> > > > > > > > connectors are set up besides kafka. > > > >> > > > > > > > > > > >> > > > > > > > Thanks, > > > >> > > > > > > > Brendan > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > > > >> > --------------------------------------------------------------------- > > > >> > > > To unsubscribe, e-mail: [email protected] > > > >> > > > For additional commands, e-mail: [email protected] > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > -- > > > >> > > > > > >> > > Matthew de Detrich > > > >> > > > > > >> > > *Aiven Deutschland GmbH* > > > >> > > > > > >> > > Immanuelkirchstraße 26, 10405 Berlin > > > >> > > > > > >> > > Amtsgericht Charlottenburg, HRB 209739 B > > > >> > > > > > >> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > >> > > > > > >> > > *m:* +491603708037 > > > >> > > > > > >> > > *w:* aiven.io *e:* [email protected] > > > >> > > > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: [email protected] > > > >> For additional commands, e-mail: [email protected] > > > >> > > > >> > > > > > > > > -- > > > > > > > > Matthew de Detrich > > > > > > > > *Aiven Deutschland GmbH* > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > > > *m:* +491603708037 > > > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > > > > > > > > -- > > > > > > Matthew de Detrich > > > > > > *Aiven Deutschland GmbH* > > > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > > > *m:* +491603708037 > > > > > > *w:* aiven.io *e:* [email protected] > > > > > > -- Matthew de Detrich *Aiven Deutschland GmbH* Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen *m:* +491603708037 *w:* aiven.io *e:* [email protected]
