I don't really like the idea of having milestone releases and all the extra voting and uptake work that this entails.
I am even less enthusiastic about a 1.1.0-M0 release that has no meaningful 1.1.0 changes and that is basically a copy of the 1.0.1 release. Bin compatibility checks for 1.1.0 can be done against 1.0.x jars like we already do today. On Wed 30 Aug 2023, 08:29 Matthew de Detrich, <[email protected]> wrote: > So I want to revive this topic now that the storm has settled a bit and the > modules > which the community wants released are now underway. > > I still think that this idea has merit for the reasons I stated (which I > don't want to repeat) > so I think going forward its just to do a vote for these milestone jar's. > > @PJ Fanning how do you feel about creating a milestone -M0 jar for pekko > core right at > the point when the branching happened, creating an ASF release for the > process (I > don't imagine there should be any issues here) so we can at least unblock > https://github.com/apache/incubator-pekko/pull/532 . > > We can also create another milestone in Pekko around now for downstream > projects > such as pekko-http to depend on, as there will be some significant changes > later on > so I think that right now marks a good point in time. > > > On Mon, Aug 7, 2023 at 5:00 PM Matthew de Detrich < > [email protected]> wrote: > > > So interestingly it's actually possible to have a mono-repo which > contains > > all of the pekko projects as > > they are now defined with each project being a sbt module within its own > > sub-folder AND each of those > > sbt modules having their own version lifecycle. Furthermore it should > also > > be possible for those sub > > folders to be loaded independently just by cd'ing into the folder. > > > > If its acceptable for Apache in the source distribution to only provide a > > subset of the sources which > > are relevant to the module being released at the time, given what was > said > > earlier about each project > > sub folder being able to loaded on its own this compromise may be able to > > give us the best of both > > worlds. > > > > On Mon, Aug 7, 2023 at 3:29 PM PJ Fanning <[email protected]> wrote: > > > >> One radical solution that we could consider is starting to merge repos > >> like pekko-http into the main pekko repo. I know this has drawbacks but > it > >> actually has a few advantages. > >> > >> Cons > >> * This main drawback is that we need to release all the modules in the > >> main repo at the same time. If we need to fix one module, we need to > >> release all the modules. Note that many of our repos already have lots > of > >> modules, so all that changes is the number of modules being released at > the > >> same time. > >> * We would need to refactor the CI so that we don't end up with > >> individual jobs running too much. We don't want to end up with a build > that > >> requires hours to verify a PR. > >> > >> Pros > >> * For users, it simplifies working out which versions of pekko modules > to > >> use. From the traffic in our online mailing lists and GitHub > discussions - > >> I think a lot of users are confused about why a module like > >> pekko-connectors-s3 is released separately from pekko-stream and has a > >> different version number. > >> * For testing the modules and how they work together, all the jars can > be > >> built into mavenLocal. > >> * No longer do we need to worry about how to get a pekko-stream change > >> tested in all the connectors > >> * For release voting, voters will not to have to work out the specific > >> circumstances of all the individual git repos that we have. > >> > >> This is a lot of work but it may be worth considering. > >> > >> > >> On 2023/08/07 11:17:18 Matthew de Detrich wrote: > >> > I made my case quite clear before, it breaks past builds if people > want > >> to > >> > load > >> > past versions of the project (and when I mean break I really do mean > >> > break, as in the build won't even load when you check out the source) > >> > and additionally there are strong arguments that this is a misuse of > >> > snapshots. > >> > > >> > The whole point of milestone is to have a persistent/permanent (this > is > >> > critical) > >> > artifact that regardless of when someone resolves a build at a > specific > >> > point > >> > in time they will always be able to load it whether it is now, 5 > days, 5 > >> > months or > >> > 5 years. If the main reason why uses will want to be loading past > >> builds to > >> > try and > >> > bisect some bug/issue in the past, it's imperative that they do > actually > >> > load the > >> > code that is relevant back then, not what is published now (because > >> > snapshots by > >> > definition only maintain the latest X versions of an artifact's > version > >> Y). > >> > Of course > >> > there are ways around this, but this basically involves users having > to > >> > carefully > >> > and transitively build all relevant dependencies in the dependency > tree > >> and > >> > the > >> > careful part here is significant because they need to build the > project > >> the > >> > same > >> > way it was built back then (think JDK versions here). > >> > > >> > This is why at least to me, using pinned snapshots as core > dependencies > >> > in Pekko modules is not negotiable from my side, I would much prefer > >> > publishing milestones as part of an Apache's Release process or just > >> > leave the core dependencies of Pekko as full release versions and > create > >> > a separate branch/CI matrix that dynamically builds the dependencies > >> > from source and/or loads snapshots to detect regressions (but even > >> > that has its own host of problems). > >> > > >> > The thing is none of the contributors have the capacity to do such > >> extensive > >> > changes, the whole point of the milstone solution is it is a trivial > and > >> > clean > >> > solution to many problems (not just this one, bringing up > >> > > >> > https://github.com/apache/incubator-pekko/pull/532#issuecomment-1667527994 > >> > again). It massively simplifies the code, makes it trivially easy for > >> pekko > >> > contributors to understand what is going on (i.e. no complex > dynamically > >> > finding > >> > out the latest snapshot as is done in > >> pekko-http/pekko-persistence-dynamodb) > >> > and doesn't have hair pulling gotchas like "why isn't this build > >> loading, > >> > oh great > >> > the snapshot is deleted because it was pruned so now I have to figure > >> out > >> > how > >> > that snapshot was built and do it all myself". > >> > > >> > Other Apache projects can use snapshots in this way, and that's fine > but > >> > Pekko > >> > already has enough issues with overburdening complexity and gotchas > >> that is > >> > making the project less approachable for contributors and using pinned > >> > snapshots in this way just makes it worse. > >> > > >> > On Mon, Aug 7, 2023 at 12:14 PM Claude Warren, Jr > >> > <[email protected]> wrote: > >> > > >> > > Matthew, your argument makes no sense to me. Let me see if I can > >> restate > >> > > it to be sure I have the salient points. > >> > > > >> > > 1. There are two modules, let's call them "core" and "child". > >> > > 2. There is a version where core and child work together. Let's > >> call > >> > > this 1.0.0 for both. I realize they can be different, but I want > >> to > >> > > keep > >> > > this as simple as possible. > >> > > 3. Now core develops a new version that has a change that may > >> cause a > >> > > regression. let's call this core-1.1.0-S1. > >> > > 4. The problem is we want "child" to be able to test against > >> > > core-1.1.0-S1 to verify that there are no regression issues. So > >> for the > >> > > immediate problem:, > >> > > 1. "child" can create a branch that tests against > core-1.1.0-S1. > >> > > Let's call this version child-1.1.0-regression (it has to be > >> the next > >> > > version of child because 1.0.0 is already released -- it could > >> > > be 1.0.1 but > >> > > I'm not going there) > >> > > 5. If there is a regression: > >> > > 1. that information will flow back to core dev and > >> core-1.1.0-S2 will > >> > > be created > >> > > 2. "child" can then test against that in > child-1.1.0-regression. > >> > > 3. In this case child-1.1.0-regression wants to test with the > >> latest > >> > > core-1.1.0-Sx version. > >> > > 6. If there was no regression there might be when core-1.1.0-S2 > is > >> > > published. > >> > > 1. In this case child-1.1.0-regression should be run > >> > > against core-1.1.0-S2 > >> > > 2. again "child" wants to run against the latest > >> "core-1.1.0-Sx". > >> > > 7. For both of the above paths the standard core-1.1.0-SNAPSHOT > >> fits the > >> > > bill. > >> > > 8. At some point "child" dev has to decide to go with core-1.1.0 > >> (after > >> > > it is released) or core-1.0.0 but that is a decision for "child" > >> and > >> > > either > >> > > will work (assuming child has no dependency on functionality > >> introduced > >> > > in > >> > > core-1.1.0). > >> > > 9. After core-1.1.0 is released core-1.1.0-Sx is deleted from the > >> > > repository, because it is a snapshot. And this leads to the > issue. > >> > > 10. At some later time someone wants to checkout > >> > > "child-1.1.0-regression" and have it build, but can not because > >> > > "core-1.1.0-Sx" is no longer found. There are 2 cases here: > >> > > > >> > > > >> > > 1. There is some issue being checked that does not involve the > >> changes > >> > > in core-1.1.0 at this moment in time, in which case core-1.1.0 > >> > > (release) > >> > > can replace core-1.1.0-Sx. > >> > > 2. There is some issue being checked with the regression tests > >> for > >> > > core-1.1.0-Sx. This feels like a very low probability issue > >> but, > >> > > let's > >> > > move forward with this discussion. > >> > > 1. At this point, the developer can checkout the core-1.1.0 > >> code > >> > > for the time frame in question and build it thus providing > >> the > >> > > library > >> > > necessary for the analysis they are performing. > >> > > 1. Is this solution optimal - no. > >> > > 2. Is it acceptable? In my opinion looking at > >> cost/benefit, I > >> > > think so. The probability that anyone will want to > >> rebuild > >> > > child-1.1.0-regression is low. In every case I can > think > >> of > >> > > 1. working with child-1.1.0 is the better solution as > >> that > >> > > is the code that was finally accepted to release, or > >> > > 2. if not released at least the accepted snapshot. > >> > > 3. If development on "child" has stalled and someone > is > >> > > picking it up again and the most advanced code is in > >> > > "child-1.1.0-regression" then they need to move the > >> > > dependency to > >> > > core-1.1.0 (if not later) and work forward from > there. > >> > > > >> > > I make some assumptions here: > >> > > > >> > > - Snapshot versions are retained in the repository by count as > >> well as > >> > > date so the last 5 or last 5 days whichever is greater sort of > >> thing. > >> > > - Snapshot versions are deleted when a release is made. > >> > > - I do not assume that child-1.1.0 will be released just > >> > > because core-1.1.0 is released, the regression check was just > that > >> and > >> > > does > >> > > not imply the version of core that will be used in child-1.1.0. > >> There > >> > > is > >> > > some argument to be made that the child regression check should > be > >> > > core-child-1.1.0-regression and that it should be the > >> responsibility of > >> > > the > >> > > core developers to make the test. But that is neither here nor > >> there as > >> > > the test is assumed to have been made for purposes of this > >> discussion. > >> > > - There is nothing stopping the "child" team from preserving any > >> version > >> > > of core-1.1.0-Sx that they wish to retain for testing or other > >> uses, > >> > > provided they don't release it. > >> > > > >> > > So if there is any disagreement please let me know which bullet > number > >> > > (and/or sub numbers) you disagree with and why. > >> > > > >> > > > >> > > On Mon, Aug 7, 2023 at 8:49 AM Matthew de Detrich > >> > > <[email protected]> wrote: > >> > > > >> > > > > I think that there is a communication probleem; crossed wires in > >> > > > terminology. Matthew talks about "general public" and most of us > >> grey > >> > > > heads (trying not to be gender specific here) in the ASF hear > >> "release". > >> > > > What I think Matthew is talking about is a wider testing audience. > >> > > People > >> > > > who understand that the code is not "released". If that is > >> > > > indeed the case, and I am certain Matthew will tell me if I am > >> wrong, > >> > > then > >> > > > the following should make sense. > >> > > > > >> > > > Yes this is completely correct > >> > > > > >> > > > > Use the snapshot repositories to their fullest. > >> > > > > >> > > > This is actually where the problem lies. We are already deploying > >> into > >> > > > snapshots > >> > > > (see > >> > > > > >> > https://repository.apache.org/content/groups/snapshots/org/apache/pekko/ > >> > > ). > >> > > > The problem is that due to the way snapshots are being deployed > >> currently > >> > > > they > >> > > > are not being pruned and INFRA has actually approached us saying > >> that at > >> > > > some point we need to fix this. In summary we cannot rely on > >> snapshots > >> > > > being > >> > > > persistent, they are designed to be pruned. > >> > > > > >> > > > Now while this may be fine for a wider general testing audience, > it > >> does > >> > > > pose > >> > > > other issues if we were to use snapshots to solve other issues, > i.e. > >> > > > testing > >> > > > merged changes in pekko main in other modules. Let me demonstrate > >> what I > >> > > > mean by this > >> > > > > >> > > > * Pekko merges some significant feature, lets say the upgrade from > >> netty3 > >> > > > to > >> > > > netty4 (which is a very good example) i.e. > >> > > > https://github.com/apache/incubator-pekko/pull/539 > >> > > > * Since this is a major we want to make sure that it doesn't cause > >> > > > unintended > >> > > > regressions in other pekko modules, such as pekko-management > >> > > > * This means that ideally we would like to update the pekko > version > >> in > >> > > > pekko-management to a version of pekko that has the netty3->netty4 > >> change > >> > > > so we can catch problems well before a release is made > >> > > > > >> > > > Now we could update the pekko dependency in pekko-management to > use > >> > > > a snapshot but as stated before the snapshot's are not meant to be > >> > > > persistent. > >> > > > This leaves us with the next best solution which is to use a > >> milestone > >> > > > which > >> > > > is designed to persist (unlike a snapshot). When enough > significant > >> > > changes > >> > > > get merged into Pekko, one of the PPMC/committers will notify > >> developers > >> > > > that a milestone is being published and when a milestone is > >> published we > >> > > > can just update the dependency in pekko-management to that > >> milestone. > >> > > > > >> > > > Now of course there are other solutions to this, i.e. pekko-http > and > >> > > > pekko-persistence-dynamodb happen to have code that will > >> > > > dynamically find the latest snapshot from Apache Maven Snapshots > >> > > > repository but this causes its own problems (see the conversation > >> > > > at > >> > > > > >> > > > > >> > > > >> > https://github.com/apache/incubator-pekko-http/issues/293#issuecomment-1666893217 > >> > > > ). > >> > > > The other solutions mentioned, i.e. using the nightlies repository > >> also > >> > > are > >> > > > less than ideal solutions i.e. publishing to nightlies repository > >> > > > requires us to manually deploy locally and move the jars via rsync > >> which > >> > > is > >> > > > quite > >> > > > brittle (i.e. the build will break whenever a folder structure > >> changes). > >> > > > More > >> > > > generally projects have actually been moving away from deploying > >> jars > >> > > into > >> > > > the nightlies repository because it's not the correct place to put > >> > > them[1], > >> > > > the > >> > > > nighlies repository is more suited to binary applications builds > >> (think > >> > > > executables for Apache OpenOffice) > >> > > > > >> > > > Deploying milestone jars will also happen to solve > >> > > > > >> > > > >> > https://github.com/apache/incubator-pekko/pull/532#issuecomment-1664741546 > >> > > > very > >> > > > cleanly. > >> > > > > >> > > > This is what I meant earlier when I said these are largely > technical > >> > > > problems. > >> > > > > >> > > > [1] > https://lists.apache.org/thread/t6598v7lsdb74q65x9grwhxqt38ntq2s > >> > > > > >> > > > On Mon, Aug 7, 2023 at 8:17 AM Claude Warren, Jr > >> > > > <[email protected]> wrote: > >> > > > > >> > > > > Here is my take on this discussion. Please stick with me for a > >> bit. > >> > > > > > >> > > > > I think that there is a communication probleem; crossed wires in > >> > > > > terminology. Matthew talks about "general public" and most of > us > >> grey > >> > > > > heads (trying not to be gender specific here) in the ASF hear > >> > > "release". > >> > > > > What I think Matthew is talking about is a wider testing > audience. > >> > > > People > >> > > > > who understand that the code is not "released". If that is > >> > > > > indeed the case, and I am certain Matthew will tell me if I am > >> wrong, > >> > > > then > >> > > > > the following should make sense. > >> > > > > > >> > > > > > >> > > > > - Pekko can build and place into the SNAPSHOT repository, > >> nightly > >> > > > > builds. These builds should be named > >> > > <pekko-module>-x.y.z-SNAPSHOT. > >> > > > > The > >> > > > > repository will take care of adding dates to them to ensure > the > >> > > > > latest SNAPSHOT is returned. (The repository should also > >> > > > automatically > >> > > > > retain only 5 or so snapshots for pekko-module-x.y.z but > that > >> is a > >> > > > > different issue) > >> > > > > - Pekko can build and place into the SNAPSHOT repository, a > >> longer > >> > > > > period (weekly,monthly) or manually triggered build that has > a > >> name > >> > > > like > >> > > > > <pekko-module>-x.y.z-M-SNAPSHOT. Where "M" is the literal > 'M'. > >> > > This > >> > > > > will > >> > > > > have the same retention properties as > >> <pekko-module>-x.y.z-SNAPSHOT. > >> > > > > - The "wider testing audience" needs to be pointed to the dev > >> > > mailing > >> > > > > list. There are several reasons for this, most notably: They > >> have > >> > > > > access > >> > > > > to the discussion about what is changing, they can be > notified > >> when > >> > > an > >> > > > > "M" > >> > > > > release is created, they can contribute ideas, suggestions > and > >> > > > concepts > >> > > > > back to the community, they become part of the community. > >> > > > > - The "wider testing audience" can depend upon > >> > > > > <pekko-module>-x.y.z.M-SNAPSHOT during their builds, provided > >> they > >> > > add > >> > > > > the > >> > > > > ASF snapshot repository. > >> > > > > - There is no "release" here, all work is in the "dev" side > of > >> the > >> > > > > house. > >> > > > > - The pekko dev team can produce Milestone ("M") builds when > >> they > >> > > > > desire, but may need to do manual cleanup of the repository > >> when the > >> > > > > actual > >> > > > > release is made. > >> > > > > - The pekko dev team can produce Regression ("R") builds that > >> > > perform > >> > > > > extensive regression testing if so desired. > >> > > > > - The pekko dev team can produce Foo ("F") builds to ensure > foo > >> > > > > correctness (whatever that is) if so desired. > >> > > > > - The pekko dev team will need to clean up all the lettered > >> versions > >> > > > > when the release is made or Infra will become very annoyed. > >> > > > > - The pekko dev team should try to limit the number of > lettered > >> > > > versions > >> > > > > so that infra does not become annoyed. > >> > > > > > >> > > > > The short bit here is that as long as the work is development > work > >> > > > > (building, testing, and verification) and not for the "general > >> public" > >> > > > > (grey head definition) then it can safely be done and "released" > >> > > > (non-grey > >> > > > > head definition) within the dev environment (source, build, > >> repository, > >> > > > and > >> > > > > community tools). > >> > > > > > >> > > > > My recommendations are: > >> > > > > > >> > > > > - Bring the "wider testing audience" into the dev > >> conversations. > >> > > > > - Announce milestone builds and internal releases in the dev > >> list. > >> > > > > - Use the snapshot repositories to their fullest. > >> > > > > - expand the pool of potential team members. > >> > > > > > >> > > > > That last point brings me to the other topic of PPMC members, > and > >> I > >> > > > > apologize for not seeing and paying attention to the discussion > >> on the > >> > > > > private list and will endeavour to do better in future. > >> > > > > > >> > > > > Claude > >> > > > > > >> > > > > On Sun, Aug 6, 2023 at 4:01 PM Matthew de Detrich > >> > > > > <[email protected]> wrote: > >> > > > > > >> > > > > > > I agree with all of Justin McLean's points. > >> > > > > > > >> > > > > > > I care about the ASF rules > >> > > > > > > >> > > > > > I am still waiting for an actual reference to a rule on this > and > >> > > Justin > >> > > > > > appears to have misunderstood what was being suggested (but I > >> > > > > > will wait for his answer) > >> > > > > > > >> > > > > > > and norms. > >> > > > > > > >> > > > > > I have seen more than non negligible amount of so-called > "norms" > >> > > > > > which turn out to be either false or some convention which > >> projects > >> > > > > > copy from other projects just because. Pekko is also not a > >> typical > >> > > > > > project in this specific regard, which by definition means its > >> not > >> > > > > > normal so I in this case I don't about norms unless there is a > >> rule > >> > > > > > behind it. > >> > > > > > > >> > > > > > > >> > > > > > On Sun, Aug 6, 2023 at 3:48 PM PJ Fanning < > [email protected] > >> > > >> > > > wrote: > >> > > > > > > >> > > > > > > I agree with all of Justin McLean's points. > >> > > > > > > > >> > > > > > > I absolutely don't care what non-ASF projects do. I care > >> about the > >> > > > ASF > >> > > > > > > rules and norms. > >> > > > > > > > >> > > > > > > On Sun, 6 Aug 2023 at 14:44, Matthew de Detrich > >> > > > > > > <[email protected]> wrote: > >> > > > > > > > > >> > > > > > > > > I remain strongly opposed to publishing to maven central > >> > > without > >> > > > a > >> > > > > > > > full release process with 2 phase voting. > >> > > > > > > > > >> > > > > > > > Can you elaborate on this opposition? There is plenty of > >> software > >> > > > > that > >> > > > > > is > >> > > > > > > > distributed on maven using commonly established milestone > >> suffix > >> > > > > > > > (i.e. -M0, -M1 etc etc) and I would argue it's far from > >> > > > controversial > >> > > > > > to > >> > > > > > > > state that it's extremely clear for users that such an > >> artifact > >> > > is > >> > > > a > >> > > > > > > > milestone, this is already established. > >> > > > > > > > > >> > > > > > > > I just want to know where you are coming from, because if > >> it's > >> > > just > >> > > > > due > >> > > > > > > > to it being a rule/Apache process this is what I want to > >> clarify > >> > > > (if > >> > > > > > > there > >> > > > > > > > is a clear specific rule on this that this is not allowed; > >> which > >> > > > has > >> > > > > > yet > >> > > > > > > > to be provided yet then yes I will drop it). > >> > > > > > > > > >> > > > > > > > On Sun, Aug 6, 2023 at 3:30 PM PJ Fanning < > >> [email protected]> > >> > > > > wrote: > >> > > > > > > > > >> > > > > > > > > I remain strongly opposed to publishing to maven central > >> > > without > >> > > > > full > >> > > > > > > > > release process with 2 phase voting. > >> > > > > > > > > > >> > > > > > > > > I can live with putting milestone jars elsewhere like > >> > > > > > > > > nightlies.apache.org. We used to have a CI job that > >> published > >> > > > jars > >> > > > > > to > >> > > > > > > > > nightlies.apache.org. This can easily be dusted down > and > >> > > > > repurposed. > >> > > > > > > > > My vague recollection was that the jars were published > in > >> a > >> > > Maven > >> > > > > > repo > >> > > > > > > > > compatible data structure so that they were easily > >> consumable > >> > > in > >> > > > > > > > > mvn/gradle/sbt builds. > >> > > > > > > > > > >> > > > > > > > > On Sun, 6 Aug 2023 at 14:18, Matthew de Detrich > >> > > > > > > > > <[email protected]> wrote: > >> > > > > > > > > > > >> > > > > > > > > > > And we can release voted on milestones. > >> > > > > > > > > > > >> > > > > > > > > > We can but for reasons stated earlier this to me seems > >> like a > >> > > > > > bandaid > >> > > > > > > > > > and as you pointed out in [1] we are having issues > >> currently > >> > > > with > >> > > > > > > getting > >> > > > > > > > > > enough votes for releases. Hence my biggest complaint > >> here is > >> > > > > that > >> > > > > > > > > > requiring votes sends the wrong signals, after all one > >> of the > >> > > > > main > >> > > > > > > points > >> > > > > > > > > > of my suggested use of milestones is to test published > >> > > > milestones > >> > > > > > in > >> > > > > > > > > > downstream modules to confirm that they are suitable > for > >> > > > general > >> > > > > > > public > >> > > > > > > > > > consumption which is the point that Justin raised > >> earlier. > >> > > > > > > > > > > >> > > > > > > > > > > In the meantime, we have: > >> > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/PEKKO/Testing+with+Pekko+Snapshot+Jars > >> > > > > > > > > > > >> > > > > > > > > > > Which is not ideal but it does provide some help. > >> > > > > > > > > > > >> > > > > > > > > > While this is (for now) fine at least for one case > (i.e. > >> > > > > > > users/developers > >> > > > > > > > > > testing quick changes which is intended use for > >> snapshots) it > >> > > > > > doesn't > >> > > > > > > > > > practically alleviate the other case regarding testing > >> > > > downstream > >> > > > > > > pekko > >> > > > > > > > > > modules with more recent upstream pekko changes. > >> > > > > > > > > > > >> > > > > > > > > > Using snapshots for this means we have to continuously > >> > > > add/remove > >> > > > > > > > > > the Apache Nexus snapshot repository in the downstream > >> > > modules > >> > > > > > build > >> > > > > > > > > > files. > >> > > > > > > > > > > >> > > > > > > > > > This point in general actually becoming increasingly > >> more > >> > > > > important > >> > > > > > > now > >> > > > > > > > > > considering we are getting significant changes merged > >> > > upstream > >> > > > in > >> > > > > > the > >> > > > > > > > > > Pekko 1.1.x series and our current approach of only > >> bringing > >> > > in > >> > > > > the > >> > > > > > > > > > changes when a release candidate is being voted on > >> means that > >> > > > > > > > > > we will be bringing in ~6-12 months of changes all at > >> once at > >> > > > an > >> > > > > > > > > > inopportune time (i.e. a release is being made) > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > [1] > >> > > > > > > >> https://lists.apache.org/thread/b57nmyrg5xhgdv3gnbq2o9s6wsqf31q6 > >> > > > > > > > > > > >> > > > > > > > > > On Sun, Aug 6, 2023 at 2:57 PM PJ Fanning < > >> > > > [email protected]> > >> > > > > > > wrote: > >> > > > > > > > > > > >> > > > > > > > > > > And we can release voted on milestones. > >> > > > > > > > > > > > >> > > > > > > > > > > In the meantime, we have: > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/PEKKO/Testing+with+Pekko+Snapshot+Jars > >> > > > > > > > > > > > >> > > > > > > > > > > Which is not ideal but it does provide some help. > >> > > > > > > > > > > > >> > > > > > > > > > > On Sun, 6 Aug 2023 at 13:38, Matthew de Detrich > >> > > > > > > > > > > <[email protected]> wrote: > >> > > > > > > > > > > > > >> > > > > > > > > > > > > Nobody is threatening to delete our recent > >> snapshots. > >> > > > There > >> > > > > > is > >> > > > > > > a > >> > > > > > > > > > > > question about what to do about older snapshots - > >> which > >> > > is > >> > > > > > > separate > >> > > > > > > > > > > > from this topic. > >> > > > > > > > > > > > > >> > > > > > > > > > > > I am not saying that people will be deleting our > >> > > snapshots, > >> > > > > but > >> > > > > > > > > rather > >> > > > > > > > > > > > there is a current problem with them being pruned > >> (or > >> > > more > >> > > > > > > correctly > >> > > > > > > > > > > > not being pruned as expected) which needs to be > >> resolved. > >> > > > > > > > > > > > > >> > > > > > > > > > > > > Anyone who wants to test the latest pekko 1.1.0 > >> preview > >> > > > can > >> > > > > > > track > >> > > > > > > > > down > >> > > > > > > > > > > > versions at > >> > > > > > > > > > > > The core point is while its possible for users to > >> > > manually > >> > > > > > track > >> > > > > > > > > > > snapshots > >> > > > > > > > > > > > and update them as new ones get released/old ones > >> get > >> > > > deleted > >> > > > > > > this > >> > > > > > > > > is not > >> > > > > > > > > > > > practical for the use case I mentioned earlier > >> where we > >> > > > want > >> > > > > to > >> > > > > > > set > >> > > > > > > > > the > >> > > > > > > > > > > > downstream Pekko modules versions to a more > >> frequently > >> > > > built > >> > > > > > non > >> > > > > > > > > release > >> > > > > > > > > > > > version so we can catch regressions and/or make > >> sure that > >> > > > > newly > >> > > > > > > > > merged > >> > > > > > > > > > > > features work as expected. > >> > > > > > > > > > > > > >> > > > > > > > > > > > If we use snapshots for this then we have the > >> overhead of > >> > > > > > having > >> > > > > > > to > >> > > > > > > > > > > > manually update snapshots versions when builds > >> break due > >> > > to > >> > > > > > > > > > > > snapshots not existing anymore (note that this > >> hasn't > >> > > been > >> > > > a > >> > > > > > > problem > >> > > > > > > > > for > >> > > > > > > > > > > now > >> > > > > > > > > > > > because the snapshot pruning is not working > >> correctly > >> > > which > >> > > > > is > >> > > > > > > the > >> > > > > > > > > > > problem > >> > > > > > > > > > > > I was referring to before). > >> > > > > > > > > > > > > >> > > > > > > > > > > > Furthermore if developers want to test newly > merged > >> > > > features > >> > > > > > into > >> > > > > > > > > > > > development/staging/prod systems to see that there > >> aren't > >> > > > > > > > > regressions, > >> > > > > > > > > > > > assuming the snapshots are working correctly > (which > >> they > >> > > > > > > currently > >> > > > > > > > > > > aren't) > >> > > > > > > > > > > > they can also hit the same issues regarding the > >> snapshots > >> > > > > being > >> > > > > > > > > pruned. > >> > > > > > > > > > > > > >> > > > > > > > > > > > At least to me, this is one of the core problems > >> that the > >> > > > > > > milestone > >> > > > > > > > > > > > concept is intended to solve, snapshots are too > >> > > > > > > frequent/granular and > >> > > > > > > > > > > > releases/release candidates are too infrequent. > >> > > > > > > > > > > > > >> > > > > > > > > > > > On Sun, Aug 6, 2023 at 2:23 PM PJ Fanning < > >> > > > > > [email protected]> > >> > > > > > > > > wrote: > >> > > > > > > > > > > > > >> > > > > > > > > > > > > Nobody is threatening to delete our recent > >> snapshots. > >> > > > There > >> > > > > > is > >> > > > > > > a > >> > > > > > > > > > > > > question about what to do about older snapshots > - > >> which > >> > > > is > >> > > > > > > separate > >> > > > > > > > > > > > > from this topic. > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > Anyone who wants to test the latest pekko 1.1.0 > >> preview > >> > > > can > >> > > > > > > track > >> > > > > > > > > down > >> > > > > > > > > > > > > versions at > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://repository.apache.org/content/groups/snapshots/org/apache/pekko/pekko-actor-typed_2.13/ > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > 1.1.0-M0+5-964dcf53-SNAPSHOT is the most recent > - > >> but > >> > > > there > >> > > > > > > will be > >> > > > > > > > > > > > > new snapshots published most nights (any day > that > >> has > >> > > > > commits > >> > > > > > > to > >> > > > > > > > > the > >> > > > > > > > > > > > > main branch). > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > On Sun, 6 Aug 2023 at 13:08, Matthew de Detrich > >> > > > > > > > > > > > > <[email protected]> wrote: > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > The question to ask is, will these artefacts > >> be > >> > > used > >> > > > by > >> > > > > > > users > >> > > > > > > > > or > >> > > > > > > > > > > > > intended > >> > > > > > > > > > > > > > for their use? i.e. people who are not > >> subscribed to > >> > > > the > >> > > > > > > mailing > >> > > > > > > > > > > list or > >> > > > > > > > > > > > > > not committers or PMC members? If so, then by > >> ASF > >> > > > > > definition, > >> > > > > > > > > that > >> > > > > > > > > > > > > artefact > >> > > > > > > > > > > > > > is a release and needs to be voted on by the > >> > > PPMC/IPMC. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > No they aren't, as I said they are treated in > >> the > >> > > exact > >> > > > > > same > >> > > > > > > way > >> > > > > > > > > > > > > snapshots > >> > > > > > > > > > > > > > are currently treated. The **ONLY** difference > >> is > >> > > that > >> > > > > they > >> > > > > > > will > >> > > > > > > > > be > >> > > > > > > > > > > > > > distributed to the Apache Nexus main > repository > >> and > >> > > > this > >> > > > > is > >> > > > > > > > > mainly > >> > > > > > > > > > > due to > >> > > > > > > > > > > > > > technical reasons stated before (i.e. > snapshots > >> are > >> > > > > > > > > semi-permanent > >> > > > > > > > > > > since > >> > > > > > > > > > > > > > they need to be pruned). > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > The primary motivation for the milestone > >> artifacts > >> > > are > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > * Internal testing between the Pekko > >> dependencies > >> > > (i.e. > >> > > > > > > > > publishing an > >> > > > > > > > > > > > > > artifact of Pekko core and using that artifact > >> in in > >> > > > > other > >> > > > > > > Pekko > >> > > > > > > > > > > modules > >> > > > > > > > > > > > > to > >> > > > > > > > > > > > > > catch regressions before a release is marked) > >> > > > > > > > > > > > > > * For Pekko developers to have the ability to > >> test > >> > > > their > >> > > > > > > features > >> > > > > > > > > > > which > >> > > > > > > > > > > > > > were merged into main in their production > >> systems > >> > > > without > >> > > > > > > having > >> > > > > > > > > to > >> > > > > > > > > > > > > > manually build Pekko (and possibly all of > >> Pekko's > >> > > > > > > dependencies) > >> > > > > > > > > > > > > themselves, > >> > > > > > > > > > > > > > which also as stated previously is quite > >> difficult > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > There is no intention of the milestones having > >> any > >> > > > formal > >> > > > > > > release > >> > > > > > > > > > > > > > announcement. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Have other incubating projects made non-ASF > >> > > releases > >> > > > > > while > >> > > > > > > in > >> > > > > > > > > > > > > incubation? > >> > > > > > > > > > > > > > In a small number of cases, yes, mostly while > >> they > >> > > were > >> > > > > > > getting > >> > > > > > > > > their > >> > > > > > > > > > > > > code > >> > > > > > > > > > > > > > base in order but not after they had made an > ASF > >> > > > release, > >> > > > > > and > >> > > > > > > > > there > >> > > > > > > > > > > we > >> > > > > > > > > > > > > very > >> > > > > > > > > > > > > > clearly labelled as non-ASF releases. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Yes and this is not meant to be a formal ASF > >> release, > >> > > > > hence > >> > > > > > > the > >> > > > > > > > > > > previous > >> > > > > > > > > > > > > > suggestion of making this clear even in the > >> > > DISCLAIMER > >> > > > > file > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Re official documentation/process that > >> describes > >> > > this > >> > > > > > > > > distinction? > >> > > > > > > > > > > Yes, > >> > > > > > > > > > > > > > that policy page sets that out. See, for > >> instance [1] > >> > > > and > >> > > > > > > [2] for > >> > > > > > > > > > > why it > >> > > > > > > > > > > > > > needs to be this way. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > [1] Is talking about source packages, this > >> discussion > >> > > > is > >> > > > > > > about > >> > > > > > > > > binary > >> > > > > > > > > > > > > > artifacts. In the last line they mention this > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Nightly Builds that are not release > >> candidates can > >> > > be > >> > > > > > > hosted at > >> > > > > > > > > > > > > > ci.apache.org projects area, just file an > INFRA > >> > > > ticket. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > The links to ci.apache.org aren't even > working > >> and I > >> > > > > doubt > >> > > > > > > we > >> > > > > > > > > can > >> > > > > > > > > > > even > >> > > > > > > > > > > > > use > >> > > > > > > > > > > > > > such a repository since we are dealing with > JVM > >> jar's > >> > > > > > > > > specifically > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Re official documentation/process that > >> describes > >> > > this > >> > > > > > > > > distinction? > >> > > > > > > > > > > Yes, > >> > > > > > > > > > > > > > that policy page sets that out. See, for > >> instance [1] > >> > > > and > >> > > > > > > [2] for > >> > > > > > > > > > > why it > >> > > > > > > > > > > > > > needs to be this way. The Incubator > distribution > >> > > > > guideline > >> > > > > > > also > >> > > > > > > > > > > covers > >> > > > > > > > > > > > > this > >> > > > > > > > > > > > > > [3]. You see that at [4] "Release candidates, > >> > > nightlys > >> > > > > and > >> > > > > > > > > snapshots > >> > > > > > > > > > > must > >> > > > > > > > > > > > > > not be advertised to the general public.” and > >> you can > >> > > > > read > >> > > > > > > [5] > >> > > > > > > > > for > >> > > > > > > > > > > why. > >> > > > > > > > > > > > > The > >> > > > > > > > > > > > > > release distribution policy also touches on > >> this e.g. > >> > > > [6] > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > As was clarified earlier, milestones are > **NOT** > >> > > > > releases, > >> > > > > > > they > >> > > > > > > > > are > >> > > > > > > > > > > > > treated > >> > > > > > > > > > > > > > the exact same way as snapshots, nightlies or > >> release > >> > > > > > > candidates > >> > > > > > > > > > > with the > >> > > > > > > > > > > > > > **ONLY** critical difference being that they > are > >> > > > deployed > >> > > > > > in > >> > > > > > > a > >> > > > > > > > > > > repository > >> > > > > > > > > > > > > > that is not snapshots and they are published > >> less > >> > > > > > frequently > >> > > > > > > > > then a > >> > > > > > > > > > > > > > snapshot/nightly is. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > With this being said I am increasingly of the > >> opinion > >> > > > > that > >> > > > > > > this > >> > > > > > > > > is > >> > > > > > > > > > > more > >> > > > > > > > > > > > > of > >> > > > > > > > > > > > > > a technical INFRA question than an ASF policy > >> > > question > >> > > > > > since > >> > > > > > > the > >> > > > > > > > > > > issues > >> > > > > > > > > > > > > > being discussed are technical in nature. There > >> was > >> > > > never > >> > > > > a > >> > > > > > > > > > > suggestion of > >> > > > > > > > > > > > > > making an alternative formal type of ASF > >> release. > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > On Sun, Aug 6, 2023 at 10:51 AM Justin Mclean > < > >> > > > > > > > > > > [email protected]> > >> > > > > > > > > > > > > > wrote: > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Hi, > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > The question to ask is, will these artefacts > >> be > >> > > used > >> > > > by > >> > > > > > > users > >> > > > > > > > > or > >> > > > > > > > > > > > > intended > >> > > > > > > > > > > > > > > for their use? i.e. people who are not > >> subscribed > >> > > to > >> > > > > the > >> > > > > > > > > mailing > >> > > > > > > > > > > list > >> > > > > > > > > > > > > or > >> > > > > > > > > > > > > > > not committers or PMC members? If so, then > by > >> ASF > >> > > > > > > definition, > >> > > > > > > > > that > >> > > > > > > > > > > > > artefact > >> > > > > > > > > > > > > > > is a release and needs to be voted on by the > >> > > > PPMC/IPMC. > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Have other incubating projects made non-ASF > >> > > releases > >> > > > > > while > >> > > > > > > in > >> > > > > > > > > > > > > incubation? > >> > > > > > > > > > > > > > > In a small number of cases, yes, mostly > while > >> they > >> > > > were > >> > > > > > > getting > >> > > > > > > > > > > their > >> > > > > > > > > > > > > code > >> > > > > > > > > > > > > > > base in order but not after they had made an > >> ASF > >> > > > > release, > >> > > > > > > and > >> > > > > > > > > > > there we > >> > > > > > > > > > > > > very > >> > > > > > > > > > > > > > > clearly labelled as non-ASF releases. > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Re official documentation/process that > >> describes > >> > > this > >> > > > > > > > > distinction? > >> > > > > > > > > > > Yes, > >> > > > > > > > > > > > > > > that policy page sets that out. See, for > >> instance > >> > > [1] > >> > > > > and > >> > > > > > > [2] > >> > > > > > > > > for > >> > > > > > > > > > > why > >> > > > > > > > > > > > > it > >> > > > > > > > > > > > > > > needs to be this way. The Incubator > >> distribution > >> > > > > > guideline > >> > > > > > > also > >> > > > > > > > > > > covers > >> > > > > > > > > > > > > this > >> > > > > > > > > > > > > > > [3]. You see that at [4] "Release > candidates, > >> > > > nightlys > >> > > > > > and > >> > > > > > > > > > > snapshots > >> > > > > > > > > > > > > must > >> > > > > > > > > > > > > > > not be advertised to the general public.” > and > >> you > >> > > can > >> > > > > > read > >> > > > > > > [5] > >> > > > > > > > > for > >> > > > > > > > > > > > > why. The > >> > > > > > > > > > > > > > > release distribution policy also touches on > >> this > >> > > e.g. > >> > > > > [6] > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > Kind Regards, > >> > > > > > > > > > > > > > > Justin > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > 1. > >> > > > > > > https://www.apache.org/legal/release-policy.html#host-rc > >> > > > > > > > > > > > > > > 2. > >> > > > > https://www.apache.org/legal/release-policy.html#why > >> > > > > > > > > > > > > > > 3. > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > https://cwiki.apache.org/confluence/display/INCUBATOR/Distribution+Guidelines > >> > > > > > > > > > > > > > > 4. > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > >> https://incubator.apache.org/guides/distribution.html#release_platforms > >> > > > > > > > > > > > > > > 5. > >> > > > > > > > > > > > >> > > > > > https://incubator.apache.org/guides/distribution.html#motivation > >> > > > > > > > > > > > > > > 6. > >> > > > > > > https://infra.apache.org/release-distribution.html#maven > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > >> > > > --------------------------------------------------------------------- > >> > > > > > > > > > > > > > > To unsubscribe, e-mail: > >> > > > > [email protected] > >> > > > > > > > > > > > > > > For additional commands, e-mail: > >> > > > > > [email protected] > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > -- > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Matthew de Detrich > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > *Aiven Deutschland GmbH* > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu > >> Valtonen > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > *m:* +491603708037 > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > >> > > > > > >> --------------------------------------------------------------------- > >> > > > > > > > > > > > > To unsubscribe, e-mail: > >> > > [email protected] > >> > > > > > > > > > > > > For additional commands, e-mail: > >> > > > [email protected] > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > -- > >> > > > > > > > > > > > > >> > > > > > > > > > > > Matthew de Detrich > >> > > > > > > > > > > > > >> > > > > > > > > > > > *Aiven Deutschland GmbH* > >> > > > > > > > > > > > > >> > > > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > > > > > > > > > >> > > > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > > > > > > > > > >> > > > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > > > > > > > > > > >> > > > > > > > > > > > *m:* +491603708037 > >> > > > > > > > > > > > > >> > > > > > > > > > > > *w:* aiven.io *e:* [email protected] > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > >> > > > --------------------------------------------------------------------- > >> > > > > > > > > > > To unsubscribe, e-mail: > >> [email protected] > >> > > > > > > > > > > For additional commands, e-mail: > >> [email protected] > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > -- > >> > > > > > > > > > > >> > > > > > > > > > Matthew de Detrich > >> > > > > > > > > > > >> > > > > > > > > > *Aiven Deutschland GmbH* > >> > > > > > > > > > > >> > > > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > > > > > > > >> > > > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > > > > > > > >> > > > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > > > > > > > > >> > > > > > > > > > *m:* +491603708037 > >> > > > > > > > > > > >> > > > > > > > > > *w:* aiven.io *e:* [email protected] > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > >> --------------------------------------------------------------------- > >> > > > > > > > > To unsubscribe, e-mail: > [email protected] > >> > > > > > > > > For additional commands, e-mail: > >> [email protected] > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > -- > >> > > > > > > > > >> > > > > > > > Matthew de Detrich > >> > > > > > > > > >> > > > > > > > *Aiven Deutschland GmbH* > >> > > > > > > > > >> > > > > > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > > > > > >> > > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > > > > > >> > > > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > > > > > > >> > > > > > > > *m:* +491603708037 > >> > > > > > > > > >> > > > > > > > *w:* aiven.io *e:* [email protected] > >> > > > > > > > >> > > > > > > > >> > > > --------------------------------------------------------------------- > >> > > > > > > To unsubscribe, e-mail: [email protected] > >> > > > > > > For additional commands, e-mail: [email protected] > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > -- > >> > > > > > > >> > > > > > Matthew de Detrich > >> > > > > > > >> > > > > > *Aiven Deutschland GmbH* > >> > > > > > > >> > > > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > > > >> > > > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > > > >> > > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > > > > >> > > > > > *m:* +491603708037 > >> > > > > > > >> > > > > > *w:* aiven.io *e:* [email protected] > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > -- > >> > > > > >> > > > Matthew de Detrich > >> > > > > >> > > > *Aiven Deutschland GmbH* > >> > > > > >> > > > Immanuelkirchstraße 26, 10405 Berlin > >> > > > > >> > > > Amtsgericht Charlottenburg, HRB 209739 B > >> > > > > >> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > > > >> > > > *m:* +491603708037 > >> > > > > >> > > > *w:* aiven.io *e:* [email protected] > >> > > > > >> > > > >> > > >> > > >> > -- > >> > > >> > Matthew de Detrich > >> > > >> > *Aiven Deutschland GmbH* > >> > > >> > Immanuelkirchstraße 26, 10405 Berlin > >> > > >> > Amtsgericht Charlottenburg, HRB 209739 B > >> > > >> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >> > > >> > *m:* +491603708037 > >> > > >> > *w:* aiven.io *e:* [email protected] > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* [email protected] > > > > > -- > > Matthew de Detrich > > *Aiven Deutschland GmbH* > > Immanuelkirchstraße 26, 10405 Berlin > > Amtsgericht Charlottenburg, HRB 209739 B > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > *m:* +491603708037 > > *w:* aiven.io *e:* [email protected] >
