Alex, There are two assumptions that deserve further discussion: 1) That tool has to be the last to build. why? it does not have more sense to build final images after everything else has been built?- 2) That the impact (in terms of effort now) on fixing CI is bigger than the impact (long term consequences) of consolidating two unrelated piece of software within the same repository.
On Wed, Mar 13, 2024 at 11:15 AM Alex Porcelli <a...@porcelli.me> wrote: > Francisco, > > This was discussed as an alternative solution, however it has major impact > on CI and there’s also the fact Tool has been always the last to build and > has no Snapshot published (actually in JavaScript world there is no > snapshot concept). > > So, based on our evaluation… the proposal here is the least disruptive and > will take less time to unblock the release. > > Regards, > _____________ > Alex Porcelli > http://porcelli.me > > > On Wed, Mar 13, 2024 at 6:09 AM Francisco Javier Tirado Sarti < > ftira...@redhat.com> wrote: > > > After kie-tools, sorry. I think we are not embracing the fact that > > kogito-images depend on kie-tools, because we want those images to > include > > tools. > > > > On Wed, Mar 13, 2024 at 11:08 AM Francisco Javier Tirado Sarti < > > ftira...@redhat.com> wrote: > > > > > Hi Tiago, > > > It can be an alternative solution to move kn-plugin-workflow to > > > kogito-images (so there is not longer dependency from tools to images) > > and > > > then build kogito-images after kogito-tools? > > > > > > On Wed, Mar 13, 2024 at 11:01 AM Enrique Gonzalez Martinez < > > > egonza...@apache.org> wrote: > > > > > >> +1 to unblock release > > >> > > >> El mié, 13 mar 2024, 10:48, Pere Fernandez (apache) < > > pefer...@apache.org> > > >> escribió: > > >> > > >> > I say +1 in order to move forward with the 10. > > >> > > > >> > On Tue, 12 Mar 2024 at 21:45, Alex Porcelli <a...@porcelli.me> > wrote: > > >> > > > >> > > +1 > > >> > > > > >> > > I spent the last day or so working closely with Tiago, exploring > > >> > different > > >> > > options and getting deeper on the impact and evaluating the > overall > > >> > release > > >> > > procedure steps required. I agree with the proposal as the most > > >> > > viable option for unblocking the 10 release in the reasonable time > > >> frame. > > >> > > > > >> > > > > >> > > On Tue, Mar 12, 2024 at 3:45 PM Tiago Bento < > tiagobe...@apache.org> > > >> > wrote: > > >> > > > > >> > > > Hi everyone, > > >> > > > > > >> > > > Unfortunately, I can't do a tl;dr this time, as this matter > > >> requires a > > >> > > > lot of context. > > >> > > > > > >> > > > This email will take you < 20 minutes to read, according to > > >> > > > https://thereadtime.com/. > > >> > > > > > >> > > > As you may have followed on a separate thread > > >> > > > ( > https://lists.apache.org/thread/nknm6j641qk2c7cl621tsy3fy98tsc69 > > ), > > >> > > > many of us were working towards removing a circular dependency > > >> > > > currently present between `kogito-apps` and `kie-tools`. As we > > >> > > > progressed towards a solution, we kept finding the circular > > >> dependency > > >> > > > pop up somewhere else. I'll do a breakdown of the things we did, > > and > > >> > > > the results we had. > > >> > > > > > >> > > > Right now, even though we started the effort to move the Quarkus > > Dev > > >> > > > UI modules to `kie-tools`, we haven't been able to do it yet, as > > >> we've > > >> > > > been busy upgrading KIE Tools to Java 17, Maven 3.9.6, and > Quarkus > > >> > > > 3.2.9, compatible with Kogito Runtimes 999-20240218-SNAPSHOT. > This > > >> > > > effort was concluded this Monday, with > > >> > > > https://github.com/apache/incubator-kie-tools/pull/2136. > > >> > > > > > >> > > > The current scenario we have is: > > >> > > > > > >> > > > 01. incubator-kie-kogito-runtimes > > >> > > > |==> 02. incubator-kie-kogito-apps > > >> > > > C | 03. incubator-kie-kogito-examples > > >> > > > Y | 04. incubator-kie-kogito-images > > >> > > > C | 05. incubator-kie-kogito-serverless-operator > > >> > > > L | ========================== > > >> > > > E | 06. incubator-kie-sandbox-quarkus-accelerator > > >> > > > |==> 07. incubator-kie-tools > > >> > > > > > >> > > > > > >> > > > * As `kie-tools`/extended-services depends on > > >> > > > `kogito-apps`/jitexecutor; > > >> > > > * and `kogito-apps`/{sonataflow,bpmn}-quarkus-devui > depend > > >> on > > >> > > > `kie-tools`/{many packages} > > >> > > > > > >> > > > > > >> > > > After moving the Quarkus Dev UIs to `kie-tools`, we would've > had: > > >> > > > > > >> > > > 01. incubator-kie-kogito-runtimes > > >> > > > 02. incubator-kie-kogito-apps > > >> > > > 03. incubator-kie-kogito-examples > > >> > > > C |==> 04. incubator-kie-kogito-images > > >> > > > Y | 05. incubator-kie-kogito-serverless-operator > > >> > > > C | ===================== > > >> > > > L | 06. incubator-kie-sandbox-quarkus-accelerator > > >> > > > E |==> 07. incubator-kie-tools > > >> > > > > > >> > > > * As `kie-tools`/kn-plugin-workflow depends on > > >> > > > `kogito-images`/kogito-swf-devmode; > > >> > > > * and `kogito-images`/kogito-swf-devmode depends on > > >> > > > `kie-tools`/sonataflow-quarkus-devui > > >> > > > > > >> > > > > > >> > > > After moving the `kogito-swf-devmode` image to `kie-tools`, we > > >> would've > > >> > > > had: > > >> > > > > > >> > > > 01. incubator-kie-kogito-runtimes > > >> > > > 02. incubator-kie-kogito-apps > > >> > > > 03. incubator-kie-kogito-examples > > >> > > > 04. incubator-kie-kogito-images > > >> > > > C |==> 05. incubator-kie-kogito-serverless-operator > > >> > > > Y | ===================== > > >> > > > C | 06. incubator-kie-sandbox-quarkus-accelerator > > >> > > > L |==> 07. incubator-kie-tools > > >> > > > E > > >> > > > > > >> > > > * As `kie-tools`/kn-plugin-workflow depends on > > >> > > > `kogito-serverless-operator`; > > >> > > > * and `kogito-serverless-operator` depends on > > >> > > > `kie-tools`/kogito-swf-devmode > > >> > > > > > >> > > > > > >> > > > Clearly, we have a much bigger problem than a simple circular > > >> > dependency. > > >> > > > > > >> > > > After multiple conversations with a lot of people, it's been > > really > > >> > > > hard coming up with a simple solution that makes it possible to > > >> build > > >> > > > Apache KIE in one shot, while preserving the way everyone is > used > > to > > >> > > > contributing to the multiple repositories we have. More than > that, > > >> > > > while making this assessment, I found more problems that, in my > > >> > > > perspective, block Apache KIE 10. > > >> > > > > > >> > > > In light of that difficulty, I'm coming forward with my proposal > > for > > >> > > > the Apache KIE release process, so we can use Apache's > mechanisms > > to > > >> > > > have a slower-paced, in-depth debate about this really > complicated > > >> > > > matter. > > >> > > > > > >> > > > I'll lay out my entire perspective about the current situation > of > > >> our > > >> > > > codebase, as well as problems I can currently see. I'll start > with > > >> an > > >> > > > analysis of the repositories and their purposes, point out some > > >> > > > problems that I believe are blocking our 10 release, explain my > > >> > > > proposal and discuss some consequences to what I'm proposing. > > >> > > > > > >> > > > Let's begin. > > >> > > > > > >> > > > > > >> > > > # THE APACHE KIE REPOS > > >> > > > > > >> > > > A. DROOLS OPTAPLANNER, & KOGITO (count: 11) > > >> > > > - incubator-kie-kogito-pipelines @ `main` > > >> > > > - incubator-kie-drools @ `main` > > >> > > > - incubator-kie-optaplanner @ `main` > > >> > > > - incubator-kie-optaplanner-quickstarts @ `main` > > >> > > > - incubator-kie-kogito-runtimes @ `main` > > >> > > > - incubator-kie-kogito-apps @ `main` > > >> > > > - incubator-kie-kogito-examples @ `main` > > >> > > > - incubator-kie-kogito-images @ `main` > > >> > > > - incubator-kie-kogito-serverless-operator @ `main` > > >> > > > - incubator-kie-kogito-docs @ `main` > > >> > > > - incubator-kie-docs @ `main-kogito` > > >> > > > > > >> > > > B. TOOLS (count: 2) > > >> > > > - incubator-kie-sandbox-quarkus-accelerator @ `0.0.0` > > >> > > > - incubator-kie-tools @ `main` > > >> > > > > > >> > > > C. BENCHMARKS (count: 2) > > >> > > > - incubator-kie-kogito-benchmarks @ `main` > > >> > > > - incubator-kie-benchmarks @ `main` > > >> > > > > > >> > > > D. ARCHIVED (count: 1) > > >> > > > - incubator-kie-kogito-operator > > >> > > > > > >> > > > E. "NON-CODE" (count: 5) > > >> > > > - incubator-kie-issues @ `main` > > >> > > > (Issues only, README should be updated @ `main`. Same for > > GitHub > > >> > > > Actions workflows.) > > >> > > > - incubator-kie-kogito-website @ `main` > > >> > > > (The Kogito website. Develop & deploy at the `main` branch.) > > >> > > > - incubator-kie-website @ `main` > > >> > > > (The KIE website. Develop @ `main`. Push @ `deploy` to > update > > >> the > > >> > > > website.) > > >> > > > - incubator-kie-kogito-online @ `gh-pages` > > >> > > > (GitHub pages used to host sandbox.kie.org and KIE Tools' > > >> Chrome > > >> > > > Extension assets.) > > >> > > > - incubator-kie-kogito-online-staging @ `main` > > >> > > > (Same as above, but for manual sanity checks during the > > staging > > >> > > > phase of a release.) > > >> > > > > > >> > > > TOTAL (count: 21) > > >> > > > > > >> > > > I grouped the repositories by category, and listed them in a > > >> > > > topological order. Keep in mind that when flattening out a tree, > > >> there > > >> > > > are multiple possibilities. For example, OptaPlanner could've > been > > >> > > > placed in any position after Drools. > > >> > > > > > >> > > > Category A repos are what I've been referring to as `drools` and > > >> > > > `kogito-*` stream. Of course OptaPlanner is inside that stream, > as > > >> the > > >> > > > way these repositories reference each other are through Maven > > >> > > > SNAPSHOTs. More specifically, the 999-SNAPSHOT version. This > > >> mechanism > > >> > > > is well-known to the team, and although flawed for intra-day > > builds > > >> > > > and disruptive for people in many different time zones, it is > > >> already > > >> > > > very comfortable for everyone to work with, I assume. > > >> > > > > > >> > > > Contributions made to Category A have some dedicated pipelines, > > >> which > > >> > > > are, at least to some extent, able to build cross-repo PRs > > together > > >> > > > and verify that the codebase will continue working as expected > > after > > >> > > > they're all merged. From what I could gather, there are some > > >> > > > "sub-streams" currently configured for cross-repo PRs. > > >> > > > > > >> > > > - kogito-pipelines > > >> > > > - drools, kogito-runtimes, kogito-apps, and kogito-examples > > >> > > > - optaplanner, and optaplanner-quickstarts > > >> > > > - kogito-images, and kogito-serverless-operator > > >> > > > - kogito-docs > > >> > > > - kie-docs > > >> > > > > > >> > > > This means that sending cross-repo PRs to any combination of > repos > > >> > > > that are not part of the same "sub-stream" cannot be verified > > before > > >> > > > merging, making our contribution model dependent on individual > > >> > > > contributors building stuff on their machines to verify that it > > >> works. > > >> > > > > > >> > > > I based this analysis on > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-kogito-pipelines/blob/main/.ci/project-dependencies.yaml > > >> > > > , > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-optaplanner/blob/main/.ci/buildchain-project-dependencies.yaml > > >> > > > , > > >> > > > and > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-kogito-pipelines/blob/main/.ci/jenkins/config/branch.yaml > > >> > > > . > > >> > > > Note that I'm not that familiar with these pipelines, so please > > >> > > > someone correct me if I'm wrong. > > >> > > > > > >> > > > Category B repos are what I've been referring to as `kie-tools` > > >> > > > stream. The first repo there is a template repository that is > used > > >> by > > >> > > > people starting projects from scratch on KIE Sandbox, similar > to a > > >> > > > Maven archetype, if you will. The other one is the KIE Tools > > >> monorepo, > > >> > > > a polyglot monorepo with `pnpm` as its build system. Currently, > > KIE > > >> > > > Tools hosts Java libraries and apps, TypeScript libraries and > > apps, > > >> Go > > >> > > > apps, Docker images, and Helm charts. The `kie-tools` monorepo > is > > >> > > > configured to work with sparse checkouts and can do partial > > builds. > > >> > > > Category B repos refer to Category A repos through timestamped > > >> > > > SNAPSHOTs. This is a new mechanism we recently introduced that > > will > > >> > > > build and publish immutable, persistent artifacts under a > version > > >> > > > following the 999-YYYYMMDD-SNAPSHOT format, published weekly > every > > >> > > > Sunday night. Timestamped SNAPSHOTs are an evolution to the > Kogito > > >> > > > releases, as we're now targeting one release for all of Apache > > KIE, > > >> so > > >> > > > we can't have Kogito releases anymore. > > >> > > > > > >> > > > An important note here is that Category B repositories have been > > >> > > > historically kept out of any automations we used to have, way > back > > >> > > > when Kogito started and we had the Business Central (a.k.a. v7) > > >> stream > > >> > > > still going on. For this reason, Category B projects have > > developed > > >> > > > their own automations, based on GitHub Actions. Category B repos > > >> have > > >> > > > always depended on Category A repos using fixed versions. If > > >> Category > > >> > > > B repos have had adopted mutable SNAPSHOTs, breaking changes on > > >> > > > Category A repositories would've had the potential to break > > >> Category B > > >> > > > silently, leaving Category B with a broken development stream, > and > > >> > > > introducing unpleasant surprises for maintainers of Category B > > >> repos, > > >> > > > as historically Category A contributors were not familiar with > > >> > > > Category B repos. > > >> > > > > > >> > > > Contributions made to Category B repos go through a GitHub > Actions > > >> > > > workflow that builds the relevant part of the `kie-tools` > monorepo > > >> for > > >> > > > the changes introduced. Changes made to the pipeline itself are > > also > > >> > > > picked up as part of PRs, allowing us to do things like > atomically > > >> > > > bumping the Node.js version, for example. More importantly, it > > >> allows > > >> > > > us to upgrade the repository to a new timestamped SNAPSHOT > > together > > >> > > > with the changes necessary to make it stay green. > > >> > > > > > >> > > > This setup, however, makes it impossible to have cross-repo PRs > > >> > > > involving Category A and Category B simultaneously, with the > > current > > >> > > > automations we have. > > >> > > > > > >> > > > Category C repos are kind of floating around, and I'm not sure > if > > >> > > > there's much activity going on there. Regardless, as they're > part > > of > > >> > > > Apache KIE, they will be part of our release, so I listed them > for > > >> us > > >> > > > to take them into consideration too. > > >> > > > > > >> > > > Category D is self explanatory. There's only one repo that has > > >> already > > >> > > > been marked for being archived. > > >> > > > > > >> > > > Category E are repos that do not host code directly, and are > > either > > >> > > > organizational entities, or host websites, that currently are > not > > >> part > > >> > > > of any pipelines we have. > > >> > > > > > >> > > > This lack of unification between Category A and Category B is, > > IMHO, > > >> > > > what allowed us to introduce the infamous circular dependency > > >> between > > >> > > > `kie-tools` and `kogito-apps`, which we now can describe as a > > >> circular > > >> > > > dependency between Category A and Category B. The way I see it, > if > > >> we > > >> > > > had a single pipeline, building everything from `drools` to > > >> > > > `kie-tools`, such flaws would've never been introduced, and we > > >> > > > wouldn't be having this huge problem in our hands right now. > > >> > > > > > >> > > > My proposal for the Apache KIE release process sees this lack of > > >> > > > unification as a central problem, not only for this release in > > >> > > > particular, but for the community as a whole. It is my belief > that > > >> we > > >> > > > are all under the same roof, and that no contribution should be > > >> > > > allowed to break any part of our codebase. With the increasing > > >> volume > > >> > > > of code, and hopefully number of contributors too, we cannot > keep > > >> > > > counting on "common sense" to avoid breaking things. We're all > > >> humans > > >> > > > after all, and it is our job to have mechanisms in place to > > prevent > > >> us > > >> > > > from unwillingly making mistakes. Especially when these mistakes > > >> > > > impact on parts of the codebase that we, individually, probably > > >> can't > > >> > > > fix. > > >> > > > > > >> > > > > > >> > > > # THE PROBLEMS WE HAVE RIGHT NOW > > >> > > > > > >> > > > P1. Quarkus Dev UIs @ `kogito-apps` depending on kiegroup's KIE > > >> Tools > > >> > > > `0.32.0`. > > >> > > > See: > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/search?q=repo%3Akiegroup%2Fkogito-apps+path%3Apackage.json+kie-tools&type=code > > >> > > > > > >> > > > > > >> > > > P2. PR open for Kogito SWF images @ `kogito-images` depending on > > >> > > > kiegroup's KIE Tools `0.32.0`. > > >> > > > See: > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-tools/tree/main/packages/sonataflow-deployment-webapp > > >> > > > > > >> > > > > > >> > > > P3. DashBuilder @ `kie-tools` depending on kiegroup's `lienzo` > and > > >> > > > `kie-soup` artifacts at version `7.59.0.Final`. > > >> > > > See: > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-tools/blob/main/packages/dashbuilder/pom.xml#L64 > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/search?q=repo%3Aapache%2Fincubator-kie-tools+path%3Apackages%2Fdashbuilder+%24%7Bversion.org.kie%7D&type=code > > >> > > > > > >> > > > > > >> > > > P4. Multiple packages @ `kogito-apps` depending on kiegroup's > > >> > > > Explainability `1.22.1.Final`. > > >> > > > * This module was removed from the KIE codebase here: > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-kogito-apps/commit/bbb22c06d37e77b97aae6496d74abe43a8cfc965 > > >> > > > and now lives on > > >> > > > > > https://github.com/trustyai-explainability/trustyai-explainability, > > >> > > > under a different GAV. > > >> > > > * This new repo depends on Kogito and OptaPlanner, pointing to > > older > > >> > > > versions. > > >> > > > See: > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/search?q=repo%3Aapache%2Fincubator-kie-kogito-apps+%3Eexplainability-core%3C&type=code > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/trustyai-explainability/trustyai-explainability/blob/main/pom.xml#L52-L53 > > >> > > > > > >> > > > > > >> > > > P5. `incubator-kie-sandbox-quarkus-accelerator` depending on > > Kogito > > >> > > > `1.32.0.Final` and Quarkus `2.15.3.Final`. > > >> > > > See: > > >> > > > - > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-sandbox-quarkus-accelerator/blob/0.0.0/pom.xml#L32-L38 > > >> > > > > > >> > > > > > >> > > > P6. Category C repos are out of date and not part of the > Category > > A > > >> > > > CI/Release pipelines. > > >> > > > * incubator-kie-kogito-benchmarks: (Current version is > > >> `2.0-SNAPSHOT`, > > >> > > > depending on Kogito without a specific version, only by using > > >> > > > `http://localhost:8080`) > > >> > > > * incubator-kie-benchmarks: (Current version is `1.0-SNAPSHOT`, > > >> > > > pointing to Drools 999-SNAPSHOT and OptaPlanner > `8.45.0-SNAPSHOT`) > > >> > > > > > >> > > > > > >> > > > P7. `kie-tools`/packages/kn-plugin-workflow has its E2E disabled > > >> after > > >> > > > upgrading to 999-20240218-SNAPSHOT. > > >> > > > > > >> > > > > > >> > > > In my perspective, P1 and P2 have the same solution, as they > both > > >> > > > suffer from the circular dependency between Category A and > > Category > > >> B. > > >> > > > As Category A and Category B are both streams that have been > > really > > >> > > > active, I see this as a blocker, as there are contributions that > > >> > > > cannot be done, given that Category A depends on Category B > with a > > >> > > > dephasing of 1 release. > > >> > > > > > >> > > > P3 and P4, although not ideal, can be understood as technical > > debt. > > >> > > > Depending on unmaintained projects is something we'll always be > > >> > > > susceptible to, given time. > > >> > > > > > >> > > > P5 and P6 are easily fixable, as it's just a matter of making > them > > >> > > > part of the play. > > >> > > > > > >> > > > P7 is an isolated problem that won't impact the structure or > > >> anything > > >> > > > that we're talking about here, but it is a regression we > > introduced > > >> > > > recently. > > >> > > > > > >> > > > Assuming P3 and P4 can be ignored for Apache KIE 10, and that > P5, > > >> P6, > > >> > > > and P7 have easy fixes, the only problems left to discuss are P1 > > and > > >> > > > P2, which can't be done without a proper proposal. > > >> > > > > > >> > > > > > >> > > > # THE PROPOSAL > > >> > > > > > >> > > > I'll try to be very meticulous here, since from my experience, > any > > >> > > > little miscalculation can lead to our release not working out in > > the > > >> > > > end. To try and avoid that as much as possible, and make > > everything > > >> we > > >> > > > can to have a successful Apache KIE 10 release, bear with me. > I'll > > >> lay > > >> > > > out a timeline of events that need to happen in order for our > > >> release > > >> > > > to be published, with all artifacts ending up in the right > places, > > >> but > > >> > > > first, we need to solve problems P1 and P2. > > >> > > > > > >> > > > As you saw at the beginning of this email, all the attempts we > > made > > >> > > > left us with the circular dependency showing up at a different > > >> place, > > >> > > > but something all these places have in common is that they're > all > > >> > > > after kogito-apps, and before to Category B. > > >> > > > > > >> > > > The first part of my proposal is the following: > > >> > > > > > >> > > > S1. We keep the original plan of moving the Quarkus Dev UIs from > > >> > > > `kogito-apps` to `kie-tools`, together with Management and Task > > >> > > > consoles from `kogito-images` to `kie-tools`. > > >> > > > S2. We move the `kogito-swf-devmode` and `kogito-swf-builder` > > images > > >> > > > from `kogito-images` to `kie-tools` too. > > >> > > > S3. We move the entire `kogito-serverless-operator` repo inside > a > > >> new > > >> > > > package on `kie-tools`, keeping Git history. > > >> > > > > > >> > > > Solutions S1, S2, and S3 together solve problems P1 and P2. Of > > >> course > > >> > > > the rest of > > >> https://github.com/apache/incubator-kie-issues/issues/967 > > >> > > > would still be done too. > > >> > > > > > >> > > > This doesn't come without consequences, of course, as the > > >> > > > `kogito-swf-devmode` and `kogito-swf-builder` images, and the > > >> > > > `kogito-serverless-operator` would be moving from Category A to > > >> > > > Category B. This move would make them have to reference > Category A > > >> > > > repos through timestamped SNAPSHOTs. Since `kogito-images` and > > >> > > > `kogito-serverless-operator` are already their own "sub-stream" > > >> inside > > >> > > > Category A, though, contributions made in a cross-repo fashion > to > > >> this > > >> > > > "sub-stream" will continue being possible, now via a single PR > to > > >> > > > `kie-tools`. Cross-repo PRs between Category A and Category B > will > > >> > > > continue not being possible, and a 1-week delay between merging > > >> > > > something on Category A and using it on Category B will still > > >> happen. > > >> > > > > > >> > > > It's worth mentioning that `kie-tools`, however, does allow for > > >> sparse > > >> > > > checkouts and partial builds, so working with a subset of the > > >> monorepo > > >> > > > is possible and encouraged. Making changes only to > > >> > > > `packages/kn-plugin-workflow`, for example, will have the PR > > checks > > >> > > > run in < 10 minutes, as you can see here: > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > https://github.com/apache/incubator-kie-tools/actions/runs/8237244382/job/22525511722?pr=2136 > > >> > > > . > > >> > > > We're not compromising when running partial builds too. We know > > that > > >> > > > the entire repo will continue working even after only building a > > >> small > > >> > > > subset of the changes. Doing partial or full builds is > > automatically > > >> > > > determined by the changes of a PR. > > >> > > > > > >> > > > Keep in mind that, even though I'm proposing we move a bunch of > > >> > > > additional stuff into `kie-tools`, I see this as a TEMPORARY > > >> solution > > >> > > > for our codebase. `kie-tools` would host some additional stuff > > >> > > > TEMPORARILY so that we can release and continue moving forward. > > >> > > > > > >> > > > As I mentioned on other places, `kie-tools` became a polyglot > > >> monorepo > > >> > > > out of necessity, and although I'm really proud of what we > > achieved > > >> > > > there so far, I don't think `kie-tools` has a setup that is > > suitable > > >> > > > for all the different nuances that compose our community. I'm > well > > >> > > > aware that a polyglot monorepo that does not follow widespread > > >> > > > conventions will scare some people away, and as much as we've > > tried > > >> to > > >> > > > make build instructions clear, we can't always get past the > > >> prejudice > > >> > > > some people have towards the "front-end" ecosystem. > > >> > > > > > >> > > > With all that said, I keep thinking this is the best course of > > >> action > > >> > > > for us right now. We keep most of our stuff unchanged, we > unblock > > >> the > > >> > > > release, and we have a working setup that will suit us well > while > > we > > >> > > > discuss and reach a conclusion regarding the future of our > > codebase > > >> > > > structure. > > >> > > > > > >> > > > Let me paint a quick picture here of what our code base would > look > > >> > > > like, repository-wise, if my proposal is accepted: > > >> > > > > > >> > > > CATEGORY REPO > > >> > > > ===================== > > >> > > > A incubator-kie-kogito-pipelines > > >> > > > A incubator-kie-drools > > >> > > > A incubator-kie-optaplanner > > >> > > > A incubator-kie-optaplanner-quickstarts > > >> > > > A incubator-kie-kogito-runtimes > > >> > > > A incubator-kie-kogito-apps > > >> > > > A incubator-kie-kogito-examples > > >> > > > A incubator-kie-kogito-images > > >> > > > A incubator-kie-kogito-docs > > >> > > > A incubator-kie-kogito-benchmarks > > >> > > > A incubator-kie-docs > > >> > > > A incubator-kie-benchmarks > > >> > > > ===================== > > >> > > > B incubator-kie-sandbox-quarkus-accelerator > > >> > > > B incubator-kie-tools > > >> > > > ===================== > > >> > > > D incubator-kie-kogito-operator > > >> > > > ===================== > > >> > > > E incubator-kie-issues > > >> > > > E incubator-kie-kogito-website > > >> > > > E incubator-kie-website > > >> > > > E incubator-kie-kogito-online > > >> > > > E incubator-kie-kogito-online-staging > > >> > > > ===================== > > >> > > > > > >> > > > * Category C becomes part of Category A, and > > >> > > > `kogito-serverless-operator` moves entirely inside `kie-tools`. > > >> > > > * With `kogito-swf-{builder,devmode}` images and > > >> > > > `kogito-serverless-operator` inside `kie-tools`, there are no > > cycles > > >> > > > anymore, as inside `kie-tools`, we can granularly build: > > >> > > > 1. packages/sonataflow-deployment-webapp > > >> > > > 2. packages/sonataflow-quarkus-devui > > >> > > > 3. packages/sonataflow-images (containing > `kogito-swf-builder` > > >> and > > >> > > > `kogito-swf-devmode`) > > >> > > > 4. packages/sonataflow-operator (contents from > > >> > > > `kogito-serverless-operator`) > > >> > > > 5. packages/kn-plugin-sonataflow > > (`packages/kn-plugin-workflow`, > > >> > > > but renamed) > > >> > > > > > >> > > > The second part of the proposal is the release process itself, > > >> > > > assuming the structure above is what we have. > > >> > > > > > >> > > > Here it is: > > >> > > > > > >> > > > 1. Define a timestamped SNAPSHOT to be used as cutting point for > > >> > > > Category A repos. > > >> > > > 2. Update Category B repos to point to this timestamped > SNAPSHOT, > > >> and > > >> > > > verify that everything is working. > > >> > > > 3. At this point, with everything working, we can branch out to > > >> > > > `10.0.x`. Category A from the timestamped SNAPSHOT tag, and > > >> Category B > > >> > > > from `main`. > > >> > > > 4. All Category A and Category B repos update their versions to > > >> > > > 10.0.0, in their `10.0.x` branches. > > >> > > > 5. Update Category B repos to point to Category A repos using > the > > >> > > > 10.0.0 version. > > >> > > > 6. At this point, we can vote on the release based on the > `10.0.x` > > >> > > > branches, given we don't expect any code changes anymore. > > >> > > > 7. After voting passes, we're good to start the release process. > > >> > > > 8. Category A repos follow their manual/automated release > process, > > >> > > > pointing to the `10.0.x` branch. Tags pushed to Git, and built > > >> > > > artifacts pushed to their registries. > > >> > > > 9. We wait a little bit for Category A artifacts to be > propagated > > on > > >> > > > registries. ~1 day. > > >> > > > 10. Category B repos follow their manual/automated release > > process, > > >> > > > pointing to the `10.0.x` branch. Tags pushed to Git, and built > > >> > > > artifacts pushed to their registries. > > >> > > > 11. Category D repos are ignored. > > >> > > > 12. Category E repos can be manually tagged with 10.0.0 from > their > > >> > > > default branches. > > >> > > > > > >> > > > More needs to be discussed if we're planning to maintain > multiple > > >> > > > release streams in parallel, but I guess it can wait for after > > >> Apache > > >> > > > KIE 10. > > >> > > > > > >> > > > Thank you for reading, and I'm looking forward to hearing back > > from > > >> > > > everyone. > > >> > > > > > >> > > > Of course, alternative solutions are possible. This email, > > however, > > >> > > > summarizes my view of how we should attack the problem, > > considering > > >> > > > disruption, required effort, the release process itself, and > > >> history. > > >> > > > Feel free to propose alternatives. This is not a voting thread. > > >> > > > > > >> > > > Regards, > > >> > > > > > >> > > > Tiago Bento > > >> > > > > > >> > > > > > >> --------------------------------------------------------------------- > > >> > > > To unsubscribe, e-mail: dev-unsubscr...@kie.apache.org > > >> > > > For additional commands, e-mail: dev-h...@kie.apache.org > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
