What I'm trying to say is that changing the streams is probably better than doing again what we did with tooling and data-index in the past (in kogito-apps repo), putting unrelated stuff into the same repo (image and tooling in this case)
On Thu, Mar 7, 2024 at 11:29 AM Francisco Javier Tirado Sarti < [email protected]> wrote: > So, is it possible to have the following build order (regardless of kie or > kogito prefix)? > 1) drools 2) runtimes 3) tools 4) images > > > > On Thu, Mar 7, 2024 at 10:51 AM Francisco Javier Tirado Sarti < > [email protected]> wrote: > >> Hi Tiago, >> Let's rewind a bit, if we ignore the kie and kogito name controversy it >> makes perfect sense that the final image that is delivered for users >> (whatever the name) includes tooling (whatever the name). >> I think we need to assume that fact and devise a solution that honors it. >> >> >> On Wed, Mar 6, 2024 at 6:50 PM Tiago Bento <[email protected]> wrote: >> >>> Dominik and Ricardo, >>> >>> I understand your concern with removing features from users. Too much >>> has grown around the circular dependency we had between `kogito-apps` >>> and `kie-tools`. We're fixing it now, and it is a challenging task. >>> >>> Rather than stating the "need" of `kogito-images` depending on >>> `kie-tools`, let's keep the focus on the end result. IMHO it is >>> irrelevant on which repo something is hosted, as long as we can do >>> atomic releases and kick-off our release train with Apache KIE 10. >>> That means defining a clear topology of repos during an Apache KIE >>> build, which, by definition, can't contain cycles. >>> >>> Now, `kogito-images` either is part of the `drools` and `kogito-*` >>> streams, or it's not. >>> >>> This PR [1] >>> https://github.com/apache/incubator-kie-kogito-images/pull/1734 >>> introduces a cycle, even without the changes proposed by me on the >>> first email of this thread, so merging this PR would be introducing a >>> blocker for the Apache KIE 10 release (and for future releases too). >>> >>> If we really wanted to have `kogito-images` depending on `kie-tools`, >>> though, we would need to: >>> >>> 1. Make sure `kie-tools` doesn't depend on `kogito-images`; >>> 2. Create a third development stream, just for `kogito-images`, >>> leaving a buffer for `kogito-images` to be updated with the latest >>> `kie-tools` version during a release. Analogous to what we do with >>> `kie-tools` and the `drools` and `kogito-*` stream; >>> 3. Create a mechanism to do timestamped SNAPSHOT releases of >>> `kie-tools`, analogous to what we have on the `drools` and `kogito-*` >>> streams now. >>> >>> This, IMHO, introduces a lot more complexity to our already convoluted >>> release process, so I wouldn't go on that route. >>> >>> With all that said, and to be clear regarding the topology of the >>> repository, to successfully build and release Apache KIE 10, we need >>> to take all 22 repositories [1] we have in Apache right now, and sort >>> them in a way that we can build them one by one, only once, without >>> referencing older versions of any 1st-party >>> package/module/image/artifact. >>> >>> We can't think about our repos in the same way we used to. KIE Tools >>> won't have a release published before nor after a Drools or Kogito >>> Runtimes release. Everything will be released at the same time. That's >>> the mindset shift we all need to get used to if we want to do atomic >>> releases of Apache KIE. >>> >>> I hope it is clearer now why we can't have `kogito-images` depending >>> on `kie-tools` if `kogito-images` continues to be part of the `drools` >>> and `kogito-*` development streams. >>> >>> The way I see it, Dominik came to this mail thread with an important >>> and valid concern, which now we need to solve together, as a >>> community. >>> >>> I'll keep my suggestions of either moving the image from >>> `kogito-images` to `kie-tools`, or removing the dependency of SWF >>> Quarkus Dev UI from it. Of course, the PR [1] can't be merged, as it >>> will introduce a dependency cycle, so I'm -1'ing that PR too, given >>> the blocker it will introduce to the Apache KIE 10 release. >>> >>> Is this image being used anywhere else? Or is `kn-plugin-workflow` the >>> only place using it? Why does it need to stay at `kogito-images` and >>> can't go to `kie-tools`? Same questions for the image being introduced >>> by [1]. >>> >>> Let me know if I'm missing something, but I feel like our best course >>> of action now would be moving the images to `kie-tools`. >>> >>> Regards, >>> >>> Tiago Bento >>> >>> >>> [1] >>> https://github.com/orgs/apache/repositories?language=&q=incubator-kie&sort=&type=all >>> >>> >>> On Wed, Mar 6, 2024 at 9:59 AM Alex Porcelli <[email protected]> wrote: >>> > >>> > Ricardo, >>> > >>> > Here's a better description [1] of the dependency that KIE Tools has >>> > with the images. If we have kogito images moved to after KIE Tools, I >>> > think we'd create a new circular dependency. >>> > >>> > [1] - >>> https://github.com/apache/incubator-kie-issues/issues/821#issuecomment-1896206199 >>> > >>> > On Wed, Mar 6, 2024 at 9:43 AM ricardo zanini fernandes >>> > <[email protected]> wrote: >>> > > >>> > > Alex, >>> > > >>> > > I'm not aware of any dependencies in KIE Tools for images. >>> > > >>> > > On Wed, Mar 6, 2024 at 9:44 AM Alex Porcelli <[email protected]> >>> wrote: >>> > > >>> > > > Ricardo, >>> > > > >>> > > > IIRC, there’s a dependency in KIE Tools for serverless workflow >>> with >>> > > > images… if this is correct, your proposal will introduce a new >>> circular >>> > > > dependency. >>> > > > >>> > > > >>> > > > On Wed, Mar 6, 2024 at 7:39 AM ricardo zanini fernandes < >>> > > > [email protected]> wrote: >>> > > > >>> > > > > Hi! >>> > > > > >>> > > > > A strong -1 from my side moving the images to kie-tools. We can >>> release >>> > > > the >>> > > > > images after a kie-tools release or patch the images with a new >>> kie-tools >>> > > > > release if that comes out. It's ok to depend on it, I just need >>> help to >>> > > > > setup the pipelines and CI. >>> > > > > >>> > > > > So, when releasing SonatFlow images, we can grab the latest >>> available >>> > > > > kie-tools version, pack it and ship it. If a new release comes >>> up and we >>> > > > > need the fix, we can of course do a patch release. >>> > > > > >>> > > > > We just need to automate the process. >>> > > > > >>> > > > > Cheers! >>> > > > > >>> > > > > On Wed, Mar 6, 2024 at 9:33 AM Alex Porcelli < >>> [email protected]> >>> > > > wrote: >>> > > > > >>> > > > > > Dominik, >>> > > > > > >>> > > > > > The plans and course of action for removing the circular >>> dependency has >>> > > > > > been exhaustively discussed in the mailing list and Zulip >>> channel among >>> > > > > > community and key stakeholders, I don’t think we have room to >>> revisit >>> > > > the >>> > > > > > decision that has been already agreed. >>> > > > > > >>> > > > > > With that being said, I don’t think the current plans removes >>> any >>> > > > > > functionality, we are just moving things from one repository to >>> > > > another. >>> > > > > > >>> > > > > > >>> > > > > > On Wed, Mar 6, 2024 at 2:19 AM Dominik Hanak < >>> [email protected]> >>> > > > > > wrote: >>> > > > > > >>> > > > > > > Hi Tiago, >>> > > > > > > >>> > > > > > > that is a problem, we are removing an important >>> functionality for our >>> > > > > SWF >>> > > > > > > users, imho it is a killer feature in this case. >>> > > > > > > I am against it, there needs to be a solution where we can >>> depend on >>> > > > > > > kie-tools with kie-kogito-images. >>> > > > > > > >>> > > > > > > Another thing that is currently open as a PR [2] that makes >>> images >>> > > > > depend >>> > > > > > > on kie-tools. Here we are following >>> > > > > > > the approach you started in kiegroup and we were not aware >>> of any >>> > > > > > changes, >>> > > > > > > now with the unexpected and sudden changes it is irrelevant? >>> > > > > > > Do you have a proposal on how to achieve what we have in [2] >>> after >>> > > > the >>> > > > > > > circular dependency issue is fixed according to the proposal >>> > > > > annoucement? >>> > > > > > > >>> > > > > > > Dominik >>> > > > > > > >>> > > > > > > >>> > > > > > > [2] >>> https://github.com/apache/incubator-kie-kogito-images/pull/1734 >>> > > > > > > >>> > > > > > > >>> > > > > > > On Wed, 6 Mar 2024 at 02:10, Tiago Bento < >>> > > > > [email protected]> >>> > > > > > > wrote: >>> > > > > > > >>> > > > > > > > Hi Dominik, >>> > > > > > > > >>> > > > > > > > That is indeed an oversight on our part. Thanks for >>> pointing that >>> > > > > out. >>> > > > > > > > >>> > > > > > > > Unfortunately, given the two development streams (`drools` >>> and >>> > > > > > > > `kogito-*`, and `kie-tools`), we can't have `kogito-images` >>> > > > depending >>> > > > > > > > on packages that will start being hosted on `kie-tools`, >>> since >>> > > > > > > > `kogito-images` is part of the `drools` and `kogito-*` >>> development >>> > > > > > > > stream, which is upstream from `kie-tools`' perspective. >>> > > > > > > > >>> > > > > > > > Since all Dev UI packages are moving to `kie-tools`, >>> images on >>> > > > > > > > `kogito-images` can't depend on them. >>> > > > > > > > >>> > > > > > > > I'm not familiar with the structure of `kogito-images`, >>> nor the >>> > > > > > > > purpose of this image that uses SWF's Quarkus Dev UI, but >>> following >>> > > > > > > > the same reasoning we did for `kogito-management-console` >>> and >>> > > > > > > > `kogito-task-console`, my suggestion would be moving this >>> image to >>> > > > > > > > `kie-tools`. Another option I can see is simply removing >>> the >>> > > > Quarkus >>> > > > > > > > Dev UI dependency on this image. >>> > > > > > > > >>> > > > > > > > I'd wait for other opinions, though, as I'm not the best >>> person to >>> > > > > > > > answer SWF-related questions. >>> > > > > > > > >>> > > > > > > > Regards, >>> > > > > > > > >>> > > > > > > > Tiago Bento >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > On Tue, Mar 5, 2024 at 9:31 AM Dominik Hanak < >>> > > > [email protected]> >>> > > > > > > > wrote: >>> > > > > > > > > >>> > > > > > > > > Hi Tiago, >>> > > > > > > > > >>> > > > > > > > > Thanks for moving this forward. >>> > > > > > > > > >>> > > > > > > > > I have one thing to highlight with >>> > > > > > > > > https://github.com/apache/incubator-kie-kogito-images >>> > > > > > > > > I noticed [1], where the swf devui is being used in an >>> image. >>> > > > Imho >>> > > > > > that >>> > > > > > > > > would require adjustments to the nightly pipelines & >>> release >>> > > > > process. >>> > > > > > > > > Would it be possible to add tasks to the plan that ensure >>> > > > required >>> > > > > > > > > adjustments to our CI once the migration is done? >>> > > > > > > > > >>> > > > > > > > > Best regards, >>> > > > > > > > > >>> > > > > > > > > Dominik >>> > > > > > > > > >>> > > > > > > > > [1] >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> https://github.com/apache/incubator-kie-kogito-images/blob/ce62b9d86c2ea0385ab85c3ef5339afe5ecdea72/modules/kogito-swf/devmode/build-config/module.yaml#L30 >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > On Fri, 1 Mar 2024 at 05:54, Tiago Bento < >>> [email protected]> >>> > > > > > > wrote: >>> > > > > > > > > >>> > > > > > > > > > Hi everyone, >>> > > > > > > > > > >>> > > > > > > > > > Following up on the status of this effort. >>> > > > > > > > > > >>> > > > > > > > > > I know I'm a bit late on the email, but it paid off, >>> as we just >>> > > > > had >>> > > > > > > > > > our first green build [1] of KIE Tools with Java 17, >>> Quarkus >>> > > > > 3.2.9, >>> > > > > > > > > > Maven 3.9.6 and Kogito 999-20240218-SNAPSHOT. We ended >>> up >>> > > > > deciding >>> > > > > > to >>> > > > > > > > > > do items 1. and 2. together after all, but we couldn't >>> use the >>> > > > > > latest >>> > > > > > > > > > timestamped SNAPSHOT due to >>> > > > > > > > > > https://issues.apache.org/jira/browse/INFRA-25546, >>> which >>> > > > > prevented >>> > > > > > > the >>> > > > > > > > > > publication of the timestamped SNAPSHOT to succeed. >>> That's why >>> > > > we >>> > > > > > > > > > picked 20240218, not 20240225. >>> > > > > > > > > > >>> > > > > > > > > > We will proceed with some manual checks now to confirm >>> that >>> > > > > > > everything >>> > > > > > > > > > is working properly, and I hope on the next status >>> update we >>> > > > will >>> > > > > > > have >>> > > > > > > > > > the big blocker removed and that we have already made >>> some >>> > > > > progress >>> > > > > > > on >>> > > > > > > > > > items other than 1. and 2.. >>> > > > > > > > > > >>> > > > > > > > > > It's been great seeing everyone working together to >>> move this >>> > > > > > forward >>> > > > > > > > > > as fast as possible. Thanks a lot Eder, Thiago, >>> Rodrigo, Alex, >>> > > > > > > > > > Fabrizio, Paulo, and Yeser! >>> > > > > > > > > > >>> > > > > > > > > > [1] >>> > > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> https://github.com/apache/incubator-kie-tools/actions/runs/8105498131/job/22153897965 >>> > > > > > > > > > >>> > > > > > > > > > On Mon, Feb 26, 2024 at 5:59 AM Francisco Javier >>> Tirado Sarti >>> > > > > > > > > > <[email protected]> wrote: >>> > > > > > > > > > > >>> > > > > > > > > > > Hi Tiago, >>> > > > > > > > > > > It was a nice morning reading ;) Thanks for all the >>> time >>> > > > > invested >>> > > > > > > on >>> > > > > > > > such >>> > > > > > > > > > > detailed writing. >>> > > > > > > > > > > @Ricardo Zanini <[email protected]> @Walter Medvedeo >>> > > > > > > > > > > <[email protected]> I guess dev ui can be >>> removed >>> > > > > from >>> > > > > > > > current >>> > > > > > > > > > > examples, at least from the ones I'm familiar with, >>> since >>> > > > their >>> > > > > > > > README >>> > > > > > > > > > > files do not really require any graphical tool. But >>> to be >>> > > > > honest, >>> > > > > > > > since I >>> > > > > > > > > > > have focused my work on SWF simple showcase >>> examples, I'm >>> > > > not >>> > > > > > 100% >>> > > > > > > > sure >>> > > > > > > > > > > that's the same case for more elaborate ones, can >>> you please >>> > > > > > check >>> > > > > > > > that >>> > > > > > > > > > it >>> > > > > > > > > > > is fine to remove dev ui? If this is not the case, >>> maybe >>> > > > those >>> > > > > > > > examples >>> > > > > > > > > > can >>> > > > > > > > > > > be moved to the tools repository (points 9 and 10 of >>> Tiago's >>> > > > > > email) >>> > > > > > > > > > > Thanks in advance. >>> > > > > > > > > > > >>> > > > > > > > > > > On Fri, Feb 23, 2024 at 2:29 AM Tiago Bento < >>> > > > > > [email protected] >>> > > > > > > > >>> > > > > > > > > > wrote: >>> > > > > > > > > > > >>> > > > > > > > > > > > DISCLAIMER: this is a long email. Feel free to >>> jump to the >>> > > > > > bottom >>> > > > > > > > for >>> > > > > > > > > > > > the concrete task list. >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > This email aims to explain the issue and lay out >>> the >>> > > > efforts >>> > > > > > that >>> > > > > > > > are >>> > > > > > > > > > > > in place to fix it. >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > Before we start digging into the problem, we need >>> to >>> > > > > > understand a >>> > > > > > > > > > > > little bit of the history of the KIE community and >>> the >>> > > > Kogito >>> > > > > > > > > > > > initiative, at least from the part I started >>> contributing >>> > > > to >>> > > > > it >>> > > > > > > > anyway >>> > > > > > > > > > > > :) >>> > > > > > > > > > > > >>> > > > > > > > > > > > Back in Business Central days, we had a single >>> stream of >>> > > > > > > > development. >>> > > > > > > > > > > > We had robust CI jobs that would build the entire >>> stack, >>> > > > from >>> > > > > > > > `drools` >>> > > > > > > > > > > > to `kie-wb-distributions`. When Kogito came about, >>> there >>> > > > was >>> > > > > a >>> > > > > > > > fission >>> > > > > > > > > > > > in our pipelines. Kogito started to release >>> independently >>> > > > > from >>> > > > > > > > > > > > everything else, and because of the organic way >>> things >>> > > > > started >>> > > > > > to >>> > > > > > > > > > > > organize themselves, the Kogito Tooling initiative >>> needed >>> > > > to >>> > > > > > have >>> > > > > > > > its >>> > > > > > > > > > > > own release cycle as well. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Up until we started the migration to Apache, >>> Business >>> > > > Central >>> > > > > > > > > > > > continued on its `7.x` stream, Kogito was at `1.x` >>> stream, >>> > > > > and >>> > > > > > > > Kogito >>> > > > > > > > > > > > Tooling (which was already called KIE Tools) was >>> still on >>> > > > > > `0.x`. >>> > > > > > > > > > > > Drools started its own release stream as well, >>> with `8.x`, >>> > > > > and >>> > > > > > > > > > > > OptaPlanner I believe had jumped to `9.x` already. >>> This >>> > > > > > > > fragmentation >>> > > > > > > > > > > > had technical implications, which we are trying to >>> overcome >>> > > > > > right >>> > > > > > > > now >>> > > > > > > > > > > > with Apache KIE 10. >>> > > > > > > > > > > > >>> > > > > > > > > > > > The Apache KIE 10 release is aimed to be an atomic >>> release, >>> > > > > > where >>> > > > > > > > the >>> > > > > > > > > > > > entire Apache KIE community follows a single >>> release >>> > > > stream. >>> > > > > > This >>> > > > > > > > > > > > means we need to make sure our code base is in >>> sync, from >>> > > > > > > `drools` >>> > > > > > > > to >>> > > > > > > > > > > > `kie-tools`. >>> > > > > > > > > > > > >>> > > > > > > > > > > > This whole conversation of course doesn't come >>> without a >>> > > > > > > > background of >>> > > > > > > > > > > > CI pipelines, organizational perspectives of what >>> Apache >>> > > > KIE >>> > > > > > is, >>> > > > > > > > and >>> > > > > > > > > > > > even poly- vs. mono-repo discussions. This is not >>> the focus >>> > > > > of >>> > > > > > > this >>> > > > > > > > > > > > issue, and I believe with time we'll converge into >>> > > > something >>> > > > > > > > everyone >>> > > > > > > > > > > > is comfortable with and that ultimately serves our >>> users >>> > > > > well. >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > Due to the lack of unification and flexibility >>> while >>> > > > > > > bootstrapping >>> > > > > > > > new >>> > > > > > > > > > > > efforts, KIE Tools had to come up with a way to >>> depend on >>> > > > the >>> > > > > > new >>> > > > > > > > > > > > stuff (i.e. Kogito) without sacrificing its own >>> integrity >>> > > > and >>> > > > > > > > > > > > stability. KIE Tools, as many of you know, evolved >>> to be a >>> > > > > > > polyglot >>> > > > > > > > > > > > monorepo, hosting Java, JavaScript/TypeScript, Go >>> and >>> > > > Docker >>> > > > > > > > images. >>> > > > > > > > > > > > There's a multitude of technologies at play on KIE >>> Tools, >>> > > > > which >>> > > > > > > > > > > > ultimately gives our users KIE Sandbox, VS Code >>> and Chrome >>> > > > > > > > Extensions, >>> > > > > > > > > > > > and Standalone Editors that can be used as regular >>> > > > libraries >>> > > > > on >>> > > > > > > web >>> > > > > > > > > > > > applications. >>> > > > > > > > > > > > >>> > > > > > > > > > > > The technical solution we found on `kie-tools` was >>> > > > depending >>> > > > > on >>> > > > > > > > Kogito >>> > > > > > > > > > > > as if it were a 3rd party dependency. We have a >>> single, >>> > > > fixed >>> > > > > > > > version >>> > > > > > > > > > > > of it and we manually upgrade it to the latest one >>> > > > available >>> > > > > > when >>> > > > > > > > we >>> > > > > > > > > > > > need to. This has worked amazingly well, allowing >>> us to own >>> > > > > the >>> > > > > > > > > > > > codebase and avoid random breaks coming from >>> SNAPSHOT >>> > > > > > artifacts. >>> > > > > > > > > > > > >>> > > > > > > > > > > > This allowed the `drools` and `kogito-*` >>> repositories to >>> > > > > evolve >>> > > > > > > > > > > > independently of `kie-tools`. This also made >>> `kie-tools` >>> > > > > > > > responsible >>> > > > > > > > > > > > for adapting itself to changes coming from >>> `drools` and >>> > > > > > > `kogito-*`. >>> > > > > > > > > > > > Luckily, we didn't have many incompatibilities, >>> but right >>> > > > now >>> > > > > > we >>> > > > > > > > have >>> > > > > > > > > > > > an important one, that we'll discuss below. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Note that at this point, and for this issue in >>> particular, >>> > > > it >>> > > > > > is >>> > > > > > > > > > > > unimportant to discuss why things were done the >>> way they >>> > > > > were, >>> > > > > > or >>> > > > > > > > what >>> > > > > > > > > > > > we should've done instead. I'm laying out the >>> facts so that >>> > > > > we >>> > > > > > > can >>> > > > > > > > > > > > understand how we're moving forward now. Of >>> course, I would >>> > > > > > > > personally >>> > > > > > > > > > > > love to debate this on a separate thread :) >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > One of the most impactful features of KIE Sandbox >>> is the >>> > > > DMN >>> > > > > > > > Runner. >>> > > > > > > > > > > > It allows people to rapidly interact with their >>> Decisions >>> > > > as >>> > > > > > they >>> > > > > > > > > > > > author them, and this was all made possible >>> because of the >>> > > > > JIT >>> > > > > > > > > > > > Executor (hosted at `kogito-apps`). KIE Sandbox >>> also >>> > > > started >>> > > > > to >>> > > > > > > > allow >>> > > > > > > > > > > > users to deploy their Decisions to OpenShift, then >>> later to >>> > > > > > > > > > > > Kubernetes, and now recently we allow users to >>> deploy >>> > > > > anything >>> > > > > > to >>> > > > > > > > > > > > OpenShift and Kubernetes, with a very high level of >>> > > > > > > customization. >>> > > > > > > > All >>> > > > > > > > > > > > these features depend on `kogito-*`, Quarkus, and >>> all the >>> > > > > > things >>> > > > > > > > that >>> > > > > > > > > > > > come with them. Including the Java version. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Logically, `kie-tools` depends on `drools` and >>> `kogito-*`, >>> > > > as >>> > > > > > > > `drools` >>> > > > > > > > > > > > and `kogito-*` can (and do!) exist and function >>> without >>> > > > > Tools, >>> > > > > > > but >>> > > > > > > > not >>> > > > > > > > > > > > the other way around. >>> > > > > > > > > > > > >>> > > > > > > > > > > > As `kie-tools` started growing, maturing, and >>> releasing >>> > > > some >>> > > > > > > > pieces of >>> > > > > > > > > > > > technology that were useful to other domains (i.e. >>> > > > > Multiplying >>> > > > > > > > > > > > Architecture), it became a dependency for other >>> projects. >>> > > > > Users >>> > > > > > > > > > > > started depending on our Standalone Editors, but >>> also on >>> > > > the >>> > > > > > > > > > > > Multiplying Architecture packages. One good >>> example is >>> > > > Kaoto, >>> > > > > > > which >>> > > > > > > > > > > > released a VS Code extension aligned with our >>> architecture. >>> > > > > > > > > > > > >>> > > > > > > > > > > > The capability of encapsulating web components in >>> any >>> > > > > > technology >>> > > > > > > > > > > > behind a well defined, statically typed API was >>> really >>> > > > > > appealing, >>> > > > > > > > and >>> > > > > > > > > > > > when the Kogito initiative started to design their >>> Runtime >>> > > > > > Tools >>> > > > > > > > > > > > (i.e., Quarkus Dev UIs, Management Console, and >>> Task >>> > > > Inbox), >>> > > > > > > using >>> > > > > > > > the >>> > > > > > > > > > > > Multiplying Architecture was the way to go. >>> > > > > > > > > > > > >>> > > > > > > > > > > > That's when the circular dependency started. We had >>> > > > > > > > > > > > `kogito-apps/ui-packages` depending on >>> `kie-tools`. Note >>> > > > that >>> > > > > > it >>> > > > > > > > was >>> > > > > > > > > > > > also logical for these packages to be part of the >>> > > > `kogito-*` >>> > > > > > > > > > > > repositories and release stream. Users upgrading >>> to a new >>> > > > > > version >>> > > > > > > > of >>> > > > > > > > > > > > Kogito would also need to have their Task Inbox, >>> Management >>> > > > > > > > Console, >>> > > > > > > > > > > > and Quarkus Dev UIs aligned with the new version >>> of Kogito. >>> > > > > Not >>> > > > > > > to >>> > > > > > > > > > > > mention Job Service, Data Index etc. >>> > > > > > > > > > > > >>> > > > > > > > > > > > For many months (or years!), this was not an >>> issue, as the >>> > > > > > > > Multiplying >>> > > > > > > > > > > > Architecture packages were very stable and very >>> rarely were >>> > > > > > > > modified. >>> > > > > > > > > > > > The last big change we had was 3 years ago, with >>> the >>> > > > > > introduction >>> > > > > > > > of >>> > > > > > > > > > > > Shared Values [1], which did not impact the >>> fire-and-forget >>> > > > > and >>> > > > > > > > > > > > Promise-based mechanisms we already had in place. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Although we were aware of this circular dependency >>> from the >>> > > > > > > > beginning, >>> > > > > > > > > > > > it was not that big of a deal for `kogito-apps` to >>> depend >>> > > > on >>> > > > > an >>> > > > > > > > older, >>> > > > > > > > > > > > dephased version of `kie-tools`. We didn't have >>> plans of >>> > > > > > unifying >>> > > > > > > > the >>> > > > > > > > > > > > streams anyway, and it was the best solution for >>> our users >>> > > > to >>> > > > > > > have >>> > > > > > > > > > > > their Runtime Tools packages released together >>> with the >>> > > > > Kogito >>> > > > > > > > stream. >>> > > > > > > > > > > > >>> > > > > > > > > > > > That of course changed after the migration to >>> Apache. >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > If you reached this point, thank you for your >>> interest in a >>> > > > > > > little >>> > > > > > > > bit >>> > > > > > > > > > > > of the history of KIE, and I hope you can now >>> understand >>> > > > why >>> > > > > > > things >>> > > > > > > > > > > > are the way they are. Let's talk about what >>> changed and >>> > > > what >>> > > > > we >>> > > > > > > are >>> > > > > > > > > > > > doing to adapt to the new requirements. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Hinting a little bit to what I mentioned above, >>> this >>> > > > > discussion >>> > > > > > > is >>> > > > > > > > > > > > very intertwined with the CI pipelines and >>> > > > > versioning/releasing >>> > > > > > > > > > > > strategies subjects, as we are essentially >>> discussing code >>> > > > > > > > > > > > organization, coupling/decoupling and granularity >>> of stuff, >>> > > > > and >>> > > > > > > to >>> > > > > > > > > > > > make things easier at least for a while, `drools` >>> and >>> > > > > > `kogito-*` >>> > > > > > > > will >>> > > > > > > > > > > > continue being a separate stream from `kie-tools`. >>> > > > > > > > > > > > >>> > > > > > > > > > > > `drools` and `kogito-*` will continue being able >>> to evolve >>> > > > > > > > > > > > independently of `kie-tools`, and `kie-tools` will >>> continue >>> > > > > > > > adapting >>> > > > > > > > > > > > itself to whatever changes on `drools` and >>> `kogito-*`. >>> > > > > > > > > > > > >>> > > > > > > > > > > > At the same time, Apache KIE 10 (and the next >>> releases >>> > > > too!) >>> > > > > > will >>> > > > > > > > be >>> > > > > > > > > > > > atomic. And for our users, a single release stream. >>> > > > > `drools@10 >>> > > > > > ` >>> > > > > > > > and >>> > > > > > > > > > > > `kogito-*@10` will be used on `kie-tools@10`. We >>> started >>> > > > > this >>> > > > > > > > effort a >>> > > > > > > > > > > > while ago, and although we should've been more >>> open about >>> > > > it >>> > > > > > from >>> > > > > > > > the >>> > > > > > > > > > > > beginning, we are also learning how to properly >>> communicate >>> > > > > > > inside >>> > > > > > > > the >>> > > > > > > > > > > > Apache structure. Better late than never :) Thanks >>> for the >>> > > > > tip >>> > > > > > > > Tibor! >>> > > > > > > > > > > > >>> > > > > > > > > > > > Here's how we'll do it: >>> > > > > > > > > > > > >>> > > > > > > > > > > > 1. A couple of weeks ago, one of our newest >>> committers. >>> > > > > Rodrigo >>> > > > > > > > > > > > Antunes, created automated jobs to weekly publish a >>> > > > > timestamped >>> > > > > > > > > > > > SNAPSHOT version of `drools` and `kogito-*`. This >>> will >>> > > > allow >>> > > > > us >>> > > > > > > to >>> > > > > > > > > > > > have a persistent, immutable version to point to on >>> > > > > > `kie-tools`, >>> > > > > > > > even >>> > > > > > > > > > > > without having a full release. Timestamped >>> SNAPSHOTs, as >>> > > > > we're >>> > > > > > > > calling >>> > > > > > > > > > > > them, are a little bit different than normal Maven >>> > > > SNAPSHOTs, >>> > > > > > as >>> > > > > > > > there >>> > > > > > > > > > > > is no `updatePolicy`, so as time passes, the >>> downloaded >>> > > > > > artifacts >>> > > > > > > > do >>> > > > > > > > > > > > not change. This allows `kie-tools` to be stable >>> and >>> > > > > > incorporate >>> > > > > > > > > > > > changes done to `drools` and `kogito-*` at its own >>> pace. We >>> > > > > > also >>> > > > > > > > moved >>> > > > > > > > > > > > the publication of `@kie-tools/jitexecutor-native` >>> to this >>> > > > > new >>> > > > > > > > job, so >>> > > > > > > > > > > > we won't have to maintain it on the `kie-tools` >>> side >>> > > > anymore. >>> > > > > > > > Thanks >>> > > > > > > > > > > > Rodrigo! >>> > > > > > > > > > > > >>> > > > > > > > > > > > 2. When we define the code freeze prior to >>> releasing Apache >>> > > > > > KIE, >>> > > > > > > > > > > > `drools` and `kogito-*` will need to be frozen a >>> little bit >>> > > > > > > earlier >>> > > > > > > > > > > > than `kie-tools`, leaving a buffer for `kie-tools` >>> to start >>> > > > > > > > pointing >>> > > > > > > > > > > > to the latest timestamped SNAPSHOT and adapting >>> itself to >>> > > > the >>> > > > > > > > changes. >>> > > > > > > > > > > > My initial thoughts are around 2 weeks, but with >>> constant >>> > > > > > updates >>> > > > > > > > and >>> > > > > > > > > > > > proper alignment between the two streams, I see >>> this time >>> > > > > > > reducing >>> > > > > > > > to >>> > > > > > > > > > > > a couple of days for upcoming releases. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 3. After `kie-tools` is fully functional pointing >>> to the >>> > > > > latest >>> > > > > > > > > > > > timestamped SNAPSHOT of `drools` and `kogito-*`, >>> we can now >>> > > > > > start >>> > > > > > > > > > > > preparing for the actual release. We will cut new >>> branches >>> > > > > for >>> > > > > > > the >>> > > > > > > > > > > > next version (i.e. 10.0.0) and update all >>> configuration >>> > > > files >>> > > > > > > that >>> > > > > > > > we >>> > > > > > > > > > > > have to. This will be done on `drools`, >>> `kogito-*`, and >>> > > > > > > > `kie-tools`. >>> > > > > > > > > > > > Every module/package will declare the same >>> version, and >>> > > > will >>> > > > > > > > depend on >>> > > > > > > > > > > > the upstream modules/packages with the same >>> version as >>> > > > well. >>> > > > > We >>> > > > > > > can >>> > > > > > > > > > > > proceed with this staging phase as normal. Running >>> > > > > > > automated/manual >>> > > > > > > > > > > > tests and eventually signing off that everything >>> is good to >>> > > > > go. >>> > > > > > > > (I'm >>> > > > > > > > > > > > not 100% on the timeline for the Apache release, >>> but with >>> > > > > this >>> > > > > > > > process >>> > > > > > > > > > > > we can guarantee that we will vote on branches >>> that won't >>> > > > > > change >>> > > > > > > > when >>> > > > > > > > > > > > we decide to release. It will be a matter of >>> pushing tags >>> > > > > > > pointing >>> > > > > > > > to >>> > > > > > > > > > > > these release branches). >>> > > > > > > > > > > > >>> > > > > > > > > > > > With the release process defined, we can now >>> discuss the >>> > > > > exact >>> > > > > > > > actions >>> > > > > > > > > > > > we're taking to remove the circular dependency we >>> have >>> > > > > between >>> > > > > > > > > > > > `kie-tools` and `kogito-apps`. All of which will >>> need to be >>> > > > > > > > addressed >>> > > > > > > > > > > > prior to starting the release process. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 1. (Fabrizio - IN PROGRESS) Make sure `kie-tools` >>> is on >>> > > > Java >>> > > > > > 17, >>> > > > > > > > Maven >>> > > > > > > > > > > > 3.9.6, and Quarkus 3. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/960 >>> > > > ) >>> > > > > > > > > > > > There's a complication here, since we still >>> have three >>> > > > > big >>> > > > > > > > > > > > packages (Serverless Workflow Diagram Editor, >>> BPMN, DMN, >>> > > > and >>> > > > > > > SceSim >>> > > > > > > > > > > > Editors, and DashBuilder) on `kie-tools` that need >>> GWT, and >>> > > > > as >>> > > > > > > you >>> > > > > > > > > > > > know, GWT together with Java upgrades usually >>> require some >>> > > > > > work. >>> > > > > > > > We're >>> > > > > > > > > > > > on GWT 2.10, which should already support Java 17, >>> but >>> > > > we'll >>> > > > > > know >>> > > > > > > > more >>> > > > > > > > > > > > once we try it. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 2. (Fabrizio - NOT YET STARTED) Make sure >>> `kie-tools` is >>> > > > > using >>> > > > > > > the >>> > > > > > > > > > > > latest timestamped SNAPSHOT from `kogito-*`. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/965 >>> > > > ) >>> > > > > > > > > > > > We have them built and published every Sunday >>> EOD, so >>> > > > on >>> > > > > > > Monday >>> > > > > > > > > > > > (Feb 25th, 2024) we'll have a brand new one to >>> start >>> > > > working >>> > > > > > > with. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 3. (Thiago - IN PROGRESS) Move >>> `kogito-apps/ui-packages` to >>> > > > > > > > > > > > `kie-tools`. ( >>> > > > > > > > > > >>> https://github.com/apache/incubator-kie-issues/issues/833) >>> > > > > > > > > > > > We started a couple of weeks ago, and we >>> should be ~80% >>> > > > > > done >>> > > > > > > > with >>> > > > > > > > > > it >>> > > > > > > > > > > > by now. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 4. (Fabrizio - PR SENT, but blocked by 1. and 2.) >>> Move the >>> > > > > SWF >>> > > > > > > > Quarkus >>> > > > > > > > > > > > Dev UI package from `kogito-apps` to `kie-tools`. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-tools/pull/2167) >>> > > > > > > > > > > > This can only be done after 1. and 2. are >>> merged, since >>> > > > > > this >>> > > > > > > > move >>> > > > > > > > > > > > will require `kie-tools` to be ready to build >>> packages with >>> > > > > > Java >>> > > > > > > > 17, >>> > > > > > > > > > > > Maven 3.9.6, and Quarkus 3. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 5. (Thiago - NOT YET STARTED) Move the BPMN >>> Quarkus Dev UI >>> > > > > > > package >>> > > > > > > > > > > > from `kogito-apps` to `kie-tools`. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/966 >>> > > > ) >>> > > > > > > > > > > > We'll follow the lead of Fabrizio's PR from 4. >>> That's >>> > > > > > > > essentially >>> > > > > > > > > > > > doing exactly the same thing for the BPMN Quarkus >>> Dev UI >>> > > > > > module. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 6. (Rodrigo - NOT YET STARTED) Change the Jenkins >>> release >>> > > > > jobs >>> > > > > > to >>> > > > > > > > > > > > include publishing both Quarkus Dev UI modules to >>> Maven >>> > > > > central >>> > > > > > > as >>> > > > > > > > > > > > part of the Apache release. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/964 >>> > > > ) >>> > > > > > > > > > > > Should be pretty straight-forward, as we >>> already have >>> > > > > > > > experience >>> > > > > > > > > > > > doing this for other repos. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 7. (Thiago - NOT YET STARTED) Change the ownership >>> of >>> > > > > > > > `task-console` >>> > > > > > > > > > > > and `management-console` images from >>> `kogito-images` to >>> > > > > > > > `kie-tools`. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/963 >>> > > > ) >>> > > > > > > > > > > > KIE Tools already hosts and publishes several >>> images, >>> > > > and >>> > > > > > > since >>> > > > > > > > > > > > the content of those images will now be hosted at >>> > > > > `kie-tools`, >>> > > > > > we >>> > > > > > > > can >>> > > > > > > > > > > > only host them there. >>> > > > > > > > > > > > This also means we'll get rid of the two >>> Quarkus apps >>> > > > [2] >>> > > > > > > that >>> > > > > > > > > > > > wrap both consoles' static assets. We could be >>> wrong about >>> > > > > the >>> > > > > > > > purpose >>> > > > > > > > > > > > of those apps, so if anyone has more information, >>> please do >>> > > > > > reach >>> > > > > > > > out. >>> > > > > > > > > > > > Otherwise, we'll skip the Quarkusification of >>> those static >>> > > > > > assets >>> > > > > > > > and >>> > > > > > > > > > > > we'll ship images with `httpd` directly, like we >>> do for KIE >>> > > > > > > > Sandbox. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 8. (Assignee TDB - NOT YET STARTED) Deleting >>> everything >>> > > > that >>> > > > > > was >>> > > > > > > > moved >>> > > > > > > > > > > > to `kie-tools` from `kogito-apps`. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/962 >>> > > > ) >>> > > > > > > > > > > > That will be the point where we actually get >>> rid of the >>> > > > > > > > circular >>> > > > > > > > > > > > dependency and remove the blocker for the release. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 9. (Assignee TDB - NOT YET STARTED) Adapt >>> `kogito-examples` >>> > > > > to >>> > > > > > > all >>> > > > > > > > the >>> > > > > > > > > > > > changes we mentioned above. >>> > > > > > > > > > > > ( >>> https://github.com/apache/incubator-kie-issues/issues/961 >>> > > > ) >>> > > > > > > > > > > > Kogito Examples keeps being part of the >>> `drools` and >>> > > > > > > `kogito-*` >>> > > > > > > > > > > > stream, meaning that example modules won't be able >>> to use >>> > > > > > neither >>> > > > > > > > of >>> > > > > > > > > > > > the Quarkus Dev UIs, as `kie-tools` is downstream >>> in >>> > > > relation >>> > > > > > to >>> > > > > > > > the >>> > > > > > > > > > > > `drools` and `kogito-*` stream. >>> > > > > > > > > > > > The same would be true for Management Console >>> and Task >>> > > > > > > Console >>> > > > > > > > > > > > images, but for them, we can use the `daily-dev` >>> tag we >>> > > > > publish >>> > > > > > > > daily >>> > > > > > > > > > > > from `kie-tools@main`. >>> > > > > > > > > > > > >>> > > > > > > > > > > > 10. (Assignee TDB - NOT YET STARTED) Create some >>> example >>> > > > apps >>> > > > > > > that >>> > > > > > > > use >>> > > > > > > > > > > > the Quarkus Dev UIs directly on the `examples` >>> directory of >>> > > > > > > > > > > > `kie-tools`. ( >>> > > > > > > > > > >>> https://github.com/apache/incubator-kie-issues/issues/959) >>> > > > > > > > > > > > This is to serve the same purpose of packages >>> that did >>> > > > > that >>> > > > > > > on >>> > > > > > > > > > > > `kogito-examples`, but now hosted on `kie-tools`. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Pheew! That's a lot of things to do, but it is the >>> result >>> > > > of >>> > > > > > two >>> > > > > > > > > > > > intense weeks assessing and planning how we would >>> address >>> > > > > this >>> > > > > > > > issue >>> > > > > > > > > > > > in a way that the Apache KIE 10 release can be >>> done in a >>> > > > way >>> > > > > > that >>> > > > > > > > > > > > feels atomic to our users, who are ultimately the >>> most >>> > > > > > important >>> > > > > > > > part >>> > > > > > > > > > > > of the community. There's an open EPIC too at >>> > > > > > > > > > > > >>> https://github.com/apache/incubator-kie-issues/issues/967. >>> > > > > > > > > > > > >>> > > > > > > > > > > > I appreciate it if you reached this point, and >>> hope you >>> > > > feel >>> > > > > > > > welcome >>> > > > > > > > > > > > to contributing to solving this lengthy and >>> historical >>> > > > > problem >>> > > > > > if >>> > > > > > > > you >>> > > > > > > > > > > > feel like doing so. >>> > > > > > > > > > > > >>> > > > > > > > > > > > I'll personally send updates to this thread every >>> week, on >>> > > > > > > Thursday >>> > > > > > > > > > > > nights. Do not hesitate to contact me or any of >>> the people >>> > > > > > > > involved in >>> > > > > > > > > > > > this effort if you have questions, or simply want >>> to chat >>> > > > > about >>> > > > > > > it. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Thanks a lot Alex, Paulo, Fabrizio, Thiago, >>> Rodrigo, and >>> > > > > others >>> > > > > > > > that >>> > > > > > > > > > > > somehow helped so far. >>> > > > > > > > > > > > >>> > > > > > > > > > > > Kind regards, >>> > > > > > > > > > > > >>> > > > > > > > > > > > Tiago Bento >>> > > > > > > > > > > > >>> > > > > > > > > > > > --- >>> > > > > > > > > > > > >>> > > > > > > > > > > > P.S.: You might've noticed we left Trusty out of >>> the >>> > > > picture >>> > > > > > for >>> > > > > > > > now. >>> > > > > > > > > > > > Thiago is conducting an investigation to >>> understand what >>> > > > > would >>> > > > > > be >>> > > > > > > > the >>> > > > > > > > > > > > best course of action for it. As much as we have >>> plans to >>> > > > > > revive >>> > > > > > > > it, >>> > > > > > > > > > > > right now our focus has to be on getting Apache >>> KIE 10 out. >>> > > > > We >>> > > > > > > can >>> > > > > > > > > > > > always rely on Git to bring it back in the future, >>> if we >>> > > > > > > ultimately >>> > > > > > > > > > > > decide that wiping out Trusty is the best strategy >>> for this >>> > > > > > > moment. >>> > > > > > > > > > > > >>> > > > > > > > > > > > [1] >>> https://github.com/apache/incubator-kie-tools/pull/691 >>> > > > > > > > > > > > >>> > > > > > > > > > > > [2] >>> > > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> https://github.com/apache/incubator-kie-kogito-apps/tree/main/task-console >>> > > > > > > > > > > > and >>> > > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> https://github.com/apache/incubator-kie-kogito-apps/tree/main/management-console >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > >>> > > > >>> --------------------------------------------------------------------- >>> > > > > > > > > > > > To unsubscribe, e-mail: >>> [email protected] >>> > > > > > > > > > > > For additional commands, e-mail: >>> [email protected] >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > >>> --------------------------------------------------------------------- >>> > > > > > > > > > To unsubscribe, e-mail: [email protected] >>> > > > > > > > > > For additional commands, e-mail: >>> [email protected] >>> > > > > > > > > > >>> > > > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > >>> --------------------------------------------------------------------- >>> > > > > > > > To unsubscribe, e-mail: [email protected] >>> > > > > > > > For additional commands, e-mail: [email protected] >>> > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > For additional commands, e-mail: [email protected] >>> > >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>>
