I forgot to explain the -1, even if I've no doubt that it does not have influence because, actually, votes have been sent without even considering the chance of a technical discussion.
Nevertheless, just for exercise: About scope: The proposal mixes two very different topics (documents and codebase) that are completely unrelated: mixing them is an example of unclearness and a fuzzy-driven approach, pretty untechnical. The proposal mention > A new read-only repository called `incubator-kie-examples` is > created, containing all examples aligned with development stream > branches, and releases candidates and releases tags. but how could be a read-only repo in sync with development ones ? And how would this be different with the current one, which already has a 10.0x stable branch ? The proposal mention > A new release artifact (Examples ZIP) starts existing, and its > contents then published to the read-only `incubator-kie-examples` > repository. but, to be published under apache, artifacts should be reproducible, and AFAIK our examples are not. The proposal presents "kie-tools" as a "flexible build system", which is a very opinionated way to present it. With the help of other senior devs, I have spent most part of last year setting up our internal CI, and the biggest issue has been, for everyone involved, that very "flexible building system". Beside, that building system is based on top of "pnpm" whose maturity and reliability, for enterprise projects, should be double checked (both in terms of current issues, actual mainteners, etc). The proposal completely ignores the fact that most of the issues we currently have are a consequence of objectionable choices, e.g. the fact that kie-tools repo uses timestamped snapshots puts it completely out of sync with all other repos. The proposal completely ignores the fact that the biggest problem lies in our repositories not reflecting the architecture of our codebase, and that is the cause of circular dependencies, version misalignment, etc. That proposal also contains other false assertions, as previously mentioned by Dominik. So, from my technical point of view, that proposal is just another naive attempt to avoid solving real problems, hiding them below another layer of complexity that, in the future, would create additional ones (has always "workarounds" lead to). I would be (or, I would have been) happy to re-evaluate all of my observations in the light of a fair and technical discussion. Then, there is a completely different matter that raised my attention, about the correctness of all that procedure. Even people that agreed with this proposal, raised concerns about the fact that such a vote, for its importance, should have been preceded by a discussion, after which anyone could have had the chance to evaluate other's opinion, and give a better informed vote. This has been prevented from the very beginning and nature of that proposal, and then my next question is: what's the real value of such votes ? I saw "+1" from people that do not even know how dependencies are managed in our examples, and probably never installed npm/pnpm stack on their machines. I saw other people popping out from the old happy past, to give a vote on something they are not even working at for a lot of time. I'm doubtful of how many people actually evaluated, in first person, the reliability of the pnpm tool, on top of which all the building systems should be based: in the past, when we had to choose between java libraries, we excluded some for much less. While "-1" votes require justification, +1 seems not, but I would be curious to know how many (beside the tools developers) have actually installed and worked on that "building system" for java development ? "Community sake" is a buzzword that is frequently pulled here and there, depending on the need of the moment. Sometimes the lack of external contributors has been mentioned as a problem (again, when it was instrumental to drive discussion in a given direction). I wonder how many of the +1 has given a slight thought about the fact that most of our code is java based, and that the core of our product are the engines, again written in java; how could we think that it would be "community attractive" to force java developers to deal with the node/npm/pnpm "flexible building system" just to test a plain java application ? Last, that proposal, knowing very well all the controversial and different strong opinions, has been made exactly while we were trying to write our own rules of games, and as it has been pushed breaks the very basilar concepts we all agreed upon, i.e. that there should be a discussion before a vote. Cheers :) Il giorno mer 5 feb 2025 alle ore 11:35 Francisco Javier Tirado Sarti < [email protected]> ha scritto: > Since this is going forward, without consensus and in the form of an > omnibus proposal that mix things that should have been already done for > 10.0.0 and others that are not really needed, whatever is finally done, > please have into account this current check for runtimes and apps PRs > should be maintained, or the examples are really going to get outdated > > [image: image.png] > I really hope to be mistaken, but carrying forward the proposal as it is > is a recipe for disaster > Good luck. > > On Wed, Feb 5, 2025 at 3:45 AM Mark Proctor <[email protected]> wrote: > >> +1 >> >> On Mon, 3 Feb 2025 at 16:31, Alex Porcelli <[email protected]> wrote: >> >> > Given the recent discussions about Docs [1] and Examples [2], Tiago >> > Bento and I got together to write, in proposal form, the combination >> > of our understanding of how we can create the baseline for content >> > (Docs, Examples, and Websites) so that our users can consume Apache >> > KIE releases in an expected, structured, and cohesive way. This >> > proposal will take you < 15 min to read. >> > >> > - >> > Alex & Tiago >> > >> > # Problem scope >> > >> > There has been a significant demand for proper guidance on using >> > Apache KIE 10.0. Users are unsure how to adopt it, as neither >> > Documentation nor Examples were available when 10.0.0 was released. An >> > attempt to manually adjust the examples to the 10.0.0 release has >> > happened for a couple of weeks now, showing how time-consuming it is >> > to defer dealing with “satellite” repositories after releases are >> > done. Even more complicated is that well-known websites (drools.org, >> > jbpm.org, kogito.kie.org) are outdated and out of sync with Apache >> > KIE’s repositories and releases, thus contributing to more confusion >> > than anything else. >> > >> > To avoid being unclear about why Docs, Examples, and Websites are >> > crucial for the community's success, they are all entry points where >> > new users can start understanding Apache KIE, how to use it and >> > whether or not it’s a fit for them. For existing users, they are all >> > reference material, including code snippets, institutional texts >> > and/or vision, licenses, migration guides, and legal material. >> > Examples, in particular, also work as playgrounds, where users can >> > start with pre-built applications and make minor tweaks to either >> > check if something they want is supported or simulate a development >> > workflow on a smaller scale. Our code is worth close to nothing if >> > there isn’t good supporting material around it. >> > >> > This proposal aims to amend all those problems by bringing Docs, >> > Examples, and Websites to the Release Procedure, PR checks, and a >> > Continuous deployment (CD) mechanism, making them all be part of our >> > daily contributions, so they are not treated as afterthoughts. >> > >> > >> > # What is not in scope >> > >> > - A complete restructuring of all Apache KIE repositories >> > - Refactoring the whole CI; this proposal aims to work with the >> > current established CI already in place >> > - Refactor content in Docs and Example applications; this proposal >> > aims to move stuff as-is, except when something is broken, then we’ll >> > fix it. >> > - Revisit all Examples to check or validate whether they are fully >> > functional. This is an important review that must be done >> > incrementally. >> > - Retroactively adjust the 10.0.x branch to include Docs, Examples, >> > and Websites. >> > - Provide a new blogging framework for Apache KIE in replacement for >> > https://blog.kie.org/ >> > >> > >> > # The proposal >> > >> > - The Apache KIE website gains a new section called "Docs" in its >> > header, where users are able to select what stream of the Docs they >> > want to see, for example: "Docs" -> "10.1.x" -> "Drools"; or "Docs" -> >> > "10.2.x" -> "SonataFlow" etc. >> > - A new read-only repository called `incubator-kie-examples` is >> > created, containing all examples aligned with development stream >> > branches, and releases candidates and releases tags. >> > - A new release artifact (Examples ZIP) starts existing, and its >> > contents then published to the read-only `incubator-kie-examples` >> > repository. >> > - The existing `incubator-kogito-examples` repository is archived >> > after 10.1 is first released. >> > - A new Continuous Deployment (CD) mechanism is created to ensure Docs >> > and Websites are kept current. >> > >> > >> > # The method >> > >> > Leverage `kie-tools` flexible build system to reduce complexity hidden >> > in Jenkins/GitHub Actions automations, which have proven to become >> > unmaintained and abandoned, by moving content (preserving Git history) >> > from all Docs-, Examples-, and Websites-related repositories into new >> > packages on `kie-tools`. Here’s the final structure this proposal aims >> > to create inside the `kie-tools` repository: >> > >> > - kie-tools/docs/drools >> > - kie-tools/docs/optaplanner >> > - kie-tools/docs/jbpm >> > - kie-tools/docs/kogito >> > - kie-tools/docs/sonataflow >> > - kie-tools/docs/tools >> > - kie-tools/websites/kie-apache-org >> > - kie-tools/examples/rules-* >> > - kie-tools/examples/decisions-* >> > - kie-tools/examples/process-* >> > - kie-tools/examples/sonataflow-* >> > - kie-tools/examples/pmml-* >> > - kie-tools/examples/trusty-* >> > >> > For each example package, Spring Boot, Quarkus, and plain Java >> > subdivisions would be done inside each example using different Maven >> > modules. This would allow users to focus on the domain rather than the >> > framework/implementation and avoid content duplication in READMEs. >> > >> > Being inside `kie-tools`, Examples would be able to benefit from >> > graphical Editors, Runtime Consoles, Quarkus Dev UIs, and Container >> > images, all crucial components for end-to-end solutions built with >> > Apache KIE. This is currently impossible in the current >> > `kogito-examples` repository due to the way our PR checks and releases >> > are structured/automated, where `kogito-examples` builds before >> > `kie-tools`. Making `kogito-examples` build after `kie-tools` for PR >> > checks is non-trivial and completely out-of-scope of this proposal, as >> > it would mean doubling-down on the existing CI systems like >> > “build-chain” for PR checks and “the Kogito framework” on Apache >> > Jenkins, both inherently wasteful on cross-repo PR checks and provenly >> > hard to maintain/operate. >> > >> > Contributions to the proposed new packages would be made following the >> > same commands we already use for all other packages on `kie-tools`. >> > Contributors would need to have `pnpm` and Node.js installed or rely >> > on Devbox to install everything automatically. After setting up the >> > environment, cloning the repo, and running `pnpm bootstrap -F >> > '@kie-docs/drools’ && pnpm -F @kie-docs/drools start` contributors >> > would be able to develop the Drools documentation with live-reload >> > capabilities. The very same would be true for Website(s). >> > >> > For the Examples applications, the process after cloning would be: >> > >> > 1. `pnpm bootstrap -F @kie-examples/some-example-app...` to install >> > 3rd party dependencies and link packages. >> > 2. `pnpm -F @kie-examples/some-example-app^... build:dev` to build >> > this example’s 1st party dependencies. >> > 3. `pnpm -F @kie-examples/some-example-app start` to start the example >> > in its default mode. >> > >> > Each example is also encouraged to define in their README how to start >> > itself using technology-specific commands, like `mvn quarkus:dev >> > -Psome-profile` or even `docker-compose up`. The `start` script is >> > only the default way to run an example, a convention already cohesive >> > with all other existing packages on `kie-tools`. >> > >> > It’s important to note again that the `pnpm` commands would not be >> > necessary in the `incubator-kie-examples` repository and users would >> > be able to run plain `mvn` commands directly inside the example’s >> > directory, for Maven-based examples at least. >> > >> > >> > #### Impact on PR checks (CI) >> > >> > Each new package will contain a `package.json` file describing scripts >> > and their dependency relationship with other packages in the >> > repository. The important scripts are `build:dev`, `build:prod`, and >> > `start`, to integrate them with the existing build system. >> > >> > PRs changing exclusively Docs packages will NOT build anything else in >> > the repository, making PR checks efficient. The same is true for >> > websites. >> > >> > PRs changing example applications will only build their dependency >> > tree and the modified example applications, excluding unrelated >> > packages from the checks while enforcing correctness. This is already >> > how `kie-tools` works. >> > >> > If you need more information on how this mechanism works in practice, >> > please come forward with clarifying questions. We’d love to show you >> > how the partial and partitioned PR checks work on `kie-tools`. >> > >> > >> > #### Impact on Release Process (“CI”) >> > >> > Docs and Websites wouldn’t be release artifacts and would be handled >> > by a new “Continuous deployment (CD)” mechanism. This is covered in >> > the next section of this proposal. >> > >> > With that said, only one new release artifact is being proposed here. >> > A ZIP file containing all examples, stripping out all >> > `kie-tools`-related boilerplate used to develop them. This new >> > artifact (Examples ZIP) would be published to a new read-only >> > repository (incubator-kie-examples) for all release candidates and >> > releases we do. This repository would be THE consumable repository for >> > users who wish to go straight to the point when testing out Apache KIE >> > and who are not necessarily committers/developers. >> > >> > To do that, changing some release automations to add a new job that >> > builds and publishes the Examples ZIP would be necessary: >> > >> > 1. Daily-dev ( >> > >> https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/kie-tools-daily-dev-publish/ >> > ) >> > 2. Release Candidates >> > ( >> > >> https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-candidate/ >> > ) >> > 3. Release Promote >> > ( >> > >> https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-promote/ >> > ) >> > >> > >> > #### New Continuous Deployment (CD) >> > >> > Websites and Docs would be treated as continually deployed packages >> > for the development stream we declare as “live”. Also, old development >> > streams would be continually built and published to make documentation >> > updates possible for older versions. Referring to Toshiya's thread on >> > documentation [1], our Docs would be “mutable”, and the granularity >> > would be “per stream”. >> > >> > Two new automations would be created for making sure our content is >> > up-to-date at all times: >> > >> > `publish-kie-apache-org-website`: Responsible for updating live Apache >> > KIE’s website (https://kie.apache.org) and also capable of generating >> > ad-hoc immutable copies of the website for Release Candidates. It >> > would be able to also be executed for the `main` stream for >> > development purposes. >> > >> > `publish-older-docs-to-live-website` ensures that older development >> > streams can have their content updated on the live website. For >> > example, if we find a seriously incorrect statement in an older >> > version, this job would ensure it’s published after being fixed. We >> > can eventually remove this for versions that become too old. >> > >> > >> > # Actionable items (plan) >> > >> > Phase 1 (Development and contributions): >> > >> > - Move all examples to `kie-tools`, already in the new >> > structure/organization, and integrate them as part of the build system >> > - Move all documentation content to `kie-tools` and integrate the >> > new packages as part of the build system >> > - Move `incubator-kie-website` content to `kie-tools` and integrate >> > it as part of the build system >> > - Create new landing pages for each major component: Drools, jBPM, >> > SonataFlow, Optaplanner, and Kogito. >> > >> > Phase 2 (Release Process) >> > >> > - Create the `incubator-kie-examples` repository >> > - Update `kie-tools` daily-dev job to publish the Examples ZIP to >> > `incubator-kie-examples@main` >> > - Update `kie-tools` Release Candidate job to publish the Examples >> > ZIP to https://dist.apache.org/repos/dist/dev/incubator/kie/ >> > - Update `kie-tools` Release Promote job to publish the Examples ZIP >> > content to incubator-kie-examples@{version}` >> > - Archive the `incubator-kie-kogito-examples` repository (post 10.1 >> > release) >> > - Update the Release Procedure for 10.1 to include Docs, Examples, >> > and Websites where necessary >> > >> > Phase 3 (Continuous deployments) >> > >> > - Create and configure new `publish-kie-apache-org-website` job >> > - Create and configure new `publish-docs-to-live-website` job >> > >> > Phase 4 (Cleanup) >> > >> > - Depublish current `drools.org`, `jbpm.org`, `optaplanner.org`, >> > `kie.org`, `sonataflow.org` and all its subdomains. >> > - Redirect all old domains to the proper landing page of Apache KIE’s >> > widely known components (E.g., https://drools.org → >> > https://kie.apache.org/drools) >> > - Preserve Docs from older versions somewhere accessible under Apache >> > KIE’s website. >> > >> > >> > # Commitment and rough timeline >> > >> > Alex Porcelli and Tiago Bento are fully committed to executing this >> > plan in collaboration with other members who want to help. >> > >> > We are committed to working on it as soon as the proposal is approved. >> > Although it’s hard to predict how long it will take, Phases 1, 2, and >> > 3 will take around 6 weeks based on the information we have today. If >> > anything changes or issues are found during execution, we can update >> > this thread. >> > >> > Phase 4 is impossible to estimate, as there might be many different >> > people and/or organizations to chase so that these websites can be >> > decommissioned and properly forward users to KIE’s new home in Apache. >> > >> > >> > # Observations on `kie-tools` >> > >> > This proposal makes Docs, Examples, and Websites available to all >> > users without touching the complex, unmaintained, black-box systems >> > currently in Apache’s Jenkins, known as “build-chain” and “the Kogito >> > CI framework”. >> > >> > We know that `kie-tools` is not unanimously popular among Apache KIE >> > committers, but it’s actively maintained and has been proven to scale >> > well both in package count and technology diversity. We invite >> > everyone to take a look at the repo’s User Manual at >> > https://github.com/apache/incubator-kie-tools/blob/main/repo/MANUAL.md >> > >> > >> > ## Non-obvious positive outcomes >> > >> > - Drools and OptaPlanner repositories won’t have dependencies on >> > JavaScript anymore. >> > - Fonts and other “binary” files won’t need to be committed as a >> > source anymore, as `kie-tools`’s build system can properly depend on >> > NPM artifacts. Also helps towards graduation, as Category X >> > dependencies won’t be present. >> > >> > >> > [1] https://lists.apache.org/thread/zjt156tk5zjw7463xsfdkojwzndwd1mn >> > [2] https://lists.apache.org/thread/qc9vhod96mdoppo5ssj4f0pkqhzt4ghd >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> > >> > >> >
