Hi Pere, This is not about strong reactions. This is about tech merit (there are cheaper and more straight forward solutions) and disregard of the main contributors in the area ( which mostly should vetoed this already). This is a way to highjack the community to flow the monorepo in multiple steps. This is not transparent or how to build a healthy community.
El mié, 5 feb 2025, 13:26, Pere Fernandez (apache) <[email protected]> escribió: > + 0 > I'm personally ok with everything that helps us to move forward and I hope > this will help in this regard. > > However, although I don't have a strong opinion about the documentation and > websites, I still think that keeping examples in kogito-examples repo still > makes sense. I understand the benefit or the idea behind having examples in > kie-tools, but from the community point of view, I'm worried that because > of this movement we could get some reactions from the traditional KIE > Contributors that have more of a Java background and likely won't want to > complicate their development setup. > > > On Wed, 5 Feb 2025 at 11:33, Francisco Javier Tirado Sarti < > [email protected]> wrote: > > > Since this is going forward, without consensus and in the form of an > > omnibus proposal that mix things that should have been already done for > > 10.0.0 and others that are not really needed, whatever is finally done, > > please have into account this current check for runtimes and apps PRs > > should be maintained, or the examples are really going to get outdated > > > > [image: image.png] > > I really hope to be mistaken, but carrying forward the proposal as it is > > is a recipe for disaster > > Good luck. > > > > On Wed, Feb 5, 2025 at 3:45 AM Mark Proctor <[email protected]> > wrote: > > > >> +1 > >> > >> On Mon, 3 Feb 2025 at 16:31, Alex Porcelli <[email protected]> wrote: > >> > >> > Given the recent discussions about Docs [1] and Examples [2], Tiago > >> > Bento and I got together to write, in proposal form, the combination > >> > of our understanding of how we can create the baseline for content > >> > (Docs, Examples, and Websites) so that our users can consume Apache > >> > KIE releases in an expected, structured, and cohesive way. This > >> > proposal will take you < 15 min to read. > >> > > >> > - > >> > Alex & Tiago > >> > > >> > # Problem scope > >> > > >> > There has been a significant demand for proper guidance on using > >> > Apache KIE 10.0. Users are unsure how to adopt it, as neither > >> > Documentation nor Examples were available when 10.0.0 was released. An > >> > attempt to manually adjust the examples to the 10.0.0 release has > >> > happened for a couple of weeks now, showing how time-consuming it is > >> > to defer dealing with “satellite” repositories after releases are > >> > done. Even more complicated is that well-known websites (drools.org, > >> > jbpm.org, kogito.kie.org) are outdated and out of sync with Apache > >> > KIE’s repositories and releases, thus contributing to more confusion > >> > than anything else. > >> > > >> > To avoid being unclear about why Docs, Examples, and Websites are > >> > crucial for the community's success, they are all entry points where > >> > new users can start understanding Apache KIE, how to use it and > >> > whether or not it’s a fit for them. For existing users, they are all > >> > reference material, including code snippets, institutional texts > >> > and/or vision, licenses, migration guides, and legal material. > >> > Examples, in particular, also work as playgrounds, where users can > >> > start with pre-built applications and make minor tweaks to either > >> > check if something they want is supported or simulate a development > >> > workflow on a smaller scale. Our code is worth close to nothing if > >> > there isn’t good supporting material around it. > >> > > >> > This proposal aims to amend all those problems by bringing Docs, > >> > Examples, and Websites to the Release Procedure, PR checks, and a > >> > Continuous deployment (CD) mechanism, making them all be part of our > >> > daily contributions, so they are not treated as afterthoughts. > >> > > >> > > >> > # What is not in scope > >> > > >> > - A complete restructuring of all Apache KIE repositories > >> > - Refactoring the whole CI; this proposal aims to work with the > >> > current established CI already in place > >> > - Refactor content in Docs and Example applications; this proposal > >> > aims to move stuff as-is, except when something is broken, then we’ll > >> > fix it. > >> > - Revisit all Examples to check or validate whether they are fully > >> > functional. This is an important review that must be done > >> > incrementally. > >> > - Retroactively adjust the 10.0.x branch to include Docs, Examples, > >> > and Websites. > >> > - Provide a new blogging framework for Apache KIE in replacement for > >> > https://blog.kie.org/ > >> > > >> > > >> > # The proposal > >> > > >> > - The Apache KIE website gains a new section called "Docs" in its > >> > header, where users are able to select what stream of the Docs they > >> > want to see, for example: "Docs" -> "10.1.x" -> "Drools"; or "Docs" -> > >> > "10.2.x" -> "SonataFlow" etc. > >> > - A new read-only repository called `incubator-kie-examples` is > >> > created, containing all examples aligned with development stream > >> > branches, and releases candidates and releases tags. > >> > - A new release artifact (Examples ZIP) starts existing, and its > >> > contents then published to the read-only `incubator-kie-examples` > >> > repository. > >> > - The existing `incubator-kogito-examples` repository is archived > >> > after 10.1 is first released. > >> > - A new Continuous Deployment (CD) mechanism is created to ensure Docs > >> > and Websites are kept current. > >> > > >> > > >> > # The method > >> > > >> > Leverage `kie-tools` flexible build system to reduce complexity hidden > >> > in Jenkins/GitHub Actions automations, which have proven to become > >> > unmaintained and abandoned, by moving content (preserving Git history) > >> > from all Docs-, Examples-, and Websites-related repositories into new > >> > packages on `kie-tools`. Here’s the final structure this proposal aims > >> > to create inside the `kie-tools` repository: > >> > > >> > - kie-tools/docs/drools > >> > - kie-tools/docs/optaplanner > >> > - kie-tools/docs/jbpm > >> > - kie-tools/docs/kogito > >> > - kie-tools/docs/sonataflow > >> > - kie-tools/docs/tools > >> > - kie-tools/websites/kie-apache-org > >> > - kie-tools/examples/rules-* > >> > - kie-tools/examples/decisions-* > >> > - kie-tools/examples/process-* > >> > - kie-tools/examples/sonataflow-* > >> > - kie-tools/examples/pmml-* > >> > - kie-tools/examples/trusty-* > >> > > >> > For each example package, Spring Boot, Quarkus, and plain Java > >> > subdivisions would be done inside each example using different Maven > >> > modules. This would allow users to focus on the domain rather than the > >> > framework/implementation and avoid content duplication in READMEs. > >> > > >> > Being inside `kie-tools`, Examples would be able to benefit from > >> > graphical Editors, Runtime Consoles, Quarkus Dev UIs, and Container > >> > images, all crucial components for end-to-end solutions built with > >> > Apache KIE. This is currently impossible in the current > >> > `kogito-examples` repository due to the way our PR checks and releases > >> > are structured/automated, where `kogito-examples` builds before > >> > `kie-tools`. Making `kogito-examples` build after `kie-tools` for PR > >> > checks is non-trivial and completely out-of-scope of this proposal, as > >> > it would mean doubling-down on the existing CI systems like > >> > “build-chain” for PR checks and “the Kogito framework” on Apache > >> > Jenkins, both inherently wasteful on cross-repo PR checks and provenly > >> > hard to maintain/operate. > >> > > >> > Contributions to the proposed new packages would be made following the > >> > same commands we already use for all other packages on `kie-tools`. > >> > Contributors would need to have `pnpm` and Node.js installed or rely > >> > on Devbox to install everything automatically. After setting up the > >> > environment, cloning the repo, and running `pnpm bootstrap -F > >> > '@kie-docs/drools’ && pnpm -F @kie-docs/drools start` contributors > >> > would be able to develop the Drools documentation with live-reload > >> > capabilities. The very same would be true for Website(s). > >> > > >> > For the Examples applications, the process after cloning would be: > >> > > >> > 1. `pnpm bootstrap -F @kie-examples/some-example-app...` to install > >> > 3rd party dependencies and link packages. > >> > 2. `pnpm -F @kie-examples/some-example-app^... build:dev` to build > >> > this example’s 1st party dependencies. > >> > 3. `pnpm -F @kie-examples/some-example-app start` to start the example > >> > in its default mode. > >> > > >> > Each example is also encouraged to define in their README how to start > >> > itself using technology-specific commands, like `mvn quarkus:dev > >> > -Psome-profile` or even `docker-compose up`. The `start` script is > >> > only the default way to run an example, a convention already cohesive > >> > with all other existing packages on `kie-tools`. > >> > > >> > It’s important to note again that the `pnpm` commands would not be > >> > necessary in the `incubator-kie-examples` repository and users would > >> > be able to run plain `mvn` commands directly inside the example’s > >> > directory, for Maven-based examples at least. > >> > > >> > > >> > #### Impact on PR checks (CI) > >> > > >> > Each new package will contain a `package.json` file describing scripts > >> > and their dependency relationship with other packages in the > >> > repository. The important scripts are `build:dev`, `build:prod`, and > >> > `start`, to integrate them with the existing build system. > >> > > >> > PRs changing exclusively Docs packages will NOT build anything else in > >> > the repository, making PR checks efficient. The same is true for > >> > websites. > >> > > >> > PRs changing example applications will only build their dependency > >> > tree and the modified example applications, excluding unrelated > >> > packages from the checks while enforcing correctness. This is already > >> > how `kie-tools` works. > >> > > >> > If you need more information on how this mechanism works in practice, > >> > please come forward with clarifying questions. We’d love to show you > >> > how the partial and partitioned PR checks work on `kie-tools`. > >> > > >> > > >> > #### Impact on Release Process (“CI”) > >> > > >> > Docs and Websites wouldn’t be release artifacts and would be handled > >> > by a new “Continuous deployment (CD)” mechanism. This is covered in > >> > the next section of this proposal. > >> > > >> > With that said, only one new release artifact is being proposed here. > >> > A ZIP file containing all examples, stripping out all > >> > `kie-tools`-related boilerplate used to develop them. This new > >> > artifact (Examples ZIP) would be published to a new read-only > >> > repository (incubator-kie-examples) for all release candidates and > >> > releases we do. This repository would be THE consumable repository for > >> > users who wish to go straight to the point when testing out Apache KIE > >> > and who are not necessarily committers/developers. > >> > > >> > To do that, changing some release automations to add a new job that > >> > builds and publishes the Examples ZIP would be necessary: > >> > > >> > 1. Daily-dev ( > >> > > >> > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/kie-tools-daily-dev-publish/ > >> > ) > >> > 2. Release Candidates > >> > ( > >> > > >> > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-candidate/ > >> > ) > >> > 3. Release Promote > >> > ( > >> > > >> > https://ci-builds.apache.org/job/KIE/job/kie-tools/job/main/job/release/job/kie-tools-release-promote/ > >> > ) > >> > > >> > > >> > #### New Continuous Deployment (CD) > >> > > >> > Websites and Docs would be treated as continually deployed packages > >> > for the development stream we declare as “live”. Also, old development > >> > streams would be continually built and published to make documentation > >> > updates possible for older versions. Referring to Toshiya's thread on > >> > documentation [1], our Docs would be “mutable”, and the granularity > >> > would be “per stream”. > >> > > >> > Two new automations would be created for making sure our content is > >> > up-to-date at all times: > >> > > >> > `publish-kie-apache-org-website`: Responsible for updating live Apache > >> > KIE’s website (https://kie.apache.org) and also capable of generating > >> > ad-hoc immutable copies of the website for Release Candidates. It > >> > would be able to also be executed for the `main` stream for > >> > development purposes. > >> > > >> > `publish-older-docs-to-live-website` ensures that older development > >> > streams can have their content updated on the live website. For > >> > example, if we find a seriously incorrect statement in an older > >> > version, this job would ensure it’s published after being fixed. We > >> > can eventually remove this for versions that become too old. > >> > > >> > > >> > # Actionable items (plan) > >> > > >> > Phase 1 (Development and contributions): > >> > > >> > - Move all examples to `kie-tools`, already in the new > >> > structure/organization, and integrate them as part of the build system > >> > - Move all documentation content to `kie-tools` and integrate the > >> > new packages as part of the build system > >> > - Move `incubator-kie-website` content to `kie-tools` and integrate > >> > it as part of the build system > >> > - Create new landing pages for each major component: Drools, jBPM, > >> > SonataFlow, Optaplanner, and Kogito. > >> > > >> > Phase 2 (Release Process) > >> > > >> > - Create the `incubator-kie-examples` repository > >> > - Update `kie-tools` daily-dev job to publish the Examples ZIP to > >> > `incubator-kie-examples@main` > >> > - Update `kie-tools` Release Candidate job to publish the Examples > >> > ZIP to https://dist.apache.org/repos/dist/dev/incubator/kie/ > >> > - Update `kie-tools` Release Promote job to publish the Examples ZIP > >> > content to incubator-kie-examples@{version}` > >> > - Archive the `incubator-kie-kogito-examples` repository (post 10.1 > >> > release) > >> > - Update the Release Procedure for 10.1 to include Docs, Examples, > >> > and Websites where necessary > >> > > >> > Phase 3 (Continuous deployments) > >> > > >> > - Create and configure new `publish-kie-apache-org-website` job > >> > - Create and configure new `publish-docs-to-live-website` job > >> > > >> > Phase 4 (Cleanup) > >> > > >> > - Depublish current `drools.org`, `jbpm.org`, `optaplanner.org`, > >> > `kie.org`, `sonataflow.org` and all its subdomains. > >> > - Redirect all old domains to the proper landing page of Apache KIE’s > >> > widely known components (E.g., https://drools.org → > >> > https://kie.apache.org/drools) > >> > - Preserve Docs from older versions somewhere accessible under Apache > >> > KIE’s website. > >> > > >> > > >> > # Commitment and rough timeline > >> > > >> > Alex Porcelli and Tiago Bento are fully committed to executing this > >> > plan in collaboration with other members who want to help. > >> > > >> > We are committed to working on it as soon as the proposal is approved. > >> > Although it’s hard to predict how long it will take, Phases 1, 2, and > >> > 3 will take around 6 weeks based on the information we have today. If > >> > anything changes or issues are found during execution, we can update > >> > this thread. > >> > > >> > Phase 4 is impossible to estimate, as there might be many different > >> > people and/or organizations to chase so that these websites can be > >> > decommissioned and properly forward users to KIE’s new home in Apache. > >> > > >> > > >> > # Observations on `kie-tools` > >> > > >> > This proposal makes Docs, Examples, and Websites available to all > >> > users without touching the complex, unmaintained, black-box systems > >> > currently in Apache’s Jenkins, known as “build-chain” and “the Kogito > >> > CI framework”. > >> > > >> > We know that `kie-tools` is not unanimously popular among Apache KIE > >> > committers, but it’s actively maintained and has been proven to scale > >> > well both in package count and technology diversity. We invite > >> > everyone to take a look at the repo’s User Manual at > >> > > https://github.com/apache/incubator-kie-tools/blob/main/repo/MANUAL.md > >> > > >> > > >> > ## Non-obvious positive outcomes > >> > > >> > - Drools and OptaPlanner repositories won’t have dependencies on > >> > JavaScript anymore. > >> > - Fonts and other “binary” files won’t need to be committed as a > >> > source anymore, as `kie-tools`’s build system can properly depend on > >> > NPM artifacts. Also helps towards graduation, as Category X > >> > dependencies won’t be present. > >> > > >> > > >> > [1] https://lists.apache.org/thread/zjt156tk5zjw7463xsfdkojwzndwd1mn > >> > [2] https://lists.apache.org/thread/qc9vhod96mdoppo5ssj4f0pkqhzt4ghd > >> > > >> > --------------------------------------------------------------------- > >> > To unsubscribe, e-mail: [email protected] > >> > For additional commands, e-mail: [email protected] > >> > > >> > > >> > > >
