Hello, reiterating on the plan regarding CI. A short summary is that PR checks for incubator-kie-kogito-images repository. I've spent over one week now on adjusting the PR check pipelines after the repository transfer.
The most problematic part turned out to be the docker environment in jenkins CI. Currently, our kogito-ci-build docker images maps and uses host's docker socket, which results in a shared docker environment across builds. Due to how the pipelines and libraries are written there were several conflicts on the way, I was able to work around, but it turned obvious that the libraries are written with the expectation of a clean docker environment. Thus I picked up on Tibor's effort to switch kogito-ci-build to be ubuntu based and used a proper dind(Docker in docker) image built on top of ubuntu. Looks as a promising way forward, code repositories build correctly with it, no blockers on the way. Now regarding pipelines - I've adjusted the existing PR check pipelines to work nicely with the Github Branch Source Plugin, so we're able to trigger builds and promote the results back into PR. I can't move forward due to test failures in cekit behave tests, I need an expert in the area, tried pinging several people on Zulip (if you know an SME please bring them in), anyway please check stream https://kie.zulipchat.com/#narrow/stream/382044-kogito-dev/topic/Publishing.20Images.20to.20Apache.20Docker.20Hub.3F/ Visible symptoms are that the tests sometime hang waiting for a container being started. And it provides zero insight into what the container was doing if anything at all. And these failures are not stable, happening often, but only to one group of tests: common-dynamic-resources.feature Those tests are messing with VM settings of the containers, so I am wondering if we're not hitting some thresholds given the big change of CI env. Unless I find someone to unblock me, I would switch to a different task, leaving the build of images unfinished. Regards Jan On Wed, 1 Nov 2023 at 09:23, Christofer Dutz <[email protected]> wrote: > Hi all, > > sorry for coming in late to this discussion. > In general, Apache would like projects to use the CI/CD infrastructure > provided by Apache. > This has the simple reason, that in the past it occurred that, when using > external infra, the service simply stopped existing, being free or the > person/company paying for it stopped paying. > Then the project is sort of in a state, where it’s not able to push out > new versions. > > So currently in order to deploy SNAPSHOT versions, Apache Jenkins would be > the place to go, as that’s setup to deploy to repository.apache.org and > other repos. > I think there’s also an option to do this on GitHub Actions runners, but > I’m not that familiar with it. Please reach out to Infra if you’ve got > questions regarding that. > > > Chris > > > Von: Alex Porcelli <[email protected]> > Datum: Dienstag, 31. Oktober 2023 um 19:00 > An: [email protected] <[email protected]> > Betreff: Re: [PROPOSAL] Images Publishing and CI > Seems reasonable. > > We need understand and move towards align with Apache direction, which > means not publish under kiegroup but in Apache instead. But, as proposed, > it’s fine to continue doing it for the short term. > > > On Fri, Oct 27, 2023 at 2:56 PM ricardo zanini fernandes < > [email protected]> wrote: > > > Good question! I think we can, I'm not seeing any problems with this. > > > > -- > > > > > > > > On Fri, Oct 27, 2023 at 3:40 PM Spolti <[email protected]> wrote: > > > > > We can keep using quay for CI test builds and dockerhub for releases? > > > > > > Em sex., 27 de out. de 2023 às 14:29, ricardo zanini fernandes < > > > [email protected]> escreveu: > > > > > > > Hello! > > > > > > > > I'm opening this thread to summarize our discussions around image > > > > publishing and CI. The overall planning is: > > > > > > > > 1. In the short term we can keep using quay.io/kiegroup > > > > 2. Verify the images CI PR checks, and the pipelines to publish > nightly > > > > builds and releases > > > > 3. Keep using Cekit [1] > > > > 4. Migrate to Dockerhub [2] and rename `sed -i > > > > /kogito-serverless-workflow/sonataflow/`. Need to investigate if > > > > Dockerfiles are required since Cekit architecture uses a different > > > > approach. > > > > > > > > Do we all agree? > > > > > > > > I'll start looking at the CI next week, Jakub Schwan is likely to > help. > > > > > > > > Cheers! > > > > > > > > [1] https://cekit.io/ > > > > [2] https://hub.docker.com/u/apache > > > > -- > > > > > > > > > > > > > -- > > > Regards, > > > --Filippe > > > > > >
