Il giorno lun 6 dic 2021 alle ore 12:18 Martin Perina <mper...@redhat.com> ha scritto:
> Hi, > > I've tried to move ovirt-engine-extensions-api > <https://github.com/ovirt/ovirt-engine-extensions-api> to github and > setup GH action to build to have some sort of a basic template for all our > Java based projects: > > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml > > During the work I discovered some parts which should be further discussed: > > 1. Most of projects requires packages from ovirt-master-release RPM > repositories to build itself > - In past we have been using > https://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm to > fetch the latest version of this RPM > - Do we plan to keep it and have this URL as the main source for oVirt > master repositories? > No. Please follow https://ovirt.org/develop/dev-process/install-nightly-snapshot.html > - ovirt-engine-extensions-api projects requires only Virt SIG repo so > I've applied below hack to provide it: > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs8.sh#L10 > - I don't like the solution, it would be much nicer to have a constant > URL for the latest ovirt-release-master RPM > Instrucitons in https://ovirt.org/develop/dev-process/install-nightly-snapshot.html whould be stable. For CentOS Virt SIG specifically I haven't yet built the 4.5 release rpms for CentOS . Once they'll be ready you should be able to just dnf install the centos release ovirt-4.5 rpm to enable them. I'll try to handle it before Christmas. > > 2. Providing RPM repositories for different OS inside single build artifact > - I wanted to build the project for both CentOS Stream 8 and 9, but I > figured out that upload-artifact action merges results for all different OS > builds into a single directory, so created repositories are mixed together. > This may have a solution, try using the wildcard pattern https://github.com/actions/upload-artifact#upload-using-a-wildcard-pattern > - I've use GH strategy option and used centos-stream-8 for CS8 and > centos-stream-9 for CS9 builds: > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L14 > - Do we already have that documented so an OST plugin to use those > artifacts per OS can be created? If not, where to put it? > Didn't see one yet. > > 3. Using maven cache for Java project builds > - When build maven projects in jenkins.ovirt.org we were using > artifactory.ovirt.org to cache maven artifacts (and also save download > bandwidth to PHX datacenter) > - It doesn't make to use artifactory.ovirt.org inside GH actions, so > I've tried to define a GH action specific cache: > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L37 > make a lot of sense. > > 4. Caching required dependencies > - Above maven dependencies helps for maven invocations before RPM > build but it doesn't help to fetch all required RPM dependencies for pro > RPM maven based build > - Right now it's almost 500 GB of packages to download and install on > top of the default container and it for ovirt-engine-extensions-api it > takes 50 % of the whole build > time just to download and install those packages (I know it's just 1 > minute, but anyway :-) > - I've noticed that VDSM is using its own customized container > <https://github.com/oVirt/vdsm/blob/master/.github/workflows/ci.yml#L8> > - Wouldn't it be worthwhile to also create customized containers for > Java based projects? > I got suggestion from @Yaniv Kaul <yk...@redhat.com> to do something as in Gluster: https://github.com/gluster/Gluster-Builds - they are building Gluster RPMs by running a container (the 'build container'), which has the required dependencies and they updates once a month, or when there's a new dependency. If someone volunteer to maintain those containers, it may save build time and bandwidth. > > 5. Differences between CS8 and CS9 > - There are again some disturbing changes between CS8 and CS9: > - PowerTools repo from CS8 is now called CRB in CS9 > - pki-deps javapackages-tools modules from CS8 don't exist in CS9 > - Core DNF plugins are not installed by default in CS9 container > - To bypass those differences I've create 2 different shell scripts > and plug them into a single workflow job: > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L33 > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs8.sh > > https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/prepare-env-cs9.sh > > 6. git commands cannot be easily used inside GH action > - GH doesn't checkout the project completely during checkout action, > so .git directory is not present > - In the past I've been using 'git archive' > <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;f=automation/build-artifacts.sh;h=965012c1195eaf288e149bde71e64b2e9647b2c4;hb=refs/heads/master#l45> > to create .tar.gz source archive, but on GH it needed to be switched to a > classic > tar > <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.automation/build-artifacts.sh#L17> > - In the past I've been using 'git rev-list' > <https://gerrit.ovirt.org/gitweb?p=ovirt-engine-extensions-api.git;a=blob;f=automation/build-artifacts.sh;h=965012c1195eaf288e149bde71e64b2e9647b2c4;hb=refs/heads/master#l39> > to fetch short git hash for snapshot builds, but on GH it needed to be > switched to below hack > <https://github.com/oVirt/ovirt-engine-extensions-api/blob/master/.github/workflows/build.yml#L47> > You can dnf install git before running checkout action and then configure checkout action to download full git content. ``` - uses: actions/checkout@v2 with: fetch-depth: 0 ``` > > > Regards, > Martin > > -- > Martin Perina > Manager, Software Engineering > Red Hat Czech s.r.o. > _______________________________________________ > Devel mailing list -- devel@ovirt.org > To unsubscribe send an email to devel-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PBUC7OKVHJJPDLI6C2YAMZNE7LGCF73A/ > -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbona...@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
_______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/5XPDRXV53MRHVFIONT6TNQH7NJDSARAM/
