Le 09/12/2016 à 12:05, Matthieu Huin a écrit : > Some additional shower thoughts: > > I'm thinking we could add a "smoke" pipeline prior to the "check" one. It > would launch one job running unit tests; > if they pass then the package building occurs and a publisher job pushes it > to swift or a repo with a guessable URI > (for example computed from the commit sha). The review gets a "smoke-verified > +1" or something, which will trigger > the proper functional tests where the test env will be installed using the > RPM(s) generated by the smoke pipeline. > > I believe this would make testing and getting feedback much faster. WDYT?
Yes. Having the unittest run + packaging (with unittest run again during the packaging) in a first step can be a good idea. There is a way to do that via a pipeline or even via a parent task in Zuul. > ----- Original Message ----- >> From: "Matthieu Huin" <[email protected]> >> To: "Fabien Boucher" <[email protected]> >> Cc: [email protected] >> Sent: Friday, December 9, 2016 11:19:08 AM >> Subject: Re: [Softwarefactory-dev] CI/Release workflow for a RPM >> packaged version of SF >> >> >> >> ----- Original Message ----- >>> From: "Fabien Boucher" <[email protected]> >>> To: [email protected] >>> Sent: Friday, December 9, 2016 11:00:55 AM >>> Subject: [Softwarefactory-dev] CI/Release workflow for a RPM packaged >>> version of SF >>> >>> Hi, >>> >>> I'm thinking about it since yesterday so and I wanted to share my thoughts >>> here. >>> >>> The idea is to have a sf-master target in Koji which is a "pot commun" >>> where >>> all >>> packages built during the CI will land. We need to have a magic tool that >>> know >>> how to manage rpm(s) build/runtime dependencies according to the >>> ZUUL_CHANGES >>> variables passed by Zuul. >>> >>> Each build of a rpm need to have the Software commit hash (for the ones we >>> host and dev >>> on Gerrit) and the tag (for the other ones like Gerrit, Storyboard). Also >>> I'm >>> thinking of a meta package called sf-<version>.rpm that contains all the >>> mandatories dependencies. >>> >>> For example: >>> * sf-3.0.0-1.rpm - depends on: >>> ** managesf-<hash>-1.rpm >>> ** managesf-ansible-<hash>-1.rpm >>> ** sf-ansible-<hash>-1.rpm >>> ** gerrit-2.X.X-1.rpm >>> ** gerrit-ansible-0.1-1.rpm >>> ** sf-docs-3.0.0-1.rpm >>> >>> This meta package is then the entry point to install SF and its mandatory >>> dependencies. >>> (Maybe it should also include a file with the list of extra component (such >>> repoxplorer, ...) >>> at the exact versions supported by this release of SF). We can even imagine >>> it contains >>> our reno notes. In this version of "dreamland" the SF Git repository should >>> only >>> contains the "(template) ?" .spec of this meta package. >>> >>> In the CI the meta package .spec file is modified according the build >>> context. For >>> example managesf is in the ZUUL_CHANGES then this meta package will be >>> rebuilt to pin >>> the freshly built version of managesf. >>> But doing that at this meta package level is not enough. For instance >>> pysflib >>> is >>> modified then the managesf's build/runtime rpm deps need to changed to pin >>> the >>> pysflib version. >>> >>> Here could be the resulting workflow of the CI to test an incoming change >>> in >>> the SF CI: >>> We bump Gerrit: >>> 1 -> A change in the gerrit-distgit repo is proposed >>> 2 -> First Gerrit is built on koji (non-scratch) build and land in the "pot >>> commun" >>> 3 -> The meta package is rebuild and pin the new version of Gerrit >>> 4 -> The NVR of the meta package can maybe an epoch or the ZUUL_UUID ? >>> 5 -> The SF image is built/or updated (if a previous on exist on the >>> slave - Then having the epoch make sense) using the "pot commun" koji >>> repo >>> in /etc/yum.repo.d. >>> 6 -> Test the SF image as usual >>> 7 -> If a success (in the gate) the gerrit-distgit proposed changed lands >>> in >>> the >>> Git repo. >> >> As you mentioned above, how do you deal with dependencies between components >> ? >> I'm thinking of a common use case whenever we add smth in the manageSF API, >> we >> usually have to change the manageSF config template in SF; both changes land >> semi-simultaneously. Thus our packaging workflow should be aware of >> depends-on >> tags to reflect that dependency in the build. >> >>> Building a master SF for our test env could be then : only install the "pot >>> commun" >>> repo in yum.repo.d and yum install sf[-<epoch>]. Note that as the meta >>> package sf use the epoch >>> as NVR so theoretically you could just yum install sf from the "pot commum" >>> but as >>> in the flow I described the meta package sf is built for each patchset then >>> we won't >>> have the guarantie the latest sf-<epoch>.rpm is a working version of SF :(. >>> >>> Working on a dev env (let's say on managesf again) could be then as follow: >>> -> Make you change locally in managesf source repo >>> -> Build the SRPM >>> -> Send it to koji (in scratch or non scratch) whatever >> >> Could the rpm building be done locally on the dev env, with a local dev repo? >> Might be faster and easier. >> >>> -> Fetch the artifacts >>> -> and install them in the devel SF >> >> Wouldn't that be broken if dependencies versions are pinned ? ie you can't >> install a newer version of this package because of dep pinning. >> >>> -> When your are satisfied with your changes -> propose them >>> on Gerrit and the CI will re-test your changes as usual. >> >>> Additional note : We want to be sure at any time that all master branch >>> of each repo[-distgit] that are part of the SF distribution are working >>> together (pass the SF tests). >>> >>> What about the SF release ? Let's say now we want to release sf-3.0.0. >>> Then we will tag (in the koji terminology) >>> https://fedoraproject.org/wiki/Koji#Package_Organization >>> a specific list of built packages. We tag them with the tag name sf-3.0.0. >>> We >>> do >>> the same when we want to release sf-3.0.1 and so on. >>> So each stable release of the SF (Distribution) will have its own RPM repo. >>> -> I'm not sure about that and need to be experimented ... >>> >>> Let's discuss about that and raise questions and issues. >>> I propose also to setup a semi-official Koji for our needs and we could >>> start >>> using it >>> to experiment. >>> >>> Cheers, >>> Fabien >>> >>> >>> _______________________________________________ >>> Softwarefactory-dev mailing list >>> [email protected] >>> https://www.redhat.com/mailman/listinfo/softwarefactory-dev >>> >> >> _______________________________________________ >> Softwarefactory-dev mailing list >> [email protected] >> https://www.redhat.com/mailman/listinfo/softwarefactory-dev >>
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Softwarefactory-dev mailing list [email protected] https://www.redhat.com/mailman/listinfo/softwarefactory-dev
