Hello, I disagree to add variables in FROM instructions simply because it obliges latest versions and then breaks Docker Automated builds. It simply useless if we rely on Docker manifests as I have already demonstrated. Manifest will also help jjobs as it avoids multiplying daily jjobs per arch.
Cédric 2017-10-14 17:29 GMT+02:00 Alexandru Avadanii <alexandru.avada...@enea.com>: > Hi, > I confirm that the current releng scripts don't support current functest > requirements, but that is within our reach. > Afaik, we are looking at 2 main issues: > 1. functest-core needs to be built first - we can solve this by using > multijobs, with an initial step of building functest-core (amd64 & arm64 in > parallel), then all other builds in parallel; > 2. functest Docker requires support for ARG in FROM - we can solve this by > upgrading Docker to a newer version on all amd64 builders (arm64 already has > a new enough version); > > We (armband) offered to take care of #1. Delia can implement the JJB changes > next week, we estimate it will take 2-3 days. > For #2, we need lab admins with access to the amd64 builder machines to do a > simple package upgrade. > As part of #2, we might need to install a new package (Docker manifest-tool) > too. > > Imo, we can adapt releng scripts without too much trouble, and we (Armband) > are willing to take care of the scripts, if you all agree. > > As for parallel building (not only amd64 and arm64, but all other containers > aside from functest-core) - this will be solved by the design we proposed in > #1. > There is no need for qemu-user-static and binfmt-misc in OPNFV, as we provide > an arm64 native builder (arm-build4). This was required for Travis CI, since > they don't have native arm64 builders. > > BR, > Alex > > >> -----Original Message----- >> From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss- >> boun...@lists.opnfv.org] On Behalf Of Cedric OLLIVIER >> Sent: Saturday, October 14, 2017 2:22 PM >> To: Fatih Degirmenci >> Cc: cedric.olliv...@orange.com; Delia Popescu; opnfv-tech-discuss >> Subject: Re: [opnfv-tech-discuss] [functest] Alpine for arch >> >> Hello Fatih, >> >> My previous mail aims at listing our requirements and again I am convinced >> that we should select releng instead of external tools for a better control. >> The travis-ci links illustrate exactly our needs and could help synchronizing >> Functest and Releng. >> >> Here are the related config files: >> - https://git.opnfv.org/functest/tree/.travis.yml >> - https://git.opnfv.org/functest/tree/.travis.yml?h=stable%2Feuphrates >> >> I had to setup external tools to beta test and share my Alpine containers >> during >> the development cycle. >> The issue was induced by the fact they were copied for OPNFV repositories (I >> precise during my holidays) instead of updating Releng. >> I have only fixed the Docker automated builds and complete them to build the >> remaining containers. >> I could have forced the sync even if I'm not in charge of that as Functest >> core >> dev. >> >> But I strongly think it's fine to compare travis-ci and today's releng jjobs >> simply >> to impove our CI/CD. >> >> Regarding tags, I fully agree that we have taken several quick decisions >> without >> analysing what OpenStack, Docker or GNU/Linux distributions do. >> Bottom up feedbacks may have helped. >> >> Cédric >> >> 2017-10-14 12:42 GMT+02:00 Fatih Degirmenci <fde...@gmail.com>: >> > Hi Cedric, >> > >> > I see lots of conversations around how to build stuff, test them, apply >> > tags but >> it is very hard to follow things. Things are going too fast and with little >> explanation. Each and every new thing coming up will make it harder for us to >> understand what you need. >> > >> > Also the builds first pushed to docker hub then travis ci now. I am not >> > sure >> about this either. I suppose you will try all the external ci services >> instead of >> telling us what you need clearly... >> > >> > So, before I say if something is possible or not, you need to come up with >> clear requirements. We can then fix whatever you need together with you. >> > >> > /Fatih >> > >> > On 14 Oct 2017, at 12:32, Cedric OLLIVIER <ollivier.ced...@gmail.com> >> wrote: >> > >> > Hello, >> > >> > We simply require concurrent jobs properties which can be simplified >> > here as parallel builds, concurrency control and dependency. >> > They are mainly required as: >> > - functest-core must be built before other Functest containers >> > - other containers should be built in parallel as soon as >> > functest-core is published >> > - amd64 containers and arm64 containers should be built in parallel >> > >> > I am not directly involved in Releng and I have read in reviews or >> > mails that today's jjobs don't support that. >> > All proposals to build Alpine containers haven't conformed with the >> > concurrency control and the dependencies. >> > @Fatih could you please confirm that? I'm quite sure that Jenkins supports >> them. >> > >> > In case of cross building arm64 images on amd64 PODs, we also require >> > several operations on PODs: >> > - to install qemu-user-static >> > - to support binfmt-misc [1] >> > >> > The current Docker Automated builds had been fine before we were asked >> > to build Alpine arm64 images and to publish stable tags: >> > - arm64 images can't be built via this CI tool as it requires >> > qemu-user-static and binfmt_misc [1] support on Docker hosts. >> > - publishing stable tags triggers useless builds simply because they >> > are already triggered by euphrates tags. >> > >> > I'm currently beta testing travis-ci to meet all requirements: >> > - https://travis-ci.org/collivier/functest/builds/287849046 >> > (stable/euphrates) >> > - https://travis-ci.org/collivier/functest/builds/287745681 (master) >> > >> > It works very well and all scripts are ran in parallel for all steps: >> > - build functest-core images >> > - publish functest-core manifests >> > - build all functest images >> > - publish all manifests >> > >> > I am considering we do switch from Docker Automated builds to >> > travis-ci for official Functest images if releng jjobs are not >> > updated. >> > But I think it's too late regarding the deadline for E as we should >> > multiply CI runs. @David, do you agree? >> > (Of course I am not allowed to configure travis-ci for OPNFV github >> > repositories). >> > >> > I will deeply update the wiki page "Docker Requirements on Releng" [2] >> > >> > [1] https://www.kernel.org/doc/html/v4.11/admin-guide/binfmt-misc.html >> > [2] >> > https://wiki.opnfv.org/display/functest/Docker+Requirements+on+Releng >> > >> > Cédric >> > >> > 2017-10-11 9:56 GMT+02:00 Jose Lausuch <jalaus...@suse.com>: >> >> Maybe late for 5.0, but not late for Euphrates 5.1. >> >> >> >> Can we collect a list the requirements we need from Releng in this wiki >> >> [1]? >> >> It will facilitate the support and I will help to speed it up. >> >> Otherwise, nothing will happen as people don’t know what we need. >> >> >> >> [1] >> >> https://wiki.opnfv.org/display/functest/Docker+Requirements+on+Releng >> >> >> >> >> >> >> >> >> >> >> >> >> >> On 11 Oct 2017, at 09:42, Cristina Pauna <cristina.pa...@enea.com> >> wrote: >> >> >> >> Hi Cedric, >> >> >> >> Which E are you refering to in this email? The one with deadline on >> >> 15th December? >> >> >> >> Cristina >> >> >> >> From: cedric.olliv...@orange.com [mailto:cedric.olliv...@orange.com] >> >> Sent: Wednesday, October 11, 2017 7:24 AM >> >> To: RICHOMME Morgan IMT/OLN <morgan.richo...@orange.com>; >> >> jalaus...@suse.com; Delia Popescu <delia.pope...@enea.com>; Alexandru >> >> Avadanii <alexandru.avada...@enea.com>; Cristina Pauna >> >> <cristina.pa...@enea.com>; wangwu...@huawei.com >> >> Cc: opnfv-tech-discuss <opnfv-tech-discuss@lists.opnfv.org> >> >> Subject: Re: [functest] Alpine for arch >> >> >> >> >> >> Hello, >> >> >> >> I quickly tested to build aarch64 Functest images via Docker >> >> automated builds what is impossible (several prerequisites are >> >> unmet). I precise the first published images were built locally. >> >> >> >> I'm thinking about an alternative way which will be too much >> >> disruptive for E release. Again it will be suitable for my own >> >> repositories. But releng should have been the target to build all >> >> Docker images (I bet it won't be ready for E). Today's releng can't meet >> functest prerequisites about Docker. >> >> >> >> I will inform as soon as my own repositories are ready. >> >> >> >> Cédric >> >> >> >> ---- Cristina Pauna a écrit ---- >> >> >> >> Hi, >> >> >> >> There has been a lot of confusion and changes around this topic and I >> >> want to clear things up going forward, so we do not waste any of our time. >> >> What I understand from all the disparate discussions around this topic is: >> >> 1. We will not do alpine for E0 release on arm, we are targeting >> >> E1/E2 >> >> 2. For the Functest-core image we will have 1 Dockerfile for x86, >> >> and >> >> a patch for arm that overrides this Dockerfile; from this file we >> >> will create one Functest-core image and thearchitecture will be >> >> mentioned in its tag >> >> 3. The subsequent images (Functest-healthcheck, Functest-smoke, etc) >> >> will be based on the previously built Functest-core image. We will do >> >> a manifest to choose the correct Functest-core image based on its >> >> tag. These dependent images will also have its arch in the tag. >> >> 4. The problem we are facing now is how to make sure that for 1 >> >> build, >> >> the Functest-core image always get built before the other ones. For >> >> x86 that is now done with a workaround directly in dockerhub. The >> >> target is to do it with Jenkins jobs builders, considering image >> >> dependencies. >> >> >> >> Is this the approach we are all agreeing on? >> >> >> >> Thanks, >> >> Cristina >> >> >> >> >> ________________________________________________________________ >> _____ >> >> ____________________________________________________ >> >> >> >> >> >> >> >> Ce message et ses pieces jointes peuvent contenir des informations >> >> confidentielles ou privilegiees et ne doivent donc >> >> >> >> pas etre diffuses, exploites ou copies sans autorisation. Si vous >> >> avez recu ce message par erreur, veuillez le signaler >> >> >> >> a l'expediteur et le detruire ainsi que les pieces jointes. Les >> >> messages electroniques etant susceptibles d'alteration, >> >> >> >> Orange decline toute responsabilite si ce message a ete altere, >> >> deforme ou falsifie. Merci. >> >> >> >> >> >> >> >> This message and its attachments may contain confidential or >> >> privileged information that may be protected by law; >> >> >> >> they should not be distributed, used or copied without authorisation. >> >> >> >> If you have received this email in error, please notify the sender >> >> and delete this message and its attachments. >> >> >> >> As emails may be altered, Orange is not liable for messages that have >> >> been modified, changed or falsified. >> >> >> >> Thank you. >> >> >> >> >> >> >> >> _______________________________________________ >> >> opnfv-tech-discuss mailing list >> >> opnfv-tech-discuss@lists.opnfv.org >> >> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >> >> >> > _______________________________________________ >> > opnfv-tech-discuss mailing list >> > opnfv-tech-discuss@lists.opnfv.org >> > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >> _______________________________________________ >> opnfv-tech-discuss mailing list >> opnfv-tech-discuss@lists.opnfv.org >> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
