Re: [OpenStack-Infra] About aarch64 third party CI
On 13 June 2017 at 01:00, Ricardo Carrillo Cruz < ricardo.carrillo.c...@gmail.com> wrote: > > > 2017-06-09 22:18 GMT+02:00 Paul Belanger : > >> On Fri, Jun 09, 2017 at 07:58:44PM +, Jeremy Stanley wrote: >> > On 2017-06-07 14:26:10 +0800 (+0800), Xinliang Liu wrote: >> > [...] >> > > we already have our own pre-built debian cloud image, could I just >> > > use it and not use the one built by diskimage-builder? >> > [...] >> > >> > The short answer is that nodepool doesn't currently have support for >> > directly using an image provided independent of its own image build >> > process. Clark was suggesting[*] in IRC today that it might be >> > possible to inject records into Zookeeper (acting as a "fake" >> > nodepool-builder daemon basically) to accomplish this, but nobody >> > has yet implemented such a solution to our knowledge. >> > >> > Longer term, I think we do want a feature in nodepool to be able to >> > specify the ID of a prebuilt image for a label/provider (at least we >> > discussed that we wouldn't reject the idea if someone proposed a >> > suitable implementation). Just be aware that nodepool's use of >> > diskimage-builder to regularly rebuild images is intentional and >> > useful since it ensures images are updated with the latest packages, >> > kernels, warm caches and whatever else you specify in your elements >> > so reducing job runtimes as they spend less effort updating these >> > things on every run. >> > >> > [*] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/% >> 23openstack-infra.2017-06-09.log.html#t2017-06-09T15:32:27-2 > >> > -- >> > Jeremy Stanley >> >> Actually, I think 458073[1] aims to fix this use case. I haven't tired it >> myself but it adds support for using images which are not built and >> managed by >> nodepool. >> >> This is currently only on feature/zuulv3 branch. >> >> [1] https://review.openstack.org/#/c/458073/ >> >> > That's right, support for cloud-images on feature/zuulv3 is now merged and > working. > I just setup a Nodepool using this new feature over the weekend. > > This is a nodepool.yaml that can help you get going: > > http://paste.openstack.org/show/612191/ > Great! I will try and see what happen. Thanks, -xinliang > > > HTH > > >> > ___ >> > OpenStack-Infra mailing list >> > OpenStack-Infra@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >> >> >> ___ >> OpenStack-Infra mailing list >> OpenStack-Infra@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About aarch64 third party CI
Hi Jeremy, Thanks for reply;-) On 10 June 2017 at 03:58, Jeremy Stanley wrote: > On 2017-06-07 14:26:10 +0800 (+0800), Xinliang Liu wrote: > [...] > > we already have our own pre-built debian cloud image, could I just > > use it and not use the one built by diskimage-builder? > [...] > > The short answer is that nodepool doesn't currently have support for > directly using an image provided independent of its own image build > process. Clark was suggesting[*] in IRC today that it might be > possible to inject records into Zookeeper (acting as a "fake" > nodepool-builder daemon basically) to accomplish this, but nobody > has yet implemented such a solution to our knowledge. > Got it, thanks. > > Longer term, I think we do want a feature in nodepool to be able to > specify the ID of a prebuilt image for a label/provider (at least we > discussed that we wouldn't reject the idea if someone proposed a > suitable implementation). Just be aware that nodepool's use of > diskimage-builder to regularly rebuild images is intentional and > useful since it ensures images are updated with the latest packages, > I think this is the point that make a really update gating image to run the tests. > kernels, warm caches and whatever else you specify in your elements > so reducing job runtimes as they spend less effort updating these > things on every run. > How often nodepool will rebuild the images? If it too frequency every day. Shall we just make a job to publish the pre-built gating images every day then other CI test just use them (something like docker image, though it is container image)? You know making a gating image need to include lots of elements, even though with warm caches, when using diskimage-builder we still need to rebuild step by step. What I mean is that building image is taking a bit time. Thanks, -xinliang > > [*] http://eavesdrop.openstack.org/irclogs/%23openstack- > infra/%23openstack-infra.2017-06-09.log.html#t2017-06-09T15:32:27-2 > > -- > Jeremy Stanley > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About aarch64 third party CI
Ricardo Carrillo Cruz writes: > This is a nodepool.yaml that can help you get going: > > http://paste.openstack.org/show/612191/ Glad it worked! You can drop 'zmq-publishers' from the config entirely. If 'images-dir' and 'diskimages' are required, then I would consider that a bug; we should have default values for those so you don't need to provide them in this case. That config snippet also illustrates something I didn't quite realize at the time I reviewed https://review.openstack.org/472959. I don't think we should be using UUIDs as keys in nodepool because they are hard for humans to distinguish from each other. It could make for somewhat error-prone configuration. So instead of: cloud-images: - name: 9e884aab-a46e-46de-b57c-a044da0f45cd pools: - name: main labels: - name: xenial cloud-image: 9e884aab-a46e-46de-b57c-a044da0f45cd If someone wants to specify an image by id, we should have: cloud-images: - name: mycloudimagename id: 9e884aab-a46e-46de-b57c-a044da0f45cd pools: - name: main labels: - name: xenial cloud-image: mycloudimagename And then if you omit the 'id' field, we should just implicitly use 'name' as before. This way it's easy to see which of several cloud-images a label uses, and, when it's time to update the UUID for that cloud image, that only needs to happen in one place. -Jim ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About aarch64 third party CI
2017-06-09 22:18 GMT+02:00 Paul Belanger : > On Fri, Jun 09, 2017 at 07:58:44PM +, Jeremy Stanley wrote: > > On 2017-06-07 14:26:10 +0800 (+0800), Xinliang Liu wrote: > > [...] > > > we already have our own pre-built debian cloud image, could I just > > > use it and not use the one built by diskimage-builder? > > [...] > > > > The short answer is that nodepool doesn't currently have support for > > directly using an image provided independent of its own image build > > process. Clark was suggesting[*] in IRC today that it might be > > possible to inject records into Zookeeper (acting as a "fake" > > nodepool-builder daemon basically) to accomplish this, but nobody > > has yet implemented such a solution to our knowledge. > > > > Longer term, I think we do want a feature in nodepool to be able to > > specify the ID of a prebuilt image for a label/provider (at least we > > discussed that we wouldn't reject the idea if someone proposed a > > suitable implementation). Just be aware that nodepool's use of > > diskimage-builder to regularly rebuild images is intentional and > > useful since it ensures images are updated with the latest packages, > > kernels, warm caches and whatever else you specify in your elements > > so reducing job runtimes as they spend less effort updating these > > things on every run. > > > > [*] http://eavesdrop.openstack.org/irclogs/%23openstack- > infra/%23openstack-infra.2017-06-09.log.html#t2017-06-09T15:32:27-2 > > > -- > > Jeremy Stanley > > Actually, I think 458073[1] aims to fix this use case. I haven't tired it > myself but it adds support for using images which are not built and > managed by > nodepool. > > This is currently only on feature/zuulv3 branch. > > [1] https://review.openstack.org/#/c/458073/ > > That's right, support for cloud-images on feature/zuulv3 is now merged and working. I just setup a Nodepool using this new feature over the weekend. This is a nodepool.yaml that can help you get going: http://paste.openstack.org/show/612191/ HTH > > ___ > > OpenStack-Infra mailing list > > OpenStack-Infra@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About aarch64 third party CI
On Fri, Jun 09, 2017 at 07:58:44PM +, Jeremy Stanley wrote: > On 2017-06-07 14:26:10 +0800 (+0800), Xinliang Liu wrote: > [...] > > we already have our own pre-built debian cloud image, could I just > > use it and not use the one built by diskimage-builder? > [...] > > The short answer is that nodepool doesn't currently have support for > directly using an image provided independent of its own image build > process. Clark was suggesting[*] in IRC today that it might be > possible to inject records into Zookeeper (acting as a "fake" > nodepool-builder daemon basically) to accomplish this, but nobody > has yet implemented such a solution to our knowledge. > > Longer term, I think we do want a feature in nodepool to be able to > specify the ID of a prebuilt image for a label/provider (at least we > discussed that we wouldn't reject the idea if someone proposed a > suitable implementation). Just be aware that nodepool's use of > diskimage-builder to regularly rebuild images is intentional and > useful since it ensures images are updated with the latest packages, > kernels, warm caches and whatever else you specify in your elements > so reducing job runtimes as they spend less effort updating these > things on every run. > > [*] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-06-09.log.html#t2017-06-09T15:32:27-2 > > > -- > Jeremy Stanley Actually, I think 458073[1] aims to fix this use case. I haven't tired it myself but it adds support for using images which are not built and managed by nodepool. This is currently only on feature/zuulv3 branch. [1] https://review.openstack.org/#/c/458073/ > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] About aarch64 third party CI
On 2017-06-07 14:26:10 +0800 (+0800), Xinliang Liu wrote: [...] > we already have our own pre-built debian cloud image, could I just > use it and not use the one built by diskimage-builder? [...] The short answer is that nodepool doesn't currently have support for directly using an image provided independent of its own image build process. Clark was suggesting[*] in IRC today that it might be possible to inject records into Zookeeper (acting as a "fake" nodepool-builder daemon basically) to accomplish this, but nobody has yet implemented such a solution to our knowledge. Longer term, I think we do want a feature in nodepool to be able to specify the ID of a prebuilt image for a label/provider (at least we discussed that we wouldn't reject the idea if someone proposed a suitable implementation). Just be aware that nodepool's use of diskimage-builder to regularly rebuild images is intentional and useful since it ensures images are updated with the latest packages, kernels, warm caches and whatever else you specify in your elements so reducing job runtimes as they spend less effort updating these things on every run. [*] http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-06-09.log.html#t2017-06-09T15:32:27-2 > -- Jeremy Stanley signature.asc Description: Digital signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] About aarch64 third party CI
Hi, This is xinliang from Linaro. I am now setting up the aarch64 third party CI in an aarch64 cloud. I create two ubunu16.04 vms, one for CI server and one for log server. And follow the steps from here[1] I am now at the step: start nodepool I realize that nodepool use diskimage-builder to build devstack gating images. I now manage to modify diskimage-builder to build a debian-minimal image and trying to make a gating image this means add more elements into the image. The question is that we already have our own pre-built debian cloud image, could I just use it and not use the one built by diskimage-builder? [1] https://docs.openstack.org/infra/openstackci/third_party_ci.html. [2] https://github.com/openstack/diskimage-builder Best, -xinliang ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
