Re: [openstack-dev] [tripleo] Updates on containerized undercloud
This is an update on what has been achieved the last month with the regard of Containerized Undercloud efforts in TripleO: ## CI - Running OVB (ovs-ha, fs001) with a containerized undercloud: it finally works, with some workarounds, all work in progress. Results can be seen here: https://logs.rdoproject.org/56/542556/79/openstack-check/ gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/ Ze1390e6e0df54a88836d75316da4b206/console.txt.gz#_2018-03-24_06_07_23_171 List of workarounds/blockers: * we need a new release of python-openstackclient that includes https://review.openstack.org/#/c/553374/ and therefore we need https://review.openstack.org/#/c/553026/ * container workflow to be finished (sbaker is on it) (in the meantime we're loading envs in quickstart). * masquerading workaround: https://review.openstack.org/#/c/553620 (long term solution will be https://review.openstack.org/#/c/553427/ but still WIP) Once we clear the workarounds/blockers and have a clean / stable deployment, we'll switch featureset001 (ovb-ha) to deploy a containerized undercloud. The target was end of rocky-m1 and we still aim for it. - Running an CI job that test upgrades from a non containerized undercloud on Queens to a containerized undercloud on Rocky. Work is in progress and can be monitored here: https://review.openstack.org/#/c/553633/ Special thanks to the Upgrade squads who help a lot on that front! ## Upgrades support We said we would provide a way to upgrade a non containerized undercloud to a containerized undercloud by rocky-m1 and we still aim for it. This is a demo of an upgrade from Queens (non containerized) to Rocky (containerized): https://youtu.be/5gLKL3YkC2c We'll wait a bit for feedback from the demo and start documenting. Note that most of the workflow remains the same as before (we still use openstack undercloud upgrade command). We'll also continue to push efforts to have this workflow tested by the CI job in progress. ## Other items - TripleO UI has been containerized. - Routed networks is still in progress by Harald (we probably aim for rocky-m2 now). - We're investigating some way to validate than an upgrade to a containerized undercloud worked fine (with Ansible?). More to come. - Containerization of Tempest so we can run Tempest against a containerized undercloud and also investigate how we could switch CI scenarios to be deployed on one node. - Port the TLS by default done in instack-undercloud. Any feedback or help on testing is very welcome. All efforts can be seen here: https://trello.com/b/nmGSNPoQ/containerized- undercloud Thanks everyone who helped in these efforts so far! On Sun, Feb 18, 2018 at 10:18 AM, Emilien Macchi wrote: > This is an update on what has been achieved this week with the regard of > Containerized Undercloud efforts in TripleO: > > TL;DR: really good efforts have been made and we can now deploy a full > (multinode) overcloud in CI. OVB testing in progress and lot of remaining > items! > > ## Bugfixes > docker-registry: add missing firewall rules - https://review.openstack. > org/#/c/545185/ > mistral-executor: mount /var/lib/mistral - https://review.openstack. > org/#/c/545143/ > docker: configure group/user for deployment_user - > https://review.openstack.org/#/c/544761/ + dependencies > Fix PublicVirtualFixedIPs in envs - https://review.openstack. > org/#/c/544744/ > Align zaqar max_messages_post_size with undercloud - > https://review.openstack.org/#/c/544756/ > undercloud_post: fix subnet name - https://review.openstack. > org/#/c/544587/ > > ## CI > We manage to run a containerized overcloud deployed by a containerized > undercloud in CI, results can be seen here: https://review. > openstack.org/#/c/542906/ > The job is running on featureser010 now (for testing purpose) but as James > mentioned in the review, we won't switch this job to run a containerized > undercloud. Note there is no impact on the job runtime. > We'll need to properly deprecate the non-containerized undercloud first > but we'll need to find a CI job that we can use for gating, so we avoid > regression during the cycle. > Now we're working on deploying featureset001 (ovb-ha), with TLS, net-iso, > Ironic/Nova/Neutron (baremetal bits) from a containerized undercloud: > https://review.openstack.org/#/c/542556/ > It's not working yet but we're working toward the blockers as they come > during testing. > > # TLS Support > All patches that were in progress have been merged, and now under testing > in ovb-ha + containerized u/c (see above). > > # UI Support > Work is still in progress, patches are ready for review, but some one them > don't pass pep8 yet. We'll hopefully fix it soon. > > # Other items > routed ctlplane networking: Harald is currently making progress on the > items, some patches are ready for review. > Create temp copy of tripleo-heat-templates before processing them: Bogdan > is working on https://review.openstack.org/#/c/542875 - the patch is > under review! > Upgrades:
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
This is an update on what has been achieved this week with the regard of Containerized Undercloud efforts in TripleO: TL;DR: really good efforts have been made and we can now deploy a full (multinode) overcloud in CI. OVB testing in progress and lot of remaining items! ## Bugfixes docker-registry: add missing firewall rules - https://review.openstack.org/#/c/545185/ mistral-executor: mount /var/lib/mistral - https://review.openstack.org/#/c/545143/ docker: configure group/user for deployment_user - https://review.openstack.org/#/c/544761/ + dependencies Fix PublicVirtualFixedIPs in envs - https://review.openstack.org/#/c/544744/ Align zaqar max_messages_post_size with undercloud - https://review.openstack.org/#/c/544756/ undercloud_post: fix subnet name - https://review.openstack.org/#/c/544587/ ## CI We manage to run a containerized overcloud deployed by a containerized undercloud in CI, results can be seen here: https://review.openstack.org/#/c/542906/ The job is running on featureser010 now (for testing purpose) but as James mentioned in the review, we won't switch this job to run a containerized undercloud. Note there is no impact on the job runtime. We'll need to properly deprecate the non-containerized undercloud first but we'll need to find a CI job that we can use for gating, so we avoid regression during the cycle. Now we're working on deploying featureset001 (ovb-ha), with TLS, net-iso, Ironic/Nova/Neutron (baremetal bits) from a containerized undercloud: https://review.openstack.org/#/c/542556/ It's not working yet but we're working toward the blockers as they come during testing. # TLS Support All patches that were in progress have been merged, and now under testing in ovb-ha + containerized u/c (see above). # UI Support Work is still in progress, patches are ready for review, but some one them don't pass pep8 yet. We'll hopefully fix it soon. # Other items routed ctlplane networking: Harald is currently making progress on the items, some patches are ready for review. Create temp copy of tripleo-heat-templates before processing them: Bogdan is working on https://review.openstack.org/#/c/542875 - the patch is under review! Upgrades: no work has been started so far but we'll probably discuss about this topic during the PTG. As usual please comment or add anything that I missed. Thanks all for your help/reviews/efforts so far, Emilien On Tue, Feb 13, 2018 at 6:41 AM, Emilien Macchi wrote: > > > On Tue, Feb 13, 2018 at 2:40 AM, Harald Jensås wrote: > >> On Fri, 2018-02-09 at 14:39 -0800, Emilien Macchi wrote: >> > On Fri, Feb 9, 2018 at 2:30 PM, James Slagle >> > wrote: >> > [...] >> > >> > > You may want to add an item for the routed ctlplane work that >> > > landed >> > > at the end of Queens. Afaik, that will need to be supported with >> > > the >> > > containerized undercloud. >> > >> > Done: https://trello.com/c/kFtIkto1/17-routed-ctlplane-networking >> > >> >> Tanks Emilien, >> >> >> I added several work items to the Trello card, and a few patches. Still >> WiP. >> >> Do we have any CI that use containerized undercloud with actual Ironic >> deployement? Or are they all using deployed-server? >> >> E.g do we have anything actually testing this type of change? >>https://review.openstack.org/#/c/543582 >> >> I belive that would have to be an ovb job with containerized undercloud? >> > > I'm working on it since last week: https://trello.com/c/ > uLqbHTip/13-switch-other-jobs-to-run-a-containerized-undercloud > But currently trying to make things stable again, we introduce regressions > and this is high prio now. > -- > Emilien Macchi > -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
On Tue, Feb 13, 2018 at 2:40 AM, Harald Jensås wrote: > On Fri, 2018-02-09 at 14:39 -0800, Emilien Macchi wrote: > > On Fri, Feb 9, 2018 at 2:30 PM, James Slagle > > wrote: > > [...] > > > > > You may want to add an item for the routed ctlplane work that > > > landed > > > at the end of Queens. Afaik, that will need to be supported with > > > the > > > containerized undercloud. > > > > Done: https://trello.com/c/kFtIkto1/17-routed-ctlplane-networking > > > > Tanks Emilien, > > > I added several work items to the Trello card, and a few patches. Still > WiP. > > Do we have any CI that use containerized undercloud with actual Ironic > deployement? Or are they all using deployed-server? > > E.g do we have anything actually testing this type of change? >https://review.openstack.org/#/c/543582 > > I belive that would have to be an ovb job with containerized undercloud? > I'm working on it since last week: https://trello.com/c/uLqbHTip/13-switch-other-jobs-to-run-a-containerized-undercloud But currently trying to make things stable again, we introduce regressions and this is high prio now. -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
On Fri, 2018-02-09 at 14:39 -0800, Emilien Macchi wrote: > On Fri, Feb 9, 2018 at 2:30 PM, James Slagle > wrote: > [...] > > > You may want to add an item for the routed ctlplane work that > > landed > > at the end of Queens. Afaik, that will need to be supported with > > the > > containerized undercloud. > > Done: https://trello.com/c/kFtIkto1/17-routed-ctlplane-networking > Tanks Emilien, I added several work items to the Trello card, and a few patches. Still WiP. Do we have any CI that use containerized undercloud with actual Ironic deployement? Or are they all using deployed-server? E.g do we have anything actually testing this type of change? https://review.openstack.org/#/c/543582 I belive that would have to be an ovb job with containerized undercloud? -- | Harald Jensås | hjensas:irc __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
On Fri, Feb 9, 2018 at 2:30 PM, James Slagle wrote: [...] You may want to add an item for the routed ctlplane work that landed > at the end of Queens. Afaik, that will need to be supported with the > containerized undercloud. > Done: https://trello.com/c/kFtIkto1/17-routed-ctlplane-networking Thanks, -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
On Fri, Feb 9, 2018 at 5:02 PM, Emilien Macchi wrote: > Quite a lot of progress has been made over the last months (and days), so I > found useful to share an update on where we are with the efforts on > containerized undercloud. > > ## CI efforts > > - tripleo-ci-centos-7-undercloud-containers job has been reworked to use the > "undercloud install" interface. Job is green and will probably start voting > during the next days. See https://review.openstack.org/#/c/517445/ and > https://review.openstack.org/#/c/517444/. > - we're looking at switching some other CI jobs (maybe one to start) to > deploy a containerized undercloud, and then deploy an overcloud (probably > featureset010). We have a few blockers but we're working on it. It's mostly > the overcloud_prep that fails: > http://logs.openstack.org/06/542906/1/check/tripleo-ci-centos-7-containers-multinode/0b18c49/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-02-09_17_07_27 > - because of that effort, we're also taking an opportunity to refactor > tripleo-quickstart-extras roles to be more "standard" for both undercloud > and overcloud (example, renaming overcloud-prep-containers to > prep-containers, etc). We'll need help from TripleO CI squad probably. > - Chandan will help us to run validate-tempest in > tripleo-ci-centos-7-undercloud-containers so we can have some actual testing > for this job, since no overcloud is deployed. > > ## Feature parity > > - TLS work is ongoing. > - TripleO UI containerization is ongoing. > - Nova join support is targeted for Rocky > - Upgrade workflow is under investigation. We'll work on re-using the > upgrade_tasks in THT to upgrade a non-containerized undercloud (Queens) to a > containerized undercloud (Rocky) like we did between Ocata and Pike with the > upgrade_tasks. We'll actually re-use the same code but will have to change > the undercloud upgrade workflow in tripleoclient. You may want to add an item for the routed ctlplane work that landed at the end of Queens. Afaik, that will need to be supported with the containerized undercloud. -- -- James Slagle -- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Updates on containerized undercloud
On Fri, Feb 9, 2018 at 2:02 PM, Emilien Macchi wrote: > Quite a lot of progress has been made over the last months (and days), so > I found useful to share an update on where we are with the efforts on > containerized undercloud. > > ## CI efforts > > - tripleo-ci-centos-7-undercloud-containers job has been reworked to use > the "undercloud install" interface. Job is green and will probably start > voting during the next days. See https://review.openstack.org/#/c/517445/ > and https://review.openstack.org/#/c/517444/. > - we're looking at switching some other CI jobs (maybe one to start) to > deploy a containerized undercloud, and then deploy an overcloud (probably > featureset010). We have a few blockers but we're working on it. It's mostly > the overcloud_prep that fails: > http://logs.openstack.org/06/542906/1/check/tripleo-ci- > centos-7-containers-multinode/0b18c49/logs/undercloud/home/ > zuul/overcloud_prep_containers.log.txt.gz#_2018-02-09_17_07_27 > - because of that effort, we're also taking an opportunity to refactor > tripleo-quickstart-extras roles to be more "standard" for both undercloud > and overcloud (example, renaming overcloud-prep-containers to > prep-containers, etc). We'll need help from TripleO CI squad probably. > - Chandan will help us to run validate-tempest in > tripleo-ci-centos-7-undercloud-containers > so we can have some actual testing for this job, since no overcloud is > deployed. > > ## Feature parity > > - TLS work is ongoing. > - TripleO UI containerization is ongoing. > - Nova join support is targeted for Rocky > - Upgrade workflow is under investigation. We'll work on re-using the > upgrade_tasks in THT to upgrade a non-containerized undercloud (Queens) to > a containerized undercloud (Rocky) like we did between Ocata and Pike with > the upgrade_tasks. We'll actually re-use the same code but will have to > change the undercloud upgrade workflow in tripleoclient. > > That highlights the current efforts, if you have any question, need more > specific or just any feedback, please go ahead. > At the PTG, we'll discuss about some technical details and hope to move > forward with this nice feature during Rocky cycle. > > Thanks, > -- > Emilien Macchi > I forgot to mention, but people working on this topic have been using Trello to collaborate: https://trello.com/b/nmGSNPoQ/containerized-undercloud To keep things in the open, here's the link and anyone is free to participate. Thanks, -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Updates on containerized undercloud
Quite a lot of progress has been made over the last months (and days), so I found useful to share an update on where we are with the efforts on containerized undercloud. ## CI efforts - tripleo-ci-centos-7-undercloud-containers job has been reworked to use the "undercloud install" interface. Job is green and will probably start voting during the next days. See https://review.openstack.org/#/c/517445/ and https://review.openstack.org/#/c/517444/. - we're looking at switching some other CI jobs (maybe one to start) to deploy a containerized undercloud, and then deploy an overcloud (probably featureset010). We have a few blockers but we're working on it. It's mostly the overcloud_prep that fails: http://logs.openstack.org/06/542906/1/check/tripleo-ci-centos-7-containers-multinode/0b18c49/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-02-09_17_07_27 - because of that effort, we're also taking an opportunity to refactor tripleo-quickstart-extras roles to be more "standard" for both undercloud and overcloud (example, renaming overcloud-prep-containers to prep-containers, etc). We'll need help from TripleO CI squad probably. - Chandan will help us to run validate-tempest in tripleo-ci-centos-7-undercloud-containers so we can have some actual testing for this job, since no overcloud is deployed. ## Feature parity - TLS work is ongoing. - TripleO UI containerization is ongoing. - Nova join support is targeted for Rocky - Upgrade workflow is under investigation. We'll work on re-using the upgrade_tasks in THT to upgrade a non-containerized undercloud (Queens) to a containerized undercloud (Rocky) like we did between Ocata and Pike with the upgrade_tasks. We'll actually re-use the same code but will have to change the undercloud upgrade workflow in tripleoclient. That highlights the current efforts, if you have any question, need more specific or just any feedback, please go ahead. At the PTG, we'll discuss about some technical details and hope to move forward with this nice feature during Rocky cycle. Thanks, -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
