Re: [openstack-dev] [tripleo] Updates from the containers squad: Where are we at?

2017-01-31 Thread Flavio Percoco 

On 31/01/17 17:17 +, Fox, Kevin M wrote:

Great to hear the containerization process is going well. :)

Can we set aside some time during the PTG to talk about the process of 
detecting stale/building fresh containers at the PTG? Its a problem that all 
the kolla deliverables would also like solved. So maybe we can work together on 
it?



Absolutely, I saw you added a topic similar to this in the tripleo agenda. I
think it'd be awesome to sit and talk about this.

Flavio


Thanks,
Kevin

From: Flavio Percoco [fla...@redhat.com]
Sent: Tuesday, January 31, 2017 8:13 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [tripleo] Updates from the containers squad: Where 
are we at?

Greetings,

In the last month, there’s been no update on what’s been happening in the
containers effort for TripleO so I thought I’d drop an email to… well… provide
updates.

The work in the DFG has been broken down into more granular steps that are
easier to account for. However, I’d like to focus a bit more on the architecture
side of things first.

There have been some major changes in how containers are deployed within TripleO
and how services are containerized. I’ll try to describe those below but bear in
mind that some details may change a little bit and that this is still a work in
progress.

* The new architecture aims to support a hybrid deployment where it’s possible
 to run some services on baremetal and others on containers. This has been done
 to ease the upgrade process from baremetal nodes to a containerize node and it
 doesn’t change the goal of having all services containerized. More details
 about this new architecture can be found here[0]. Please, check out the README
 file.

* There’s no heat agent container anymore. Instead, we’re installing puppet in
 the base image for every service and then run puppet from the service
 container. A bit of more details about this change can be found in this blog
 post[0] (Thanks Dan)

* The old copy-json file that used to create the config.json for the kolla image
 was replaced with a heat-hook. More about this on this patch.[2]

* Several patches for containerizing services have been proposed. TBH, we’ve not
 done a great job at using a single topic BUT if you open the patch[0], you’ll
 see a list of patches depending on it that propose the containerization of
 several projects.

* There’s a non-voting CI job that currently only runs on patches that modify
 files related to the containers work. You can read more about this on this
 patch[3]. The goal is to make this job voting as soon as Pike starts.

* Perhaps more RDO specific but there’s also been great progress in the building
 pipeline for containers. We’re using kolla images as our base container images
 and the image build process is being integrated with the TripleO CI pipeline.
 This means that new images will be created whenever there’s a promotion in
 TripleO. We’d like to take this one step further for Pike and be able to build
 images on every patch merged in the various projects, just like we’d do with
 RPM packages. I wanted to mention this because I love the fact that this helps
 us testing kolla images thoroughly and also give back to the community in some
 way. More on this can be found here[4]

* Support for a containerized deployment has been added to tripleo-quickstart
 and it’s being used for the CI job.

* The containers squad will start the work of documenting the current
 architecture as soon as possible. As mentioned in the points above, there have
 been several changes to the architecture that made starting the documentation
 process not worthwhile. We can start documenting things now that we feel more
 comfortable with the current version of the framework.

* We’re starting to work on a strategy for upgrades for baremetal nodes to
 containerized nodes. Again, this work was waiting for the architecture to be
 polished out a bit more and the composable upgrades work to mature. Now it
 seems like a good time to kick this off.

Eventually (and likely soon) this will impact other squads and there will be
need of a joint work towards finalizing the containerization process. More
details will be provided as part of the documentation and in future emails.

There are still many things to do, though. There are services that still need to
be containerized, there are more CI jobs to be added and all the services that
have been containerized so far must also be tested in the overcloud. That being
said, I’m personally happy with the progress made so far and I hope you are as
well and that this email is useful to you.

Rock on,
Flavio

[0] https://review.openstack.org/#/c/416421/
[1] https://dprince.github.io/docker-puppet.html
[2] https://review.openstack.org/#/c/416420/
[3] https://review.openstack.org/#/c/423519/
[4] https://etherpad.openstack.org/p/rdo-kolla-build

--
@flaper87
Flavio Percoco

Re: [openstack-dev] [tripleo] Updates from the containers squad: Where are we at?

2017-01-31 Thread Emilien Macchi 
On Tue, Jan 31, 2017 at 11:13 AM, Flavio Percoco  wrote:
> Greetings,
>
> In the last month, there’s been no update on what’s been happening in the
> containers effort for TripleO so I thought I’d drop an email to… well…
> provide
> updates.
>
> The work in the DFG has been broken down into more granular steps that are
> easier to account for. However, I’d like to focus a bit more on the
> architecture
> side of things first.
>
> There have been some major changes in how containers are deployed within
> TripleO
> and how services are containerized. I’ll try to describe those below but
> bear in
> mind that some details may change a little bit and that this is still a work
> in
> progress.
>
> * The new architecture aims to support a hybrid deployment where it’s
> possible
>  to run some services on baremetal and others on containers. This has been
> done
>  to ease the upgrade process from baremetal nodes to a containerize node and
> it
>  doesn’t change the goal of having all services containerized. More details
>  about this new architecture can be found here[0]. Please, check out the
> README
>  file.
>  * There’s no heat agent container anymore. Instead, we’re installing puppet
> in
>  the base image for every service and then run puppet from the service
>  container. A bit of more details about this change can be found in this
> blog
>  post[0] (Thanks Dan)
>  * The old copy-json file that used to create the config.json for the kolla
> image
>  was replaced with a heat-hook. More about this on this patch.[2]
>  * Several patches for containerizing services have been proposed. TBH,
> we’ve not
>  done a great job at using a single topic BUT if you open the patch[0],
> you’ll
>  see a list of patches depending on it that propose the containerization of
>  several projects.
>  * There’s a non-voting CI job that currently only runs on patches that
> modify
>  files related to the containers work. You can read more about this on this
>  patch[3]. The goal is to make this job voting as soon as Pike starts.

Excellent plan and it's aligned with our TripleO CI with oooq roadmap.

>  * Perhaps more RDO specific but there’s also been great progress in the
> building
>  pipeline for containers. We’re using kolla images as our base container
> images
>  and the image build process is being integrated with the TripleO CI
> pipeline.
>  This means that new images will be created whenever there’s a promotion in
>  TripleO. We’d like to take this one step further for Pike and be able to
> build
>  images on every patch merged in the various projects, just like we’d do
> with
>  RPM packages. I wanted to mention this because I love the fact that this
> helps
>  us testing kolla images thoroughly and also give back to the community in
> some
>  way. More on this can be found here[4]
>  * Support for a containerized deployment has been added to
> tripleo-quickstart
>  and it’s being used for the CI job.
>  * The containers squad will start the work of documenting the current
>  architecture as soon as possible. As mentioned in the points above, there
> have
>  been several changes to the architecture that made starting the
> documentation
>  process not worthwhile. We can start documenting things now that we feel
> more
>  comfortable with the current version of the framework.
>  * We’re starting to work on a strategy for upgrades for baremetal nodes to
>  containerized nodes. Again, this work was waiting for the architecture to
> be
>  polished out a bit more and the composable upgrades work to mature. Now it
>  seems like a good time to kick this off.
>
> Eventually (and likely soon) this will impact other squads and there will be
> need of a joint work towards finalizing the containerization process. More
> details will be provided as part of the documentation and in future emails.
>
> There are still many things to do, though. There are services that still
> need to
> be containerized, there are more CI jobs to be added and all the services
> that
> have been containerized so far must also be tested in the overcloud. That
> being
> said, I’m personally happy with the progress made so far and I hope you are
> as
> well and that this email is useful to you.

Excellent summary Flavio, I wish you could send more updates on $topic :-)
It has been very useful so far.

Thanks,

> Rock on,
> Flavio
>
> [0] https://review.openstack.org/#/c/416421/
> [1] https://dprince.github.io/docker-puppet.html
> [2] https://review.openstack.org/#/c/416420/
> [3] https://review.openstack.org/#/c/423519/
> [4] https://etherpad.openstack.org/p/rdo-kolla-build
>
> --
> @flaper87
> Flavio Percoco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

Re: [openstack-dev] [tripleo] Updates from the containers squad: Where are we at?

2017-01-31 Thread Fox, Kevin M 
Great to hear the containerization process is going well. :)

Can we set aside some time during the PTG to talk about the process of 
detecting stale/building fresh containers at the PTG? Its a problem that all 
the kolla deliverables would also like solved. So maybe we can work together on 
it?

Thanks,
Kevin

From: Flavio Percoco [fla...@redhat.com]
Sent: Tuesday, January 31, 2017 8:13 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [tripleo] Updates from the containers squad: Where 
are we at?

Greetings,

In the last month, there’s been no update on what’s been happening in the
containers effort for TripleO so I thought I’d drop an email to… well… provide
updates.

The work in the DFG has been broken down into more granular steps that are
easier to account for. However, I’d like to focus a bit more on the architecture
side of things first.

There have been some major changes in how containers are deployed within TripleO
and how services are containerized. I’ll try to describe those below but bear in
mind that some details may change a little bit and that this is still a work in
progress.

* The new architecture aims to support a hybrid deployment where it’s possible
  to run some services on baremetal and others on containers. This has been done
  to ease the upgrade process from baremetal nodes to a containerize node and it
  doesn’t change the goal of having all services containerized. More details
  about this new architecture can be found here[0]. Please, check out the README
  file.

* There’s no heat agent container anymore. Instead, we’re installing puppet in
  the base image for every service and then run puppet from the service
  container. A bit of more details about this change can be found in this blog
  post[0] (Thanks Dan)

* The old copy-json file that used to create the config.json for the kolla image
  was replaced with a heat-hook. More about this on this patch.[2]

* Several patches for containerizing services have been proposed. TBH, we’ve not
  done a great job at using a single topic BUT if you open the patch[0], you’ll
  see a list of patches depending on it that propose the containerization of
  several projects.

* There’s a non-voting CI job that currently only runs on patches that modify
  files related to the containers work. You can read more about this on this
  patch[3]. The goal is to make this job voting as soon as Pike starts.

* Perhaps more RDO specific but there’s also been great progress in the building
  pipeline for containers. We’re using kolla images as our base container images
  and the image build process is being integrated with the TripleO CI pipeline.
  This means that new images will be created whenever there’s a promotion in
  TripleO. We’d like to take this one step further for Pike and be able to build
  images on every patch merged in the various projects, just like we’d do with
  RPM packages. I wanted to mention this because I love the fact that this helps
  us testing kolla images thoroughly and also give back to the community in some
  way. More on this can be found here[4]

* Support for a containerized deployment has been added to tripleo-quickstart
  and it’s being used for the CI job.

* The containers squad will start the work of documenting the current
  architecture as soon as possible. As mentioned in the points above, there have
  been several changes to the architecture that made starting the documentation
  process not worthwhile. We can start documenting things now that we feel more
  comfortable with the current version of the framework.

* We’re starting to work on a strategy for upgrades for baremetal nodes to
  containerized nodes. Again, this work was waiting for the architecture to be
  polished out a bit more and the composable upgrades work to mature. Now it
  seems like a good time to kick this off.

Eventually (and likely soon) this will impact other squads and there will be
need of a joint work towards finalizing the containerization process. More
details will be provided as part of the documentation and in future emails.

There are still many things to do, though. There are services that still need to
be containerized, there are more CI jobs to be added and all the services that
have been containerized so far must also be tested in the overcloud. That being
said, I’m personally happy with the progress made so far and I hope you are as
well and that this email is useful to you.

Rock on,
Flavio

[0] https://review.openstack.org/#/c/416421/
[1] https://dprince.github.io/docker-puppet.html
[2] https://review.openstack.org/#/c/416420/
[3] https://review.openstack.org/#/c/423519/
[4] https://etherpad.openstack.org/p/rdo-kolla-build

--
@flaper87
Flavio Percoco

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org

[openstack-dev] [tripleo] Updates from the containers squad: Where are we at?

2017-01-31 Thread Flavio Percoco 

Greetings,

In the last month, there’s been no update on what’s been happening in the
containers effort for TripleO so I thought I’d drop an email to… well… provide
updates.

The work in the DFG has been broken down into more granular steps that are
easier to account for. However, I’d like to focus a bit more on the architecture
side of things first.

There have been some major changes in how containers are deployed within TripleO
and how services are containerized. I’ll try to describe those below but bear in
mind that some details may change a little bit and that this is still a work in
progress.

* The new architecture aims to support a hybrid deployment where it’s possible
 to run some services on baremetal and others on containers. This has been done
 to ease the upgrade process from baremetal nodes to a containerize node and it
 doesn’t change the goal of having all services containerized. More details
 about this new architecture can be found here[0]. Please, check out the README
 file.
 
* There’s no heat agent container anymore. Instead, we’re installing puppet in

 the base image for every service and then run puppet from the service
 container. A bit of more details about this change can be found in this blog
 post[0] (Thanks Dan)
 
* The old copy-json file that used to create the config.json for the kolla image

 was replaced with a heat-hook. More about this on this patch.[2]
 
* Several patches for containerizing services have been proposed. TBH, we’ve not

 done a great job at using a single topic BUT if you open the patch[0], you’ll
 see a list of patches depending on it that propose the containerization of
 several projects.
 
* There’s a non-voting CI job that currently only runs on patches that modify

 files related to the containers work. You can read more about this on this
 patch[3]. The goal is to make this job voting as soon as Pike starts.
 
* Perhaps more RDO specific but there’s also been great progress in the building

 pipeline for containers. We’re using kolla images as our base container images
 and the image build process is being integrated with the TripleO CI pipeline.
 This means that new images will be created whenever there’s a promotion in
 TripleO. We’d like to take this one step further for Pike and be able to build
 images on every patch merged in the various projects, just like we’d do with
 RPM packages. I wanted to mention this because I love the fact that this helps
 us testing kolla images thoroughly and also give back to the community in some
 way. More on this can be found here[4]
 
* Support for a containerized deployment has been added to tripleo-quickstart

 and it’s being used for the CI job.
 
* The containers squad will start the work of documenting the current

 architecture as soon as possible. As mentioned in the points above, there have
 been several changes to the architecture that made starting the documentation
 process not worthwhile. We can start documenting things now that we feel more
 comfortable with the current version of the framework.
 
* We’re starting to work on a strategy for upgrades for baremetal nodes to

 containerized nodes. Again, this work was waiting for the architecture to be
 polished out a bit more and the composable upgrades work to mature. Now it
 seems like a good time to kick this off.

Eventually (and likely soon) this will impact other squads and there will be
need of a joint work towards finalizing the containerization process. More
details will be provided as part of the documentation and in future emails.

There are still many things to do, though. There are services that still need to
be containerized, there are more CI jobs to be added and all the services that
have been containerized so far must also be tested in the overcloud. That being
said, I’m personally happy with the progress made so far and I hope you are as
well and that this email is useful to you.

Rock on,
Flavio

[0] https://review.openstack.org/#/c/416421/
[1] https://dprince.github.io/docker-puppet.html
[2] https://review.openstack.org/#/c/416420/
[3] https://review.openstack.org/#/c/423519/
[4] https://etherpad.openstack.org/p/rdo-kolla-build

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev