Re: [openstack-dev] [TripleO] Passing along some field feedback

[Alex Schultz]

On Fri, May 26, 2017 at 3:34 PM, Ben Nemec  wrote:
> Hi,
>
> As some of you may know, I have a weekly meeting with some of the Red Hat
> folks who are out in the field using TripleO to deploy OpenStack at customer
> sites.  The past couple of weeks have been particularly interesting, so I
> thought I'd take a moment to disseminate the comments to the broader
> developer community.  I'll try to keep it from getting too wall-of-texty,
> but no promises. :-)
>
> Here goes:
>
> * There was interest in using split stack to avoid having to use TripleO's
> baremetal provisioning.  I let them know that technically split stack was
> possible in Newton and that further usability improvements were coming in
> Ocata and Pike.  Keep in mind that for many of these companies Newton is
> still a new shiny thing that they're looking to move onto, so even though
> this functionality seems like it's been around for a long time to us that
> isn't necessarily the case in the field.
>
> * Some concerns around managing config with TripleO were raised,
> specifically two points: 1) Stack updates can cause service outages, even if
> you're just looking to change one minor config value in a single service.
> 2) Stack updates take a long time.  45 minutes to apply one config value
> seems a little heavy from a user perspective.
>

Yea we keep trying to work on 1 and I think we're in a good spot for
that (at least better than previous releases). But it's also on
developers to ensure when you add/change when something is run that
you are doing it in the right step to make sure configurations don't
get removed/re-added on updates. This is particularly bad with
services that are fronted by apache as that occurs in Step 3.  If you
add your service in Step 4, on updates it's removed from the
configuration (service restart) in step 3 and then reapplied in step 4
(service restart).

> For 1, I did mention that we had been working to reduce the number of
> spurious service restarts on update.  2 is obviously a little trickier. One
> thing that was requested was a mode where a Puppet step doesn't get run
> unless something changes - which is the exact opposite of the feedback we
> got initially on TripleO.  Back then people wanted a stack update to assert
> state every time, like regular Puppet would.  There probably weren't enough
> people in this discussion to take immediate action on this, but I thought it
> was an interesting potential change in philosophy.  Also it drives home the
> need for better performance of TripleO as a whole.
>

So the problem with the 'only run if something changes' is that it
assumes nothing else has changed outside of the configuration managed
by tripleo. And as we all know, people login to servers and mess with
settings and then forgot they left them like that.  The only way we
can be assured the state is correct is to go through the entire update
process.  So I think this goes back to the way folks manage their
configurations and when you start getting two conflicting tools that
manage things slightly differently you end up with operational
problems.  I would say it would be better to work on shortening the
actual update process time rather than increasing the complexity by
trying to add in support to not always run something.  That being
said, if heat could determine when the compiled data has changed
between updates and only run on those nodes, maybe it wouldn't be so
hard.

> * For large scale deployments there was interest in leveraging the
> oslo.messaging split RPC/notifications.  Initially just separate Rabbit
> clusters, but potentially in the future completely different drivers that
> better suit the two different use cases.  Since this is something I believe
> operators are already using in the field, it would be good to get support
> for it in TripleO.  I suggested they file an RFE, and I went ahead and
> opened a wishlist bug upstream:
> https://bugs.launchpad.net/tripleo/+bug/1693928
>

It's in the underlying puppet modules so technically it could be
deployed if you can setup two distinct messaging infrastructures with
tripleo.

> And although I have a bunch more, I'm going to stop here in an attempt to
> avoid complete information overload.  I'll send another email (or two...)
> another day with some of the other topics that came up.
>
> Thanks.
>
> -Ben
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [TripleO] Passing along some field feedback

[Ben Nemec]

Hi,

As some of you may know, I have a weekly meeting with some of the Red 
Hat folks who are out in the field using TripleO to deploy OpenStack at 
customer sites.  The past couple of weeks have been particularly 
interesting, so I thought I'd take a moment to disseminate the comments 
to the broader developer community.  I'll try to keep it from getting 
too wall-of-texty, but no promises. :-)


Here goes:

* There was interest in using split stack to avoid having to use 
TripleO's baremetal provisioning.  I let them know that technically 
split stack was possible in Newton and that further usability 
improvements were coming in Ocata and Pike.  Keep in mind that for many 
of these companies Newton is still a new shiny thing that they're 
looking to move onto, so even though this functionality seems like it's 
been around for a long time to us that isn't necessarily the case in the 
field.


* Some concerns around managing config with TripleO were raised, 
specifically two points: 1) Stack updates can cause service outages, 
even if you're just looking to change one minor config value in a single 
service.  2) Stack updates take a long time.  45 minutes to apply one 
config value seems a little heavy from a user perspective.


For 1, I did mention that we had been working to reduce the number of 
spurious service restarts on update.  2 is obviously a little trickier. 
One thing that was requested was a mode where a Puppet step doesn't get 
run unless something changes - which is the exact opposite of the 
feedback we got initially on TripleO.  Back then people wanted a stack 
update to assert state every time, like regular Puppet would.  There 
probably weren't enough people in this discussion to take immediate 
action on this, but I thought it was an interesting potential change in 
philosophy.  Also it drives home the need for better performance of 
TripleO as a whole.


* For large scale deployments there was interest in leveraging the 
oslo.messaging split RPC/notifications.  Initially just separate Rabbit 
clusters, but potentially in the future completely different drivers 
that better suit the two different use cases.  Since this is something I 
believe operators are already using in the field, it would be good to 
get support for it in TripleO.  I suggested they file an RFE, and I went 
ahead and opened a wishlist bug upstream: 
https://bugs.launchpad.net/tripleo/+bug/1693928


And although I have a bunch more, I'm going to stop here in an attempt 
to avoid complete information overload.  I'll send another email (or 
two...) another day with some of the other topics that came up.


Thanks.

-Ben


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev