Re: [openstack-dev] [puppet] Puppet OpenStack PTL non-candidacy

[Clayton O'Neill]

On Fri, Sep 9, 2016 at 12:05 PM, Emilien Macchi  wrote:
> However, I think it's time to pass the PTL torch for Ocata cycle.
> Don't worry, I'll still be around and bother you when CI is broken ;-)
>
> Again, a big thank you for those who work with me,

I think it’s impossible to overstate how much the Puppet modules for
Openstack have improved over the past 18 months.  Thank you for all
hard work and long hours you’ve put in to help get them there!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Puppet] Suggestions for Dynamic enabling of SRIOV nic agent

[Clayton O'Neill]

I would probably create a custom fact to identify which ones have
sriov capable NICs and use that to selectively enable installation of
the correct packages, setup aggregates, etc.

On Fri, Jun 24, 2016 at 7:19 AM, Sanjay Upadhyay  wrote:
> Hello Folks,
>
> Wondering what is the best approach to dynamically generating info of SRIOV
> capable hosts in a cluster?
>
> The problem is, user want to deploy sriov-nic-agent on compute nodes, with
> mixed set of h/w configuration, ie set of compute nodes which do not have
> SR-IOV nics and a set of nodes which have SR-IOV capable nics.
>
> From the perspective of nova, nova can spawn hosts requiring sr-iov via host
> aggregation or flavors (scheduler_default_filters =
> RamFilter,ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,PciPassthroughFilter
> )
>
> However, from puppet side, we should enable sriov nic agent only on the
> nodes with sr-iov nics. What is the best approach to address this?
>
> regards
> /sanjay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] Stepping down from puppet core

[Clayton O'Neill]

I’d like to step down as a core reviewer for the OpenStack Puppet
modules.  For the last cycle I’ve had very little time to spend
reviewing patches, and I don’t expect that to change in the next
cycle.  In addition, it used to be that I was contributing regularly
because we were early upgraders and the modules always needed some
work early in the cycle.  Under Emilien’s leadership this situation
has changed significantly and I find that the puppet modules generally
“just work” for us in most cases.

I intend to still be contribute when I can and I’d like to thank
everyone for the hard work for the last two cycles.  The OpenStack
Puppet modules are really in great shape these days.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum][keystone][all] Using Keystone /v3/credentials to store TLS certificates

[Clayton O'Neill]

On Wed, Apr 13, 2016 at 10:26 AM, rezroo  wrote:
> Hi Kevin,
>
> I understand that this is how it is now. My question is how bad would it be
> to wrap the Barbican client library calls in another class and claim, for
> all practical purposes, that Magnum has no direct dependency on Barbican?
> What is the negative of doing that?
>
> Anyone who wants to use another mechanism should be able to do that with a
> simple change to the Magnum conf file. Nothing more complicated. That's the
> essence of my question.

For us, the main reason we’d want to be able to deploy without
Barbican is mostly to lower the initial barrier of entry.  We’re not
running anything else that would require Barbican for a multi-node
deployment, so for us to do a realistic evaluation of Magnum, we’d
have to get two “new to us” services up and running in a development
environment.  Since we’re not running Barbican or Magnum, that’s a big
time commitment for something we don’t really know if we’d end up
using.  From that perspective, something that’s less secure might be
just fine in the short term.  For example, I’d be completely fine with
storing certificates in the Magnum database as part of an evaluation,
knowing I had to switch from that before going to production.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] A proposal to separate the design summit

[Clayton O'Neill]

On Mon, Feb 29, 2016 at 4:32 AM, Thierry Carrez  wrote:
> That was considered... and I would not necessarily be against it, but I
> would like to understand what the benefit would be. Tom advocated for
> keeping it as a separate event (or a set of regional separated events) that
> would be ops-branded, making it easier for ops to justify travel to it. If
> the goal is to co-locate so that users can more easily participate to the
> contributor-oriented event, I think that would be counter-productive for
> most operators: as I said earlier those are project team meetings for
> project team members to work together -- you should only join a room if you
> consider yourself part of the team that will do the work. We won't be
> rediscussing requirements or gathering feedback there, those will have been
> discussed at the strategic planning sessions at the summit already.

I’d strongly prefer to see the new dev event and an ops event
co-located.  I’m not against regional events, but I’m not sure why
regional events would *replace* the existing event.  We’re not doing
that for any other sort of mid-cycle/meetup/conference are we?

One suggestion regarding the funding and branding: Have the non-summit
dev event and the ops event separately branded but co-located  Have
different tickets, but have them both grant entrance to both events.
Issue different color badges or lanyards if that seems helpful in
having developers and operators find each other.  Ideally the two
events would run concurrently, or substantially overlap.  Something
like Mon-Wed for ops event, Wed-Fri for dev event or vice versa.  You
could have cross project discussions and dedicated ops/dev breakouts
on an overlapping day.  People not interested in the overlapping
sessions could arrive a day late or leave a day early.

One of my very few complaints about the past few summits has been that
the ops and dev events run concurrently.  For example, in Tokyo I had
to choose between going to cross-project design sessions, or going to
the operators session.  I picked the cross-project sessions in most
cases and I felt like that was probably a good choice.  However, on
Friday there was nothing but dev work sessions was scheduled.  Kris
Lindgren put together an informal ops session on that day and we had
40-50 people attend with no prior planning and no official space.  If
the Ops sessions had been at the end of the week instead of the
beginning there would have been less conflicts.

I’ve seen many developers express the opinion that they desperately
want more operator feedback.  Personally I’m not going to be able to
go to a 2 summits, 2 ops events and 2 dev events, and I think most
operators are in the same situation.  If I have to choose, i’m going
to skip the dev events.  I think it needs to be well thought out how
ops and dev combined sessions are going to be run if the split goes
into place.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] A proposal to separate the design summit

[Clayton O'Neill]

Is the expectation that the ops mid-cycle would continue separately,
or be held with the meeting formerly known as the Design Summit?

Personally I’d prefer they be held together, but scheduled with the
thought that operators aren’t likely to be interested in work
sessions, but that a good number of us would be interested in
cross-project and some project specific planning sessions.  This would
also open up the possibility of having some sessions specific intended
for operator/developer feedback sessions.

On Mon, Feb 22, 2016 at 12:15 PM, Lauren Sell  wrote:
>
>> On Feb 22, 2016, at 8:52 AM, Clayton O'Neill  wrote:
>>
>> I think this is a great proposal, but like Matt I’m curious how it
>> might impact the operator sessions that have been part of the Design
>> Summit and the Operators Mid-Cycle.
>>
>> As an operator I got a lot out of the cross-project designs sessions
>> in Tokyo, but they were scheduled at the same time as the Operator
>> sessions.  On the other hand, the work sessions clearly aren’t as
>> useful to me.  It would be nice would be worked out so that the new
>> design summit replacement was in the same location, and scheduled so
>> that the operator specific parts were overlapping the work sessions
>> instead of the more big picture sessions.
>
> Great question. The current plan is to maintain the ops summit and mid-cycle 
> activities.
>
> The new format would allow us to reduce overlap between ops summit and cross 
> project sessions at the main event, both for the operators and developers who 
> want to be involved in either activity.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] A proposal to separate the design summit

[Clayton O'Neill]

I think this is a great proposal, but like Matt I’m curious how it
might impact the operator sessions that have been part of the Design
Summit and the Operators Mid-Cycle.

As an operator I got a lot out of the cross-project designs sessions
in Tokyo, but they were scheduled at the same time as the Operator
sessions.  On the other hand, the work sessions clearly aren’t as
useful to me.  It would be nice would be worked out so that the new
design summit replacement was in the same location, and scheduled so
that the operator specific parts were overlapping the work sessions
instead of the more big picture sessions.

On Mon, Feb 22, 2016 at 11:32 AM, Matt Fischer  wrote:
> Cross-post to openstack-operators...
>
> As an operator, there's value in me attending some of the design summit
> sessions to provide feedback and guidance. But I don't really need to be in
> the room for a week discussing minutiae of implementations. So I probably
> can't justify 2 extra trips just to give a few hours of feedback/discussion.
> If this is indeed the case for some other folks we'll need to do a good job
> of collecting operator feedback at the operator sessions (perhaps hopefully
> with reps from each major project?). We don't want projects operating in a
> vacuum when it comes to major decisions.
>
> Also where do the current operators design sessions and operators midcycle
> fit in here?
>
> (apologies for not replying directly to the first message, gmail seems to
> have lost it).

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

[Clayton O'Neill]

On Thu, Jan 7, 2016 at 10:44 AM, Mike Bayer  wrote:
> On 01/07/2016 07:39 AM, Clayton O'Neill wrote:
>> On Thu, Jan 7, 2016 at 2:49 AM, Roman Podoliaka  
>> wrote:
>>> In each child process eventlet WSGI server calls accept() in a loop to
>>> get a client socket from the kernel and then puts into a greenlet from
>>> a pool for processing:
>>
>> It’s worse than that.  What I’ve seen (via strace) is that eventlet actually
>> converts socket into a non-blocking socket, then converts that accept() into 
>> a
>> epoll()/accept() pair in every child.  Then when a connection comes in, every
>> child process wakes up out of poll and races to try to accept on the the
>> non-blocking socket, and all but one of them fails.
>
> is that eventlet-specific or would we see the same thing in gevent ?

I’m not sure.  For eventlet it’s a natural consequence of most of this being
implemented in Python.  It looks like some of this is implemented in C in
gevent, so they may handle the situation differently.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] proposing Alex Schultz part of core team

[Clayton O'Neill]

+1

On Tue, Jan 5, 2016 at 1:15 PM, Potter, Nathaniel <
nathaniel.pot...@intel.com> wrote:

> I'm not a core, but Alex has been very helpful to me in the past and I
> think he's definitely earned it, +1.
>
> Thanks,
> Nate
>
> -Original Message-
> From: Emilien Macchi [mailto:emil...@redhat.com]
> Sent: Tuesday, January 5, 2016 9:55 AM
> To: OpenStack Development Mailing List
> Subject: [openstack-dev] [puppet] proposing Alex Schultz part of core team
>
> Hi,
>
> Alex Schultz (mwhahaha on IRC) has been a very active contributor over the
> last months in the Puppet OpenStack group:
> * He's doing a lot of reviews and they are very valuable. He's in my
> opinion fully aware of our conventions and has nice insights to improve our
> modules.
> * He's very helpful to work on bugs or new features when needed.
> * Always present during meetings and actively participating.
> * Always on IRC, he never hesitates to give a hand on something or help
> people.
>
> I think we're very lucky to have Alex part of our group and I would like
> to promote him core reviewer of all our modules.
>
> Team, please vote if you like the idea,
>
> Thanks,
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] proposing Cody Herriges part of Puppet OpenStack core

[Clayton O'Neill]

+1

On Tue, Dec 8, 2015 at 4:15 PM, Matt Fischer  wrote:

> +1
>
> On Tue, Dec 8, 2015 at 2:07 PM, Rich Megginson 
> wrote:
>
>> On 12/08/2015 09:49 AM, Emilien Macchi wrote:
>>
>> Hi,
>>
>> Back in "old days", Cody was already core on the modules, when they were
>> hosted by Puppetlabs namespace.
>> His contributions [1] are very valuable to the group:
>> * strong knowledge on Puppet and all dependencies in general.
>> * very helpful to debug issues related to Puppet core or dependencies
>> (beaker, etc).
>> * regular attendance to our weekly meeting
>> * pertinent reviews
>> * very understanding of our coding style
>>
>> I would like to propose having him back part of our core team.
>> As usual, we need to vote.
>>
>>
>> +1
>>
>> Thanks,
>>
>> [1]http://stackalytics.openstack.org/?metric=commits&release=all&project_type=all&user_id=ody-cat
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] proposing Sofer Athlan Guyot part of puppet-keystone core team

[Clayton O'Neill]

+1 from me.

On Thu, Dec 3, 2015 at 3:05 PM, Emilien Macchi  wrote:

> Hi,
>
> For some months, Puppet OpenStack group has been very lucky to have
> Sofer working with us.
> He became a huge contributor to puppet-keystone, he knows the module
> perfectly and wrote insane amount of code recently, to bring new
> features that our community requested (some stats: [1]).
> He's always here to help on IRC and present during our weekly meetings.
>
> Core contributors, please vote if you agree to add him part of
> puppet-keystone core team.
>
> Thanks,
>
> [1]
>
> http://stackalytics.openstack.org/?user_id=sofer-athlan-guyot&release=all&metric=loc
> --
> Emilien Macchi
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Config support for oslo.config.cfg.MultiStrOpt

[Clayton O'Neill]

For puppetlabs-inifile that’s a bigger question.  For our purposes the
answer is “whatever ConfigParser does”.

On Wed, Dec 2, 2015 at 10:32 PM, Cody Herriges  wrote:

> Martin,
>
> I see no reason this shouldn't just be pushed into puppetlabs-inifile.  I
> can't actually find a real "spec" for INI file and even the Wiki link[3]
> calls out that there is no actual spec.
>
> On Fri, Nov 27, 2015 at 5:04 AM, Martin Mágr  wrote:
>
>> Greetings,
>>
>>   I've submitted patch to puppet-openstacklib [1] which adds provider for
>> parsing INI files containing duplicated variables (a.k.a MultiStrOpt [2]).
>> Such parameters are used for example to set
>> service_providers/service_provider for Neutron LBaaSv2. There has been a
>> thought raised, that the patch should rather be submitted to
>> puppetlabs-inifile module instead. The reason I did not submitted the patch
>> to inifile module is that IMHO duplicate variables are not in the INI file
>> spec [3]. Thoughts?
>>
>> Regards,
>> Martin
>>
>>
>> [1] https://review.openstack.org/#/c/234727/
>> [2]
>> https://docs.openstack.org/developer/oslo.config/api/oslo.config.cfg.html#oslo.config.cfg.MultiStrOpt
>> [3] https://en.wikipedia.org/wiki/INI_file#Duplicate_names
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] weekly meeting #59

[Clayton O'Neill]

On Tue, Nov 17, 2015 at 4:38 PM, Cody Herriges  wrote:

> Now that the standard is $::os_service_default does it mean that current
> changes up for review with parameters set to the string ' DEFAULT>' should be changed to $::os_service_default before merge?
>

The decision was to grandfather any existing patches and any new patches
after today would be required to use $:os_service_default.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] about $::os_service_default

[Clayton O'Neill]

What Matt said.

I think we don’t need to explicitly support people that aren’t using
distribution packages, we just want to try to avoid making harder things
harder.  I don’t see a need to go out of our way to support it.  Anyone
that isn’t using distro packages should be able to figure out how to set
logdir on their own.

On Fri, Nov 13, 2015 at 1:46 PM, Matt Fischer  wrote:

> This work is already being done by Clayton (and to a lesser extent me).
> From the openstack modules POV it mainly involves moving the packaging code
> into a separate place [1][2] and then integrating with puppet-os_docker[3].
> This os_docker work is only done for designate and heat and of course
> requires os_docker which is not official and is rather specific to us.
>
> As long as the external hooks are in place folks could plugin their own
> venv, docker, tarball, or whatever way to install code.
>
> [1] -
> https://github.com/openstack/puppet-designate/commit/5a37cc81276cb8f8ee6dca9b9b532930e6ac86de
> [2] -
> https://github.com/openstack/puppet-heat/commit/dca9fe942b99b9c30e31167e4736058767738f21
> [3] - https://github.com/twc-openstack/puppet-os_docker
>
>
>
> On Fri, Nov 13, 2015 at 11:13 AM, Cody Herriges 
> wrote:
>
>> Yanis Guenane wrote:
>> >
>> > On 11/03/2015 02:57 PM, Emilien Macchi wrote:
>> >> I'm seeing a lot of patches using the new $::os_service_default.
>> >>
>> >> Please stop trying to using it at this time. The feature is not stable
>> >> yet and we're testing it only for puppet-cinder module.
>> >> I've heard Yanis found something that is not backward compatible with
>> >> logging, but he's away this week so I suggest we wait next week.
>> >>
>> >> In the meantime, please do not use $::os_service_default outside
>> >> puppet-cinder.
>> >>
>> >> Thanks a lot,
>> > After a deeper investigation, the issue with logging[1] is only true if
>> > a user is using the puppet-openstack to only configure the component and
>> > not relying on it to install the RDO/UCA packages.
>> >
>> > On RDO, the file /usr/lib/systemd/system/openstack-cinder-api.service is
>> > provided. It specifies :
>> >
>> >   ExecStart=/usr/bin/cinder-api --config-file
>> > /usr/share/cinder/cinder-dist.conf \
>> > --config-file /etc/cinder/cinder.conf --logfile
>> /var/log/cinder/api.log
>> >
>> > On Ubuntu, the file /etc/init/cinder-api.conf is provided. It specfies :
>> >
>> >   exec start-stop-daemon --start --chuid cinder --exec
>> /usr/bin/cinder-api \
>> >  -- --config-file=/etc/cinder/cinder.conf
>> > --log-file=/var/log/cinder/cinder-api.log
>> >
>> > In my understanding, this means that when using packages none of log-dir
>> > and log-file will ever be taken in account.
>> >
>> > So the only use case moving those values to $::os_service_default might
>> > impact are for the people relying directly on the python package.
>> >
>> > This raises two questions I'd like to ask :
>> >
>> >   * Do lot of people use puppet-openstack modules relying on the python
>> > package directly ?
>> >   * Should we be opinionated here ? If user relies on the python
>> > packages, we can consider that an advanced use-case and expect the user
>> > to know exactly what she needs to configure. Plus we do not handle the
>> > use case where we want a file for cinder-volume.log and
>> cinder-backup.log.
>> >
>> > [1]
>> >
>> https://trello.com/c/XLJJJBF0/71-move-modules-to-the-os-service-default-pattern
>> >
>>
>> My opinion is that installing directly from python pip is not currently
>> officially supported in the modules and specifically trying to take that
>> use case into account when we do not support it either leaves us in a
>> place where we have to go full in on supporting them or put the modules
>> in a state that thoroughly frustrates and misleads users.
>>
>> If we were going to put priority on which packaging systems to support
>> next; I'd prefer docker over pip.
>>
>>
>> --
>> Cody
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] ::db classes

[Clayton O'Neill]

On Wed, Nov 11, 2015 at 9:50 AM, Clayton O'Neill  wrote:

> I discovered this issue last night and opened a bug on it (
> https://bugs.launchpad.net/puppet-tuskar/+bug/1515273).
>
> This effects most of the modules, and the short version of it is that the
> defaults in all the ::db classes are wrong for max_pool_size
> and max_overflow.  We’re setting test to 10 and 20, but oslo_db actually
> has no internal default.
>

To clarify: The modules following this pattern are setting max_pool_size
and max_overflow to 10 and 20 respectively, but oslo_db has no internal
default.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] ::db classes

[Clayton O'Neill]

I discovered this issue last night and opened a bug on it (
https://bugs.launchpad.net/puppet-tuskar/+bug/1515273).

This effects most of the modules, and the short version of it is that the
defaults in all the ::db classes are wrong for max_pool_size
and max_overflow.  We’re setting test to 10 and 20, but oslo_db actually
has no internal default.

The two options I see for fixing this are to either put in place the old
traditional:

if $option_name {
  # ensure value
} else {
  # ensure absent
}

and do that for at least the two values with the wrong defaults, or
preferably, change the defaults for all of these parameters to use the
service default fact.

I prefer the latter approach.  I know there has been some discussion on
Trello about how much we want to be using the service default fact, but as
near as I can tell, the concerns seem to mostly about not accidentally
reverting intentionally different values and breaking existing installs.

This scenario seems to be the ideal candidate for just not setting a value
unless the deployer has specifically asked for something special.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] about $::os_service_default

[Clayton O'Neill]

What is the issue with logging?  Can someone other than Yanis look into
this?

On Tue, Nov 3, 2015 at 8:57 AM, Emilien Macchi  wrote:

> I'm seeing a lot of patches using the new $::os_service_default.
>
> Please stop trying to using it at this time. The feature is not stable
> yet and we're testing it only for puppet-cinder module.
> I've heard Yanis found something that is not backward compatible with
> logging, but he's away this week so I suggest we wait next week.
>
> In the meantime, please do not use $::os_service_default outside
> puppet-cinder.
>
> Thanks a lot,
> --
> Emilien Macchi
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Creating puppet-keystone-core and proposing Richard Megginson core-reviewer

[Clayton O'Neill]

+1

Big thanks for Rich for all the work he’s done so far.

On Mon, Nov 2, 2015 at 3:34 AM, Adam Young  wrote:

> On 10/31/2015 10:55 AM, Emilien Macchi wrote:
>
> At the Summit we discussed about scaling-up our team.
> We decided to investigate the creation of sub-groups specific to our
> modules that would have +2 power.
>
> I would like to start with puppet-keystone:
> https://review.openstack.org/240666
>
> And propose Richard Megginson part of this group.
>
> Rich is leading puppet-keystone work since our Juno cycle. Without his
> leadership and skills, I'm not sure we would have Keystone v3 support in
> our modules.
> He's a good Puppet reviewer and takes care of backward compatibility. He
> also has strong knowledge at how Keystone works. He's always willing to
> lead our roadmap regarding identity deployment in OpenStack.
>
> Having him on-board is for us an awesome opportunity to be ahead of other
> deployments tools and supports many features in Keystone that real
> deployments actually need.
>
> I would like to propose him part of the new puppet-keystone-core group.
>
>
> As a Keystone developer I have to say I am indebted to Rich for his
> efforts.  +1 from me.
>
>
> Thank you Rich for your work, which is very appreciated.
> --
> Emilien Macchi
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Incompatible default parameters

[Clayton O'Neill]

I think the only people that would be affected by the puppet-openstack
module default not matching the keystone default are people that are trying
to retrofit puppet into an already running environment.  Those people
already have a ton of work laid out in front of them, so I’m ok with making
it very slightly more difficult for them (if those people even exist).

However, for existing users of the modules, this is potentially a big pain,
since we already have existing catalogs with the old (wrong) name.

For new users of the module, this might be slightly annoying, but again,
they can override it.

Personally I’d prefer to leave it as is, instead of requiring all existing
users of the modules to change.

On Wed, Oct 21, 2015 at 9:31 AM, Martin Mágr  wrote:

> Greetings,
>
>   I'd like to discuss bug 1506061 ([1]). The main problem I'm trying to
> solve here is that using default parameters of classes cinder::api and
> nova::keystone::auth migration won't work, because Cinder will search for
> endpoint with different name ('Compute service' instead of 'nova'). I've
> submitted fix for that in first patchset of [2], but it was denied as
> backward incompatible change, which I agree with, and instead I just added
> warnings about rename in next cycle [3].
>
>   So the question is if we should start following endpoint naming
> according to Keystone's default catalog or just change default value of
> $nova_catalog_.*info parameters in cinder::api to match endpoint created by
> nova::keystone::auth? I don't mind going one way or another, but I do think
> that default parameters has to create fully functional environment.
>
> Thanks in advance for answers,
> Martin
>
> [1] https://bugs.launchpad.net/puppet-nova/+bug/1506061
> [2] https://review.openstack.org/#/c/222120/1
> [3]
> https://review.openstack.org/#/q/status:open+branch:master+topic:bug/1506061,n,z
> [4] https://review.openstack.org/#/c/219284/2/manifests/api.pp
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet][Fuel] Using Native Ruby Client for Openstack Providers

[Clayton O'Neill]

I agree that ideally, using a native ruby library would be better, but I
also share Matt's concern.  We'd need a commitment from more than one
person to maintain the library if we went that route.

I think the big advantages I see with the ruby client would be:

   - Potentially better performance
   - Faster turn around time for enhancements/bug fixes.  My concern here
   is that we're currently limited by how quickly distros pick up new versions
   of OpenStack Client.


I think if we did end up using a ruby library, we'd also want to make sure
it was not only vendored, but also usable independently, to increase the
audience.

On Tue, Oct 13, 2015 at 8:16 AM, Vladimir Kuklin 
wrote:

> Matt
>
> Thanks for your input. So, I mentioned the following - Fuel guys can
> contribute into Ruby client for OpenStack as we are also interested in
> making it faster. That's why I asked for support in case we invest
> substantial effort (as we do not want to waste our time on things that will
> not land into upstream) and asked if the approach that I proposed is OK
> with you.
>
> On Tue, Oct 13, 2015 at 6:07 PM, Matt Fischer 
> wrote:
>
>> From a technical point of view, not forking and using a native library
>> makes total sense. I think it would likely be faster and certainly cleaner
>> than parsing output. Unfortunately I don't think that we have the resources
>> to actively maintain the library. I think that's the main blocker for me.
>>
>> On Tue, Oct 13, 2015 at 7:13 AM, Vladimir Kuklin 
>> wrote:
>>
>>> Puppetmaster and Fuelers,
>>>
>>> Last week I mentioned that I would like to bring the theme of using
>>> native ruby OpenStack client and use it within the providers.
>>>
>>> Emilien told me that I had already been late and the decision was made
>>> that puppet-openstack decided to not work with Aviator based on [0]. I went
>>> through this thread and did not find any unresolvable issues with using
>>> Aviator in comparison with potential benefits it could have brought up.
>>>
>>> What I saw actually was like that:
>>>
>>> * Pros
>>>
>>> 1) It is a native ruby client
>>> 2) We can import it in puppet and use all the power of Ruby
>>> 3) We will not need to have a lot of forks/execs for puppet
>>> 4) You are relying on negotiated and structured output provided by API
>>> (JSON) instead of introducing workarounds for client output like [1]
>>>
>>> * Cons
>>>
>>> 1) Aviator is not actively supported
>>> 2) Aviator does not track all the upstream OpenStack features while
>>> native OpenStack client does support them
>>> 3) Ruby community is not really interested in OpenStack (this one is
>>> arguable, I think)
>>>
>>> * Proposed solution
>>>
>>> While I completely understand all the cons against using Aviator right
>>> now, I see that Pros above are essential enough to change our mind and
>>> invest our own resources into creating really good OpenStack binding in
>>> Ruby.
>>> Some are saying that there is not so big involvement of Ruby into
>>> OpenStack. But we are actually working with Puppet/Ruby and are invloved
>>> into community. So why should not we own this by ourselves and lead by
>>> example here?
>>>
>>> I understand that many of you do already have a lot of things on their
>>> plate and cannot or would not want to support things like additional
>>> library when native OpenStack client is working reasonably well for you.
>>> But if I propose the following scheme to get support of native Ruby client
>>> for OpenStack:
>>>
>>> 1) we (community) have these resources (speaking of the company I am
>>> working for, we at Mirantis have a set of guys who could be very interested
>>> in working on Ruby client for OpenStack)
>>> 2) we gradually improve Aviator code base up to the level that it
>>> eliminates issues that are mentioned in  'Cons' section
>>> 3) we introduce additional set of providers and allow users and
>>> operators to pick whichever they want
>>> 4) we leave OpenStackClient default one
>>>
>>> Would you support it and allow such code to be merged into upstream
>>> puppet-openstack modules?
>>>
>>>
>>> [0]
>>> https://groups.google.com/a/puppetlabs.com/forum/#!searchin/puppet-openstack/aviator$20openstackclient/puppet-openstack/GJwDHNAFVYw/ayN4cdg3EW0J
>>> [1]
>>> https://github.com/openstack/puppet-swift/blob/master/lib/puppet/provider/swift_ring_builder.rb#L21-L86
>>> --
>>> Yours Faithfully,
>>> Vladimir Kuklin,
>>> Fuel Library Tech Lead,
>>> Mirantis, Inc.
>>> +7 (495) 640-49-04
>>> +7 (926) 702-39-68
>>> Skype kuklinvv
>>> 35bk3, Vorontsovskaya Str.
>>> Moscow, Russia,
>>> www.mirantis.com 
>>> www.mirantis.ru
>>> vkuk...@mirantis.com
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __

Re: [openstack-dev] [fuel][puppet] Fuel Library and upstream modules

[Clayton O'Neill]

Emilien, debtor looks like an interesting tool, thanks for the link.

As far as tracking upstream branches from local forks and carrying local
patches, we've had good luck with using the git-upstream tool on stack
forge: https://github.com/stackforge/git-upstream

It allows you to merge in upstream changes on a regular basis and will
automatically drop local changes if there is a matching Change-Id field in
the upstream branch.  It also allows you to manually tag specific changes
to drop.  We have this setup to run as a regular job and the merge commit
for the changes gets pushed to our internal Gerrit instance for review and
all our normal CI tools run against that so we can gauge the impact of the
merge.



On Wed, Jul 29, 2015 at 6:16 PM, Emilien Macchi  wrote:

>
>
> On 07/29/2015 03:48 PM, Alex Schultz wrote:
> > Hello everyone,
> >
> > I have put together a wiki describing the proposed interactions between
> > fuel-library and upstream modules based on previous talks around
> > librarian-puppet[0].  Please take some time to
> > review https://wiki.openstack.org/wiki/Fuel/Library_and_Upstream_Modules
> .
> > This page provides a brief history of this change, details of the
> > changes, best practices, and workflows for dealing with upstream modules.
> >
> > I have updated this page based on the feedback previously solicited
> > around my suggestion to use librarian-puppet to manage the inclusion of
> > upstream modules into fuel-library. During the implementation, we found
> > that packaging librarian-puppet and the dependency management became
> > burdensome. As a substitute, we have packaged librarian-puppet-simple
> > and will only be using git repository references for inclusion of
> > upstream modules.  Also based on our build requirements, we are also
> > leveraging a fuel-infra mirror of upstream repositories to allow for
> > local builds and CI to not have to fetch modules continuously from
> > upstream github repositories.
> >
> > I have also taken the time to update the proposed patches[1] with the
> > librarian-puppet-simple configurations so they they are ready to merge
> > if there is an agreement on this process.
> >
> > Thanks,
> > -Alex
> >
> > [0]
> http://lists.openstack.org/pipermail/openstack-dev/2015-June/067806.html
> > [1]
> https://review.openstack.org/#/q/topic:bp/fuel-puppet-librarian+project:stackforge/fuel-library,n,z
> >
>
> This is a very good initiative and I'm happy to see that happening. It
> reflects what I was asking in our collaboration request.
>
> Though I have one suggestion for "Custom Upstream Module Changes" section.
> I suggest you:
>
> 1/ first submit the change in the upstream module
> 2/ cherry-pick the commit in Fuel custom branch
> 3/ keep track of upstream patch (address reviews, etc), and establish a
> sort of scoring to evaluate the debt [1] between downstream (Fuel) and
> upstream (OpenStack). It will make sure a maximum of code is upstream
> and downstream patches won't be forgotten.
> 4/ when an upstream patch is merged, drop the cherry-pick by rebasing.
>
> This is the workflow I try to push at Red Hat (in RDO) and to me, this
> is the way to go for having Puppet modules the closest as possible from
> upstream with the freedom to have custom patches downstream.
>
> Best,
>
> [1] https://github.com/redhat-cip/debtor
> --
> Emilien Macchi
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Proposing Yanis Guenane core

[Clayton O'Neill]

+1

On Tue, Jul 28, 2015 at 9:00 AM, Sebastien Badia  wrote:

> On Mon, Jul 27, 2015 at 03:06:56PM (-0400), Emilien Macchi wrote:
> > I would like to open the vote to promote Yanis part of Puppet OpenStack
> > core reviewers.
>
> Hi,
>
> Yes, of course!
> A big +1
>
> Seb
>
> --
> Sebastien Badia
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] puppet-designate POC implementation of virtualenv and docker support.

[Clayton O'Neill]

On Wed, Jul 15, 2015 at 6:11 PM, Mike Dorman  wrote:

>   I have been meaning to ask you about this, so thanks for posting.
>
>  I like the approach.  Definitely a lot cleaner than the somewhat
> hardcoded dependencies and subscriptions that are in the modules now.
>
>  Do you envision that long term the docker/venv/whatever else
> implementation (like you have in designate_ext) would actually be part of
> the upstream Puppet module?  Or would we provide the hooks that you
> describe, and leave it up to other modules to handle the non-package-based
> installs?
>

I think it may make sense to integrate them long term.  Right now I think
that it's hard to do virtualenv or docker support for the Puppet modules
that's not fairly biased towards a specific implementation or set of
requirements.  In the short term I'd like to see the Designate hook patch
get merged and we'll start working on similar patches for other modules if
everyone agrees that's a reasonable approach.  Right now projects like
Designate, Keystone, Heat are obvious candidates since they're simpler from
a deployment standpoint.

Over time I suspect we'll develop a better idea of what best practices are
in this area and we could consider having a single module that adds docker
or virtualenv support to the existing modules, and long term perhaps merge
that support into the existing modules.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] puppet-designate POC implementation of virtualenv and docker support.

[Clayton O'Neill]

Last year I put together a virtualenv patch for the Designate puppet
module, but the patch was too invasive of a change and too opinionated to
be practical to merge.  I've taken another shot at this with the approach
of implementing well defined hooks for various phases of the module. This
should  allow external support for alternative ways of installing and
running services (such as virtualenv, and docker).  I think this patch is
now mostly ready for some outside reviews (we'll be running the virtualenv
support in production soon).

The puppet-designate change to support this can be found here:
https://review.openstack.org/#/c/197172/

The supporting puppet-designate_ext module can be found here:
https://github.com/twc-openstack/puppet-designate_ext

The basic approach is to split the module dependency chain into 3 phases:

 * install begin/end
 * config begin/end
 * service begin/end

Each of these phases have a pair of corresponding anchors that are used
internally for dependencies and notifications.  This allows external
modules to hook into this flow without having to change the module.  For
example, the virtualenv support will build the virtualenv environment
between the designate::install::begin and designate::install::end anchors.
Additionally, the virtualenv support will notify the
designate::install::end anchor.  This allows other resources to subscribe
to this anchor without needing to know if the software is being installed
as a package, virtualenv, or docker image.

I think this approach could be applied mostly as is to at least some of the
existing modules with similar levels of changes.  For example, horizon,
keystone & heat would probably be fairly straightforward.  I suspect this
approach would need refinement for more complex services like neutron and
nova.  We would need to work out how to manage things like external
packages that would still be needed if running a virtualenv based install,
but probably not needed if running a docker based install.  We would
probably also want to consider how to be more granular about service
notifications.

I'd love to get some feedback on this approach if people have time to look
it over.  We're still trying to move away from using packages for service
installs and I'd like to figure out how to do that without carrying
heavyweight and fragile patches to the openstack puppet modules.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] (officially) deprecate stable/{grizzly, havana} branches

[Clayton O'Neill]

I'd vote for stable -2.  I think a number of people do an upgrade maybe
once a year.  I believe CERN just upgraded to juno recently.

On Tue, Jun 16, 2015 at 1:36 PM, Richard Raseley 
wrote:

> Matt Fischer wrote:
>
>> +1 from me for deprecation.
>>
>> I'd also like to know or have an official policy for future
>> deprecations, such as when will we deprecate Icehouse?
>>
>> On Tue, Jun 16, 2015 at 9:50 AM, Emilien Macchi > > wrote:
>>
>> Hi,
>>
>> Some of our modules have stable/grizzly and stable/havana branches.
>> Some
>> of them have the CI broken due to rspec issues that would require some
>> investigation and time if we wanted to fix it.
>>
>> We would like to know who plan to backport some patches in these
>> branches?
>>
>> If nobody plans to do that, we will let the branches as they are now
>> but
>> won't officially support them.
>>
>> By support I mean maintaining the CI jobs green (rspec, syntax, etc),
>> fixing bugs and adding new features.
>>
>> Any feedback is welcome!
>>
>> Regards,
>> --
>> Emilien Macchi
>>
>>
> I echo your +1.
>
> Perhaps most current stable supported, -1 stable version?
>
> In that example, once the Liberty release of modules (or a particular
> module) is cut we would support Liberty and Kilo. When the same happens for
> M, we would deprecate Kilo and support M and Liberty.
>
> Stable -2 also seems sane - I don't have a good sense of how far people
> are generally behind.
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] drop monolithic plugins in neutron module

[Clayton O'Neill]

Makes sense to drop them to me.

On Wed, Jun 10, 2015 at 7:20 PM, Emilien Macchi  wrote:

> Hi,
>
> Monolithic plugins have been dropped in Neutron tree since Juno, I think
> it's time to drop the code from puppet-neutron (I guess everyone is
> using ML2, at least I hope for them).
>
> If anyone is running master against OpenStack Icehouse, please raise
> your voice here and please vote in this patch:
> https://review.openstack.org/#/c/190395
>
> Thanks,
> --
> Emilien Macchi
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] [nova] Glance bug with Kilo upgrade & Nova

[Clayton O'Neill]

Flavio's fix was merged to master and back ported to the stable/kilo
master.  We're hoping to run it through our upgrade testing tomorrow.

On Tue, Jun 9, 2015 at 8:33 AM, Flavio Percoco  wrote:

> On 09/06/15 07:38 +, Bhandaru, Malini K wrote:
>
>> A DB migrate script that puts some token default string only for glance
>> properties that are NULL. Should not change anything else.
>> Hope it works Flavio.
>> Regards
>> Malini
>>
>
> Proposed fix, please read the commit message and comments:
>
> https://review.openstack.org/#/c/189661/
>
>
>
>> -Original Message-
>> From: Flavio Percoco [mailto:fla...@redhat.com]
>> Sent: Tuesday, June 09, 2015 12:33 AM
>> To: Bhandaru, Malini K
>> Cc: OpenStack Development Mailing List (not for usage questions);
>> openstack-operat...@lists.openstack.org
>> Subject: Re: [openstack-dev] [glance] [nova] Glance bug with Kilo upgrade
>> & Nova
>>
>> On 09/06/15 09:22 +0200, Flavio Percoco wrote:
>>
>>> On 09/06/15 07:09 +, Bhandaru, Malini K wrote:
>>>
>>>> Flavio, would a DB script that writes an empty string or NOP or
>>>> something instead of NULL In the column do the trick?
>>>> Then the problem degenerates to a new DB upgrade script.
>>>>
>>>
>>> I now remembered about a change[0] - that I wrote myself - that
>>> required a bump on the API version - which we did - that allows None
>>> values to be returned in the API. This is probably what is causing this
>>> behavior.
>>>
>>> A DB migration would certainly work, I'd love to avoid it but I guess
>>> that's the best solution in this case.
>>>
>>
>> Actually, we can't just migrate the database as there might be custom
>> properties that were explicitly set to None.
>>
>> I'll keep you posted
>>
>>
>>> Cheers,
>>> Flavio
>>>
>>> [0] https://review.openstack.org/#/c/138184/
>>>
>>>
>>>> Regards
>>>> Malini
>>>>
>>>> -Original Message-
>>>> From: Flavio Percoco [mailto:fla...@redhat.com]
>>>> Sent: Monday, June 08, 2015 11:47 PM
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> Cc: openstack-operat...@lists.openstack.org
>>>> Subject: Re: [openstack-dev] [glance] [nova] Glance bug with Kilo
>>>> upgrade & Nova
>>>>
>>>> On 08/06/15 11:46 -0400, Clayton O'Neill wrote:
>>>>
>>>>> We tested testing Kilo upgrades in our hardware dev environments last
>>>>> week and the second time through ran into this bug which right now is
>>>>> probably a show-stopper for us.
>>>>>
>>>>> https://bugs.launchpad.net/glance/+bug/1419823
>>>>>
>>>>> The issue here is that the v1 Glance API allows you to create images
>>>>> with properties that are 'NULL' in the Glance database.  For example:
>>>>>
>>>>> glance image-create --name cirros_test --disk-format qcow2
>>>>> --container-format bare --file cirros-0.3.4-x86_64-disk.img
>>>>> --is-public True --is-protected True --progress --property
>>>>> description=
>>>>>
>>>>> It's apparently also fairly easy to end up with a NULL description
>>>>> when editing images properties via Horizon.
>>>>>
>>>>> The issue is that the v2 Glance API returns these NULL properties to
>>>>> the client, which then validates them against the schema returned by
>>>>> the v2 API.
>>>>> This schema returns specifies that the description property *must* be
>>>>> a string.
>>>>>
>>>>> In the Kilo release, Nova has been changed to use the v2 API, so
>>>>> suddenly this matters.  The net effect is that end users can pretty
>>>>> easily create properties with NULL values, and then won't be able to
>>>>> boot instances using those images.
>>>>> What makes this worse is that it's completely opaque to end users,
>>>>> since this just reports that no node was available to schedule the
>>>>> instance.
>>>>>
>>>>> However, Nova *only* uses the v2 api to list images if the
>>>>> glance.allowed_direct_url_schemes config key is set in the config file.
>>>>> However, this config item defaults to

[openstack-dev] [glance] [nova] Glance bug with Kilo upgrade & Nova

[Clayton O'Neill]

We tested testing Kilo upgrades in our hardware dev environments last week
and the second time through ran into this bug which right now is probably a
show-stopper for us.

https://bugs.launchpad.net/glance/+bug/1419823

The issue here is that the v1 Glance API allows you to create images with
properties that are 'NULL' in the Glance database.  For example:

glance image-create --name cirros_test --disk-format qcow2
--container-format bare --file cirros-0.3.4-x86_64-disk.img --is-public
True --is-protected True --progress --property description=

It's apparently also fairly easy to end up with a NULL description when
editing images properties via Horizon.

The issue is that the v2 Glance API returns these NULL properties to the
client, which then validates them against the schema returned by the v2
API.  This schema returns specifies that the description property *must* be
a string.

In the Kilo release, Nova has been changed to use the v2 API, so suddenly
this matters.  The net effect is that end users can pretty easily create
properties with NULL values, and then won't be able to boot instances using
those images.  What makes this worse is that it's completely opaque to end
users, since this just reports that no node was available to schedule the
instance.

However, Nova *only* uses the v2 api to list images if the
glance.allowed_direct_url_schemes config key is set in the config file.
However, this config item defaults to an empty array, meaning that by
default it's *always* set.  There doesn't appear to be a way to unset a
value with oslo-config that has a default value, blocking off that route to
work around the issue.  Disabling the v2 Glance API we don't think will
work, since Nova appears to assume the v2 API is available.

Another work around we've looked at is to change the DB schema for image
properties (yuck) to not allow NULL values.  This results in Glance
returning a 500 error since glance-api is attempting to insert an invalid
value.  This is better than instances failing in an opaque fashion, but
still pretty horrible.

Has anyone else run into this issue yet?  Are there other work arounds that
we've not thought of other than "Don't create images with NULL properties?"
  User education is definitely an option, but given the failure mode, it's
not a great solution for us.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] RabbitMQ deprecation

[Clayton O'Neill]

I've spent some time trying to eliminate the number of deprecation warnings
we have in preparation for our upgrade to Kilo.  One of the ones that I got
stuck on is the nova::rabbitmq::rabbitmq_class parameter.

If this parameter is specified, and it is by default, then the class will
include the rabbitmq class given by name, but it also the sets up useful
dependencies between nova and RabbitMQ.  For example, it will ensure that
RabbitMQ is running before nova is started.

It seems to me that this needs to be split into two sets of functionality.

*rabbitmq_class*
* Not deprecated
* if provided then it will be used to set up dependencies.
* Defaults to rabbitmq.
* This should perhaps just be hard coded to be rabbitmq now.

*manage_rabbitmq*
* Deprecated
* If specified, configures and sets up rabbitmq, including the class.
* Defaults to ???

Thoughts?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] Managing config file values and parameter defaults

[Clayton O'Neill]

How to handle config file values, defaults and how they relate to manifest
parameters was brought up in the weekly meeting up a few weeks ago.  I
promised to put something together.  I've put written up my thoughts and my
understanding of the other viewpoints presented there in an ether pad and
you can find the link below.  I apologize if I've not properly captured
things accurately.  I'd like to get feedback on this, either in the ether
pad or on the list.  I realize we might not be able to work this out
entirely before the summit, but we may be able to do some of the heavy
lifting and discuss it further there

Ultimately I'd like to end up with a blueprint that describes how we want
to handle different types of config file options.  It will likely be a huge
amount of work to completely implement any policy we come up with.
However, if we can agree on a policy then we can at least ensure any new
changes follow it, and over time start working on the rest of the code base.

https://etherpad.openstack.org/p/puppet-config-parameter-defaults
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] rabbit/kombu settings deprecations

[Clayton O'Neill]

On Thu, Apr 16, 2015 at 2:10 PM, Richard Raseley 
wrote:

> I am certainly sympathetic to your situation - having the world change
> beneath your feet is never a good place to be. =]
>
> It does seem, however, that there is a disconnect between your
> expectations (as I understand them) of what the 'master' branch represents,
> and what it has traditionally represented - the 'bleeding edge'
>
> Master is volatile - it is expected to be volatile. As the code progresses
> between releases it is expected that things can (and will) break.
>
> The solution, in my mind, is not to redefine what it means to be master,
> but rather to be more aggressive about back-porting changes to stable
> branches.
>

I agree that in theory, it should work like you suggest.  However, as they
say: “In theory, theory and practice are the same. In practice, they are
not.”  In practice, we're not as good at back porting things as everyone
would like.  I'll be as happy as anyone else to see this continue to
improve, but it's a given that it'll never be perfect.   In practice, if
you wanted to upgrade to Juno, we needed to be on master, because even in
January it wasn't ready for the sort of configurations we were running, and
we're not doing anything overly weird.

Additionally, as Matt said, those of us that are upgrading production
environments are the ones contributing back fixes that have to land in
master first, and then we have to wait for an additional back port.  We've
several times found bugs that break production deployments and had to carry
local patches, but we try to contribute everything back as quickly as
possible.  We'll continue to do this regardless, but we certainly have less
incentive to do it quickly if we're going to have to carry it locally for
the length of time a back port takes.

I am very much in favor of discussions that include your proposed solution
> above (i.e. 'current-1' support), as long as it is identified as a
> transitional step; I do pretty strongly believe that the right long-term
> model is to treat master as the 'bleeding edge' and to only pin yourself to
> stable releases.
>

We're not following master blindly.  We import most modules once a week,
run them through CI before merging them and QA them before deploying.
Doing that is *dramatically* easier than trying to update to a new,
massively different puppet module twice a year.  If there is no overlapping
compatibility between  and  then
upgrading requires making massive changes to infrastructure while making
massive changes to the automation that drives it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] rabbit/kombu settings deprecations

[Clayton O'Neill]

On Thu, Apr 16, 2015 at 10:50 AM, Emilien Macchi  wrote:

> We do our best now to backport what is backportable to stable/juno.
>

This certainly has gotten much better, but I don't think it's 100% there
yet either.  It's just a ton of work and we probably need better tooling
around this to expect it to be as good as it should be.


> FWIW, even without rabbit/kombu topic, master won't work on Juno, there
> is plenty of things that are brought in Kilo.
>

That may be the case in some areas, but we're using it without issues
(until now) on Ubuntu with the features we need.


> My opinion is we should follow other projects that use stable branches
> with doing deprecation for one (or more?) release (currently our master)
> and then drop the deprecations after some time.
>
> So I would propose this policy:
> * for new features, patch master with backward compatibility
>

Agreed, I think some of these might also be candidates for back port if
they're new "module features".  For example a new cinder backend that
existed in the previous release might get back ported if they're just
adding a new class.


> * backports relevant patches from master to stable branches (mainly bugs)
>
Agreed.


> * in the case of rabbit or any update in OpenStack upstream, update
> master without backward compatibility, except if we accept to have a lot
> of if/else in our code, and a lot of backwards to support; I'm not in
> favor of that.
>

I think I agree here also.  However, I'd like to see us avoid making
breaking changes solely to avoid deprecation warnings until x amount of
time after a release comes out.  If we're able to support some level of
backwards compatibility, then it also makes upgrading between releases a
lot easier.  Upgrading all of your packages, db schemas, etc is a lot less
scary and easier to test than upgrading all that + every OpenStack puppet
module you use at the same time.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev