Re: [openstack-dev] [tricircle] Zuul v3 integration status
Hi Boden！ Thanks for report this bug. we will talk about this bug in our meeting this week wednesday 9:00 beijing time. if you have time i would like you join it in the openstack-meeting channel. At 2018-06-15 21:56:29, "Boden Russell" wrote: >Is there anyone who can speak to the status of tricircle's adoption of >Zuul v3? > >As per [1] it doesn't seem like the project is setup properly for Zuul >v3. Thus, it's difficult/impossible to land patches like [2] that >require neutron/master + a depends on patch. > >Assuming tricircle is still being maintained, IMO we need to find a way >to get it up to speed with zuul v3; otherwise some of our neutron >efforts will be held up, or tricircle will fall behind with respect to >neutron-lib adoption. > >Thanks > > >[1] https://bugs.launchpad.net/tricircle/+bug/1776922 >[2] https://review.openstack.org/#/c/565879/ > >__ >OpenStack Development Mailing List (not for usage questions) >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cloudkitty] configuration, deployment or packaging issue?
On Mon, Jun 18, 2018 at 4:08 PM, Tobias Urdin wrote: > Hello CloudKitty team, > > > I'm having an issue with this review not going through and being stuck after > staring at it for a while now [1]. > > Is there any configuration[2] issue that are causing the error[3]? Or is the > package broken? > Likely due to https://review.openstack.org/#/c/538256/ which appears to change the metrics.yaml format. It doesn't look backwards compatible so the puppet module probably needs updating. > > Thanks for helping out! > > Best regards > > > [1] https://review.openstack.org/#/c/569641/ > > [2] > http://logs.openstack.org/41/569641/1/check/puppet-openstack-beaker-centos-7/ee4742c/logs/etc/cloudkitty/ > > [3] > http://logs.openstack.org/41/569641/1/check/puppet-openstack-beaker-centos-7/ee4742c/logs/cloudkitty/processor.txt.gz > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cloudkitty] configuration, deployment or packaging issue?
Hello CloudKitty team, I'm having an issue with this review not going through and being stuck after staring at it for a while now [1]. Is there any configuration[2] issue that are causing the error[3]? Or is the package broken? Thanks for helping out! Best regards [1] https://review.openstack.org/#/c/569641/ [2] http://logs.openstack.org/41/569641/1/check/puppet-openstack-beaker-centos-7/ee4742c/logs/etc/cloudkitty/ [3] http://logs.openstack.org/41/569641/1/check/puppet-openstack-beaker-centos-7/ee4742c/logs/cloudkitty/processor.txt.gz __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Sprint 15 Planning Summary (CI, Tempest squads)
Greetings, The TripleO CI & Tempest squads have begun work on Sprint 15. Like most of our sprints these are three weeks long and are planned on a Thursday or Friday (depending on squad) and end with a retrospective on Wednesdays. Sprint 15 runs from 2018-06-14 to 2018-07-03 More information regarding our process is available in the tripleo-specs repository [1]. Ongoing meeting notes and other detail are always available in the Squad Etherpads [2,3]. # Ruck / Rover: * weshay + quiquell * https://review.rdoproject.org/etherpad/p/ruckrover-sprint15 # CI Squad Epic: https://trello.com/c/bQuQ9aWF/802-sprint-15-ci-goals Tasks: http://ow.ly/3kDB30kypjH This sprint the CI squad is transitioning to a new topic! We are spending the first of at least three sprints on the topic of migrating tripleo CI to Zuul v3. We’ve split the team into two “tiger teams” [4] for the first portion of the sprint. One is focused on issues of networking and multinode support, the other looking at generating job configuration parent jobs, from which other jobs will derive. This work will inform a number of activities in this sprint and sprints to come. This includes topics such as refactoring of TOCI gate scripts (bash → ansible/python), full native ansible tooling, and putting into place jobs to address python3, RHEL8 (future centos), RDO on RHEL, and other topics enabled by support for an RH internal deployment of Software Factory. We will also be monitoring efforts by upstream openstack-infra to transition jobs @ review.rdoproject.org to zuul v3, on standby to assist as needed. # Tempest Squad Epic: https://trello.com/c/6QKG0HkU/801-sprint-15-python-tempestconf Tasks: http://ow.ly/XnB530kyppQ The Tempest Squad this sprint is operating with UA (chandankumar) on PTO for most of the sprint. We are focused on 3 main topics: * Closing out the remaining python-tempestconf refactor tasks for core OpenStack services * Create a presentation covering tempest plugin creation * Targeted tasks addressing RefStack certification for rhos 11,12,13 For any questions please find us in #tripleo Thanks, Matt [1] https://github.com/openstack/tripleo-specs/blob/master/specs/policy/ci-team-structure.rst [2] https://etherpad.openstack.org/p/tripleo-ci-squad-meeting [3] https://etherpad.openstack.org/p/tripleo-tempest-squad-meeting [4] https://en.wikipedia.org/wiki/Tiger_team __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc] [ptl] PTL E-mail addresses on rendered team pages
On 2018-06-18 11:54:00 -0700 (-0700), Kendall Nelson wrote: [...] > I'm 100% in agreement with Doug and Tony about not giving more work to > election officials. Speaking as an election official, its already tedious > and one more thing to keep track of (even if its automated) is not > something I would wish on future election officials. Yep, TC member hat off and election official hat on for a moment, the proposed changes (there's a similar one for showing the TC member E-mail addresses) just involve us copying over one piece of additional information we already have in the election data so is really no increased burden in that regard. -- Jeremy Stanley signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc] [ptl] PTL E-mail addresses on rendered team pages
On Sun, Jun 17, 2018 at 5:49 PM Tony Breeds wrote: > On Fri, Jun 15, 2018 at 03:05:51PM -0400, Doug Hellmann wrote: > > Excerpts from Jean-Philippe Evrard's message of 2018-06-15 17:37:02 > +0200: > > > > Not sure it'd help but one option we do is to create aliases based on > > > > the title. Though since the PTLs don't have addresses on the > openstack > > > > domain an alias may not make as much sense, it'd have to be a full > > > > account forward. It's useful for centralized spam filtering. > > > > > > I foresee this: > > > 1) We create an alias to PTL email > > > 2) PTL think that kind of emails are worth sharing with a team to > balance work > > > 3) We now have a project mailing list > > > 4) People stop using openstack-dev lists. > > > > > > But that's maybe me... > > > > > > > Yeah, setting all of that up feels like it would just be something > > else we would have to remember to do every time we have an election. > > I'm trying to reduce the number those kinds of tasks we have, so > > let's not add a new one. > > While I'm not sure that JP's scenario would eventuate I am against > adding the aliases and adding additional work for the election > officials. It's not that this would be terribly hard to automate it > just seems like duplication of data/effort whereas the change under > review is pretty straight forward. > I'm 100% in agreement with Doug and Tony about not giving more work to election officials. Speaking as an election official, its already tedious and one more thing to keep track of (even if its automated) is not something I would wish on future election officials. -Kendall (diablo_rojo) > > Yours Tony. > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] NUMA-aware live migration: easy but incomplete vs complete but hard
> For what it's worth, I think the previous patch languished for a number of > reasons other than the complexity of the code...the original author left, > the coding style was a bit odd, there was an attempt to make it work even if > the source was an earlier version, etc. I think a fresh implementation > would be less complicated to review. I'm afraid of unknowns in the resource tracker and claims mechanism. For snips and giggles, I submitted a quick patch that attempts to use a claim [1] when live migrating instances. Assuming it somehow passes CI, I have no idea if I've just opened rabbit hole of people telling me "oh but you need to do this other thing in this other place." How knows the claims code well, anyways? [1] https://review.openstack.org/576222 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack-operators] [openstack-operators][heat][oslo.db] Configure maximum number of db connections
+openstack-dev since I believe this is an issue with the Heat source code. On 06/18/2018 11:19 AM, Spyros Trigazis wrote: Hello list, I'm hitting quite easily this [1] exception with heat. The db server is configured to have 1000 max_connnections and 1000 max_user_connections and in the database section of heat conf I have these values set: max_pool_size = 22 max_overflow = 0 Full config attached. I ended up with this configuration based on this formula: num_heat_hosts=4 heat_api_workers=2 heat_api_cfn_workers=2 num_engine_workers=4 max_pool_size=22 max_overflow=0 num_heat_hosts * (max_pool_size + max_overflow) * (heat_api_workers + num_engine_workers + heat_api_cfn_workers) 704 What I have noticed is that the number of connections I expected with the above formula is not respected. Based on this formula each node (every node runs the heat-api, heat-api-cfn and heat-engine) should use up to 176 connections but they even reach 400 connections. Has anyone noticed a similar behavior? Looking through the Heat code, I see that there are many methods in the /heat/db/sqlalchemy/api.py module that use a SQLAlchemy session but never actually call session.close() [1] which means that the session will not be released back to the connection pool, which might be the reason why connections keep piling up. Not sure if there's any setting in Heat that will fix this problem. Disabling connection pooling will likely not help since connections are not properly being closed and returned to the connection pool to begin with. Best, -jay [1] Heat apparently doesn't use the oslo.db enginefacade transaction context managers either, which would help with this problem since the transaction context manager would take responsibility for calling session.flush()/close() appropriately. https://github.com/openstack/oslo.db/blob/43af1cf08372006aa46d836ec45482dd4b5b5349/oslo_db/sqlalchemy/enginefacade.py#L626 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [barbican] default devstack barbican secret store ? and big picture question ?
Hey ... a couple of NEWBY question for the Barbican Team.
I just setup a devstack with Barbican @ stable/queens .
Ran through the “Verify operation” commands (
https://docs.openstack.org/barbican/latest/install/verify.html ) ... Everything
worked.
stack@barbican:~/devstack$ openstack secret list
stack@barbican:~/devstack$ openstack secret store --name mysecret --payload
j4=]d21
+---++
| Field | Value
|
+---++
| Secret href |
http://10.10.10.17/key-manager/v1/secrets/87eb0f18-e417-45a8-ae49-187f8d8c98d1 |
| Name | mysecret
|
| Created | None
|
| Status| None
|
| Content types | None
|
| Algorithm | aes
|
| Bit length| 256
|
| Secret type | opaque
|
| Mode | cbc
|
| Expiration| None
|
+---++
stack@barbican:~/devstack$
stack@barbican:~/devstack$
stack@barbican:~/devstack$ openstack secret list
++--+---++-+---++-+--++
| Secret href
| Name | Created | Status | Content types
| Algorithm | Bit length | Secret type | Mode | Expiration |
++--+---++-+---++-+--++
|
http://10.10.10.17/key-manager/v1/secrets/87eb0f18-e417-45a8-ae49-187f8d8c98d1
| mysecret | 2018-06-18T14:47:45+00:00 | ACTIVE | {u'default': u'text/plain'} |
aes |256 | opaque | cbc | None |
++--+---++-+---++-+--++
stack@barbican:~/devstack$ openstack secret get
http://10.10.10.17/key-manager/v1/secrets/87eb0f18-e417-45a8-ae49-187f8d8c98d1
+---++
| Field | Value
|
+---++
| Secret href |
http://10.10.10.17/key-manager/v1/secrets/87eb0f18-e417-45a8-ae49-187f8d8c98d1 |
| Name | mysecret
|
| Created | 2018-06-18T14:47:45+00:00
|
| Status| ACTIVE
|
| Content types | {u'default': u'text/plain'}
|
| Algorithm | aes
|
| Bit length| 256
|
| Secret type | opaque
|
| Mode | cbc
|
| Expiration| None
|
+---++
stack@barbican:~/devstack$ openstack secret get
http://10.10.10.17/key-manager/v1/secrets/87eb0f18-e417-45a8-ae49-187f8d8c98d1
--payload
+-+-+
| Field | Value |
+-+-+
| Payload | j4=]d21 |
+-+-+
stack@barbican:~/devstack$
QUESTIONS:
· In this basic devstack setup, what is being used as the secret store ?
oE.g. /etc/barbican/barbican.conf for devstack is
Re: [openstack-dev] [tc][all] A culture change (nitpicking)
Excerpts from Zane Bitter's message of 2018-06-18 12:58:07 -0400: > Replying to myself one more time... > > On 12/06/18 17:35, Zane Bitter wrote: > > On 11/06/18 18:49, Zane Bitter wrote: > >> It's had a week to percolate (and I've seen quite a few people viewing > >> the etherpad), so here is the review: > >> > >> https://review.openstack.org/574479 > > > > In response to comments, I moved the change to the Project Team Guide > > instead of the Contributor Guide (since the latter is aimed only at new > > contributors, but this is for everyone). The new review is here: > > > > https://review.openstack.org/574888 > > > > The first review is still up, but it's now just adding links from the > > Contributor Guide to this new doc. > > This is now live: > > https://docs.openstack.org/project-team-guide/review-the-openstack-way.html > > Thanks to everyone who contributed/reviewed/commented. Let's also > remember to make this a living document, so we all keep learning from > each other :) > > cheers, > Zane. > +1 Nice work on this initiative, Julia and Zane. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all] A culture change (nitpicking)
Replying to myself one more time... On 12/06/18 17:35, Zane Bitter wrote: On 11/06/18 18:49, Zane Bitter wrote: It's had a week to percolate (and I've seen quite a few people viewing the etherpad), so here is the review: https://review.openstack.org/574479 In response to comments, I moved the change to the Project Team Guide instead of the Contributor Guide (since the latter is aimed only at new contributors, but this is for everyone). The new review is here: https://review.openstack.org/574888 The first review is still up, but it's now just adding links from the Contributor Guide to this new doc. This is now live: https://docs.openstack.org/project-team-guide/review-the-openstack-way.html Thanks to everyone who contributed/reviewed/commented. Let's also remember to make this a living document, so we all keep learning from each other :) cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [stable][horizon] Adding Ivan Kolodyazhny to horizon-stable-maint
+1 On Mon, Jun 18, 2018 at 9:15 AM Sean McGinnis wrote: > > On Mon, Jun 18, 2018 at 12:03:52PM +1000, Tony Breeds wrote: > > Hello folks, > > Recently Ivan became the Horizon PTL and as with past PTLs (Hi Rob) > > isn't a member of the horizon-stable-maint team. Ivan is a member of > > the Cinder stable team and as such has demonstrated an understanding of > > the stable policy. Since the Dublin PTG Ivan has been doing consistent > > high quality reviews on Horizon's stable branches. > > > > As such I'm suggesting adding him to the existing stable team. > > > > Without strong objections I'll do that on (my) Monday 25th June. > > > > Yours Tony. > > Speaking with both stable and Cinder hats on, Ivan has been doing good stable > reviews and I do not have any concerns about him being in any *-stable-maint > groups. > > Sean > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sdk] announcing first release of rust-openstack (+ call for contributors)
Hi all, I'd like to announce my hobby project that I've been working on for some time. rust-openstack [1], as the name suggests, is an SDK for OpenStack written in Rust! I released version 0.1.0 last week, and now the project is ready for early testers and contributors. Currently only a small subset of Compute, Networking and Image API is implemented, as well as password authentication against Identity v3. If you're interested in the Rust language, this may be your chance :) I have written a short contributor's guide [2] to help understanding the code structure. Special thanks to the OpenLab project for providing the CI for the project. Cheers, Dmitry [1] https://docs.rs/openstack/latest/openstack/ [2] https://github.com/dtantsur/rust-openstack/blob/master/CONTRIBUTING.md __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Puppet debugging help?
On Mon, Jun 18, 2018 at 11:31:08AM -0400, Mohammed Naser wrote:
> Hey Lars,
>
> Do you have a full job that's running which shows those issues?
I don't. I have a local environment where I'm doing my testing.
--
Lars Kellogg-Stedman | larsks @ {irc,twitter,github}
http://blog.oddbit.com/|
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Puppet debugging help?
On Mon, Jun 18, 2018 at 9:13 AM, Lars Kellogg-Stedman wrote:
> Hey folks,
>
> I'm trying to patch puppet-keystone to support multi-valued
> configuration options (like trusted_dashboard). I have a patch that
> works, mostly, but I've run into a frustrating problem (frustrating
> because it would seem to be orthogonal to my patches, which affect the
> keystone_config provider and type).
>
> During the initial deploy, running tripleo::profile::base::keystone
> fails with:
>
> "Error: Could not set 'present' on ensure: undefined method `new'
> for nil:NilClass at
> /etc/puppet/modules/tripleo/manifests/profile/base/keystone.pp:274",
>
It's likely erroring in the keystone_domain provider.
https://github.com/openstack/puppet-keystone/blob/master/lib/puppet/provider/keystone_domain/openstack.rb#L115-L122
or
https://github.com/openstack/puppet-keystone/blob/master/lib/puppet/provider/keystone_domain/openstack.rb#L155-L161
Providers are notoriously bad at their error messaging. Usually this
error happens when we get a null back from the underlying command and
we're still trying to do something. This could point to a
misconfiguration of keystone if it's not getting anything back.
> The line in question is:
>
> 70: if $step == 3 and $manage_domain {
> 71: if hiera('heat_engine_enabled', false) {
> 72: # create these seperate and don't use ::heat::keystone::domain since
> 73: # that class writes out the configs
> 74: keystone_domain { $heat_admin_domain:
> ensure => 'present',
> enabled => true
> }
>
> The thing is, despite the error...it creates the keystone domain
> *anyway*, and a subsequent run of the module will complete without any
> errors.
>
> I'm not entirely sure that the error is telling me, since *none* of
> the puppet types or providers have a "new" method as far as I can see.
> Any pointers you can offer would be appreciated.
>
> Thanks!
>
> --
> Lars Kellogg-Stedman | larsks @ {irc,twitter,github}
> http://blog.oddbit.com/|
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] NUMA-aware live migration: easy but incomplete vs complete but hard
On 06/18/2018 08:16 AM, Artom Lifshitz wrote: Hey all, For Rocky I'm trying to get live migration to work properly for instances that have a NUMA topology [1]. A question that came up on one of patches [2] is how to handle resources claims on the destination, or indeed whether to handle that at all. I think getting the live migration to work at all is better than having it stay broken, so even without resource claiming on the destination it's an improvement over the status quo and I think it'd be a desirable change. However, *not* doing resource claiming means that until the migration is complete and the regular resource audit runs on the destination (which could be a minute later by default) you could end up having other instances try to use the same resources, causing various operations to fail. I think we'd want to have a very clear notice in the release notes about the limitations if we go this route. I'm a little bit worried that waiting for support in placement will result in "fully-functional" live migration with dedicated resources being punted out indefinitely. One of the reasons why the spec[1] called for using the existing resource tracker was that we don't expect placement to be functional for all NUMA-related stuff for a while yet. For what it's worth, I think the previous patch languished for a number of reasons other than the complexity of the code...the original author left, the coding style was a bit odd, there was an attempt to make it work even if the source was an earlier version, etc. I think a fresh implementation would be less complicated to review. Given the above, my personal preference would be to merge it even without claims, but then try to get the claims support merged as well. (Adding claims support later on wouldn't change any on-the-wire messaging, it would just make things work more robustly.) Chris [1] https://github.com/openstack/nova-specs/blob/master/specs/rocky/approved/numa-aware-live-migration.rst __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sig][upgrade] Upgrade SIG IRC meeting 1600 UTC Tuesday
Hi All Just a quick reminder that the Upgrade SIG IRC meeting will be held at 1600 UTC tomorrow (Tuesday) in #openstack-meeting-4. If you're interested in helping improve the OpenStack upgrade experience be sure to attend! See [0] for previous meeting minutes and our standing agenda. Regards James [0] https://etherpad.openstack.org/p/upgrades-sig-meeting __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Puppet debugging help?
Hey Lars,
Do you have a full job that's running which shows those issues?
Thanks,
Mohammed
On Mon, Jun 18, 2018 at 11:13 AM, Lars Kellogg-Stedman wrote:
> Hey folks,
>
> I'm trying to patch puppet-keystone to support multi-valued
> configuration options (like trusted_dashboard). I have a patch that
> works, mostly, but I've run into a frustrating problem (frustrating
> because it would seem to be orthogonal to my patches, which affect the
> keystone_config provider and type).
>
> During the initial deploy, running tripleo::profile::base::keystone
> fails with:
>
> "Error: Could not set 'present' on ensure: undefined method `new'
> for nil:NilClass at
> /etc/puppet/modules/tripleo/manifests/profile/base/keystone.pp:274",
>
> The line in question is:
>
> 70: if $step == 3 and $manage_domain {
> 71: if hiera('heat_engine_enabled', false) {
> 72: # create these seperate and don't use ::heat::keystone::domain since
> 73: # that class writes out the configs
> 74: keystone_domain { $heat_admin_domain:
> ensure => 'present',
> enabled => true
> }
>
> The thing is, despite the error...it creates the keystone domain
> *anyway*, and a subsequent run of the module will complete without any
> errors.
>
> I'm not entirely sure that the error is telling me, since *none* of
> the puppet types or providers have a "new" method as far as I can see.
> Any pointers you can offer would be appreciated.
>
> Thanks!
>
> --
> Lars Kellogg-Stedman | larsks @ {irc,twitter,github}
> http://blog.oddbit.com/|
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Mohammed Naser — vexxhost
-
D. 514-316-8872
D. 800-910-1726 ext. 200
E. mna...@vexxhost.com
W. http://vexxhost.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] increasing the number of allowed volumes attached per instance > 26
Hi, this looks reasonable to me but I would prefer B. In this case the operator can configure the hard limit. I don't think we more granularity or expose it using the API. Belmiro On Fri, Jun 8, 2018 at 3:46 PM Dan Smith wrote: > > Some ideas that have been discussed so far include: > > FYI, these are already in my order of preference. > > > A) Selecting a new, higher maximum that still yields reasonable > > performance on a single compute host (64 or 128, for example). Pros: > > helps prevent the potential for poor performance on a compute host > > from attaching too many volumes. Cons: doesn't let anyone opt-in to a > > higher maximum if their environment can handle it. > > I prefer this because I think it can be done per virt driver, for > whatever actually makes sense there. If powervm can handle 500 volumes > in a meaningful way on one instance, then that's cool. I think libvirt's > limit should likely be 64ish. > > > B) Creating a config option to let operators choose how many volumes > > allowed to attach to a single instance. Pros: lets operators opt-in to > > a maximum that works in their environment. Cons: it's not discoverable > > for those calling the API. > > This is a fine compromise, IMHO, as it lets operators tune it per > compute node based on the virt driver and the hardware. If one compute > is using nothing but iSCSI over a single 10g link, then they may need to > clamp that down to something more sane. > > Like the per virt driver restriction above, it's not discoverable via > the API, but if it varies based on compute node and other factors in a > single deployment, then making it discoverable isn't going to be very > easy anyway. > > > C) Create a configurable API limit for maximum number of volumes to > > attach to a single instance that is either a quota or similar to a > > quota. Pros: lets operators opt-in to a maximum that works in their > > environment. Cons: it's yet another quota? > > Do we have any other quota limits that are per-instance like this would > be? If not, then this would likely be weird, but if so, then this would > also be an option, IMHO. However, it's too much work for what is really > not a hugely important problem, IMHO, and both of the above are > lighter-weight ways to solve this and move on. > > --Dan > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [barbican] NEW weekly meeting time
Based on popular demand, the new meeting time is now active. We will meet at Tuesday 12:00 UTC starting this week. redrobot and Dave will chair the next two meetings as I'm on vacation. Ade On Sat, 2018-06-16 at 11:11 +0300, Juan Antonio Osorio wrote: > +1 I dig > > On Fri, 15 Jun 2018, 17:41 Dave McCowan (dmccowan), om> wrote: > > +1 > > This is a great time. > > > > On 6/14/18, 4:30 PM, "Ade Lee" wrote: > > > > >The new time slot has been pretty difficult for folks to attend. > > >I'd like to propose a new time slot, which will hopefully be more > > >amenable to everyone. > > > > > >Tuesday 12:00 UTC > > > > > >https://www.timeanddate.com/worldclock/fixedtime.html?hour=12= > > 00 > > >c=0 > > > > > >This works out to 8 am EST, around 1pm in Europe, and 8 pm in > > China. > > >Please vote by responding to this email. > > > > > >Thanks, > > >Ade > > > > > >__ > > > > >OpenStack Development Mailing List (not for usage questions) > > >Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:uns > > ubscribe > > >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > ___ > > ___ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu > > bscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _ > _ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [stable][horizon] Adding Ivan Kolodyazhny to horizon-stable-maint
On Mon, Jun 18, 2018 at 12:03:52PM +1000, Tony Breeds wrote: > Hello folks, > Recently Ivan became the Horizon PTL and as with past PTLs (Hi Rob) > isn't a member of the horizon-stable-maint team. Ivan is a member of > the Cinder stable team and as such has demonstrated an understanding of > the stable policy. Since the Dublin PTG Ivan has been doing consistent > high quality reviews on Horizon's stable branches. > > As such I'm suggesting adding him to the existing stable team. > > Without strong objections I'll do that on (my) Monday 25th June. > > Yours Tony. Speaking with both stable and Cinder hats on, Ivan has been doing good stable reviews and I do not have any concerns about him being in any *-stable-maint groups. Sean __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Puppet debugging help?
Hey folks,
I'm trying to patch puppet-keystone to support multi-valued
configuration options (like trusted_dashboard). I have a patch that
works, mostly, but I've run into a frustrating problem (frustrating
because it would seem to be orthogonal to my patches, which affect the
keystone_config provider and type).
During the initial deploy, running tripleo::profile::base::keystone
fails with:
"Error: Could not set 'present' on ensure: undefined method `new'
for nil:NilClass at
/etc/puppet/modules/tripleo/manifests/profile/base/keystone.pp:274",
The line in question is:
70: if $step == 3 and $manage_domain {
71: if hiera('heat_engine_enabled', false) {
72: # create these seperate and don't use ::heat::keystone::domain since
73: # that class writes out the configs
74: keystone_domain { $heat_admin_domain:
ensure => 'present',
enabled => true
}
The thing is, despite the error...it creates the keystone domain
*anyway*, and a subsequent run of the module will complete without any
errors.
I'm not entirely sure that the error is telling me, since *none* of
the puppet types or providers have a "new" method as far as I can see.
Any pointers you can offer would be appreciated.
Thanks!
--
Lars Kellogg-Stedman | larsks @ {irc,twitter,github}
http://blog.oddbit.com/|
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova]Notification update week 25
Hi, Here is the latest notification subteam update. Bugs No update on bugs and we have no new bugs tagged with notifications. Features Sending full traceback in versioned notifications ~ https://blueprints.launchpad.net/nova/+spec/add-full-traceback-to-error-notifications We are really close to merge https://review.openstack.org/#/c/564092/ but some nits still needs to be addressed. Add notification support for trusted_certs ~~ The notification impact of the trusted_certs bp has been merged. \o/ Introduce Pending VM state ~~ The spec https://review.openstack.org/#/c/554212 still not exactly define what will be in the select_destination notification payload. Add the user id and project id of the user initiated the instance action to the notification https://blueprints.launchpad.net/nova/+spec/add-action-initiator-to-instance-action-notifications Good progress on the implementation https://review.openstack.org/#/c/536243 No progress: * Versioned notification transformation https://review.openstack.org/#/q/topic:bp/versioned-notification-transformation-rocky+status:open * Introduce instance.lock and instance.unlock notifications https://blueprints.launchpad.net/nova/+spec/trigger-notifications-when-lock-unlock-instances Blocked: * Add versioned notifications for removing a member from a server group https://blueprints.launchpad.net/nova/+spec/add-server-group-remove-member-notifications Weekly meeting -- The next meeting will be held on 19th of June on #openstack-meeting-4 https://www.timeanddate.com/worldclock/fixedtime.html?iso=20180619T17 Cheers, gibi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Reminder: UC Meeting Today 1800UTC
Hey everyone, Please see https://wiki.openstack.org/wiki/Governance/Foundation/ UserCommittee for UC meeting info and add additional agenda items if needed. -- Kind regards, Melvin Hillsman mrhills...@gmail.com mobile: (832) 264-2646 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Newton branch is End-Of-Life
After many postpones, we finally went ahead and closed Newton branch for TripleO repositories A last tag was created and from now we won't accept backports in this branch. RPMs in RDO will be updated with this last tag. If there is any question or concern, please let us know. PS: Thanks to the stable-maint/release-managers who helped in that process. -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][python3] advice needed with updating lib-forward-testing jobs
On Mon, 18 Jun 2018 22:04:38 +0900 Doug Hellmann
wrote
> Excerpts from Ghanshyam Mann's message of 2018-06-18 20:38:00 +0900:
> > On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann
> > wrote
> > > Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
> > > > Doug Hellmann writes:
> > > >
> > > > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
> > > >
> > > > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
> > > > >> explicitly. But gate running on any repo does run job on current
> > > > >> change set of that repo which is nothing but "master + current
> > patch
> > > > >> changes" . For example, any job running on oslo.config patch will
> > > > >> take oslo.config source code from that patch which is "master +
> > > > >> current change". You can see the results in this patch -
> > > > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
> > > > >> and gate jobs (including tempest-full-py3) fails as they run on
> > > > >> current change set of neutron-lib code not on pypi version(which
> > > > >> would pass the tests).
> > > > >
> > > > > The tempest-full-py3 job passed for that patch, though. Which seems
> > to
> > > > > indicate that the neutron-lib repository was not used in the test
> > job,
> > > > > even though it was checked out.
> > > >
> > > > The automatic generation of LIBS_FROM_GIT only includes projects which
> > > > appear in required-projects. So in this case neutron-lib does not
> > > > appear in LIBS_FROM_GIT[1], so the change is not actually tested by
> > that
> > > > job.
> >
> > Yes, this is now clear to me. I was in impressions of treating the lib
> > same way as service for testing repo always from source but that's not the
> > case.
> >
> > > >
> > > > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would
> > work,
> > > > but anytime LIBS_FROM_GIT is set explicitly, it turns off the
> > automatic
> > > > generation, so more complex jobs (which may want to inherit from that
> > > > job but need multiple libraries) would also have to override
> > > > LIBS_FROM_GIT and add the full set of projects.
> > > >
> > > > The code that automatically sets LIBS_FROM_GIT is fairly simple and
> > > > could be modified to automatically add the project of the change under
> > > > test. We could do that for all jobs, or we could add a flag which
> > > > toggles the behavior. The question to answer here is: is there ever a
> > > > case where a devstack job should not install the change under test
> > from
> > > > source? I think the answer is no, and even though under Zuul v2
> > > > devstack-gate didn't automatically add the project under test to
> > > > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
> > > > templating which did.
> > >
> > > Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
> > > like the behavior I would expect from the job.
> >
> > ++
> >
> > >
> > > > A further thing to consider is what the desired behavior is for a
> > series
> > > > of changes. If a change to neutron-lib depends on a change to
> > > > oslo.messaging, when the forward testing job runs on neutron-lib,
> > should
> > > > it also add oslo.messaging to LIBS_FROM_GIT? That's equally easy to
> > > > implement (but would certainly need a flag as it essentially would add
> > > > everything in the change series to LIBS_FROM_GIT which defeats the
> > > > purpose of the restriction for the jobs which need it), but I honestly
> > > > am not certain what's desired.
> > >
> > > I think going ahead and adding everything in the dependency chain
> > > also makes sense. If I have 2 changes in libraries and a change in
> > > a service and I want to test them all together I would expect to
> > > be able to do that by using Depends-On and then for all 3 to be
> > > installed from source in the job that runs.
> >
> > Yes, I agree on testing all series(either alone repo or depends-on on lib
> > or service) with installed from source.
> >
> > >
> > > >
> > > > For each type of project (service, lib, lib-group (eg
> > oslo.messaging)),
> > > > what do we want to test from git vs pypi?
> > >
> > > We want to test changes to service projects with libraries from
> > > PyPI so that we do not end up with services that rely on unreleased
> > > features of libraries.
> > >
> > > We want to test changes to libraries with some services installed
> > > from git so that we know changes to the library do not break (current)
> > > master of the service. The set of interesting services may vary,
> > > but a default set that represents the tightly coupled services that
> > > run in the integrated gate now is reasonable.
> > >
> > > > How many jobs are needed to accomplish that?
> > >
> > > Ideally 1?
[openstack-dev] [nova] NUMA-aware live migration: easy but incomplete vs complete but hard
Hey all, For Rocky I'm trying to get live migration to work properly for instances that have a NUMA topology [1]. A question that came up on one of patches [2] is how to handle resources claims on the destination, or indeed whether to handle that at all. The previous attempt's approach [3] (call it A) was to use the resource tracker. This is race-free and the "correct" way to do it, but the code is pretty opaque and not easily reviewable, as evidenced by [3] sitting in review purgatory for literally years. A simpler approach (call it B) is to ignore resource claims entirely for now and wait for NUMA in placement to land in order to handle it that way. This is obviously race-prone and not the "correct" way of doing it, but the code would be relatively easy to review. For the longest time, live migration did not keep track of resources (until it started updating placement allocations). The message to operators was essentially "we're giving you this massive hammer, don't break your fingers." Continuing to ignore resource claims for now is just maintaining the status quo. In addition, there is value in improving NUMA live migration *now*, even if the improvement is incomplete because it's missing resource claims. "Best is the enemy of good" and all that. Finally, making use of the resource tracker is just work that we know will get thrown out once we start using placement for NUMA resources. For all those reasons, I would favor approach B, but I wanted to ask the community for their thoughts. Thanks! [1] https://review.openstack.org/#/q/topic:bp/numa-aware-live-migration+(status:open+OR+status:merged) [2] https://review.openstack.org/#/c/567242/ [3] https://review.openstack.org/#/c/244489/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tc] Technical Committee status for 18 June
This is the weekly summary of work being done by the Technical Committee members. The full list of active items is managed in the wiki: https://wiki.openstack.org/wiki/Technical_Committee_Tracker We also track TC objectives for the cycle using StoryBoard at: https://storyboard.openstack.org/#!/project/923 == Recent Activity == Project updates: * Add afsmon project under infra, https://review.openstack.org/572527 * Add new roles to OpenStack-Ansible, https://review.openstack.org/572556 Other approved changes: * document house-rule for chair-proposed typo fixes, https://review.openstack.org/572811 * showing PTL and TC email addresses on gov site: https://review.openstack.org/#/c/575554/2 and https://review.openstack.org/#/c/575797/2 * the principle "We Value Constructive Peer Review": https://review.openstack.org/#/c/570940/ * related changes to the project team guide to add review guidelines: https://review.openstack.org/#/c/574888/ and https://docs.openstack.org/project-team-guide/review-the-openstack-way.html * The consensus seemed to be that we should go ahead and update past goal documents when projects complete them, so I approved the patch to add details to the python 3.5 goal for Kolla: https://review.openstack.org/#/c/557863 Office hour logs from last week: * http://eavesdrop.openstack.org/meetings/tc/2018/tc.2018-06-12-09.02.html * http://eavesdrop.openstack.org/meetings/tc/2018/tc.2018-06-13-01.00.html * http://eavesdrop.openstack.org/meetings/tc/2018/tc.2018-06-14-15.01.html A few folks expressed concern that using the meeting bot to record the office hours made them more like a meeting. It would be useful to have some feedback from the community about whether having the logs separate is helfpul, or if linking to the timestamp in the regular channel logs would be sufficient. == Ongoing Discussions == I filled out the remaining liaison assignments for TC members to be responsible for reaching out to project teams. Our goal is to check in with each team between now and the PTG, and record notes in the wiki. Information about several teams is already available there. * http://lists.openstack.org/pipermail/openstack-dev/2018-June/131507.html * https://wiki.openstack.org/wiki/OpenStack_health_tracker The resolution layout out the Python 2 deprecation timeline and deadline for supporting Python 3 is approved and I have started writing up a "python 3 first" goal to be considered for Stein. * https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html * https://review.openstack.org/#/c/575933/ Zane's proposal to clarify diversity requirements for new projects is receiving some discussion. * https://review.openstack.org/567944 Zane has started a draft of a technical vision for OpenStack * https://etherpad.openstack.org/p/tech-vision-2018 == TC member actions/focus/discussions for the coming week(s) == Jeremy's proposal to add a "Castellan-compatible key store" to the base services list seems to have good support but has not yet reached majority. TC members, please review. * https://review.openstack.org/572656 Ian's update to the PTI for documentation translation and PDF generation has some review feedback and needs to be updated. * https://review.openstack.org/572559 == Contacting the TC == The Technical Committee uses a series of weekly "office hour" time slots for synchronous communication. We hope that by having several such times scheduled, we will have more opportunities to engage with members of the community from different timezones. Office hour times in #openstack-tc: * 09:00 UTC on Tuesdays * 01:00 UTC on Wednesdays * 15:00 UTC on Thursdays If you have something you would like the TC to discuss, you can add it to our office hour conversation starter etherpad at:https://etherpad.openstack.org/p/tc-office-hour-conversation-starters Many of us also run IRC bouncers which stay in #openstack-tc most of the time, so please do not feel that you need to wait for an office hour time to pose a question or offer a suggestion. You can use the string "tc-members" to alert the members to your question. If you expect your topic to require significant discussion or to need input from members of the community other than the TC, please start a mailing list discussion on openstack-dev at lists.openstack.org and use the subject tag "[tc]" to bring it to the attention of TC members. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config-download/ansible next steps
On Mon, Jun 18, 2018 at 1:51 PM, Dmitry Tantsur wrote:
> On 06/13/2018 03:17 PM, James Slagle wrote:
>>
>> On Wed, Jun 13, 2018 at 6:49 AM, Dmitry Tantsur
>> wrote:
>>>
>>> Slightly hijacking the thread to provide a status update on one of the
>>> items
>>> :)
>>
>>
>> Thanks for jumping in.
>>
>>
>>> The immediate plan right now is to wait for metalsmith 0.4.0 to hit the
>>> repositories, then start experimenting. I need to find a way to
>>> 1. make creating nova instances no-op
>>> 2. collect the required information from the created stack (I need
>>> networks,
>>> ports, hostnames, initial SSH keys, capabilities, images)
>>> 3. update the config-download code to optionally include the role [2]
>>> I'm not entirely sure where to start, so any hints are welcome.
>>
>>
>> Here are a couple of possibilities.
>>
>> We could reuse the OS::TripleO::{{role.name}}Server mappings that we
>> already have in place for pre-provisioned nodes (deployed-server).
>> This could be mapped to a template that exposes some Ansible tasks as
>> outputs that drives metalsmith to do the deployment. When
>> config-download runs, it would execute these ansible tasks to
>> provision the nodes with Ironic. This has the advantage of maintaining
>> compatibility with our existing Heat parameter interfaces. It removes
>> Nova from the deployment so that from the undercloud perspective you'd
>> roughly have:
>>
>> Mistral -> Heat -> config-download -> Ironic (driven via
>> ansible/metalsmith)
>
>
> One thing that came to my mind while planning this work is that I'd prefer
> all nodes to be processed in one step. This will help avoiding some issues
> that we have now. For example, the following does not work reliably:
>
> compute-0: just any profile:compute
> compute-1: precise node=abcd
> control-0: any node
>
> This has two issues that will pop up randomly:
> 1. compute-0 can pick node abcd designated for compute-1
> 2. control-0 can pick a compute node, failing either compute-0 or compute-1
>
> This problem is hard to fix if all deployment requests are processed
> separately, but is quite trivial if the decision is done based on the whole
> deployment plan. I'm going to work on a bulk scheduler like that in
> metalsmith.
>
>>
>> A further (or completely different) iteration might look like:
>>
>> Step 1: Mistral -> Ironic (driven via ansible/metalsmith)
>> Step 2: Heat -> config-download
>
>
> Step 1 will still use provided environment to figure out the count of nodes
> for each role, their images, capabilities and (optionally) precise node
> scheduling?
> I'm a bit worried about the last bit: IIRC we rely on Heat's %index%
> variable currently. We can, of course, ask people to replace it with
> something more explicit on upgrade.
>
>>
>> Step 2 would use the pre-provisioned node (deployed-server) feature
>> already existing in TripleO and treat the just provisioned by Ironic
>> nodes, as pre-provisioned from the Heat stack perspective. Step 1 and
>> Step 2 would also probably be driven by a higher level Mistral
>> workflow. This has the advantage of minimal impact to
>> tripleo-heat-templates, and also removes Heat from the baremetal
>> provisioning step. However, we'd likely need some python compatibility
>> libraries that could translate Heat parameter values such as
>> HostnameMap to ansible vars for some basic backwards compatibility.
>
>
> Overall, I like this option better. It will allow an operator to isolate the
> bare metal provisioning step from everything else.
>
>>
>>>
>>> [1] https://github.com/openstack/metalsmith
>>> [2] https://metalsmith.readthedocs.io/en/latest/user/ansible.html
>>>
Obviously we have things to consider here such as backwards
compatibility
and
upgrades, but overall, I think this would be a great simplification to
our
overall deployment workflow.
>>>
>>> Yeah, this is tricky. Can we make Heat "forget" about Nova instances?
>>> Maybe
>>> by re-defining them to OS::Heat::None?
>>
>>
>> Not exactly, as Heat would delete the previous versions of the
>> resources. We'd need some special migrations, or could support the
>> existing method forever for upgrades, and only deprecate it for new
>> deployments.
>
>
> Do I get it right that if we redefine OS::TripleO::{{role.name}}Server to be
> OS::Heat::None, Heat will delete the old {{role.name}}Server instances on
> the next update? This is sad..
>
> I'd prefer not to keep Nova support forever, this is going to be hard to
> maintain and cover by the CI. Should we extend Heat to support "forgetting"
> resources? I think it may have a use case outside of TripleO.
This is already supported, it's just not the default:
https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#resources-section
you can used e.g deletion_policy: retain to skip the deletion of the
underlying heat-managed resource.
Steve
__
OpenStack Development Mailing
Re: [openstack-dev] [horizon][plugins] Introduce horizonlib (again)
Hi Doug, We discussed this option a bit too. Maybe we need to think about this again. >From my point of view, it would be better to keep current release model for now, because we've got a very small amount of active horizon contributors, so current release model helps us deliver the project in time. From the other side, your option is less complicated and could be easier to implement. Let's get more feedback from the team before further discussion. Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Wed, Jun 13, 2018 at 6:43 PM, Doug Hellmann wrote: > Excerpts from Ivan Kolodyazhny's message of 2018-06-13 18:01:26 +0300: > > Hi team, > > > > Last week on the Horizon meeting we discussed [1] possible options for > > Horizon release model to address current issues for plugins maintainers. > > Some background could be found here [2]. > > > > The main issue is that we should have some stable API for plugins and be > > able to release it as needed. We're trying to cover several use cases > with > > this effort. E.g: > > - do not break plugins with Horizon changes (cross-project CI would help > > with some issues here too) > > - provide an easy way to develop plugins which require specific Horizon > > version and features > > > > For now, most of the plugins use 'horizon' package to implement > > dashboard extensions. Some plugins use parts of 'openstack_dashboard' > > package. In such case, it becomes complicated to develop plugins based on > > current master and have CI up and running. > > > > The idea is to introduce something like 'horizonlib' or 'horizon-sdk' > with > > a stable API for plugin development. We're going to collect everything > > needed for this library, so plugins developers could consume only it and > do > > not relate on any internal Horizon things. > > > > We'd got horizonlib in the past. Unfortunately, we missed information > about > > what was good or bad but we'll do our best to succeed in this. > > > > > > If you have any comments or questions, please do not hesitate to drop few > > words into this conversation or ping me in IRC. We're going to collect as > > much feedback as we can before we'll discuss it in details during the > next > > PTG. > > > > > > [1] > > http://eavesdrop.openstack.org/meetings/horizon/2018/ > horizon.2018-06-06-15.01.log.html#l-29 > > [2] > > http://lists.openstack.org/pipermail/openstack-dev/2018- > March/128310.html > > > > Regards, > > Ivan Kolodyazhny, > > http://blog.e0ne.info/ > > Another solution that may end up being less work is to release Horizon > using the cycle-with-intermediary model and publish the releases to > PyPI. Those two changes would let you release changes at any point in > the cycle, to support your plugin authors, and would not require > reorganizing the code in Horizon to build a new release artifact. > > The release team would be happy to offer advice about how to make the > changes, if you want to talk about it. > > Doug > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon][plugins] Third-party JavaScript libraries and Xstatic Python packages
Fixed horizon tag in the subject :( Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ On Mon, Jun 18, 2018 at 4:11 PM, Ivan Kolodyazhny wrote: > Hi team, > > As you may know, Horizon uses both Python and JavaScript dependencies as > well. All of our JS dependencies are packed into the xstatic-* packages > which could be installed like a regular python package. All current > xstatic-* packages could be found on the Horizon's deliverables list [1]. > > We need all of these things to make development and packaging processes > easier. > > Of course, we can't cover all cases and JS libs, so there is a manual on > how to create new xstatic package [2]. > > > Historically, all xstatic-* projects were maintained by Horizon Core > team. Probably, they were introduced even before we've got plugins > implemented but it doesn't matter at this point. > > Today, when we've got dozens of horizon plugins [3], some of them would > like to use new xstatic packages which are not existing at the moment. E.g.: > - heat-dashboard uses some JS libs which are nor required by Horizon > itself. > - vitrage-dashboard team is going to use React in their plugin. > > We discussed it briefly on the last meeting [4], As Horizon team, we don't > want to forbid using some 3rd-party JS lib if it's acceptable by license. > From the other side, Horizon Core team can't maintain all xstatic-* > packages which could be needed by plugins. That's why we decided to add > some folks form heat-dashboard team to xstatic-* core for that projects, > which are used only be Heat Dashboard now. IMO, it looks like a reasonable > and fair enough solution. > > I think we're OK if plugin teams will share the responsibility to maintain > their xstatic-* dependencies both with Horizon team following current > guidelines [2]. > > Maintaining xstatic-* project means: > - create this repo according to the current guidelines > - follow community rules according to stable policy > - publish a new version of xstatic project if it's required by bugfixes, > security fixes, new features > - help other teams to integrate this project into their plugin. > > > I hope if we agree on things above it helps both Horizon and plugin teams > to deliver new versions faster with a limited teams capacity. > > > > [1] https://governance.openstack.org/tc/reference/projects/horizon.html# > deliverables > [2] https://docs.openstack.org/horizon/latest/contributor/ > contributing.html#javascript-and-css-libraries-using-xstatic > [3] https://docs.openstack.org/horizon/latest/install/plugin-registry.html > [4] http://eavesdrop.openstack.org/meetings/horizon/2018/horizon.2018-06- > 13-15.04.log.html#l-124 > > > > Regards, > Ivan Kolodyazhny, > http://blog.e0ne.info/ > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horiozn][plugins] Third-party JavaScript libraries and Xstatic Python packages
Hi team, As you may know, Horizon uses both Python and JavaScript dependencies as well. All of our JS dependencies are packed into the xstatic-* packages which could be installed like a regular python package. All current xstatic-* packages could be found on the Horizon's deliverables list [1]. We need all of these things to make development and packaging processes easier. Of course, we can't cover all cases and JS libs, so there is a manual on how to create new xstatic package [2]. Historically, all xstatic-* projects were maintained by Horizon Core team. Probably, they were introduced even before we've got plugins implemented but it doesn't matter at this point. Today, when we've got dozens of horizon plugins [3], some of them would like to use new xstatic packages which are not existing at the moment. E.g.: - heat-dashboard uses some JS libs which are nor required by Horizon itself. - vitrage-dashboard team is going to use React in their plugin. We discussed it briefly on the last meeting [4], As Horizon team, we don't want to forbid using some 3rd-party JS lib if it's acceptable by license. >From the other side, Horizon Core team can't maintain all xstatic-* packages which could be needed by plugins. That's why we decided to add some folks form heat-dashboard team to xstatic-* core for that projects, which are used only be Heat Dashboard now. IMO, it looks like a reasonable and fair enough solution. I think we're OK if plugin teams will share the responsibility to maintain their xstatic-* dependencies both with Horizon team following current guidelines [2]. Maintaining xstatic-* project means: - create this repo according to the current guidelines - follow community rules according to stable policy - publish a new version of xstatic project if it's required by bugfixes, security fixes, new features - help other teams to integrate this project into their plugin. I hope if we agree on things above it helps both Horizon and plugin teams to deliver new versions faster with a limited teams capacity. [1] https://governance.openstack.org/tc/reference/projects/horizon.html#deliverables [2] https://docs.openstack.org/horizon/latest/contributor/contributing.html#javascript-and-css-libraries-using-xstatic [3] https://docs.openstack.org/horizon/latest/install/plugin-registry.html [4] http://eavesdrop.openstack.org/meetings/horizon/2018/horizon.2018-06-13-15.04.log.html#l-124 Regards, Ivan Kolodyazhny, http://blog.e0ne.info/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][python3] advice needed with updating lib-forward-testing jobs
Excerpts from Ghanshyam Mann's message of 2018-06-18 20:38:00 +0900:
> On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann
> wrote
> > Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
> > > Doug Hellmann writes:
> > >
> > > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
> > >
> > > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
> > > >> explicitly. But gate running on any repo does run job on current
> > > >> change set of that repo which is nothing but "master + current patch
> > > >> changes" . For example, any job running on oslo.config patch will
> > > >> take oslo.config source code from that patch which is "master +
> > > >> current change". You can see the results in this patch -
> > > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
> > > >> and gate jobs (including tempest-full-py3) fails as they run on
> > > >> current change set of neutron-lib code not on pypi version(which
> > > >> would pass the tests).
> > > >
> > > > The tempest-full-py3 job passed for that patch, though. Which seems to
> > > > indicate that the neutron-lib repository was not used in the test job,
> > > > even though it was checked out.
> > >
> > > The automatic generation of LIBS_FROM_GIT only includes projects which
> > > appear in required-projects. So in this case neutron-lib does not
> > > appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
> > > job.
>
> Yes, this is now clear to me. I was in impressions of treating the lib same
> way as service for testing repo always from source but that's not the case.
>
> > >
> > > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
> > > but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
> > > generation, so more complex jobs (which may want to inherit from that
> > > job but need multiple libraries) would also have to override
> > > LIBS_FROM_GIT and add the full set of projects.
> > >
> > > The code that automatically sets LIBS_FROM_GIT is fairly simple and
> > > could be modified to automatically add the project of the change under
> > > test. We could do that for all jobs, or we could add a flag which
> > > toggles the behavior. The question to answer here is: is there ever a
> > > case where a devstack job should not install the change under test from
> > > source? I think the answer is no, and even though under Zuul v2
> > > devstack-gate didn't automatically add the project under test to
> > > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
> > > templating which did.
> >
> > Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
> > like the behavior I would expect from the job.
>
> ++
>
> >
> > > A further thing to consider is what the desired behavior is for a series
> > > of changes. If a change to neutron-lib depends on a change to
> > > oslo.messaging, when the forward testing job runs on neutron-lib, should
> > > it also add oslo.messaging to LIBS_FROM_GIT? That's equally easy to
> > > implement (but would certainly need a flag as it essentially would add
> > > everything in the change series to LIBS_FROM_GIT which defeats the
> > > purpose of the restriction for the jobs which need it), but I honestly
> > > am not certain what's desired.
> >
> > I think going ahead and adding everything in the dependency chain
> > also makes sense. If I have 2 changes in libraries and a change in
> > a service and I want to test them all together I would expect to
> > be able to do that by using Depends-On and then for all 3 to be
> > installed from source in the job that runs.
>
> Yes, I agree on testing all series(either alone repo or depends-on on lib or
> service) with installed from source.
>
> >
> > >
> > > For each type of project (service, lib, lib-group (eg oslo.messaging)),
> > > what do we want to test from git vs pypi?
> >
> > We want to test changes to service projects with libraries from
> > PyPI so that we do not end up with services that rely on unreleased
> > features of libraries.
> >
> > We want to test changes to libraries with some services installed
> > from git so that we know changes to the library do not break (current)
> > master of the service. The set of interesting services may vary,
> > but a default set that represents the tightly coupled services that
> > run in the integrated gate now is reasonable.
> >
> > > How many jobs are needed to accomplish that?
> >
> > Ideally 1? Or 2? That's what I'm trying to work out.
>
> Currently "tempest-full/-py3" job does not run the 'slow' marked scenario
> tests and they run in separate job
> "tempest-scenario-multinode-lvm-multibackend"(which i am working to make it
> more clean)
>
> I think "tempest-full/-py3" will cover the most of the code/lib usage
> coverage.
It does seem like enough coverage to start out, and matches
Re: [openstack-dev] [TripleO] config-download/ansible next steps
On 06/13/2018 03:17 PM, James Slagle wrote:
On Wed, Jun 13, 2018 at 6:49 AM, Dmitry Tantsur wrote:
Slightly hijacking the thread to provide a status update on one of the items
:)
Thanks for jumping in.
The immediate plan right now is to wait for metalsmith 0.4.0 to hit the
repositories, then start experimenting. I need to find a way to
1. make creating nova instances no-op
2. collect the required information from the created stack (I need networks,
ports, hostnames, initial SSH keys, capabilities, images)
3. update the config-download code to optionally include the role [2]
I'm not entirely sure where to start, so any hints are welcome.
Here are a couple of possibilities.
We could reuse the OS::TripleO::{{role.name}}Server mappings that we
already have in place for pre-provisioned nodes (deployed-server).
This could be mapped to a template that exposes some Ansible tasks as
outputs that drives metalsmith to do the deployment. When
config-download runs, it would execute these ansible tasks to
provision the nodes with Ironic. This has the advantage of maintaining
compatibility with our existing Heat parameter interfaces. It removes
Nova from the deployment so that from the undercloud perspective you'd
roughly have:
Mistral -> Heat -> config-download -> Ironic (driven via ansible/metalsmith)
One thing that came to my mind while planning this work is that I'd prefer all
nodes to be processed in one step. This will help avoiding some issues that we
have now. For example, the following does not work reliably:
compute-0: just any profile:compute
compute-1: precise node=abcd
control-0: any node
This has two issues that will pop up randomly:
1. compute-0 can pick node abcd designated for compute-1
2. control-0 can pick a compute node, failing either compute-0 or compute-1
This problem is hard to fix if all deployment requests are processed separately,
but is quite trivial if the decision is done based on the whole deployment plan.
I'm going to work on a bulk scheduler like that in metalsmith.
A further (or completely different) iteration might look like:
Step 1: Mistral -> Ironic (driven via ansible/metalsmith)
Step 2: Heat -> config-download
Step 1 will still use provided environment to figure out the count of nodes for
each role, their images, capabilities and (optionally) precise node scheduling?
I'm a bit worried about the last bit: IIRC we rely on Heat's %index% variable
currently. We can, of course, ask people to replace it with something more
explicit on upgrade.
Step 2 would use the pre-provisioned node (deployed-server) feature
already existing in TripleO and treat the just provisioned by Ironic
nodes, as pre-provisioned from the Heat stack perspective. Step 1 and
Step 2 would also probably be driven by a higher level Mistral
workflow. This has the advantage of minimal impact to
tripleo-heat-templates, and also removes Heat from the baremetal
provisioning step. However, we'd likely need some python compatibility
libraries that could translate Heat parameter values such as
HostnameMap to ansible vars for some basic backwards compatibility.
Overall, I like this option better. It will allow an operator to isolate the
bare metal provisioning step from everything else.
[1] https://github.com/openstack/metalsmith
[2] https://metalsmith.readthedocs.io/en/latest/user/ansible.html
Obviously we have things to consider here such as backwards compatibility
and
upgrades, but overall, I think this would be a great simplification to our
overall deployment workflow.
Yeah, this is tricky. Can we make Heat "forget" about Nova instances? Maybe
by re-defining them to OS::Heat::None?
Not exactly, as Heat would delete the previous versions of the
resources. We'd need some special migrations, or could support the
existing method forever for upgrades, and only deprecate it for new
deployments.
Do I get it right that if we redefine OS::TripleO::{{role.name}}Server to be
OS::Heat::None, Heat will delete the old {{role.name}}Server instances on the
next update? This is sad..
I'd prefer not to keep Nova support forever, this is going to be hard to
maintain and cover by the CI. Should we extend Heat to support "forgetting"
resources? I think it may have a use case outside of TripleO.
I'd like to help with this work. I'll start by taking a look at what
you've got so far. Feel free to reach out if you'd like some
additional dev assistance or testing.
Thanks!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo][heat][jinja] resources.RedisVirtualIP: Property error: resources.VipPort.properties.network: Error validating value 'internal_api': Unable to find network with name or id
On Thu, Jun 14, 2018 at 1:48 PM, Mark Hamzy wrote:
> I am trying to delete the Storage, StorageMgmt, Tenant, and Management
> networks and trying to deploy using TripleO.
>
> The following patch
> https://hamzy.fedorapeople.org/0001-RedisVipPort-error-internal_api.patchapplied
> on top of /usr/share/openstack-tripleo-heat-templates from
> openstack-tripleo-heat-templates-8.0.2-14.el7ost.noarch
>
> yields the following error:
>
> (undercloud) [stack@oscloud5 ~]$ openstack overcloud deploy --templates -e
> ~/templates/node-info.yaml -e ~/templates/overcloud_images.yaml -e
> ~/templates/environments/network-environment.yaml -e
> ~/templates/environments/network-isolation.yaml -e
> ~/templates/environments/config-debug.yaml --ntp-server pool.ntp.org
> --control-scale 1 --compute-scale 1 --control-flavor control
> --compute-flavor compute 2>&1 | tee output.overcloud.deploy
> ...
> overcloud.RedisVirtualIP:
> resource_type: OS::TripleO::Network::Ports::RedisVipPort
> physical_resource_id:
> status: CREATE_FAILED
> status_reason: |
> resources.RedisVirtualIP: Property error:
> resources.VipPort.properties.network: Error validating value 'internal_api':
> Unable to find network with name or id 'internal_api'
> ...
>
> The following patch seems to fix it:
>
> 8<-8<-8<-8<-8<-
> diff --git a/environments/network-isolation.j2.yaml
> b/environments/network-isolation.j2.yaml
> index 3d4f59b..07cb748 100644
> --- a/environments/network-isolation.j2.yaml
> +++ b/environments/network-isolation.j2.yaml
> @@ -20,7 +20,13 @@ resource_registry:
>{%- for network in networks if network.vip and
> network.enabled|default(true) %}
>OS::TripleO::Network::Ports::{{network.name}}VipPort:
> ../network/ports/{{network.name_lower|default(network.name.lower())}}.yaml
>{%- endfor %}
> +{%- for role in roles -%}
> + {%- if internal_api in role.networks|default([]) and
> internal_api.enabled|default(true) %}
>OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip.yaml
> + {%- else %}
> + # Avoid weird jinja2 bugs that don't output a newline...
> + {%- endif %}
> +{%- endfor -%}
Does this actually work, or just suppress the error because your
network_data.yaml has deleted the internal_api network?
From the diff it looks like you're also deleting the internal_api and
external network which won't work with the default ServiceNetMap:
https://github.com/openstack/tripleo-heat-templates/blob/master/network/service_net_map.j2.yaml#L27
Can you please provide the network_data.yaml to confirm this?
If you really want to delete the internal_api network then you'll need
to pass a ServiceNetMap to specify a new bind network for every
service (and any other deleted networks where used as a value in the
ServiceNetMapDefaults).
Thanks,
Steve
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] DeployArtifacts considered...complicated?
On Sat, Jun 16, 2018 at 3:06 AM, Lars Kellogg-Stedman wrote: > I've been working on a series of patches to enable support for > keystone federation in tripleo. I've been making good use of the > DeployArtifacts support for testing puppet modules...until today. > > I have some patches that teach puppet-keystone about multi-valued > configuration options (like trusted_dashboard). They replace the > keystone_config provider (and corresponding type) with ones that work > with the 'openstackconfig' provider (instead of ini_settings). These > work great when I test them in isolation, but whenever I ran them as > part of an "overcloud deploy" I would get erroneous output. > > After digging through the various layers I found myself looking at > docker-puppet.py [1], which ultimately ends up calling puppet like > this: > > puppet apply ... > --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules ... > > It's that --modulepath argument that's the culprit. DeployArtifacts > (when using the upload-puppet-modules script) works by replacing the > symlinks in /etc/puppet/modules with the directories from your upload > directory. Even though the 'keystone' module in /etc/puppet/modules > takes precedence when doing something like 'include ::keystone', *all > the providers and types* in lib/puppet/* in > /usr/share/openstack-puppet/modules will be activated. Is this the same issue Carlos is trying to fix via https://review.openstack.org/#/c/494517/ ? I think there was some confusion on that patch around the underlying problem, but I think your explanation here helps, e.g the problem is you can conceivably end up with a mix of old/new modules? Thanks, Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [stable][horizon] Adding Ivan Kolodyazhny to horizon-stable-maint
+1 to add Ivan to the horizon stable maint team. 2018年6月18日(月) 11:04 Tony Breeds : > Hello folks, > Recently Ivan became the Horizon PTL and as with past PTLs (Hi Rob) > isn't a member of the horizon-stable-maint team. Ivan is a member of > the Cinder stable team and as such has demonstrated an understanding of > the stable policy. Since the Dublin PTG Ivan has been doing consistent > high quality reviews on Horizon's stable branches. > > As such I'm suggesting adding him to the existing stable team. > > Without strong objections I'll do that on (my) Monday 25th June. > > Yours Tony. > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][python3] advice needed with updating lib-forward-testing jobs
On Sat, 16 Jun 2018 04:01:45 +0900 Doug Hellmann
wrote
> Excerpts from corvus's message of 2018-06-15 08:46:40 -0700:
> > Doug Hellmann writes:
> >
> > > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900:
> >
> > >> Yes, It will not be set on LIBS_FROM_GIT as we did not set it
> > >> explicitly. But gate running on any repo does run job on current
> > >> change set of that repo which is nothing but "master + current patch
> > >> changes" . For example, any job running on oslo.config patch will
> > >> take oslo.config source code from that patch which is "master +
> > >> current change". You can see the results in this patch -
> > >> https://review.openstack.org/#/c/575324/ . Where I deleted a module
> > >> and gate jobs (including tempest-full-py3) fails as they run on
> > >> current change set of neutron-lib code not on pypi version(which
> > >> would pass the tests).
> > >
> > > The tempest-full-py3 job passed for that patch, though. Which seems to
> > > indicate that the neutron-lib repository was not used in the test job,
> > > even though it was checked out.
> >
> > The automatic generation of LIBS_FROM_GIT only includes projects which
> > appear in required-projects. So in this case neutron-lib does not
> > appear in LIBS_FROM_GIT[1], so the change is not actually tested by that
> > job.
Yes, this is now clear to me. I was in impressions of treating the lib same way
as service for testing repo always from source but that's not the case.
> >
> > Doug's approach of adding {{zuul.project}} to LIBS_FROM_GIT would work,
> > but anytime LIBS_FROM_GIT is set explicitly, it turns off the automatic
> > generation, so more complex jobs (which may want to inherit from that
> > job but need multiple libraries) would also have to override
> > LIBS_FROM_GIT and add the full set of projects.
> >
> > The code that automatically sets LIBS_FROM_GIT is fairly simple and
> > could be modified to automatically add the project of the change under
> > test. We could do that for all jobs, or we could add a flag which
> > toggles the behavior. The question to answer here is: is there ever a
> > case where a devstack job should not install the change under test from
> > source? I think the answer is no, and even though under Zuul v2
> > devstack-gate didn't automatically add the project under test to
> > LIBS_FROM_GIT, we probably had that behavior anyway due to some JJB
> > templating which did.
>
> Adding the project-under-test to LIBS_FROM_GIT unconditionally feels
> like the behavior I would expect from the job.
++
>
> > A further thing to consider is what the desired behavior is for a series
> > of changes. If a change to neutron-lib depends on a change to
> > oslo.messaging, when the forward testing job runs on neutron-lib, should
> > it also add oslo.messaging to LIBS_FROM_GIT? That's equally easy to
> > implement (but would certainly need a flag as it essentially would add
> > everything in the change series to LIBS_FROM_GIT which defeats the
> > purpose of the restriction for the jobs which need it), but I honestly
> > am not certain what's desired.
>
> I think going ahead and adding everything in the dependency chain
> also makes sense. If I have 2 changes in libraries and a change in
> a service and I want to test them all together I would expect to
> be able to do that by using Depends-On and then for all 3 to be
> installed from source in the job that runs.
Yes, I agree on testing all series(either alone repo or depends-on on lib or
service) with installed from source.
>
> >
> > For each type of project (service, lib, lib-group (eg oslo.messaging)),
> > what do we want to test from git vs pypi?
>
> We want to test changes to service projects with libraries from
> PyPI so that we do not end up with services that rely on unreleased
> features of libraries.
>
> We want to test changes to libraries with some services installed
> from git so that we know changes to the library do not break (current)
> master of the service. The set of interesting services may vary,
> but a default set that represents the tightly coupled services that
> run in the integrated gate now is reasonable.
>
> > How many jobs are needed to accomplish that?
>
> Ideally 1? Or 2? That's what I'm trying to work out.
Currently "tempest-full/-py3" job does not run the 'slow' marked scenario tests
and they run in separate job
"tempest-scenario-multinode-lvm-multibackend"(which i am working to make it
more clean)
I think "tempest-full/-py3" will cover the most of the code/lib usage coverage.
>
> > What should happen with a change series with other
> > projects in it?
>
> I expect all of the patches in a series to be installed from source
> somewhere in the chain. That works today if we have a library patch
> that depends on a service patch because that patched version of the
> service is used in the
[openstack-dev] [qa][integrated] Multinode base job
Dear all, the Tempest / devstack multinode integration base job in it's Zuulv3 native incarnation is finally working, and the patch [0] to making it voting on Tempest is approved. The name of the new job is "tempest-multinode-full". This effectively replaces the legacy "neutron-tempest-multinode-full". Since "neutron-tempest-multinode-full" is used as non-voting by all projects that use the "integrated-gate" job template, I'd propose to: - add "tempest-multinode-full" as non-voting to the integrated gate for master, queens and pike - fix any issue that may show up on queens/pike - remove neutron-tempest-multinode-full legacy job I would also like to make the multinode job voting, at least on devstack, possibly on all integrated gate repos. Please let me know if anyone as any concern with this plan. Andrea __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][python3] advice needed with updating lib-forward-testing jobs
On Fri, 15 Jun 2018 22:41:41 +0900 Doug Hellmann wrote > Excerpts from Ghanshyam's message of 2018-06-15 09:04:35 +0900: > > > > > > > > On Fri, 15 Jun 2018 06:17:34 +0900 Doug Hellmann > > wrote > > > Excerpts from Doug Hellmann's message of 2018-06-14 13:02:31 -0400: > > > > Excerpts from Ghanshyam's message of 2018-06-14 16:54:33 +0900: > > > > > > > > > > > Could it be as simple as adding tempest-full-py3 with the > > > > > > > > required-projects list updated to include the current > > repository? So > > > > > > > > there isn't a special separate job, and we would just reuse > > > > > > > > tempest-full-py3 for this? > > > > > > > > > > This can work if lib-forward-testing is going to run against > > current lib repo only not cross lib or cross project. For example, if > > neutron want to tests neutron change against neutron-lib src then this > > will not work. But from history [1] this does not seems to be scope of > > lib-forward-testing. > > > > > > > > > > Even we do not need to add current repo to required-projects list > > or in LIBS_FROM_GIT . That will always from master + current patch > > changes. So this makes no change in tempest-full-py3 job and we can > > directly use tempest-full-py3 job in lib-forward-testing. Testing in [2]. > > > > > > > > Does it? So if I add tempest-full-py3 to a *library* that library is > > > > installed from source in the job? I know the source for the library > > > > will be checked out, but I'm surprised that devstack would be > > configured > > > > to use it. How does that work? > > > > > > Based on my testing, that doesn't seem to be the case. I added it to > > > oslo.config and looking at the logs [1] I do not set LIBS_FROM_GIT set > > > to include oslo.config and the check function is returning false so that > > > it is not installed from source [2]. > > > > Yes, It will not be set on LIBS_FROM_GIT as we did not set it explicitly. > > But gate running on any repo does run job on current change set of that > > repo which is nothing but "master + current patch changes" . For example, > > any job running on oslo.config patch will take oslo.config source code > > from that patch which is "master + current change". You can see the > > results in this patch - https://review.openstack.org/#/c/575324/ . Where I > > deleted a module and gate jobs (including tempest-full-py3) fails as they > > run on current change set of neutron-lib code not on pypi version(which > > would pass the tests). > > The tempest-full-py3 job passed for that patch, though. Which seems to > indicate that the neutron-lib repository was not used in the test job, > even though it was checked out. oops, I saw the other job failure and overlooked tempest-full-py3 (friday night effect :)). Your point is correct on LIBS_FROM_GIT . -gmann > > > > > In that case, lib's proposed change will be tested against integration > > tests job to check any regression. If we need to run cross lib/project > > testing of any lib then, yes we need the 'tempest-full-py3-src' job but > > that is separate things as you mentioned. > > > > -gmann > > > > > > > > So, I think we need the tempest-full-py3-src job. I will propose an > > > update to the tempest repo to add that. > > > > > > Doug > > > > > > [1] > > http://logs.openstack.org/64/575164/2/check/tempest-full-py3/9aa50ad/job-output.txt.gz > > > [2] > > http://logs.openstack.org/64/575164/2/check/tempest-full-py3/9aa50ad/job-output.txt.gz#_2018-06-14_19_40_56_223136 > > > > > > > > __ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements][nova] weird error on 'Validating lower constraints of test-requirements.txt'
Thanks all for helping , Saw this patch [1] merged and assume that's the fix for the issue , we will rebase based on it then try again, [1] https://review.openstack.org/#/c/575872/4 Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com Phone: +86-10-82451493 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC From: Doug Hellmann To: openstack-dev Date: 06/16/2018 08:14 AM Subject:Re: [openstack-dev] [requirements][nova] weird error on 'Validating lower constraints of test-requirements.txt' Excerpts from Eric Fried's message of 2018-06-15 18:09:49 -0500: > Doug- > > > The lower constraints tests only look at files in the same repo. > > The minimum versions of dependencies set in requirements.txt, > > test-requirements.txt, etc. need to match the values in > > lower-constraints.txt. > > > > In this case, the more detailed error message is a few lines above the > > error quoted by Chen CH Ji. The detail say "Requirement for package > > retrying has no lower bound" which means that there is a line in > > requirements.txt indicating a dependency on "retrying" but without > > specifying a minimum version. That is the problem. > > The patch didn't change the retrying constraint in requirements.txt [1]; > why isn't this same failure affecting every other patch in nova? > > [1] https://review.openstack.org/#/c/523387/51/requirements.txt@65 > > -efried > Earlier this cycle I updated the requirements check job to verify that all of the settings are correct any time any changes to the dependency lists are made. We used to only look at the line being changed, but that allowed incorrect settings to stay in place for a long time so we weren't actually testing with good settings. We still only run that job when the dependency list is modified in some way. Earlier this week, Matt Thode updated the job to be more strict and to require that all dependencies have a minimum version specified [2]. We did this because some project teams thought that after we dropped the minimums from the global-requirements.txt list they were supposed to (or allowed to) drop them from their project dependency lists, too. My guess is that this dependency in nova never had a lower bound and that this is the first patch to touch the dependency list, so now it's being blocked on the fact that the list has a validation error. I recommend using a separate patch to fix the minimum version of retrying and then rebasing 523387 on top of the new patch. Doug [2] https://review.openstack.org/#/c/574367/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
