Re: [openstack-dev] [oslo][oslo.log] JSON logs are missing the request ID
A bug would be fine. I'm not sure how many people are keeping an eye on oslo.log bugs at this point, so be realistic in when it might get looked at. On 01/18/2018 03:06 AM, Saverio Proto wrote: > Hello Sean, > after the brief chat we had on IRC, do you think I should open a bug > about this issue ? > > thank you > > Saverio > > > On 13.01.18 07:06, Saverio Proto wrote: >>> I don't think this is a configuration problem. >>> >>> Which version of the oslo.log library do you have installed? >> >> Hello Doug, >> >> I use the Ubuntu packages, at the moment I have this version: >> >> python-oslo.log 3.16.0-0ubuntu1~cloud0 >> >> thank you >> >> Saverio >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] cyclomatic complexity check in flake8 & Python 3
On 12/12/2017 08:08 AM, Thierry Carrez wrote: > Zane Bitter wrote: >> Here's how the allegedly most complex function in Heat[5] shakes out, >> for example: >> >> mccabe py27 py36 >> 0.2.1 14 6 >> 0.3.1 23 23 >> 0.6.1 23 23 >> >> I don't have a solution to suggest, but I burned most of my day digging >> into this so I just wanted to let y'all know how screwed we are. > Thanks for digging into this, Zane. I'd say we have two options, based > on how useful running those tests is: > > 1/ If they are useful, we should bump the version, at the same time > adjusting max-complexity per repo to match the most complex function > (with some slack factor). Encourage more projects to use cyclomatic > complexity checks and fix the biggest offenders using cycle goals. > > 2/ If they are not useful, just drop cyclomatic complexity tests overall > rather than relying on wrong results and wasting CPU cycles > > So I'd like to hear from the 60+ repos on whether using those tests lead > to any good results :) I don't know how useful these end up being (#1), though I've seen patches from time to time with people trying to reduce them. The current max in Nova is 35. Moving to mccabe 0.6.1 generates 2 errors: Running flake8 on all files ./nova/compute/manager.py:763:5: C901 'ComputeManager._init_instance' is too complex (37) ./nova/tests/functional/api_samples_test_base.py:144:5: C901 'ApiSampleTestBase._compare_result' is too complex (36) I honestly think this is one of the things that you declare a flag day a couple weeks out, warn everyone they are going to have to adjust their max, and move forward. It's a very easy fix when pep8 starts failing people. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api][nova] Why we return a redirect (302) when request to 'GET /v2.1'?
On 11/01/2017 11:04 PM, Alex X wrote: There is bug complain about the redirect which returned by the 'GET /v2.1' https://launchpad.net/bugs/1728732 'GET /v2.1' will return a redirect to the 'GET /v2.1/'. The response of 'GET /v2.1/' is the API version info. This seems nova API behaviour for a long time. In the keystone catalog, the endpoint should be the version API I think. For nova, it is 'GET /v2.1' which return a redirect instead of version info. Is there anybody knowing that why we return a redirect? I thought it was an artifact of the way that paste builds pipelines, and the way our resources need urls. I was trying to see if we generate it on our side, but I'm not seeing it, so I suspect this is just a consequence of the resource mapper and paste. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Unified Limits work stalled - next steps, need for new leadership
That sounds great, thank you! Let's synchronize on IRC to do a quick bootstrapping about where I think the next steps are, then you all can run with it. -Sean On 10/17/2017 11:31 PM, 王玺源 wrote: > HI Sean, > I think that we(Lance and I) from Huawei are willing to take over and > continue this feature. I hope that we can get some information and > suggestion from you. Such as: what’s the progress now? What should I do > now? IMHO, the main work now is to update and polish these two specs[1], > right? I'll continue to update them if no objection. > > Thanks. > > > [1]: https://review.openstack.org/#/c/441203 > https://review.openstack.org/#/c/455709 > > 2017-09-08 3:18 GMT+08:00 Sean Dague <s...@dague.net > <mailto:s...@dague.net>>: > > This is an email that's probably overdue, but time gets away from > all of us. Coming out of Atlanta and even Boston we were pushing > pretty hard on a new approach to unify limits, as well as building > the infrastructure to have hierarchical limits possible in OpenStack > (keystone merged spec - > > https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html > > <https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html>) > > However, this effort pretty much as run out of gas, and really needs > a new driver (or set of drivers) to make forward progress. My > current set of responsibilities doesn't really make it possible for > me to own driving this (though I will be happy to help others). That > seems to also be true of many of the others that were engaged. > > This is going out in advance of the PTG in case any developer(s), or > organizations feel that hierarchical limits are a priority for them, > and would be willing to step up there. If so please feel free to > reach out, and we can carve out some PTG time in helping a new push > organize here. > > However, without new leadership here, the current plan is to just > put the effort to rest. > > -Sean > > -- > Sean Dague > http://dague.net > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][infra][devstack] Changes to devstack-core for zuul v3 migration
On 10/10/2017 04:22 PM, Dean Troyer wrote: > On Tue, Oct 10, 2017 at 1:15 PM, Andrea Frittoli > <andrea.fritt...@gmail.com> wrote: >> - we will treat +1 from infra-core members on devstack changes in the >> ansible bits as +2 >> - add clarkb to devstack-core, since he's quite aware of the the devstack >> codebase > > +2 on both of these proposals! Agreed +2. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ptl][tc] Accessible upgrade support
On 10/06/2017 07:23 PM, Kevin Benton wrote: >The neutron story is mixed on accessable upgrade, because at least in some cases, like ovs, upgrade might trigger a network tear down / rebuild that generates an outage (though typically a pretty small one). This shouldn't happen. If it does it should be reported as a bug. All existing OVS flows are left in place during agent initialization and we don't get rid of the old ones until the agent finishes setting up the new ones. I thought it was literally a problem with the daemon restart where it is stateful and does a teardown / rebuild when started. I'd be happy to be wrong about that, but was told in the past that it was just an OVS limitation. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Should we make rebuild + new image on a volume-backed instance fail fast?
On 10/06/2017 01:22 PM, Matt Riedemann wrote: > So with all of this in mind, should we at least consider, until at least > someone owns supporting this, that the API should fail with a 400 > response if you're trying to rebuild with a new image on a volume-backed > instance? That way it's a fast failure in the API, similar to trying to > backup a volume-backed instance fails fast. > > If we did, that would change the API response from a 202 today to a 400, > which is something we normally don't do. I don't think a microversion > would be necessary if we did this, however, because essentially what the > user is asking for isn't what we're actually giving them, so it's a > failure in an unexpected way even if there is no fault recorded, it's > not what the user asked for. I might not be thinking of something here > though, like interoperability for example - a cloud without this change > would blissfully return 202 but a cloud with the change would return a > 400...so that should be considered. Yes, and this is the kind of thing I think we'd backport to all stable branches as well. Right now this is effectively a silent 500 (giving a 202 that we know will never actually do what is asked). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Infra][Nova] s390x zKVM CI
On 10/06/2017 04:07 AM, Andreas Scheuring wrote: > Hi, > yesterday the nova s390x (zkvm) CI got the permission to vote +1/-1 on > nova again [3]. Technically it was just an addition the the gerri group > [1]. Now Jenkins is showing up behind the “Verified” field in the review > with its +1/-1. However it’s not yet showing up in the long table > underneath that and therefore (probably) also not in the third party ci > watch [2] page. What else needs to be done that our CI appears there as > well? > > Thanks! > > [1] https://review.openstack.org/#/admin/groups/511,members > [2] http://ci-watch.tintri.com/project?project=nova > [3] > http://lists.openstack.org/pipermail/openstack-dev/2017-October/123195.html > > > Andreas Scheuring (andreas_s) Look at how the markup for the jenkins and power vm results posts are: http://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/nova/11/457711/35/check/tempest-dsvm-full-xenial/ec523a2/; rel="nofollow">tempest-dsvm-full-xenial SUCCESS in 53m 59s Those css classes are used as selectors by the Toggle CI code to extract test results. The zKVM results posting is missing them. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ptl][tc] Accessible upgrade support
On 10/05/2017 07:08 AM, Graham Hayes wrote: > > > On Thu, 5 Oct 2017, at 09:50, Thierry Carrez wrote: >> Matt Riedemann wrote: >>> What's the difference between this tag and the zero-impact-upgrades tag? >>> I guess the accessible one is, can a user still ssh into their VM while >>> the nova compute service is being upgraded. The zero-impact-upgrade one >>> is more to do with performance degradation during an upgrade. I'm not >>> entirely sure what that might look like, probably need operator input. >>> For example, while upgrading, you're live migrating VMs all over the >>> place which is putting extra strain on the network. >> >> The zero-impact-upgrade tag means no API downtime and no measurable >> impact on performance, while the accessible-upgrade means that while >> there can be API downtime, the resources provisioned are still >> accessible (you can use the VM even if nova-api is down). >> >> I still think we have too many of those upgrade tags, and amount of >> information they provide does not compensate the confusion they create. >> If you're not clear on what they mean, imagine a new user looking at the >> Software Navigator... >> >> In particular, we created two paths in the graph: >> * upgrade < accessible-upgrade >> * upgrade < rolling-upgrade < zero-downtime < zero-impact >> >> I personally would get rid of zero-impact (not sure there is that much >> additional information it conveys beyond zero-downtime). >> >> If we could make the requirements of accessible-upgrade a part of >> rolling-upgrade, that would also help (single path in the graph, only 3 >> "levels"). Is there any of the current rolling-upgrade things (cinder, >> neutron, nova, swift) that would not qualify for accessible-upgrade as >> well ? > > Well, there is projects (like designate) that qualify for accessible > upgrade, but not rolling upgrade. The neutron story is mixed on accessable upgrade, because at least in some cases, like ovs, upgrade might trigger a network tear down / rebuild that generates an outage (though typically a pretty small one). I still think it's hard to describe to folks what is going on without pictures. And the tag structure might just be the wrong way to describe the world, because they are a set of positive assertions, and upgrade expectations are really about: "how terrible will this be". If I was an operator the questions I might have is: 1) Really basic, will my db roll forward? 2) When my db rolls forward, is it going to take a giant table lock that is effectively an outage? 3) Is whatever date I created, computes, networks going to stay up when I do all this? (i.e. no customer workload interuption) 4) If the service is more than 1 process, can they arbitrarily work with N-1 so I won't have a closet outage when services restart. 5) If the service runs on more than 1 host, can I mix host levels, or will there be an outage as I upgrade nodes 6) If the service talks to other openstack services, is there a strict version lock in which means I've got to coordinate with those for upgrade? If so, what order is that and is it clear? 7) Can I seamlessly hide my API upgrade behind HA-Proxy / Istio / (or similar) so that there is no API service interruption 8) Is there any substantial degradation in running "mixed mode" even if it's supported, so that I know whether I can do this over a longer window of time when time permits 9) What level of validation exists to ensure that any of these "should work" do work? The tags were really built around grouping a few of these, but even with folks that are near the problem, they got confusing quick. I really think that some more pictoral upgrade safety cards or something explaining the things you need to consider, and what parts projects handle for you would be really useful. And then revisit whatever the tagging structure is going to be later. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Important information for people with in-repo Zuul v3 config
On 10/04/2017 07:23 AM, Boden Russell wrote: > > On 10/3/17 1:37 PM, Monty Taylor wrote: >> The partial rollback of Zuulv3 is in place now. Zuulv2 is acting as your >> gate keeper once again. > > Since the rollback, one of the neutron-lib (v2) jobs has been > consistently failing [1] with: > > c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory > > Is this a known issue? Yes, this is unrelated. cffi 1.11.1 was released today on pypi, without wheels. It's installed super early in the job run because ansible is used to drive things. There is a stop gap work around that's running tests right now - https://review.openstack.org/#/c/509394/1/devstack-vm-gate-wrap.sh If it passes I'll approve it to get folks back to work. A longer term solution is probably needed. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Security of Meta-Data
There is an assumption that you've got solid network security on the path between your guests and your nova-API. Either because you've got a secure network path, or you run the neutron proxy server on the host itself, and so this is a no hop call. Because this is a bootstrapping problem, and the guests are coming up blank and *asking* the service how they should be configured, it's kind of hard to have generically better security than that. A lot of how that path is configured is very specific to deployment's networking setup and topology, so the options are on the table without a specific recommendation. If you still have concerns about that, it's always possible to bake your own config management daemon into your images, and do more sensitive data pulled via a certificate secured model. You do then have to manage certificate rotation in guest images, but that moves the bootstrapping problem elsewhere. -Sean On 10/03/2017 06:00 PM, Giuseppe de Candia wrote: > Hi Folks, > > > Are there any documented conventions regarding the security model for > MetaData? > > > Note that CloudInit allows passing user and ssh service public/private > keys via MetaData service (or ConfigDrive). One assumes it must be > secure, but I have not found a security model or documentation. > > > My understanding of the Neutron reference implementation is that > MetaData requests are HTTP (not HTTPS) and go from the VM to the > MetaData proxy on the Network Node (after which they are proxied to Nova > meta-data API server). The path from VM to Network Node using HTTP > cannot guarantee confidentiality and is also susceptible to > Man-in-the-Middle attacks. > > > > Some Neutron drivers proxy Metadata requests locally from the node > hosting the VM that makes the query. I have mostly seen this > presented/motivated as a way of removing dependency on the Network node, > but it should also increase security. Yet, I have not seen explicit > discussions of the security model, nor any attempt to set a standard for > security of the meta-data. > > Finally, there do not seem to be granular controls over what meta-data > is presented over ConfigDrive (when enabled) vs. meta-data REST API. As > an example, Nova vendor data is presented over both, if both are > enabled; config drive is presumably more secure. > > thanks, > Pino > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] key_pair update on rebuild (a whole lot of conversations)
There is currently a spec up for being able to specify a new key_pair name during the rebuild operation in Nova - https://review.openstack.org/#/c/375221/ For those not completely familiar with Nova operations, rebuild triggers the "reset this vm to initial state" by throwing out all the disks, and rebuilding them from the initial glance images. It does however keep the IP address and device models when you do that. So it's useful for ephemeral but repeating workloads, where you'd rather not have the network information change out from under you. The spec is a little vague about when this becomes really useful, because this will not save you from "I lost my private key, and I have important data on that disk". Because the disk is destroyed. That's the point of rebuild. We once added this preserve_ephemeral flag to rebuild for trippleo on ironic, but it's so nasty we've scoped it to only work with ironic backends. Ephemeral should mean ephemeral. Rebuild bypasses the scheduler. A rebuilt server stays on the same host as it was before, which means the operation has a good chance of being faster than a DELETE + CREATE, as the image cache on that host should already have the base image for you instance. A bunch of data was collected today in a lot of different IRC channels (#openstack-nova, #openstack-infra, #openstack-operators). = OpenStack Operators = mnaser said that for their customers this would be useful. Keys get lost often, but keeping the IP is actually valuable. They would also like this. penick said that for their existing environment, they have a workflow where this would be useful. But they are moving away from using nova for key distribution because in Nova keys are user owned, which actually works poorly given that everything else is project owned. So they are building something to do key distribution after boot in the guest not using nova's metadata. Lots of people said they didn't use nova's keypair interfaces, they just did it all in config management after the fact. = Also on reboot? = Because the reason people said they wanted it was: "I lost my private key", the question at PTG was "does that mean you want it on reboot?" But as we dive through the constraints of that, people that build "pet" VMs typically delete or disable cloud-init (or similar systems) after first boot. Without that kind of agent, this isn't going to work anyway. So also on reboot seems very fragile and unuseful. = Infra = We asked the infra team if this is useful to them, the answer was no. What would be useful them is if keypairs could be updated. They use a symbolic name for a keypair but want to do regular key rotation. Right now they do this by deleting then recreating keypairs, but that does mean there is a window where there is no keypair with that name, so server creates fail. It is agreed that something supporting key rotation in the future would be handy, that's not in this scope. = Barbican = In the tradition of making a simple fix a generic one, it does look like there is a longer term part of this where Nova should really be able to specify a Barbican resource url for a key so that things like rotation could be dealt with in a system that specializes in that. It also would address the very weird oddity of user vs. project scoping. That's a bigger more nebulous thing. Other folks would need to be engaged on that one. = Where I think we are? = I think with all this data we're at the following: Q: Should we add this to rebuild A: Yes, probably - after some enhancement to the spec * * - we really should have much better use cases about the situations it is expected to be used in. We spend a lot of time 2 and 3 years out trying to figure out how anyone would ever use a feature, and adding another one without this doesn't seem good Q: should this also be on reboot? A: NO - it would be too fragile I also think figuring out a way to get Nova out of the key storage business (which it really shouldn't be in) would be good. So if anyone wants to tackle Nova using Barbican for keys, that would be ++. Rebuild doesn't wait on that, but Barbican urls for keys seems like a much better world to be in. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Zuul v3 Status - and Rollback Information
On 10/03/2017 01:06 PM, Jeremy Stanley wrote: > On 2017-10-03 12:00:44 -0500 (-0500), Matt Riedemann wrote: >> On 10/3/2017 11:40 AM, Monty Taylor wrote: >>> Our nodepool quota will be allocated 80% to Zuul v2, and 20% to ZUul v3. >>> This will slow v2 down slightly, but allow us to continue to exercise v3 >>> enough to find problems. >>> >>> Zuul v2 and v3 can not both gate a project or set of projects. In >>> general, Zuul v2 will be gating all projects, except the few projects that >>> are specifically v3-only: zuul-jobs, openstack-zuul-jobs, project-config, >>> and zuul itself. >> >> So if v3 is in check and periodic, and will be restarted and offline at >> times, doesn't that mean we could have patches waiting for an extended >> period of time on v3 results when the v2 jobs are long done? Or do the v3 >> jobs just timeout and being non-voting shouldn't impact the overall score? > > Not at all, since v2 will also be running check jobs itself and it > only pays attention to its own results for that puropose. So > basically while we're rolled back to v2 you'll see both "Jenkins" > (v2 account) and "Zuul" (v3 account) reporting check results most of > the time but you can consider the v3 results merely an advisory > indicator of whether your v3 jobs are working correctly. Right, so zuul v3 will effectively be treated like 3rd party CI during the transition. Sounds good. The 80 / 20 split seems very reasonable, and a good way to let teams get back to work while letting the v3 effort make forward progress with real load to smoke out the issues. Thanks for flipping over to this model. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Update on Zuul v3 Migration - and what to do about issues
Any update on where we stand on issues now? Because every single patch I tried to land yesterday was killed by POST_FAILURE in various ways. Including some really small stuff - https://review.openstack.org/#/c/324720/ That also includes the patch I'm told fixes some issues with zuul v3 in the base devstack jobs - https://review.openstack.org/#/c/508344/3 It also appears that many of the skips stopped being a thing - https://review.openstack.org/#/c/507527/ got a Tempest test run attempted on it (though everything ended in Node failure). Do we have a defined point on the calendar for getting the false negatives back below the noise threshold otherwise a rollback is implemented so that some of these issues can be addressed in parallel without holding up community development? -Sean On 09/29/2017 10:58 AM, Monty Taylor wrote: > Hey everybody! > > tl;dr - If you're having issues with your jobs, check the FAQ, this > email and followups on this thread for mentions of them. If it's an > issue with your job and you can spot it (bad config) just submit a patch > with topic 'zuulv3'. If it's bigger/weirder/you don't know - we'd like > to ask that you send a follow up email to this thread so that we can > ensure we've got them all and so that others can see it too. > > ** Zuul v3 Migration Status ** > > If you haven't noticed the Zuul v3 migration - awesome, that means it's > working perfectly for you. > > If you have - sorry for the disruption. It turns out we have a REALLY > complicated array of job content you've all created. Hopefully the pain > of the moment will be offset by the ability for you to all take direct > ownership of your awesome content... so bear with us, your patience is > appreciated. > > If you find yourself with some extra time on your hands while you wait > on something, you may find it helpful to read: > > https://docs.openstack.org/infra/manual/zuulv3.html > > We're adding content to it as issues arise. Unfortunately, one of the > issues is that the infra manual publication job stopped working. > > While the infra manual publication is being fixed, we're collecting FAQ > content for it in an etherpad: > > https://etherpad.openstack.org/p/zuulv3-migration-faq > > If you have a job issue, check it first to see if we've got an entry for > it. Once manual publication is fixed, we'll update the etherpad to point > to the FAQ section of the manual. > > ** Global Issues ** > > There are a number of outstanding issues that are being worked. As of > right now, there are a few major/systemic ones that we're looking in to > that are worth noting: > > * Zuul Stalls > > If you say to yourself "zuul doesn't seem to be doing anything, did I do > something wrong?", we're having an issue that jeblair and Shrews are > currently tracking down with intermittent connection issues in the > backend plumbing. > > When it happens it's an across the board issue, so fixing it is our > number one priority. > > * Incorrect node type > > We've got reports of things running on trusty that should be running on > xenial. The job definitions look correct, so this is also under > investigation. > > * Multinode jobs having POST FAILURE > > There is a bug in the log collection trying to collect from all nodes > while the old jobs were designed to only collect from the 'primary'. > Patches are up to fix this and should be fixed soon. > > * Branch Exclusions being ignored > > This has been reported and its cause is currently unknown. > > Thank you all again for your patience! This is a giant rollout with a > bunch of changes in it, so we really do appreciate everyone's > understanding as we work through it all. > > Monty > > ______ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Nova docker replaced by zun?
On 09/29/2017 10:48 AM, ADAMS, STEVEN E wrote: > Can anyone point me to some background on why nova docker was > discontinued and how zun is the heir? > > Thx, > > Steve Adams > > AT > > https://github.com/openstack/nova-docker/blob/master/README.rst The nova-docker driver discontinued because it was not maintained. In the entire OpenStack community we could not find a second person to help with the maintenance of it (it was only Dims doing any needed fixes). This was even though the driver was known to be running in multiple production clouds. The project was shut down for that reason so that no one would mistakenly assume there was any maintenance or support on it. If you or others want to revive the project, that would be fine, as long as we can identify 2 individuals who will step up as maintainers. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tc] [election] NON nomination for TC
I'd like to announce that after 4 years serving on the OpenStack Technical Committee, I will not be running in this fall's election. Over the last 4 years we've navigated some rough seas together, including the transition to more inclusion of projects, the dive off the hype curve, the emergence of real interoperability between clouds, and the beginnings of a new vision of OpenStack pairing with more technologies beyond our community. There remains a ton of good work to be done. But it's also important that we have a wide range of leaders to do that. Those opportunities only exist if we make space for new leaders to emerge. Rotational leadership is part of what makes OpenStack great, and is part of what will ensure that this community lasts far beyond any individuals within it. I plan to still be around in the community, and contribute where needed. So this is not farewell. However it will be good to see new faces among the folks leading the next steps in the community. I would encourage all members of the community that are interested in contributing to the future of OpenStack to step forward and run. It's important to realize what the TC is and can be. This remains a community driven by consensus, and the TC reflects that. Being a member of the TC does raise your community visibility, but it does not replace the need to listen, understand, communicate clearly, and realize that hard work comes through compromise. Good luck to all our candidates this fall, and thanks for letting me represent you the past 4 years. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack-operators] [tc][nova][ironic][mogan] Evaluate Mogan project
On 09/27/2017 09:31 AM, Julia Kreger wrote: > [...] >>> The short explanation which clicked for me (granted it's probably an >>> oversimplification, but still) was this: Ironic provides an admin >>> API for managing bare metal resources, while Mogan gives you a user >>> API (suitable for public cloud use cases) to your Ironic backend. I >>> suppose it could have been implemented in Ironic, but implementing >>> it separately allows Ironic to be agnostic to multiple user >>> frontends and also frees the Ironic team up from having to take on >>> yet more work directly. >> >> >> ditto! >> >> I had a similar question at the PTG and this was the answer that convinced >> be >> may be worth the effort. >> >> Flavio >> > > For Ironic, the question did come at the PTG up of tenant aware > scheduling of owned hardware, as in Customer A and B are managed by > the same ironic, only customer A's users should be able to schedule on > to Customer A's hardware, with API access control restrictions such > that specific customer can take action on their own hardware. > > If we go down the path of supporting such views/logic, it could become > a massive undertaking for Ironic, so there is absolutely a plus to > something doing much of that for Ironic. Personally, I think Mogan is > a good direction to continue to explore. That being said, we should > improve our communication of plans/directions/perceptions between the > teams so we don't adversely impact each other and see where we can > help each other moving forward. My biggest concern with Mogan is that it forks Nova, then starts changing interfaces. Nova's got 2 really big API surfaces. 1) The user facing API, which is reasonably well documented, and under tight control. Mogan has taken key things at 95% similarity and changed bits. So servers includes things like a partitions parameter. https://github.com/openstack/mogan/blob/master/api-ref/source/v1/servers.inc#request-4 This being nearly the same but slightly different ends up being really weird. Especially as Nova evolves it's code with microversions for things like embedded flavor info. 2) The guest facing API of metadata/config drive. This is far less documented or tested, and while we try to be strict about adding in information here in a versioned way, it's never seen the same attention as the user API on either documentation or version rigor. That's presumably getting changed, going to drift as well, which means discovering multiple implementations that are nearly, but not exactly the same that drift. The point of licensing things under and Apache 2 license was to enable folks to do all kind of experiments like this. And experiments are good. But part of the point of experiments is to learn lessons to bring back into the fold. Digging out of the multi year hole of "close but not exactly the same" API differences between nova-net and neutron really makes me want to make sure we never intentionally inflict that confusion on folks again. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] reset key pair during rebuilding
On 09/27/2017 05:15 AM, Marcus Furlong wrote: > On 27 September 2017 at 09:23, Michael Still <mi...@stillhq.com> wrote: >> >> Operationally, why would I want to inject a new keypair? The scenario I can >> think of is that there's data in that instance that I want, and I've lost >> the keypair somehow. Unless that data is on an ephemeral, its gone if we do >> a rebuild. > > This is quite a common scenario - staff member who started the > instance leaves, and you want to access data on the instance, or > maintain/debug the service running on the instance. > > Hitherto, I have used direct db calls to update the key, so it would > be nice if there was an API call to do so. But you also triggered a rebuild in the process? Or you tweaked the keys and did a reboot? This use case came up in the room, but then we started trying to figure out if the folks that mostly had it would also need it on reboot. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] patches for simple typo fixes
On 09/25/2017 09:28 AM, Doug Hellmann wrote:
> I'm less concerned with the motivation of someone submitting the
> patches than I am with their effect. Just like the situation we had
> with the bug squash days a year or so ago, if we had a poorly timed
> set of these trivial patches coming in at our feature freeze deadline,
> it would be extremely disruptive. So to me the fact that we're
> seeing them in large batches means we have people who are not fully
> engaged with the community and don't understand the impact they're
> having. My goal is to reach out and try to improve that engagement,
> and try to help them become more fully constructive contributors.
I think that quantifying how big that impact is would be good before
deciding it needs to be a priority to act upon. There are lots of things
that currently swamp our system, and on my back of the envelope math and
spot checking on resources used, these really aren't a big concern.
But it's harder to see that until we really start accounting for CI time
by project / person, and what kinds of things really do consume the system.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] patches for simple typo fixes
On 09/25/2017 07:56 AM, Chris Dent wrote: > On Fri, 22 Sep 2017, Paul Belanger wrote: > >> This is not a good example of encouraging anybody to contribute to the >> project. > > Yes. This entire thread was a bit disturbing to read. Yes, I totally > agree that mass patches that do very little are a big cost to > reviewer and CI time but a lot of the responses sound like: "go away > you people who don't understand our special culture and our > important work". > > That's not a good look. > > Matt's original comment is good in and of itself: I saw a thing, > let's remember to curtail this stuff and do it in a nice way. > > But then we generate a long thread about it. It's odd to me that > these threads sometimes draw more people out then discussions about > actually improving the projects. > > It's also odd that if OpenStack were small and differently > structured, any self-respecting maintainer would be happy to see > a few typo fixes and generic cleanups. Anything to push the quality > forward is nice. But because of the way we do review and because of > the way we do CI these things are seen as expensive distractions[1]. > We're old and entrenched enough now that our tooling enforces our > culture and our culture enforces our tooling. > > [1] Note that I'm not denying they are expensive distractions nor > that they need to be managed as such. They are, but a lot of that > is on us. I was trying to ignore the thread in the hopes it would die out quick. But torches and pitchforks all came out from the far corners, so I'm going to push back on that a bit. I'm not super clear why there is always so much outrage about these patches. They are fixing real things. When I encounter them, I just approve them to get them merged quickly and not backing up the review queue, using more CI later if they need rebasing. They are fixing real things. Maybe there is a CI cost, but the faster they are merged the less likely someone else is to propose it in the future, which keeps down the CI cost. And if we have a culture of just fixing typos later, then we spend less CI time on patches the first time around with 2 or 3 iterations catching typos. I think the concern is the ascribed motive for why people are putting these up. That's fine to feel that people are stat padding (and that too many things are driven off metrics). But, honestly, that's only important if we make it important. Contributor stats are always going to be pretty much junk stats. They are counting things to be the same which are wildly variable in meaning (number of patches, number of Lines of Code). My personal view is just merge things that fix things that are wrong, don't care why people are doing it. If it gets someone a discounted ticket somewhere, so be it. It's really not any skin off our back in the process. If people are deeply concerned about CI resources, step one is to get some better accounting into the existing system to see where resources are currently spent, and how we could ensure that time is fairly spread around to ensure maximum productivity by all developers. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [skip-level-upgrades][fast-forward-upgrades] PTG summary
On 09/20/2017 02:25 PM, Doug Hellmann wrote: Excerpts from Dan Smith's message of 2017-09-20 10:09:54 -0700: - Modify the `supports-upgrades`[3] and `supports-accessible-upgrades`[4] tags I have yet to look into the formal process around making changes to these tags but I will aim to make a start ASAP. We've previously tried to avoid changing assert tag definitions because we then have to re-review all of the projects that already have the tags to ensure they meet the new criteria. It might be easier to add a new tag for assert:supports-fast-forward-upgrades with the criteria that are unique to this use case. We already have a confusing array of upgrade tags, so I would really rather not add more that overlap in complicated ways. Most of the change here is clarification of things I think most people assume, so I don't think the validation effort will be a lot of work. --Dan OK, I'll wait to see what the actual updates look like before commenting further, but it sounds like working on the existing tag definitions is a good first step. Agreed. We're already at 5 upgrade tags now? I think honestly we're going to need a picture to explain the differences between them. Based on the confusion that kept seeming to come during discussions at the PTG, I think we need to circle around and figure out if there are different ways to explain this to have greater clarity. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] dashboard query changes since upgrade
On 09/19/2017 09:00 AM, Sean Dague wrote: > I've been iterating through gerrit dashboards this morning trying to > figure out why they no longer show any changes. > > It looks like the query: label:Code-Review>=-2,self now matches changes > you haven't voted on. Previously (probably to a bug) this only matched > patches where you had a -2,-1,+1,+2 vote on them. > > I'll be poking around today to figure out what the options are to get > equivalent functionality is out of the system, then update the gerrit > dashboards in gerrit-dash-creator based on that. It appears that reviewedby:self actually works like we were hacking the old one to work. I've pushed through a bulk fix for most of the dashboards here - https://review.openstack.org/#/c/505247/ However local versions will need local patching. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] dashboard query changes since upgrade
I've been iterating through gerrit dashboards this morning trying to figure out why they no longer show any changes. It looks like the query: label:Code-Review>=-2,self now matches changes you haven't voted on. Previously (probably to a bug) this only matched patches where you had a -2,-1,+1,+2 vote on them. I'll be poking around today to figure out what the options are to get equivalent functionality is out of the system, then update the gerrit dashboards in gerrit-dash-creator based on that. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] removing screen from devstack - RSN
I would love to. Those were mostly left because devstack-gate (and related tooling like elasticsearch) is not branch aware, so things get ugly on the conditionals for changing expected output files. That might be a good popup infra topic at PTG. On 09/08/2017 04:17 PM, John Villalovos wrote: > Does this mean we can now get more user friendly names for the log files? > > Currently I see names like: > screen-dstat.txt.gz > screen-etcd.txt.gz > screen-g-api.txt.gz > screen-g-reg.txt.gz > screen-ir-api.txt.gz > screen-ir-cond.txt.gz > screen-keystone.txt.gz > screen-n-api-meta.txt.gz > screen-n-api.txt.gz > screen-n-cauth.txt.gz > screen-n-cond.txt.gz > screen-n-cpu.txt.gz > screen-n-novnc.txt.gz > screen-n-sch.txt.gz > screen-peakmem_tracker.txt.gz > screen-placement-api.txt.gz > screen-q-agt.txt.gz > screen-q-dhcp.txt.gz > screen-q-l3.txt.gz > screen-q-meta.txt.gz > screen-q-metering.txt.gz > screen-q-svc.txt.gz > screen-s-account.txt.gz > screen-s-container.txt.gz > screen-s-object.txt.gz > screen-s-proxy.txt.gz > > People new to OpenStack don't really know that 'q' means neutron. > > > > On Thu, Sep 7, 2017 at 5:45 AM, Sean Dague <s...@dague.net > <mailto:s...@dague.net>> wrote: > > On 08/31/2017 06:27 AM, Sean Dague wrote: > > The work that started last cycle to make devstack only have a single > > execution mode, that was the same between automated QA and local, is > > nearing it's completion. > > > > https://review.openstack.org/#/c/499186/ > <https://review.openstack.org/#/c/499186/> is the patch that will remove > > screen from devstack (which was only left as a fall back for things like > > grenade during Pike). Tests are currently passing on all the gating jobs > > for it. And experimental looks mostly useful. > > > > The intent is to merge this in about a week (right before PTG). So, if > > you have a complicated devstack plugin you think might be affected by > > this (and were previously making jobs pretend to be grenade to keep > > screen running), now is the time to run tests against this patch and see > > where things stand. > > This patch is in the gate and now merging, and with it devstack now has > a single run mode, using systemd units, which is the same between test > and development. > > Thanks to everyone helping with the transition! > > -Sean > > -- > Sean Dague > http://dague.net > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev> > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] removing screen from devstack - RSN
On 09/07/2017 04:54 PM, Eric Fried wrote: All- The plain pdb doc patch [1] is merging. On clarkb's suggestion, I took a look at remote-pdb [2], and it turned out to be easy-peasy to use. I submitted a followon doc patch for that [3]. Thanks, John, for speaking up and getting this rolling. Eric Approved for merge, should be in shortly. Eric, thanks again for stepping up and pulling this all together. Very much appreciated. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] removing screen from devstack - RSN
On 09/07/2017 01:52 PM, Eric Fried wrote: John- You're not the only one for whom the transition to systemd has been painful. However... It *is* possible (some would argue just as easy) to do all things with systemd that were done with screen. For starters, have you seen [1] ? Though looking at that again, I realize it could use a section on how to do pdb - I'll propose something for that. In the meantime, feel free to find me in #openstack-dev and I can talk you through it. [1] https://docs.openstack.org/devstack/latest/systemd.html Thanks, Eric Fried (efried) Thank you Eric. Would love to get a recommended pdb path into the docs. Ping me as soon as it's up for review, and I'll get it merged quickly. Thanks for stepping up here, it's highly appreciated. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Unified Limits work stalled - next steps, need for new leadership
This is an email that's probably overdue, but time gets away from all of us. Coming out of Atlanta and even Boston we were pushing pretty hard on a new approach to unify limits, as well as building the infrastructure to have hierarchical limits possible in OpenStack (keystone merged spec - https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/unified-limits.html) However, this effort pretty much as run out of gas, and really needs a new driver (or set of drivers) to make forward progress. My current set of responsibilities doesn't really make it possible for me to own driving this (though I will be happy to help others). That seems to also be true of many of the others that were engaged. This is going out in advance of the PTG in case any developer(s), or organizations feel that hierarchical limits are a priority for them, and would be willing to step up there. If so please feel free to reach out, and we can carve out some PTG time in helping a new push organize here. However, without new leadership here, the current plan is to just put the effort to rest. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] removing screen from devstack - RSN
On 08/31/2017 06:27 AM, Sean Dague wrote: > The work that started last cycle to make devstack only have a single > execution mode, that was the same between automated QA and local, is > nearing it's completion. > > https://review.openstack.org/#/c/499186/ is the patch that will remove > screen from devstack (which was only left as a fall back for things like > grenade during Pike). Tests are currently passing on all the gating jobs > for it. And experimental looks mostly useful. > > The intent is to merge this in about a week (right before PTG). So, if > you have a complicated devstack plugin you think might be affected by > this (and were previously making jobs pretend to be grenade to keep > screen running), now is the time to run tests against this patch and see > where things stand. This patch is in the gate and now merging, and with it devstack now has a single run mode, using systemd units, which is the same between test and development. Thanks to everyone helping with the transition! -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] remove uml code from nova tree
As some of the libvirt refactorings have hit recently, especially around privsep, one bit of cruft is UML (user mode linux) support. For people that do not yet have gray hairs, this was the earliest form of upstream "virtualization" that Linux had. Allowed you to run a kernel as a user process with a special loop filesystem. It predates Xen. Very early virtual host companies used it (and all switched off of it a long time ago). It shows up in commit #1 in Nova - bf6e6e718cdc7488e2da87b21e258ccc065fe499 - because it was proposed for early testing. However, with the gate doing nested qemu, this hasn't been used for a very long time. I think this classifies as the kind of thing we can straight remove, and it will simplify a few things if we do. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] removing screen from devstack - RSN
The work that started last cycle to make devstack only have a single execution mode, that was the same between automated QA and local, is nearing it's completion. https://review.openstack.org/#/c/499186/ is the patch that will remove screen from devstack (which was only left as a fall back for things like grenade during Pike). Tests are currently passing on all the gating jobs for it. And experimental looks mostly useful. The intent is to merge this in about a week (right before PTG). So, if you have a complicated devstack plugin you think might be affected by this (and were previously making jobs pretend to be grenade to keep screen running), now is the time to run tests against this patch and see where things stand. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposing Balazs Gibizer for nova-core
On 08/22/2017 09:18 PM, Matt Riedemann wrote: > I'm proposing that we add gibi to the nova core team. He's been around > for awhile now and has shown persistence and leadership in the > multi-release versioned notifications effort, which also included > helping new contributors to Nova get involved which helps grow our > contributor base. > > Beyond that though, gibi has a good understanding of several areas of > Nova, gives thoughtful reviews and feedback, which includes -1s on > changes to get them in shape before a core reviewer gets to them, > something I really value and look for in people doing reviews who aren't > yet on the core team. He's also really helpful with not only reporting > and triaging bugs, but writing tests to recreate bugs so we know when > they are fixed, and also works on fixing them - something I expect from > a core maintainer of the project. > > So to the existing core team members, please respond with a yay/nay and > after about a week or so we should have a decision (knowing a few cores > are on vacation right now). > +1 -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] O search where art thou?
On 08/11/2017 08:22 AM, Matt Riedemann wrote: > Before the great docs migration, searching for something in the nova > devref was restricted to the nova devref: > > https://docs.openstack.org/nova/ocata/search.html?q=rbd_keywords=yes=default > > > Now searching for something in the nova docs searches docs.o.o, ask.o.o, > maybe other places, but it's basically so many unrelated results it's > not usable for me: > > https://docs.openstack.org/nova/latest/search.html#stq=rbd=1 > > Is there a way we can just get the content-specific (restricted to > whatever is in the nova repo for docs) search results back and if people > want more, they go to docs.o.o to search for stuff? > > Because when I'm in nova docs looking for rbd stuff, I don't want to > sift through forum questions or glance docs or cinder docs, etc. The following change will bring back local search behavior like that - https://review.openstack.org/#/c/493107/ It will need to be landed and release, and probably backported to stable/pike given that all the branches have been cut. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] O search where art thou?
On 08/11/2017 10:49 AM, Anne Gentle wrote: > On Fri, Aug 11, 2017 at 8:10 AM, Sean Dague <s...@dague.net> wrote: >> On 08/11/2017 08:50 AM, Anne Gentle | Just Write Click wrote: >>> Yeah, I need to circle back in the theme work to make sure both search >>> scopes are available. My prior attempt had some wonky CSS debugging and I >>> needed to separate patches more. >>> >>> I'll put up another patch to the theme today to bring in the Sphinx search >>> in the sidebar as it was before. >>> >>> I'm not sure how to solve the sort problem Sean notes, would like help >>> there. >>> Hope this helps - >>> Anne >> >> If we have the option to use sphinx baked in search, instead of the >> bounce out to google swift_search that's being installed, it's all >> solved. The sphinx search builds the search terms into a compiled >> javascript file on build, and will only link to content in the tree. >> >> I think the thing to do is probably enhance openstackdocstheme to have >> some toggle of "local_search = True" which would get rid of swift_search >> and just use baked in local search. Then on a per site basic people >> could turn it on / off, and openstackdocstheme could still be used for >> sites that want global search. > > The global search will always be available in the header, so my idea > is to add the scoped-to-project's-collection search to the sidebar. > Let me know if that works and I'll get going on it. The sidebar would be good. I still think it's a little confusing to have 2 different search fields, but maybe if "Search" in the toolbar was "Search all of OpenStack", and the sidebar was more clearly "Search $project Docs", it would be more clear. The big technical bit is actually the openstackdocstheme replaced search.html with a form that uses swift_search. I was hacking on this this morning and it seems that in able to get that back to something that works with built in search is going to need to redo that page, and also add the ability to inject script_footers (as all the javascript is late loaded in openstackdocstheme). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] O search where art thou?
On 08/11/2017 08:50 AM, Anne Gentle | Just Write Click wrote: > Yeah, I need to circle back in the theme work to make sure both search scopes > are available. My prior attempt had some wonky CSS debugging and I needed to > separate patches more. > > I'll put up another patch to the theme today to bring in the Sphinx search in > the sidebar as it was before. > > I'm not sure how to solve the sort problem Sean notes, would like help there. > Hope this helps - > Anne If we have the option to use sphinx baked in search, instead of the bounce out to google swift_search that's being installed, it's all solved. The sphinx search builds the search terms into a compiled javascript file on build, and will only link to content in the tree. I think the thing to do is probably enhance openstackdocstheme to have some toggle of "local_search = True" which would get rid of swift_search and just use baked in local search. Then on a per site basic people could turn it on / off, and openstackdocstheme could still be used for sites that want global search. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] O search where art thou?
On 08/11/2017 08:22 AM, Matt Riedemann wrote: > Before the great docs migration, searching for something in the nova > devref was restricted to the nova devref: > > https://docs.openstack.org/nova/ocata/search.html?q=rbd_keywords=yes=default > > > Now searching for something in the nova docs searches docs.o.o, ask.o.o, > maybe other places, but it's basically so many unrelated results it's > not usable for me: > > https://docs.openstack.org/nova/latest/search.html#stq=rbd=1 > > Is there a way we can just get the content-specific (restricted to > whatever is in the nova repo for docs) search results back and if people > want more, they go to docs.o.o to search for stuff? > > Because when I'm in nova docs looking for rbd stuff, I don't want to > sift through forum questions or glance docs or cinder docs, etc. Equally problematic, in the rbd search above it returns content from all published branches, and seems to be coming back in reverse order. So mitaka content is the first link for folks for searching from latest docs. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [docs] importing openstack-manuals questions, looking for strategies
On 08/09/2017 09:53 AM, Doug Hellmann wrote:
>
> The option discovery stuff in oslo.config works using entry point
> plugins. oslo.vmware doesn't defined any plugins for the oslo.config.opts
> namespace.
>
> Doug
Some more interacting debugging with Doug, and this is what I discovered.
* oslo.vmware wasn't actually what I wanted, it was nova.conf.vmware
* in order for nova.conf.vmware to be a valid reference it needs an
entry point in setup.cfg -
https://github.com/openstack/nova/blob/338ae5ef97ab7dac0831fe4e641be0d2a838b61c/setup.cfg#L30
* for that entry point to work, nova.conf.vmware needs a list_opts
function. Which we have, but it does not provide the expected format for
the importer
The net of it is that we can't use this method without changes in the
nova tree that are probably not worth it during the freeze period
(especially as a lot of this imported documentation is going to need
some heavy pruning / editing next cycle to get into shape).
For now it's just a copy and paste of the old opts.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [docs] importing openstack-manuals questions, looking for strategies
On 08/09/2017 09:24 AM, Sean Dague wrote: > On 08/09/2017 09:19 AM, Doug Hellmann wrote: >> Excerpts from Sean Dague's message of 2017-08-08 17:01:29 -0400: >>> When trying to import >>> https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/compute/hypervisor-vmware.rst#L2 >>> into the Nova admin config reference >>> (https://review.openstack.org/#/c/491853), a couple of interesting >>> challenges popped up. These pop up in a couple of other places, but this >>> one file nicely contains both of them. >>> >>> The first is the common autogenerated files (like >>> https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/tables/nova-vmware.rst#L11). >>> To the best of my knowledge we don't have that autogeneration tooling in >>> the projects. Should we just be copy/pasting this content in? Is there >>> another better strategy there? >> >> oslo.config does have an extension for generating a table-format >> list of all of the configuration options, and the idea is to replace >> those included tables with the 'show-options' directive. It uses >> the same inputs as the sample file generator. See >> https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html for >> details > > So in this case we could: > > .. show-options:: > >oslo.vmware > > > And that would have the same impact? I'm attempting to do this, and just getting: Warning, treated as error: Could not load oslo.vmware .tox/docs/bin/pip freeze shows that oslo.vmware is installed the tox env. Any ideas what I'm doing incorrectly? -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [docs] importing openstack-manuals questions, looking for strategies
On 08/09/2017 09:19 AM, Doug Hellmann wrote: > Excerpts from Sean Dague's message of 2017-08-08 17:01:29 -0400: >> When trying to import >> https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/compute/hypervisor-vmware.rst#L2 >> into the Nova admin config reference >> (https://review.openstack.org/#/c/491853), a couple of interesting >> challenges popped up. These pop up in a couple of other places, but this >> one file nicely contains both of them. >> >> The first is the common autogenerated files (like >> https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/tables/nova-vmware.rst#L11). >> To the best of my knowledge we don't have that autogeneration tooling in >> the projects. Should we just be copy/pasting this content in? Is there >> another better strategy there? > > oslo.config does have an extension for generating a table-format > list of all of the configuration options, and the idea is to replace > those included tables with the 'show-options' directive. It uses > the same inputs as the sample file generator. See > https://docs.openstack.org/oslo.config/latest/reference/sphinxext.html for > details So in this case we could: .. show-options:: oslo.vmware And that would have the same impact? -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [docs] importing openstack-manuals questions, looking for strategies
When trying to import https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/compute/hypervisor-vmware.rst#L2 into the Nova admin config reference (https://review.openstack.org/#/c/491853), a couple of interesting challenges popped up. These pop up in a couple of other places, but this one file nicely contains both of them. The first is the common autogenerated files (like https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/tables/nova-vmware.rst#L11). To the best of my knowledge we don't have that autogeneration tooling in the projects. Should we just be copy/pasting this content in? Is there another better strategy there? The second is cross references that don't yet exist. In this case - https://github.com/openstack/openstack-manuals/blob/6f9fc171800e8a435011f38cd4558e900884ce86/doc/config-reference/source/compute/hypervisor-vmware.rst#L981. Which is :ref:`block_storage_vmdk_driver`, that would be in the cinder manual, but does not seem to yet be there. I'm not sure what the best strategy is here. Just a TODO to find the right cinder ref once it shows up? If anyone has thoughts on the best resolutions here, that would be great. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [docs] Ideal landing page
On 08/07/2017 06:58 PM, Doug Hellmann wrote: > Excerpts from Sean Dague's message of 2017-08-07 17:07:37 -0400: >> The integration of all the manual docs, plus all of our native docs >> trying to get restructured a bit, has left us with a first order mess >> when it comes to our front page (and the Table of Contents in the >> sidebar) - https://docs.openstack.org/nova/latest/ >> >> I started trying to pull things out of the sidebar TOC by creating some >> interior landing pages for Contributors and a Deep Dive Technical >> Reference. In staring at what was left on the page for a good hour today >> I came up with the following ideal front page outline. >> >> https://etherpad.openstack.org/p/ideal-nova-docs-landing-page >> >> This probably turns into 3 additional patches on topic:bp/doc-migration >> * For Users >> * For Operators >> * Index page smooth over >> >> The original docs spec didn't really separate out users / operators as >> target audiences. None of this will involve moving subpages around (so > > End-user content goes in /user. Operator content goes in /admin. See > http://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html#proposed-change > for details. Ok, I guess a lot of things got binned incorrectly during the transition. I think the definition of user was less clearly understood. But it's fine, we can collect things together in subpages however works. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] Concerns with docs migration
On 08/07/2017 05:04 PM, Clark Boylan wrote: > On Wed, Aug 2, 2017, at 01:44 PM, Clark Boylan wrote: >> On Wed, Aug 2, 2017, at 11:37 AM, Sean Dague wrote: >>> On 08/02/2017 12:28 PM, Clark Boylan wrote: >>>> On Wed, Aug 2, 2017, at 07:55 AM, Matt Riedemann wrote: >>>>> Now that Stephen Finucane is back from enjoying his youth and >>>>> gallivanting all over Europe, and we talked about a few things in IRC >>>>> this morning on the docs migration for Nova, I wanted to dump my >>>>> concerns here for broader consumption. >>>>> >>>>> 1. We know we have to fix a bunch of broken links by adding in redirects >>>>> [1] which sdague started here [2]. However, that apparently didn't catch >>>>> everything, e.g. [3], so I'm concerned we're missing other broken links. >>>>> Is there a way to find out? >>>> >>>> The infra team can generate lists of 404 urls fairly easily on the docs >>>> server. This won't show you everything but will show you what urls >>>> people are finding/using that 404. >>> >>> If we could get a weekly report of 404 urls posted somewhere public, >>> that would be extremely useful, because the easy ones based on git >>> renames are done, and everything else is going to require human >>> inspection to figure out what the right landing target is. >>> >> >> I've pushed https://review.openstack.org/#/c/490175/ which will generate >> a report each day for roughly the last day's worth of 404s. You should >> be able to see them at https://docs.openstack.org/404s once the change >> merges and the cron job fires. >> >> You can see what that will look like (from my test runs) at >> http://paste.openstack.org/show/617322/. Note that this isn't a complete >> file because paste.openstack.org truncated it but you'll get the full >> data from the wbeserver once this change merges. > > http://files.openstack.org/docs-404s/ is now live (note it moved to > http://files.openstack.org/docs-404s because that is where we are > hosting raw bits of utility info for this hosting service). The current > content there was just generated by running the scripts manually, but it > should update daily at 0700UTC going forward. Awesome. Thank you Clark. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] [docs] Ideal landing page
The integration of all the manual docs, plus all of our native docs trying to get restructured a bit, has left us with a first order mess when it comes to our front page (and the Table of Contents in the sidebar) - https://docs.openstack.org/nova/latest/ I started trying to pull things out of the sidebar TOC by creating some interior landing pages for Contributors and a Deep Dive Technical Reference. In staring at what was left on the page for a good hour today I came up with the following ideal front page outline. https://etherpad.openstack.org/p/ideal-nova-docs-landing-page This probably turns into 3 additional patches on topic:bp/doc-migration * For Users * For Operators * Index page smooth over The original docs spec didn't really separate out users / operators as target audiences. None of this will involve moving subpages around (so no additional redirects), it will just be a couple of sections and landing documents where appropriate. Please chime in if you have opinions. Otherwise I'm going to start writing the rest of this tomorrow to try to get it all sorted for RC. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon][Nova] Editing flavor causing instance flavor display error
On 08/03/2017 06:13 AM, Zhenyu Zheng wrote: > I was thinking, the current "edit" in Horizon is delete-and-create, and > it is there maybe just because > flavor has many fields, user may want to have a new flavor but just > modify one of the old flavor, so > they don't want to manually copy all other fields. And it is the > automatic delete action that causes > all the problem. Maybe horizon can provide a copy-and-modify action and > leave the deletion of > the flavor to the admin. For what it is worth, it is already an admin level permission. I do think that "Copy and Modify" is a better paradigm. Or "New Flavor based on X" which will prefill based on an existing one. The Delete flavor button should come with a giant warning of "This will make a lot of information in your environment confusing, you should never do this". -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon][Nova] Editing flavor causing instance flavor display error
On 08/02/2017 10:12 PM, Zhenyu Zheng wrote: > Hi All, > > Horizon provided the feature to allow user edit existing flavors, no > matter it is currently used by any instances or not. While Nova doesn't > provide this kind of ability, Horizon achieved this by deleting the old > flavor first and create a new flavor with the requested properties. And > the flavor ID will be changed to a new UUID. > > This causes problem when display instances with latest Nova, Horizon > display flavor name by request flavor details using the flavor id > returned by get_instance request, because Nova now moved flavor to > api_db, and when a flavor got deleted, it will be directly removed from > the DB, and when displaying, the field will be "Not Available". > > Should we stop support editing existing flavors? It is actually > deleting-and-create. Yes, it should not be a feature in Horizon, because it's extremely misleading what it does. It's actually really breaking their environment. It's unfortunate that it was implemented there at all. :( > Maybe we should at least add WARNING notes about this when editing > flavors, about how it is actually done and what kind of influence will > be to the existing instances. > > Nova now(> microversion 2.47) can reply embedded flavor details > including ram, vcpus, original_name etc. > > Since we provided this flavor editing feature and we displayed "name" as > a identifier, after we done some flavor edit, even we fix the above > mentioned problem and displayed the correct flavor name, the flavor > details will be different than the actual instance type. There was an extremely long an heated discussion in the Nova room in Atlanta about including that name in server document because of this Horizon behavior. I came down on the side of showing it, because people that use the flavor "rename" function are actually basically breaking their environment (in many ways). Lots of people don't do that, so this is useful to them so that they can create new instances like the ones there. The only way to change the flavor of an instance is to resize it. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] Concerns with docs migration
aram going to remove with deprecated process. > > IMO, Nova doc should talk about more conceptual or operation things > and then give reference to CLI doc as example. > For example: > "Launch VM doc in Nova" tells about all precondition to launch VM and > APIs (it can be API name or API ref or Curl example) to get required > data and launch VM and at the end give reference to CLI (novaclient or > osc) doc where set of CLI are shown for that operation. If no CLI doc > exist for particular operation, then nova doc still holds good > information of doing that using APIs. This is about audience and usage. We have the following audience and users: * CLI users (nova cli or openstack client) * Horizon users * API users (possibly using python APIs, possibly others) When a person shows up at our site from where ever, we should assume they are a CLI user. It's universal, and easy to show examples. It's also just simpler to integrate these examples then it is to integrate Horizon usage. API users aren't going to be reading the using nova guide, they are going to be deep in the API ref, and be looking for their examples there. We shouldn't be putting API examples in line in the admin or user guides. It's fine and good to reference that Nova is API driven, but again, you present curl commands to users that are just trying to learn the system, and you've lost them. The other important point about documentation, is that duplication of content is not a sin. It's fine and good to have then entire clear set of steps needed in one place without having to chase back references. It's totally fine if "how to start a server" is content complete in the Nova doc, and the openstack client doc, and the nova cli doc. The boundaries that we draw around these in our head are because after years of use, we think about things in certain buckets, but for a new user finding our stuff on google, they don't have those buckets. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] Concerns with docs migration
On 08/02/2017 03:32 PM, Dean Troyer wrote: > On Wed, Aug 2, 2017 at 11:20 AM, Doug Hellmann <d...@doughellmann.com> wrote: >> My gut response is to say that if the example uses nova-manage or >> one of the nova service executables, then that makes sense to leave >> in the nova tree, but otherwise it should go into either the >> python-novaclient or python-openstackclient repositories (depending >> on which command line tool is used in the example). > > I don't think this is a good rule of thumb... > >> On the other hand, I can see that being somewhat confusing for >> someone who lands at the nova docs and can't find instructions for >> how to use it. Maybe less confusing, though, than if I am not >> *running* nova but am trying to use a nova deployed by someone else >> and I start from the python-novaclient or python-openstackclient >> docs because I installed one of those in order to talk to nova. > > When the point of the example is to illustrate configuring nova, the > example should stay with the nova code, even if it uses novaclient or > osc. The examples that are about _using_ novaclient or osc belong in > those repos, but where they are just a means to configuring something > in nova, they should remain with nova and use the tools that users are > expected to be familiar with (novaclient and/or osc in this example). > >> I thought "put the docs with the code" would be a simple enough >> rule that we wouldn't have to think too hard about where to put >> individual example files, which would speed up the migration. > > If I find a doc that tells me how to set up a VM with a Neutron > network and ports and subnets and floating IPs that uses curl, I'm not > reading farther. ++ that is actually kind of a punch in the face to our users -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] Concerns with docs migration
On 08/02/2017 12:28 PM, Clark Boylan wrote: > On Wed, Aug 2, 2017, at 07:55 AM, Matt Riedemann wrote: >> Now that Stephen Finucane is back from enjoying his youth and >> gallivanting all over Europe, and we talked about a few things in IRC >> this morning on the docs migration for Nova, I wanted to dump my >> concerns here for broader consumption. >> >> 1. We know we have to fix a bunch of broken links by adding in redirects >> [1] which sdague started here [2]. However, that apparently didn't catch >> everything, e.g. [3], so I'm concerned we're missing other broken links. >> Is there a way to find out? > > The infra team can generate lists of 404 urls fairly easily on the docs > server. This won't show you everything but will show you what urls > people are finding/using that 404. If we could get a weekly report of 404 urls posted somewhere public, that would be extremely useful, because the easy ones based on git renames are done, and everything else is going to require human inspection to figure out what the right landing target is. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][docs] Concerns with docs migration
On 08/02/2017 12:20 PM, Doug Hellmann wrote: > Excerpts from Akihiro Motoki's message of 2017-08-03 01:02:59 +0900: >> 2017-08-03 0:52 GMT+09:00 Doug Hellmann <d...@doughellmann.com>: >>> Excerpts from Stephen Finucane's message of 2017-08-02 16:35:23 +0100: >>>> On Wed, 2017-08-02 at 09:28 -0600, Chris Friesen wrote: >>>>> On 08/02/2017 09:22 AM, Stephen Finucane wrote: >>>>>> On Wed, 2017-08-02 at 09:55 -0500, Matt Riedemann wrote: >>>>>>> 3. The patch for the import of the admin guide [8] is missing some CLI >>>>>>> specific pages which are pretty useful given they aren't documented >>>>>>> anywhere else, like the forced_host part of the compute API [9]. >>>>>>> Basically anything that's cli-nova-* in the admin guide should be in the >>>>>>> Nova docs. It's also missing the compute-flavors page [10] which is >>>>>>> pretty important for using OpenStack at all. >>>>>> >>>>>> This is a tricky one. Based on previous discussions with dhellmann, the >>>>>> plan >>>>>> seems to be to replace any references to 'nova xxx' or 'openstack xxx' >>>>>> commands >>>>>> (i.e. commands using python-novaclient or python-openstackclient) in >>>>>> favour >>>>>> of >>>>>> 'curl'-based requests. The idea here is that the Python clients are not >>>>>> the >>>>>> only clients available, and we shouldn't be "mandating" their use by >>>>>> referencing them in the docs. I get this, though I don't fully agree with >>>>>> it >>>>>> (who really uses curl?) >>>>> >>>>> Are we going to document the python clients elsewhere then? >>>> >>>> I think the docs would go into the respective python clients docs? >>> >>> Right. I don't remember the details of the curl discussion, but I >>> think what I was trying to say there was that the "nova" command >>> line tool installed via python-novaclient should be documented as >>> part of the openstack/python-novaclient repo rather than openstack/nova. >>> A CLI reference for nova-manage, which I think is in openstack/nova, >>> could be documented in openstack/nova. >>> >>> Basically, put the docs with the code, which is the whole point of this >>> migration. >> >> It is true for CLI reference. >> The gray zone is CLI stuffs in the admin guide and/or end-user guide. >> I think this is the point Matt raised. >> cli-nova-* stuffs in the admin guide was per topic like launching instance, >> migrating instances and so on. It is actually beyond the CLI reference to me. >> It is about how to use specific features of nova. >> IMHO this kind of stuffs can be covered by the admin guide or user guide, >> so it fits into openstack/nova (or other server projects). > > Yeah, I don't know. > > My gut response is to say that if the example uses nova-manage or > one of the nova service executables, then that makes sense to leave > in the nova tree, but otherwise it should go into either the > python-novaclient or python-openstackclient repositories (depending > on which command line tool is used in the example). > > On the other hand, I can see that being somewhat confusing for > someone who lands at the nova docs and can't find instructions for > how to use it. Maybe less confusing, though, than if I am not > *running* nova but am trying to use a nova deployed by someone else > and I start from the python-novaclient or python-openstackclient > docs because I installed one of those in order to talk to nova. > > I thought "put the docs with the code" would be a simple enough > rule that we wouldn't have to think too hard about where to put > individual example files, which would speed up the migration. It's not so simple because years ago we decided nova-manage was mostly a bad thing (as you needed to run it on the API node and be able to read db creds with it), and exposed as much as possible of the administrative interfaces over the API. The policy of openstack client was that administrative commands would not go in openstack client. So a bunch of what was once nova-manage (5 years ago), is part of the nova cli now, and not really implemented by other toolkits. This is also why the python nova client is not going away any time soon. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] [ironic] [nova] Trying again on wait_for_compute in devstack
An issue with the xenserver CI was identified. Once we get this patch in, and backported to ocata, it should also address a frequent grenade multinode fail scenario which is plaguing the gate. -Sean On 08/02/2017 07:17 AM, Sean Dague wrote: The 3 node scenarios in Neutron (which are still experimental nv) are typically failing to bring online the 3rd compute. In cells v2 you have to explicitly add nodes to the cells. There is a nova-manage command "discover-hosts" that takes all the compute nodes which have checked in, but aren't yet assigned to a cell, and puts them into a cell of your choosing. We do this in devstack-gate in the gate. However... subnodes don't take very long to setup (so few services). And the nova-compute process takes about 30s before it's done all it's initialization and actually checks in to the cluster. It's a real possibility that discover_hosts will run before subnode 3 checks in. We see it in logs. This also really could come and bite us on any multinode job, and I'm a bit concerned some of the multinode jobs aren't running multinode some times because of it. One way to fix this, without putting more logic in devstack-gate, is ensure that by the time stack.sh finishes, the compute node is up. This was tried previously, but it turned out that we totally missed that it broke Ironic (the check happened too early, ironic was not yet running, so we always failed), Cells v1 (munges hostnames :( ), and PowerVM (their nova-compute was never starting correctly, and they were working around it with a restart later). This patch https://review.openstack.org/#/c/488381/ tries again. The check is moved very late, Ironic seems to be running fine with it. Cells v1 is just skipped, that's deprecated in Nova now, and we're not going to use it in multinode scenarios. The PowerVM team fixed their nova-compute start issues, so they should be good to go as well. This is an FYI that we're going to land this again soon. If you think this impacts your CI / jobs, please speak up. The CI runs on both the main and experimental queue on devstack for this change look pretty good, so I think we're safe to move forward this time. But we also thought that the last time, and were wrong. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [devstack] [ironic] [nova] Trying again on wait_for_compute in devstack
The 3 node scenarios in Neutron (which are still experimental nv) are typically failing to bring online the 3rd compute. In cells v2 you have to explicitly add nodes to the cells. There is a nova-manage command "discover-hosts" that takes all the compute nodes which have checked in, but aren't yet assigned to a cell, and puts them into a cell of your choosing. We do this in devstack-gate in the gate. However... subnodes don't take very long to setup (so few services). And the nova-compute process takes about 30s before it's done all it's initialization and actually checks in to the cluster. It's a real possibility that discover_hosts will run before subnode 3 checks in. We see it in logs. This also really could come and bite us on any multinode job, and I'm a bit concerned some of the multinode jobs aren't running multinode some times because of it. One way to fix this, without putting more logic in devstack-gate, is ensure that by the time stack.sh finishes, the compute node is up. This was tried previously, but it turned out that we totally missed that it broke Ironic (the check happened too early, ironic was not yet running, so we always failed), Cells v1 (munges hostnames :( ), and PowerVM (their nova-compute was never starting correctly, and they were working around it with a restart later). This patch https://review.openstack.org/#/c/488381/ tries again. The check is moved very late, Ironic seems to be running fine with it. Cells v1 is just skipped, that's deprecated in Nova now, and we're not going to use it in multinode scenarios. The PowerVM team fixed their nova-compute start issues, so they should be good to go as well. This is an FYI that we're going to land this again soon. If you think this impacts your CI / jobs, please speak up. The CI runs on both the main and experimental queue on devstack for this change look pretty good, so I think we're safe to move forward this time. But we also thought that the last time, and were wrong. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Broken nova-lxd
On 07/28/2017 12:58 AM, Tony Breeds wrote: > On Fri, Jul 28, 2017 at 10:55:53AM +0800, ChangBo Guo wrote: >> For the specific case with method "last_bytes", it's also used in other >> projects [1], an alternative solution is that add this method in >> oslo.utils.fileutils, then consume it from oslo. >> >> [1] http://codesearch.openstack.org/?q=last_bytes=nope== > > That does look like a thing that a number of projects could use. > > I'm not certain about the interaction with privsep > (https://review.openstack.org/472229) But it is something that worth > looking at when we open again for queens. I think it's also really important to set expectations though that the internals of Nova are the internals of Nova. They are subject to change and refactoring at any time. Just because a particular module has not changed in 4 years, doesn't mean it's not being radically changed tomorrow. Part of the reason to do the hard work of getting a Nova driver upstream is that you don't have any of these issues, because the Nova team will prevent things from breaking your driver (or fix it themselves). Choosing to do a driver out of tree is not recommended and comes with all of these risks. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] 404s on docs website after the great reorg
In the #openstack-nova channel this morning we were debugging some cells v2 things, and ran into the fact that the online docs for this - https://docs.openstack.org/nova/latest/cells.html go to a 404. That's a previously well known link, people have it in their browser history, bookmarks, wiki pages, other websites. My understanding of big moves like this is that redirects are important. Things going blackhole like that not only is an inconvenience to users, but impacts our search engine rankings, and takes a while for them to all sift out. I know in sites I run I'm still regularly getting in bounds to paths on the site that haven't been there for 8 years. It would be really good if we had a way (manual or automated) to have 301 redirects, that are fixable by the teams that now own the documentation (the project teams). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] help required regarding devstack
Also see - https://docs.openstack.org/devstack/latest/systemd.html On 07/24/2017 02:09 PM, Kristi Nikolla wrote: > Hi, > > I do not know about VM migration, however for restarting nova the right > service names are devstack@n-api, devstack@n-cpu, etc. > > ubuntu@k2k1:~$ systemctl list-units | grep devstack > devstack@c-api.service > loaded active running Devstack devstack@c-api.service > devstack@c-sch.service > loaded active running Devstack devstack@c-sch.service > devstack@c-vol.service > loaded active running Devstack devstack@c-vol.service > devstack@dstat.service > loaded active running Devstack devstack@dstat.service > devstack@etcd.service > loaded active running Devstack devstack@etcd.service > devstack@g-api.service > loaded active running Devstack devstack@g-api.service > devstack@g-reg.service > loaded active running Devstack devstack@g-reg.service > devstack@keystone.service > loaded active running Devstack devstack@keystone.service > devstack@n-api-meta.service > loaded active running Devstack devstack@n-api-meta.service > devstack@n-api.service > loaded active running Devstack devstack@n-api.service > devstack@n-cauth.service > loaded active running Devstack devstack@n-cauth.service > devstack@n-cond.service > loaded active running Devstack devstack@n-cond.service > devstack@n-cpu.service > loaded active running Devstack devstack@n-cpu.service > devstack@n-novnc.service > loaded active running Devstack devstack@n-novnc.service > devstack@n-sch.service > loaded active running Devstack devstack@n-sch.service > devstack@placement-api.service > loaded active running Devstack devstack@placement-api.service > devstack@q-agt.service > loaded active running Devstack devstack@q-agt.service > devstack@q-dhcp.service > loaded active running Devstack devstack@q-dhcp.service > devstack@q-l3.service > loaded active running Devstack devstack@q-l3.service > devstack@q-meta.service > loaded active running Devstack devstack@q-meta.service > devstack@q-svc.service > loaded active running Devstack devstack@q-svc.service > system-devstack.slice > loaded active activesystem-devstack.slice > >> On Jul 24, 2017, at 9:43 AM, Ziad Nayyer <ziadnay...@gmail.com> wrote: >> >> Dear, >> >> I am a PhD candidate at COMSATS, lahore, Pakistan. I am working on devstack. >> Just wanted to know whether it supports VM Migration between two devstacks >> installed on two different physical machines as currently I am unable to >> find any lead. Also please let me know how to restart a particular service >> on devstack version pike on centos7. >> >> The screen file is not being generated in devstack folder ->stack-screenrc >> and systemctl only restarts keystone not any other like nova-compute >> >> sudo systemctl restart devstack@keystone (works) >> sudo systemctl restart devstack@nova >> sudo systemctl restart devstack@nova-compute >> >> and any other does not work. >> >> I'll be very thankful. >> >> >> -- >> Regards, >> >> Muhammad Ziad Nayyer Dar >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Deve
[openstack-dev] [oslo] trying to understand CORS deprecation
I'm trying to knock out common deprecation messages which are generating noise in the test runs. There is a deprecation message emitted all the time during test runs in Nova which is: DeprecationWarning: Method 'CORS.set_latent()' has moved to 'method.set_defaults()': CORS.set_latent has been deprecated in favor of oslo_middleware.cors.set_defaults" But from what I can see it's primary caller is the cors middleware itself - https://github.com/openstack/oslo.middleware/blob/1cf39ee5c3739c18fed78946532438550f56356f/oslo_middleware/cors.py#L133-L137 At least I'm having a hard time finding anyone else in this stack calling set_latent. Is this just a circular bug on the cors.py module? -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 07/20/2017 06:20 PM, Nematollah Bidokhti wrote: > Hi, > > I have missed the original email on this subject. > We [Fault Genes WG] have been doing some machine learning analysis on Nova > bugs/issues from 3 different sources (Launchpad, Stackoverflow, > ask.openstack.org). We have been able to take all the issues and bring them > down to 15 clusters. > We have tried to find open source tools that can help us define the fault > classifications, but have not been able to find any tool. > > Therefore, our team have come to the conclusion that we need the support of > some Nova experts to help define the classifications. I would like to have > some discussions with Sean and others that have an interest in this area and > compare notes and see how we can collaborate. > > The goal of our WG is to apply the same technique to all key OpenStack > projects. Sure, would be happy to. All this went a little bit on hold as I was off on vacation and a conference, and am now trying to help getting freeze critical patches back in. But I'll probably start looking again more deeply after the feature freeze. If you can provide me pointers to your machine learning work that clustered things, I'd happily take a look and see how that matches with domain experts. Thanks a bunch for also diving in here! -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache - PLAN OF ACTION
On 07/20/2017 09:27 AM, Sean Dague wrote: > Here is a starting patch that gets us close (no tests yet) - > https://review.openstack.org/#/c/485602/ - it's going to require a paste > change, which is less than idea. After some #openstack-nova IRC discussion this morning, we decided the following: 1) we need something like this! 2) changing paste.ini, and having an upgrade note, is not the end of the world. If you are cutting over from eventlet to uwsgi/apache, you are going to need to do other configuration management changes in your environment, adding this to the mix would be another part of that manual change. 3) try to get this to go silent if enabled under eventlet (to prevent duplicate lines) as a stretch goal (which I think I just got working). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache
On 07/19/2017 06:28 PM, Matt Riedemann wrote: > On 7/19/2017 6:16 AM, Sean Dague wrote: >> We hit a similar issue with placement, and added custom >> paste middleware for that. Maybe we need to consider a similar thing >> here, that would only emit if running under uwsgi/apache? > > For example, this: > > http://logs.openstack.org/97/479497/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a0fb17/logs/screen-placement-api.txt.gz#_Jul_19_03_41_21_429324 > > > If it's not optional for placement, why would we make it optional for > the compute API? Would turning it on always make it log the request IDs > twice or something? > > Is this a problem for glance/cinder/neutron/keystone and whoever else is > logging request IDs in the API? Here is a starting patch that gets us close (no tests yet) - https://review.openstack.org/#/c/485602/ - it's going to require a paste change, which is less than idea. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache
On 07/19/2017 06:28 PM, Matt Riedemann wrote: > On 7/19/2017 6:16 AM, Sean Dague wrote: >> We hit a similar issue with placement, and added custom >> paste middleware for that. Maybe we need to consider a similar thing >> here, that would only emit if running under uwsgi/apache? > > For example, this: > > http://logs.openstack.org/97/479497/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a0fb17/logs/screen-placement-api.txt.gz#_Jul_19_03_41_21_429324 Placement can't run under eventlet, so there was no reason to make it optional (or to not emit if we're under eventlet). I'm fine with it being mandatory, but for niceness not run when we're under eventlet server. > If it's not optional for placement, why would we make it optional for > the compute API? Would turning it on always make it log the request IDs > twice or something? That was my concern. Right now the path for logging the INFO request lines comes from following: * http server is started via oslo.service * oslo.service is a wrapper around eventlet.wsgi * eventlet.wsgi takes a log object in, and uses that for logging * that log object is our log object, and it uses our .info method to emit Which means it has the context, which includes things like global request-id, request-id, project, user, domain, etc. > Is this a problem for glance/cinder/neutron/keystone and whoever else is > logging request IDs in the API? It will be the same issue for anyone else going from oslo.service -> wsgi. I had forgotten that bit of the problem when we did our cut over, but it's a pretty big problem, and it actually makes most of the global request id work somewhat pointless, because we threw away the REST call tracing entirely if people run under the uwsgi/apache model. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache
On 07/19/2017 09:46 PM, Matt Riedemann wrote: > On 7/19/2017 6:16 AM, Sean Dague wrote: >> I was just starting to look through some logs to see if I could line up >> request ids (part of global request id efforts), when I realized that in >> the process to uwsgi by default, we've entirely lost the INFO wsgi >> request logs. :( >> >> Instead of the old format (which was coming out of oslo.service) we get >> the following - >> http://logs.openstack.org/97/479497/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a0fb17/logs/screen-n-api.txt.gz#_Jul_19_03_44_58_233532 >> >> >> >> That definitely takes us a step backwards in understanding the world, as >> we lose our request id on entry that was extremely useful to match up >> everything. We hit a similar issue with placement, and added custom >> paste middleware for that. Maybe we need to consider a similar thing >> here, that would only emit if running under uwsgi/apache? >> >> Thoughts? >> >> -Sean >> > > I'm noticing some other weirdness here: > > http://logs.openstack.org/65/483565/4/check/gate-tempest-dsvm-py35-ubuntu-xenial/9921636/logs/screen-n-sch.txt.gz#_Jul_19_20_17_18_801773 > > > The first part of the log message got cut off: > > Jul 19 20:17:18.801773 ubuntu-xenial-infracloud-vanilla-9950433 > nova-scheduler[22773]: > -01dc-4de3-9da7-8eb3de9e305e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active'), > 'a4eba582-075a-4200-ae6f-9fc7797c95dd': No, it's the log message exceeded buffer limits in systemd journal, and was split across lines. It starts 2 more lines up. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova] Persistent application credentials
On 07/19/2017 10:00 PM, Adrian Turjak wrote: > The problem is then entirely procedural within a team. Do they rotate > all keys when one person leaves? Anything less is the same problem. All > we can do is make rotation less of a pain, but it will still be painful > no matter what, and depending on the situation the team makes the choice > of how to handle rotation if at all. > > The sole reason for project level ownership of these application > credentials is so that a user leaving/being deleted isn't a scramble to > replace keys, and a team has the option/time to do it if they care about > the possibility of that person having known the keys (again, not our > problem, not a security flaw in code). Anything else, pretty much makes > this feature useless for teams. :( > > Having both options (owned by project vs user) is useful, but the > 'security issues' are kind of implied by using project owned app creds. > It's a very useful feature with some 'use at your own risk' attached. I think this is a pretty good summary. In many situations the situation of removing people from projects (termination) will also physically remove their path to said clouds (as they are beyond the firewall). It's not true with public clouds, but it's not making the situation any worse, because right now it's shared passwords to accounts. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] loss of WSGI request logs with request-id under uwsgi/apache
I was just starting to look through some logs to see if I could line up request ids (part of global request id efforts), when I realized that in the process to uwsgi by default, we've entirely lost the INFO wsgi request logs. :( Instead of the old format (which was coming out of oslo.service) we get the following - http://logs.openstack.org/97/479497/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/5a0fb17/logs/screen-n-api.txt.gz#_Jul_19_03_44_58_233532 That definitely takes us a step backwards in understanding the world, as we lose our request id on entry that was extremely useful to match up everything. We hit a similar issue with placement, and added custom paste middleware for that. Maybe we need to consider a similar thing here, that would only emit if running under uwsgi/apache? Thoughts? -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 07/12/2017 02:19 PM, Kendall Nelson wrote: Hey Sean :) So we discussed the issue of tag collisions in the SB meeting we had today. Basically, we came to the conclusion that projects should append their project to the start of the tag, thereby avoiding collision i.e. ironic-compute, nova-compute, manila-storage, swift-storage, cinder-storage. If we can ask bug triagers in their respective projects to follow and uphold the convention, we should be fine. It might also be helpful to add this to any directions projects might have about filing bugs so new contributors start off on the right foot. It would be really nice if we could this as first class support and not bolt on later, because I fear we're going to get pretty complicated adhoc schema definitions here that have to be done client side. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 07/11/2017 04:31 PM, Jeremy Stanley wrote: On 2017-07-10 07:33:28 -0400 (-0400), Sean Dague wrote: [...] Ideally storyboard would just be a lot more receptive to these kinds of things, by emitting a more native event stream, Well, there is http://git.openstack.org/cgit/openstack-infra/storyboard/tree/storyboard/notifications/publisher.py > so replacing or partnering its RabbitMQ publisher with something like an MQTT publisher into firehose.openstack.org is probably not terribly hard for someone with interest in that and would be generally useful. and having really good tag support (preferably actually project scoped tags, so setting it on the nova task doesn't impact the neutron tasks on the same story, as an for instance) [...] Your queries (including those used to build automatic tasklists and boards) could just include project in addition to tag, right? Or is this more of a UI concern, being able to click on an arbitrary tag in the webclient and only get back a set of tagged stories for the same project rather than across all projects? My concern is based on current limitations in launchpad, and to make sure they don't get encoded into Storyboard. Tags in launchpad are at the Bug level. Bugs map to projects as Tasks. Which is why you can have 1 Bug set to be impacting both Nova and Neutron. You get lots of weirdness today when for instance a bug is assigned to Nova and Ironic, and the Nova team tags it "ironic" in triage, but that means that now Ironic has a bug with the "ironic" tag. Then if later Nova is removed from the bug, it ends up really all looking odd and confusing. Or the fact that "compute" as a Nova tag means the compute worker, but other teams tag things with compute to just mean Nova is involved. Project scoped tags would help clarify what context it is in. -Sean __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc] Status update, July 7th
On 07/11/2017 12:14 PM, Dean Troyer wrote: On Fri, Jul 7, 2017 at 3:19 AM, Thierry Carrez <thie...@openstack.org> wrote: == Need for a TC meeting next Tuesday == [...] others). Who is up for discussing those items at our usual meeting slot time on Tuesday ? I am unlikely to make the meeting, travel plans are more fluid than I would like today. Will be there if possible. I will also be unable to attend today due to being in a metal tube at the time of the meeting. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] scheduling oslosphinx for retirement at the start of queens
On 07/10/2017 09:10 AM, Doug Hellmann wrote: > Oslo team, > > With all documentation now moving to use the openstackdocs theme, > I propose that we retire the oslosphinx repository during Queens. > We should go ahead and create the stable/pike branch at the end of > this cycle, so that we have a way to deal with bugs in existing > pike releases, but I think we can retire the repository at any point > after that. > > Thoughts? +1 and thanks for building the oslosphinx module years ago to start us down this standardization on themes. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 07/05/2017 03:23 PM, Emilien Macchi wrote: > > I also believe that some of the scripts could be transformed into > native features of Storyboard where bugs could be auto-triaged > periodically without human intervention. > Maybe it would convince more OpenStack projects to leave Launchpad and > adopt Storyboard? > I would certainly one of those and propose such a change for TripleO & > related projects. Maybe... my concern there is that workflow encoded into trackers is pretty static, and it's hard to evolve, because it impacts all users of that platform. Where as a script that processes bugs externally can adapt really quickly based on what's working / not working with a particular team. There is no 1 right way to handle bugs, it's just about making every bug handling team the most effective that they can be. Which means I assume that different teams would find different parts of this useful, and other parts things they wouldn't want to use at all. That's why I tried to make every "processing unit" it's own cli. Ideally storyboard would just be a lot more receptive to these kinds of things, by emitting a more native event stream, and having really good tag support (preferably actually project scoped tags, so setting it on the nova task doesn't impact the neutron tasks on the same story, as an for instance) so the hack we need to do on LP isn't needed. But, actually, beyond that, keeping the processing logic team specific is a good thing. It's much like the fact that we've largely done gerrit review dashboards client side, because they are fast to iterate on, then server side. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 07/05/2017 03:07 PM, Emilien Macchi wrote: > On Wed, Jun 28, 2017 at 7:33 AM, Ben Nemec <openst...@nemebean.com> wrote: >> >> >> On 06/23/2017 11:52 AM, Sean Dague wrote: >>> >>> The Nova bug backlog is just over 800 open bugs, which while >>> historically not terrible, remains too large to be collectively usable >>> to figure out where things stand. We've had a few recent issues where we >>> just happened to discover upgrade bugs filed 4 months ago that needed >>> fixes and backports. >>> >>> Historically we've tried to just solve the bug backlog with volunteers. >>> We've had many a brave person dive into here, and burn out after 4 - 6 >>> months. And we're currently without a bug lead. Having done a big giant >>> purge in the past >>> >>> (http://lists.openstack.org/pipermail/openstack-dev/2014-September/046517.html) >>> I know how daunting this all can be. >>> >>> I don't think that people can currently solve the bug triage problem at >>> the current workload that it creates. We've got to reduce the smart >>> human part of that workload. >>> >>> But, I think that we can also learn some lessons from what active github >>> projects do. >>> >>> #1 Bot away bad states >>> >>> There are known bad states of bugs - In Progress with no open patch, >>> Assigned but not In Progress. We can just bot these away with scripts. >>> Even better would be to react immediately on bugs like those, that helps >>> to train folks how to use our workflow. I've got some starter scripts >>> for this up at - https://github.com/sdague/nova-bug-tools >> >> >> Just saw the update on https://bugs.launchpad.net/nova/+bug/1698010 and I >> don't agree that assigned but not in progress is an invalid state. If it >> persists for a period of time then sure, but to me assigning yourself a bug >> is a signal that you're working on it and nobody else needs to. Otherwise >> you end up with multiple people working a bug without realizing someone else >> already was. I've seen that happen more than once. >> >> Would it be possible to only un-assign such bugs if they've been in that >> state for a week? At that point it seems safe to say the assignee has >> either moved on or that the bug is tricky and additional input would be >> useful anyway. >> >> Otherwise, big +1 to a bug bot. I need to try running it against the ~700 >> open TripleO bugs... > > I just tried, please send complains to me if I broke something. > > Sean, the tool is really awesome and I was wondering if we could move > it to https://github.com/openstack-infra/release-tools so we > centralize the tools. Probably yes, eventually? Right now I'm honestly trying to figure out the things that are useful here, and the ones that aren't, so a more fluid experimentation is worth while. My end game is to get some of these running and responding in real time which means the core processing logic is going to evolve away from scripts and into some kind of function plugin (the batch interface will just still call them). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 06/28/2017 10:33 AM, Ben Nemec wrote: > > > On 06/23/2017 11:52 AM, Sean Dague wrote: >> The Nova bug backlog is just over 800 open bugs, which while >> historically not terrible, remains too large to be collectively usable >> to figure out where things stand. We've had a few recent issues where we >> just happened to discover upgrade bugs filed 4 months ago that needed >> fixes and backports. >> >> Historically we've tried to just solve the bug backlog with volunteers. >> We've had many a brave person dive into here, and burn out after 4 - 6 >> months. And we're currently without a bug lead. Having done a big giant >> purge in the past >> (http://lists.openstack.org/pipermail/openstack-dev/2014-September/046517.html) >> >> I know how daunting this all can be. >> >> I don't think that people can currently solve the bug triage problem at >> the current workload that it creates. We've got to reduce the smart >> human part of that workload. >> >> But, I think that we can also learn some lessons from what active github >> projects do. >> >> #1 Bot away bad states >> >> There are known bad states of bugs - In Progress with no open patch, >> Assigned but not In Progress. We can just bot these away with scripts. >> Even better would be to react immediately on bugs like those, that helps >> to train folks how to use our workflow. I've got some starter scripts >> for this up at - https://github.com/sdague/nova-bug-tools > > Just saw the update on https://bugs.launchpad.net/nova/+bug/1698010 and > I don't agree that assigned but not in progress is an invalid state. If > it persists for a period of time then sure, but to me assigning yourself > a bug is a signal that you're working on it and nobody else needs to. > Otherwise you end up with multiple people working a bug without > realizing someone else already was. I've seen that happen more than once. The other case, where folks assign themselves and never do anything, happens about 100 times a month. We don't live in an exclusive lock environment, anyone can push a fix for a bug, and gerrit assigns it to them. I don't see why we'd treat LP any differently. Yes, this sometimes leads to duplicate fixes, however in the current model it's far more frequent for bugs to be blocked away as "assigned" when no one is working on them. A future version might be smarter and give folks a 7 day window or something, but parsing back the history to understand the right logic there is tricky enough that it's a future enhancement at best. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] New upgrade test tool proposal.
On 06/27/2017 03:19 PM, Dean Troyer wrote: > On Tue, Jun 27, 2017 at 1:37 PM, Jay Pipes <jaypi...@gmail.com> wrote: >> Hi Castulo, sorry for the delayed response on this. Has your team moved >> forward on any of this? > > IIRC this work was impacted by the OSIC shutdown, I believe it is not > currently on anyone's radar. > >> What about the Grenade testing framework that uses devstack as its >> deployment system was not useful or usable for you? > > I can take at least part of the blame in encouraging them to not > attempt to leverage Grenade directly. Grenade needs to be replaced as > it has far out-lived its expected life[0]. > > Grenade was built to do static in-place upgrades and the fact that it > has been pushed as far as it has is a happy surprise to me. However, > it is fundamentally limited in its abilities as a test orchestrator, > implementing robust multi-node capabilities and the granularity that > is required to properly do upgrade testing really needs a reboot. In > a well-funded world that would include replacing DevStack too, which > while nice is not strictly necessary to achieve the testing goals they > had. > > The thing that Grenade and DevStack have going for them besides > inertia is that they are not otherwise tied to any deployment > strategy. Starting over from scratch really is not an option at this > point, something existing really does need to be leveraged even though > it may hurt some feelings along the way for the project(s) not chosen. > > dt > > > [0] Seriously, I never expected Grenade (or DevStack for that matter) > to have survived this long, but they have mostly because they were/are > just barely good enough that nobody wants to fund replacing them. Well, we also lengthened their life and usefulness by adding an external plugin interface, that let people leverage what we had without having to wait on review queues. But, I also agree that grenade is largely pushed to its limits. That also includes the fact that I'm more or less the only "active" reviewer at this point. The only way this is even vaguely tenable is by the complexity being capped so it's straight forward enough to think through implications of patches. The complicated upgrade orchestration including multiple things executing at the same time, breaks that. It's cool if someone wants to do it, however it definitely needs more dedicated folks on the review side to be successful. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Turning TC/UC workgroups into OpenStack SIGs
On 06/27/2017 09:42 AM, Thierry Carrez wrote: > Sean Dague wrote: >> I also think it's fine to rebrand WG to SIG, but we should also be >> honest that it's mostly a rebrand to consolidate on terminology that k8s >> and cncf have used that people find easier to understand so it's a way >> in which openstack is not different than those. Consolidating on terms >> isn't a bad thing, but it's really a minor part of the workflow issue. > > It's both a consolidation and the signal of a change. If we continued to > call them "workgroups" I suspect we'd carry some of the traditions > around them (or would end up calling them new-style WG vs. old-style WG). I still think I've missed, or not grasped, during this thread how a SIG functions differently than a WG, besides name. Both in theory and practice. The API WG doesn't seem like a great example, because it was honestly a couple of people that were interested in API consumption, but mostly had a historical view of how the API worked in OpenStack. The transition in from developers was largely because some reality checking needed to be put in place, and then people changed roles / jobs, and those showing up stayed on the dev side. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Turning TC/UC workgroups into OpenStack SIGs
On 06/21/2017 01:10 PM, Michał Jastrzębski wrote: > One of key components which, imho, made SIGs successful in k8s is > infrastructure behind it. > > When someone proposes an issue, they can tag SIG to it. Everyone in > this SIG will be notified that there is an issue they might be > interested it, they check it out and provide feedback. That also > creates additional familiarity with dev toolset for non-dev sig > members. I think what would be important for OpenStack SIGs to be > successful is connecting SIGs to both Launchpad and Gerrit. I think this is a key point. The simpler tools that github has, which require that you build a workflow based on tags outside of the tools, actually enables the effectiveness here. Does k8s community currently have the same level of operators that aren't developers participating as OpenStack? I wonder if we're going down this path, if some kind of tooling like standard tags for issues/patches should be added to the mix to help gain the effectiveness that the k8s team seems to have here. I also think it's fine to rebrand WG to SIG, but we should also be honest that it's mostly a rebrand to consolidate on terminology that k8s and cncf have used that people find easier to understand so it's a way in which openstack is not different than those. Consolidating on terms isn't a bad thing, but it's really a minor part of the workflow issue. It might also be a good idea that any SIG that is going to be "official" has the requirement that they write up a state of the sig every month or two with what's done, what's happening, what's next, and what's challenging. At a project the scale of OpenStack one of the biggest issues is actually having a good grasp on the wide range of efforts, and these summaries by teams are pretty critical to increasing the shared understanding. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all][ptl] Most Supported Queens Goals and Improving Goal Completion
On 06/27/2017 04:50 AM, Thierry Carrez wrote: > gordon chung wrote: >> do we know why they're not being completed? indifference? lack of resources? > > I would say it's a mix of reasons. Sometimes it's a resource issue, but > most of the time it's a prioritization issue (everyone waiting for > someone else to pick it up), and in remaining cases it's pure > procrastination (it's not that much work, I'll do it tomorrow). > >> i like the champion idea although i think its scope should be expanded. >> i didn't mention this in meeting and the following has no legit research >> behind it so feel free to disregard but i imagine some of the >> indifference towards the goals is because: >> >> - it's often trivial (but important) work >> many projects are already flooded with a lot of non-trivial, >> self-interest goals AND a lot trivial (and unimportant) copy/paste >> patches already so it's hard to feel passionate and find motivation to >> do it. the champion stuff may help here. >> >> - there is a disconnect between the TC and the projects. >> it seems there is a requirement for the projects to engage the TC but >> not necessarily the other way around. for many projects, i'm fairly >> certain nothing would change whether they actively engaged the TC or >> just left relationship as is and had minimal/no interaction. i apologise >> if that's blunt but just based on my own prior experience. >> >> i don't know if the TC wants to become PMs but having the goals i feel >> sort of requires the TC to be PMs and to actually interact with the PTLs >> regularly, not just about the goal itself but the project and it's role >> in openstack. maybe it's as designed, but if there's no relationship >> there, i don't think 'TC wants you to do this' will get something done. >> it's in the same vein as how it's easier to get a patch approved if >> you're engaged in a project for some time as oppose to a patch out of >> the blue (disclaimer: i did not study sociology). > When we look at goals, the main issue is generally not writing the > patches, it's more about getting that prioritized in code review and > tracking completion. That's where I think champions will help. Sometimes > teams will need help writing patches, sometimes they will just need > reminders to prioritize code review up. Someone has to look at the big > picture and care for the completion of the goal. Having champions will > also make it look a lot less like 'TC wants you to do this' and more > like 'we are in this together, completing this goal will make openstack > better'. ++ Having worked on a number of things that have touched a bunch of projects, it turns out that the needs of every project are different. The reason that multi project efforts seem to take so long, or die out, is they need a reasonable amount of project management to be effective. There are lots of demands on teams, and having someone that can represent a bigger goal, knows what it looks like when complete, and can go to the affected teams with "here is the next one thing I need from you to make this whole" really speeds up the process. At least 2 - 3x (if not more). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] bug triage experimentation
On 06/26/2017 04:49 AM, Sylvain Bauza wrote: > > > Le 23/06/2017 18:52, Sean Dague a écrit : >> The Nova bug backlog is just over 800 open bugs, which while >> historically not terrible, remains too large to be collectively usable >> to figure out where things stand. We've had a few recent issues where we >> just happened to discover upgrade bugs filed 4 months ago that needed >> fixes and backports. >> >> Historically we've tried to just solve the bug backlog with volunteers. >> We've had many a brave person dive into here, and burn out after 4 - 6 >> months. And we're currently without a bug lead. Having done a big giant >> purge in the past >> (http://lists.openstack.org/pipermail/openstack-dev/2014-September/046517.html) >> I know how daunting this all can be. >> >> I don't think that people can currently solve the bug triage problem at >> the current workload that it creates. We've got to reduce the smart >> human part of that workload. >> > > Thanks for sharing ideas, Sean. > >> But, I think that we can also learn some lessons from what active github >> projects do. >> >> #1 Bot away bad states >> >> There are known bad states of bugs - In Progress with no open patch, >> Assigned but not In Progress. We can just bot these away with scripts. >> Even better would be to react immediately on bugs like those, that helps >> to train folks how to use our workflow. I've got some starter scripts >> for this up at - https://github.com/sdague/nova-bug-tools >> > > Sometimes, I had no idea why but I noticed the Gerrit hook not working > (ie. amending the Launchpad bug with the Gerrit URL) so some of the bugs > I was looking for were actively being worked on (and I had the same > experience myself although my commit msg was pretty correctly marked AFAIR). > > Either way, what you propose sounds reasonable to me. If you care about > fixing a bug by putting yourself owner of that bug, that also means you > engage yourself on a resolution sooner than later (even if I do fail > applying that to myself...). > >> #2 Use tag based workflow >> >> One lesson from github projects, is the github tracker has no workflow. >> Issues are openned or closed. Workflow has to be invented by every team >> based on a set of tags. Sometimes that's annoying, but often times it's >> super handy, because it allows the tracker to change workflows and not >> try to change the meaning of things like "Confirmed vs. Triaged" in your >> mind. >> >> We can probably tag for information we know we need at lot easier. I'm >> considering something like >> >> * needs.system-version >> * needs.openstack-version >> * needs.logs >> * needs.subteam-feedback >> * has.system-version >> * has.openstack-version >> * has.reproduce >> >> Some of these a bot can process the text on and tell if that info was >> provided, and comment how to provide the updated info. Some of this >> would be human, but with official tags, it would probably help. >> > > The tags you propose seem to me related to an "Incomplete" vs. > "Confirmed" state of the bug. > > If I'm not able to triage the bug because I'm missing information like > the release version or more logs, I put the bug as Incomplete. > I could add those tags, but I don't see where a programmatical approach > could help us. We always want that information, and the odds of us getting it from a user decline over time. When we end up looking at bugs that are year old, it becomes a big guessing game on their relevancy. The theory here is that tags like that would be applied by a bot immediately after the bug is filed. Catching the owner within 5 minutes of their bug filing with a response which is "we need the following" means we should get a pretty decent attach rate on that information. And then you don't have to spend 10 minutes of real human time realizing that you really need that before moving forward. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] bug triage experimentation
The Nova bug backlog is just over 800 open bugs, which while historically not terrible, remains too large to be collectively usable to figure out where things stand. We've had a few recent issues where we just happened to discover upgrade bugs filed 4 months ago that needed fixes and backports. Historically we've tried to just solve the bug backlog with volunteers. We've had many a brave person dive into here, and burn out after 4 - 6 months. And we're currently without a bug lead. Having done a big giant purge in the past (http://lists.openstack.org/pipermail/openstack-dev/2014-September/046517.html) I know how daunting this all can be. I don't think that people can currently solve the bug triage problem at the current workload that it creates. We've got to reduce the smart human part of that workload. But, I think that we can also learn some lessons from what active github projects do. #1 Bot away bad states There are known bad states of bugs - In Progress with no open patch, Assigned but not In Progress. We can just bot these away with scripts. Even better would be to react immediately on bugs like those, that helps to train folks how to use our workflow. I've got some starter scripts for this up at - https://github.com/sdague/nova-bug-tools #2 Use tag based workflow One lesson from github projects, is the github tracker has no workflow. Issues are openned or closed. Workflow has to be invented by every team based on a set of tags. Sometimes that's annoying, but often times it's super handy, because it allows the tracker to change workflows and not try to change the meaning of things like "Confirmed vs. Triaged" in your mind. We can probably tag for information we know we need at lot easier. I'm considering something like * needs.system-version * needs.openstack-version * needs.logs * needs.subteam-feedback * has.system-version * has.openstack-version * has.reproduce Some of these a bot can process the text on and tell if that info was provided, and comment how to provide the updated info. Some of this would be human, but with official tags, it would probably help. #3 machine assisted functional tagging I'm playing around with some things that might be useful in mapping new bugs into existing functional buckets like: libvirt, volumes, etc. We'll see how useful it ends up being. #4 reporting on smaller slices Build some tooling to report on the status and change over time of bugs under various tags. This will help visualize how we are doing (hopefully) and where the biggest piles of issues are. The intent is the normal unit of interaction would be one of these smaller piles. Be they the 76 libvirt bugs, 61 volumes bugs, or 36 vmware bugs. It would also highlight the rates of change in these piles, and what's getting attention and what is not. This is going to be kind of an ongoing experiment, but as we currently have no one spear heading bug triage, it seemed like a good time to try this out. Comments and other suggestions are welcomed. The tooling will have the nova flow in mind, but I'm trying to make it so it takes a project name as params on all the scripts, so anyone can use it. It's a little hack and slash right now to discover what the right patterns are. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][all][ptl] Most Supported Queens Goals and Improving Goal Completion
On 06/23/2017 05:44 AM, Thierry Carrez wrote: > Lance Bragstad wrote: >> Is the role of a goal "champion" limited to a single person? Can it be >> distributed across multiple people pending actions are well communicated? > > I'm a bit torn on that. On one hand work can definitely (and probably > should) be covered by multiple people. But I like the idea that it's > someone's responsibility to ensure that progress is made (even if that > person ends up delegating all the work). The trick is, it's easy for > everyone to assume that the work is covered since someone has signed up > for it. > > It's like the PTL situation -- work is done by the group and it's great > to have a clear go-to person to keep track of things, until the PTL has > to do everything because they end up as the default assignee for everything. I agree, there should be a single name here. They can delegate and collect up a group, but at the end of the day one person should be responsible for it. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][placement] Add API Sample tests for Placement APIs
On 06/22/2017 01:22 PM, Matt Riedemann wrote:
> Rocky, we have tests, we just don't have API samples for documentation
> purposes like in the compute API reference docs.
>
> This doesn't have anything to do with interop guidelines, and it
> wouldn't, since the Placement APIs are all admin-only and interop is
> strictly about non-admin APIs.
I think the other important thing to remember is that the consumers of
the placement api are currently presumed to be other OpenStack projects.
Definitely not end users. So the documentation priority of providing
lots of examples so that people don't need to look at source code is not
as high.
I'm firmly in the camp that sample request / response is a good thing,
but from a priorities perspective it's way more important to get that on
end user APIs than quasi internal ones.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/21/2017 05:35 PM, Jeremy Stanley wrote: > On 2017-06-21 13:52:11 -0500 (-0500), Lauren Sell wrote: > [...] >> To make this actionable...Github is just a mirror of our >> repositories, but for better or worse it's the way most people in >> the world explore software. If you look at OpenStack on Github >> now, it’s impossible to tell which projects are official. Maybe we >> could help by better curating the Github projects (pinning some of >> the top projects, using the new new topics feature to put tags >> like openstack-official or openstack-unofficial, coming up with >> more standard descriptions or naming, etc.). > > I hadn't noticed the pinned repositories option until you mentioned > it: appears they just extended that feature to orgs back in October > (and introduced the topics feature in January). I could see > potentially integrating pinning and topic management into the > current GH API script we run when creating new mirrors > there--assuming these are accessible via their API anyway--and yes > normalizing the descriptions to something less freeform is something > else we'd discussed to be able to drive users back to the official > locations for repositories (or perhaps to the project navigator). > > I've already made recent attempts to clarify our use of GH in the > org descriptions and linked the openstack org back to the project > navigator too, since those were easy enough to do right off the bat. It doesn't look like either (pinned repos or topics) are currently available over the API (topics in get format are experimental, but no edit as of yet). The pinned repositories aren't such a big deal, we're talking a handful here. The topics / tags maintenance would be more, but those aren't changing so fast that I think they'd be too unweildly to keep close. As I have strong feelings that this all would help, I would be happy to volunteer to manually update that information. Just need enough access to do that. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/21/2017 09:52 PM, Chris Hoge wrote: > >> On Jun 21, 2017, at 2:35 PM, Jeremy Stanley <fu...@yuggoth.org> wrote: >> >> On 2017-06-21 13:52:11 -0500 (-0500), Lauren Sell wrote: >> [...] >>> To make this actionable...Github is just a mirror of our >>> repositories, but for better or worse it's the way most people in >>> the world explore software. If you look at OpenStack on Github >>> now, it’s impossible to tell which projects are official. Maybe we >>> could help by better curating the Github projects (pinning some of >>> the top projects, using the new new topics feature to put tags >>> like openstack-official or openstack-unofficial, coming up with >>> more standard descriptions or naming, etc.). >> >> I hadn't noticed the pinned repositories option until you mentioned >> it: appears they just extended that feature to orgs back in October >> (and introduced the topics feature in January). I could see >> potentially integrating pinning and topic management into the >> current GH API script we run when creating new mirrors >> there--assuming these are accessible via their API anyway--and yes >> normalizing the descriptions to something less freeform is something >> else we'd discussed to be able to drive users back to the official >> locations for repositories (or perhaps to the project navigator). >> >> I've already made recent attempts to clarify our use of GH in the >> org descriptions and linked the openstack org back to the project >> navigator too, since those were easy enough to do right off the bat. >> >>> Same goes for our repos…if there’s a way we could differentiate >>> between official and unofficial projects on this page it would be >>> really useful: https://git.openstack.org/cgit/openstack/ >> >> I have an idea as to how to go about that by generating custom >> indices rather than relying on the default one cgit provides; I'll >> mull it over. >> >>> 2) Create a simple structure within the official set of projects >>> to provide focus and a place to get started. The challenge (again >>> to our success, and lots of great work by the community) is that >>> even the official project set is too big for most people to >>> follow. >> >> This is one of my biggest concerns as well where high-cost (in the >> sense of increasingly valuable Infra team member time) solutions are >> being tossed around to solve the "what's official?" dilemma, while >> not taking into account that the overwhelming majority of active Git >> repositories we're hosting _are_ already deliverables for official >> teams. I strongly doubt that just labelling the minority as >> unofficial will any any way lessen the overall confusion about the >> *more than one thousand* official Git repositories we're >> maintaining. > > Another instance where the horse is out of the barn, but this > is one of the reasons why I don’t like it when config-management > style efforts are organized as one-to-one mapping of repositories > to corresponding project. It created massive sprawl > within the ecosystem, limited opportunities for code sharing, > and made refactoring a nightmare. I lost count of the number > of times we submitted n inconsistent patches to change > similar behavior across n+1 projects. Trying to build a library > helped but was never as powerful as being able to target a > single repository. ++ The micro repositories for config management and packaging create this overwhelming wall of projects from the outside. I realize that git repos are cheap from a dev perspective, but they are expensive from a concept perspective. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/22/2017 04:33 AM, Thierry Carrez wrote: > Sean Dague wrote: >> [...] >> I think even if it was only solvable on github, and not cgit, it would >> help a lot. The idea of using github project tags and pinning suggested >> by Lauren seems great to me. >> >> If we replicated the pinning on github.com/openstack to "popular >> projects" here - https://www.openstack.org/software/, and then even just >> start with the tags as defined in governance - >> https://governance.openstack.org/tc/reference/tags/index.html it would >> go a long way. >> >> I think where the conversation is breaking down is realizing that >> different people process the information we put out there in different >> ways, and different things lock in and make sense to them. Lots of >> people are trained to perceive github structure as meaningful, because >> it is 98% of the time. As such I'd still also like to see us using that >> structure well, and mirroring only things we tag as official to >> github.com/openstack, and the rest to /openstack-ecosystem or something. >> >> Even if that's flat inside our gerrit and cgit environment. > > I would even question the need for us to mirror the rest. Those are > hosted projects, if they want presence on GitHub they would certainly > welcome the idea of setting up an organization for their project. I > wouldn't be shocked to find a fuel-ccp or a stacktach GitHub org. And if > they don't care about their GitHub presence, then just don't do > anything. I'm not sure why we would make that choice for us. > > Jeremy is right that the GitHub mirroring goes beyond an infrastructure > service: it's a marketing exercise, an online presence more than a > technical need. As such it needs to be curated, and us doing that for > "projects that are not official but merely hosted" is an anti-feature. > No real value for the hosted project, and extra confusion as a result. Good point, I hadn't thought of that. I'd be totally fine only mirroring official projects. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/21/2017 02:52 PM, Lauren Sell wrote: > Two things we should address: > > 1) Make it more clear which projects are “officially” part of > OpenStack. It’s possible to find that information, but it’s not obvious. > I am one of the people who laments the demise of stackforge…it was very > clear that stackforge projects were not official, but part of the > OpenStack ecosystem. I wish it could be resurrected, but I know that’s > impractical. > > To make this actionable...Github is just a mirror of our repositories, > but for better or worse it's the way most people in the world > explore software. If you look at OpenStack on Github now, it’s > impossible to tell which projects are official. Maybe we could help by > better curating the Github projects (pinning some of the top projects, > using the new new topics feature to put tags like openstack-official or > openstack-unofficial, coming up with more standard descriptions or > naming, etc.). Same goes for our repos…if there’s a way we could > differentiate between official and unofficial projects on this page it > would be really useful: https://git.openstack.org/cgit/openstack/ I think even if it was only solvable on github, and not cgit, it would help a lot. The idea of using github project tags and pinning suggested by Lauren seems great to me. If we replicated the pinning on github.com/openstack to "popular projects" here - https://www.openstack.org/software/, and then even just start with the tags as defined in governance - https://governance.openstack.org/tc/reference/tags/index.html it would go a long way. I think where the conversation is breaking down is realizing that different people process the information we put out there in different ways, and different things lock in and make sense to them. Lots of people are trained to perceive github structure as meaningful, because it is 98% of the time. As such I'd still also like to see us using that structure well, and mirroring only things we tag as official to github.com/openstack, and the rest to /openstack-ecosystem or something. Even if that's flat inside our gerrit and cgit environment. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][scheduler][placement] Trying to understand the proposed direction
On 06/21/2017 04:43 AM, sfinu...@redhat.com wrote: > On Tue, 2017-06-20 at 16:48 -0600, Chris Friesen wrote: >> On 06/20/2017 09:51 AM, Eric Fried wrote: >>> Nice Stephen! >>> >>> For those who aren't aware, the rendered version (pretty, so pretty) can >>> be accessed via the gate-nova-docs-ubuntu-xenial jenkins job: >>> >>> http://docs-draft.openstack.org/10/475810/1/check/gate-nova-docs-ubuntu-xen >>> ial/25e5173//doc/build/html/scheduling.html?highlight=scheduling >> >> Can we teach it to not put line breaks in the middle of words in the text >> boxes? > > Doesn't seem configurable in its current form :( This, and the defaulting to > PNG output instead of SVG (which makes things ungreppable) are my biggest bug > bear. > > I'll go have a look at the sauce and see what can be done about it. If not, > still better than nothing? I've actually looked through the blockdiag source (to try to solve a similar problem). There is no easy way to change it. If people find it confusing, the best thing to do would be short labels on boxes, then explain in more detail in footnotes. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][qa][glance] some recent tempest problems
On 06/16/2017 10:46 AM, Eric Harney wrote: > On 06/16/2017 10:21 AM, Sean McGinnis wrote: >> >> I don't think merging tests that are showing failures, then blacklisting >> them, is the right approach. And as Eric points out, this isn't >> necessarily just a failure with Ceph. There is a legitimate logical >> issue with what this particular test is doing. >> >> But in general, to get back to some of the earlier points, I don't think >> we should be merging tests with known breakages until those breakages >> can be first addressed. >> > > As another example, this was the last round of this, in May: > > https://review.openstack.org/#/c/332670/ > > which is a new tempest test for a Cinder API that is not supported by > all drivers. The Ceph job failed on the tempest patch, correctly, the > test was merged, then the Ceph jobs broke: > > https://bugs.launchpad.net/glance/+bug/1687538 > https://review.openstack.org/#/c/461625/ > > This is really not a sustainable model. > > And this is the _easy_ case, since Ceph jobs run in OpenStack infra and > are easily visible and trackable. I'm not sure what the impact is on > Cinder third-party CI for other drivers. Ah, so the issue is that gate-tempest-dsvm-full-ceph-plugin-src-glance_store-ubuntu-xenial is Voting, because when the regex was made to stop ceph jobs from voting (which they aren't on Nova, Tempest, Glance, or Cinder), it wasn't applied there. It's also a question about why a library is doing different back end testing through full stack testing, instead of more targeted and controlled behavior. Which I think is probably also less than ideal. Both would be good things to fix. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][qa][glance] some recent tempest problems
On 06/16/2017 09:51 AM, Sean McGinnis wrote: >> >> It would be useful to provide detailed examples. Everything is trade >> offs, and having the conversation in the abstract is very difficult to >> understand those trade offs. >> >> -Sean >> > > We've had this issue in Cinder and os-brick. Usually around Ceph, but if > you follow the user survey, that's the most popular backend. > > The problem we see is the tempest test that covers this is non-voting. > And there have been several cases so far where this non-voting job does > not pass, due to a legitimate failure, but the tempest patch merges anyway. > > > To be fair, these failures usually do point out actual problems that need > to be fixed. Not always, but at least in a few cases. But instead of it > being addressed first to make sure there is no disruption, it's suddenly > a blocking issue that holds up everything until it's either reverted, skipped, > or the problem is resolved. > > Here's one recent instance: https://review.openstack.org/#/c/471352/ So, before we go further, ceph seems to be -nv on all projects right now, right? So I get there is some debate on that patch, but is it blocking anything? Again, we seem to be missing specifics and a set of events here, which lacking that everyone is trying to guess what the problems are, which I don't think is effective. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][qa][glance] some recent tempest problems
On 06/16/2017 09:51 AM, Sean McGinnis wrote: >> >> It would be useful to provide detailed examples. Everything is trade >> offs, and having the conversation in the abstract is very difficult to >> understand those trade offs. >> >> -Sean >> > > We've had this issue in Cinder and os-brick. Usually around Ceph, but if > you follow the user survey, that's the most popular backend. > > The problem we see is the tempest test that covers this is non-voting. > And there have been several cases so far where this non-voting job does > not pass, due to a legitimate failure, but the tempest patch merges anyway. > > > To be fair, these failures usually do point out actual problems that need > to be fixed. Not always, but at least in a few cases. But instead of it > being addressed first to make sure there is no disruption, it's suddenly > a blocking issue that holds up everything until it's either reverted, skipped, > or the problem is resolved. > > Here's one recent instance: https://review.openstack.org/#/c/471352/ Sure, if ceph is the primary concern, that feels like it should be a reasonable specific thing to fix. It's not a grand issue, it's a specific mismatch on what configs should be common. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] etcd v3.2.0?
On 06/15/2017 10:06 PM, Tony Breeds wrote:
> Hi All,
> I just push a review [1] to bump the minimum etcd version to
> 3.2.0 which works on intel and ppc64le. I know we're pretty late in the
> cycle to be making changes like this but releasing pike with a dependacy
> on 3.1.x make it harder for users on ppc64le (not many but a few :D)
>
> Yours Tony.
>
> [1] https://review.openstack.org/474825
It should be fine, no one is really using these much at this point.
However it looks like mirroring is not happening automatically? The
patch fails on not existing in the infra mirror.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Strict validation in query parameters
On 06/15/2017 10:01 PM, Matt Riedemann wrote: > On 6/15/2017 8:43 PM, Alex Xu wrote: >> We added new decorator 'query_schema' to support validate the query >> parameters by JSON-Schema. >> >> It provides more strict valiadation as below: >> * set the 'additionalProperties=False' in the schema, it means that >> reject any invalid query parameters and return HTTPBadRequest 400 to >> the user. >> * use the marco function 'single_param' to declare the specific query >> parameter only support single value. For example, the 'marker' >> parameters for the pagination actually only one value is the valid. If >> the user specific multiple values "marker=1=2", the validation >> will return 400 to the user. >> >> Currently there is patch related to this: >> https://review.openstack.org/#/c/459483/13/nova/api/openstack/compute/schemas/server_migrations.py >> >> >> So my question is: >> Are we all good with this strict validation in all the future >> microversion? >> >> I didn't remember we explicit agreement this at somewhere, just want >> to double check this is the direction everybody want to go. >> >> Thanks >> Alex >> >> >> __ >> >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > I think this is fine and makes sense for new microversions. The spec for > consistent query parameter validation does talk about it a bit: > > https://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/consistent-query-parameters-validation.html#proposed-change > > > "The behaviour additionalProperties as below: > > * When the value of additionalProperties is True means the extra query > parameters are allowed. But those extra query parameters will be > stripped out. > * When the value of additionalProperties is False means the extra query > aren’t allowed. > > The value of additionalProperties will be True until we decide to > restrict the parameters in the future, and it will be changed with new > microversion." > > I don't see a point in allowing someone to specify a query parameter > multiple times if we only pick the first one from the list and use that. Agreed. The point of doing strict validation and returning a 400 is to help the user eliminate bugs in their program. If they specified marker twice either they thought it did something, or they made a mistake. Both are wrong. When we are silent on that front it means they may not be getting the behavior they were expecting, which hurts their experience with the API. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][qa][glance] some recent tempest problems
On 06/15/2017 01:04 PM, Brian Rosmaita wrote: > This isn't a glance-specific problem though we've encountered it quite > a few times recently. > > Briefly, we're gating on Tempest jobs that tempest itself does not > gate on. This leads to a situation where new tests can be merged in > tempest, but wind up breaking our gate. We aren't claiming that the > added tests are bad or don't provide value; the problem is that we > have to drop everything and fix the gate. This interrupts our current > work and forces us to prioritize bugs to fix based not on what makes > the most sense for the project given current priorities and resources, > but based on whatever we can do to get the gates un-blocked. > > As we said earlier, this situation seems to be impacting multiple projects. > > One solution for this is to change our gating so that we do not run > any Tempest jobs against Glance repositories that are not also gated > by Tempest. That would in theory open a regression path, which is why > we haven't put up a patch yet. Another way this could be addressed is > by the Tempest team changing the non-voting jobs causing this > situation into voting jobs, which would prevent such changes from > being merged in the first place. The key issue here is that we need > to be able to prioritize bugs based on what's most important to each > project. > > We want to be clear that we appreciate the work the Tempest team does. > We abhor bugs and want to squash them too. The problem is just that > we're stretched pretty thin with resources right now, and being forced > to prioritize bug fixes that will get our gate un-blocked is > interfering with our ability to work on issues that may have a higher > impact on end users. > > The point of this email is to find out whether anyone has a better > suggestion for how to handle this situation. It would be useful to provide detailed examples. Everything is trade offs, and having the conversation in the abstract is very difficult to understand those trade offs. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/15/2017 10:57 AM, Thierry Carrez wrote: > Jeremy Stanley wrote: >> On 2017-06-15 11:15:36 +0200 (+0200), Thierry Carrez wrote: >> [...] >>> I'd like to propose that we introduce a new concept: "OpenStack-Hosted >>> projects". There would be "OpenStack projects" on one side, and >>> "Projects hosted on OpenStack infrastructure" on the other side (all >>> still under the openstack/ git repo prefix). >> >> I'm still unconvinced a term is needed for this. Can't we just have >> "OpenStack Projects" (those under TC governance) and "everything >> else?" Why must the existence of any term require a term for its >> opposite? > > Well, we tried that for 2.5 years now, and people are still confused > about which projects are an Openstack project and what are not. The > confusion led to the perception that everything under openstack/ is an > openstack project. It led to the perception that "big tent" means > "anything goes in" or "flea market". > > Whether we like it or not, giving a name to that category, a name that > people can refer to (not "projects under openstack infrastructure that > are not officially recognized by the TC"), is I think the only way out > of this confusion. > > Obviously we are not the target audience for that term. I think we are > deep enough in OpenStack and technically-focused enough to see through > that. But reality is, the majority of the rest of the world is confused, > and needs help figuring it out. Giving the category a name is a way to > do that. Right, I think the point is that we both need to have an internal understanding, as well as be able to effectively communicate the state of things with people that aren't reading yaml out of git trees. The human tendency to need a category for things (especially things they aren't 100% familiar with the inner workings of) is way too strong to ignore. And the fact that namespaces in github mean something to the rest of the world, even if they don't mean it to us, as that's a first order piece of data the way other projects organize. I do kind of wonder if we returned the stackforge or friends-of-openstack or whatever to the github namespace when we mirrored if it would clear a bunch of things up for people. It would just need to be an extra piece of info in our project list about where those projects should mirror to (which may not be the same namespace as in gerrit). -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Moving away from "big tent" terminology
On 06/15/2017 05:15 AM, Thierry Carrez wrote: > Hi everyone, > > Back in 2014, OpenStack was facing a problem. Our project structure, > inherited from days where Nova, Swift and friends were the only game in > town, was not working anymore. The "integrated release" that we ended up > producing was not really integrated, already too big to be installed by > everyone, and yet too small to accommodate the growing interest in other > forms of "open infrastructure". The incubation process (from stackforge > to incubated, from incubated to integrated) created catch-22s that > prevented projects from gathering enough interest to reach the upper > layers. Something had to give. > > The project structure reform[1] that resulted from those discussions > switched to a simpler model: project teams would be approved based on > how well they fit the OpenStack overall mission and community > principles, rather than based on a degree of maturity. It was nicknamed > "the big tent" based on a blogpost[2] that Monty wrote -- mostly > explaining that things produced by the OpenStack community should be > considered OpenStack projects. > > So the reform removed the concept of incubated vs. integrated, in favor > of a single "official" category. Tags[3] were introduced to better > describe the degree of maturity of the various official things. "Being > part of the big tent" was synonymous to "being an official project" (but > people kept saying the former). > > At around the same time, mostly for technical reasons around the > difficulty of renaming git repositories, the "stackforge/" git > repository prefix was discontinued (all projects hosted on OpenStack > infrastructure would be created under an "openstack/" git repository > prefix). > > All those events combined, though, sent a mixed message, which we are > still struggling with today. "Big tent" has a flea market connotation of > "everyone can come in". Combined with the fact that all git repositories > are under the same prefix, it created a lot of confusion. Some people > even think the big tent is the openstack/ namespace, not the list of > official projects. We tried to stop using the "big tent" meme, but (I > blame Monty), the name is still sticking. I think it's time to more > aggressively get rid of it. We tried using "unofficial" and "official" > terminology, but that did not stick either. > > I'd like to propose that we introduce a new concept: "OpenStack-Hosted > projects". There would be "OpenStack projects" on one side, and > "Projects hosted on OpenStack infrastructure" on the other side (all > still under the openstack/ git repo prefix). We'll stop saying "official > OpenStack project" and "unofficial OpenStack project". The only > "OpenStack projects" will be the official ones. We'll chase down the > last mentions of "big tent" in documentation and remove it from our > vocabulary. > > I think this new wording (replacing what was previously Stackforge, > replacing what was previously called "unofficial OpenStack projects") > will bring some clarity as to what is OpenStack and what is beyond it. I think those are all fine. The other term that popped into my head was "Friends of OpenStack" as a way to describe the openstack-hosted efforts that aren't official projects. It may be too informal, but I do think the OpenStack-Hosted vs. OpenStack might still mix up in people's head. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] os-api-ref 1.4.0 about to hit upper-constraints
There were some changes in Sphinx 1.6.x that removed functions that os-api-ref was using to warn for validation. Which meant that when things failed instead of getting the warning you got a huge cryptic stack trace. :( Those are fixed in 1.4.0, however with the other changes in Sphinx about how warnings work, I'm not 100% confident this is going to be smooth for everyone. If you hit an issue in your api-ref building after this lands, please pop up on #openstack-dev and we'll try to work through it. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc][glance] Glance needs help, it's getting critical
On 06/12/2017 03:20 PM, Flavio Percoco wrote: > Could you please elaborate more on why you think switching code bases is > going > to solve the current problem? In your email you talked about Glance's > over-engineered code as being the thing driving people away and while I > disagree > with that statement, I'm wondering whether you really think that's the > motivation or there's something else. > > Let's not talk about proxy API's or ways we would migrate users. I'd > like to > understand why *you* (or others) might think that a complete change of > projects > is a good solution to this problem. > > Ultimatedly, I believe Glance, in addition to not being the "sexiest" > project in > OpenStack, is taking the hit of the recent lay-offs, which it kinda > managed to > avoid last year. As someone from the outside the glance team, I'd really like to avoid the artifacts path. I feel like 2 years ago there was a promise that if glance headed in that direction it would bring in new people, and everything would be great. But, it didn't bring in folks solving the class of issues that current glance users are having. 80+ GB disk images could be classified as a special case of Artifacts, but it turns optimizing for their specialness is really important to a well functioning cloud. Glance might not be the most exciting project, but what seems to be asked for is help on the existing stuff. I'd rather focus there. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc][glance] Glance needs help, it's getting critical
On 06/09/2017 01:07 PM, Flavio Percoco wrote: > (sorry if duplicate, having troubles with email) > > Hi Team, > > I've been working a bit with the Glance team and trying to help where I > can and > I can't but be worried about the critical status of the Glance team. > Unfortunately, the number of participants in the Glance team has been > reduced a > lot resulting in the project not being able to keep up with the goals, the > reviews required, etc.[0] > > I've always said that Glance is one of those critical projects that not many > people notice until it breaks. It's in every OpenStack cloud sitting in > a corner > and allowing for VMs to be booted. So, before things get even worse, I'd > like us to brainstorm a bit on what solutions/options we have now. > > I know Glance is not the only project "suffering" from lack of > contributors but > I don't want us to get to the point where there won't be contributors left. > > How do people feel about adding Glance to the list of "help wanted" areas of > interest? > > Would it be possible to get help w/ reviews from folks from teams like > nova/cinder/keystone? Any help is welcomed, of course, but I'm trying to > think > about teams that may be familiar with the Glance code/api already. I'm happy to help here, I just went through and poked at a few things. It is going to be tough to make meaningful contributions there without approve authority, especially given the normal trust building exercise for core teams takes 3+ months. It might be useful to figure out if there are a set of folks already in the community that the existing core team would be happy to provisionally promote to help worth the current patch backlog and get things flowing. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][requirements][all] removing global pins for linters
On 06/11/2017 11:30 AM, Doug Hellmann wrote: > The recent thread about updating the pylint version in > global-requirements.txt raised an issue because it was trying to > update the pylint version for all projects using it, but some teams > were not ready for the new tests in the latest version. I thought > we had dealt with that kind of case in the past by treating linter > projects like pylint and flake8 as special cases, and leaving them > out of the global requirements list. The requirements repo has a > separate file (blacklist.txt) for projects that should not be synced > into repositories and tested against the global-requirements.txt > list, and pylint is included there along with several other linter > tools. > > I'm not sure why the linters were also being added to > global-requirements.txt, but it seems like a mistake. I have proposed > a patch [2] to remove them, which should allow projects that want > to update pylint to do so while not forcing everyone to update at > the same time. If we find issues with the requirements sync after > removing the entries from the global list, we should fix the syncing > scripts so we can keep the linters blacklisted. > > Doug > > [1] http://lists.openstack.org/pipermail/openstack-dev/2017-June/118085.html > [2] https://review.openstack.org/473094 Are you sure that works as expected? My understanding is that the requirements enforcement jobs only let you set requirements lines to what are in that file. So that effectively prevents anyone from changing the linters lines ever (see http://logs.openstack.org/69/473369/1/check/gate-nova-requirements/b425844/console.html) -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Is Routes==2.3.1 a binary only package or something?
On 06/12/2017 04:29 AM, Michael Still wrote: > Hi, > > I'm trying to explain this behaviour in stable/newton, which specifies > Routes==2.3.1 in upper-constraints: > > $ pip install --no-binary :all: Routes==2.3.1 > ... > Could not find a version that satisfies the requirement Routes==2.3.1 > (from versions: 1.5, 1.5.1, 1.5.2, 1.6, 1.6.1, 1.6.2, 1.6.3, 1.7, 1.7.1, > 1.7.2, 1.7.3, 1.8, 1.9, 1.9.1, 1.9.2, 1.10, 1.10.1, 1.10.2, 1.10.3, > 1.11, 1.12, 1.12.1, 1.12.3, 1.13, 2.0, 2.1, 2.2, 2.3, 2.4.1) > Cleaning up... > No matching distribution found for Routes==2.3.1 > > There is definitely a 2.3.1 on pip: > > $ pip install Routes==2.3.1 > ... > Successfully installed Routes-2.3.1 repoze.lru-0.6 six-1.10.0 > > This implies to me that perhaps Routes version 2.3.1 is a binary-only > release and that stable/newton is therefore broken for people who don't > like binary packages (in my case because they're building an install > image for an architecture which doesn't match their host architecture). > > Am I confused? I'd love to be enlightened. Routes 2.3.1 appears to be any arch wheel. Is there a specific reason that's not going to work for you? (e.g. Routes-2.3.1-py2.py3-none-any.whl) -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Call for check: is your project ready for pylint 1.7.1?
On 06/08/2017 02:53 PM, Akihiro Motoki wrote: > Hi all, > > Is your project ready for pylint 1.7.1? > If you use pylint in your pep8 job, it is worth checked. > > Our current version of pylint is 1.4.5 but it is not safe in python 3.5. > The global-requirements update was merged once [1], > However, some projects (at least neutron) are not ready for pylint > 1.7.1 and it was reverted [2]. > it is reasonable to give individual projects time to cope with pylint 1.7.1. > > I believe bumping pylint version to 1.7.1 (or later) is the right > direction in long term. > I would suggest to make your project ready for pylint 1.7.1 soon (two > weeks or some?) > You can disable new rules in pylint 1.7.1 temporarily and clean up > your code later > as neutron does [3]. As far as I checked, most rules are reasonable > and worth enabled. > > Thanks, > Akihiro Motoki > > [1] https://review.openstack.org/#/c/470800/ > [2] https://review.openstack.org/#/c/471756/ > [3] https://review.openstack.org/#/c/471763/ Please only make changes like this in the first milestone of the cycle. Lint requirements changes are distracting, and definitely shouldn't be happening during the final milestone. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [glance] [cinder] [neutron] [ironic] - Global Request ID progress - update 6/9
A bunch more work landed this week, here is where we stand: STATUS oslo.context / oslo.middleware - DONE devstack logging additional global_request_id - DONE cinder: DONE - client supports global_request_id - DONE - call Nova & Glance with global_request_id - DONE neutron: BLOCKED - client supports global_request_id - DONE - neutron calls Nova with global_request_id - BLOCKED (see below) nova: DONE - Convert to oslo.middleware (to accept global_request_id) - DONE - client supports global_request_id - DONE - call Neutron / Cinder / Glance with global_request_id - DONE glance: BLOCKED - client supports global_request_id - DONE - Glance supports setting global_request_id - BLOCKED (see below) ironic (NEW): in progress - Ironic supports accepting global_request_id - IN REVIEW BLOCKED ITEMS Neutron: There is a mailing list post out here http://lists.openstack.org/pipermail/openstack-dev/2017-June/118031.html. The neutron code for interactions back to Nova is wildly different than the patterns in other services, so I'm actually stumped on the right path forward. Some questions are there. Any neutron experts that could advise or help dive in would be appreciated. Glance: The review that would set the global_request_id in the context is blocked - https://review.openstack.org/#/c/468443/ over different perspectives on API change here. There are only 2 of us in this review so far, so it would be good to get more perspectives from folks as well. STRETCH GOALS Ironic: My original intent was to get through Nova, Neutron, Glance, Cinder this cycle. As that is nearly done, I thought that the next logical service to loop in would be Ironic. There is an initial patch there to add the global_request_id inbound - https://review.openstack.org/#/c/472258/. Ironic reviews to get that into shape for merge would be appreciated. Comments / questions welcomed. As well as anyone that's interested in expanding this support to additional services. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] plumbing in request-id passing back to nova
We've now got Nova -> Glance, Neutron, Cinder. Cinder -> Nova, Glance is
up for review. I started looking at the Neutron code for this, and it's
wildly different, so need some help on what the right way forward is.
It appears that the novaclient construction happens only once per run
inside this Notifier construction -
https://github.com/openstack/neutron/blob/8d9fcb2d3037004cd1ad5136c449d80cdc5a5865/neutron/notifiers/nova.py#L47-L77
(where there is no context available).
Also, it appears that the way these API calls are emitted is through
implicit binds in the DBPlugin
(https://github.com/openstack/neutron/blob/8d9fcb2d3037004cd1ad5136c449d80cdc5a5865/neutron/db/db_base_plugin_v2.py#L152-L166)
so they are happening potentially well outside of any active context.
So... the questions, in order:
1) is construction path so disconnect from request processing that we've
got to give up on that pattern (my guess is yes).
2) when these events are emitted is there some way to have access to a
the context explicitly or do we have to do the magic reach back into the
tls for it?
3) is there any chance in the case of Nova -> Neutron -> Nova that we're
going to be able to keep track of the global_request_id coming from Nova
originally, so that the admin events coming back to Nova are tagged with
the original Nova initiating request?
Any advise here, or help in understanding is very welcomed. Thanks in
advance.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] [glance] [cinder] [neutron] - Global Request ID progress
Some good progress has been made so far on Global Request ID work in the core IaaS layer, here is where we stand. STATUS oslo.context / oslo.middleware - everything DONE devstack logging additional global_request_id - DONE cinder: - client supports global_request_id - DONE - Cinder calls Nova with global_request_id - TODO (waiting on Novaclient release) - Cinder calls Glance with global_request_id - TODO neutron: - client supports global_request_id - IN PROGRESS (this landed, released, but the neutron client release had to be blocked for unrelated issues). - Neutron calls Nova with global_request_id - TODO (waiting on Novaclient release) nova: - Convert to oslo.middleware (to accept global_request_id) - DONE - client supports global_request_id - IN PROGRESS (waiting for release here - https://review.openstack.org/#/c/471323/) - Nova calls cinder with global_request_id - DONE - Nova calls neutron with global_request_id - TODO (waiting on working neutronclient release) - Nova calls Glance with global request id - IN PROGRESS (review needs final +2 here https://review.openstack.org/#/c/467242/) glance: - client supports global_request_id - DONE - Glance supports setting global_request_id - IN REVIEW (https://review.openstack.org/#/c/468443/) *(some debate on this). Everything except the last glance change is uncontroversial, and it's just mechanics and project management to get things through in the correct order. The Glance support for global_request_id has hit a bump in the review process as there is a concern that it's changing the API. Though from an end user perspective that's not happening, it's just changing which field things get logged into. We'll see if we can work through that. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance] request_id middleware in glance
I had not realized this until doing the research at the Summit about request_id middleware in the base iaas services, that glance did something quite different than the others (and has always allowed the request_id to be set). I'd actually like to modify that behavior to setting the new global_request_id variable in oslo.context instead - https://review.openstack.org/#/c/468443/ which gives you 2 ids that can be tracked, one per inbound request, and one that might be set globally. It's a small change (a larger change to use the oslo.middleware would be a bit more complicated, and while good long term, is beyond scope right now), but I wanted to get an idea how this sits with the glance team. Thanks in advance, -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] [tc] [all] more tempest plugins (was Re: [tc] [all] TC Report 22)
On 06/01/2017 06:09 AM, Chris Dent wrote:
> It's clear from this thread and other conversations that the
> management of tempest plugins is creating a multiplicity of issues
> and confusions:
>
> * Some projects are required to use plugins and some are not. This
> creates classes of projects.
While this is true, there are also reasons for that. We decided to break
up the compute service into distinct parts years ago to help let each
part grow dedicated expertise (images, networking, block storage).
However, there is a ton of coupling here even though these are broken up.
My continued resistance to decomposing the QA side of those projects is
getting that integration testing right, and debugging it is hard,
because there are so many interactions required to have a working server
started. And Nova, Neutron, Cinder are the top three most active
projects in OpenStack. So the rate of change individually is quite high.
Forcing those services out into plugins because of the feeling that
something doesn't look fair on paper is just generating more work to
create spherical elephants, instead of acknowledging the amount of work
the QA team has on it's shoulders, and letting it optimize for a better
experience by OpenStack users. Especially given limited resources.
-Sean
--
Sean Dague
http://dague.net
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
