[OpenStack-Infra] Stepping down from core
Hi everyone, After taking some time to rest, I came to the simple (yet difficult) realization that despite my best efforts, I can't do everything that I'd like to do and so I'll be stepping down from infra-core. I've somehow ended up with too many professional and familial responsibilities on my shoulders at once and I need to take a step back to better understand what it is that I can do without burning myself out. Although this week will be my last at Red Hat, I will remain involved with OpenStack in my next adventure and I plan to help where I can as an individual contributor. As for ARA, I'll continue contributing to the project because it's useful for a lot of people and I think it's fun. Thank you for hosting it ! I've really appreciated having the opportunity to work with you and I hope we get to collaborate again in the future. See you around, David Moreau Simard dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] ARA 1.0 deployment plans
On Tue, Jun 11, 2019 at 7:45 PM James E. Blair wrote: > Given that our direction is to remove logs.o.o from the system along > with all of its proxy services and only serve static files from swift, I > think we will have to continue to use the old version of ARA until the > story with static generation is worked out for 1.0. We're currently experimenting with a proof of concept [1] to generate static reports in 1.x -- you can see what it looks like in the uploaded logs here [2]. In a nutshell, it uses the offline API client to retrieve data and then it's rendered using simple html templates. No added dependencies, no javascript (yet?) and the only css is included from a CDN. The challenge with static reports remains the same, though: if you run 100 tasks against 10 hosts, it's 1000 task results. If we keep results in their own pages to keep the playbook pages lightweight, we end up with 1000+ small files. Inlcuding those task results in the same pages as the playbooks reduces the amount of files to generate but it also results in heavier pages to load in users' browsers. Another option is to have arguments or parameters to omit data from the report which would result in less bloat. For example, there could be a flag to omit (detailed) results for "OK" and "SKIPPED" tasks or there could be a flag to only include failed playbooks in the report. Doing something like this isn't complicated since the API already supports filtering by playbook and task status. I'll think about it a little bit but I'm open to suggestions if anyone has an idea. In the meantime, we've landed an implementation of the sqlite middleware for the API [3]. I know it wouldn't work with Swift (unless we use something like s3ql [4]?) but I think that an API to browse and query any job's results could be useful. [1]: https://review.opendev.org/#/c/670597/ [2]: http://logs.openstack.org/97/670597/7/check/ansible-role-ara-api-fedora-devel/1dcf203/logs/static/ [3]: https://ara.readthedocs.io/en/latest/distributed-sqlite-backend.html [4]: https://github.com/s3ql/s3ql David Moreau Simard dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] ARA 1.0 deployment plans
Thanks for starting this thread Ian. On Tue, Jun 11, 2019 at 4:09 AM Ian Wienand wrote: > It seems ARA 1.0 has moved in some directions we're not handling right > now. Playing with [1] I've got ARA generating and uploading it's > database. Patch looks good to me. Thanks for playing with it :) > Currently, Apache matches an ara-report/ directory on > logs.openstack.org and sent to the ARA wsgi application which serves > the response from the sqlite db in that directory [2]. Correct. > If I'm understanding, we now need ara-web [3] to display the report > page we all enjoy. However this web app currently only gets data from > an ARA server instance that provides a REST interface with the info? Correct. > I'm not really seeing how this fits with the current middleware > deployment? (unfortunately [4] or an analogue in the new release seems > to have disappeared). Do we now host a separate ARA server on > logs.openstack.org on some known port that knows how to turn > /*/ara-report/ URL requests into access of the .sqlite db on disk and > thus provide the REST interface? The sqlite middleware doesn't have an equivalent in 1.0 right now. Although it was first implemented as somewhat of a hack to address the lack of scalability of HTML generation, I've gotten to like the design principle of isolating a job's result in a single database. It easy to scale and keeps latency to a minimum compared to a central database server. I'm convinced that implementing a similar approach for 1.0 would make sense. I would happily accept any input here. > And then somehow we host a ara-web instance that knows how to > request from this ? Right now the API server is defined by the configuration [1]. We would need to implement a more "dynamic" way of being able to specify an API endpoint to query. For example, we recently added support for ara-web to prompt a login page if the API server requires authentication. The credentials are stored locally and then used to authenticate queries against the API. It might not be too much of a stretch to implement a way to store the API endpoint locally like we do for credentials. We wouldn't want the user(s) to type in the API endpoint every time, though. There is a little thinking to do about how to glue the things together. The combination of the rewrite rule, the wsgi middleware and the fact that 0.x was a monolithic app definitely made this easier. > Given I can't see us wanting to do a bunch of puppet hacking to get > new services on logs.openstack.org, but yet also it requiring fairly > non-trivial effort to get the extant bits and pieces on that server > migrated to an all-Ansible environment, I think we have to give some > thought as to how we'll roll this out (plus add in containers, > possible logs on swift, etc ... for extra complexity :) > > So does anyone have thoughts on a high-level view of how this might > hang together? For now I've created an etherpad [2] as a starting point to summarize some of what has been written here as well as other points which are perhaps more Zuul-specific. [1]: https://github.com/ansible-community/ara-web/blob/master/public/config.json [2]: https://etherpad.openstack.org/p/ara-1.0-in-zuul David Moreau Simard dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Move packstack from openstack to redhat-openstack organization in github
Hi, github.com/openstack/packstack was moved successfully to github.com/redhat-openstack/packstack. Note that the replication needs to be set up to keep the repository up to date. This can be done by inheriting from the upload-git-mirror [1] job. [1]: http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005007.html David Moreau Simard dmsimard = [irc, github, twitter] On Wed, May 1, 2019 at 1:17 PM Clark Boylan wrote: > > On Tue, Apr 30, 2019, at 7:51 AM, Alfredo Moralejo Alonso wrote: > > Hi, > > > > After migration to opendev an moving openstack/packstack to > > x/packstack, mirroring to github has been lost as announced in [1]. > > > > Could we get the repository moved to redhat-openstack organization in > > github?, we'd like to get redirection from old repo to the new one > > what's require to transfer the repo using github iiuc. > > > > Best regards, > > > > Alfredo > > > > [1] > > http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005629.html > > > > Thank you for sending this out so that we have a record of why the move was > made. I have no objections to making this move in GitHub. If dmsimard still > has time to do the move I think we can go ahead with that. Otherwise, we'll > likely get to it once we return to our daily lives after the summit and PTG. > > Clark > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[openstack-dev] [all] Update to flake8 and failures despite # flake8: noqa
Hi openstack-dev, I stumbled on odd and sudden pep8 failures with ARA recently and brought it up in #openstack-infra [1]. It was my understanding that appending " # flake8: noqa" to a line of code would have flake8 ignore this line if it happened to violate any linting rules. It turns out that, at least according to the flake8 release notes [2], "flake8: noqa" is actually meant to ignore the linting on an entire file. The correct way to ignore a specific line appears to be to append " # noqa" to the line... without "flake8: ". Looking at codesearch [3], there is a lot of projects using the "flake8: noqa" approach with the intent of ignoring a specific line. It would be important to fix that in order to make sure we're only ignoring the specific lines we're interested in ignoring and prevent upcoming failures in the jobs. [1]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-10-26.log.html#t2018-10-26T16:18:38 [2]: http://flake8.pycqa.org/en/latest/release-notes/3.6.0.html [3]: http://codesearch.openstack.org/?q=flake8%3A%20noqa=nope== David Moreau Simard dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] Upcoming changes in ARA Zuul job reports
Hi, It took longer than ancitipated but the necessary changes have landed in both ARA and logs.openstack.org. The reports should now be faster and more reliable. Please let me know if you see anything out of the ordinary. Thanks ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Wed, Apr 4, 2018 at 6:16 PM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi, > > You might have noticed that the performance (and reliability) of the > new reports aren't up to par. > If you see failures in loading content, a refresh will usually fix the issue. > > We have different fixes to improve the performance and the reliability > of the reports and we hope > to be able to land them soon. > > In the meantime, please let us know if there is any report that > appears to be particularly > problematic. > > Thanks ! > > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] > > > On Thu, Mar 29, 2018 at 6:14 PM, David Moreau Simard > <dmsim...@redhat.com> wrote: >> Hi, >> >> By default, all jobs currently benefit from the generation of a static >> ARA report located in the "ara" directory at the root of the log >> directory. >> Due to scalability concerns, these reports were only generated when a >> job failed and were not available on successful runs. >> >> I'm happy to announce that you can expect ARA reports to be available >> for every job from now on -- including the successful ones ! >> >> You'll notice a subtle but important change: the report directory will >> henceforth be named "ara-report" instead of "ara". >> >> Instead of generating and saving a HTML report, we'll now only save >> the ARA database in the "ara-report" directory. >> This is a special directory from the perspective of the >> logs.openstack.org server and ARA databases located in such >> directories will be loaded dynamically by a WSGI middleware. >> >> You don't need to do anything to benefit from this change -- it will >> be pushed to all jobs that inherit from the base job by default. >> >> However, if you happen to be using a "nested" installation of ARA and >> Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this >> means that you can also leverage this feature. >> In order to do that, you'll want to create an "ara-report" directory >> and copy your ARA database inside before your logs are collected and >> uploaded. >> >> To help you visualize: >> /ara-report <-- This is the default Zuul report >> /logs/ara <-- This wouldn't be loaded dynamically >> /logs/ara-report <-- This would be loaded dynamically >> /logs/some/directory/ara-report <-- This would be loaded dynamically >> >> For more details on this feature of ARA, you can refer to the documentation >> [1]. >> >> Let me know if you have any questions ! >> >> [1]: https://ara.readthedocs.io/en/latest/advanced.html >> >> David Moreau Simard >> Senior Software Engineer | OpenStack RDO >> >> dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Recent failures to use ARA or generate reports in the gate
Hi, I forgot to follow up but this was resolved last Monday April 30th when Flask released 0.12.4. Please let me know if you see anything out of the ordinary. Thanks ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Apr 27, 2018 at 11:41 AM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi, > > I was made aware today that new installations of ARA were not working > or failing to generate reports in a variety of gate jobs with a stack > trace that ends with: > AttributeError: 'Blueprint' object has no attribute 'json_encoder' > > The root cause was identified to be a new release of Flask, 0.12.3, > which shipped broken packages to PyPi [1]. > This should be fixed momentarily once upstream ships a fixed 0.12.4 package. > > In the meantime, we're going to merge a requirements.txt update to > blacklist 0.12.3 but it won't be effective until we cut a new release > of ARA which we hope to be able to do sometime next week. > > I'll take the opportunity to remind users of ARA that we're > transitioning away from statically generated reports [3] and you > should do that too if you haven't already. > > [1]: https://github.com/pallets/flask/issues/2728 > [2]: > https://github.com/openstack/requirements/blob/a5537a6f4b9cc477067949e1f9136415ac216f21/upper-constraints.txt# > L480 > [3]: http://lists.openstack.org/pipermail/openstack-dev/2018-March/128902.html > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Recent failures to use ARA or generate reports in the gate
Hi, I was made aware today that new installations of ARA were not working or failing to generate reports in a variety of gate jobs with a stack trace that ends with: AttributeError: 'Blueprint' object has no attribute 'json_encoder' The root cause was identified to be a new release of Flask, 0.12.3, which shipped broken packages to PyPi [1]. This should be fixed momentarily once upstream ships a fixed 0.12.4 package. In the meantime, we're going to merge a requirements.txt update to blacklist 0.12.3 but it won't be effective until we cut a new release of ARA which we hope to be able to do sometime next week. I'll take the opportunity to remind users of ARA that we're transitioning away from statically generated reports [3] and you should do that too if you haven't already. [1]: https://github.com/pallets/flask/issues/2728 [2]: https://github.com/openstack/requirements/blob/a5537a6f4b9cc477067949e1f9136415ac216f21/upper-constraints.txt# L480 [3]: http://lists.openstack.org/pipermail/openstack-dev/2018-March/128902.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][kolla][rdo] Collaboration with Kolla for the RDO test days
Hi, For a mix of good and bad reasons, we haven't been able to do this until now. The upcoming RDO test days will be May 3rd and 4th and we'll be testing the latest content from Rocky now that M1 has been released. We can re-use the pad we had started last time around [1]. I'll be in touch. [1]: https://etherpad.openstack.org/p/kolla-rdo-m3 David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Jan 29, 2018 at 8:29 AM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi ! > > For those who might be unfamiliar with the RDO [1] community project: > we hang out in #rdo, we don't bite and we build vanilla OpenStack > packages. > > These packages are what allows you to leverage one of the deployment > projects such as TripleO, PackStack or Kolla to deploy on CentOS or > RHEL. > The RDO community collaborates with these deployment projects by > providing trunk and stable packages in order to let them develop and > test against the latest and the greatest of OpenStack. > > RDO test days typically happen around a week after an upstream > milestone has been reached [2]. > The purpose is to get everyone together in #rdo: developers, users, > operators, maintainers -- and test not just RDO but OpenStack itself > as installed by the different deployment projects. > > We tried something new at our last test day [3] and it worked out great. > Instead of encouraging participants to install their own cloud for > testing things, we supplied a cloud of our own... a bit like a limited > duration TryStack [4]. > This lets users without the operational knowledge, time or hardware to > install an OpenStack environment to see what's coming in the upcoming > release of OpenStack and get the feedback loop going ahead of the > release. > > We used Packstack for the last deployment and invited Packstack cores > to deploy, operate and troubleshoot the installation for the duration > of the test days. > The idea is to rotate between the different deployment projects to > give every interested project a chance to participate. > > Last week, we reached out to Kolla to see if they would be interested > in participating in our next RDO test days [5] around February 8th. > We supply the bare metal hardware and their core contributors get to > deploy and operate a cloud with real users and developers poking > around. > All around, this is a great opportunity to get feedback for RDO, Kolla > and OpenStack. > > We'll be advertising the event a bit more as the test days draw closer > but until then, I thought it was worthwhile to share some context for > this new thing we're doing. > > Let me know if you have any questions ! > > Thanks, > > [1]: https://www.rdoproject.org/ > [2]: https://www.rdoproject.org/testday/ > [3]: > https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ > [4]: http://trystack.org/ > [5]: > http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-01-24-16.00.log.html > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Changes to Zuul role checkouts
If you're not familiar with the "override-checkout" configuration, you can find the documentation about it here [1] and some example usage here [2]. [1]: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.override-checkout [2]: http://codesearch.openstack.org/?q=override-checkout David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Apr 9, 2018 at 12:55 PM, James E. Blair <cor...@inaugust.com> wrote: > Hi, > > We recently fixed a subtle but important bug related to how Zuul checks > out repositories it uses to find Ansible roles for jobs. > > This may result in a behavior change, or even an error, for jobs which > use roles defined in projects with multiple branches. > > Previously, Zuul would (with some exceptions) generally check out the > 'master' branch of any repository which appeared in the 'roles:' stanza > in the job definition. Now Zuul will follow its usual procedure of > trying to find the most appropriate branch to check out. That means it > tries the project override-checkout branch first, then the job > override-checkout branch, then the branch of the change, and finally the > default branch of the project. > > This should produce more predictable behavior which matches the > checkouts of all other projects involved in a job. > > If you find that the wrong branch of a role is being checked out, > depending on circumstances, you may need to set a job or project > override-checkout value to force the correct one, or you may need to > backport a role to an older branch. > > If you encounter any problems related to this, please chat with us in > #openstack-infra. > > Thanks, > > Jim > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] Upcoming changes in ARA Zuul job reports
Hi, You might have noticed that the performance (and reliability) of the new reports aren't up to par. If you see failures in loading content, a refresh will usually fix the issue. We have different fixes to improve the performance and the reliability of the reports and we hope to be able to land them soon. In the meantime, please let us know if there is any report that appears to be particularly problematic. Thanks ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Thu, Mar 29, 2018 at 6:14 PM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi, > > By default, all jobs currently benefit from the generation of a static > ARA report located in the "ara" directory at the root of the log > directory. > Due to scalability concerns, these reports were only generated when a > job failed and were not available on successful runs. > > I'm happy to announce that you can expect ARA reports to be available > for every job from now on -- including the successful ones ! > > You'll notice a subtle but important change: the report directory will > henceforth be named "ara-report" instead of "ara". > > Instead of generating and saving a HTML report, we'll now only save > the ARA database in the "ara-report" directory. > This is a special directory from the perspective of the > logs.openstack.org server and ARA databases located in such > directories will be loaded dynamically by a WSGI middleware. > > You don't need to do anything to benefit from this change -- it will > be pushed to all jobs that inherit from the base job by default. > > However, if you happen to be using a "nested" installation of ARA and > Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this > means that you can also leverage this feature. > In order to do that, you'll want to create an "ara-report" directory > and copy your ARA database inside before your logs are collected and > uploaded. > > To help you visualize: > /ara-report <-- This is the default Zuul report > /logs/ara <-- This wouldn't be loaded dynamically > /logs/ara-report <-- This would be loaded dynamically > /logs/some/directory/ara-report <-- This would be loaded dynamically > > For more details on this feature of ARA, you can refer to the documentation > [1]. > > Let me know if you have any questions ! > > [1]: https://ara.readthedocs.io/en/latest/advanced.html > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] Selecting New Priority Effort(s)
It won't be very exciting but we really need to do one of the following two things soon: 1) Ansiblify control plane [1] 2) Update our puppet things to puppet 4 (or 5?) Puppet 3 has been end of life since Dec 31, 2016. [2] The longer we draw this out, the more work it'll be :( [1]: https://review.openstack.org/#/c/469983/ [2]: https://groups.google.com/forum/#!topic/puppet-users/IdutL5FTW7w David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Tue, Apr 3, 2018 at 4:23 PM, Clark Boylan <cboy...@sapwetik.org> wrote: > Hello everyone, > > I just approved the change to mark the Zuul v3 priority effort as completed > in the infra-specs repo. Thank you to everyone that made that possible. With > Zuul v3 work largely done we can now look forward to our next priority > efforts. > > Currently the only task marked as a priority is the task-tracker spec which > at this point is migrating projects into storyboard. I think we can likely > add one or two new priority efforts to this list. > > After some quick initial brainstorming these were the ideas I had for getting > onto that list (note some may require we actually write a spec): > > * Gerrit upgrade to 2.14/2.15 > * Control Plane operating system upgrades to Xenial > * Bringing wiki under config management management > > My bias here is I've personally been working to try and pay down some of this > tech debt we've built up simply due to bit rot, but I know we have other > specs and I'm sure we can make good arguments for why other efforts should be > made a priority. I'd love to get feedback on what others think would make > good priority efforts. > > Let's use this thread to identify candidates then whittle the list down to > one or two to focus on for the next little while. > > Thank you, > Clark > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [all][infra] Upcoming changes in ARA Zuul job reports
On Thu, Mar 29, 2018 at 9:05 PM, Jeffrey Zhang <zhang.lei@gmail.com> wrote: > cool. kolla will try to implement it. Cool ! For reference, openstack-ansible already retooled their log collection to copy the database instead of generating the report [1]. [1]: https://review.openstack.org/#/c/557921/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] Our job is done, time to close up shop.
The requirements project isn't even required. Oh, the irony. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Sat, Mar 31, 2018 at 11:57 PM, Matthew Thode <prometheanf...@gentoo.org> wrote: > The requirements project had a good run, but things seem to be winding > down. We only break openstack a couple times a cycle now, and that's > just not acceptable. The graph must go up and to the right. So, it's > time for the requirements project to close up shop. So long and thanks > for all the fish. > > -- > Matthew Thode (prometheanfire) > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all][infra] Upcoming changes in ARA Zuul job reports
Hi, By default, all jobs currently benefit from the generation of a static ARA report located in the "ara" directory at the root of the log directory. Due to scalability concerns, these reports were only generated when a job failed and were not available on successful runs. I'm happy to announce that you can expect ARA reports to be available for every job from now on -- including the successful ones ! You'll notice a subtle but important change: the report directory will henceforth be named "ara-report" instead of "ara". Instead of generating and saving a HTML report, we'll now only save the ARA database in the "ara-report" directory. This is a special directory from the perspective of the logs.openstack.org server and ARA databases located in such directories will be loaded dynamically by a WSGI middleware. You don't need to do anything to benefit from this change -- it will be pushed to all jobs that inherit from the base job by default. However, if you happen to be using a "nested" installation of ARA and Ansible (i.e, OpenStack-Ansible, Kolla-Ansible, TripleO, etc.), this means that you can also leverage this feature. In order to do that, you'll want to create an "ara-report" directory and copy your ARA database inside before your logs are collected and uploaded. To help you visualize: /ara-report <-- This is the default Zuul report /logs/ara <-- This wouldn't be loaded dynamically /logs/ara-report <-- This would be loaded dynamically /logs/some/directory/ara-report <-- This would be loaded dynamically For more details on this feature of ARA, you can refer to the documentation [1]. Let me know if you have any questions ! [1]: https://ara.readthedocs.io/en/latest/advanced.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Prototyping dedicated roles with unique repositories for Ansible tasks in TripleO
Nice! I don't have a strong opinion about this but what I might recommend would be to chat with the openshift-ansible [1] and the kolla-ansible [2] folks. I'm happy to do the introductions if necessary ! Their models, requirements or context might be different than ours but at the end of the day, it's a set of Ansible roles and playbooks to install something. It would be a good idea just to informally chat about the reasons why their things are set up the way they are, what are the pros, cons.. or their challenges. I'm not saying we should structure our things like theirs. What I'm trying to say is that they've surely learned a lot over the years these projects have existed and it's surely worthwhile to chat with them so we don't repeat some of the same mistakes. Generally just draw from their experience, learn from their conclusions and take that into account before committing to any particular model we'd like to have in TripleO ? [1]: https://github.com/openshift/openshift-ansible [2]: https://github.com/openstack/kolla-ansible David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Thu, Mar 29, 2018, 12:34 PM David Peacock, <dpeac...@redhat.com> wrote: > Hi everyone, > > During the recent PTG in Dublin, it was decided that we'd prototype a way > forward with Ansible tasks in TripleO that adhere to Ansible best > practises, creating dedicated roles with unique git repositories and RPM > packaging per role. > > With a view to moving in this direction, a couple of us on the TripleO > team have begun developing tooling to facilitate this. Initially we've > worked on a tool [0] to extract Ansible tasks lists from > tripleo-heat-templates and move them into new formally structured Ansible > roles. > > An example with the existing keystone docker service [1]: > > The upgrade_tasks block will become: > > ``` > upgrade_tasks: > - import_role: > name: tripleo-role-keystone > tasks_from: upgrade.yaml > ``` > > The fast_forward_upgrade_tasks block will become: > > ``` > fast_forward_upgrade_tasks: > - import_role: > name: tripleo-role-keystone > tasks_from: fast_forward_upgrade.yaml > ``` > > And this role [2] will be structured: > > ``` > tripleo-role-keystone/ > └── tasks > ├── fast_forward_upgrade.yaml > ├── main.yaml > └── upgrade.yaml > ``` > > We'd love to hear any feedback from the community as we move towards this. > > Thank you, > David Peacock > > [0] https://github.com/davidjpeacock/openstack-role- > extract/blob/master/role-extractor-creator.py > [1] https://github.com/openstack/tripleo-heat-templates/blob/ > master/docker/services/keystone.yaml > [2] https://github.com/davidjpeacock/tripleo-role-keystone > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?
The talk was this week and it's up on YouTube [1]. During the talk which was basically a long live demo, we... - Sent a patch to fix a typo in the talk [2] - Fixed a Zuul job through speculative testing [3] - Updated the openstack-infra IRC meeting chair [4]. Oh, and we also added an item on the next meeting to talk about this talk [5]. It was fun. [1]: https://youtu.be/6gTsL7E7U7Q [2]: https://review.openstack.org/#/c/556738/ [3]: https://review.openstack.org/#/c/556615/ [4]: https://review.openstack.org/#/c/557095/ [5]: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Sat, Mar 24, 2018 at 9:28 PM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi -infra, > > I'll be presenting a talk at a local OpenStack meetup next week [1] > that will highlight some examples about how people can help and > contribute to the infrastructure project. > The talk will be recorded and should hopefully serve as a form of > informal documentation. > > I'd like to disclose some semi-official numbers (as I'd personally > pull them up) to let people have an idea of the scale our contributors > are maintaining. > I suppose this data is already somewhat public if you know where to > look but I don't think it's been written down in a digestable format > in recent history. > > Unless there's any objection, I'd have a slide with up to date numbers such > as: > - # of projects hosted (as per git.openstack.org) > - # of servers (in aggregate of all our regions) > -- (Maybe some big highlights like the size of logstash, logs.o.o, Zuul) > - Nodepool capacity (number of clouds, aggregate capacity) > - # of jobs and Ansible playbooks per month ran by Zuul > - Approximate number of maintained and hosted services (irc, > gerritbot, meetbot, gerrit, git, mailing lists, wiki, ask.openstack, > storyboard, codesearch, etc.) > - Probably some high level numbers from Stackalytics > - Maybe something else I haven't thought about > > The idea of the talk is not to brag about all the stuff we're doing > but rather, "hey, you don't need to be a pro in OpenStack to > contribute, we got all these different things you can help with". > > I realize it's a bit last minute but please let me know if you see > anything wrong with this ! > > [1]: https://www.meetup.com/Montreal-OpenStack/events/248344351/ > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?
Good point. I'll work with that instead. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Mar 26, 2018 at 4:30 PM, James E. Blair <cor...@inaugust.com> wrote: > David Moreau Simard <dmsim...@redhat.com> writes: > >> On Mon, Mar 26, 2018 at 10:20 AM, James E. Blair <cor...@inaugust.com> wrote: >>>> - # of jobs and Ansible playbooks per month ran by Zuul >>> >>> I'm curious about this one -- how were you planning on defining these >>> values and obtaining them? >>> >> >> I've needed to pull statistics out of Zuul in the past for RDO (i.e, >> justifying budget for CI resources) >> and I use the sql reporter data to do it. >> It looks like this: >> >> $range = "'2018-02-01 00:00:00' AND '2018-02-28 23:59:59'" >> SELECT job_name, >>result, >>start_time, >>end_time, >>TIMEDIFF(end_time, start_time) as duration >> FROM zuul_build >> WHERE >> start_time BETWEEN $range >> >> This gets me the amount of monthly *jobs* and I can extrapolate (over >> N playbooks..) >> by estimating a number knowing that: >> - base and post playbooks are fairly consistently X playbooks >> - there is at least one "run" playbook >> >> So pretending that 1000 jobs ran, I can say something like: >> 1000 jobs and over [1000 * (X+1)] playbooks >> >> It's not a perfect number but we know we run more playbooks than that. >> >> What I have also been thinking about is, if I want to get a more >> accurate number, I could do a sum of all the executor playbook results >> (which are in graphite) but the history for those don't go too far >> back. >> Ex: stats.zuul.executor.ze*_openstack_org.phase.*.* > > The SQL query gets the number of completed jobs which are *reported*. > It doesn't get you two other numbers, which are the jobs *launched* > (many of which may have been aborted before completion), or the jobs > *completed* (the results of many of which may have been discarded due to > changes in the environment). In reality, the system is likely to be > significantly busier than the number of jobs reported will indicate. > > Both of the other values can be obtained from graphite or by parsing > logs. I think for this purpose, graphite might be sufficient. (The > only time I'd recommend going to logs is when we need to find > project-specific resource usage information.) > > stats_counts.zuul.executor.*.builds should be all jobs launched. > stats_counts.zuul.tenant.*.pipeline.*.all_jobs should be all jobs completed. > > -Jim ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?
On Mon, Mar 26, 2018 at 10:20 AM, James E. Blair <cor...@inaugust.com> wrote: >> - # of jobs and Ansible playbooks per month ran by Zuul > > I'm curious about this one -- how were you planning on defining these > values and obtaining them? > I've needed to pull statistics out of Zuul in the past for RDO (i.e, justifying budget for CI resources) and I use the sql reporter data to do it. It looks like this: $range = "'2018-02-01 00:00:00' AND '2018-02-28 23:59:59'" SELECT job_name, result, start_time, end_time, TIMEDIFF(end_time, start_time) as duration FROM zuul_build WHERE start_time BETWEEN $range This gets me the amount of monthly *jobs* and I can extrapolate (over N playbooks..) by estimating a number knowing that: - base and post playbooks are fairly consistently X playbooks - there is at least one "run" playbook So pretending that 1000 jobs ran, I can say something like: 1000 jobs and over [1000 * (X+1)] playbooks It's not a perfect number but we know we run more playbooks than that. What I have also been thinking about is, if I want to get a more accurate number, I could do a sum of all the executor playbook results (which are in graphite) but the history for those don't go too far back. Ex: stats.zuul.executor.ze*_openstack_org.phase.*.* David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] Public numbers about the scale of the infrastructure/CI ?
Hi -infra, I'll be presenting a talk at a local OpenStack meetup next week [1] that will highlight some examples about how people can help and contribute to the infrastructure project. The talk will be recorded and should hopefully serve as a form of informal documentation. I'd like to disclose some semi-official numbers (as I'd personally pull them up) to let people have an idea of the scale our contributors are maintaining. I suppose this data is already somewhat public if you know where to look but I don't think it's been written down in a digestable format in recent history. Unless there's any objection, I'd have a slide with up to date numbers such as: - # of projects hosted (as per git.openstack.org) - # of servers (in aggregate of all our regions) -- (Maybe some big highlights like the size of logstash, logs.o.o, Zuul) - Nodepool capacity (number of clouds, aggregate capacity) - # of jobs and Ansible playbooks per month ran by Zuul - Approximate number of maintained and hosted services (irc, gerritbot, meetbot, gerrit, git, mailing lists, wiki, ask.openstack, storyboard, codesearch, etc.) - Probably some high level numbers from Stackalytics - Maybe something else I haven't thought about The idea of the talk is not to brag about all the stuff we're doing but rather, "hey, you don't need to be a pro in OpenStack to contribute, we got all these different things you can help with". I realize it's a bit last minute but please let me know if you see anything wrong with this ! [1]: https://www.meetup.com/Montreal-OpenStack/events/248344351/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [tc] [all] TC Report 18-12
In case people have missed it, Jim Blair sent an email recently to shed some light on where Zuul is headed [1]. [1]: http://lists.openstack.org/pipermail/openstack-dev/2018-March/128396.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Tue, Mar 20, 2018 at 7:24 PM, Chris Dent <cdent...@anticdent.org> wrote: > > HTML: https://anticdent.org/tc-report-18-12.html > > This week's TC Report goes off in the weeds a bit with the editorial > commentary from yours truly. I had trouble getting started, so had > to push myself through some thinking by writing stuff that at least > for the last few weeks I wouldn't normally be including in the > summaries. After getting through it, I realized that the reason I > was struggling is because I haven't been including these sorts of > things. Including them results in a longer and more meandering report > but it is more authentically my experience, which was my original > intention. > > # Zuul Extraction and the Difficult Nature of Communication > > Last [Tuesday > Morning](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-13.log.html#t2018-03-13T17:22:38) > we had some initial discussion about Zuul being extracted from > OpenStack governance as a precursor to becoming part of the CI/CD > strategic area being born elsewhere in the OpenStack Foundation. > > Then on > [Thursday](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-15.log.html#t2018-03-15T15:08:06) > we revisited the topic, especially as it related to how we > communicate change in the community and how we invite participation > in making decisions about change. In this case by "community" we're > talking about anything under the giant umbrella of "stuff associated > with the OpenStack Foundation". > > Plenty of people expressed that though they were not surprised by > the change, it was because they are insiders and could understand > how some, who are not, might be surprised by what seemed like a big > change. This led to addressing the immediate shortcomings and > clarifying the history of the event. > > There was also > [concern](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-15.log.html#t2018-03-15T15:27:22) > that some of the reluctance to talk openly about the change appeared > to stem from needing to preserve the potency of a Foundation marketing > release. > > I [expressed some > frustration](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-15.log.html#t2018-03-15T15:36:50): > "...as usual, we're getting caught up in > details of a particular event (one that in the end we're all happy > to see happen), rather than the general problem we saw with it > (early transparency etc). Solving the immediate problem is easy, but > since we _keep doing it_, we've got a general issues to resolve." > > We went round and round about the various ways in which we have tried > and failed to do good communication in the past, and while we make > some progress, we fail to establish a pattern. As Doug [pointed > out](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-15.log.html#t2018-03-15T15:41:33), > no method can be 100% successful, but if we pick a method and stick to > it, people can learn that method. > > We have a cycle where we not only sometimes communicate poorly but > we also communicate poorly about that poor communication. So when I > come round to another week of writing this report, and am reminded > that these issues persist and I am once again communicating about > them, it's frustrating. Communicating, a lot, is generally a good > thing, but if things don't change as a result, that can be a strain. > If I'm still writing these things in a year's time, and we haven't > managed to achieve at least a bit more grace, consistency, and > transparency in the ways that we share information within and > between groups (including, and maybe especially, the Foundation > executive wing) in the wider community, it will be a shame and I will > have a sad. > > In a somewhat related and good sign, there is [great > thread](http://lists.openstack.org/pipermail/openstack-operators/2018-March/014994.html) > on the operators list that raises the potential of merging the Ops > Meeting and the PTG into some kind of "OpenStack Community Working > Gathering". > > # Encouraging Upstream Contribution > > On > [Friday](http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-03-16.log.html#t2018-03-16T14:29:21), > tbarron raised some interesting questions about how the summit talk > selection process might relate to the [four &
Re: [openstack-dev] [tripleo] storyboard evaluation
I have nothing to add but I wanted to mention that it seems like a great exercise to have UI/UX minded folks test our Storyboard implementation. Storyboard will be what we make of it based on the feedback and contributions it gets. It's an interesting opportunity. I'd actually like to encourage Horizon and the different UI components developers to test Storyboard, provide feedback and contribute ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Mar 2, 2018 at 4:24 AM, Emilien Macchi <emil...@redhat.com> wrote: > A quick update: > > - Discussed with Jiri Tomasek from TripleO UI squad and he agreed that his > squad would start to use Storyboard, and experiment it. > - I told him I would take care of making sure all UI bugs created in > Launchpad would be moved to Storyboard. > - Talked with Kendall and we agreed that we would move forward and migrate > TripleO UI bugs to Storyboard. > - TripleO UI Squad would report feedback about storyboard to the storyboard > team with the help of other TripleO folks (me at least, I'm willing to > help). > > Hopefully this is progress and we can move forward. More updates to come > about migration during the next days... > > Thanks everyone involved in these productive discussions. > > On Wed, Jan 17, 2018 at 12:33 PM, Thierry Carrez <thie...@openstack.org> > wrote: >> >> Clint Byrum wrote: >> > [...] >> > That particular example board was built from tasks semi-automatically, >> > using a tag, by this script running on a cron job somewhere: >> > >> > >> > https://git.openstack.org/cgit/openstack-infra/zuul/tree/tools/update-storyboard.py?h=feature/zuulv3 >> > >> > We did this so that we could have a rule "any task that is open with >> > the zuulv3 tag must be on this board". Jim very astutely noticed that >> > I was not very good at being a robot that did this and thus created the >> > script to ease me into retirement from zuul project management. >> > >> > The script adds new things in New, and moves tasks automatically to >> > In Progress, and then removes them when they are completed. We would >> > periodically groom the "New" items into an appropriate lane with the >> > hopes >> > of building what you might call a rolling-sprint in Todo, and calling >> > out blocked tasks in a regular meeting. Stories were added manually as >> > a way to say "look in here and add tasks", and manually removed when >> > the larger effort of the story was considered done. >> > >> > I rather like the semi-automatic nature of it, and would definitely >> > suggest that something like this be included in Storyboard if other >> > groups find the board building script useful. This made a cross-project >> > effort between Nodepool and Zuul go more smoothly as we had some more >> > casual contributors to both, and some more full-time. >> >> That's a great example that illustrates StoryBoard design: rather than >> do too much upfront feature design, focus on primitives and expose them >> fully through a strong API, then let real-world usage dictate patterns >> that might result in future features. >> >> The downside of this approach is of course getting enough usage on a >> product that appears a bit "raw" in terms of features. But I think we >> are closing on getting that critical mass :) >> >> -- >> Thierry Carrez (ttx) >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ openstack-dev ][ tripleo ] unplanned outtage in RDO-Cloud
Please be wary of approving changes since the Third Party CI is out of order until this is resolved. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Feb 23, 2018 at 5:40 PM, Wesley Hayutin <whayu...@redhat.com> wrote: > > > On Fri, Feb 23, 2018 at 12:09 PM, Wesley Hayutin <whayu...@redhat.com> > wrote: >> >> Greetings, >> >> The TripleO CI in RDO-Cloud has experienced an unplanned outage and is >> down at this time. We will update this thread with more information >> regarding when the CI will be brought back online as it becomes available. >> >> >> Thank you! >> Wes Hayutin > > > FYI.. > The latest estimate for the unplanned outtage to TripleO-CI in RDO-Cloud is > that it will take a number of business days to resolve the issues. > > Thank you! > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Deep dive on Ansible Integration Thursday Feb 22 1400UTC
Thanks for recording it! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Feb 19, 2018 at 9:41 AM, James Slagle <james.sla...@gmail.com> wrote: > As mentioned in the TripleO meeting last week, I volunteered to give a > deep dive on the state of TripleO and Ansible integration with > config-download. > > I'll do that this week on Thursday February 22nd at 1400UTC. > > Anyone can join via bluejeans: https://bluejeans.com/7754237859/ > Etherpad: https://bluejeans.com/7754237859/ > Optional pre-reading: > https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ansible_config_download.html > > The session will be recorded and later uploaded to Youtube at: > https://www.youtube.com/channel/UCNGDxZGwUELpgaBoLvABsTA > > -- > -- James Slagle > -- > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [release][ptl] Final Queens RC Deadline
On Mon, Feb 19, 2018 at 10:44 AM, Sean McGinnis <sean.mcgin...@gmx.com> wrote: > Hey everyone, > > Just a quick reminder that Thursday, 22 March, is the deadline for any final > Queens release candidates. After this point we will enter a quiet period for a > week in preparation of tagging the final Queens release during the PTG week. February, right ? David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo][python3] python3 readiness?
On Wed, Feb 14, 2018 at 11:19 AM, Ben Nemec <openst...@nemebean.com> wrote: > > I have to admit I don't entirely understand this constraint. CentOS 7 is in > support until 2024. I would think RHEL 7's timeline is similar or even > longer. If Python 2 is going out of support in 2020, does that mean there > will be no supported Python on CentOS for the last four years of its > lifecycle? The OpenStack community is definitely not expected to support py2 beyond 2020. If RHEL and CentOS wants to support py2 beyond that date, the burden is on them. The RHEL 8 release date is unknown. We can only speculate that it should be "sometime soon" based on previous release dates [1]. I don't know if it's going to be an official goal to drop py27 support in OpenStack for Rocky but we can't wait at the last minute -- py3 support has been a goal for a long time [2]. It doesn't mean that RDO has to support a python2 version of OpenStack on EL7 after upstream has dropped support for it. It's similar to how EL6 support was eventually dropped after moving on from py26 (or was it py25?) and we started shipping on EL7. > In fact, the more I think about this the more I feel like there's a > fundamental problem with the way we're handling this transition. We're not > the only ones who are going to feel the pain of having disjoint Python > releases from 7 to 8. Anyone running a Python application now gets to not > only do a major OS upgrade, but also a major Python upgrade. Sure, it's > worse for us because we need to support EL 8 at release, but _everyone_ is > going to feel some variation on this pain as they move forward. Python 3 has been out since 2008 [3], yup, 10 years ago.. and here we are. I remember when most of this board was red [4]. > I realize this is a discussion that's probably above my pay grade, but I > feel I would be remiss if I didn't point out that our Python support > strategy seems very flawed. It's no use questioning the decisions that lead RHEL7 to ship without py3 in 2014, we can only look forward at this point :) [1]: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Version_history [2]: https://governance.openstack.org/tc/goals/pike/python35.html [3]: https://en.wikipedia.org/wiki/History_of_Python#Version_3 [4]: https://python3wos.appspot.com/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo][python3] python3 readiness?
On Tue, Feb 13, 2018 at 5:53 PM, Ben Nemec <openst...@nemebean.com> wrote: > > I guess if RDO has chosen this path then we don't have much choice. This makes it sound like we had a choice to begin with. We've already had a lot of discussions around the topic but we're ultimately stuck between a rock and a hard place. We're in this together and it's important that everyone understands what's going on. It's not a secret to anyone that Fedora is more or less the upstream to RHEL. There's no py3 available in RHEL 7. The alternative to making things work in Fedora is to use Software Collections [1]. If you're not familiar with Software Collections for python, it's more or less the installation of RPM packages in a virtualenv. Installing the "rh-python35" SCL would: - Set up a chroot in /opt/rh/rh-python35/root - Set up a py35 interpreter at /opt/rh/rh-python35/root/usr/bin/python3 And then, when you would install packages *against* that SCL, they would end up being installed in /opt/rh/rh-python35/root/usr/lib/python3.5/site-packages/. That means that you need *all* of your python packages to be built against the software collections and installed in the right path. Python script with a #!/usr/bin/python shebang ? Probably not going to work. Need python-requests ? Nope, sclo-python35-python-requests. Need one of the 1000+ python packages maintained by RDO ? Those need to be re-built and maintained against the SCL too. If you want to see what it looks like in practice, here's a Zuul spec file [2] or the official docs for SCL [3]. Making stuff work on Fedora is not going to be easy for anyone but it sure beats messing with 1500+ packages that we'd need to untangle later. Most of the hard work for Fedora is already done as far as packaging is concerned, we never really stopped building packages for Fedora [4]. It means we should be prepared once RHEL 8 comes out. [1]: https://www.softwarecollections.org/en/ [2]: https://softwarefactory-project.io/r/gitweb?p=scl/zuul-distgit.git;a=blob;f=zuul.spec;h=6bba6a79c1f8ff844a9ea3715ab2cef1b12d323f;hb=refs/heads/master [3]: https://www.softwarecollections.org/en/docs/guide/#chap-Packaging_Software_Collections [4]: https://trunk.rdoproject.org/fedora-rawhide/report.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] Updating default Docker registry and namespace
Hi, I've submitted a series of patches: https://review.openstack.org/#/q/topic:default-registry In these patches, I am merely doing the following: #1 s/trunk.registry.rdoproject.org/docker.io/ trunk.registry.rdoproject.org is not meant for production or stable use, it should only be used as a staging ground so we don't spam docker.io with hundreds of images needlessly. We're pushing and tagging tested and promoted images to docker.io -- trunk.registry.rdoproject.org should not be advertised. #2 s/latest/current-tripleo/ We don't use the "latest" tag, we push tags based on their trunk repository hash (DLRN) or names such as "current-tripleo", "current-tripleo-rdo", etc. #3 s/tripleoupstream/tripleomaster/ The docker.io/tripleoupstream namespace is unmaintained. Images are now pushed and tagged in the tripleomaster namespace. Patches for these should be backported to Pike while replacing "tripleomaster" for "tripleopike" once they have landed. Please validate the patches properly as I can't pretend to have tested these before sending them. Thanks, David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][kolla][rdo] Collaboration with Kolla for the RDO test days
Please note that the RDO test days have currently been re-scheduled to (at least) next week, February 15th and 16th. We are currently working our way through different issues and hope they'll be sorted out in time by then. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Feb 5, 2018 at 10:31 AM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi everyone, > > We've started planning the deployment with the Kolla team, you can see > the etherpad from the "operator" perspective here: > https://etherpad.openstack.org/p/kolla-rdo-m3 > > We'll advertise the test days and how users can participate soon. > > Thanks, > > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] > > > On Mon, Jan 29, 2018 at 8:29 AM, David Moreau Simard > <dmsim...@redhat.com> wrote: >> Hi ! >> >> For those who might be unfamiliar with the RDO [1] community project: >> we hang out in #rdo, we don't bite and we build vanilla OpenStack >> packages. >> >> These packages are what allows you to leverage one of the deployment >> projects such as TripleO, PackStack or Kolla to deploy on CentOS or >> RHEL. >> The RDO community collaborates with these deployment projects by >> providing trunk and stable packages in order to let them develop and >> test against the latest and the greatest of OpenStack. >> >> RDO test days typically happen around a week after an upstream >> milestone has been reached [2]. >> The purpose is to get everyone together in #rdo: developers, users, >> operators, maintainers -- and test not just RDO but OpenStack itself >> as installed by the different deployment projects. >> >> We tried something new at our last test day [3] and it worked out great. >> Instead of encouraging participants to install their own cloud for >> testing things, we supplied a cloud of our own... a bit like a limited >> duration TryStack [4]. >> This lets users without the operational knowledge, time or hardware to >> install an OpenStack environment to see what's coming in the upcoming >> release of OpenStack and get the feedback loop going ahead of the >> release. >> >> We used Packstack for the last deployment and invited Packstack cores >> to deploy, operate and troubleshoot the installation for the duration >> of the test days. >> The idea is to rotate between the different deployment projects to >> give every interested project a chance to participate. >> >> Last week, we reached out to Kolla to see if they would be interested >> in participating in our next RDO test days [5] around February 8th. >> We supply the bare metal hardware and their core contributors get to >> deploy and operate a cloud with real users and developers poking >> around. >> All around, this is a great opportunity to get feedback for RDO, Kolla >> and OpenStack. >> >> We'll be advertising the event a bit more as the test days draw closer >> but until then, I thought it was worthwhile to share some context for >> this new thing we're doing. >> >> Let me know if you have any questions ! >> >> Thanks, >> >> [1]: https://www.rdoproject.org/ >> [2]: https://www.rdoproject.org/testday/ >> [3]: >> https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ >> [4]: http://trystack.org/ >> [5]: >> http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-01-24-16.00.log.html >> >> David Moreau Simard >> Senior Software Engineer | OpenStack RDO >> >> dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][kolla][rdo] Collaboration with Kolla for the RDO test days
Hi everyone, We've started planning the deployment with the Kolla team, you can see the etherpad from the "operator" perspective here: https://etherpad.openstack.org/p/kolla-rdo-m3 We'll advertise the test days and how users can participate soon. Thanks, David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Jan 29, 2018 at 8:29 AM, David Moreau Simard <dmsim...@redhat.com> wrote: > Hi ! > > For those who might be unfamiliar with the RDO [1] community project: > we hang out in #rdo, we don't bite and we build vanilla OpenStack > packages. > > These packages are what allows you to leverage one of the deployment > projects such as TripleO, PackStack or Kolla to deploy on CentOS or > RHEL. > The RDO community collaborates with these deployment projects by > providing trunk and stable packages in order to let them develop and > test against the latest and the greatest of OpenStack. > > RDO test days typically happen around a week after an upstream > milestone has been reached [2]. > The purpose is to get everyone together in #rdo: developers, users, > operators, maintainers -- and test not just RDO but OpenStack itself > as installed by the different deployment projects. > > We tried something new at our last test day [3] and it worked out great. > Instead of encouraging participants to install their own cloud for > testing things, we supplied a cloud of our own... a bit like a limited > duration TryStack [4]. > This lets users without the operational knowledge, time or hardware to > install an OpenStack environment to see what's coming in the upcoming > release of OpenStack and get the feedback loop going ahead of the > release. > > We used Packstack for the last deployment and invited Packstack cores > to deploy, operate and troubleshoot the installation for the duration > of the test days. > The idea is to rotate between the different deployment projects to > give every interested project a chance to participate. > > Last week, we reached out to Kolla to see if they would be interested > in participating in our next RDO test days [5] around February 8th. > We supply the bare metal hardware and their core contributors get to > deploy and operate a cloud with real users and developers poking > around. > All around, this is a great opportunity to get feedback for RDO, Kolla > and OpenStack. > > We'll be advertising the event a bit more as the test days draw closer > but until then, I thought it was worthwhile to share some context for > this new thing we're doing. > > Let me know if you have any questions ! > > Thanks, > > [1]: https://www.rdoproject.org/ > [2]: https://www.rdoproject.org/testday/ > [3]: > https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ > [4]: http://trystack.org/ > [5]: > http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-01-24-16.00.log.html > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[Openstack-operators] [all][kolla][rdo] Collaboration with Kolla for the RDO test days
Hi ! For those who might be unfamiliar with the RDO [1] community project: we hang out in #rdo, we don't bite and we build vanilla OpenStack packages. These packages are what allows you to leverage one of the deployment projects such as TripleO, PackStack or Kolla to deploy on CentOS or RHEL. The RDO community collaborates with these deployment projects by providing trunk and stable packages in order to let them develop and test against the latest and the greatest of OpenStack. RDO test days typically happen around a week after an upstream milestone has been reached [2]. The purpose is to get everyone together in #rdo: developers, users, operators, maintainers -- and test not just RDO but OpenStack itself as installed by the different deployment projects. We tried something new at our last test day [3] and it worked out great. Instead of encouraging participants to install their own cloud for testing things, we supplied a cloud of our own... a bit like a limited duration TryStack [4]. This lets users without the operational knowledge, time or hardware to install an OpenStack environment to see what's coming in the upcoming release of OpenStack and get the feedback loop going ahead of the release. We used Packstack for the last deployment and invited Packstack cores to deploy, operate and troubleshoot the installation for the duration of the test days. The idea is to rotate between the different deployment projects to give every interested project a chance to participate. Last week, we reached out to Kolla to see if they would be interested in participating in our next RDO test days [5] around February 8th. We supply the bare metal hardware and their core contributors get to deploy and operate a cloud with real users and developers poking around. All around, this is a great opportunity to get feedback for RDO, Kolla and OpenStack. We'll be advertising the event a bit more as the test days draw closer but until then, I thought it was worthwhile to share some context for this new thing we're doing. Let me know if you have any questions ! Thanks, [1]: https://www.rdoproject.org/ [2]: https://www.rdoproject.org/testday/ [3]: https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ [4]: http://trystack.org/ [5]: http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-01-24-16.00.log.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[openstack-dev] [all][kolla][rdo] Collaboration with Kolla for the RDO test days
Hi ! For those who might be unfamiliar with the RDO [1] community project: we hang out in #rdo, we don't bite and we build vanilla OpenStack packages. These packages are what allows you to leverage one of the deployment projects such as TripleO, PackStack or Kolla to deploy on CentOS or RHEL. The RDO community collaborates with these deployment projects by providing trunk and stable packages in order to let them develop and test against the latest and the greatest of OpenStack. RDO test days typically happen around a week after an upstream milestone has been reached [2]. The purpose is to get everyone together in #rdo: developers, users, operators, maintainers -- and test not just RDO but OpenStack itself as installed by the different deployment projects. We tried something new at our last test day [3] and it worked out great. Instead of encouraging participants to install their own cloud for testing things, we supplied a cloud of our own... a bit like a limited duration TryStack [4]. This lets users without the operational knowledge, time or hardware to install an OpenStack environment to see what's coming in the upcoming release of OpenStack and get the feedback loop going ahead of the release. We used Packstack for the last deployment and invited Packstack cores to deploy, operate and troubleshoot the installation for the duration of the test days. The idea is to rotate between the different deployment projects to give every interested project a chance to participate. Last week, we reached out to Kolla to see if they would be interested in participating in our next RDO test days [5] around February 8th. We supply the bare metal hardware and their core contributors get to deploy and operate a cloud with real users and developers poking around. All around, this is a great opportunity to get feedback for RDO, Kolla and OpenStack. We'll be advertising the event a bit more as the test days draw closer but until then, I thought it was worthwhile to share some context for this new thing we're doing. Let me know if you have any questions ! Thanks, [1]: https://www.rdoproject.org/ [2]: https://www.rdoproject.org/testday/ [3]: https://dmsimard.com/2017/11/29/come-try-a-real-openstack-queens-deployment/ [4]: http://trystack.org/ [5]: http://eavesdrop.openstack.org/meetings/kolla/2018/kolla.2018-01-24-16.00.log.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [zuul] Cannot view log outputs in browser
That's odd. What should happen is that while the file is zipped as .txt.gz, Apache decompresses the file on-the-fly and serves it to you as plain text [1]. This should work even with something as basic as curl. There's no javascript or fancy things involved : $ curl -s http://logs.openstack.org/23/536623/1/check/build-openstack-sphinx-docs/3bbaf99/job-output.txt.gz |wc -l 1569 To my knowledge, this hasn't changed even throughout the gradual rollout of Zuul v3. There is likely a redirect happening behind the scenes in order to use the os-loganalyze middleware which is what makes the timestamps clickable, add colors and things like that [2]. Would you perhaps have an extension or plugin that would block something like that ? [1]: http://git.openstack.org/cgit/openstack-infra/puppet- openstackci/tree/templates/logs.vhost.erb#n21 [2]: http://git.openstack.org/cgit/openstack-infra/puppet- openstackci/tree/templates/logs.vhost.erb#n107 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Jan 23, 2018 7:52 AM, "Paul Bourke" <paul.bou...@oracle.com> wrote: Apologies if this has been asked before. It seems as of late (I think since the roll out of zuul v3, I can't seem to view job outputs directly in my browser. E.g. when I click link[0], I have to download 'job-output.txt.gz', unzip it, rename the extension to '.html', and finally open it in a browser. I've tried this with both Chrome 63.0.3239.132 and Firefox 57.0.4, OS Ubuntu 16.04. Is anyone else seeing this issue? Thanks, -Paul [0] http://logs.openstack.org/71/535671/11/check/kolla-ansible-o raclelinux-source-ceph/e63c60c/job-output.txt.gz __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[OpenStack-Infra] Talks for the Vancouver CFP ?
Hi, Did we want to brainstorm around topics and talks suggestions from an openstack-infra perspective for Vancouver [1] ? The deadline is February 8th and the tracks are the following: - CI / CD - Container Infrastructure - Edge Computing - HPC / GPU / AI - Open Source Community - Private & Hybrid Cloud - Public Cloud - Telecom & NFV CI/CD has Zuul and Nodepool written all over it, of course. FWIW I'm already planning on submitting a talk that covers how a commit in an upstream project ends up being released by RDO which includes the upstream Zuul and RDO's instance of Zuul (amongst other things). I started an etherpad [2], we can brainstorm there ? [1]: https://www.openstack.org/summit/vancouver-2018/call-for-presentations/ [2]: https://etherpad.openstack.org/p/infra-vancouver-cfp David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Hostnames
When I compared ze10 with ze09 today, I noticed that ze09's "hostname" command returned "ze09" while ze10 had "ze10.openstack.org". However, both nodes had the full fqdn when doing "hostname -f". I didn't dig deeper since we're the weekend and all that but there might be a clue in my experience above. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Jan 6, 2018 1:04 PM, "James E. Blair" <cor...@inaugust.com> wrote: > Hi, > > It seems that every time we boot a new server, it either randomly has a > hostname of foo, or foo.openstack.org. And maybe that changes between > the first boot and second. > > The result of this is that our services which require that they know > their hostname (which is a lot, especially the complicated ones) end up > randomly working or not. We waste time repeating the same diagnosis and > manual fix each time. > > What is the cause of this, and how do we fix this correctly? > > -Jim > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] Xenial Upgrade Sprint Recap
+1, my day to day tends to be a bit hectic so it's hard to commit to a full sprint but I have time here and there to help move things along. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Dec 15, 2017 8:09 PM, "James E. Blair" <cor...@inaugust.com> wrote: > Clark Boylan <cboy...@sapwetik.org> writes: > > > Hello everyone, > > > > Just wanted to quickly recap what we got done this week during our > > control plane upgrade to Xenial sprint. > > Thanks! > > Now that this is over, I wonder if we should start a 'marathon' (as > opposed to a sprint) to finish the rest of the servers? > > Now that we've established the pattern, perhaps if we all agreed to do > one or two servers each per week, we'd knock the rest out in good time? > > I wasn't able to spend as much time dedicated to this as I would > normally have liked, but did find that due to the latency, I could fit > in a few minutes here and there into my schedule fairly easily. > > -Jim > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [all] Switching to longer development cycles
On Wed, Dec 13, 2017 at 11:17 AM, Thierry Carrez <thie...@openstack.org> wrote: > - It doesn't mean that teams can only meet in-person once a year. > Summits would still provide a venue for team members to have an > in-person meeting. I also expect a revival of the team-organized > midcycles to replace the second PTG for teams that need or want to meet > more often. > Are we convinced that the ~3 day formula for the Forum is one we want to stick with ? With longer cycles and no PTGs in-between, it actually sounds like a good opportunity to revisit the "Forum" events to include the "design" and "ops" summits back into the event. Perhaps not in the previous format, because it had its flaws, but with longer cycles I feel we need something to close that gap. I've had a lot of discussions and some people that always went to summits now no longer attend neither the PTG (because they don't necessarily contribute upstream) or the Forum because it's only three days, costs a lot of money, jet lag, there's little to no ops/design exposure.. and the talks end up on YouTube anyway. > - It doesn't give us LTS. The cost of maintaining branches is not really > due to the number of them we need to maintain in parallel, it is due to > the age of the oldest one. We might end up being able to support > branches for slightly longer as a result of having to maintain less of > them in parallel, but we will not support our stable branches for a > significantly longer time as a direct result of this change. The real > solution here is being discussed by the (still forming) LTS SIG and > involves having a group step up to continue to maintain some branches > past EOL. > The complaints about the pace of the development and support cycles are numerous and I won't repeat them here. While this is not LTS, I can't help but think this is a compromise or a middle ground even though it might not be the intent behind the idea. Right now we ship a release every 6 months and support each release for 1 year which means at any given time, there are 3 ongoing "releases", including trunk. What I'm curious about... is how does this translate into the support cycle ? - Pike was released 2017-08-30 and the scheduled EOL is 2018-09-03. - Ocata was released 2017-02-22 and the scheduled EOL is 2018-02-26. This can go down a few different ways, I guess we can: 1) Extend the support of a stable release by a full year This keeps the two rolling stable releases plus the one in development. Not quite LTS, but you know, it's still 2 years of support instead of one. 2) Keep the support cycle to a year This would basically mean that as soon as there is a new release, the previous one would become EOL ? This is what seems suggested here and I'm really not confident this would be a great idea. It would force people to upgrade within weeks after a new release to stay on a supported version. As some others have mentioned in the thread, there are pros and cons to moving to a year-long cycle and I think it's great that we are having this discussion as it will help us make an informed decision. It won't be possible to please everyone so leaving this up to the technical committee to vote on this matter seems fair. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[Openstack-operators] Fwd: [rdo-users] Queens Milestone 2 test day, December 14th, 15th
Hi openstack-operators ! I'm sharing this in case anyone is interested in participating in the upcoming RDO community test day. We're trying something new to get more people interested in testing the latest and the greatest of OpenStack's upcoming release. Feel free to ask me if you have any questions ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] -- Forwarded message -- From: Rich Bowen <rbo...@redhat.com> Date: Mon, Dec 4, 2017 at 4:43 PM Subject: [rdo-users] Queens Milestone 2 test day, December 14th, 15th To: us...@lists.rdoproject.org Join us next Thursday and Friday as we test Queens Milestone 2 - the next version of OpenStack. http://rdoproject.org/testday/queens/milestone2/ This time, we're doing something a little different. For those of you who don't have the time, patience, or hardware to spin up your own OpenStack cloud, we'll have one ready for you. You can just log in, try it out, try your work loads, try to break things. We're hoping, with this change, to open test day to a wider audience of users and operators. If you're interested in participating in this test, and seeing what's coming in February, read this blog post [1] and sign up here [2] for your login credentials. Please also consider amplifying this invitation by following @RDOCommunity on Twitter, and retweeting the invitations to test day, as well as by sending this message to other relevant mailing lists. Thanks! [1] https://dmsimard.com/2017/11/29/come-try-a-real-openstack-qu eens-deployment/ [2] https://etherpad.openstack.org/p/rdo-queens-m2-cloud -- Rich Bowen - rbo...@redhat.com http://rdoproject.org @RDOCommunity ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[openstack-dev] [TripleO] Planning for job execution outside the gate with Zuul v3
Hi, As the migration of review.rdoproject.org to Zuul v3 draws closer, I'd like to open up the discussion around how we want to approach an eventual migration to Zuul v3 outside the gate. I'd like to take this opportunity to allow ourselves to think outside the box, think about how we would like to shape the CI of TripleO from upstream to the product and then iterate to reach that goal. The reason why I mention "outside the gate" is because one of the features of Zuul v3 is to dynamically construct its configuration by including different repositories. For example, the Zuul v3 from review.rdoproject.org can selectively include parts of git.openstack.org/openstack-infra/tripleo-ci [1] and it will load the configuration found there for jobs, nodesets, projects, etc. This opens a great deal of opportunities for sharing content or centralizing the different playbooks, roles and job parameters in one single repository rather than spread across different repositories across the production chain. If we do things right, this could give us the ability to run the same jobs (which can be customized with parameters depending on the environment, release, scenario, etc.) from the upstream gate down to review.rdoproject.org and the later productization steps. There's pros and cons to the idea, but this is just an example of what we can do with Zuul v3. Another example of an interesting thought from Sagi is to boot virtual machines directly with pre-built images instead of installing the undercloud/overcloud every time. Something else to think about is how can we leverage all the Ansible things from TripleO Quickstart in Zuul v3 natively. There's of course constraints about what we can and can't do in the upstream gate... but let's avoid prematurely blocking ourselves and try to think about what we want to do ideally and figure out if, and how, we can do it. Whether it's about the things that we would like to do, can't do, or the things that don't work, I'm sure the feedback and outcome of this could prove useful to improve Zuul. How would everyone like to proceed ? Should we start an etherpad ? Do some "design dession" meetings ? I'm willing to help get the ball rolling and spearhead the effort but this is a community effort :) Thanks ! [1]: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/zuul.d David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] [elections] Technical Committee Election Results
Was it just me or did the "official" period for campaigning/questions was awfully short ? The schedule [1] went: TC Campaigning: (Start) Oct 11, 2017 23:59 UTC (End) Oct 14, 2017 23:45 UTC That's three days, one of which was a saturday. Was it always this short ? It seems to me that this is not a lot of time to the community to ask (read, and answer) thoughful questions. I realize this doesn't mean you can't keep asking questions once the actual election voting start but I wonder if we should cut a few days from the nomination and give it to the campaigning. [1]: https://governance.openstack.org/election/#openstack-election David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Oct 20, 2017 at 8:20 PM, Tony Breeds <t...@bakeyournoodle.com> wrote: > > Hi All, > With the election behind us it's somewhat traditional to look at > some simple stats from the elections: > > +--+---+---+ > ---+ > | Election | Electorate (delta %) | Voted (delta %) | Turnout % > (delta %) | > +--+---+---+ > ---+ > | 10/2013 | 1106 (nan) | 342 (nan) | 30.92 ( > nan) | > | 04/2014 | 1510 ( 36.53) | 448 ( 30.99) | 29.67 ( > -4.05) | > | 10/2014 | 1893 ( 25.36) | 506 ( 12.95) | 26.73 ( > -9.91) | > | 04/2015 | 2169 ( 14.58) | 548 ( 8.30) | 25.27 ( > -5.48) | > | 10/2015 | 2759 ( 27.20) | 619 ( 12.96) | 22.44 ( > -11.20) | > | 04/2016 | 3284 ( 19.03) | 652 ( 5.33) | 19.85 ( > -11.51) | > | 10/2016 | 3517 ( 7.10) | 801 ( 22.85) | 22.78 ( > 14.71) | > | 04/2017 | 3191 ( -9.27) | 427 ( -46.69) | 13.38 ( > -41.25) | > | 10/2017 | 2430 ( -23.85) | 420 ( -1.64) | 17.28 ( > 29.16) | > +--+---+---+ > ---+ > > Election CIVS links > 10/2014: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > c105db929e6c11f4 > 04/2015: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > ef1379fee7b94688 > 10/2015: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > 4ef58718618691a0 > 04/2016: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > fef5cc22eb3dc27a > 10/2016: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > 356e6c1b16904010 > 04/2017: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > 072c4cd7ff0673b5 > 10/2017: http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_ > ce86063991ef8aae > > I don't have a feel for with the Pike electorate decreased but my gut > feel is that it was organic drop-off possibly in part to the shorter > Ocata development cycle. The Queens drop-off was due to a new[1] > membership API being available that meant we could validate Foundation > membership instead of using gerrit permission as a proxy. > > I'd like to call out that with Pike we had a very dramatic decrease in > voter turnout both in absolute and relative terms. As a community it's > worth trying to understand whether this is a problem and/or a trend that > needs to change. > > Yours Tony. > > [1] It wasn't that new it was also used during the PTL election[2] > [2] See: > http://lists.openstack.org/pipermail/openstack-dev/2017- > July/119786.html ; and > http://lists.openstack.org/pipermail/openstack-dev/2017- > August/120544.html > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][infra][CI] Moving OVB jobs from RH1 cloud to RDO cloud, plan
We're currently running with a max-servers of 80 for the TripleO tenant. This number doesn't include OVB nodes. When taking into account OVB nodes, we are already nearing vCPU capacity and could consider raising the overcommit ratio from 2.0 to 4.0 to make use of the available RAM. See the rough maths in my comment here [1]. [1]: https://review.rdoproject.org/r/#/c/10249/1/nodepool/nodepool.yaml@133 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Oct 25, 2017 1:39 PM, "Ben Nemec" <openst...@nemebean.com> wrote: > Overall sounds good. A couple of comments inline. > > On 10/23/2017 05:46 AM, Sagi Shnaidman wrote: > >> Hi, >> >> as you know we prepare transition of all OVB jobs from RH1 cloud to RDO >> cloud, also a few long multinode upgrades jobs as well. We prepared a >> workflow of transition below, please feel free to comment. >> >> >> 1) We run one job (ovb-ha-oooq) on every patch in following repos: oooq, >> oooq-extras, tripleo-ci. We run rest of ovb jobs (containers and fs024) as >> experimental in rdo cloud for following repos: oooq, oooq-extras, >> tripleo-ci, tht, tripleo-common. It should cover most of our testing. This >> step is completed. >> >> Currently it's blocked by newton bug in RDO cloud: >> https://bugs.launchpad.net/heat/+bug/1626256 , where cloud release >> doesn't contain its fix: https://review.openstack.org/#/c/501592/ . From >> other side, the upgrade to Ocata release (which would solve this issue too) >> is blocked by bug: https://bugs.launchpad.net/tripleo/+bug/1724328 >> So we are in blocked state right now with moving. >> >> Next steps: >> >> 2) We solve all issues with running on every patch job (ovb-ha-oooq) so >> that it's passing (or failing exactly for same results as on rh1) for a 2 >> regular working days. (not weekend). >> 3) We should trigger experimental jobs in this time on various patches in >> tht and tripleo-common and solve all issues for experimental jobs so all >> ovb jobs pass. >> 4) We need to monitor all this time resources in openstack-nodepool >> tenant (with help of rhops maybe) and be sure that it has the capacity to >> run configured jobs. >> > > I assume we will have a max jobs limit in nodepool (or whatever we're > using for that purpose) that will ensure we stay within capacity regardless > of what jobs are configured. We probably want to keep that limit low > initially so we don't have to worry about throwing a huge number of jobs at > the cloud accidentally (say someone submits a large patch series that > triggers our subset of jobs). > > Obviously as we add jobs we'll need to bump the concurrent jobs limit, but > I think that should be the primary variable we change and that we add more > jobs as necessary to fill the configured limit. Also, rather than set a > time period of two days or whatever, ensure we run at the configured limit > for some period of time before increasing it. There are slow days in ci > where we might not get much useful information so we need to make sure we > don't get a false positive result from a step just because of the quirks of > ci load. > > 5) We set ovb-ha-oooq job as running for every patch in all places where >> it's running in rh1 (in parallel with existing rh1 job). We monitor RDO >> cloud that it doesn't fail and still have resources - 1.5 working days >> 6) We add featureset024 ovb job to run in every patch where it runs in >> rh1. We continue to monitor RDO cloud - 1.5 working days >> 7) We add last containers ovb job to all patches where it runs on rh1. We >> continue monitor RDO cloud - 2 days. >> 8) In case if everything is OK in all previous points and RDO cloud still >> performs well, we remove ovb jobs from rh1 configuration and make them as >> experimental. >> 9) During next few days we monitor ovb jobs and run rh1 ovb jobs as >> experimental to check if we have the same results (or better :) ) >> 10) OVB jobs on rh1 cloud stay in experimental pipeline in tripleo for a >> next month or two. >> >> -- >> Best regards >> Sagi Shnaidman >> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Migrating TripleO jobs to native Zuul v3
Reminder that this session is taking place in ~1h20minutes from the time of this email. I sent out Bluejeans invites to the people who signed up, if you didn't get one, please poke me on IRC (dmsimard). David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Wed, Oct 18, 2017 at 5:26 PM, David Moreau Simard <d...@redhat.com> wrote: > It looks like the best slot for everyone [1] is monday October 23rd @ > 10:00AM EST (2:00PM UTC). > > I'll send invites out, don't forget to start writing down things > you're interested in learning > about in the etherpad [2]. > > [1]: https://beta.doodle.com/poll/hfkcgrahwskm2ggv > [2]: https://etherpad.openstack.org/p/migrating-tripleo-zuulv3 > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] > > > On Mon, Oct 16, 2017 at 11:08 AM, David Moreau Simard <d...@redhat.com> > wrote: > > Hi, > > > > Unless you've been hiding under a rock (which is totally > > understandable), you know that Zuul v3 is here. > > > > Zuul v3 can be seen as a hindrance as of late because it's been > > messing up with the gate jobs or preventing your patches to merge, > > etc. > > Hopefully, the "legacy" Zuul v3 jobs are all fixed up now and things > > are able to merge without too much troubles. > > I'm here to tell you that Zuul v3 is in fact awesome and I'd like > > TripleO to benefit from all it's glory ASAP. > > > > I encourage everyone, not just people typically involved in CI, to > > take a read at the Zuul v3 migration guide [1]. > > Zuul v3 opens a lot of opportunities to streamline, improve and > > simplify the CI of TripleO in upstream. > > > > I'd like to open up an informal "ask me anything" regarding what's > > implied in properly migrating TripleO jobs to "native" Zuul v3 and > > have set up an etherpad to start collecting questions [2]. > > We could do a recorded Bluejeans session sometime early next week. > > I've set up a Doodle, please tell us what time would work best for you > [3]. > > > > Paul Belanger and I will be there to answer questions from a Zuul v3 > > and upstream infrastructure perspective. > > > > We might also have questions for you ! > > For example, how do we plan on keeping jobs "compatible" between > > review.openstack.org and review.rdoproject.org ? > > > > Thanks ! > > > > [1]: https://docs.openstack.org/infra/manual/zuulv3.html > > [2]: https://etherpad.openstack.org/p/migrating-tripleo-zuulv3 > > [3]: https://doodle.com/poll/hfkcgrahwskm2ggv > > > > David Moreau Simard > > Senior Software Engineer | OpenStack RDO > > > > dmsimard = [irc, github, twitter] > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Migrating TripleO jobs to native Zuul v3
It looks like the best slot for everyone [1] is monday October 23rd @ 10:00AM EST (2:00PM UTC). I'll send invites out, don't forget to start writing down things you're interested in learning about in the etherpad [2]. [1]: https://beta.doodle.com/poll/hfkcgrahwskm2ggv [2]: https://etherpad.openstack.org/p/migrating-tripleo-zuulv3 David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Oct 16, 2017 at 11:08 AM, David Moreau Simard <d...@redhat.com> wrote: > Hi, > > Unless you've been hiding under a rock (which is totally > understandable), you know that Zuul v3 is here. > > Zuul v3 can be seen as a hindrance as of late because it's been > messing up with the gate jobs or preventing your patches to merge, > etc. > Hopefully, the "legacy" Zuul v3 jobs are all fixed up now and things > are able to merge without too much troubles. > I'm here to tell you that Zuul v3 is in fact awesome and I'd like > TripleO to benefit from all it's glory ASAP. > > I encourage everyone, not just people typically involved in CI, to > take a read at the Zuul v3 migration guide [1]. > Zuul v3 opens a lot of opportunities to streamline, improve and > simplify the CI of TripleO in upstream. > > I'd like to open up an informal "ask me anything" regarding what's > implied in properly migrating TripleO jobs to "native" Zuul v3 and > have set up an etherpad to start collecting questions [2]. > We could do a recorded Bluejeans session sometime early next week. > I've set up a Doodle, please tell us what time would work best for you [3]. > > Paul Belanger and I will be there to answer questions from a Zuul v3 > and upstream infrastructure perspective. > > We might also have questions for you ! > For example, how do we plan on keeping jobs "compatible" between > review.openstack.org and review.rdoproject.org ? > > Thanks ! > > [1]: https://docs.openstack.org/infra/manual/zuulv3.html > [2]: https://etherpad.openstack.org/p/migrating-tripleo-zuulv3 > [3]: https://doodle.com/poll/hfkcgrahwskm2ggv > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] Issue with Zuul v3 / Gerrit / TripleO Jobs
This was confirmed to be a client-side javascript issue in Gerrit earlier this week [1]. [1]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-10-16.log.html#t2017-10-16T12:51:40 David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Wed, Oct 18, 2017 at 2:30 PM, Emilien Macchi <emil...@redhat.com> wrote: > Since we migrated to Zuul v3 and Zuul is now the default user which > vote in Gerrit, we now have a bug with jobs running in the 'check' and > the 'check-tripleo' pipeline. > Jobs finish in the 'check' pipeline and report their status and then > jobs finish in the 'check-tripleo' pipeline, overwriting/discarding > the results from the 'check' pipeline. > In v2, the check pipeline results were 'additive' but not in v3, it seems. > > It might be some javascript display in Gerrit but we haven't figured out. > > Which means... please be careful when you review and check both > pipelines results and not the latest one displayed in Gerrit. Please > scroll-down to the comments to see first pipeline results as well, so > we make sure we're not merging patches that didn't pass some > non-voting jobs (specially the ones in check-tripleo). > > Thank you, > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO] Migrating TripleO jobs to native Zuul v3
Hi, Unless you've been hiding under a rock (which is totally understandable), you know that Zuul v3 is here. Zuul v3 can be seen as a hindrance as of late because it's been messing up with the gate jobs or preventing your patches to merge, etc. Hopefully, the "legacy" Zuul v3 jobs are all fixed up now and things are able to merge without too much troubles. I'm here to tell you that Zuul v3 is in fact awesome and I'd like TripleO to benefit from all it's glory ASAP. I encourage everyone, not just people typically involved in CI, to take a read at the Zuul v3 migration guide [1]. Zuul v3 opens a lot of opportunities to streamline, improve and simplify the CI of TripleO in upstream. I'd like to open up an informal "ask me anything" regarding what's implied in properly migrating TripleO jobs to "native" Zuul v3 and have set up an etherpad to start collecting questions [2]. We could do a recorded Bluejeans session sometime early next week. I've set up a Doodle, please tell us what time would work best for you [3]. Paul Belanger and I will be there to answer questions from a Zuul v3 and upstream infrastructure perspective. We might also have questions for you ! For example, how do we plan on keeping jobs "compatible" between review.openstack.org and review.rdoproject.org ? Thanks ! [1]: https://docs.openstack.org/infra/manual/zuulv3.html [2]: https://etherpad.openstack.org/p/migrating-tripleo-zuulv3 [3]: https://doodle.com/poll/hfkcgrahwskm2ggv David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Proposing Alfredo Moralejo core on puppet-openstack-integration
+1, Alfredo rocks. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Oct 13, 2017 at 1:21 PM, Emilien Macchi <emil...@redhat.com> wrote: > Alfredo has been doing an incredible work on maintaining Puppet > OpenStack CI; by always testing OpenStack from trunk and taking care > of issues. He has been involved in fixing the actual CI problems in > OpenStack projects but also maintaining puppet-openstack-integration > repository in a consistent and solid manner. > Also, he has an excellent understanding how things work in this > project and I would like to propose him part of p-o-i maintainers > (among the rest of the whole core team and also dmsimard). > > As usual, feel free to vote and give feedback. > > Thanks, > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] grafana doesn't display result of periodic job
This is a known issue of the migration to Zuul v3. The metric data from jobs (duration, status, etc) running under Zuul v3 is not yet available. It's something we still need to add support for. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Oct 3, 2017 5:14 PM, "fumihiko kakuma" <kak...@valinux.co.jp> wrote: > Hi infra team, > > I noticed that grafana doesn't display the result of periodic jobs[1] > since 9/29. > Health results are also similar[2]. > Does the periodic job work? > > Thanks, > kakuma > > [1] http://grafana.openstack.org/dashboard/db/neutron-failure- > rate?panelId=4 > [2] http://status.openstack.org/openstack-health/#/job/ > periodic-tempest-dsvm-neutron-with-ryu-master-ubuntu-xenial?duration=P14D > http://status.openstack.org/openstack-health/#/job/ > periodic-neutron-dynamic-routing-dsvm-tempest-with-ryu- > master-scenario-ipv4?duration=P14D > > -- > fumihiko kakuma <kak...@valinux.co.jp> > > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] Zuul v3 and you: FAQs and how to troubleshoot your new (and legacy) jobs
Jim Blair rightfully pointed out that fleshed out documentation of the Zuul v3 documentation can be found in the infra-manual docs [1] which everyone should definitely read. The etherpad is not meant as a replacement for the documentation but rather as a collaborative effort around the questions which are most often asked or are not covered (yet) in the infra-manual. [1]: https://docs.openstack.org/infra/manual/zuulv3.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Sep 29, 2017 at 10:16 AM, David Moreau Simard <d...@redhat.com> wrote: > Hi ! > > So Zuul v3, years in the making, is now live. > > Congrats to everyone involved in making it happen and thanks everyone > else for their patience, even now as we work towards fully migrating and > fixing problems. > > In the interest of keeping this short as possible, Zuul jobs are now driven > by Ansible playbooks and roles. > > We realize that not everyone is familiar with Ansible, much less Zuul v3, > so I've started an etherpad with some FAQs and tips on how to get started. > > You'll find the etherpad here [1]. > > [1]: https://etherpad.openstack.org/p/zuulv3-migration-faq > > David Moreau Simard > Senior Software Engineer | OpenStack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Zuul v3 and you: FAQs and how to troubleshoot your new (and legacy) jobs
Hi ! So Zuul v3, years in the making, is now live. Congrats to everyone involved in making it happen and thanks everyone else for their patience, even now as we work towards fully migrating and fixing problems. In the interest of keeping this short as possible, Zuul jobs are now driven by Ansible playbooks and roles. We realize that not everyone is familiar with Ansible, much less Zuul v3, so I've started an etherpad with some FAQs and tips on how to get started. You'll find the etherpad here [1]. [1]: https://etherpad.openstack.org/p/zuulv3-migration-faq David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Garbage patches for simple typo fixes
Wow, Matt, that's excellent timing. Day for day, an exact year after the last thread of this kind [1]. Thanks for speaking up, I didn't want to it'd seem like encouraging a stereotype or prejudice towards certain kind of contributions (or contributors) but I've also been rolling my eyes a lot recently. It does look like those users are just seeking to pad statistics, but this isn't the first time this kind of topic comes up. Some other contributors with similar patterns: - https://review.openstack.org/#/q/author:%255E.*unionpay.* - https://review.openstack.org/#/q/author:%255E.*fiberhome.* - https://review.openstack.org/#/q/author:%255E.*sina.* There are some thoughts in the thread from last year but I don't think any concrete measures were put in place to encourage better and more meaningful contributions. [1]: http://lists.openstack.org/pipermail/openstack-dev/2016-September/104173.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Thu, Sep 21, 2017 at 10:21 PM, Matt Riedemann <mriede...@gmail.com> wrote: > I just wanted to highlight to people that there seems to be a series of > garbage patches in various projects [1] which are basically doing things > like fixing a single typo in a code comment, or very narrowly changing http > to https in links within docs. > > Also +1ing ones own changes. > > I've been trying to snuff these out in nova, but I see it's basically a > pattern widespread across several projects. > > This is the boilerplate comment I give with my -1, feel free to employ it > yourself. > > "Sorry but this isn't really a useful change. Fixing typos in code comments > when the context is still clear doesn't really help us, and mostly seems > like looking for padding stats on stackalytics. It's also a drain on our CI > environment. > > If you fixed all of the typos in a single module, or in user-facing > documentation, or error messages, or something in the logs, or something > that actually doesn't make sense in code comments, then maybe, but this > isn't one of those things." > > I'm not trying to be a jerk here, but this is annoying to the point I felt > the need to say something publicly. > > [1] https://review.openstack.org/#/q/author:%255E.*inspur.* > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?
On Tue, Sep 19, 2017 at 9:03 AM, Jeremy Stanley <fu...@yuggoth.org> wrote: > > In order to reduce image sizes and the time it takes to build > images, once we had local package caches in each provider we stopped > pre-retrieving packages onto the images. Is the time spent at this > stage mostly while downloading package files (which is what that > used to alleviate) or is it more while retrieving indices or > installing the downloaded packages (things having them pre-retrieved > on the images never solved anyway)? > At what point does it become beneficial to build more than one image per OS that is more aggressively tuned/optimized for a particular purpose ? We could take more freedom in a devstack-specific image like pre-install packages that are provided out of base OS, etc. Different projects could take this kind of freedom to optimize build times according to their needs as well. Here's an example of something we once did in RDO: 1) Aggregate the list of every package installed (rpm -qa) at the end of several jobs 2) From that sorted and uniq'd list, work out which repositories each package came from 3) Blacklist every package that was not installed from a base operating system repository (i.e, blacklist every package and dependencies from RDO, since we'll be testing these) 4) Pre-install every package that were not blacklisted in our images The end result was a list of >700 packages [1] completely unrelated to OpenStack that ended up being installed anyway throughout different jobs. To give an idea of numbers, a fairly vanilla CentOS image has ~400 packages installed. You can find the (rudimentary) script to achieve this filtering is here [2]. [1]: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/nodepool/scripts/weirdo-packages.txt [2]: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/nodepool/scripts/filter_packages.sh David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Release-job-failures][ara] Release of openstack/ara failed
Ack, I must have missed that. Thanks for letting me know. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Sep 19, 2017 3:22 PM, "Doug Hellmann" <d...@doughellmann.com> wrote: Excerpts from jenkins's message of 2017-09-19 19:10:12 +: > Build failed. > > - ara-tarball http://logs.openstack.org/89/89289c511c8c48c8409dd1fb997fc2 4e64d1dae6/release/ara-tarball/a75a87f/ : SUCCESS in 3m 17s > - ara-tarball-signing http://logs.openstack.org/89/ 89289c511c8c48c8409dd1fb997fc24e64d1dae6/release/ara- tarball-signing/f812ce9/ : FAILURE in 41s > - ara-pypi-both-upload http://logs.openstack.org/89/ 89289c511c8c48c8409dd1fb997fc24e64d1dae6/release/ara-pypi- both-upload/b3367cd/ : SUCCESS in 19s > - hook-ara-rtfd http://logs.openstack.org/89/ 89289c511c8c48c8409dd1fb997fc24e64d1dae6/release/hook-ara-rtfd/574ecf9/ : SUCCESS in 4s > I think we're still under release freeze until the infra team gives us the all-clear, but when that's done the ara team may want to re-release this version (or at least coordinate with the infra team to re-run the failed signing job). Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] TripleO/Ansible PTG session
On Mon, Sep 18, 2017 at 2:14 PM, Joshua Harlow <harlo...@fastmail.com> wrote: > Was there any discussion at the PTG on how the newly released AWX[1] will > affect tripleo/ansible (will it?) or ara or such? Thoughts there? > > [1] https://github.com/ansible/awx I wasn't at this particular session (wish I could've cloned myself several times to attend everything) but AWX doesn't change anything as far as ARA is concerned. The use cases are different. One provides things like ACL/RBAC/authentication/reporting/auditing/running/editing/etc but also requires memcached, postgres, rabbitmq, a dedicated server and running your playbooks through it. The other one provides... reporting and requires 'pip install ara' on your laptop :) The folks from Ansible and Tower have reached out to me about collaborating and talked about integration with AWX/Tower. I don't know yet what this means and what it will translate to but I'll have an idea in the next few weeks. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] [openstack-dev] [PTG] Video interviews at PTG, now scheduling
Rich, Where are the interviews taking place ? David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Tue, Aug 22, 2017 at 8:43 AM, Rich Bowen <rbo...@redhat.com> wrote: > Now that the PTG schedule is up, I'd like to invite you to sign up for > my video interview series. I'll be conducting interviews at the PTG, > with the goal of: > > * Telling our users what's new in Pike, and what to expect in Queens > * Put a human face on the upstream developer community > * Show the awesome cooperation and collaboration between projects and > between companies > > Sign up at: > https://docs.google.com/spreadsheets/d/1KNHuo9Yb5kbjZAYGQ_PAo-YFndD8QTdaKzaPoct_aaU/edit?usp=sharing > > A few tips: > > * Talk with your project about who should do the interview, and what you > want to highlight. > * Consider having several people sign up to do an interview (no more > than 3, please) > * Consider doing an interview that is cross-project, to talk about this > collaboration > * If you know of a user/customer who will be at the PTG who has a cool > use-case, encourage them to sign up > * Please read the notes on the "Planning for your interview" tab of the > spreadsheet. > > If you'd like to see examples of what I'm going for, I did this on a > much smaller scale in Atlanta, with just Red Hat engineers. Please see: > https://www.youtube.com/watch?v=5kT-Sv3rkTw=PLOuHvpVx7kYksG0NFaCaQsSkrUlj3Oq4S > > Thanks! > > -- > Rich Bowen - rbo...@redhat.com > RDO Community Liaison > http://rdoproject.org > @RDOCommunity > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [PTG] Video interviews at PTG, now scheduling
Rich, Where are the interviews taking place ? David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Tue, Aug 22, 2017 at 8:43 AM, Rich Bowen <rbo...@redhat.com> wrote: > Now that the PTG schedule is up, I'd like to invite you to sign up for > my video interview series. I'll be conducting interviews at the PTG, > with the goal of: > > * Telling our users what's new in Pike, and what to expect in Queens > * Put a human face on the upstream developer community > * Show the awesome cooperation and collaboration between projects and > between companies > > Sign up at: > https://docs.google.com/spreadsheets/d/1KNHuo9Yb5kbjZAYGQ_PAo-YFndD8QTdaKzaPoct_aaU/edit?usp=sharing > > A few tips: > > * Talk with your project about who should do the interview, and what you > want to highlight. > * Consider having several people sign up to do an interview (no more > than 3, please) > * Consider doing an interview that is cross-project, to talk about this > collaboration > * If you know of a user/customer who will be at the PTG who has a cool > use-case, encourage them to sign up > * Please read the notes on the "Planning for your interview" tab of the > spreadsheet. > > If you'd like to see examples of what I'm going for, I did this on a > much smaller scale in Atlanta, with just Red Hat engineers. Please see: > https://www.youtube.com/watch?v=5kT-Sv3rkTw=PLOuHvpVx7kYksG0NFaCaQsSkrUlj3Oq4S > > Thanks! > > -- > Rich Bowen - rbo...@redhat.com > RDO Community Liaison > http://rdoproject.org > @RDOCommunity > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [devstack] SUSE jobs started failing on peakmem_tracker
Hi, I was trying to make sure the existing openSUSE jobs passed on Zuul v3 but even the regular v2 jobs are hitting a bug I filed here [1]. As far as I know, these jobs were passing until recently. This is preventing us from sanity checking that everything works out of the box for the suse devstack job for the v3 migration. I've disabled this service through localconf in the v3 job for the time being [2] to test further. [1]: https://bugs.launchpad.net/devstack/+bug/1716066 [2]: https://review.openstack.org/#/c/502147/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[Openstack-operators] RDO at the OpenStack PTG
Hi! The RDO community project [1] provides RPM OpenStack packages for deploying on RHEL and CentOS. RDO is analogous to Ubuntu Cloud Archive (UCA) which provides OpenStack packages for installing on Ubuntu. I'm no marketing or sales guy, I'm part of the RDO engineering team in charge of building, packaging, testing and shipping RDO. If you happen to be at the PTG and would like to chat about RDO, please let me know off-thread so we can set aside some time. I'm always excited to talk with users and operators ! Thanks and see you there ! [1]: http://rdoproject.org David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [openstack-dev] [tripleo] PTG Agenda draft - action required
On Sep 1, 2017 9:21 AM, "Giulio Fidente" <gfide...@redhat.com> wrote: - upgrade ceph to luminous (maybe also in Kolla) I think this is a no-brainer as Luminous is the new Ceph LTS release and brings many improvements like bluestore so we'll want to implement it in Queens ASAP. It's a good thing to discuss how we want to do this upgrade from Pike to Queens (i.e, with ceph-ansible?) though. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] Infra Team PTG Dinner
Do we want to set up a scheduling doodle [1] or something to get the best possible time ? As the PTG draws near, different events are being set up. There's the RDO release party wednesday [2] (everyone is invited!) and I'm trying to juggle tuesday right now. [1]: http://doodle.com/ [2]: https://www.redhat.com/archives/rdo-list/2017-August/msg00057.html David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Wed, Aug 23, 2017 at 5:21 PM, Clark Boylan <cboy...@sapwetik.org> wrote: > Hello, > > I brought this up briefly in yesterday's meeting, but are we interested > in having a team dinner while at the PTG? I expect we are so I've tried > to start putting information together at > https://etherpad.openstack.org/p/infra-ptg-team-dinner. > > The biggest issue is there does not appear to be a lot of options near > the PTG hotel. That said, there appears to be a good beer garden option > about 4 miles away. Beer gardens were a hit in Darmstadt because they > can accomodate large groups, we don't have to pre negotiate reservations > or who is paying, and of course because beer. > > If interested please add your name to the list on the etherpad and the > evenings you are available for the dinner. Also, if anyone else has > better ideas (seriously I'm not great at this) or wants to organize go > ahead and edit the etherpad and/or let me know. > > Thanks, > Clark > > ___ > OpenStack-Infra mailing list > OpenStack-Infra@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[openstack-dev] [TripleO] What's the plan for shipping Pike TripleO containers ?
Hi, At the the last few RDO meetings [1][2][3], we brought up that no one had stepped up to do the work to build and ship what ends up being stable TripleO containers for Pike. There was different mentions that we should bring this to the MLs but we never did, so here it is. TL;DR, Do we want stable TripleO containers ? Who's going to be building and shipping them ? Where ? From my perspective, RDO is in kind of a weird spot: - Technically speaking, the RDO project and community provides packages for installers (such as TripleO) to consume - The container images are only consumable by the TripleO project - The tooling to build containers are included in Kolla and TripleO which are packaged by RDO - There has been resistance in TripleO to adopt containers that have been built by RDO during the Pike development cycle because TripleO should be the upstream - Upstream projects, such as TripleO and Kolla, are already producing and shipping containers derived from RDO to DockerHub and tarballs.openstack.org The definition of done for shipping RDO releases [4] includes successful testing coverage of different Packstack and TripleO scenarios, the same scenarios that continuously run throughout the development cycle. I'll concede that the current definition of done (strangely) doesn't include bits around packaging -- how packages are built and shipped. It probably should and maybe this should be part of the discussion. With my RDO community hat on, I can't help but feel that it's important to keep RDO as agnostic, neutral and open as possible. We have to be careful about any bias and direct or indirect special treatment RDO might be providing to TripleO. This ensures a fair and level playing field for other projects that are interested in either using/consuming RDO or want to be included in the packaging distribution. This is what makes it easier for RDO to "grow": not just in terms of packages and contributors but in mind and market share. RDO allows operators deploy their OpenStack clouds with vanilla packages for Red Hat based distributions, no matter their software, hardware, hypervisor, drivers, backends and installer preferences. Now, back with my TripleO hat on, someone has to do the work. If we want to build and ship containers to DockerHub, we can do that already and automatically through periodic builds. The role that we have been using to build containers in RDO supports pushing to any registry, including DockerHub. If we want to build and ship containers to the CentOS official registry, "registry.centos.org", it's more work [5]. The container pipeline runs from Jenkins and builds Dockerfiles out of pseudo dist-git repositories. We would need to: - Adapt the tooling that we have to generate only Dockerfiles (~trivial) - Push them out in an organized fashion - Explicitly define the dependency and build order of the containers (complicated) It's probably also worth asking if we want to be shipping stable containers at all ? Who will be the users of those stable containers ? The tooling to build containers is included in (stable) packages provided by RDO, we have packages for Kolla and TripleO. Users will already be pushing containers to the TripleO undercloud registry, perhaps they could be expected to build the containers as well ? Let's discuss and figure out the plan moving forward. Thanks, [1]: http://eavesdrop.openstack.org/meetings/rdo_meeting___2017_08_09/2017/rdo_meeting___2017_08_09.2017-08-09-15.00.log.html#l-207 [2]: http://eavesdrop.openstack.org/meetings/rdo_meeting___2017_08_16/2017/rdo_meeting___2017_08_16.2017-08-16-15.00.log.html#l-33 [3]: http://eavesdrop.openstack.org/meetings/rdo_meeting___2017_08_23/2017/rdo_meeting___2017_08_23.2017-08-23-15.00.log.html#l-194 [4]: https://www.rdoproject.org/blog/2016/05/technical-definition-of-done/ [5]: https://wiki.centos.org/ContainerPipeline David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] CI design session at the PTG
On Mon, Aug 28, 2017 at 10:25 AM, Wesley Hayutin <whayu...@redhat.com> wrote: > +1 from me, I'm sure John, Sagi, and Arx are also interested. Yes, of course, I just went with whom I knew were going to the PTG. Anyone else is welcome to join as well ! David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Mon, Aug 28, 2017 at 10:25 AM, Wesley Hayutin <whayu...@redhat.com> wrote: > > > On Mon, Aug 28, 2017 at 10:19 AM, Paul Belanger <pabelan...@redhat.com> > wrote: >> >> On Mon, Aug 28, 2017 at 09:42:45AM -0400, David Moreau Simard wrote: >> > Hi, >> > >> > (cc whom I would at least like to attend) >> > >> > The PTG would be a great opportunity to talk about CI design/layout >> > and how we see things moving forward in TripleO with Zuul v3, upstream >> > and in review.rdoproject.org. >> > >> > Can we have a formal session on this scheduled somewhere ? >> > >> Wednesday onwards likely is best for me, otherwise, I can find time during >> Mon-Tues if that is better. >> > > +1 from me, I'm sure John, Sagi, and Arx are also interested. > > Thanks > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] CI design session at the PTG
On Mon, Aug 28, 2017 at 10:58 AM, Emilien Macchi <emil...@redhat.com> wrote: > Yeah, this session would be interesting to do. > Feel free to add it on https://etherpad.openstack.org/p/tripleo-ptg-queens > We need to work on scheduling before the PTG but it would likely > happen between Wednesday and Friday morning. Good idea, I've added it to the etherpad [1] and I've created a pad for the session as well [2]. [1]: https://etherpad.openstack.org/p/tripleo-ptg-queens [2]: https://etherpad.openstack.org/p/tripleo-ptg-queens-ci David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO] CI design session at the PTG
Hi, (cc whom I would at least like to attend) The PTG would be a great opportunity to talk about CI design/layout and how we see things moving forward in TripleO with Zuul v3, upstream and in review.rdoproject.org. Can we have a formal session on this scheduled somewhere ? David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[OpenStack-Infra] Can we add ARA to the Zuul v3 projects ?
Hi, There are major changes coming to ARA 1.0 [1] and it currently has no Zuul v3 test coverage at all. I'd like to add coverage with the same approach Monty is taking here [2]. Is it possible for ARA to use Zuul v3 with an in-repo .zuul.yaml file ? Thanks ! [1]: https://github.com/openstack/ara/compare/feature/1.0 [2]: https://review.openstack.org/#/c/498209/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [kolla] [tripleo] [openstack-ansible] [deployment] Collaboration at PTG
Good idea for the etherpad. I've added an item about the oslo/etcd integration we talked about while back. It seemed like a common interest to align how configuration is achieved between deployment projects. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Fri, Aug 25, 2017 at 8:00 PM, Emilien Macchi <emil...@redhat.com> wrote: > I created an etherpad: https://etherpad.openstack.org/p/deployment-queens > > Please bring topics (and names) - we'll try to be prepared before the > PTG with an agenda and some scheduling. > > Thanks, > > On Fri, Aug 25, 2017 at 4:19 PM, Michał Jastrzębski <inc...@gmail.com> wrote: >> We (Kolla) planned some time for that discussion:) It would be awesome >> if we could have them on Mon-Tue, because Wed-Fri we'll have >> kolla-specific design room. That being said if needed we can use it >> for our cross-community discussions. >> >> Biggest one for me is new direction of tripleo (k8s+ansible) and how >> that corresponds to kolla-k8s (k8s+ansible). >> >> On 25 August 2017 at 15:53, Jean-Philippe Evrard >> <jean-phili...@evrard.me> wrote: >>> Hello Emilien, >>> >>> The Discussion room is a good idea. I like it. >>> Most of the OpenStack-Ansible crew will be available the whole week, so we >>> can even think of doing a conversation outside the Wed-Friday timeframe. >>> >>> If you/we all have enough time, maybe we could organise two sessions, >>> probably with different formats? >>> For example, a brainstorming session about how we collaborated on previous >>> cycles and how we could collaborate in the future, and another session with >>> the real action points based on the first conversation? >>> >>> On top of that, I have extra points I'd like to discuss with you: >>> - Architecture of LB + web server + uwsgi >>> - Possibility of sharing infrastructure (mariadb/rabbitmq/...) >>> experience/code between projects. >>> >>> Thank you in advance. >>> >>> Best regards, >>> JP >>> >>> On Fri, Aug 25, 2017 at 8:16 PM, Emilien Macchi <emil...@redhat.com> wrote: >>>> >>>> Cool, sounds like some people are interested (I haven't hear from >>>> Kolla yet but I'm sure they are as well). >>>> >>>> I was wondering if we should take benefit of Discussion Rooms, useful >>>> for inter-projects topics: >>>> https://ethercalc.openstack.org/Queens-PTG-Discussion-Rooms >>>> >>>> There is still some place, let me know what you think and we can block >>>> a slot (maybe 2h?) >>>> I want to hear from Kolla and OpenStack Ansible at least and know if >>>> you have schedule constraints otherwise I'll go ahead and block a >>>> slot. >>>> >>>> Thanks, >>>> >>>> On Fri, Aug 18, 2017 at 4:37 AM, Flavio Percoco <fla...@redhat.com> wrote: >>>> > On 17/08/17 10:24 -0500, Major Hayden wrote: >>>> >> >>>> >> On 08/17/2017 09:30 AM, Emilien Macchi wrote: >>>> >>> >>>> >>> If you're working on Kolla / OpenStack-Ansible - please let us know if >>>> >>> you have specific constraints on the schedule, so we can maybe block a >>>> >>> timeslot in the agenda from now. >>>> >>> We'll have a "Packaging" room which is reserved for all topics related >>>> >>> to OpenStack deployments, so we can use this one. >>>> >> >>>> >> >>>> >> I don't have any constraints (that I'm aware of), but I'd be interested >>>> >> in >>>> >> participating! Performance in the gate jobs has been one of my tasks >>>> >> lately >>>> >> and I'd like to see if we can collaborate there to make improvements >>>> >> without >>>> >> ruining infra's day. ;) >>>> >> >>>> >> As long as you can put up with a few Dad jokes, I'll be there. >>>> > >>>> > >>>> > ++ I'm interested in this topic too! >>>> > >>>> > Flavio >>>> > >>>> > -- >>>> > @flaper87 >>>> > Flavio Percoco >>>> > >>>> > >>>> > __ >>>> > OpenStack Development Mailing List (not for usage questi
Re: [OpenStack-Infra] Plan for devstack Zuul v3 jobs
On Fri, Aug 25, 2017 at 4:50 PM, James E. Blair <cor...@inaugust.com> wrote: > We create a devstack-legacy job in Zuul v3 which attempts to run > devstack-gate in the manner closest to that in which it runs today. > This means that it will use the Zuul-provided git repos rather than > performing its own git fetch operations, and supply config files and > environment variables which are compatible with the way Zuul v2 works. > > Simultaneously, we also create a new devstack job which utilizes all the > new features of Zuul v3 and is structured in the way we envisioned > earlier. We can start very simply here and avoid carrying all of the > design baggage from the earlier job. This will be a job that projects > can build off of and migrate to over time, once we have completed the > migration. I'm OK with this. Should we get to a point where the scripts can be used by both Zuul v2 and Zuul v3 simultaneously so that there is no "migration" or "cutover" so to speak ? It might mean some amount of work but it shouldn't be too bad, I think ? Ansible (Zuul v3) will potentially end up running Ansible but we've seen worse and it'll be temporary. It would allow for a smooth transition because we would just essentially stop running the v2 jobs when we are ready. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [tripleo] How to report tripleo-quickstart results to DLRN API
> Idea #3: Use "post-build scripts" in the promotion jobs. We can pattern match > for failed/passed jobs and report the result accordingly. The problem with > this is that it's environment dependent. While we can certainly do this with > post-build scripts in Jenkins Job Builder on CentOS CI, it's not clear how to > solve this in Zuul queues. Probably we just need to make the shell scripts of > the jobs more involved (not fail on quickstart.sh's nonzero exit). Besides > these complications, it also means that we have to keep the reporting method > in sync across multiple environments. So what you are talking about is basically the "postbuildscript" [1] publisher from Jenkins. It's how we are currently reporting both successes and failures with WeIRDO out of ci.centos.org jobs right now [2][3]. Interestingly enough, it so happens that I've asked #openstack-infra about "postbuildscript" recently, but for other purposes [4]. You're correct in saying that there's no such thing in the current Zuul implementation, however, it will be easy to accomplish in Zuul v3 which is planned to be deployed right before the PTG in less than a month. As far as how to do the implementation in TripleO Quickstart, I don't feel adding "DLRN API reporting" to tripleo-quickstart is a good fit. It has nothing to do with TripleO or TripleO Quickstart, it's just tooling that helps RDO keep track of what is working and what isn't. The logic to report to DLRN API should be deferred to a job or a separate playbook without "deep" integration with tripleo-quickstart IMO. That said, Zuul v3 is still a bit away so if we want to get started now, let's compromise on not reporting failures for the time being and revisit this after the PTG. [1]: https://docs.openstack.org/infra/jenkins-job-builder/publishers.html#publishers.postbuildscript [2]: https://github.com/rdo-infra/ci-config/blob/bcd70953fd69a2d375f81c301b6eca8b796becda/jenkins/jobs/weirdo-defaults.yml#L92 [3]: https://github.com/rdo-infra/weirdo/blob/master/playbooks/dlrn-api-report.yml [4]: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-08-08.log.html#t2017-08-08T18:58:54 David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] On Tue, Aug 22, 2017 at 11:22 AM, Attila Darazs <adar...@redhat.com> wrote: > Hi folks, > > I'm trying to come up with a good design for $subject, and there are several > different methods with pros and cons. I'd like to get your opinion about > them. > > For a bit of context, DLRN API[1] is a new extension of DLRN, our package > and repo building solution for RDO. It's designed to be a central point of > information about jobs that ran on certain hashes in various stages of > testing and handle "promotions", which are really just symlinks to certain > hashes. > > We want to report back job results on multiple levels (upstream, RDO CI > phase1 & phase2) and then use the information to promote new hashes at every > stage. > > If we would only be interested in reporting successful runs, the solution is > fairly simple: add a reporting step to the quickstart-extras.yml[2] playbook > at the end if a "report" variable is set. > > However it would be probably useful in the long term to also report back > failures (for statistics) and that's where things get complicated. > > It would be great if we could report the failed status within the same > quickstart.sh run instead of having a second run, because this way we don't > have to touch the shell scripts in multiple places (upstream, phase1, > phase2), just get the reporting done with config file changes. > > This is not simple, because the Ansible play can exit at any failed task. We > would need to wrap each task in rescue blocks[3] to avoid skipping the > failure. > > Idea #1: Create a "run successful" marker file at the reporting step, and > report failure in case the file is not found (also making sure the file does > not exist at the start of the run). This would still require multiple run of > ansible-playbook, but we could integrate the functionality into > quickstart.sh by creating a --report option, making it available at every > environment at the same time. > > Idea #2: Don't fail on *any* step, just register variables and check for > success. An example where we already do this is the overcloud-deploy role. > We don't fail on errors[4], but write out a file with the result and fail > later[5]. We would need to do this at almost all shell parts to be > reasonably certain we won't miss any failure. This requires a lot of > alterations to playbooks and it seems a bit forced on Ansible without the > usage of the rescue block, which we can't put in every single task
Re: [OpenStack-Infra] Opendayight Internship (Jenkins job builder) Week 12 status update August 14th - August 20th 2017
Hey Yolande, I see these emails every week and I just had to send some warm thanks for the work you've been doing on JJB. JJB is an important piece of software that's made my life easier when dealing with Jenkins or Zuul environments. So thanks again for your time and dedication to the project! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Aug 21, 2017 5:38 PM, "Yolande Amate" <yolandeam...@gmail.com> wrote: Hi, Last week I did the following: - Included tests and complete adding support for jira_changelog_trigger[1] - Added support for the Branch API Plugin[2] - Began adding support for CloudBees Docker Hub Notification Plugin(draft patch)[3] - Updated archive based on code review to use convert_mapping_to_xml()[4] This week, I plan to: - Complete adding support for CloudBees Docker Hub Notification Plugin - Add support for Email Extension Plugin - Add support for Jira Issue Updater Plugin - Respond to and do code review Attached below is a link to my Jenkins job builder project proposal document with a table of plugins(under activity details) that I have worked on so far[5]. Cheers, Yolande [1]https://review.openstack.org/#/c/493336/ [2]https://review.openstack.org/#/c/494031/ [3]https://review.openstack.org/#/c/494754/ [4]https://review.openstack.org/#/c/476960/ [5]https://docs.google.com/document/d/13YGxDwt76K5mhRUlmC_bzXK_ mCRSiVGStp-CJL4VYtA/edit# ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [kolla] [tripleo] [openstack-ansible] [deployment] Collaboration at PTG
Happy to join in case my perspective from infra and RDO can be useful! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Aug 17, 2017 10:32 AM, "Emilien Macchi" <emil...@redhat.com> wrote: > Hey folks, > > As usual, we'll meet in Denver and I hope we can spend some time > together (in a meeting room first) to have face to face discussions on > the recent topics that we had. > Right now, TripleO sessions are not scheduled in our agenda, so we're > pretty flexible: https://etherpad.openstack.org/p/tripleo-ptg-queens > > I would like to propose one topic (happy to coordinate the discussion) > on some efforts regarding doing configuration management with Ansible, > and k8s integration as well. > Flavio made some progress [1] - I really hope we can make progress here. > > [1] http://lists.openstack.org/pipermail/openstack-dev/2017- > July/119696.html > > If you're working on Kolla / OpenStack-Ansible - please let us know if > you have specific constraints on the schedule, so we can maybe block a > timeslot in the agenda from now. > We'll have a "Packaging" room which is reserved for all topics related > to OpenStack deployments, so we can use this one. > > Looking forward to meeting you at PTG! > Thanks, > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra]Plan to support Python 3.6 ?
On Wed, Aug 9, 2017 at 9:31 AM, Doug Hellmann <d...@doughellmann.com> wrote: > > Are Ubuntu LTS and CentOS shipping Python 3.6? > CentOS doesn't even carry python 3 at all right now, outside of perhaps python 3.4 from EPEL. Unless mistaken, RHEL 8 (and by extension CentOS 8) is bound to ship with 3.5. We're still a bit far away from RHEL 8. David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ara] Best practices on what to return from REST API ?
On Wed, Aug 9, 2017 at 10:02 AM, Chris Dent <cdent...@anticdent.org> wrote:
>
> If people remember to actually read this and respond, you'll likely get
> as many different responses to this as there are options.
>
I expected as much but I very much appreciate your reply regardless!
\o/
>
> From there rather than /{collection} for list and
> /{collection}?id={id} for single entity I'd do:
>
> /{collection} # list
> /{collection}?filter=to_smaller_list # e.g. playbook_id
> /{collection}/id # single entity
>
That makes sense.
>
> In the collection response only include the bare minimum of data, in
> the single entity include:
>
> - whatever is thought of the entity
> - anything that you think of as a child, represent as links
> elsewhere
> - it is makes sense to have /{collection}/id/{child} and {child} is
> actually some other /{collection}/id, consider making the child
> URI a redirect (that is /{collection/id/{child} redirects to
> /{childtype}/{child id}
> - if a child is actually children then you'll want to adjust the
> grammar somewhat: /{collection}/id/{child collection}/id
>
I've read about HATEOAS [1] and it's something I will likely implement.
So, if I'm understanding correctly, you would perhaps only include
references to the childs but not their actual data ?
With the concept of HATEOAS, it would go something like:
# Example task
{
"id": 8,
"links": [
{
"href": "/api/v1/tasks/8",
"rel": "self"
}
],
"name": "Ansible >=2.2: include_role tasks",
"playbook_id": 5,
"results": [
{
"href": "/api/v1/results/1", # or /api/v1/tasks/8/results/1
"rel": "1"
},
{
"href": "/api/v1/results/2",
"rel": "2"
},
# ...
]
}
I think that would be manageable and it wasn't something I had considered.
> This provides you with good urls for things, straightforward ways of
> getting at data, and the capacity to spawn off concurrent requests
> in response to an initial query.
>
> This is essentially the same as your option 2, but with more
> structure to the URLs.
Thanks for the input !
[1]: https://spring.io/understanding/HATEOAS
David Moreau Simard
Senior Software Engineer | OpenStack RDO
dmsimard = [irc, github, twitter]
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [ara] Best practices on what to return from REST API ?
Hi, So I'm making what I think is good progress towards the python and REST API implementation for ARA but now I have a question. I've made the following API "GET" endpoints: - files (saved Ansible files) - hosts (hosts involved in playbooks and their facts, if available) - playbooks (data about the actual playbook and ansible parameters) - plays (data about plays) - results (actual results like 'ok', 'changed' as well as the data from the Ansible module output) - tasks (data about the task, like name, action and file path) Each endpoint is self-sufficient and allows consumers to search individual endpoints as required, for example: (See attached links for actual and real output) - GET /api/v1/plays # [1] List all plays - GET /api/v1/plays?id=1 # [2] Get play with id 1 - GET /api/v1/plays?playbook_id=1 # [3] List all plays for playbook_id 1 The mechanism and logic is the same for all endpoints. Where I get a bit doubtful is because some components have relationships, like the following: - files have file contents (file.content) [4] - hosts have facts (host.facts) [5] - tasks have results (task.results) [6] - playbooks basically aggregate everything (playbook.files, playbook.hosts, playbook.plays, playbook.results, playbook.tasks) so it's potentially huge. So, what is the best practice here ? I'm thinking that returning all the available data on a LIST call (GET /api/v1/playbooks) is nuts because of the potential amount of data involved. I'm torn between two different approaches but I would be happy to be convinced there's a better way. 1) Only return "child" resources on specific call So, when doing a GET on something that is expected to return a list (GET /api/v1/playbooks), do not return child resources. And, when doing a GET on the specific resource (GET /api/v1/playbooks?id=1), display all the child resources. This would be sort of analogous to how python-openstackclient has different fields when doing "openstack server list" and "openstack server show". However, the amount of data for even a single playbook is likely very significant at scale. 2) Require sub calls for different components In this method, we would not return child resources when doing a list call (GET /api/v1/playbooks) or when doing a specific call (GET /api/v1/playbooks?id=1). To get child resources, you would need to pick them up one by one, ex: for playbook in playbooks: files = FileApi.get(playbook_id=playbook['id'] hosts = HostApi.get(playbook_id=playbook['id'] plays = PlayApi.get(playbook_id=playbook['id'] [...] I feel like approach #2 would be best -- for running at scale but also in the context of loading data asynchronously and concurrently. Any opinions ? Thanks ! [1]: http://paste.openstack.org/raw/617868/ [2]: http://paste.openstack.org/raw/617869/ [3]: http://paste.openstack.org/raw/617870/ [4]: http://paste.openstack.org/raw/617871/ [5]: http://paste.openstack.org/raw/617872/ [6]: http://paste.openstack.org/raw/617873/ David Moreau Simard Senior Software Engineer | OpenStack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] Ocata Updates at CentOS Repo
Hi, > Nova latest version is 15.0.6 but at CentOS repo ( > http://mirror.centos.org/centos/7/cloud/x86_64/openstack-ocata/) the > latest available version is 15.0.4 > > Does anyone know how long does it take to be updated? We are getting better at automatically building, testing and releasing new versions for stable release but it's still a work in progress. As you can see directly on the mirror [1] we have recently flushed a large batch of updates that were released on July 31st. As far as Nova is concerned, there was an issue with it that was resolved earlier this morning and it was tagged for release so it should hit the mirrors after package signing and CDN distribution. If you feel we're lagging behind on stable releases, please don't hesitate to ping us on #rdo on freenode or rdo-list [2]. We're doing our best to keep track of the 1500+ packages we are maintaining but sometimes it's possible one falls through the cracks :) Thanks, [1]: http://mirror.centos.org/centos/7/cloud/x86_64/openstack-ocata/?C=M;O=D [2]: https://www.redhat.com/mailman/listinfo/rdo-list David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
[openstack-dev] [TripleO] Non-voting job for testing container builds against tripleo-common
Hi, We temporarily stood up a third party job from review.rdoproject.org for the tripleo-common project to test that container builds are not broken as a result of a patch. For example, recently we took out EPEL with overrides and all containers built fine. After that, we introduced barbican which relied on EPEL and once more broke container builds [1]. This new job first builds a package for tripleo-common (DLRN-rpmbuild) and then builds containers (rdo-kolla-build-integration) with that new tripleo-common package using the overcloud_containers and overrides files from it. Since the job is third party, it is non-voting but I please ask core reviewers on tripleo-common to pay attention to the job results so that we don't break containers. You can see an example of the job running here [2]. If you have any questions, please reach out to me. Thanks, [1]: https://trello.com/c/KIO2UiWo/227-kolla-error-creating-barbican-api-container-image-after-disabling-epel [2]: https://review.openstack.org/#/c/489224/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] Zuul v3: Giant pile of thoughts around zuul-web things
Hi, A bit out of my comfort zone for the content of your giant pile of thoughts, but I'll jump on this bit: > We have a desire to add something that knows how to > render the job-output.json in some nice ways. The next things on my plate for ARA are: - Introduce the concept of API to abstract model access - Input drivers based on the API (callback, zuul json, mqtt, etc.) - Output drivers based on the API (html, junit, subunit, influx, graphite, etc.) This is no small task but I'm hoping to have made significant progress by the time the PTG rolls around. Paul showed interest in having those job-output.json files imported into ARA. I could see this taking a similar approach than how we are currently importing subunit files into openstack-health. I'll let you know if I have anything to show ! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [RDO] Major maintenance on review.rdoproject.org @ 2017-07-25 at 23:00 UTC
Hi, The maintenance is completed and jobs are once again running. Please note that pending additional changes that will be made ASAP, email delivery from review.rdoproject.org can either be delayed or not received at all. Other than email delivery, we've tested various jobs and everything seems to be in working order on that end. Please reach out to us by replying to this thread or on #rdo if you notice anything out of the ordinary. Thanks ! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Tue, Jul 25, 2017 at 10:46 AM, David Moreau Simard <d...@redhat.com> wrote: > Please note that this maintenance is re-scheduled to happen two hours > later, so at 2017-07-26 01:00 UTC. > > The maintenance window of two hours and the impacts remain the same. > > David Moreau Simard > Senior Software Engineer | Openstack RDO > > dmsimard = [irc, github, twitter] > > > On Sun, Jul 23, 2017 at 10:55 AM, David Moreau Simard <d...@redhat.com> wrote: >> Hi, >> >> (+openstack-dev for wider audience) >> >> Please note that we are planning a 2 hour maintenance window where >> review.rdoproject.org could be unavailable this next tuesday evening, >> July 25th at 23:00 UTC. >> We will proceed with an upgrade of the instance software and will >> migrate the review.rdoproject.org infrastructure to a new cloud >> environment. >> >> Please note that while this maintenance will have no impact on the >> availability of RDO mirrors, new package builds or tags could be >> delayed until the maintenance is finished. >> >> If you have any questions, please do not hesitate to reply here. >> >> Thanks ! >> >> David Moreau Simard >> Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [infra] Non-candidacy for Queens cycle Infrastructure PTL
Thanks Jeremy for all your great work ! Keep being awesome :) David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Jul 25, 2017 7:39 PM, "Jeremy Stanley" <fu...@yuggoth.org> wrote: It has been a pleasure and a privilege to captain the good ship Infrastructure for the past four cycles, but it's time to let someone else take a turn at the wheel. I am confident in the ability of my shipmates to step up to the responsibilities of the position (quite likely even better than I have done), and I look forward to assisting them as they steer us through the challenging waters ahead. I couldn't have accomplished everything I did without the aid and support of the rest of our crew, so it's only right I should give the same as I received. Thank you all for making it so much easier than I ever could have anticipated. I'm definitely not jumping ship, though! Much like my Infra PTL emeritus predecessors, I'm excited to be able to return to building, fixing and improving the systems on which our community relies so heavily day in and day out. This should hopefully also allow me to redouble efforts on my TC and VMT responsibilities, and I'm eager to be able to set course for long-awaited necessities such as consolidated authentication and finally putting an end to the ICLA. To those running for PTL, remember that we're only a few days away from the nomination period so start polishing your platforms; I eagerly await the opportunity to read through them all as a regular voter rather than a candidate for the first time in years! -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [RDO] Major maintenance on review.rdoproject.org @ 2017-07-25 at 23:00 UTC
Please note that this maintenance is re-scheduled to happen two hours later, so at 2017-07-26 01:00 UTC. The maintenance window of two hours and the impacts remain the same. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Sun, Jul 23, 2017 at 10:55 AM, David Moreau Simard <d...@redhat.com> wrote: > Hi, > > (+openstack-dev for wider audience) > > Please note that we are planning a 2 hour maintenance window where > review.rdoproject.org could be unavailable this next tuesday evening, > July 25th at 23:00 UTC. > We will proceed with an upgrade of the instance software and will > migrate the review.rdoproject.org infrastructure to a new cloud > environment. > > Please note that while this maintenance will have no impact on the > availability of RDO mirrors, new package builds or tags could be > delayed until the maintenance is finished. > > If you have any questions, please do not hesitate to reply here. > > Thanks ! > > David Moreau Simard > Senior Software Engineer | Openstack RDO > > dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [RDO] Major maintenance on review.rdoproject.org @ 2017-07-25 at 23:00 UTC
Hi, (+openstack-dev for wider audience) Please note that we are planning a 2 hour maintenance window where review.rdoproject.org could be unavailable this next tuesday evening, July 25th at 23:00 UTC. We will proceed with an upgrade of the instance software and will migrate the review.rdoproject.org infrastructure to a new cloud environment. Please note that while this maintenance will have no impact on the availability of RDO mirrors, new package builds or tags could be delayed until the maintenance is finished. If you have any questions, please do not hesitate to reply here. Thanks ! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo] [kolla] [rdo] EPEL and RDO in Kolla containers
Hi, Just FYI... I took a bit of time to document what seems to be most (if not all) dependencies that vanilla Kolla currently pulls from EPEL on CentOS Binary builds. This list is documented here [1]. TripleO doesn't use all the CentOS binary containers from Kolla, only a subset of those is relevant to the projects supported in TripleO right now. In that context, removing EPEL means adding a few overrides [2] to get the image builds to work in the first place. I haven't tested the containers deployed with these overrides so there might be issues, I'm not sure. There's some odd package mismatches due to the use of EPEL that haven't been picked up, too. For example, both "python2-msgpack" and "python2-crypto" are provided by EPEL, however these packages aren't available from RDO and are instead respectively named "python-msgpack" and "python-crypto". [1]: https://etherpad.openstack.org/p/kolla-epel [2]: https://review.openstack.org/#/c/48/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [OpenStack-Infra] Collected list of repos to retire
I don't want to sidetrack the thread but should someone also be taking
care of cleaning up stale branches in Devstack ?
It goes all the way back to kilo.
David Moreau Simard
Senior Software Engineer | Openstack RDO
dmsimard = [irc, github, twitter]
On Mon, Jul 10, 2017 at 11:05 AM, Andreas Jaeger <a...@suse.com> wrote:
> infra admins, could somebody please retire the following branches from
> repositories:
>
> openstack/sahara-extra stable/icehousePlease retire
> openstack/sahara-extra stable/mitaka Please retire
> openstack/packstack stable/kiloPlease retire
> openstack/packstack stable/liberty Please retire
> openstack/packstack stable/mitaka Please retire
> openstack/bareon-ironic stable/mitaka Please retire
> openstack/rpm-packaging stable/mitaka Please retire
> openstack/training-labs stable/mitaka Please retire
>
> Not done in
> https://gist.githubusercontent.com/tbreeds/c99e62bf8da19380e4eb130be8783be7/raw/6d02deb40e07516ce8fc529d2ba8c74af11a5a6b/mitaka_eol_data.txt
>
> openstack/astara stable/mitaka Please retire
>
>
> Special treatment:
> openstack/training-labs icehouse-eol (just delete branch, tag exists)
> openstack/training-labs juno-eol (delete branch, create tag instead)
>
> Note that packstack has eol tags already created.
>
> thanks,
> Andreas
> --
> Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [TripleO] Forming our plans around Ansible
That sounds like a good fit for an Ansible plugin to control how variables or host inventories are designed [1] and is the intended use case for extending Ansible behavior. [1]: http://docs.ansible.com/ansible/dev_guide/developing_plugins.html#vars-plugins David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Mon, Jul 10, 2017 at 9:31 AM, Steven Hardy <sha...@redhat.com> wrote: > On Sun, Jul 9, 2017 at 8:44 AM, Yolanda Robla Mota <yrobl...@redhat.com> > wrote: >> What i'd like to dig more is how Ansible and Heat can live together. And >> what features do Heat offer that are not covered by Ansible as well? Is >> there still the need to have Heat as the main engine, or could that be >> replaced by Ansible totally in the future? > > The main interface provided by Heat which AFAIK cannot currently be > replaced by Ansible is the parameters schema, where the template > parameters are exposed (that include description, type and constraint > data) in a format that is useful to e.g building the interfaces for > tripleo-ui > > Ansible has a different approach to role/playbook parameters AFAICT, > which is more a global namespace with no type validation, no way to > include description data or tags with variable declarations, and no > way to specify constraints (other than perhaps hainvg custom modules > or playbook patterns that perform the validations early in the > deployment). > > This is kind of similar to how the global namespace for hiera works > with our puppet model, although that at least has the advantage of > namespacing foo::something::variable, which again doesn't have a > direct equivalent in the ansible role model AFAIK (happy to be > corrected here, I'm not an ansible expert :) > > For these reasons (as mentioned in my reply to James), I think a first > step of a "split stack" model where heat deploys the nodes/networks > etc, then outputs data that can be consumed by Ansible is reasonable - > it leaves the operator interfaces alone for now, and gives us time to > think about the interface changes that may be needed long term, while > still giving most of the operator-debug and usability/scalabilty > benefits that I think folks pushing for Ansible are looking for. > > Steve > > > > >> On Sat, Jul 8, 2017 at 12:20 AM, James Slagle <james.sla...@gmail.com> >> wrote: >>> >>> On Fri, Jul 7, 2017 at 5:31 PM, David Moreau Simard <d...@redhat.com> >>> wrote: >>> > On Fri, Jul 7, 2017 at 1:50 PM, James Slagle <james.sla...@gmail.com> >>> > wrote: >>> >> (0) tripleo-quickstart which follows the common and well accepted >>> >> approach to bundling a set of Ansible playbooks/roles. >>> > >>> > I don't want to de-rail the thread but I really want to bring some >>> > attention to a pattern that tripleo-quickstart has been using across >>> > it's playbooks and roles. >>> > I sincerely hope that we can find a better implementation should we >>> > start developing new things from scratch. >>> >>> Yes, just to clarify...by "well accepted" I just meant how the git >>> repo is organized and how you are expected to interface with those >>> playbooks and roles as opposed to what those playbooks/roles actually >>> do. >>> >>> > I'll sound like a broken record for those that have heard me mention >>> > this before but for those that haven't, here's a concrete example of >>> > how things are done today: >>> > (Sorry for the link overload, making sure the relevant information is >>> > available) >>> > >>> > For an example tripleo-quickstart job, here's the console [1] and it's >>> > corresponding ARA report [2]: >>> > - A bash script is created [3][4][5] from a jinja template [6] >>> > - A task executes the bash script [7][8][9] >>> >>> From my limited experience, I believe the intent was that the >>> playbooks should do what a user is expected to do so that it's as >>> close to reproducing the user interface of TripleO 1:1. >>> >>> For example, we document users running commands from a shell prompt. >>> Therefore, oooq ought to do the same thing as close as possible. >>> Obviously there will be gaps, just as there is with tripleo.sh, but I >>> feel that both tools (tripleo.sh/oooq) were trying to be faithful to >>> our published docs as mush as possible, and I think there's something >>> to be commended there. >>> >>> Not saying it's right
[OpenStack-Infra] Removing EOL branches from Packstack
Hi -infra, We've tagged -eol for stable/kilo, stable/liberty as well as stable/mitaka for Packstack. We're a bit late! Sorry about that. The branches can be deleted now. Let me know if you have any questions, David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [openstack-dev] [TripleO] Forming our plans around Ansible
]: http://logs.openstack.org/99/477599/2/check/gate-openstack-ansible-openstack-ansible-ceph-centos-7-nv/c1efa30/logs/ara/ David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] weekly meetings on #tripleo
We have been holding RDO meetings in #rdo for as long as I can remember and I don't see anyone complaining. If someone asks a "support" type question during the meeting, we invite them to participate in the meeting and mention that we can help after the meeting is over. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Jul 5, 2017 1:41 PM, "Ben Nemec" <openst...@nemebean.com> wrote: > I don't have a particularly strong opinion one way or another (probably > because I tend to skip most of the meetings lately anyway...), but some > thoughts inline. > > On 07/05/2017 11:56 AM, Emilien Macchi wrote: > >> After reading http://lists.openstack.org/pip >> ermail/openstack-dev/2017-June/118899.html >> - we might want to collect TripleO's community feedback on doing >> weekly meetings on #tripleo instead of #openstack-meeting-alt. >> >> I see some direct benefits: >> - if you come up late in meetings, you could easily read backlog in >> #tripleo >> > > Most people idle in the meeting channels anyway, so +0 on this one. > > - newcomers not aware about the meeting channel wouldn't have to search >> for it >> > > We generally announce meetings in #tripleo before starting, don't we? > > - meeting would maybe get more activity and we would expose the >> information more broadly >> > > I guess this would preclude non-meeting related activity in #tripleo, > which in my experience is a thing that happens pretty frequently during the > meeting. Whether that's a good thing or not is a matter of opinion. :-) > > >> Any feedback on this proposal is welcome before we make any change (or >> not). >> >> Thanks, >> >> > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] How to deal with confusion around "hosted projects"
On Wed, Jun 28, 2017 at 5:50 PM, Monty Taylor <mord...@inaugust.com> wrote: > Such people are under the misguided impression that kicking cloudkitty out > of OpenStack will somehow cause Nova features to land quicker. I can't even > begin to express all of the ways in which it's wrong. So much this. I wonder where this perception is coming from, but I've definitely seen it around. Perhaps most notably from Mr. Shuttleworth [1] ? Kicking projects like Cloudkitty isn't going to make the developers working on Cloudkitty magically start working on Nova, Cinder or Keystone. [1]: http://www.eweek.com/cloud/ubuntu-linux-founder-tells-openstack-to-focus-on-things-that-matter David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] How to deal with confusion around "hosted projects"
Hi ! I thought my perspective might be valuable as the author of a project that would likely end up being slashed: ARA [1]. In a nutshell, ARA provides easy and seamless Ansible reporting on playbook runs. It has nothing to do with OpenStack, to be honest: it doesn't require OpenStack to run, it doesn't run on OpenStack. We highlight that ARA is not an "official" OpenStack project in the contributors' documentation [2]. But, it was born out of necessity inside the RDO community (call that the extended OpenStack community, if you will) due to the vast amount of our workloads that are driven by Ansible. I like to draw a parallel between ARA and JJB [3] (Jenkins Job Builder) as two tools that the OpenStack community develops and use without slapping the OpenStack brand on them. If we take a step back, would projects like Nodepool or Zuul also be affected by this kind of cut ? Where do we draw the line ? Are they OpenStack projects or not ? For me, applying for ARA to be hosted as a "community" project [4], or what would've been a "Stackforge" project long ago, was a completely logical choice to me. I could go and praise how the Gerrit workflow is vastly superior than GitHub pull requests or that the Zuul-driven CI is completely awesome. But... The truth is that I sincerely hoped it would help the project gain exposure and credibility in order to drive adoption *inside* the OpenStack community. We say that hindsight is 20/20, but, in retrospect [5], I guess we could say it worked. Maybe out of luck, maybe out of sheer dedication and long weekends spent working on the project. The reality is that today, ARA is used by many different projects inside the OpenStack ecosystem -- to name a few: - OpenStack-Ansible - TripleO - Browbeat - Kolla/Kolla-Ansible - Devstack-Gate But also, like Jenkins Job Builder, it has gained adoption outside of the OpenStack community -- such as inside the OpenShift Ansible community or BonnyCI. Being part of the OpenStack "ecosystem" allows ARA to do things like easily implement a gate job almost verbatim from OpenStack-Ansible [6] to make sure that each new commit doesn't break them. I feel that this is useful and powerful not just for ARA but for OpenStack-Ansible and the other projects that use it as well. Ironically, some projects feel hindered by this ecosystem (see: Gnocchi) and have made an exit. I do not share these feelings and I would in fact be very sad to be shown the door. My 0.02$CAD (not worth much right now) [1]: https://github.com/openstack/ara [2]: http://ara.readthedocs.io/en/latest/contributing.html#contributing [3]: https://github.com/openstack-infra/jenkins-job-builder [4]: https://review.openstack.org/#/c/321226/ [5]: https://dmsimard.com/2017/05/08/ara-is-one-year-old-a-look-back-at-the-past-year/ [6]: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ara.yaml#L21-L89 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Jun 28, 2017 6:46 PM, "James E. Blair" <cor...@inaugust.com> wrote: Thierry Carrez <thie...@openstack.org> writes: > Removing the root cause would be a more radical move: stop offering > hosting to non-OpenStack projects on OpenStack infrastructure > altogether. We originally did that for a reason, though. The benefits of > offering that service are: > > 1- it lets us set up code repositories and testing infrastructure before > a project applies to be an official OpenStack project. > > 2- it lets us host things that are not openstack but which we work on > (like abandoned Python libraries or GPL-licensed things) in a familiar > environment > > 3- it spreads "the openstack way" (Gerrit, Zuul) beyond openstack itself I think this omits what I consider the underlying reason for why we did it: It helps us build a community around OpenStack. Early on we had so many people telling us that we needed to support "ecosystem" projects better. That was the word they used at the time. Many of us said "hey, you're free to use github" and they told us that wasn't enough. We eventually got the message and invited them in, and it surpassed our expectations and I think surprised even the most optimistic of us. We ended up in a place where anyone with an OpenStack related idea can try it out and collaborate frictionlessly with everyone else in the OpenStack community on it, and in doing so, become recognized in the community for that. The ability for someone to build something on top of OpenStack as part of the OpenStack community has been empowering. I confess to being a skeptic and a convert. I wasn't thrilled about the unbounded additional responsibility when we started this, but now that we're here, I think it's one of the best things about the project and I would hate to cleave our community by ending it. -Jim ___
Re: [Openstack-operators] [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading
I've mentioned this elsewhere but writing here for posterity... Making N to N+1 upgrades seamless and work well is already challenging today which is one of the reasons why people aren't upgrading in the first place. Making N to N+1 upgrades work as well as possible already puts a great strain on developers and resources, think about the testing and CI involved in making sure things really work. My opinion is that of upgrades were made out to be a simple, easy and seamless operation, it wouldn't be that much of a problem to upgrade from N to N+3 by upgrading from release to release (three times) until you've caught up. But then, if upgrades are awesome, maybe operators won't be lagging 3 releases behind anymore. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Thu, May 25, 2017 at 9:55 PM, Carter, Kevin <ke...@cloudnull.com> wrote: > Hello Stackers, > > As I'm sure many of you know there was a talk about doing "skip-level"[0] > upgrades at the OpenStack Summit which quite a few folks were interested in. > Today many of the interested parties got together and talked about doing > more of this in a formalized capacity. Essentially we're looking for cloud > upgrades with the possibility of skipping releases, ideally enabling an N+3 > upgrade. In our opinion it would go a very long way to solving cloud > consumer and deployer problems it folks didn't have to deal with an upgrade > every six months. While we talked about various issues and some of the > current approaches being kicked around we wanted to field our general chat > to the rest of the community and request input from folks that may have > already fought such a beast. If you've taken on an adventure like this how > did you approach it? Did it work? Any known issues, gotchas, or things folks > should be generally aware of? > > > During our chat today we generally landed on an in-place upgrade with known > API service downtime and little (at least as little as possible) data plane > downtime. The process discussed was basically: > a1. Create utility "thing-a-me" (container, venv, etc) which contains the > required code to run a service through all of the required upgrades. > a2. Stop service(s). > a3. Run migration(s)/upgrade(s) for all releases using the utility > "thing-a-me". > a4. Repeat for all services. > > b1. Once all required migrations are complete run a deployment using the > target release. > b2. Ensure all services are restarted. > b3. Ensure cloud is functional. > b4. profit! > > Obviously, there's a lot of hand waving here but such a process is being > developed by the OpenStack-Ansible project[1]. Currently, the OSA tooling > will allow deployers to upgrade from Juno/Kilo to Newton using Ubuntu 14.04. > While this has worked in the lab, it's early in development (YMMV). Also, > the tooling is not very general purpose or portable outside of OSA but it > could serve as a guide or just a general talking point. Are there other > tools out there that solve for the multi-release upgrade? Are there any > folks that might want to share their expertise? Maybe a process outline that > worked? Best practices? Do folks believe tools are the right way to solve > this or would comprehensive upgrade documentation be better for the general > community? > > As most of the upgrade issues center around database migrations, we > discussed some of the potential pitfalls at length. One approach was to > roll-up all DB migrations into a single repository and run all upgrades for > a given project in one step. Another was to simply have mutliple python > virtual environments and just run in-line migrations from a version specific > venv (this is what the OSA tooling does). Does one way work better than the > other? Any thoughts on how this could be better? Would having N+2/3 > migrations addressable within the projects, even if they're not tested any > longer, be helpful? > > It was our general thought that folks would be interested in having the > ability to skip releases so we'd like to hear from the community to validate > our thinking. Additionally, we'd like to get more minds together and see if > folks are wanting to work on such an initiative, even if this turns into > nothing more than a co-op/channel where we can "phone a friend". Would it be > good to try and secure some PTG space to work on this? Should we try and > create working group going? > > If you've made it this far, please forgive my stream of consciousness. I'm > trying to ask a lot of questions and distill long form conversation(s) into > as little text as possible all without writing a novel. With that said, I > hope this finds you well, I look forward
Re: [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading
I've mentioned this elsewhere but writing here for posterity... Making N to N+1 upgrades seamless and work well is already challenging today which is one of the reasons why people aren't upgrading in the first place. Making N to N+1 upgrades work as well as possible already puts a great strain on developers and resources, think about the testing and CI involved in making sure things really work. My opinion is that of upgrades were made out to be a simple, easy and seamless operation, it wouldn't be that much of a problem to upgrade from N to N+3 by upgrading from release to release (three times) until you've caught up. But then, if upgrades are awesome, maybe operators won't be lagging 3 releases behind anymore. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Thu, May 25, 2017 at 9:55 PM, Carter, Kevin <ke...@cloudnull.com> wrote: > Hello Stackers, > > As I'm sure many of you know there was a talk about doing "skip-level"[0] > upgrades at the OpenStack Summit which quite a few folks were interested in. > Today many of the interested parties got together and talked about doing > more of this in a formalized capacity. Essentially we're looking for cloud > upgrades with the possibility of skipping releases, ideally enabling an N+3 > upgrade. In our opinion it would go a very long way to solving cloud > consumer and deployer problems it folks didn't have to deal with an upgrade > every six months. While we talked about various issues and some of the > current approaches being kicked around we wanted to field our general chat > to the rest of the community and request input from folks that may have > already fought such a beast. If you've taken on an adventure like this how > did you approach it? Did it work? Any known issues, gotchas, or things folks > should be generally aware of? > > > During our chat today we generally landed on an in-place upgrade with known > API service downtime and little (at least as little as possible) data plane > downtime. The process discussed was basically: > a1. Create utility "thing-a-me" (container, venv, etc) which contains the > required code to run a service through all of the required upgrades. > a2. Stop service(s). > a3. Run migration(s)/upgrade(s) for all releases using the utility > "thing-a-me". > a4. Repeat for all services. > > b1. Once all required migrations are complete run a deployment using the > target release. > b2. Ensure all services are restarted. > b3. Ensure cloud is functional. > b4. profit! > > Obviously, there's a lot of hand waving here but such a process is being > developed by the OpenStack-Ansible project[1]. Currently, the OSA tooling > will allow deployers to upgrade from Juno/Kilo to Newton using Ubuntu 14.04. > While this has worked in the lab, it's early in development (YMMV). Also, > the tooling is not very general purpose or portable outside of OSA but it > could serve as a guide or just a general talking point. Are there other > tools out there that solve for the multi-release upgrade? Are there any > folks that might want to share their expertise? Maybe a process outline that > worked? Best practices? Do folks believe tools are the right way to solve > this or would comprehensive upgrade documentation be better for the general > community? > > As most of the upgrade issues center around database migrations, we > discussed some of the potential pitfalls at length. One approach was to > roll-up all DB migrations into a single repository and run all upgrades for > a given project in one step. Another was to simply have mutliple python > virtual environments and just run in-line migrations from a version specific > venv (this is what the OSA tooling does). Does one way work better than the > other? Any thoughts on how this could be better? Would having N+2/3 > migrations addressable within the projects, even if they're not tested any > longer, be helpful? > > It was our general thought that folks would be interested in having the > ability to skip releases so we'd like to hear from the community to validate > our thinking. Additionally, we'd like to get more minds together and see if > folks are wanting to work on such an initiative, even if this turns into > nothing more than a co-op/channel where we can "phone a friend". Would it be > good to try and secure some PTG space to work on this? Should we try and > create working group going? > > If you've made it this far, please forgive my stream of consciousness. I'm > trying to ask a lot of questions and distill long form conversation(s) into > as little text as possible all without writing a novel. With that said, I > hope this finds you well, I look forward
[openstack-dev] [TripleO] overcloud_containers.yaml: container versioning, tags and use cases ?
velopers need to be pulling from that "stable" source and from a "stable" tag. So... what do we do ? [1]: https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-master-current-tripleo/ [2]: http://i.imgur.com/I26njTz.png [3]: https://review.openstack.org/#/c/466305/ [4]: http://logs.openstack.org/51/467651/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-containers-oooq-nv/dc61632/logs/oooq/undercloud/home/jenkins/overcloud_prep_containers.log.txt.gz#_2017-05-24_19_24_13 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-ansible] Running openstack-ansible-tests functional job in the ARA gate ?
Okay, so I did find something just now. First, > However, the ARA check jobs get a permission denied when running tox > (?), Lack of caffeine -- it's a file not found error (I knew that). It turns out that the "zuul-git-prep-upper-constraints" uses $ZUUL_PROJECT in the clone map [1]. $ZUUL_PROJECT in this context resolves to openstack/ara, not openstack/openstack-ansible-tests. The upper constraints file is likely not being put in the expected place: === invocation failed (errno 2), args: ['/home/jenkins/workspace/gate-ara-integration-openstack-ansible-func-centos-7-nv/openstack-ansible-tests/.tox/functional/bin/pip', 'install', '-c/home/jenkins/workspace/gate-ara-integration-openstack-ansible-func-centos-7-nv/openstack-ansible-tests/upper-constraints.txt' === I'll try to straigthen things out and see what happens. [1]: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/macros.yaml#L40-L71 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Mon, Apr 24, 2017 at 5:45 PM, Paul Belanger <pabelan...@redhat.com> wrote: > On Mon, Apr 24, 2017 at 05:25:26PM -0400, David Moreau Simard wrote: >> Hi! >> >> I'm trying to run the openstack-ansible-tests func jobs in the ARA >> check gate but it's not working. >> I'm probably missing something obvious and someone can hopefully point it >> out ! >> >> This review [1] landed today so I figured I would give it another shot >> but it didn't fix the problem I had -- here's a test review in the ARA >> gate [2]. >> So, let's compare. >> >> As I understand, openstack-ansible-tests uses this template [3] to run the >> job. >> I've re-implemented this as an ARA job [4] with the subtle difference >> that we zuul-clone osa-tests first.. it's otherwise the same. >> >> However, the ARA check jobs get a permission denied when running tox >> (?), example here [5]. >> For comparison, you can see the same block from an osa-tests review here [6]. >> >> Anyone know what could be the problem ? >> > You need a testenv:functional in your tox.ini file. The job is running: > > tox -efunctional > > and failing. > >> Thanks ! >> >> [1]: https://review.openstack.org/#/c/453298/ >> [2]: https://review.openstack.org/#/c/436098/ >> [3]: >> https://github.com/openstack-infra/project-config/blob/45d26f75d150febc121b91ebd5aa0435ecba7315/jenkins/jobs/ansible-role-jobs.yaml#L66-L102 >> [4]: >> https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ara.yaml#L21-L72 >> [5]: >> http://logs.openstack.org/98/436098/7/check/gate-ara-integration-openstack-ansible-func-centos-7-nv/6445cfb/console.html#_2017-04-24_21_20_58_603926 >> [6]: >> http://logs.openstack.org/98/453298/21/check/gate-openstack-ansible-tests-ansible-func-centos-7/235e59a/console.html#_2017-04-24_18_29_31_510532 >> >> David Moreau Simard >> Senior Software Engineer | Openstack RDO >> >> dmsimard = [irc, github, twitter] >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-ansible] Running openstack-ansible-tests functional job in the ARA gate ?
Hi! I'm trying to run the openstack-ansible-tests func jobs in the ARA check gate but it's not working. I'm probably missing something obvious and someone can hopefully point it out ! This review [1] landed today so I figured I would give it another shot but it didn't fix the problem I had -- here's a test review in the ARA gate [2]. So, let's compare. As I understand, openstack-ansible-tests uses this template [3] to run the job. I've re-implemented this as an ARA job [4] with the subtle difference that we zuul-clone osa-tests first.. it's otherwise the same. However, the ARA check jobs get a permission denied when running tox (?), example here [5]. For comparison, you can see the same block from an osa-tests review here [6]. Anyone know what could be the problem ? Thanks ! [1]: https://review.openstack.org/#/c/453298/ [2]: https://review.openstack.org/#/c/436098/ [3]: https://github.com/openstack-infra/project-config/blob/45d26f75d150febc121b91ebd5aa0435ecba7315/jenkins/jobs/ansible-role-jobs.yaml#L66-L102 [4]: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ara.yaml#L21-L72 [5]: http://logs.openstack.org/98/436098/7/check/gate-ara-integration-openstack-ansible-func-centos-7-nv/6445cfb/console.html#_2017-04-24_21_20_58_603926 [6]: http://logs.openstack.org/98/453298/21/check/gate-openstack-ansible-tests-ansible-func-centos-7/235e59a/console.html#_2017-04-24_18_29_31_510532 David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack-operators] Migrating glance images to a new backend
Oh, hey, that's my blog. Happy to answer questions as much as my memory allows.. I haven't revisited this particular topic, it was a one-time thing for me. I would certainly hope for there to be a better way by now, almost two years later! Curious to see if other operators have any insight to share as my method was fairly hacked together. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Mar 24, 2017 11:58 AM, "Massimo Sgaravatto" <massimo.sgarava...@gmail.com> wrote: Hi In our Mitaka cloud we are currently using Gluster as storage backend for Glance and Cinder. We are now starting the migration to ceph: the idea is then to dismiss gluster when we have done. I have a question concerning Glance. I have understood (or at least I hope so) how to add ceph as store backend for Glance so that new images will use ceph while the previously created ones on the file backend will be still usable. My question is how I can migrate the images from the file backend to ceph when I decide to dismiss the gluster based storage. The only documentation I found is this one: https://dmsimard.com/2015/07/18/migrating-glance-images-to- a-different-backend/ Could you please confirm that there aren't other better (simpler) approaches for such image migration ? Thanks, Massimo ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [openstack-dev] ARA - Ansible Run Analysis: Would you like to help ?
On Thu, Mar 23, 2017 at 7:49 AM, Paul Belanger <pabelan...@redhat.com> wrote: > Yes? Yay \o/ > > We are in the process of shutting down puppetboard.o.o for openstack-infra. > I'd > like to get an infra-spec in place to potentially use ARA for our ansible > runs. > So it is possible you'll get some patches your way. Let me know if you need help getting that going ! David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] ARA - Ansible Run Analysis: Would you like to help ?
This is a question that comes up often and thus made its way to the documentation FAQ [1] :) Yes, ARA only provides playbook run recording and reporting/viewing. The web interface is 100% passive, there are no actions that can be taken like editing or retrying a playbook. In fact, it's so passive that there is a feature to statically generate the whole web application to HTML like StackViz. [1]: http://ara.readthedocs.io/en/latest/faq.html#why-don-t-you-use-ansible-tower-rundeck-or-semaphore David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Mar 22, 2017 8:48 PM, "Joshua Harlow" <harlo...@fastmail.com> wrote: Sounds neat, So this would be similar to what tower or semaphore also have (I would assume they have something very like ARA internally) but instead of providing the whole start/stop/inventory workflow this just provides the viewing component? David Moreau Simard wrote: > Hi openstack-dev, > > There's this project I'm passionate that I want to tell you about: ARA [1]. > So, what's ARA ? > > ARA is an Ansible callback plugin that you can set up anywhere you run > Ansible today. > The next time you run an ansible-playbook command, it'll automatically > record and organize all the data and provide an intuitive interface > for you to browse the playbook results. > > In practice, you can find a video demonstration of what the user > interface looks like here [2]. > > ARA doesn't require you to change your existing workflows, it doesn't > require you to re-write your playbooks. > It's offline, self-contained, standalone and decentralized by default. > You can run it on your laptop for a single playbook or run it across > thousands of runs, recording millions of tasks in a centralized > database. > You can read more about the project's core values and philosophies in > the documented manifesto [3]. > > ARA is already used by many different projects that leverage Ansible > to fulfill their needs, for example: > - OpenShift-Ansible > - OpenStack-Ansible > - Kolla-Ansible > - TripleO-Quickstart > - Browbeat > - devstack-gate > > ARA's also garnered quite a bit of interest outside the OpenStack > community and there is already a healthy amount of users hanging out > in IRC on #ara. > > So, it looks like the project is going well. Why am I asking for help ? > > ARA has been growing in popularity, that's definitely something I am > very happy about. > However, this also means that there are more users, more feedback, > more questions, more bugs, more feature requests, more use cases and > unfortunately, ARA doesn't happen to be my full time job. > ARA is a tool that I created to make my job easier ! > > Also, as much as I hate to admit it, I am by no means a professional > python developer -- even less so in frontend (html/css/js). > Being honest, there are things that we should be doing in the project > that I don't have the time or the skills to accomplish. > > Examples of what I would need help with, aside from what's formally on > StoryBoard [4]: > - Help the community (answer questions, triage bugs, etc) > - Flask experts (ARA is ultimately a flask application) > - Better separation of components (decouple things properly into a > server/client/api interface) > - Full python3 compatibility, test coverage and gating > - Improve/optimize SQL models/performance > > Contributing to ARA in terms of code is no different than any other > OpenStack project but I've documented the process if you are not > familiar with it [5]. > ARA has good unit and integration test coverage and I love to think > it's not a project that is hard to develop for. > > If you feel the project is interesting and would like to get involved, > I'd love to welcome you on board. > > Let's chat. > > [1]: https://github.com/openstack/ara > [2]: https://www.youtube.com/watch?v=aQiN5wBXZ4g > [3]: http://ara.readthedocs.io/en/latest/manifesto.html > [4]: https://storyboard.openstack.org/#!/project/843 > [5]: http://ara.readthedocs.io/en/latest/contributing.html > > David Moreau Simard > Senior Software Engineer | Openstack RDO > > dmsimard = [irc, github, twitter] > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-de
[openstack-dev] ARA - Ansible Run Analysis: Would you like to help ?
Hi openstack-dev, There's this project I'm passionate that I want to tell you about: ARA [1]. So, what's ARA ? ARA is an Ansible callback plugin that you can set up anywhere you run Ansible today. The next time you run an ansible-playbook command, it'll automatically record and organize all the data and provide an intuitive interface for you to browse the playbook results. In practice, you can find a video demonstration of what the user interface looks like here [2]. ARA doesn't require you to change your existing workflows, it doesn't require you to re-write your playbooks. It's offline, self-contained, standalone and decentralized by default. You can run it on your laptop for a single playbook or run it across thousands of runs, recording millions of tasks in a centralized database. You can read more about the project's core values and philosophies in the documented manifesto [3]. ARA is already used by many different projects that leverage Ansible to fulfill their needs, for example: - OpenShift-Ansible - OpenStack-Ansible - Kolla-Ansible - TripleO-Quickstart - Browbeat - devstack-gate ARA's also garnered quite a bit of interest outside the OpenStack community and there is already a healthy amount of users hanging out in IRC on #ara. So, it looks like the project is going well. Why am I asking for help ? ARA has been growing in popularity, that's definitely something I am very happy about. However, this also means that there are more users, more feedback, more questions, more bugs, more feature requests, more use cases and unfortunately, ARA doesn't happen to be my full time job. ARA is a tool that I created to make my job easier ! Also, as much as I hate to admit it, I am by no means a professional python developer -- even less so in frontend (html/css/js). Being honest, there are things that we should be doing in the project that I don't have the time or the skills to accomplish. Examples of what I would need help with, aside from what's formally on StoryBoard [4]: - Help the community (answer questions, triage bugs, etc) - Flask experts (ARA is ultimately a flask application) - Better separation of components (decouple things properly into a server/client/api interface) - Full python3 compatibility, test coverage and gating - Improve/optimize SQL models/performance Contributing to ARA in terms of code is no different than any other OpenStack project but I've documented the process if you are not familiar with it [5]. ARA has good unit and integration test coverage and I love to think it's not a project that is hard to develop for. If you feel the project is interesting and would like to get involved, I'd love to welcome you on board. Let's chat. [1]: https://github.com/openstack/ara [2]: https://www.youtube.com/watch?v=aQiN5wBXZ4g [3]: http://ara.readthedocs.io/en/latest/manifesto.html [4]: https://storyboard.openstack.org/#!/project/843 [5]: http://ara.readthedocs.io/en/latest/contributing.html David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [telemetry] Moving Gnocchi out
I'm really curious about this decision too. Not so much about why the project wants to move out but more about what it plans on doing in terms of contribution (code, issue) workflow. I happen to drive a project that's not OpenStack-specific either: ARA [1]. ARA was first created in GitHub and was "incubated" there until we felt it was "good enough" to be proposed as an OpenStack ecosystem project. We chose the "OpenStack workflow" for two main reasons: - The original authors were already intimate with it and we were very satisfied with the rigid process it provided - ARA would be used by different OpenStack projects and it would be a good fit to be a part of the "family" I did not find it hard to find *users* outside of the OpenStack bubble, however, I still felt I needed to document a "FAQ" [2] about how, yes, the project can be used outside of OpenStack. It is definitely challenging to find contributors outside the OpenStsack ecosystem, however. Even with an attempt at providing simplified contribution guidelines [3]. Signing up for Launchpad and OpenStackid accounts, learning git-review, setting up and using Gerrit and tracking things in launchpad or storyboard are things we take for granted. Whatever way we put it, though, it's a higher barrier to entry than just browsing the GitHub repository and filing issues or creating pull requests there. So, what's the alternative ? Use the GitHub workflow ? How well is this working out for projects that attracts (or intends to attract) a lot of users and developers ? Look at the Ansible GitHub repository [4] for an extreme case: 2600 contributors, more than 1700 issues and almost 1000 pull requests. How do you make sense out of that ? Ansible has had to create a bunch of custom software to wrap around the workflow. Triaging bots [5], custom tools [6] to sift through the amount of content they have and so on. I'm not saying the OpenStack workflow is better than the GitHub one -- just that there are pros and cons that the project must weigh based on it's priorities and resources. That said, I'll re-iterate that I'm really curious on what Gnocchi intends on doing. [1]: https://github.com/openstack/ara [2]: http://ara.readthedocs.io/en/latest/faq.html#can-ara-be-used-outside-the-context-of-openstack-or-continuous-integration [3]: http://ara.readthedocs.io/en/latest/contributing.html [4]: https://github.com/ansible/ansible [5]: https://github.com/ansible/ansibullbot [6]: http://jctanner.mynetgear.com:5000/issuesearch/programmer-defection-vacuum/created_at/desc David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Mon, Mar 20, 2017 at 12:57 PM, Ian Cordasco <sigmaviru...@gmail.com> wrote: > -Original Message- > From: Chris Friesen <chris.frie...@windriver.com> > Reply: OpenStack Development Mailing List (not for usage questions) > <openstack-dev@lists.openstack.org> > Date: March 20, 2017 at 11:39:38 > To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [telemetry] Moving Gnocchi out > >> On 03/20/2017 10:10 AM, Chris Dent wrote: >> > On Mon, 20 Mar 2017, Thomas Goirand wrote: >> > >> >> I really don't understand why the Telemetry team insists in being >> >> release-independent, out of big tent and such, when the reality is that >> >> all of released Telemetry components are *very tightly* bound to a >> >> specific versions of OpenStack. IMO, it doesn't make sense upstream, or >> >> downstream of Telemetry. >> > >> > This simply isn't the case with gnocchi. Gnocchi is an independent >> > timeseries, metrics and resources data service that _happens_ to >> > work with OpenStack. >> > >> > By making it independent of OpenStack, its ability to draw >> > contribution and engagement from people outside the OpenStack >> > community increases. As a result it can become a better tool for >> > more people, including OpenStack people. Not all, or even many, of >> > the OpenStack projects are like that, but gnocchi is. More eyes, >> > less bugs, right? >> >> I'm curious why being independent of OpenStack would make it more attractive. >> >> Is the perception that requiring people to sign the Contributor Agreement is >> holding back external contribution? Or is it just that the mere idea of it >> being an OpenStack project is discouraging people from getting involved? >> >> Just as an example, if I want to get involved with libvirt because I have an >> itch to scratch the fact that it's basically a RedHat project isn't going to >> turn me off... > > Contributing to OpenStack
Re: [OpenStack-Infra] [openstack-dev] [infra][security] Encryption in Zuul v3
I don't have a horse in this race or a strong opinion on the topic, in fact I'm admittedly not very knowledgeable when it comes to low-level encryption things. However, I did have a question, even if just to generate discussion. Did we ever consider simply leaving secrets out of Zuul and offloading that "burden" to something else ? For example, end-users could use something like git-crypt [1] to crypt files in their git repos and Zuul could have a mean to decrypt them at runtime. There is also ansible-vault [2] that could perhaps be leveraged. Just trying to make sure we're not re-inventing any wheels, implementing crypto is usually not straightfoward. [1]: https://www.agwa.name/projects/git-crypt/ [2]: http://docs.ansible.com/ansible/playbooks_vault.html David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Tue, Mar 21, 2017 at 12:36 PM, James E. Blair <cor...@inaugust.com> wrote: > Hi, > > In working on the implementation of the encrypted secrets feature of > Zuul v3, I have found some things that warrant further discussion. It's > important to be deliberate about this and I welcome any feedback. > > For reference, here is the relevant portion of the Zuul v3 spec: > > http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#secrets > > And here is an implementation of that: > > https://review.openstack.org/#/q/status:open+topic:secrets+project:openstack-infra/zuul > > The short version is that we want to allow users to store private keys > in the public git repos which Zuul uses to run jobs. To do this, we > propose to use asymmetric cryptography (RSA) to encrypt the data. The > specification suggests implementing PKCS#1-OAEP, a standard for > implementing RSA encryption. > > Note that RSA is not able to encrypt a message longer than the key, and > PKCS#1 includes some overhead which eats into that. If we use 4096 bit > RSA keys in Zuul, we will be able to encrypt 3760 bits (or 470 bytes) of > information. > > Further, note that value only holds if we use SHA-1. It has been > suggested that we may want to consider using SHA-256 with PKCS#1. If we > do, we will be able to encrypt slightly less data. However, I'm not > sure that the Python cryptography library allows this (yet?). Also, see > this answer for why it may not be necessary to use SHA-256 (and also, > why we may want to anyway): > > https://security.stackexchange.com/questions/112029/should-sha-1-be-used-with-rsa-oaep > > One thing to note is that the OpenSSL CLI utility uses SHA-1. Right > now, I have a utility script which uses that to encrypt secrets so that > it's easy for anyone to encrypt a secret without installing many > dependencies. Switching to another hash function would probably mean we > wouldn't be able to use that anymore. But that's also true for other > systems (see below). > > In short, PKCS#1 pros: Simple, nicely packaged asymmetric encryption, > hides plaintext message length (up to its limit). Cons: limited to 470 > bytes (or less). > > Generally, when faced with the prospect of encrypting longer messages, > the advice is to adopt a hybrid encryption scheme (as opposed to, say, > chaining RSA messages together, or increasing the RSA key size) which > uses symmetric encryption with a single-use key for the message and > asymmetric encryption to hide the key. If we want Zuul to support the > encryption of longer secrets, we may want to adopt the hybrid approach. > A frequent hybrid approach is to encrypt the message with AES, and then > encrypt the AES key with RSA. > > The hiera-eyaml work which originally inspired some of this is based on > PKCS#7 with AES as the cipher -- ultimately a hybrid approach. An > interesting aspect of that implementation is that the use of PKCS#7 as a > message passing format allows for multiple possible underlying ciphers > since the message is wrapped in ASN.1 and is self-descriptive. We might > have simply chosen to go with that except that there don't seem to be > many good options for implementing this in Python, largely because of > the nightmare that is ASN.1 parsing. > > The system we have devised for including encrypted content in our YAML > files involves a YAML tag which specifies the encryption scheme. So we > can evolve our use to add or remove systems as needed in the future. > > So to break this down into a series of actionable questions: > > 1) Do we want a system to support encrypting longer secrets? Our PKCS#1 > system supports up to 470 bytes. That should be sufficient for most > passwords and API keys, but unlikely to be sufficient for some > certificate related systems, etc. > > 2) If so, what system should we use? > >2.1a) GPG? This has hybr
Re: [openstack-dev] [infra][security] Encryption in Zuul v3
I don't have a horse in this race or a strong opinion on the topic, in fact I'm admittedly not very knowledgeable when it comes to low-level encryption things. However, I did have a question, even if just to generate discussion. Did we ever consider simply leaving secrets out of Zuul and offloading that "burden" to something else ? For example, end-users could use something like git-crypt [1] to crypt files in their git repos and Zuul could have a mean to decrypt them at runtime. There is also ansible-vault [2] that could perhaps be leveraged. Just trying to make sure we're not re-inventing any wheels, implementing crypto is usually not straightfoward. [1]: https://www.agwa.name/projects/git-crypt/ [2]: http://docs.ansible.com/ansible/playbooks_vault.html David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Tue, Mar 21, 2017 at 12:36 PM, James E. Blair <cor...@inaugust.com> wrote: > Hi, > > In working on the implementation of the encrypted secrets feature of > Zuul v3, I have found some things that warrant further discussion. It's > important to be deliberate about this and I welcome any feedback. > > For reference, here is the relevant portion of the Zuul v3 spec: > > http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#secrets > > And here is an implementation of that: > > https://review.openstack.org/#/q/status:open+topic:secrets+project:openstack-infra/zuul > > The short version is that we want to allow users to store private keys > in the public git repos which Zuul uses to run jobs. To do this, we > propose to use asymmetric cryptography (RSA) to encrypt the data. The > specification suggests implementing PKCS#1-OAEP, a standard for > implementing RSA encryption. > > Note that RSA is not able to encrypt a message longer than the key, and > PKCS#1 includes some overhead which eats into that. If we use 4096 bit > RSA keys in Zuul, we will be able to encrypt 3760 bits (or 470 bytes) of > information. > > Further, note that value only holds if we use SHA-1. It has been > suggested that we may want to consider using SHA-256 with PKCS#1. If we > do, we will be able to encrypt slightly less data. However, I'm not > sure that the Python cryptography library allows this (yet?). Also, see > this answer for why it may not be necessary to use SHA-256 (and also, > why we may want to anyway): > > https://security.stackexchange.com/questions/112029/should-sha-1-be-used-with-rsa-oaep > > One thing to note is that the OpenSSL CLI utility uses SHA-1. Right > now, I have a utility script which uses that to encrypt secrets so that > it's easy for anyone to encrypt a secret without installing many > dependencies. Switching to another hash function would probably mean we > wouldn't be able to use that anymore. But that's also true for other > systems (see below). > > In short, PKCS#1 pros: Simple, nicely packaged asymmetric encryption, > hides plaintext message length (up to its limit). Cons: limited to 470 > bytes (or less). > > Generally, when faced with the prospect of encrypting longer messages, > the advice is to adopt a hybrid encryption scheme (as opposed to, say, > chaining RSA messages together, or increasing the RSA key size) which > uses symmetric encryption with a single-use key for the message and > asymmetric encryption to hide the key. If we want Zuul to support the > encryption of longer secrets, we may want to adopt the hybrid approach. > A frequent hybrid approach is to encrypt the message with AES, and then > encrypt the AES key with RSA. > > The hiera-eyaml work which originally inspired some of this is based on > PKCS#7 with AES as the cipher -- ultimately a hybrid approach. An > interesting aspect of that implementation is that the use of PKCS#7 as a > message passing format allows for multiple possible underlying ciphers > since the message is wrapped in ASN.1 and is self-descriptive. We might > have simply chosen to go with that except that there don't seem to be > many good options for implementing this in Python, largely because of > the nightmare that is ASN.1 parsing. > > The system we have devised for including encrypted content in our YAML > files involves a YAML tag which specifies the encryption scheme. So we > can evolve our use to add or remove systems as needed in the future. > > So to break this down into a series of actionable questions: > > 1) Do we want a system to support encrypting longer secrets? Our PKCS#1 > system supports up to 470 bytes. That should be sufficient for most > passwords and API keys, but unlikely to be sufficient for some > certificate related systems, etc. > > 2) If so, what system should we use? > >2.1a) GPG? This has hybr
Re: [openstack-dev] [tc][appcat] The future of the App Catalog
The App Catalog, to me, sounds sort of like a weird message that OpenStack somehow requires applications to be packaged/installed/deployed differently. If anything, perhaps we should spend more effort on advertising that OpenStack provides bare metal or virtual compute resources and that apps will work just like any other places. David Moreau Simard Senior Software Engineer | Openstack RDO dmsimard = [irc, github, twitter] On Wed, Mar 8, 2017 at 9:41 AM, Jay Pipes <jaypi...@gmail.com> wrote: > On 03/06/2017 06:26 AM, Thierry Carrez wrote: >> >> Hello everyone, >> >> The App Catalog was created early 2015 as a marketplace of pre-packaged >> applications that you can deploy using Murano. Initially a demo by >> Mirantis, it was converted into an open upstream project team, and >> deployed as a "beta" as apps.openstack.org. >> >> Since then it grew additional categories (Glance images, Heat & Tosca >> templates), but otherwise did not pick up a lot of steam. The website >> (still labeled "beta") features 45 glance images, 6 Tosca templates, 13 >> heat templates and 94 murano packages (~30% of which are just thin >> wrappers around Docker containers). Traffic stats show around 100 visits >> per week, 75% of which only read the index page. >> >> In parallel, Docker developed a pretty successful containerized >> application marketplace (the Docker Hub), with hundreds of thousands of >> regularly-updated apps. Keeping the App Catalog around (including its >> thinly-wrapped Docker container Murano packages) make us look like we >> are unsuccessfully trying to compete with that ecosystem, while >> OpenStack is in fact completely complementary. >> >> In the past we have retired projects that were dead upstream. The App >> Catalog is not in this case: it has an active maintenance team, which >> has been successfully maintaining the framework and accepting >> applications. If we end up retiring the App Catalog, it would clearly >> not be a reflection on that team performance, which has been stellar >> despite limited resources. It would be because the beta was arguably not >> successful in building an active marketplace of applications, and >> because its continuous existence is not a great fit from a strategy >> perspective. Such removal would be a first for our community, but I >> think it's now time to consider it. >> >> Before we discuss or decide anything at the TC level, I'd like to >> collect everyone thoughts (and questions) on this. Please feel free to >> reply to this thread (or reach out to me privately if you prefer). Thanks >> ! > > > Mirantis' position is that the App Catalog was a good idea, but we agree > with you that other application repositories like DockerHub and Quay.io are > both more useful and more actively used. > > The OpenStack App Catalog does indeed seem to unnecessarily compete with > those application repositories, and we would support its retirement if that > is what the community would like to do. We'll provide resources and help in > winding anything down if needed. > > Best, > -jay > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
