Thank you for the write up. Having missed being there in person it is much appreciated :-)
On Wed, Dec 7, 2016 at 5:56 AM, Jeremy Stanley <fu...@yuggoth.org> wrote: > I'm Cc'ing this to the openstack-infra ML but setting MFT to direct > subsequent discussion to the openstack-dev ML so we can hopefully > avoid further cross-posting as much as possible. If you're replying > on a particular session topic, please update the Subject so that the > subthreads are easier to keep straight. > > Apologies for the _extreme_ delay in getting this composed and sent > out! > > > Firehose > -------- > > https://etherpad.openstack.org/p/ocata-infra-firehose > > This was primarily a brainstorming/roadmap session on possible > future plans for the firehose.openstack.org service. Discussed was > potential to have Zuul (post-v3) both consume and emit events over > MQTT, as well as having StoryBoard (should probably support an > analog of the events handled by lpmqtt at a minimum, probably easy > to add given it already has an RabbitMQ one) and Nodepool event > streams. The gerritbot consumer PoC was mentioned, but determined > that it would be better to shelve that until planning for an > Errbot-based gerritbot reimplementation is fleshed out. > > We talked about how the current logstash MQTT stream implementation, > while interesting, has significant scaling (especially bandwidth) > issues with the volume of logging we do in tests while only offering > limited benefit. We could potentially make use of it in concern with > a separate logstash for production server and Ansible logs, but it's > efficacy for our job logs was called into question. > > We also spent much of the timeslot talking about possible > integration with FedMesg (particularly that they're considering > pluggable backend support which could include an MQTT > implementation), which yields much opportunity for collaboration > between our projects. > > One other topic which came up was how to do a future HA > implementation, either by having publishers send to multiple brokers > and configure consumers to have a primary/fallback behavior or my > trying to implement a more integrated clustering solution with load > balancing proxies. We concluded that current use cases don't demand > anywhere near 100% message delivery and 100% uptime, so we can dig > deeper when there's an actual use case. > > > Status update and plans for task tracking > ----------------------------------------- > > https://etherpad.openstack.org/p/ocata-infra-community-task-tracking > > As is traditional, we held a fishbowl on our ongoing task tracking > woes. We started with a brief introduction of stakeholders who > attended and the groups whose needs they were there to represent. > After that, some presentation was made of recent StoryBoard > development progress since Austin (including Gerrit integration, > private story support for embargoed security issues, improved event > timelines, better discoverability for boards and worklists, flexible > task prioritization), as well as the existing backlog of blockers. > > We revisited the Infra spec on task tracking > http://specs.openstack.org/openstack-infra/infra-specs/ > specs/task-tracker.html > for the benefit of those present, and Kendall Nelson (diablo_rojo) > agreed to pick up and continue with the excellent stakeholder > blocking issues outreach/coordination work begun by Anita Kuno > (anteaya). > > > Next steps for infra-cloud > -------------------------- > > https://etherpad.openstack.org/p/ocata-infra-infra-cloud > > This was sort of a catch-all opportunity to hash out current plans > and upcoming needs for the infra-cloud. We determined that the > current heterogeneous hardware in the in-progress "chocolate" region > should be split into two homogeneous regions named "chocolate" and > "strawberry" (our "vanilla" region was already homogeneous). We also > talked about ongoing work to get a quote from OSUOSL for hosting the > hardware so that we can move it out of HPE data centers, and > attempting to find funding once we have some figures firmed up. > > There were also some sideline discussions on possible monitoring and > high-availability options for the underlying services. > Containerization was, as always, brought up but the usual "not a fit > for this use case" answers abounded. It was further suggested that > using infra-cloud resources for things like special TripleO tests or > Docker registry hosting were somehow in scope, but there are other > solutions to these needs which should not be conflated with the > purpose of the infra-cloud effort. > > > Interactive infra-cloud debugging > --------------------------------- > > https://etherpad.openstack.org/p/ocata-infra-infra-cloud-debugging > > The original intent for this session was to try to gather > leaders/representatives from the various projects that we're relying > on in the infra-cloud deployment and step through an interactive > session debugging the sorts of failures we see arise on the servers. > The idea was that this would be potentially educational for some > since this is a live bare metal "production" deployment of Nova, > Neutron, Keystone, Glance, et cetera with all the warts and rough > edges that operators handle on a daily basis but our developers may > not have directly experienced. > > As well-intentioned as it was, the session suffered from several > issues. First and foremost we didn't realize the Friday morning > workroom we got was going to lack a projector (only so many people > can gather around one laptop, and if it's mine then fewer still!). > Trying to get people from lots of different projects to show up for > the same slot on a day that isn't for cross-project sessions is > pretty intractable. And then there's the fact that we were all > approaching burnout as it was the last day of the week and coffee > was all the way at the opposite end of the design summit space. :/ > > Instead the time was spent partly continuing the "future of > infra-cloud" discussion, and partly just talking about random things > like troubleshooting CI jobs (some people misunderstood the session > description and thought that's what we had planned) or general Infra > team wishlist items. Not a complete waste, but some lessons learned > if we ever want to try this idea again at a future summit. > > > Test environment expectations > ----------------------------- > > https://etherpad.openstack.org/p/ocata-infra-test-env-expectations > > After the morning break we managed to perk back up again and discuss > test platform expectations. This was a remarkably productive > brainstorming session where we assembled a rough list of > expectations developers can and, more importantly, can't make about > the systems on which our CI jobs run. The culmination of these > musings can since be found in a shiny new page of the Infra Manual: > > http://docs.openstack.org/infra/manual/testing.html > > > Xenial jobs transition for stable/newton > ---------------------------------------- > > https://etherpad.openstack.org/p/ocata-infra-xenial-stable-newton > > Another constructive session right on the heels of the last... > planning the last mile of the cut-over from Ubuntu 14.04 to 16.04 > testing. We confirmed that we would switch all jobs for > stable/newton as well as master (since the implementation started > early in the Newton cycle and we need to be consistent across > projects in a stable branch). We decided to set a date (which > incidentally is TODAY) to finalize the transition. The plan was > announced to the dev ML a month ago: > > http://lists.openstack.org/pipermail/openstack-dev/2016- > November/106906.html > > The (numerous) changes in flight today to switch the lingering jobs > are covered under a common review topic: > > https://review.openstack.org/#/q/topic:st-nicholas-xenial > > > Unconference afternoon > ---------------------- > > https://etherpad.openstack.org/p/ocata-infra-contributors-meetup > > At this stage things were starting to wind up and a lot of people > with early departures had already bowed out. Those of us who > remained were treated to our own room for the first time in many > summits (no offense to the Release and QA teams, but it was nice to > not need to share for a change). Since we were a little more at > liberty to set our own pace this time we treated it as a sort of > home base from which many of us set forth to pitch in on > Infra-related planning discussions in other teams' spaces, then > regroup and disseminate what we'd done (from translation platform > upgrades to release automation designs). > > We also got in some good one-on-one time to work through topics > which weren't covered in scheduled sessions, such as Zuul v3 spec > additions or changes to the pep8 jobs to guard against missing sdist > build dependencies. As the afternoon progressed and the crowd > dwindled further we said our goodbyes and split up into smaller > groups to go out for one last meal, commiserate with those who found > themselves newly in search of employment and generally celebrate a > successful week in Barcelona. > > > That concludes my recollection of these sessions over the course of > the week--thanks for reading this far--feel free to follow up (on > the openstack-dev ML please) with any corrections/additions. Many > thanks to all who attended, and to those who could not: we missed > you. I hope to see lots of you again at the PTG in Atlanta, only a > couple months away now. Don't forget to register and book your > flights/lodging! > -- > Jeremy Stanley > > _______________________________________________ > OpenStack-Infra mailing list > openstack-in...@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev