Re: [openstack-dev] [neutron] multinode CI jobs in the gate
Hey, There is already an experimental job called "gate-tempest-dsvm-neutron-dvr-ha-multinode-full-ubuntu-xenial-nv" (which was added by [1]). However, the job is currently a "one dvr_snat nodes, two dvr nodes" deployment (instead of the "two dvr_snat nodes, one dvr node" that it should be in order to make L3HA work there). The patch which makes the final transition is being worked on in [2], and it's ready to be merged afaik. Once it merges, we'll have a DVR+HA gate in the experimental queue. John. [1]: https://review.openstack.org/#/c/383742/ [2]: https://review.openstack.org/#/c/383827/ On Fri, Dec 16, 2016 at 2:23 AM, Armando M. <arma...@gmail.com> wrote: > Hi Neutrinos, > > Infra patch proposed in [1] got me thinking again about what we shall do > when it comes to multinode testing in the gate and how to focus our testing > and CI efforts upstream going forward. My line of thinking has always been > that multinode resources should be devoted to configurations whose coverage > fully benefit from the inherent nature of the distributed deployment, and > that means giving priority to DVR+HA and demote other configurations that > use centralized routing. > > I know that some of you in team have worked on this, but I am a bit behind > on the latest status update. Any good soul willing to bring a strapped (for > time) PTL up to speed? > > Cheers, > Armando > > [1] https://review.openstack.org/#/c/411263/ > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] bug deputy report
On Mon, Oct 24, 2016 at 7:39 AM, Das, Anindita <anindita@intel.com> wrote: > 1. https://bugs.launchpad.net/neutron/+bug/1635554 - Delete Router / > race condition > I've triaged this bug and assigned it to myself. I'll make sure this either gets solved or gets the proper attention. -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Neutron team social event in Barcelona
+1 - great initiative, can't wait to re-meet everybody :) On Sat, Oct 15, 2016 at 8:11 PM, Paul Michali <p...@michali.net> wrote: > +1 Thanks for setting this up! > > > On Sat, Oct 15, 2016, 9:53 AM Alonso Hernandez, Rodolfo > <rodolfo.alonso.hernan...@intel.com> wrote: >> >> +1 >> >> >> >> From: Doug Wiegley [mailto:doug...@parksidesoftware.com] >> Sent: Saturday, October 15, 2016 1:57 AM >> To: OpenStack Development Mailing List (not for usage questions) >> <openstack-dev@lists.openstack.org> >> Subject: Re: [openstack-dev] [Neutron] Neutron team social event in >> Barcelona >> >> >> >> +1 >> >> Doug >> >> >> On Oct 14, 2016, at 6:24 PM, Kevin Benton <ke...@benton.pub> wrote: >> >> +1 >> >> >> >> On Oct 14, 2016 1:33 PM, "Miguel Lavalle" <mig...@mlavalle.com> wrote: >> >> Dear Neutrinos, >> >> I am organizing a social event for the team on Thursday 27th at 19:30. >> After doing some Google research, I am proposing Raco de la Vila, which is >> located in Poblenou: http://www.racodelavila.com/en/index.htm. The menu is >> here: http://www.racodelavila.com/en/carta-racodelavila.htm >> >> It is easy to get there by subway from the Summit venue: >> https://goo.gl/maps/HjaTEcBbDUR2. I made a reservation for 25 people under >> 'Neutron' or "Miguel Lavalle". Please confirm your attendance so we can get >> a final count. >> >> Here's some reviews: >> https://www.tripadvisor.com/Restaurant_Review-g187497-d1682057-Reviews-Raco_De_La_Vila-Barcelona_Catalonia.html >> >> >> Cheers >> >> >> >> Miguel >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] Skip-only tests
Hi all, The Neutron community has recently started contributing Tempest scenario tests in the Neutron tree and I'd like to discuss a general issue we're hitting. Mainly, we're talking about required hardware for some features and/or VM images that support more features, which makes it hard (or impossible) to test certain features at the gate end-to-end. Specifically, tests that require SR-IOV or OVS-DPDK, and tests relating to VLAN Aware VMs. For this reason, we would like to bring up the idea of introducing in a new concept to the "testing arena" - skip-only tests. These will be tests that will be merged upstream and skipped unless some pre-conditions are met (example: "@skip_unless(has_sriov=True)"). The key idea here though, is that we don't expect upstream gates to add support for these pre-conditions any time soon. Instead, the tests will be merged upstream and used on the various downstream gates most of us have (with specific hardware/deployment that match our respective interests). Another way to look at it is: "I want to contribute tests that require specific hardware/software/deployments, and I want it to be robust and shared amongst the community as per the Open Source way, even though it won't be actually run on the upstream gates". Obviously, we would continue to investigate ways to "unskip" these kind of tests where possible. Examples where this would be extremely helpful include (but aren't limited to): 1. SR-IOV - Sure, the Mellanox CI currently tests SR-IOV in VF mode, however this isn't enough; SR-IOV in PF mode isn't checked and additional interesting use-cases (such as QoS + SR-IOV) aren't tested in Mellanox' CI (let alone any other upstream gate). Also, considering the current QoS test - in order to test it specifically with SR-IOV, one must be able to create a port with vnic_type='direct'. 2. VLAN Aware VMs - while tested (or planned to be tested) upstream in the unit/functional/api/fullstack levels, it isn't tested (nor is it planned to be, afaik) in a tempest scenario. The reason is simple: VMs require 802.1Q [1] in order to support VLAN Aware VMs, and it isn't currently in the Cirros image. While we're currently blocking on these tests until the Cirros images are updated, we could easily test the feature using our respective images downstream. This is also a good example where eventually these tests can be un-skipped when the upstream Cirros images are updated. 3. SCTP tests for security groups - also blocked because Cirros doesn't carry a netcat which supports this. ...And so on and so forth. Here's a brief QA section that should include some common questions/complaints: Q: The VLAN Aware VMs issue can be solved by adding Cirros support for 802.1Q - why should we in the mean time merge tests that need to be maintained? A: No one has guarantees when the new Cirros will be released and that it will work with all the other gate jobs and tests. Though we have already started pushing for a new version with support, we have (downstream-wise) other VMs that do carry the support required - we are just missing the tests. Q: So write the tests downstream - why should I care? A: We want to collaborate on the tests upstream so we (and you) can gain the standard benefits of improved and better maintained code. This will also (hopefully) help removing duplication of efforts in the testing scene. Q: So how will we know if a downstream test fails? A: This is a currently open question. One suggestion we had was that any time these skip-only tests are explicitly changed, the author has to show logs of a successful run. While we don't see how this can be relevant to certain trivial project-wide changes (and such changes can be exempt from such proofs), explicit modification of tests, logic or infra that involve these tests will need to prove that they actually work downstream. We are now open for ideas about this idea. [1]: https://en.wikipedia.org/wiki/IEEE_802.1Q -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] Newton Midcycle - Tuesday dinner
Hi guys, For those of us who'll arrive on Tuesday to Cork, Martin Hickey has arranged a dinner at "Gourmet Burger Bistro" [1], 8 Bridge Street, at 19:30. Last I heard the reservation was for 15 people so this should accommodate all who filled out in [2] that they will arrive on Tuesday. [1]: http://www.gourmetburgerbistro.ie/ [2]: https://etherpad.openstack.org/p/newton-neutron-midcycle See you then, -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [tooz] DLM benchmark results
Yes, the backends were deployed in cluster configuration (the configurations are available in the appendix). I'll make a change to the doc to make sure this is reflected properly. On Fri, Jul 22, 2016 at 11:29 AM, Kevin Benton <ke...@benton.pub> wrote: > Were the backends (zookeeper, etcd) deployed in a cluster configuration? I > can't quite tell from the doc. > > On Fri, Jul 22, 2016 at 12:58 AM, John Schwarz <jschw...@redhat.com> wrote: >> >> You're right Joshua. >> >> Tooz HEAD points to 0f4e1198fdcbd6a29d77c67d105d201ed0fbd9e0. >> >> With regards to etcd and zookeeper's versions, they are: >> zookeeper-3.4.5+28-1.cdh4.7.1.p0.13.el6.x86_64, >> etcd-2.2.5-2.el7.0.1.x86_64. >> >> John. >> >> On Thu, Jul 21, 2016 at 8:14 PM, Joshua Harlow <harlo...@fastmail.com> >> wrote: >> > Hi John, >> > >> > Thanks for gathering this info, >> > >> > Do you have the versions of the backend that were used here >> > (particularly >> > relevant for etcd which has a new release pretty frequently). >> > >> > It'd be useful to capture that info also :) >> > >> > John Schwarz wrote: >> >> >> >> Hi everyone, >> >> >> >> Following [1], a few of us sat down during the last day of the Austin >> >> Summit and discussed the possibility of adding formal support for >> >> Tooz, specifically for the locking mechanism it provides. The >> >> conclusion we reached was that benchmarks should be done to show if >> >> and how Tooz affects the normal operation of Neutron (i.e. if locking >> >> a resource using Zookeeper takes 3 seconds, it's not worthwhile at >> >> all). >> >> >> >> We've finally finished the benchmarks and they are available at [2]. >> >> They test a specific case: when creating an HA router a lock-free >> >> algorithm is used to assign a vrid to a router (this is later used for >> >> keepalived), and the benchmark specifically checks the effects of >> >> locking that function with either Zookeeper or Etcd, using the no-Tooz >> >> case as a baseline. The locking was checked in 2 different ways - one >> >> which presents no contention (acquire() always succeeds immediately) >> >> and one which presents contentions (acquire() may block until a >> >> similar process for the invoking tenant is complete). >> >> >> >> The benchmarks show that while using Tooz does raise the cost of an >> >> operation, the effects are not as bad as we initially feared. In the >> >> simple, single simultaneous request, using Zookeeper raised the >> >> average time it took to create a router by 1.5% (from 11.811 to 11.988 >> >> seconds). On the more-realistic case of 6 simultaneous requests, >> >> Zookeeper raised the cost by 3.74% (from 16.533 to 17.152 seconds). >> >> >> >> It is important to note that the setup itself was overloaded - it was >> >> built on a single baremetal hosting 5 VMs (4 of which were >> >> controllers) and thus we were unable to go further - for example, 10 >> >> concurrent requests overloaded the server and caused some race >> >> conditions to appear in the L3 scheduler (bugs will be opened soon), >> >> so for this reason we haven't tested heavier samples and limited >> >> ourselves to 6 simultaneous requests. >> >> >> >> Also important to note that some kind of race condition was noticed in >> >> tooz's etcd driver. We've discussed this with the tooz devs and >> >> provided a patch that is supposed to fix them [3]. >> >> Lastly, races in the L3 HA Scheduler were found and we are yet to dig >> >> into them and find out their cause - bugs will be opened for these as >> >> well. >> >> >> >> I've opened the summary [2] for comments so you're welcome to open a >> >> discussion about the results both in the ML and on the doc itself. >> >> >> >> (CC to all those who attended the Austin Summit meeting and other >> >> interested parties) >> >> Happy locking, >> >> >> >> [1]: >> >> >> >> http://lists.openstack.org/pipermail/openstack-dev/2016-April/093199.html >> >> [2]: >> >> >> >> https://docs.google.com/document/d/1jdI8gkQKBE0G9koR0nLiW02d5rwyWv_-gAp7yavt4w8 >> >> [3]: https://review.openstack.org/#/c/342096/ >> >> >> >> -- >
Re: [openstack-dev] [neutron] [tooz] DLM benchmark results
You're right Joshua. Tooz HEAD points to 0f4e1198fdcbd6a29d77c67d105d201ed0fbd9e0. With regards to etcd and zookeeper's versions, they are: zookeeper-3.4.5+28-1.cdh4.7.1.p0.13.el6.x86_64, etcd-2.2.5-2.el7.0.1.x86_64. John. On Thu, Jul 21, 2016 at 8:14 PM, Joshua Harlow <harlo...@fastmail.com> wrote: > Hi John, > > Thanks for gathering this info, > > Do you have the versions of the backend that were used here (particularly > relevant for etcd which has a new release pretty frequently). > > It'd be useful to capture that info also :) > > John Schwarz wrote: >> >> Hi everyone, >> >> Following [1], a few of us sat down during the last day of the Austin >> Summit and discussed the possibility of adding formal support for >> Tooz, specifically for the locking mechanism it provides. The >> conclusion we reached was that benchmarks should be done to show if >> and how Tooz affects the normal operation of Neutron (i.e. if locking >> a resource using Zookeeper takes 3 seconds, it's not worthwhile at >> all). >> >> We've finally finished the benchmarks and they are available at [2]. >> They test a specific case: when creating an HA router a lock-free >> algorithm is used to assign a vrid to a router (this is later used for >> keepalived), and the benchmark specifically checks the effects of >> locking that function with either Zookeeper or Etcd, using the no-Tooz >> case as a baseline. The locking was checked in 2 different ways - one >> which presents no contention (acquire() always succeeds immediately) >> and one which presents contentions (acquire() may block until a >> similar process for the invoking tenant is complete). >> >> The benchmarks show that while using Tooz does raise the cost of an >> operation, the effects are not as bad as we initially feared. In the >> simple, single simultaneous request, using Zookeeper raised the >> average time it took to create a router by 1.5% (from 11.811 to 11.988 >> seconds). On the more-realistic case of 6 simultaneous requests, >> Zookeeper raised the cost by 3.74% (from 16.533 to 17.152 seconds). >> >> It is important to note that the setup itself was overloaded - it was >> built on a single baremetal hosting 5 VMs (4 of which were >> controllers) and thus we were unable to go further - for example, 10 >> concurrent requests overloaded the server and caused some race >> conditions to appear in the L3 scheduler (bugs will be opened soon), >> so for this reason we haven't tested heavier samples and limited >> ourselves to 6 simultaneous requests. >> >> Also important to note that some kind of race condition was noticed in >> tooz's etcd driver. We've discussed this with the tooz devs and >> provided a patch that is supposed to fix them [3]. >> Lastly, races in the L3 HA Scheduler were found and we are yet to dig >> into them and find out their cause - bugs will be opened for these as >> well. >> >> I've opened the summary [2] for comments so you're welcome to open a >> discussion about the results both in the ML and on the doc itself. >> >> (CC to all those who attended the Austin Summit meeting and other >> interested parties) >> Happy locking, >> >> [1]: >> http://lists.openstack.org/pipermail/openstack-dev/2016-April/093199.html >> [2]: >> https://docs.google.com/document/d/1jdI8gkQKBE0G9koR0nLiW02d5rwyWv_-gAp7yavt4w8 >> [3]: https://review.openstack.org/#/c/342096/ >> >> -- >> John Schwarz, >> Senior Software Engineer, >> Red Hat. >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] [tooz] DLM benchmark results
Hi everyone, Following [1], a few of us sat down during the last day of the Austin Summit and discussed the possibility of adding formal support for Tooz, specifically for the locking mechanism it provides. The conclusion we reached was that benchmarks should be done to show if and how Tooz affects the normal operation of Neutron (i.e. if locking a resource using Zookeeper takes 3 seconds, it's not worthwhile at all). We've finally finished the benchmarks and they are available at [2]. They test a specific case: when creating an HA router a lock-free algorithm is used to assign a vrid to a router (this is later used for keepalived), and the benchmark specifically checks the effects of locking that function with either Zookeeper or Etcd, using the no-Tooz case as a baseline. The locking was checked in 2 different ways - one which presents no contention (acquire() always succeeds immediately) and one which presents contentions (acquire() may block until a similar process for the invoking tenant is complete). The benchmarks show that while using Tooz does raise the cost of an operation, the effects are not as bad as we initially feared. In the simple, single simultaneous request, using Zookeeper raised the average time it took to create a router by 1.5% (from 11.811 to 11.988 seconds). On the more-realistic case of 6 simultaneous requests, Zookeeper raised the cost by 3.74% (from 16.533 to 17.152 seconds). It is important to note that the setup itself was overloaded - it was built on a single baremetal hosting 5 VMs (4 of which were controllers) and thus we were unable to go further - for example, 10 concurrent requests overloaded the server and caused some race conditions to appear in the L3 scheduler (bugs will be opened soon), so for this reason we haven't tested heavier samples and limited ourselves to 6 simultaneous requests. Also important to note that some kind of race condition was noticed in tooz's etcd driver. We've discussed this with the tooz devs and provided a patch that is supposed to fix them [3]. Lastly, races in the L3 HA Scheduler were found and we are yet to dig into them and find out their cause - bugs will be opened for these as well. I've opened the summary [2] for comments so you're welcome to open a discussion about the results both in the ML and on the doc itself. (CC to all those who attended the Austin Summit meeting and other interested parties) Happy locking, [1]: http://lists.openstack.org/pipermail/openstack-dev/2016-April/093199.html [2]: https://docs.google.com/document/d/1jdI8gkQKBE0G9koR0nLiW02d5rwyWv_-gAp7yavt4w8 [3]: https://review.openstack.org/#/c/342096/ -- John Schwarz, Senior Software Engineer, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][ovo] NeutronDbObject concurrency issues
The incorporation of tooz and Neutron is being discussed as part of https://bugs.launchpad.net/neutron/+bug/1552680 as an RFE for Newton. Hopefully I'll have some news to break in on this matter in the upcoming days (and if I do I'll update on the launchpad to eliminate duplicity). On Tue, May 24, 2016 at 8:54 AM, Gary Kotton <gkot...@vmware.com> wrote: > Hi, > > We have used tooz to enable concurrency. Zookeeper and Redis worked well. I > think that it is certainly something that we need to consider. The challenge > becomes a deployment. > > Thanks > > Gary > > > > From: Damon Wang <damon.dev...@gmail.com> > Reply-To: OpenStack List <openstack-dev@lists.openstack.org> > Date: Tuesday, May 24, 2016 at 5:58 AM > To: OpenStack List <openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [neutron][ovo] NeutronDbObject concurrency > issues > > > > Hi, > > > > I want to add an option which handle by another project Tooz. > > > > https://github.com/openstack/tooz > > > > with redis or some other drivers, it seems pretty a good choice. > > > > Any comments? > > > > Wei Wang > > > > 2016-05-17 6:53 GMT+08:00 Ilya Chukhnakov <ichukhna...@mirantis.com>: > > > > On 16 May 2016, at 20:01, Michał Dulko <michal.du...@intel.com> wrote: > > > It's not directly related, but this reminds me of tests done by geguileo > [1] some time ago that were comparing different methods of preventing DB > race conditions in concurrent environment. Maybe you'll also find them > useful as you'll probably need to do something like conditional update > to increment a revision number. > > [1] https://github.com/Akrog/test-cinder-atomic-states > > > > Thanks for the link. The SQLA revisions are similar to the > 'solutions/update_with_where', > > but they use the dedicated column for that [2]. And as long as it is > properly configured, > > it happens 'automagically' (SQLA will take care of adding proper 'where' to > 'update'). > > > > [2] http://docs.sqlalchemy.org/en/latest/orm/versioning.html > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > ______ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Discussing DLMs in the Unplugged Track
Hi guys, We'll meet at 11:00 at Salon C, Hilton. John. On Sun, Apr 24, 2016 at 8:59 AM, Joshua Harlow <harlo...@fastmail.com> wrote: > I'll try to be there as well, since I know a little bit about tooz ;) > > -Josh > > > Gary Kotton wrote: >> >> Hi, >> I suggest that you speak with Kobi Samoray - he implemented this for the >> vmware_nsx repository using tooz. Its pretty cool. >> Thanks >> Gary >> >> >> >> >> On 4/23/16, 5:16 PM, "John Schwarz"<jschw...@redhat.com> wrote: >> >>> Hi guys, >>> >>> I'm interested in discussing the DLM RFE [1] during the unplugged >>> track on Friday morning, around 11:00am. A face-to-face talk about >>> this with all parties interested will surely prove fruitful and will >>> set us up on the track we want to go through in respect to this >>> feature. >>> >>> You can find past discussions about this topic in previous driver >>> meetings here: [2], [3]. >>> >>> If you're interested in this topic, please come to talk :) >>> >>> [1]: https://bugs.launchpad.net/neutron/+bug/1552680 >>> [2]: >>> http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-03-31-22.00.log.html#l-231 >>> [3]: >>> http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-04-14-22.00.log.html#l-153 >>> >>> -- >>> John Schwarz, >>> Red Hat. >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Social at the summit
+1 On 25 Apr 2016 16:48, "Carl Baldwin"wrote: > +1 > On Apr 25, 2016 10:57 AM, "Kyle Mestery" wrote: > >> Ihar, Henry and I were talking and we thought Thursday night makes sense >> for a Neutron social in Austin. If others agree, reply on this thread and >> we'll find a place. >> >> Thanks! >> Kyle >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] Discussing DLMs in the Unplugged Track
Hi guys, I'm interested in discussing the DLM RFE [1] during the unplugged track on Friday morning, around 11:00am. A face-to-face talk about this with all parties interested will surely prove fruitful and will set us up on the track we want to go through in respect to this feature. You can find past discussions about this topic in previous driver meetings here: [2], [3]. If you're interested in this topic, please come to talk :) [1]: https://bugs.launchpad.net/neutron/+bug/1552680 [2]: http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-03-31-22.00.log.html#l-231 [3]: http://eavesdrop.openstack.org/meetings/neutron_drivers/2016/neutron_drivers.2016-04-14-22.00.log.html#l-153 -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] L3 HA testing on scale
This is some awesome work, Ann. It's very neat to see that all the races we've struggled with w.r.t. the l3 scheduler has paid off. I would definitely like to see how these results are effected by https://review.openstack.org/#/c/305774/ but understandably 49 physical nodes are hard to come by. Also, we should see how to best handle of the issue Ann found (and is tracked at https://review.openstack.org/#/c/305774/). Specifically, reproducing this should be our goal. John. On Mon, Apr 18, 2016 at 5:15 PM, Anna Kamyshnikova <akamyshnik...@mirantis.com> wrote: > Hi guys! > > As a developer I use Devstack or multinode OpenStack installation (4-5 > nodes) for work, but these are "abstract" environments, where you are not > able to perform some scenarios as your machine is not powerful enough. But > it is really important to understand the issues that real deployments have. > > Recently I've performed testing of L3 HA on the scale environment 49 nodes > (3 controllers, 46 computes) Fuel 8.0. On this environment I ran shaker and > rally tests and also performed some manual destructive scenarios. I think > that this is very important to share these results. Ideally, I think that we > should collect statistics for different configurations each release to > compare and check it to make sure that we are heading the right way. > > The results of shaker and rally tests [1]. I put detailed report in google > doc [2]. I would appreciate all comments on these results. > > [1] - http://akamyshnikova.github.io/neutron-benchmark-results/ > [2] - > https://docs.google.com/a/mirantis.com/document/d/1TFEUzRRlRIt2HpsOzFh-RqWwgTzJPBefePPA0f0x9uw/edit?usp=sharing > > Regards, > Ann Kamyshnikova > Mirantis, Inc > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Newton Design summit schedule - Draft
Hi guys, Note that the wiki page's timestamps<->session title was a bit outdated so I've corrected where I've seen the discrepancies. Specifically the "future of Neutron architecture" and "future of Neutron client" were swapped. John. On Tue, Apr 12, 2016 at 9:47 PM, Armando M. <arma...@gmail.com> wrote: > > > On 12 April 2016 at 07:08, Michael Johnson <johnso...@gmail.com> wrote: >> >> Armando, >> >> Is there any way we can move the "Neutron: Development track: future >> of *-aas projects" session? I overlaps with the LBaaS talk: >> >> https://www.openstack.org/summit/austin-2016/summit-schedule/events/6893?goback=1 >> >> Michael > > > Swapped with the first slot of the day. I also loaded etherpads here: > > https://wiki.openstack.org/wiki/Design_Summit/Newton/Etherpads#Neutron > > Cheers, > Armando >> >> >> >> On Mon, Apr 11, 2016 at 9:56 PM, Armando M. <arma...@gmail.com> wrote: >> > Hi folks, >> > >> > A provisional schedule for the Neutron project is available [1]. I am >> > still >> > working with the session chairs and going through/ironing out some >> > details >> > as well as gathering input from [2]. >> > >> > I hope I can get something more final by the end of this week. In the >> > meantime, please free to ask questions/provide comments. >> > >> > Many thanks, >> > Armando >> > >> > [1] >> > >> > https://www.openstack.org/summit/austin-2016/summit-schedule/global-search?t=Neutron%3A >> > [2] https://etherpad.openstack.org/p/newton-neutron-summit-ideas >> > >> > >> > __ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- John Schwarz, Red Hat. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] client noauth deprecation
Adding to what Miguel said, I've merged a few months back a patch [1] which actually fixed the noauth feature in favour of the full-stack effort (the full-stack patches use neutronclient in noauth mode for easy access to neutron-server). We'll probably continue to use neutronclient until some other alternative is mature enough (for example, API testing providing thorough interface to neutron-server). Needless to say that noauth is currently working just fine and if it were to stop working I'll be very sad ;-) [1]: https://review.openstack.org/#/c/125022/ On 01/07/2015 11:44 AM, Miguel Ángel Ajo wrote: Seems like a good reason to keep it, this allows us to test internal integration in isolation from keystone. Miguel Ángel Ajo On Wednesday, 7 de January de 2015 at 10:05, Assaf Muller wrote: - Original Message - The option to disable keystone authentication in the neutron client was marked for deprecation in August as part of a Keystone support upgrade.[1] What was the reason for this? As far as I can tell, Neutron works fine in the 'noauth' mode and there isn't a lot of code that tightly couples neutron to Keystone that I can think of. It was actually broken until John fixed it in: https://review.openstack.org/#/c/125022/ We plan on using it in the Neutron in-tree full-stack testing. I'd appreciate if the functionality was not removed or otherwise broken :) 1. https://github.com/openstack/python-neutronclient/commit/2203b013fb66808ef280eff0285318ce21d9bc67#diff-ba2e4fad85e66d9aabb6193f222fcc4cR438 -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][Spec freeze exception] Rootwrap daemon ode support
+1. I think this is an important feature and I'd be happy to do reviews when needed - hopefully this can get in by Kilo! :) On 11/08/2014 12:24 PM, Armando M. wrote: Hi Miguel, Thanks for picking this up. Pull me in and I'd be happy to help! Cheers, Armando On 7 November 2014 10:05, Miguel Ángel Ajo majop...@redhat.com mailto:majop...@redhat.com wrote: Hi Yorik, I was talking with Mark Mcclain a minute ago here at the summit about this. And he told me that now at the start of the cycle looks like a good moment to merge the spec the root wrap daemon bits, so we have a lot of headroom for testing during the next months. We need to upgrade the spec [1] to the new Kilo format. Do you have some time to do it?, I can allocate some time and do it right away. [1] https://review.openstack.org/#/c/93889/ -- Miguel Ángel Ajo Sent with Sparrow http://www.sparrowmailapp.com/?sig On Thursday, 24 de July de 2014 at 01:42, Miguel Angel Ajo Pelayo wrote: +1 Sent from my Android phone using TouchDown (www.nitrodesk.com http://www.nitrodesk.com) -Original Message- From: Yuriy Taraday [yorik@gmail.com mailto:yorik@gmail.com] Received: Thursday, 24 Jul 2014, 0:42 To: OpenStack Development Mailing List [openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org] Subject: [openstack-dev] [Neutron][Spec freeze exception] Rootwrap daemon mode support Hello. I'd like to propose making a spec freeze exception for rootwrap-daemon-mode spec [1]. Its goal is to save agents' execution time by using daemon mode for rootwrap and thus avoiding python interpreter startup time as well as sudo overhead for each call. Preliminary benchmark shows 10x+ speedup of the rootwrap interaction itself. This spec have a number of supporters from Neutron team (Carl and Miguel gave it their +2 and +1) and have all code waiting for review [2], [3], [4]. The only thing that has been blocking its progress is Mark's -2 left when oslo.rootwrap spec hasn't been merged yet. Now that's not the case and code in oslo.rootwrap is steadily getting approved [5]. [1] https://review.openstack.org/93889 [2] https://review.openstack.org/82787 [3] https://review.openstack.org/84667 [4] https://review.openstack.org/107386 [5] https://review.openstack.org/#/q/project:openstack/oslo.rootwrap+topic:bp/rootwrap-daemon-mode,n,z -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] non-deterministic gate failures due to unclosed eventlet Timeouts
Hi, Long story short: for future reference, if you initialize an eventlet Timeout, make sure you close it (either with a context manager or simply timeout.close()), and be extra-careful when writing tests using eventlet Timeouts, because these timeouts don't implicitly expire and will cause unexpected behaviours (see [1]) like gate failures. In our case this caused non-deterministic failures on the dsvm-functional test suite. Late last week, a bug was found ([2]) in which an eventlet Timeout object was initialized but not closed. This instance was left inside eventlet's inner-workings and triggered non-deterministic Timeout: 10 seconds errors and failures in dsvm-functional tests. As mentioned earlier, initializing a new eventlet.timeout.Timeout instance also registers it to inner mechanisms that exist within the library, and the reference remains there until it is explicitly removed (and not until the scope leaves the function block, as some would have thought). Thus, the old code (simply creating an instance without assigning it to a variable) left no way to close the timeout object. This reference remains throughout the life of a worker, so this can (and did) effect other tests and procedures using eventlet under the same process. Obviously this could easily effect production-grade systems with very high load. For future reference: 1) If you run into a Timeout: %d seconds exception whose traceback includes hub.switch() and self.greenlet.switch() calls, there might be a latent Timeout somewhere in the code, and a search for all eventlet.timeout.Timeout instances will probably produce the culprit. 2) The setup used to reproduce this error for debugging purposes is a baremetal machine running a VM with devstack. In the baremetal machine I used some 6 dd if=/dev/zero of=/dev/null to simulate high CPU load (full command can be found at [3]), and in the VM I ran the dsvm-functional suite. Using only a VM with similar high CPU simulation fails to produce the result. [1] http://eventlet.net/doc/modules/timeout.html#eventlet.timeout.eventlet.timeout.Timeout.Timeout.cancel [2] https://review.openstack.org/#/c/119001/ [3] http://stackoverflow.com/questions/2925606/how-to-create-a-cpu-spike-with-a-bash-command -- John Schwarz, Software Engineer, Red Hat. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] [LBaaS] LBaaS v2 API syntax additions/changes
On 08/25/2014 10:06 PM, Brandon Logan wrote: 2. Therefor, there should be some configuration to specifically enable either version (not both) in case LBaaS is needed. In this case, the other version is disabled (ie. a REST query for non-active version should return a not activated error). Additionally, adding a 'lb-version' command to return the version currently active seems like a good user-facing idea. We should see how this doesn't negatively effect the db migration process (for example, allowing read-only commands for both versions?) A /version endpoint can be added for both v1 and v2 extensions and service plugins. If it doesn't already exist, it would be nice if neutron had an endpoint that would return the list of loaded extensions and their versions. There is 'neutron ext-list', but I'm not familiar enough with it or with the REST API to say if we can use that. 3. Another decision that's needed to be made is the syntax for v2. As mentioned, the current new syntax is 'neutron lbaas-object-command' (against the old 'lb-object-action'), keeping in mind that once v1 is deprecated, a syntax like 'lbv2-object-action' would be probably unwanted. Is 'lbaas-object-command' okay with everyone? That is the reason we with with lbaas because lbv2 looks ugly and we'd be stuck with it for the lifetime of v2, unless we did another migration back to lb for it. Which seemed wrong to do, since then we'd have to accept both lbv2 and lb commands, and then deprecate lbv2. I assume this also means you are fine with the prefix in the API resource of /lbaas as well then? I don't mind, as long there is a similar mechanism which disables the non-active REST API commands. Does anyone disagree? 4. If we are going for different API between versions, appropriate patches also need to be written for lbaas-related scripts and also Tempest, and their maintainers should probably be notified. Could you elaborate on this? I don't understand what you mean by different API between version. The intention was that the change of the user-facing API also forces changes on other levels - not only neutronclient needs to be modified accordingly, but also tempest system tests, horizon interface regarding LBaaS... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron] [LBaaS] LBaaS v2 API syntax additions/changes
Hi, With the ongoing development of LBaaS v2, support for v2 of LBaaS in neutronclient is also being developed, as can be seen in [1]. The current implementation adds a new syntax for v2; Whereas the v1 syntax is 'neutron lb-object-command', the new v2 syntax is 'neutron lbaas-object-command'. We fear that this can lead to some level of confusion on the users' side. Currently, nothing prevents a user from trying to create a v1 pool and then trying to add v2 members. Of course the second command will fail, but the error message will be a non-intuitive one. This was discussed at the last weekly IRC meeting ([2]). Some bulletins: 1. As was discussed in the hackathon, there shouldn't be more than one version active at any given time - either v1 or v2. Also, using the same syntax for both v1 and v2 is not a good idea (db migration will be difficult when both version share syntax). We believe this should also be enforced in the server side. 2. Therefor, there should be some configuration to specifically enable either version (not both) in case LBaaS is needed. In this case, the other version is disabled (ie. a REST query for non-active version should return a not activated error). Additionally, adding a 'lb-version' command to return the version currently active seems like a good user-facing idea. We should see how this doesn't negatively effect the db migration process (for example, allowing read-only commands for both versions?) 3. Another decision that's needed to be made is the syntax for v2. As mentioned, the current new syntax is 'neutron lbaas-object-command' (against the old 'lb-object-action'), keeping in mind that once v1 is deprecated, a syntax like 'lbv2-object-action' would be probably unwanted. Is 'lbaas-object-command' okay with everyone? 4. If we are going for different API between versions, appropriate patches also need to be written for lbaas-related scripts and also Tempest, and their maintainers should probably be notified. Are there any issues with one of the points raised above? Does anyone see any other problems which we forgot to write down? Thanks a lot, John Schwarz, Yair Fried, Redhat. [1]: https://review.openstack.org/#/c/111475 [2]: http://eavesdrop.openstack.org/meetings/neutron_lbaas/2014/neutron_lbaas.2014-08-21-14.00.log.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev