Re: [Openstack-operators] Bandwidth limitations
Hi there, It looks like QOS is already available within the Mitaka release. Maybe it doesn't have all the features you need, but looks to be a good start. http://docs.openstack.org/mitaka/networking-guide/adv-config-qos.html I haven't used it yet, but maybe someone else will pipe up with some expierence. --Joe On Wed, Jun 29, 2016 at 12:36 PM, Daniel Levy wrote: > Hi all, > I'd like to learn about potential solutions anyone out there is using for > bandwidth limitations on VMs. Potentially applying QOS (quality of service) > rules on the VM ports in an automated fashion. > If there are no current solutions, I might submit a blue print to tackle > this issue > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
We've been using this for some time now (since at least Kilo). We set them per flavor not per instance. https://wiki.openstack.org/wiki/InstanceResourceQuota Bandwidth limits Nova Extra Specs keys: - vif_inbound_average - vif_outbound_average - vif_inbound_peak - vif_outbound_peak - vif_inbound_burst - vif_outbound_burst On Wed, Jun 29, 2016 at 10:36 AM, Daniel Levy wrote: > Hi all, > I'd like to learn about potential solutions anyone out there is using for > bandwidth limitations on VMs. Potentially applying QOS (quality of service) > rules on the VM ports in an automated fashion. > If there are no current solutions, I might submit a blue print to tackle > this issue > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
I would also look at seeing how its doing it. IN the past what it did was drop packets over a specific threshold which is really really really terrible. We do some traffic policing on some of our vm's – but we do it outside of openstack via a qemu hook and setting up our own qdisc and ifb device for each tap device that we want to police. https://github.com/godaddy/openstack-traffic-shaping ___ Kris Lindgren Senior Linux Systems Engineer GoDaddy From: Joseph Bajin mailto:josephba...@gmail.com>> Date: Wednesday, June 29, 2016 at 10:43 AM To: Daniel Levy mailto:dl...@us.ibm.com>> Cc: OpenStack Operators mailto:openstack-operators@lists.openstack.org>> Subject: Re: [Openstack-operators] Bandwidth limitations Hi there, It looks like QOS is already available within the Mitaka release. Maybe it doesn't have all the features you need, but looks to be a good start. http://docs.openstack.org/mitaka/networking-guide/adv-config-qos.html I haven't used it yet, but maybe someone else will pipe up with some expierence. --Joe On Wed, Jun 29, 2016 at 12:36 PM, Daniel Levy mailto:dl...@us.ibm.com>> wrote: Hi all, I'd like to learn about potential solutions anyone out there is using for bandwidth limitations on VMs. Potentially applying QOS (quality of service) rules on the VM ports in an automated fashion. If there are no current solutions, I might submit a blue print to tackle this issue ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org<mailto:OpenStack-operators@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
On Wed, Jun 29, 2016 at 12:43 PM, Joseph Bajin wrote: > Hi there, > > It looks like QOS is already available within the Mitaka release. Maybe it > doesn't have all the features you need, but looks to be a good start. > http://docs.openstack.org/mitaka/networking-guide/adv-config-qos.html It's available from Neutron's Liberty release even. The new feature provides a new QoS bandwidth limitation API, which when using the OVS agent, is implemented via an OVS feature as such [1]. It sets the 'ingress_policing_rate' and 'ingress_policing_burst' attributes on the VM's interface record in the ovsdb. Internally to OVS that is implemented via 'tc' and by dropping packets over the specified rate as detailed here [2]. [1] https://github.com/openstack/neutron/blob/stable/liberty/neutron/agent/common/ovs_lib.py#L539 [2] http://openvswitch.org/support/config-cookbooks/qos-rate-limiting/ > > I haven't used it yet, but maybe someone else will pipe up with some > expierence. > > --Joe > > On Wed, Jun 29, 2016 at 12:36 PM, Daniel Levy wrote: >> >> Hi all, >> I'd like to learn about potential solutions anyone out there is using for >> bandwidth limitations on VMs. Potentially applying QOS (quality of service) >> rules on the VM ports in an automated fashion. >> If there are no current solutions, I might submit a blue print to tackle >> this issue >> >> >> ___ >> OpenStack-operators mailing list >> OpenStack-operators@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >> > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
Thanks for the responses; I'm aware of the QOS policies in Openstack, however I'd like them to be applied automatically. Using predefined flavors as described by Matt Fischer above seems like a good approach, are there any solutions for non-predefined flavors? - Original message -From: Assaf Muller To: Joseph Bajin Cc: Daniel Levy/Austin/IBM@IBMUS, OpenStack Operators Subject: Re: [Openstack-operators] Bandwidth limitationsDate: Wed, Jun 29, 2016 12:46 PM On Wed, Jun 29, 2016 at 12:43 PM, Joseph Bajin wrote:> Hi there,>> It looks like QOS is already available within the Mitaka release. Maybe it> doesn't have all the features you need, but looks to be a good start.> http://docs.openstack.org/mitaka/networking-guide/adv-config-qos.htmlIt's available from Neutron's Liberty release even. The new featureprovides a new QoS bandwidth limitation API, which when using the OVSagent, is implemented via an OVS feature as such [1].It sets the 'ingress_policing_rate' and 'ingress_policing_burst'attributes on the VM's interface record in the ovsdb. Internally toOVS that is implemented via 'tc' and by dropping packets over thespecified rate as detailed here [2].[1] https://github.com/openstack/neutron/blob/stable/liberty/neutron/agent/common/ovs_lib.py#L539[2] http://openvswitch.org/support/config-cookbooks/qos-rate-limiting/>> I haven't used it yet, but maybe someone else will pipe up with some> expierence.>> --Joe>> On Wed, Jun 29, 2016 at 12:36 PM, Daniel Levy wrote: Hi all,>> I'd like to learn about potential solutions anyone out there is using for>> bandwidth limitations on VMs. Potentially applying QOS (quality of service)>> rules on the VM ports in an automated fashion.>> If there are no current solutions, I might submit a blue print to tackle>> this issue>> ___>> OpenStack-operators mailing list>> OpenStack-operators@lists.openstack.org>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators> ___> OpenStack-operators mailing list> OpenStack-operators@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators> ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
We automate all our flavor creation and don't allow people to make their own, therefore everyone gets a flavor with some restriction, but that may not fit your use case. On Wed, Jun 29, 2016 at 12:03 PM, Daniel Levy wrote: > Thanks for the responses; I'm aware of the QOS policies in Openstack, > however I'd like them to be applied automatically. Using predefined flavors > as described by Matt Fischer above seems like a good approach, are there > any solutions for non-predefined flavors? > > > - Original message - > From: Assaf Muller > To: Joseph Bajin > Cc: Daniel Levy/Austin/IBM@IBMUS, OpenStack Operators < > openstack-operators@lists.openstack.org> > Subject: Re: [Openstack-operators] Bandwidth limitations > Date: Wed, Jun 29, 2016 12:46 PM > > On Wed, Jun 29, 2016 at 12:43 PM, Joseph Bajin > wrote: > > Hi there, > > > > It looks like QOS is already available within the Mitaka release. > Maybe it > > doesn't have all the features you need, but looks to be a good start. > > http://docs.openstack.org/mitaka/networking-guide/adv-config-qos.html > > It's available from Neutron's Liberty release even. The new feature > provides a new QoS bandwidth limitation API, which when using the OVS > agent, is implemented via an OVS feature as such [1]. > > It sets the 'ingress_policing_rate' and 'ingress_policing_burst' > attributes on the VM's interface record in the ovsdb. Internally to > OVS that is implemented via 'tc' and by dropping packets over the > specified rate as detailed here [2]. > > [1] > https://github.com/openstack/neutron/blob/stable/liberty/neutron/agent/common/ovs_lib.py#L539 > [2] http://openvswitch.org/support/config-cookbooks/qos-rate-limiting/ > > > > > I haven't used it yet, but maybe someone else will pipe up with some > > expierence. > > > > --Joe > > > > On Wed, Jun 29, 2016 at 12:36 PM, Daniel Levy wrote: > >> > >> Hi all, > >> I'd like to learn about potential solutions anyone out there is using > for > >> bandwidth limitations on VMs. Potentially applying QOS (quality of > service) > >> rules on the VM ports in an automated fashion. > >> If there are no current solutions, I might submit a blue print to tackle > >> this issue > >> > >> > >> ___ > >> OpenStack-operators mailing list > >> OpenStack-operators@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >> > > > > > > ___ > > OpenStack-operators mailing list > > OpenStack-operators@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > > > > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Bandwidth limitations
Hello, We are also using it even in Juno. There is one problem with outgoing traffic limitation made this way if You are using openvswitch also. This limit is set by libvirt using tc as policing on ingress qdisc. Openvswitch is using same policing on ingress qdisc if You set 'ingress_policing_rate' for interface. Problem is that if You spawn new vm, libvirt configures bandwidth limit with tc but then neutron ovs agent wants to configure port and is setting vlan_tag in this "tap" interface. After that operation ovs will clear policing on ingress qdisc because in ovsdb there is no any info about ingress_policing_rate which should be set. Details can be found on http://openvswitch.org/pipermail/discuss/2013-July/010584.html We solved it by setting libvirt qemu hook (https://www.libvirt.org/hooks.html) which sets ingress_policing_rate and ingress_policing_burst in ovs with same values which are in instance description in libvirt. About neutron QoS, for now it is possible only to set max bandwidth limits for traffic outgoing from vm. There is no possiblity to set limit for incoming traffic. -- Best regards / Pozdrawiam Sławek Kapłoński sla...@kaplonski.pl On Wed, 29 Jun 2016, Matt Fischer wrote: > We've been using this for some time now (since at least Kilo). We set them > per flavor not per instance. > > https://wiki.openstack.org/wiki/InstanceResourceQuota > > Bandwidth limits > > Nova Extra Specs keys: > >- vif_inbound_average >- vif_outbound_average >- vif_inbound_peak >- vif_outbound_peak >- vif_inbound_burst >- vif_outbound_burst > > > > On Wed, Jun 29, 2016 at 10:36 AM, Daniel Levy wrote: > > > Hi all, > > I'd like to learn about potential solutions anyone out there is using for > > bandwidth limitations on VMs. Potentially applying QOS (quality of service) > > rules on the VM ports in an automated fashion. > > If there are no current solutions, I might submit a blue print to tackle > > this issue > > > > > > ___ > > OpenStack-operators mailing list > > OpenStack-operators@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > > > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators signature.asc Description: Digital signature ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators