Hi All,
Running with the other_config suggestion from Ben & Darrell far below, here is
some more details on what an implementation of this could look like. I'll start
coding an RFC patch shortly.
Comments, criticisms & improvements welcome.
Regards,
Billy.
RFC
Proposed Feature: Interface ingress scheduling
Purpose:
To allow certain packets ingressing an Interface to be prioritized. Which means
such packets are:
a) forwarded to their destination port before non-prioritized packets
and/or
b) are less likely to be dropped in an overloaded situation than prioritized
packets.
Details:
Ingress scheduling is supported with the best effort of the Interface. It may
be dependant on the interface type and it's supporting implementation devices.
So different interface types may have different levels of support for the
feature and the same interface type attached to different devices (e.g physical
NICs or vhost ports, the device driver used, the NIC model) may also have
different levels of support.
It's unlikely that any implementation will allow prioritization on an arbitrary
combinarion of ovs-fields (7) - check the Interface type's documentation for
details.
If for any reason the prioritizion request cannot be fully honored a warning is
logged.
The general form of control of the feature is
ovs-vsctl set Interface <iface> other_config:prioritize=<ovs-fields(7) match
specifier>
Example:
To give priority to packets with an IP DSCP field value of 1:
ovs-vsctl set Interface dpdk0 other_config:prioritize=ip_dscp=1
----
* Intially only support one ingress priority match. Multiple priorities could
be specified later.
* The prioritization algorithm (e.g strict priority queueing and so on) is
currently unspecified and implementation dependent.
* This proposal's interaction with bonding is to be defined.
* The subset of match fields supported is dependent on the interface type and
it's underlying implementation devices (NICs, vhost ports etc).
* If for any reason the prioritize request cannot be honored a warning is logged
* the configuration stays in the Interface table it is not rolled back (this
is normal once schema constraints are met the entry stays in the db and db
watchers (such as ovs-vswitchd are triggered))
* possible failure reaons (for netdev-dpdk Interfaces)
* the prioritize fields are not supported
* NIC rx multi-queue is not enabled (ie n_rxq < 2) - implementation on dpdk
ports will be via h/w filtering (supported by dpdk interface) but which
requires multiqueue to be enabled. It would be too surprising to change the
n_rxq setting implicitly.
* pmd-rxq-affinity is set. A simple & sensible interaction between these
two features is not simple to define. For now using the existing
pmd-rqx-affinity feature precludes other_config:prioritize.
> -----Original Message-----
> From: Ben Pfaff [mailto:b...@ovn.org]
> Sent: Monday, April 24, 2017 4:14 PM
> To: O Mahony, Billy <billy.o.mah...@intel.com>
> Cc: ovs-discuss@openvswitch.org; Darrell Ball <db...@vmware.com>; Kevin
> Traynor <ktray...@redhat.com>
> Subject: Re: [ovs-discuss] prioritizing latency-sensitive traffic
>
> If that's useful, then it makes sense to me and I'd have no objection.
>
> On Mon, Apr 24, 2017 at 10:06:17AM +0000, O Mahony, Billy wrote:
> > Hi Ben, Darrell,
> >
> > I've done some PoC work on this kind of traffic prioritization. However with
> OVS-DPDK's run-to-completion model the issue I find the same issue as you
> outlined - by the time the priority of the packet has been determined most
> of the effort to process the packet has already been spent.
> >
> > So, I relied on using hardware offload i.e. FlowDirector on the NIC to
> analyse and allocate packets to high and low priority queues and then
> modifying the PMD (dpif_netdev_run ) to read preferentially from the high
> priority queue. The results were good for overload situations - packets on
> the priority queue not dropped. In terms of latency there was an
> improvement but there was still a long tail to the latency profile. i..e The
> latency profile moved down but not left.
> >
> > As Darrell pointed out for egress scheduling, perhaps this kind of ingress
> prioritization can be encapsulated as an optional property of a port?
> >
> > If the port can implement the prioritization (either by offloading to
> hardware or in software) it can accept the property being set; If not it can
> return ENOTSUPP?
> >
> > There is currently HWOL use-cases being gathered for OVS-DPDK:
> https://docs.google.com/document/d/1A8adu1xzg53bzcFKINhffYyC0nCQjq
> wqnI1jAFx2sck/edit?usp=sharing + Kevin who is co-ordinating that.
> >
> > Thanks,
> > Billy.
> >
> >
> > > -----Original Message-----
> > > From: Ben Pfaff [mailto:b...@ovn.org]
> > > Sent: Friday, April 21, 2017 10:39 PM
> > > To: O Mahony, Billy <billy.o.mah...@intel.com>
> > > Cc: ovs-discuss@openvswitch.org; Darrell Ball <db...@vmware.com>
> > > Subject: Re: [ovs-discuss] prioritizing latency-sensitive traffic
> > >
> > > Thanks for letting us know. I'm happy to continue the conversation
> > > if there are interesting ideas; it's a frustrating situation,
> > > frankly, and I'd love to hear creative approaches.
> > >
> > > On Tue, Apr 18, 2017 at 10:01:28AM +0000, O Mahony, Billy wrote:
> > > > Hi Ben, Darrell,
> > > >
> > > > It sounds like the general feeling is that any kind of tc
> > > > pre-processing is not
> > > worth it and the existing egress queing/QoS facilities should suffice.
> > > >
> > > > Thanks for your comments.
> > > >
> > > > /Billy
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Ben Pfaff [mailto:b...@ovn.org]
> > > > > Sent: Thursday, April 13, 2017 7:47 PM
> > > > > To: O Mahony, Billy <billy.o.mah...@intel.com>
> > > > > Cc: ovs-discuss@openvswitch.org
> > > > > Subject: Re: [ovs-discuss] prioritizing latency-sensitive
> > > > > traffic
> > > > >
> > > > > I don't know how much more OVS can contribute to this than it
> > > > > already
> > > does.
> > > > > By the time that OVS has classified a packet to the extent that
> > > > > is necessary to determine that it should be handled with a high
> > > > > priority, OVS has already done most of the work that it does on a
> packet.
> > > > [[BO'M]] I'm investigating how I could go about classifying
> > > > packets before the main The work to transmit the
> > > > > packet is not part of OVS's job, it is the job of the driver,
> > > > > and at most OVS can mark the packet with a priority or a queue.
> > > > > OVS can already do that. So the usual answer is that it's a
> > > > > matter of configuring QoS in the driver to do what the user wants.
> > > > >
> > > > > On Mon, Apr 10, 2017 at 09:30:12AM +0000, O Mahony, Billy wrote:
> > > > > > Hi Everybody,
> > > > > >
> > > > > > I just wanted to reflag this discussion below about possible
> > > > > > methods of
> > > > > how to prioritize certain types of traffic handled by OVS.
> > > > > >
> > > > > > By prioritize I mean either or both of:
> > > > > > a) 'priority' packets are sent to their destination port
> > > > > > faster than other packets
> > > > > > b) in an overloaded situation the switch drops non-prioritized
> > > > > > packets
> > > > > rather than prioritized packets.
> > > > > >
> > > > > > Also just to be clear I am discussing kernel ovs here. Also
> > > > > > I'm looking at
> > > > > doing this without writing new code - ie is it possible and if
> > > > > so how is it configured using existing OVS.
> > > > > >
> > > > > > Thanks again,
> > > > > > Billy.
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: ovs-discuss-boun...@openvswitch.org
> > > > > > > [mailto:ovs-discuss- boun...@openvswitch.org] On Behalf Of O
> > > > > > > Mahony, Billy
> > > > > > > Sent: Friday, November 25, 2016 5:04 PM
> > > > > > > To: ovs-discuss@openvswitch.org
> > > > > > > Subject: [ovs-discuss] prioritizing latency-sensitive
> > > > > > > traffic
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I have been performing tests investigating latency profiles
> > > > > > > of low-bandwidth time-sensitive traffic when the system is
> > > > > > > busy with
> > > > > 'normal' traffic.
> > > > > > > Unsurprisingly the latency-sensitive traffic is affected by
> > > > > > > the normal traffic and has basically the same latency
> > > > > > > profile as the normal
> > > > > traffic.
> > > > > > >
> > > > > > > I would like to be able to perform prioritization of traffic
> > > > > > > as some protocols such as PTP would benefit greatly from
> > > > > > > having it's packets 'jump
> > > > > the queue'.
> > > > > > >
> > > > > > > From skimming the documentation it looks that ingress QoS
> > > > > > > offers only policing (rate-limiting). Is this actually the
> > > > > > > case or maybe I'm not looking in the right place?
> > > > > > >
> > > > > > > But if so, I am looking at some alternatives:
> > > > > > >
> > > > > > > a) create two separate egress ports and have PTP listen on
> > > > > > > one port, everything else listen on the other port and use
> > > > > > > normal forwarding rules to send PTP traffic incoming from
> > > > > > > eth0 to it's own port. Something
> > > > > like:
> > > > > > >
> > > > > > > other apps ptp_daemon
> > > > > > > + +
> > > > > > > + +
> > > > > > > if_norm if_ptp
> > > > > > > + +
> > > > > > > | |
> > > > > > > | |
> > > > > > > ++------------++
> > > > > > > | |
> > > > > > > | ovs |
> > > > > > > | |
> > > > > > > +-----+--------+
> > > > > > > |
> > > > > > > +
> > > > > > > eth0
> > > > > > >
> > > > > > > b) create prioritized queues on a port and use match and
> > > > > > > actions such as
> > > > > > > set_queue(queue) and enqueue(port, queue) on ingress traffic
> > > > > > > to forward the PTP traffic to the higher priority queue.
> > > > > > > However I think queue priority for this case only relates to
> > > > > > > which queue get to consume the bandwidth of the port first
> > > > > > > and not about changing the order in which the packets egress the
> port.
> > > > > > >
> > > > > > > c) Or perhaps I can re-use tc PRIO or CBQ qdiscs by passing
> > > > > > > all traffic to tc first before ovs?
> > > > > > >
> > > > > > > other apps
> > > > > > > |
> > > > > > > |
> > > > > > > if_norm
> > > > > > > +
> > > > > > > |
> > > > > > > |
> > > > > > > +--------------+
> > > > > > > | |
> > > > > > > | ovs |
> > > > > > > | |
> > > > > > > +-----+--------+
> > > > > > > |
> > > > > > > |
> > > > > > > tc ----- if_ptp ---- ptp_daemon
> > > > > > > +
> > > > > > > eth0
> > > > > > >
> > > > > > > Any thoughts, ideas or clarifications most welcome.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Billy.
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > discuss mailing list
> > > > > > > disc...@openvswitch.org
> > > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
> > > > > > _______________________________________________
> > > > > > discuss mailing list
> > > > > > disc...@openvswitch.org
> > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
