On 06/24/2015 05:06 AM, Pravin Shelar wrote:
On Tue, Jun 23, 2015 at 2:51 PM, Jesse Gross <je...@nicira.com> wrote:
On Mon, Jun 22, 2015 at 8:08 PM, Pravin Shelar <pshe...@nicira.com> wrote:
On Fri, Jun 19, 2015 at 11:24 AM, Daniele Di Proietto
<diproiet...@vmware.com> wrote:
On 18/06/2015 23:57, "Traynor, Kevin" <kevin.tray...@intel.com> wrote:
-----Original Message-----
From: dev [mailto:dev-boun...@openvswitch.org] On Behalf Of Daniele Di
Proietto
Sent: Tuesday, June 16, 2015 7:39 PM
To: dev@openvswitch.org
Subject: [ovs-dev] [PATCH] dpif-netdev: Check for PKT_RX_RSS_HASH flag.
DPDK mbufs contain a valid RSS hash only if PKT_RX_RSS_HASH is
set in 'ol_flags'. Otherwise the hash is garbage and doesn't
relate to the packet.
This fixes an issue with vhost, which, being a virtual NIC, doesn't
compute the hash.
Unfortunately the ixgbe vPMD doesn't set the PKT_RX_RSS_HASH, forcing
OVS to compute an hash is software. This has a significant impact on
performance (-30% throughput in a single flow setup) which can be
mitigated in the CPU supports crc32c instructions.
As per the other thread on this I'm a bit concerned about the performance
drop from this patch, so I did some testing of this and alternative/
complimentary solutions.
Here's the options I looked at and some comments:
1. This patch in isolation: vhost drops about ~15% vhost-vhost and
phy-vhost-phy (because of sw hash) but also there is drops of ~25% for
phy-phy and ~15% drop for phy-ivshmem-phy.
2. Leave the code as is and let EMC misses happen for vhost rx pkts:
I measure this at ~35% drop if missed *everytime* for vhost-vhost. We
see in testing that it can also never happen, but this is not realistic.
There should be no impact to other DPDK interfaces.
3. Add hash reset for packets from vhost: This is another way of forcing
the software hash for vhost rx and it is roughly equivalent in performance
to 1. for vhost-vhost (~15% drop). While there is a no significant drop
for phy-vhost-phy. There should be no impact to other DPDK interfaces.
4. Apply this patch and turn off Rx Vectorisation. vhost-vhost will drop
~15% as per 1. and there should be nothing significant for phy-vhost-phy.
We would lose the 10% gain that rx vectorisation gave us for phy-phy.
There should be no impact for dpdkr ports.
In terms of not knowing whether the hw hash is valid or not if the flag is
not checked, I would have expected the pmd to return an error on config if
the hash wasn't supported, but I'm not sure that it does.
In the worst case where there was an incorrect hash, it would miss the EMC
which is about a 45% drop for phy-phy. I would think it's pretty safe that
if we configure it, the hash will be correct but I guess there is a
possibility it wouldn't be.
Even if it is possible to get a smaller patch to fix the underlying issue
in DPDK, it would be in DPDK 2.1 at the earliest meaning the performance
would remain low until sometime in August. If it's DPDK 2.2, then it would
be sometime in December. This would mean any performance drops would be
present in OVS 2.4 and possibly OVS 2.5.
Sorry :( but based on the performance drop with this patch in isolation it
would be a NAK from me. My preference would be 3 which gives best
performance,
or 4 which is a bit lower for phy-phy but safer.
Kevin.
Thanks for all the testing. I guess it might make sense to stretch our
interpretation of the API in this case, because it wouldn't affect
correctness.
Unless there any other objection I'm fine with the 3rd approach.
We can use 3rd approach to fix issue on branch 2.4. Then have patch to
check the PKT_RX_RSS_HASH flag on master. By the time we release
branch 2.5 we will have proper fix in DPDK and performance will bounce
back.
I think this is probably a reasonable compromise. I think it's better
to not keep a workaround in for an unbounded amount of time, otherwise
we'll forget about it and it will come back to bite us in the future.
ok, Once the DPDK fix is backported to DPDK 2.0, we can remove the workaround.
That's assuming there will be a DPDK 2.0.1 release, but I have seen no
evidence of such plans in the DPDK camp.
- Panu -
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
