On 2019-04-15 12:58, Alex McWhirter wrote:
> On 2019-04-15 12:43, Darrell Budic wrote: Interesting. Who's 10g cards and
> which offload settings did you disable? Did you do that on the servers or the
> vm host clients or both?
>
> On Apr 15, 2019, at 11:37 AM, Alex McWhirter <a...@triadic.us> wrote:
>
> I went in and disabled TCP offload on all the nics, huge performance boost.
> went from 110MB/s to 240MB/s seq writes, reads lost a bit of performance
> going down to 680MB/s, but that's a decent trade off. Latency is still really
> high though, need to work on that. I think some more TCP tuning might help.
>
> Those changes didn't do a whole lot, but i ended up enabling
> performance.read-ahead on the gluster volume. my blockdev read ahead values
> were already 8192, which seemed good enough. Not sure if ovirt set those, or
> if it's just the defaults of my raid controller.
>
> Anyways up to 350MB/s writes, 700MB/s reads. Which so happens to correlate
> with the saturation of my 10G network. Latency is still a slight issue, but
> at least now im not blocking :)
>
> _______________________________________________
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/message/5COPHAIVCVK42KMMGWZQVMNGDH6Q32ZC/
These are dual port qlogic QLGE cards, plugging into dual Cisco Nexus
3064's with VPC to allow me to LACP across two switches. These are
FCoE/10GBE cards, so on the Cisco Switches i had to disable lldp on the
ports to stop FCoE initiator errors from disabling ports (as i don't use
FCoE atm)
bond options are "mode=4 lacp_rate=1 miimon=100 xmit_hash_policy=1"
then i have the following /sbin/ifup-local script that triggers on
storage network creation
#!/bin/bash
case "$1" in
Storage)
/sbin/ethtool -K ens2f0 tx off rx off tso off gso off
/sbin/ethtool -K ens2f1 tx off rx off tso off gso off
/sbin/ip link set dev ens2f0 txqueuelen 10000
/sbin/ip link set dev ens2f1 txqueuelen 10000
/sbin/ip link set dev bond2 txqueuelen 10000
/sbin/ip link set dev Storage txqueuelen 10000
;;
*)
;;
esac
exit 0
if you have lro, disable it too IMO, these cards do not do lro so it's
not applicable to me.
This did cut down my read performance by about 50MB/s, but my write went
from 98-110MB/s to about 240MB/s, then enabling read-ahead got me to the
350MB/s it should have been.
Oh and i did it on both, the VM hosts and storage machines. Same cards
in all of them._______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/EZF2G3AWM3GSDR35CLP2SKABALQ6H4KN/
