On Tue, Sep 19, 2017 at 5:34 PM, Samudrala, Sridhar <sridhar.samudr...@intel.com> wrote: > On 9/12/2017 3:53 PM, Tom Herbert wrote: >> >> On Tue, Sep 12, 2017 at 3:31 PM, Samudrala, Sridhar >> <sridhar.samudr...@intel.com> wrote: >>> >>> >>> On 9/12/2017 8:47 AM, Eric Dumazet wrote: >>>> >>>> On Mon, 2017-09-11 at 23:27 -0700, Samudrala, Sridhar wrote: >>>>> >>>>> On 9/11/2017 8:53 PM, Eric Dumazet wrote: >>>>>> >>>>>> On Mon, 2017-09-11 at 20:12 -0700, Tom Herbert wrote: >>>>>> >>>>>>> Two ints in sock_common for this purpose is quite expensive and the >>>>>>> use case for this is limited-- even if a RX->TX queue mapping were >>>>>>> introduced to eliminate the queue pair assumption this still won't >>>>>>> help if the receive and transmit interfaces are different for the >>>>>>> connection. I think we really need to see some very compelling >>>>>>> results >>>>>>> to be able to justify this. >>>>> >>>>> Will try to collect and post some perf data with symmetric queue >>>>> configuration. > > > Here is some performance data i collected with memcached workload over > ixgbe 10Gb NIC with mcblaster benchmark. > ixgbe is configured with 16 queues and rx-usecs is set to 1000 for a very > low > interrupt rate. > ethtool -L p1p1 combined 16 > ethtool -C p1p1 rx-usecs 1000 > and busy poll is set to 1000usecs > sysctl net.core.busy_poll = 1000 > > 16 threads 800K requests/sec > ============================= > rtt(min/avg/max)usecs intr/sec contextswitch/sec > ----------------------------------------------------------------------- > Default 2/182/10641 23391 61163 > Symmetric Queues 2/50/6311 20457 32843 > > 32 threads 800K requests/sec > ============================= > rtt(min/avg/max)usecs intr/sec contextswitch/sec > ------------------------------------------------------------------------ > Default 2/162/6390 32168 69450 > Symmetric Queues 2/50/3853 35044 35847 > No idea what "Default" configuration is. Please report how xps_cpus is being set, how many RSS queues there are, and what the mapping is between RSS queues and CPUs and shared caches. Also, whether and threads are pinned.
Thanks, Tom
