[PATCHv2 dontapply] vhost-net tx tuning

2011-02-01 Thread Michael S. Tsirkin 
OK, so thinking about it more, maybe the issue is this:
tx becomes full. We process one request and interrupt the guest,
then it adds one request and the queue is full again.

Maybe the following will help it stabilize?  By default with it we will
only interrupt when we see an empty ring.
Which is liklely too much: pls try other values
in the middle: e.g. make bufs half the ring,
or bytes some small value like half ring * 200, or packets some
small value etc.

Set any one parameter to 0 to get current
behaviour (interrupt immediately when enabled).

Warning: completely untested.

Signed-off-by: Michael S. Tsirkin m...@redhat.com

---

diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index aac05bc..6769cdc 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -32,6 +32,13 @@
  * Using this limit prevents one virtqueue from starving others. */
 #define VHOST_NET_WEIGHT 0x8
 
+int tx_bytes_coalesce = 10;
+module_param(tx_bytes_coalesce, int, 0644);
+int tx_bufs_coalesce = 10;
+module_param(tx_bufs_coalesce, int, 0644);
+int tx_packets_coalesce = 10;
+module_param(tx_packets_coalesce, int, 0644);
+
 enum {
VHOST_NET_VQ_RX = 0,
VHOST_NET_VQ_TX = 1,
@@ -127,6 +134,9 @@ static void handle_tx(struct vhost_net *net)
int err, wmem;
size_t hdr_size;
struct socket *sock;
+   int bytes_coalesced = 0;
+   int bufs_coalesced = 0;
+   int packets_coalesced = 0;
 
/* TODO: check that we are running from vhost_worker? */
sock = rcu_dereference_check(vq-private_data, 1);
@@ -196,14 +206,26 @@ static void handle_tx(struct vhost_net *net)
if (err != len)
pr_debug(Truncated TX packet: 
  len %d != %zd\n, err, len);
-   vhost_add_used_and_signal(net-dev, vq, head, 0);
total_len += len;
+   packets_coalesced += 1;
+   bytes_coalesced += len;
+   bufs_coalesced += out;
+   if (unlikely(packets_coalesced  tx_packets_coalesce ||
+bytes_coalesced  tx_bytes_coalesce ||
+bufs_coalesced  tx_bufs_coalesce))
+   vhost_add_used_and_signal(net-dev, vq, head, 0);
+   else
+   vhost_add_used(vq, head, 0);
if (unlikely(total_len = VHOST_NET_WEIGHT)) {
vhost_poll_queue(vq-poll);
break;
}
}
 
+   if (likely(packets_coalesced 
+  bytes_coalesced 
+  bufs_coalesced))
+   vhost_signal(net-dev, vq);
mutex_unlock(vq-mutex);
 }
 
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHv2 dontapply] vhost-net tx tuning

2011-02-01 Thread Sridhar Samudrala 
On Tue, 2011-02-01 at 17:52 +0200, Michael S. Tsirkin wrote:
 OK, so thinking about it more, maybe the issue is this:
 tx becomes full. We process one request and interrupt the guest,
 then it adds one request and the queue is full again.
 
 Maybe the following will help it stabilize?  By default with it we will
 only interrupt when we see an empty ring.
 Which is liklely too much: pls try other values
 in the middle: e.g. make bufs half the ring,
 or bytes some small value like half ring * 200, or packets some
 small value etc.
 
 Set any one parameter to 0 to get current
 behaviour (interrupt immediately when enabled).
 
 Warning: completely untested.
 
 Signed-off-by: Michael S. Tsirkin m...@redhat.com
 
 ---
 
 diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
 index aac05bc..6769cdc 100644
 --- a/drivers/vhost/net.c
 +++ b/drivers/vhost/net.c
 @@ -32,6 +32,13 @@
   * Using this limit prevents one virtqueue from starving others. */
  #define VHOST_NET_WEIGHT 0x8
 
 +int tx_bytes_coalesce = 10;
 +module_param(tx_bytes_coalesce, int, 0644);
 +int tx_bufs_coalesce = 10;
 +module_param(tx_bufs_coalesce, int, 0644);
 +int tx_packets_coalesce = 10;
 +module_param(tx_packets_coalesce, int, 0644);
 +
  enum {
   VHOST_NET_VQ_RX = 0,
   VHOST_NET_VQ_TX = 1,
 @@ -127,6 +134,9 @@ static void handle_tx(struct vhost_net *net)
   int err, wmem;
   size_t hdr_size;
   struct socket *sock;
 + int bytes_coalesced = 0;
 + int bufs_coalesced = 0;
 + int packets_coalesced = 0;
 
   /* TODO: check that we are running from vhost_worker? */
   sock = rcu_dereference_check(vq-private_data, 1);
 @@ -196,14 +206,26 @@ static void handle_tx(struct vhost_net *net)
   if (err != len)
   pr_debug(Truncated TX packet: 
 len %d != %zd\n, err, len);
 - vhost_add_used_and_signal(net-dev, vq, head, 0);
   total_len += len;
 + packets_coalesced += 1;
 + bytes_coalesced += len;
 + bufs_coalesced += out;
 + if (unlikely(packets_coalesced  tx_packets_coalesce ||
 +  bytes_coalesced  tx_bytes_coalesce ||
 +  bufs_coalesced  tx_bufs_coalesce))
 + vhost_add_used_and_signal(net-dev, vq, head, 0);

I think the counters that exceed the limits need to be reset to 0 here.
Otherwise we keep signaling for every buffer once we hit this condition.

Thanks
Sridhar

 + else
 + vhost_add_used(vq, head, 0);
   if (unlikely(total_len = VHOST_NET_WEIGHT)) {
   vhost_poll_queue(vq-poll);
   break;
   }
   }
 
 + if (likely(packets_coalesced 
 +bytes_coalesced 
 +bufs_coalesced))
 + vhost_signal(net-dev, vq);
   mutex_unlock(vq-mutex);
  }
 
 --
 To unsubscribe from this list: send the line unsubscribe netdev in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHv2 dontapply] vhost-net tx tuning

2011-02-01 Thread Shirley Ma 
On Tue, 2011-02-01 at 15:07 -0800, Sridhar Samudrala wrote:
 I think the counters that exceed the limits need to be reset to 0
 here.
 Otherwise we keep signaling for every buffer once we hit this
 condition. 

I will modify the patch to rerun the test to see the difference.

Shirley

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHv2 dontapply] vhost-net tx tuning

2011-02-01 Thread Michael S. Tsirkin 
On Tue, Feb 01, 2011 at 03:07:38PM -0800, Sridhar Samudrala wrote:
 On Tue, 2011-02-01 at 17:52 +0200, Michael S. Tsirkin wrote:
  OK, so thinking about it more, maybe the issue is this:
  tx becomes full. We process one request and interrupt the guest,
  then it adds one request and the queue is full again.
  
  Maybe the following will help it stabilize?  By default with it we will
  only interrupt when we see an empty ring.
  Which is liklely too much: pls try other values
  in the middle: e.g. make bufs half the ring,
  or bytes some small value like half ring * 200, or packets some
  small value etc.
  
  Set any one parameter to 0 to get current
  behaviour (interrupt immediately when enabled).
  
  Warning: completely untested.
  
  Signed-off-by: Michael S. Tsirkin m...@redhat.com
  
  ---
  
  diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
  index aac05bc..6769cdc 100644
  --- a/drivers/vhost/net.c
  +++ b/drivers/vhost/net.c
  @@ -32,6 +32,13 @@
* Using this limit prevents one virtqueue from starving others. */
   #define VHOST_NET_WEIGHT 0x8
  
  +int tx_bytes_coalesce = 10;
  +module_param(tx_bytes_coalesce, int, 0644);
  +int tx_bufs_coalesce = 10;
  +module_param(tx_bufs_coalesce, int, 0644);
  +int tx_packets_coalesce = 10;
  +module_param(tx_packets_coalesce, int, 0644);
  +
   enum {
  VHOST_NET_VQ_RX = 0,
  VHOST_NET_VQ_TX = 1,
  @@ -127,6 +134,9 @@ static void handle_tx(struct vhost_net *net)
  int err, wmem;
  size_t hdr_size;
  struct socket *sock;
  +   int bytes_coalesced = 0;
  +   int bufs_coalesced = 0;
  +   int packets_coalesced = 0;
  
  /* TODO: check that we are running from vhost_worker? */
  sock = rcu_dereference_check(vq-private_data, 1);
  @@ -196,14 +206,26 @@ static void handle_tx(struct vhost_net *net)
  if (err != len)
  pr_debug(Truncated TX packet: 
len %d != %zd\n, err, len);
  -   vhost_add_used_and_signal(net-dev, vq, head, 0);
  total_len += len;
  +   packets_coalesced += 1;
  +   bytes_coalesced += len;
  +   bufs_coalesced += out;
  +   if (unlikely(packets_coalesced  tx_packets_coalesce ||
  +bytes_coalesced  tx_bytes_coalesce ||
  +bufs_coalesced  tx_bufs_coalesce))
  +   vhost_add_used_and_signal(net-dev, vq, head, 0);
 
 I think the counters that exceed the limits need to be reset to 0 here.
 Otherwise we keep signaling for every buffer once we hit this condition.
 
 Thanks
 Sridhar

Correct, good catch.

  +   else
  +   vhost_add_used(vq, head, 0);
  if (unlikely(total_len = VHOST_NET_WEIGHT)) {
  vhost_poll_queue(vq-poll);
  break;
  }
  }
  
  +   if (likely(packets_coalesced 
  +  bytes_coalesced 
  +  bufs_coalesced))
  +   vhost_signal(net-dev, vq);
  mutex_unlock(vq-mutex);
   }
  
  --
  To unsubscribe from this list: send the line unsubscribe netdev in
  the body of a message to majord...@vger.kernel.org
  More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html