Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-29 Thread Yuzhou (C) 
Hi,

About rx zerocopy, I have a question:

If some application make a socket, then listen and accept, the client 
sends packets to it, but it doesn't recv from this socket right now, all 
persistent grant page would be in used.
So other application cannot receive any packets.  Is my guess right or wrong?

YuZhou

-Original Message-
From: xen-devel-boun...@lists.xen.org [mailto:xen-devel-boun...@lists.xen.org] 
On Behalf Of Joao Martins
Sent: Friday, May 22, 2015 6:27 PM
To: Wei Liu
Cc: ian.campb...@citrix.com; net...@vger.kernel.org; david.vra...@citrix.com; 
xen-de...@lists.xenproject.org; boris.ostrov...@oracle.com
Subject: Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net 
drivers


On 19 May 2015, at 17:39, Wei Liu wei.l...@citrix.com wrote:

 On Tue, May 12, 2015 at 07:18:24PM +0200, Joao Martins wrote:
 
 There have been recently[3] some discussions and issues raised on 
 persistent grants for the block layer, though the numbers above show 
 some significant improvements specially on more network intensive 
 workloads and provide a margin for comparison against future 
 map/unmap improvements.
 
 Any comments or suggestions are welcome, Thanks!
 
 Thanks, the numbers certainly look interesting.
 
 I'm just a bit concerned about the complexity of netback. I've 
 commented on individual patches, we can discuss the issues there.

Thanks a lot for the review! It does add more complexity, mainly for the TX 
path, but I also would like to mention that a portion of this changeset is also 
the persistent grants ops that could potentially live outside.

Joao

 [1] http://article.gmane.org/gmane.linux.network/249383
 [2] http://bit.ly/1IhJfXD
 [3] 
 http://lists.xen.org/archives/html/xen-devel/2015-02/msg02292.html
 
 Joao Martins (13):
  xen-netback: add persistent grant tree ops
  xen-netback: xenbus feature persistent support
  xen-netback: implement TX persistent grants
  xen-netback: implement RX persistent grants
  xen-netback: refactor xenvif_rx_action
  xen-netback: copy buffer on xenvif_start_xmit()
  xen-netback: add persistent tree counters to debugfs
  xen-netback: clone skb if skb-xmit_more is set
  xen-netfront: move grant_{ref,page} to struct grant
  xen-netfront: refactor claim/release grant
  xen-netfront: feature-persistent xenbus support
  xen-netfront: implement TX persistent grants
  xen-netfront: implement RX persistent grants
 
 drivers/net/xen-netback/common.h|  79 
 drivers/net/xen-netback/interface.c |  78 +++-
 drivers/net/xen-netback/netback.c   | 873 
 ++--
 drivers/net/xen-netback/xenbus.c|  24 +
 drivers/net/xen-netfront.c  | 362 ---
 5 files changed, 1216 insertions(+), 200 deletions(-)
 
 --
 2.1.3



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-29 Thread Joao Martins 

On 29 May 2015, at 08:53, Yuzhou (C) vitas.yuz...@huawei.com wrote:
 Hi,
 
   About rx zerocopy, I have a question:
 
   If some application make a socket, then listen and accept, the client 
 sends packets to it, but it doesn't recv from this socket right now, all 
 persistent grant page would be in used.
 So other application cannot receive any packets.  Is my guess right or wrong?

I believe that doesn’t happen: before the skb gets delivered to the protocol 
stack,
skb_orphan_frags gets called which releases the original pages (i.e. the 
persistent
grants) and memcpy to new ones (if the skb is fragmented). This happens because 
I
previously set the flag SKBTX_DEV_ZEROCOPY, which also invokes a callback
in such event.

Once the callback is invoked, the released pages are added a pool within
xen-netfront which later will use for new requests to the backend. Note that 
part
of the data is previously copied on pskb_pull_tail and may unref the initial 
frag
before skb_orphan_frags is called. The callback is still invoked and this page 
is added
to the pool as well.

Joao

 YuZhou
 
 -Original Message-
 From: xen-devel-boun...@lists.xen.org 
 [mailto:xen-devel-boun...@lists.xen.org] On Behalf Of Joao Martins
 Sent: Friday, May 22, 2015 6:27 PM
 To: Wei Liu
 Cc: ian.campb...@citrix.com; net...@vger.kernel.org; david.vra...@citrix.com; 
 xen-de...@lists.xenproject.org;boris.ostrov...@oracle.com
 Subject: Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net 
 drivers
 
 
 On 19 May 2015, at 17:39, Wei Liu wei.l...@citrix.com wrote:
 
 On Tue, May 12, 2015 at 07:18:24PM +0200, Joao Martins wrote:
 
 There have been recently[3] some discussions and issues raised on 
 persistent grants for the block layer, though the numbers above show 
 some significant improvements specially on more network intensive 
 workloads and provide a margin for comparison against future 
 map/unmap improvements.
 
 Any comments or suggestions are welcome, Thanks!
 
 Thanks, the numbers certainly look interesting.
 
 I'm just a bit concerned about the complexity of netback. I've 
 commented on individual patches, we can discuss the issues there.
 
 Thanks a lot for the review! It does add more complexity, mainly for the TX 
 path, but I also would like to mention that a portion of this changeset is 
 also the persistent grants ops that could potentially live outside.
 
 Joao
 
 [1] http://article.gmane.org/gmane.linux.network/249383
 [2] http://bit.ly/1IhJfXD
 [3] 
 http://lists.xen.org/archives/html/xen-devel/2015-02/msg02292.html
 
 Joao Martins (13):
 xen-netback: add persistent grant tree ops
 xen-netback: xenbus feature persistent support
 xen-netback: implement TX persistent grants
 xen-netback: implement RX persistent grants
 xen-netback: refactor xenvif_rx_action
 xen-netback: copy buffer on xenvif_start_xmit()
 xen-netback: add persistent tree counters to debugfs
 xen-netback: clone skb if skb-xmit_more is set
 xen-netfront: move grant_{ref,page} to struct grant
 xen-netfront: refactor claim/release grant
 xen-netfront: feature-persistent xenbus support
 xen-netfront: implement TX persistent grants
 xen-netfront: implement RX persistent grants
 
 drivers/net/xen-netback/common.h|  79 
 drivers/net/xen-netback/interface.c |  78 +++-
 drivers/net/xen-netback/netback.c   | 873 
 ++--
 drivers/net/xen-netback/xenbus.c|  24 +
 drivers/net/xen-netfront.c  | 362 ---
 5 files changed, 1216 insertions(+), 200 deletions(-)
 
 --
 2.1.3
 
 
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

===
João Martins
Research Scientist, Networked Systems and Data Analytics Group

NEC Laboratories Europe
Kurfuerstenanlage 36
D-69115 Heidelberg

Tel. +49 (0)6221 4342-208
Fax: +49 (0)6221 4342-155
e-mail:  joao.mart...@neclab.eu
===
NEC Europe Ltd | Registered Office: Athene, Odyssey Business Park,
West End Road, London, HA4 6QE, GB | Registered in England 2832014


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-22 Thread Joao Martins 

On 19 May 2015, at 17:39, Wei Liu wei.l...@citrix.com wrote:

 On Tue, May 12, 2015 at 07:18:24PM +0200, Joao Martins wrote:
 
 There have been recently[3] some discussions and issues raised on
 persistent grants for the block layer, though the numbers above
 show some significant improvements specially on more network intensive
 workloads and provide a margin for comparison against future map/unmap
 improvements.
 
 Any comments or suggestions are welcome,
 Thanks!
 
 Thanks, the numbers certainly look interesting.
 
 I'm just a bit concerned about the complexity of netback. I've
 commented on individual patches, we can discuss the issues there.

Thanks a lot for the review! It does add more complexity, mainly for
the TX path, but I also would like to mention that a portion of this
changeset is also the persistent grants ops that could potentially live
outside.

Joao

 [1] http://article.gmane.org/gmane.linux.network/249383
 [2] http://bit.ly/1IhJfXD
 [3] http://lists.xen.org/archives/html/xen-devel/2015-02/msg02292.html
 
 Joao Martins (13):
  xen-netback: add persistent grant tree ops
  xen-netback: xenbus feature persistent support
  xen-netback: implement TX persistent grants
  xen-netback: implement RX persistent grants
  xen-netback: refactor xenvif_rx_action
  xen-netback: copy buffer on xenvif_start_xmit()
  xen-netback: add persistent tree counters to debugfs
  xen-netback: clone skb if skb-xmit_more is set
  xen-netfront: move grant_{ref,page} to struct grant
  xen-netfront: refactor claim/release grant
  xen-netfront: feature-persistent xenbus support
  xen-netfront: implement TX persistent grants
  xen-netfront: implement RX persistent grants
 
 drivers/net/xen-netback/common.h|  79 
 drivers/net/xen-netback/interface.c |  78 +++-
 drivers/net/xen-netback/netback.c   | 873 
 ++--
 drivers/net/xen-netback/xenbus.c|  24 +
 drivers/net/xen-netfront.c  | 362 ---
 5 files changed, 1216 insertions(+), 200 deletions(-)
 
 -- 
 2.1.3



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-19 Thread Wei Liu 
On Tue, May 12, 2015 at 07:18:24PM +0200, Joao Martins wrote:

 There have been recently[3] some discussions and issues raised on
 persistent grants for the block layer, though the numbers above
 show some significant improvements specially on more network intensive
 workloads and provide a margin for comparison against future map/unmap
 improvements.
 
 Any comments or suggestions are welcome,
 Thanks!

Thanks, the numbers certainly look interesting.

I'm just a bit concerned about the complexity of netback. I've
commented on individual patches, we can discuss the issues there.

Wei.

 Joao
 
 [1] http://article.gmane.org/gmane.linux.network/249383
 [2] http://bit.ly/1IhJfXD
 [3] http://lists.xen.org/archives/html/xen-devel/2015-02/msg02292.html
 
 Joao Martins (13):
   xen-netback: add persistent grant tree ops
   xen-netback: xenbus feature persistent support
   xen-netback: implement TX persistent grants
   xen-netback: implement RX persistent grants
   xen-netback: refactor xenvif_rx_action
   xen-netback: copy buffer on xenvif_start_xmit()
   xen-netback: add persistent tree counters to debugfs
   xen-netback: clone skb if skb-xmit_more is set
   xen-netfront: move grant_{ref,page} to struct grant
   xen-netfront: refactor claim/release grant
   xen-netfront: feature-persistent xenbus support
   xen-netfront: implement TX persistent grants
   xen-netfront: implement RX persistent grants
 
  drivers/net/xen-netback/common.h|  79 
  drivers/net/xen-netback/interface.c |  78 +++-
  drivers/net/xen-netback/netback.c   | 873 
 ++--
  drivers/net/xen-netback/xenbus.c|  24 +
  drivers/net/xen-netfront.c  | 362 ---
  5 files changed, 1216 insertions(+), 200 deletions(-)
 
 -- 
 2.1.3

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-13 Thread David Vrabel 
On 12/05/15 18:18, Joao Martins wrote:
 
 Packet I/O Tests:
 
 Measured on a Intel Xeon E5-1650 v2, Xen 4.5, no HT. Used pktgen burst 1
 and clone_skbs 10 (to avoid alloc skb overheads) with various pkt
 sizes. All tests are DomU - Dom0, unless specified otherwise.

Are all these measurements with a single domU with a single VIF?

The biggest problem with a persistent grant method is the amount of
grant table and maptrack resources it requires.  How well does this
scale to 1000s of VIFs?

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH 00/13] Persistent grant maps for xen net drivers

2015-05-13 Thread Joao Martins 

On 13 May 2015, at 12:50, David Vrabel david.vra...@citrix.com wrote:

 On 12/05/15 18:18, Joao Martins wrote:
 
 Packet I/O Tests:
 
 Measured on a Intel Xeon E5-1650 v2, Xen 4.5, no HT. Used pktgen burst 1
 and clone_skbs 10 (to avoid alloc skb overheads) with various pkt
 sizes. All tests are DomU - Dom0, unless specified otherwise.
 
 Are all these measurements with a single domU with a single VIF?
 
 The biggest problem with a persistent grant method is the amount of
 grant table and maptrack resources it requires.  How well does this
 scale to 1000s of VIFs?

Correct. I was more focused on throughput benefits with persistent grants,
as opposed to scalability with a large number of guests. I will do more tests
with more guests and provide you the numbers. Most likely it won't scale to
that numbers of VIFs, given that the maptrack size increases much quicker 
(nr_vifs * 512 * nr_queues grants mapped) thus I also added the option of not
exposing feature-persistent if xen-netback.max_persistent_gnts module param is
set to 0. I am aware of these issues with persistent grants, but the case I had
in mind, was fewer VMs with higher throughput, which I believe to be the 
compromise
with persistent grants.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel