[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 3/5/2016 2:17 AM, Stephen Hemminger wrote:
> Resending them now. I don't understand the issue with merge-able header.
> Virtio negotiation is symmetric, if receiver is using merge-able header
> then the transmitter needs to send it also.

Yes, both receiver and transmitter use the same format of header though
merge-able is actually only meaningful in RX path.

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Stephen Hemminger]

On Fri, 4 Mar 2016 06:18:17 +
"Xie, Huawei"  wrote:

> On 1/14/2016 9:49 PM, Xie, Huawei wrote:
> > On 1/6/2016 8:04 PM, Thomas Monjalon wrote:
> >> 2016-01-05 08:10, Xie, Huawei:
> >>> On 10/26/2015 10:06 PM, Xie, Huawei wrote:
>  On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
> > This is a tested version of the virtio Tx performance improvements
> > that I posted earlier on the list, and described at the DPDK Userspace
> > meeting in Dublin. Together they get a 25% performance improvement for
> > both small packet and large multi-segment packet case when testing
> > from DPDK guest application to Linux KVM host.
> >
> > Stephen Hemminger (5):
> >   virtio: clean up space checks on xmit
> >   virtio: don't use unlikely for normal tx stuff
> >   virtio: use indirect ring elements
> >   virtio: use any layout on transmit
> >   virtio: optimize transmit enqueue
>  There is one open why merge-able header is used in tx path. Since old
>  implementation is also using the merge-able header in tx path if this
>  feature is negotiated, i choose to ack the patch and address this later
>  if not now.
> 
>  Acked-by: Huawei Xie 
> >>> Thomas:

Resending them now. I don't understand the issue with merge-able header.
Virtio negotiation is symmetric, if receiver is using merge-able header
then the transmitter needs to send it also.

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 1/14/2016 9:49 PM, Xie, Huawei wrote:
> On 1/6/2016 8:04 PM, Thomas Monjalon wrote:
>> 2016-01-05 08:10, Xie, Huawei:
>>> On 10/26/2015 10:06 PM, Xie, Huawei wrote:
 On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
> This is a tested version of the virtio Tx performance improvements
> that I posted earlier on the list, and described at the DPDK Userspace
> meeting in Dublin. Together they get a 25% performance improvement for
> both small packet and large multi-segment packet case when testing
> from DPDK guest application to Linux KVM host.
>
> Stephen Hemminger (5):
>   virtio: clean up space checks on xmit
>   virtio: don't use unlikely for normal tx stuff
>   virtio: use indirect ring elements
>   virtio: use any layout on transmit
>   virtio: optimize transmit enqueue
 There is one open why merge-able header is used in tx path. Since old
 implementation is also using the merge-able header in tx path if this
 feature is negotiated, i choose to ack the patch and address this later
 if not now.

 Acked-by: Huawei Xie 
>>> Thomas:
>>> This patch isn't in the patchwork. Does Stephen need to send a new one?
>> Yes please, I cannot find them in patchwork.
> ping

Ping.


>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 1/6/2016 8:04 PM, Thomas Monjalon wrote:
> 2016-01-05 08:10, Xie, Huawei:
>> On 10/26/2015 10:06 PM, Xie, Huawei wrote:
>>> On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
 This is a tested version of the virtio Tx performance improvements
 that I posted earlier on the list, and described at the DPDK Userspace
 meeting in Dublin. Together they get a 25% performance improvement for
 both small packet and large multi-segment packet case when testing
 from DPDK guest application to Linux KVM host.

 Stephen Hemminger (5):
   virtio: clean up space checks on xmit
   virtio: don't use unlikely for normal tx stuff
   virtio: use indirect ring elements
   virtio: use any layout on transmit
   virtio: optimize transmit enqueue
>>> There is one open why merge-able header is used in tx path. Since old
>>> implementation is also using the merge-able header in tx path if this
>>> feature is negotiated, i choose to ack the patch and address this later
>>> if not now.
>>>
>>> Acked-by: Huawei Xie 
>> Thomas:
>> This patch isn't in the patchwork. Does Stephen need to send a new one?
> Yes please, I cannot find them in patchwork.

ping

>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Thomas Monjalon]

2016-01-05 08:10, Xie, Huawei:
> On 10/26/2015 10:06 PM, Xie, Huawei wrote:
> > On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
> >> This is a tested version of the virtio Tx performance improvements
> >> that I posted earlier on the list, and described at the DPDK Userspace
> >> meeting in Dublin. Together they get a 25% performance improvement for
> >> both small packet and large multi-segment packet case when testing
> >> from DPDK guest application to Linux KVM host.
> >>
> >> Stephen Hemminger (5):
> >>   virtio: clean up space checks on xmit
> >>   virtio: don't use unlikely for normal tx stuff
> >>   virtio: use indirect ring elements
> >>   virtio: use any layout on transmit
> >>   virtio: optimize transmit enqueue
> > There is one open why merge-able header is used in tx path. Since old
> > implementation is also using the merge-able header in tx path if this
> > feature is negotiated, i choose to ack the patch and address this later
> > if not now.
> >
> > Acked-by: Huawei Xie 
> 
> Thomas:
> This patch isn't in the patchwork. Does Stephen need to send a new one?

Yes please, I cannot find them in patchwork.

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/26/2015 10:06 PM, Xie, Huawei wrote:
> On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
>> This is a tested version of the virtio Tx performance improvements
>> that I posted earlier on the list, and described at the DPDK Userspace
>> meeting in Dublin. Together they get a 25% performance improvement for
>> both small packet and large multi-segment packet case when testing
>> from DPDK guest application to Linux KVM host.
>>
>> Stephen Hemminger (5):
>>   virtio: clean up space checks on xmit
>>   virtio: don't use unlikely for normal tx stuff
>>   virtio: use indirect ring elements
>>   virtio: use any layout on transmit
>>   virtio: optimize transmit enqueue
> There is one open why merge-able header is used in tx path. Since old
> implementation is also using the merge-able header in tx path if this
> feature is negotiated, i choose to ack the patch and address this later
> if not now.
>
> Acked-by: Huawei Xie 

Thomas:
This patch isn't in the patchwork. Does Stephen need to send a new one?

>
>
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Stephen Hemminger]

You need to use the extended mergeable rx buffer format.
It is a virtio spec requirement, look at Linux virtio network driver or ask
the virtio maintainers for Linux
if you need more clarification.

On Tue, Oct 27, 2015 at 10:56 AM, Xie, Huawei  wrote:

> On 10/27/2015 7:52 AM, Stephen Hemminger wrote:
> > On Fri, 23 Oct 2015 09:00:38 +
> > "Xie, Huawei"  wrote:
> >
>  Why use merge-able rx header here in the tx region?
> >>> If mergeable rx is negotiated then the header must be used for
> >>> both Tx and Rx. I chose to allocate the largest possible header
> >>> needed, rather than having to deal with variable size data structure.
> >> Our original code is also using merge-able header for TX descriptor if
> >> this negotiated.
> >> I checked the virtio spec, all of the merge-able header is about
> >> receiving buffers, which is expected. That is why i feel weird here.
> >> Maybe not a big deal?
> > Since num_buffers is only in merge-able header, the negotiation is
> implied
> > to be symmetric.
> >
> Can we come to the conclusion that in tx case, we use merge-able header
> though number_buffers is not used at all?
> > Reading 0.95 spec
> >
> > Under "Packet Transmission"
> >  3. If the driver negotatied the VIRTIO_NET_F_MGR_RXBUF feature
> > the num_buffers field is set to zero.
> >
> >
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Stephen Hemminger]

On Fri, 23 Oct 2015 09:00:38 +
"Xie, Huawei"  wrote:

> >> Why use merge-able rx header here in the tx region?  
> > If mergeable rx is negotiated then the header must be used for
> > both Tx and Rx. I chose to allocate the largest possible header
> > needed, rather than having to deal with variable size data structure.  
> Our original code is also using merge-able header for TX descriptor if
> this negotiated.
> I checked the virtio spec, all of the merge-able header is about
> receiving buffers, which is expected. That is why i feel weird here.
> Maybe not a big deal?

Since num_buffers is only in merge-able header, the negotiation is implied
to be symmetric.


Reading 0.95 spec 

Under "Packet Transmission"
 3. If the driver negotatied the VIRTIO_NET_F_MGR_RXBUF feature
the num_buffers field is set to zero.

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/27/2015 10:23 AM, Stephen Hemminger wrote:
> You need to use the extended mergeable rx buffer format.
> It is a virtio spec requirement, look at Linux virtio network driver
> or ask the virtio maintainers for Linux
> if you need more clarification.
Yes, it is spec requirement as far as we know though num_buffers might
not be used.
So far not a big deal for us. Let us get clarification from mst later.
>
> On Tue, Oct 27, 2015 at 10:56 AM, Xie, Huawei  > wrote:
>
> On 10/27/2015 7:52 AM, Stephen Hemminger wrote:
> > On Fri, 23 Oct 2015 09:00:38 +
> > "Xie, Huawei"  > wrote:
> >
>  Why use merge-able rx header here in the tx region?
> >>> If mergeable rx is negotiated then the header must be used for
> >>> both Tx and Rx. I chose to allocate the largest possible header
> >>> needed, rather than having to deal with variable size data
> structure.
> >> Our original code is also using merge-able header for TX
> descriptor if
> >> this negotiated.
> >> I checked the virtio spec, all of the merge-able header is about
> >> receiving buffers, which is expected. That is why i feel weird
> here.
> >> Maybe not a big deal?
> > Since num_buffers is only in merge-able header, the negotiation
> is implied
> > to be symmetric.
> >
> Can we come to the conclusion that in tx case, we use merge-able
> header
> though number_buffers is not used at all?
> > Reading 0.95 spec
> >
> > Under "Packet Transmission"
> >  3. If the driver negotatied the VIRTIO_NET_F_MGR_RXBUF feature
> > the num_buffers field is set to zero.
> >
> >
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/27/2015 7:52 AM, Stephen Hemminger wrote:
> On Fri, 23 Oct 2015 09:00:38 +
> "Xie, Huawei"  wrote:
>
 Why use merge-able rx header here in the tx region?  
>>> If mergeable rx is negotiated then the header must be used for
>>> both Tx and Rx. I chose to allocate the largest possible header
>>> needed, rather than having to deal with variable size data structure.  
>> Our original code is also using merge-able header for TX descriptor if
>> this negotiated.
>> I checked the virtio spec, all of the merge-able header is about
>> receiving buffers, which is expected. That is why i feel weird here.
>> Maybe not a big deal?
> Since num_buffers is only in merge-able header, the negotiation is implied
> to be symmetric.
>
Can we come to the conclusion that in tx case, we use merge-able header
though number_buffers is not used at all?
> Reading 0.95 spec 
>
> Under "Packet Transmission"
>  3. If the driver negotatied the VIRTIO_NET_F_MGR_RXBUF feature
> the num_buffers field is set to zero.
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/19/2015 1:16 PM, Stephen Hemminger wrote:
> This is a tested version of the virtio Tx performance improvements
> that I posted earlier on the list, and described at the DPDK Userspace
> meeting in Dublin. Together they get a 25% performance improvement for
> both small packet and large multi-segment packet case when testing
> from DPDK guest application to Linux KVM host.
>
> Stephen Hemminger (5):
>   virtio: clean up space checks on xmit
>   virtio: don't use unlikely for normal tx stuff
>   virtio: use indirect ring elements
>   virtio: use any layout on transmit
>   virtio: optimize transmit enqueue
There is one open why merge-able header is used in tx path. Since old
implementation is also using the merge-able header in tx path if this
feature is negotiated, i choose to ack the patch and address this later
if not now.

Acked-by: Huawei Xie 

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/23/2015 12:05 AM, Stephen Hemminger wrote:
> On Thu, 22 Oct 2015 10:38:33 +
> "Xie, Huawei"  wrote:
>
>> On 10/21/2015 9:20 PM, Thomas Monjalon wrote:
>>> 2015-10-18 22:16, Stephen Hemminger:
 This is a tested version of the virtio Tx performance improvements
 that I posted earlier on the list, and described at the DPDK Userspace
 meeting in Dublin. Together they get a 25% performance improvement for
 both small packet and large multi-segment packet case when testing
 from DPDK guest application to Linux KVM host.

 Stephen Hemminger (5):
   virtio: clean up space checks on xmit
   virtio: don't use unlikely for normal tx stuff
   virtio: use indirect ring elements
   virtio: use any layout on transmit
   virtio: optimize transmit enqueue
>>> Huawei, do you ack this series?
>>>
>> Okay with this patchset with two remained questions,
>>
>> +/* Region reserved to allow for transmit header and indirect ring */
>> +#define VIRTIO_MAX_TX_INDIRECT 8
>> +struct virtio_tx_region {
>> +struct virtio_net_hdr_mrg_rxbuf tx_hdr;
>>
>> Why use merge-able rx header here in the tx region?
> If mergeable rx is negotiated then the header must be used for
> both Tx and Rx. I chose to allocate the largest possible header
> needed, rather than having to deal with variable size data structure.
Our original code is also using merge-able header for TX descriptor if
this negotiated.
I checked the virtio spec, all of the merge-able header is about
receiving buffers, which is expected. That is why i feel weird here.
Maybe not a big deal?
>>> +   struct vring_desc tx_indir[VIRTIO_MAX_TX_INDIRECT]
>>> +  __attribute__((__aligned__(16)));
>> WARNING: __aligned(size) is preferred over __attribute__((aligned(size)))
> That is true in kernel, but there is no __aligned macro in DPDK code.
ok, then we ignore this warning as __rte_cache_aligned is also using
this style.
> It could be changed to __rte_aligned_16?
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/22/2015 6:39 PM, Xie, Huawei wrote:
> On 10/21/2015 9:20 PM, Thomas Monjalon wrote:
>> 2015-10-18 22:16, Stephen Hemminger:
>>> This is a tested version of the virtio Tx performance improvements
>>> that I posted earlier on the list, and described at the DPDK Userspace
>>> meeting in Dublin. Together they get a 25% performance improvement for
>>> both small packet and large multi-segment packet case when testing
>>> from DPDK guest application to Linux KVM host.
>>>
>>> Stephen Hemminger (5):
>>>   virtio: clean up space checks on xmit
>>>   virtio: don't use unlikely for normal tx stuff
>>>   virtio: use indirect ring elements
>>>   virtio: use any layout on transmit
>>>   virtio: optimize transmit enqueue
>> Huawei, do you ack this series?
>>
> Okay with this patchset with two remained questions,
Forget to cc Stephen.
>
> +/* Region reserved to allow for transmit header and indirect ring */
> +#define VIRTIO_MAX_TX_INDIRECT 8
> +struct virtio_tx_region {
> + struct virtio_net_hdr_mrg_rxbuf tx_hdr;
>
> Why use merge-able rx header here in the tx region?
>
>> +struct vring_desc tx_indir[VIRTIO_MAX_TX_INDIRECT]
>> +   __attribute__((__aligned__(16)));
> WARNING: __aligned(size) is preferred over __attribute__((aligned(size)))
> [...]
>
>
>
>

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Xie, Huawei]

On 10/21/2015 9:20 PM, Thomas Monjalon wrote:
> 2015-10-18 22:16, Stephen Hemminger:
>> This is a tested version of the virtio Tx performance improvements
>> that I posted earlier on the list, and described at the DPDK Userspace
>> meeting in Dublin. Together they get a 25% performance improvement for
>> both small packet and large multi-segment packet case when testing
>> from DPDK guest application to Linux KVM host.
>>
>> Stephen Hemminger (5):
>>   virtio: clean up space checks on xmit
>>   virtio: don't use unlikely for normal tx stuff
>>   virtio: use indirect ring elements
>>   virtio: use any layout on transmit
>>   virtio: optimize transmit enqueue
> Huawei, do you ack this series?
>
Okay with this patchset with two remained questions,

+/* Region reserved to allow for transmit header and indirect ring */
+#define VIRTIO_MAX_TX_INDIRECT 8
+struct virtio_tx_region {
+   struct virtio_net_hdr_mrg_rxbuf tx_hdr;

Why use merge-able rx header here in the tx region?

> + struct vring_desc tx_indir[VIRTIO_MAX_TX_INDIRECT]
> +__attribute__((__aligned__(16)));

WARNING: __aligned(size) is preferred over __attribute__((aligned(size)))
[...]

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Stephen Hemminger]

On Thu, 22 Oct 2015 10:38:33 +
"Xie, Huawei"  wrote:

> On 10/21/2015 9:20 PM, Thomas Monjalon wrote:
> > 2015-10-18 22:16, Stephen Hemminger:
> >> This is a tested version of the virtio Tx performance improvements
> >> that I posted earlier on the list, and described at the DPDK Userspace
> >> meeting in Dublin. Together they get a 25% performance improvement for
> >> both small packet and large multi-segment packet case when testing
> >> from DPDK guest application to Linux KVM host.
> >>
> >> Stephen Hemminger (5):
> >>   virtio: clean up space checks on xmit
> >>   virtio: don't use unlikely for normal tx stuff
> >>   virtio: use indirect ring elements
> >>   virtio: use any layout on transmit
> >>   virtio: optimize transmit enqueue
> > Huawei, do you ack this series?
> >
> Okay with this patchset with two remained questions,
> 
> +/* Region reserved to allow for transmit header and indirect ring */
> +#define VIRTIO_MAX_TX_INDIRECT 8
> +struct virtio_tx_region {
> + struct virtio_net_hdr_mrg_rxbuf tx_hdr;
> 
> Why use merge-able rx header here in the tx region?

If mergeable rx is negotiated then the header must be used for
both Tx and Rx. I chose to allocate the largest possible header
needed, rather than having to deal with variable size data structure.

> 
> > +   struct vring_desc tx_indir[VIRTIO_MAX_TX_INDIRECT]
> > +  __attribute__((__aligned__(16)));
> 
> WARNING: __aligned(size) is preferred over __attribute__((aligned(size)))

That is true in kernel, but there is no __aligned macro in DPDK code.
It could be changed to __rte_aligned_16?

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Thomas Monjalon]

2015-10-18 22:16, Stephen Hemminger:
> This is a tested version of the virtio Tx performance improvements
> that I posted earlier on the list, and described at the DPDK Userspace
> meeting in Dublin. Together they get a 25% performance improvement for
> both small packet and large multi-segment packet case when testing
> from DPDK guest application to Linux KVM host.
> 
> Stephen Hemminger (5):
>   virtio: clean up space checks on xmit
>   virtio: don't use unlikely for normal tx stuff
>   virtio: use indirect ring elements
>   virtio: use any layout on transmit
>   virtio: optimize transmit enqueue

Huawei, do you ack this series?

[dpdk-dev] [PATCH v2 0/5] virtio: Tx performance improvements

[Stephen Hemminger]

This is a tested version of the virtio Tx performance improvements
that I posted earlier on the list, and described at the DPDK Userspace
meeting in Dublin. Together they get a 25% performance improvement for
both small packet and large multi-segment packet case when testing
from DPDK guest application to Linux KVM host.

Stephen Hemminger (5):
  virtio: clean up space checks on xmit
  virtio: don't use unlikely for normal tx stuff
  virtio: use indirect ring elements
  virtio: use any layout on transmit
  virtio: optimize transmit enqueue

 drivers/net/virtio/virtio_ethdev.c |  38 +++---
 drivers/net/virtio/virtio_ethdev.h |   4 +-
 drivers/net/virtio/virtio_rxtx.c   | 150 -
 drivers/net/virtio/virtqueue.h |  19 +
 4 files changed, 130 insertions(+), 81 deletions(-)

-- 
2.1.4