[dpdk-dev] [PATCH v6 0/4] new crypto software based device

[De Lara Guarch, Pablo]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Slawomir Mrozowicz
> Sent: Tuesday, October 04, 2016 8:11 AM
> To: dev at dpdk.org
> Cc: Mrozowicz, SlawomirX
> Subject: [dpdk-dev] [PATCH v6 0/4] new crypto software based device
> 
> This code provides the initial implementation of the libcrypto poll mode
> driver.
> All cryptography operations are using Openssl library crypto API.
> Each algorithm uses EVP_ interface from openssl API - which is recommended
> by
> Openssl maintainers.
> 
> For more information about how to use this driver, go to:
> doc/guides/cryptodevs/libcrypto.rst
> 
> Changes in V6:
> - fix checkpatch warnings
> 
> Changes in V5:
> - reduce source of big data test
> 
> Changes in V4:
> - move aes test rework to another patch
> - move big data test to another patch
> - checking if libcrypto pmd is available
> 
> Changes in V3:
> - add nagative verification tests
> - add big data test
> - fix pmd according to negative verification tests
> - change gmac aad max size
> - update documentation and commits comments
> 
> Changes in V2:
> - add gcm/gmac algorithm correction
> - unit test rework
> 
> Slawomir Mrozowicz (1):
>   libcrypto_pmd: initial implementation of SW crypto device
> 
> Piotr Azarewicz (2)
>   app/test: cryptodev AES tests rework
>   app/test: added tests for libcrypto PMD
> 
> Daniel Mrzyglod (1)
>   examples/l2fwd-crypto: updated example for libcrypto PMD
> 
>  MAINTAINERS|4 +
>  app/test/Makefile  |2 +-
>  app/test/test_cryptodev.c  | 1584 
> ++--
>  app/test/test_cryptodev.h  |1 +
>  app/test/test_cryptodev_aes.c  |  687 -
>  app/test/test_cryptodev_aes.h  | 1124 --
>  app/test/test_cryptodev_aes_test_vectors.h | 1097 ++
>  app/test/test_cryptodev_blockcipher.c  |  538 +++
>  app/test/test_cryptodev_blockcipher.h  |  125 ++
>  app/test/test_cryptodev_des_test_vectors.h |  955 
>  app/test/test_cryptodev_gcm_test_vectors.h |   36 +-
>  app/test/test_cryptodev_hash_test_vectors.h|  491 ++
>  app/test/test_cryptodev_perf.c |  712 -
>  config/common_base |6 +
>  doc/guides/cryptodevs/index.rst|1 +
>  doc/guides/cryptodevs/libcrypto.rst|  116 ++
>  doc/guides/rel_notes/release_16_11.rst |   23 +-
>  drivers/crypto/Makefile|1 +
>  drivers/crypto/libcrypto/Makefile  |   60 +
>  drivers/crypto/libcrypto/rte_libcrypto_pmd.c   | 1062 +
>  drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c   |  708 +
>  .../crypto/libcrypto/rte_libcrypto_pmd_private.h   |  174 +++
>  .../crypto/libcrypto/rte_pmd_libcrypto_version.map |3 +
>  examples/l2fwd-crypto/main.c   |9 +
>  lib/librte_cryptodev/rte_cryptodev.h   |5 +-
>  mk/rte.app.mk  |   23 +-
>  26 files changed, 7621 insertions(+), 1926 deletions(-)
>  delete mode 100644 app/test/test_cryptodev_aes.c
>  delete mode 100644 app/test/test_cryptodev_aes.h
>  create mode 100644 app/test/test_cryptodev_aes_test_vectors.h
>  create mode 100644 app/test/test_cryptodev_blockcipher.c
>  create mode 100644 app/test/test_cryptodev_blockcipher.h
>  create mode 100644 app/test/test_cryptodev_des_test_vectors.h
>  create mode 100644 app/test/test_cryptodev_hash_test_vectors.h
>  create mode 100644 doc/guides/cryptodevs/libcrypto.rst
>  create mode 100644 drivers/crypto/libcrypto/Makefile
>  create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd.c
>  create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c
>  create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_private.h
>  create mode 100644
> drivers/crypto/libcrypto/rte_pmd_libcrypto_version.map
> 
> --
> 2.5.0

Series-acked-by: Pablo de Lara 

Thanks for all the rework!

[dpdk-dev] [PATCH v7] net/virtio: add set_mtu in virtio

[Dey, Souvik]

Hi All,
Is there any further comments or modifications required for this patch, 
or what next steps do you guys suggest here ?

--
Regards,
Souvik

-Original Message-
From: Dey, Souvik 
Sent: Saturday, October 1, 2016 10:09 AM
To: mark.b.kavanagh at intel.com; yuanhan.liu at linux.intel.com; stephen at 
networkplumber.org; dev at dpdk.org
Subject: RE: [PATCH v7] net/virtio: add set_mtu in virtio

Hi Liu/Stephen/Mark,

I have submitted Version 7 of this patch. Do let me know if this looks 
proper.

--
Regards,
Souvik  

-Original Message-
From: Dey, Souvik 
Sent: Thursday, September 29, 2016 4:32 PM
To: mark.b.kavanagh at intel.com; yuanhan.liu at linux.intel.com; stephen at 
networkplumber.org; dev at dpdk.org
Cc: Dey, Souvik 
Subject: [PATCH v7] net/virtio: add set_mtu in virtio


Virtio interfaces do not currently allow the user to specify a particular 
Maximum Transmission Unit (MTU).Consequently, the MTU of Virtio interfaces 
is typically set to the Ethernet default value of 1500.
This is problematic in the case of cloud deployments, in which a specific
(and potentially non-standard) MTU needs to be set by a DHCP server, which 
needs to be honored by all interfaces across the traffic path.To acheive 
this Virtio interfaces should support setting of MTU.
In case when GRE/VXLAN tunneling is used for internal communication, there 
will be an overhead added by the infrastructure in the packet over and 
above the ETHER MTU of 1518. So to take care of this overhead in these 
cases the DHCP server corrects the L3 MTU to 1454. But since virtio 
interfaces was not having the MTU set functionality that MTU sent by the 
DHCP server was ignored and the instance will still send packets with 1500 
MTU which after encapsulation will become more than 1518 and eventually 
gets dropped in the infrastructure. 
By adding an additional 'set_mtu' function to the Virtio driver, we can 
honor the MTU sent by the DHCP server. The dhcp server/controller can 
then leverage this 'set_mtu' functionality to resolve the above 
mentioned issue of packets getting dropped due to incorrect size.


Signed-off-by: Souvik Dey 

---
Changes in v7:
- Replaced the CRC_LEN with the merge rx buf header length.
- Changed the frame_len max validation to VIRTIO_MAX_RX_PKTLEN.
Changes in v6:
- Description of change corrected
- Corrected the identations
- Corrected the subject line too
- The From line was also not correct
- Re-submitting as the below patch was not proper
Changes in v5: 
- Fix log message for out-of-bounds MTU parameter in virtio_mtu_set
- Calculate frame size, based on 'mtu' parameter
- Corrected the upper bound and lower bound checks in virtio_mtu_set
Changes in v4: Incorporated review comments.
Changes in v3: Corrected few style errors as reported by sys-stv.
Changes in v2: Incorporated review comments.

 drivers/net/virtio/virtio_ethdev.c | 16 
 1 file changed, 16 insertions(+)

diff --git a/drivers/net/virtio/virtio_ethdev.c 
b/drivers/net/virtio/virtio_ethdev.c
index 423c597..1dbfea6 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -653,12 +653,20 @@ virtio_dev_allmulticast_disable(struct rte_eth_dev *dev)
PMD_INIT_LOG(ERR, "Failed to disable allmulticast");
 } 

+#define VLAN_TAG_LEN   4/* 802.3ac tag (not DMA'd) */
+static int  virtio_mtu_set(struct rte_eth_dev *dev, uint16_t mtu)
+{
+   struct virtio_hw *hw = dev->data->dev_private;
+   uint32_t ether_hdr_len = ETHER_HDR_LEN + VLAN_TAG_LEN +
+   hw->vtnet_hdr_size;
+   uint32_t frame_size = mtu + ether_hdr_len;
+
+   if (mtu < ETHER_MIN_MTU || frame_size > VIRTIO_MAX_RX_PKTLEN) {
+   PMD_INIT_LOG(ERR, "MTU should be between %d and %d\n",
+   ETHER_MIN_MTU, VIRTIO_MAX_RX_PKTLEN);
+   return -EINVAL;
+   }
+   return 0;
+}

 /*
  * dev_ops for virtio, bare necessities for basic operation
  */
@@ -677,7 +685,6 @@ static const struct eth_dev_ops virtio_eth_dev_ops = {
.allmulticast_enable = virtio_dev_allmulticast_enable,
.allmulticast_disable= virtio_dev_allmulticast_disable,
+   .mtu_set = virtio_mtu_set,
.dev_infos_get   = virtio_dev_info_get,
.stats_get   = virtio_dev_stats_get,
.xstats_get  = virtio_dev_xstats_get,
-- 
2.9.3.windows.1

[dpdk-dev] [PATCH v2]:rte_timer:timer lag issue correction

[Karmarkar Suyash]

Thanks !! So as next steps I will push the patch .

-Original Message-
From: Sanford, Robert [mailto:rsanf...@akamai.com] 
Sent: Tuesday, October 4, 2016 5:40 PM
To: Karmarkar Suyash ; dev at dpdk.org; 
thomas.monjalon at 6wind.com; reshma.pattan at intel.com
Subject: Re: [PATCH v2]:rte_timer:timer lag issue correction

Yes, this change makes sense. I ran timer tests and they passed.

Acked-by: Robert Sanford 

Thanks,
Robert



On 9/29/16, 10:27 AM, "Karmarkar Suyash"  wrote:

Hello,

Can you please review the changes and suggest next steps? Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash
Sent: Wednesday, September 21, 2016 4:54 PM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Cc: Karmarkar Suyash 
Subject: [PATCH v2]:rte_timer:timer lag issue correction

For Periodic timers ,if the lag gets introduced, the current code added 
additional delay when the next peridoc timer was initialized by not taking into 
account the delay added, with this fix the code would start the next occurrence 
of timer keeping in account the lag added.Corrected the behavior.

Fixes: 9b15ba89 ("timer: use a skip list")

Karmarkar Suyash (1):
Signed-off-by: Karmarkar Suyash 

 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


---
 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c index 
43da836..18782fa 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -613,7 +613,7 @@ void rte_timer_manage(void)
status.owner = (int16_t)lcore_id;
rte_wmb();
tim->status.u32 = status.u32;
-   __rte_timer_reset(tim, cur_time + tim->period,
+   __rte_timer_reset(tim, tim->expire + tim->period,
tim->period, lcore_id, tim->f, tim->arg, 1);
rte_spinlock_unlock(&priv_timer[lcore_id].list_lock);
}

--
2.9.3.windows.1

[dpdk-dev] [RFC] libeventdev: event driven programming model framework for DPDK

[Vangati, Narender]

Hi Jerin,



Here are some comments on the libeventdev RFC.

These are collated thoughts after discussions with you & others to understand 
the concepts and rationale for the current proposal.



1. Concept of flow queues. This is better abstracted as flow ids and not as 
flow queues which implies there is a queueing structure per flow. A s/w 
implementation can do atomic load balancing on multiple flow ids more 
efficiently than maintaining each event in a specific flow queue.



2. Scheduling group. A scheduling group is more a steam of events, so an event 
queue might be a better abstraction.



3. An event queue should support the concept of max active atomic flows 
(maximum number of active flows this queue can track at any given time) and max 
active ordered sequences (maximum number of outstanding events waiting to be 
egress reordered by this queue). This allows a scheduler implementation to 
dimension/partition its resources among event queues.



4. An event queue should support concept of a single consumer. In an 
application, a stream of events may need to be brought together to a single 
core for some stages of processing, e.g. for TX at the end of the pipeline to 
avoid NIC reordering of the packets. Having a 'single consumer' event queue for 
that stage allows the intensive scheduling logic to be short circuited and can 
improve throughput for s/w implementations.



5. Instead of tying eventdev access to an lcore, a higher level of abstraction 
called event port is needed which is the application i/f to the eventdev. Event 
ports are connected to event queues and is the object the application uses to 
dequeue and enqueue events. There can be more than one event port per lcore 
allowing multiple lightweight threads to have their own i/f into eventdev, if 
the implementation supports it. An event port abstraction also encapsulates 
dequeue depth and enqueue depth for a scheduler implementations which can 
schedule multiple events at a time and output events that can be buffered.



6. An event should support priority. Per event priority is useful for 
segregating high priority (control messages) traffic from low priority within 
the same flow. This needs to be part of the event definition for 
implementations which support it.



7. Event port to event queue servicing priority. This allows two event ports to 
connect to the same event queue with different priorities. For implementations 
which support it, this allows a worker core to participate in two different 
workflows with different priorities (workflow 1 needing 3.5 cores, workflow 2 
needing 2.5 cores, and so on).



8. Define the workflow as schedule/dequeue/enqueue. An implementation is free 
to define schedule as NOOP. A distributed s/w scheduler can use this to 
schedule events; also a centralized s/w scheduler can make this a NOOP on 
non-scheduler cores.



9. The schedule_from_group API does not fit the workflow.



10. The ctxt_update/ctxt_wait breaks the normal workflow. If the normal 
workflow is a dequeue -> do work based on event type -> enqueue,  a pin_event 
argument to enqueue (where the pinned event is returned through the normal 
dequeue) allows application workflow to remain the same whether or not an 
implementation supports it.



11. Burst dequeue/enqueue needed.



12. Definition of a closed/open system - where open system is memory backed and 
closed system eventdev has limited capacity. In such systems, it is also useful 
to denote per event port how many packets can be active in the system. This can 
serve as a threshold for ethdev like devices so they don't overwhelm core to 
core events.



13. There should be sort of device capabilities definition to address different 
implementations.




vnr
---

[dpdk-dev] [PATCH v2]:rte_timer:timer lag issue correction

[Sanford, Robert]

Yes, this change makes sense. I ran timer tests and they passed.

Acked-by: Robert Sanford 

Thanks,
Robert



On 9/29/16, 10:27 AM, "Karmarkar Suyash"  wrote:

Hello,

Can you please review the changes and suggest next steps? Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash 
Sent: Wednesday, September 21, 2016 4:54 PM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Cc: Karmarkar Suyash 
Subject: [PATCH v2]:rte_timer:timer lag issue correction

For Periodic timers ,if the lag gets introduced, the current code 
added additional delay when the next peridoc timer was initialized 
by not taking into account the delay added, with this fix the code 
would start the next occurrence of timer keeping in account the 
lag added.Corrected the behavior.

Fixes: 9b15ba89 ("timer: use a skip list")

Karmarkar Suyash (1):
Signed-off-by: Karmarkar Suyash 

 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


---
 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c
index 43da836..18782fa 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -613,7 +613,7 @@ void rte_timer_manage(void)
status.owner = (int16_t)lcore_id;
rte_wmb();
tim->status.u32 = status.u32;
-   __rte_timer_reset(tim, cur_time + tim->period,
+   __rte_timer_reset(tim, tim->expire + tim->period,
tim->period, lcore_id, tim->f, tim->arg, 1);
rte_spinlock_unlock(&priv_timer[lcore_id].list_lock);
}

-- 
2.9.3.windows.1

[dpdk-dev] Getting corrupted ESP packet

[Pankaj Joshi]

Hello All,

I am using QAT library for data encryption ( for coletocreek card).
I am sending 98 byte ICMP data to the hardware, at successful time it is
returning 166 byte of data as ESP packet.
But sometimes it is returning through callback function 180 byte of data ,
which is corrupted one .
Can anyone tell, how I can resolve this issue and why it is happening when
I am sending same data to the hardware.

Regards,
Pankaj Joshi

[dpdk-dev] [PATCH v2] examples: fix ip_pipeline to load PMD driver correctly

[Dumitrescu, Cristian]

> -Original Message-
> From: Gowrishankar [mailto:gowrishankar.m at linux.vnet.ibm.com]
> Sent: Tuesday, October 4, 2016 1:43 PM
> To: dev at dpdk.org
> Cc: Chao Zhu ; Thomas Monjalon
> ; Dumitrescu, Cristian
> ; Christian Ehrhardt
> ; Pradeep ;
> Gowrishankar Muthukrishnan 
> Subject: [PATCH v2] examples: fix ip_pipeline to load PMD driver correctly
> 
> From: Gowrishankar Muthukrishnan 
> 
> v2: minor correction in patch to avoid space between -d option and driver
> path
> 


Acked-by: Cristian Dumitrescu 

[dpdk-dev] [PATCH v2]:rte_timer:timer lag issue correction

[Sanford, Robert]

Sorry, just saw this. I will take a look and get back shortly.

--
Regards,
Robert



On 10/4/16, 3:31 PM, "Karmarkar Suyash"  wrote:

Hello Robert/Thomas,

Can you please review the changes in V2 of the Patch and suggest next steps? 
Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash 
Sent: Thursday, September 29, 2016 10:27 AM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Subject: RE: [PATCH v2]:rte_timer:timer lag issue correction

Hello,

Can you please review the changes and suggest next steps? Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash
Sent: Wednesday, September 21, 2016 4:54 PM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Cc: Karmarkar Suyash 
Subject: [PATCH v2]:rte_timer:timer lag issue correction

For Periodic timers ,if the lag gets introduced, the current code added 
additional delay when the next peridoc timer was initialized by not taking into 
account the delay added, with this fix the code would start the next occurrence 
of timer keeping in account the lag added.Corrected the behavior.

Fixes: 9b15ba89 ("timer: use a skip list")

Karmarkar Suyash (1):
Signed-off-by: Karmarkar Suyash 

 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


---
 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c index 
43da836..18782fa 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -613,7 +613,7 @@ void rte_timer_manage(void)
status.owner = (int16_t)lcore_id;
rte_wmb();
tim->status.u32 = status.u32;
-   __rte_timer_reset(tim, cur_time + tim->period,
+   __rte_timer_reset(tim, tim->expire + tim->period,
tim->period, lcore_id, tim->f, tim->arg, 1);
rte_spinlock_unlock(&priv_timer[lcore_id].list_lock);
}

--
2.9.3.windows.1

[dpdk-dev] [PATCH v2 1/3] mem: fix hugepage mapping error messages

[Sergio Gonzalez Monroy]

On 04/10/2016 18:17, Jean Tourrilhes wrote:
> Running secondary is tricky due to the need to map the memory region
> at the right place in VM, which is whatever primary has chosen. If the
> base address for primary happens to by already mapped in the
> secondary, we will hit precisely these error messages (depending if we
> fail on the config region or the hugepages). This is why there is
> already a comment about ASLR.
>
> The issue is that in most cases, remapping does not happen and "errno"
> is not changed and therefore stale. In our case, we got a "permission
> denied", which sent us down the wrong track. It's such a common error
> for secondary that I feel this error message should be unambiguous and
> helpful.
> The call to close was also moved because close() may override errno.
>
> Signed-off-by: Jean Tourrilhes 
> ---
>   lib/librte_eal/linuxapp/eal/eal.c| 14 +++---
>   lib/librte_eal/linuxapp/eal/eal_memory.c | 16 
>   2 files changed, 23 insertions(+), 7 deletions(-)

Acked-by: Sergio Gonzalez Monroy 

[dpdk-dev] [PATCH v2]:rte_timer:timer lag issue correction

[Karmarkar Suyash]

Hello Robert/Thomas,

Can you please review the changes in V2 of the Patch and suggest next steps? 
Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash 
Sent: Thursday, September 29, 2016 10:27 AM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Subject: RE: [PATCH v2]:rte_timer:timer lag issue correction

Hello,

Can you please review the changes and suggest next steps? Thanks

Regards
Suyash Karmarkar

-Original Message-
From: Karmarkar Suyash
Sent: Wednesday, September 21, 2016 4:54 PM
To: dev at dpdk.org; thomas.monjalon at 6wind.com; rsanford at akamai.com; 
reshma.pattan at intel.com
Cc: Karmarkar Suyash 
Subject: [PATCH v2]:rte_timer:timer lag issue correction

For Periodic timers ,if the lag gets introduced, the current code added 
additional delay when the next peridoc timer was initialized by not taking into 
account the delay added, with this fix the code would start the next occurrence 
of timer keeping in account the lag added.Corrected the behavior.

Fixes: 9b15ba89 ("timer: use a skip list")

Karmarkar Suyash (1):
Signed-off-by: Karmarkar Suyash 

 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)


---
 lib/librte_timer/rte_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_timer/rte_timer.c b/lib/librte_timer/rte_timer.c index 
43da836..18782fa 100644
--- a/lib/librte_timer/rte_timer.c
+++ b/lib/librte_timer/rte_timer.c
@@ -613,7 +613,7 @@ void rte_timer_manage(void)
status.owner = (int16_t)lcore_id;
rte_wmb();
tim->status.u32 = status.u32;
-   __rte_timer_reset(tim, cur_time + tim->period,
+   __rte_timer_reset(tim, tim->expire + tim->period,
tim->period, lcore_id, tim->f, tim->arg, 1);
rte_spinlock_unlock(&priv_timer[lcore_id].list_lock);
}

--
2.9.3.windows.1

[dpdk-dev] Getting corrupted ESP packet

[Trahe, Fiona]

Hi Pankaj,

I can't think of any way the QAT PMD could return a larger packet than it's 
been sent, can you provide some more details of your use-case please, e.g. 
which cipher algorithm, which auth algorithm are you using?
Are you using out-of-place or in-place? i.e. are the m_src and m_dst mbuf 
pointers in the rte_crypto_sym_op the same or different?
Can you try doing the same operation using the AESNI_MB PMD?

Regards,
Fiona


> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Pankaj Joshi
> Sent: Tuesday, October 4, 2016 5:04 PM
> To: qat-linux ; dev at dpdk.org
> Subject: [dpdk-dev] Getting corrupted ESP packet
> 
> Hello All,
> 
> I am using QAT library for data encryption ( for coletocreek card).
> I am sending 98 byte ICMP data to the hardware, at successful time it is
> returning 166 byte of data as ESP packet.
> But sometimes it is returning through callback function 180 byte of data ,
> which is corrupted one .
> Can anyone tell, how I can resolve this issue and why it is happening when
> I am sending same data to the hardware.

> 
> Regards,
> Pankaj Joshi

[dpdk-dev] [PATCH] dpdk-procinfo: free allocated xstats memory upon failure

[Reshma Pattan]

Some of the failures cases inside the nic_xstats_display()
function doesn't free the allocated memory for the xstats and
their names, memory is freed now.

Fixes: e2aae1c1 ("ethdev: remove name from extended statistic fetch")
Fixes: 22561383 ("app: replace dump_cfg by proc_info")

Signed-off-by: Reshma Pattan 
---
 app/proc_info/main.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/app/proc_info/main.c b/app/proc_info/main.c
index 8246fb2..2c56d10 100644
--- a/app/proc_info/main.c
+++ b/app/proc_info/main.c
@@ -268,7 +268,7 @@ nic_xstats_display(uint8_t port_id)
if (len != rte_eth_xstats_get_names(
port_id, xstats_names, len)) {
printf("Cannot get xstat names\n");
-   return;
+   goto err;
}

printf("## NIC extended statistics for port %-2d #\n",
@@ -278,8 +278,7 @@ nic_xstats_display(uint8_t port_id)
ret = rte_eth_xstats_get(port_id, xstats, len);
if (ret < 0 || ret > len) {
printf("Cannot get xstats\n");
-   free(xstats);
-   return;
+   goto err;
}

for (i = 0; i < len; i++)
@@ -289,6 +288,7 @@ nic_xstats_display(uint8_t port_id)

printf("%s\n",
   nic_stats_border);
+err:
free(xstats);
free(xstats_names);
 }
-- 
2.7.4

[dpdk-dev] [PATCH v6 4/4] examples/l2fwd-crypto: updated example for libcrypto PMD

[Slawomir Mrozowicz]

Libcrypto PMD has support for:

Supported cipher algorithms:
RTE_CRYPTO_CIPHER_3DES_CBC
RTE_CRYPTO_CIPHER_AES_CBC
RTE_CRYPTO_CIPHER_AES_CTR
RTE_CRYPTO_CIPHER_3DES_CTR
RTE_CRYPTO_CIPHER_AES_GCM

Supported authentication algorithms:
RTE_CRYPTO_AUTH_AES_GMAC
RTE_CRYPTO_AUTH_MD5
RTE_CRYPTO_AUTH_SHA1
RTE_CRYPTO_AUTH_SHA224
RTE_CRYPTO_AUTH_SHA256
RTE_CRYPTO_AUTH_SHA384
RTE_CRYPTO_AUTH_SHA512
RTE_CRYPTO_AUTH_MD5_HMAC
RTE_CRYPTO_AUTH_SHA1_HMAC
RTE_CRYPTO_AUTH_SHA224_HMAC
RTE_CRYPTO_AUTH_SHA256_HMAC
RTE_CRYPTO_AUTH_SHA384_HMAC
RTE_CRYPTO_AUTH_SHA512_HMAC

Signed-off-by: Daniel Mrzyglod 
---
v3:
- change description
---
 examples/l2fwd-crypto/main.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/examples/l2fwd-crypto/main.c b/examples/l2fwd-crypto/main.c
index 0593734..dae45f5 100644
--- a/examples/l2fwd-crypto/main.c
+++ b/examples/l2fwd-crypto/main.c
@@ -340,15 +340,22 @@ fill_supported_algorithm_tables(void)
strcpy(supported_auth_algo[i], "NOT_SUPPORTED");

strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_AES_GCM], "AES_GCM");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_AES_GMAC], "AES_GMAC");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_MD5_HMAC], "MD5_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_MD5], "MD5");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_NULL], "NULL");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_AES_XCBC_MAC],
"AES_XCBC_MAC");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA1_HMAC], "SHA1_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA1], "SHA1");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA224_HMAC], "SHA224_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA224], "SHA224");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA256_HMAC], "SHA256_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA256], "SHA256");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA384_HMAC], "SHA384_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA384], "SHA384");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA512_HMAC], "SHA512_HMAC");
+   strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SHA512], "SHA512");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_SNOW3G_UIA2], "SNOW3G_UIA2");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_ZUC_EIA3], "ZUC_EIA3");
strcpy(supported_auth_algo[RTE_CRYPTO_AUTH_KASUMI_F9], "KASUMI_F9");
@@ -363,6 +370,8 @@ fill_supported_algorithm_tables(void)
strcpy(supported_cipher_algo[RTE_CRYPTO_CIPHER_SNOW3G_UEA2], 
"SNOW3G_UEA2");
strcpy(supported_cipher_algo[RTE_CRYPTO_CIPHER_ZUC_EEA3], "ZUC_EEA3");
strcpy(supported_cipher_algo[RTE_CRYPTO_CIPHER_KASUMI_F8], "KASUMI_F8");
+   strcpy(supported_cipher_algo[RTE_CRYPTO_CIPHER_3DES_CTR], "3DES_CTR");
+   strcpy(supported_cipher_algo[RTE_CRYPTO_CIPHER_3DES_CBC], "3DES_CBC");
 }


-- 
2.5.0

[dpdk-dev] [PATCH v6 3/4] app/test: added tests for libcrypto PMD

[Slawomir Mrozowicz]

This patch contains unit tests for libcrypto PMD. User can
use app/test application to check how to use this pmd and to
verify crypto processing.

Test name is cryptodev_libcrypto_autotest.
For performance test cryptodev_libcrypto_perftest can be used.

Signed-off-by: Piotr Azarewicz 
Signed-off-by: Marcin Kerlin 
Signed-off-by: Daniel Mrzyglod 
---
v2:
- rename AES-named functions to blockcipher
- replace different test cases with blockcipher functions pattern
- add 3DES tests into QuickAssist PMD testsuite

v3:
- add nagative verification tests
- add big data test

v4:
- move aes test rework to another patch
- move big data test to another patch
- checking if libcrypto pmd is available

v5:
- add reduced big data test

v6:
- fix checkpatch warnings
---
 app/test/test_cryptodev.c   | 1505 ++-
 app/test/test_cryptodev.h   |1 +
 app/test/test_cryptodev_aes_test_vectors.h  |  306 +-
 app/test/test_cryptodev_blockcipher.c   |   22 +
 app/test/test_cryptodev_blockcipher.h   |1 +
 app/test/test_cryptodev_des_test_vectors.h  |  955 +
 app/test/test_cryptodev_gcm_test_vectors.h  |   36 +-
 app/test/test_cryptodev_hash_test_vectors.h |  491 +
 app/test/test_cryptodev_perf.c  |  712 -
 9 files changed, 3954 insertions(+), 75 deletions(-)
 create mode 100644 app/test/test_cryptodev_des_test_vectors.h
 create mode 100644 app/test/test_cryptodev_hash_test_vectors.h

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 8f03157..9767704 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -45,6 +45,8 @@

 #include "test_cryptodev_blockcipher.h"
 #include "test_cryptodev_aes_test_vectors.h"
+#include "test_cryptodev_des_test_vectors.h"
+#include "test_cryptodev_hash_test_vectors.h"
 #include "test_cryptodev_kasumi_test_vectors.h"
 #include "test_cryptodev_kasumi_hash_test_vectors.h"
 #include "test_cryptodev_snow3g_test_vectors.h"
@@ -167,7 +169,7 @@ testsuite_setup(void)
/* Not already created so create */
ts_params->mbuf_pool = rte_pktmbuf_pool_create(
"CRYPTO_MBUFPOOL",
-   NUM_MBUFS, MBUF_CACHE_SIZE, 0, MBUF_SIZE,
+   NUM_MBUFS, MBUF_CACHE_SIZE, 0, UINT16_MAX,
rte_socket_id());
if (ts_params->mbuf_pool == NULL) {
RTE_LOG(ERR, USER1, "Can't create CRYPTO_MBUFPOOL\n");
@@ -308,6 +310,28 @@ testsuite_setup(void)
}
}

+   /* Create 2 LIBCRYPTO devices if required */
+   if (gbl_cryptodev_type == RTE_CRYPTODEV_LIBCRYPTO_PMD) {
+#ifndef RTE_LIBRTE_PMD_LIBCRYPTO
+   RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_LIBCRYPTO must be"
+   " enabled in config file to run this testsuite.\n");
+   return TEST_FAILED;
+#endif
+   nb_devs = rte_cryptodev_count_devtype(
+   RTE_CRYPTODEV_LIBCRYPTO_PMD);
+   if (nb_devs < 2) {
+   for (i = nb_devs; i < 2; i++) {
+   ret = rte_eal_vdev_init(
+   RTE_STR(CRYPTODEV_NAME_LIBCRYPTO_PMD),
+   NULL);
+
+   TEST_ASSERT(ret == 0, "Failed to create "
+   "instance %u of pmd : %s", i,
+   RTE_STR(CRYPTODEV_NAME_LIBCRYPTO_PMD));
+   }
+   }
+   }
+
 #ifndef RTE_LIBRTE_PMD_QAT
if (gbl_cryptodev_type == RTE_CRYPTODEV_QAT_SYM_PMD) {
RTE_LOG(ERR, USER1, "CONFIG_RTE_LIBRTE_PMD_QAT must be enabled "
@@ -877,6 +901,315 @@ static const uint8_t 
catch_22_quote_2_512_bytes_AES_CBC_HMAC_SHA1_digest[] = {
0x18, 0x8c, 0x1d, 0x32
 };

+
+/* Multisession Vector context Test */
+/*Begin Session 0 */
+static uint8_t ms_aes_cbc_key0[] = {
+   0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
+   0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff
+};
+
+static uint8_t ms_aes_cbc_iv0[] = {
+   0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
+   0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff
+};
+
+static const uint8_t ms_aes_cbc_cipher0[] = {
+   0x3C, 0xE4, 0xEE, 0x42, 0xB6, 0x9B, 0xC3, 0x38,
+   0x5F, 0xAD, 0x54, 0xDC, 0xA8, 0x32, 0x81, 0xDC,
+   0x7A, 0x6F, 0x85, 0x58, 0x07, 0x35, 0xED, 0xEB,
+   0xAD, 0x79, 0x79, 0x96, 0xD3, 0x0E, 0xA6, 0xD9,
+   0xAA, 0x86, 0xA4, 0x8F, 0xB5, 0xD6, 0x6E, 0x6D,
+   0x0C, 0x91, 0x2F, 0xC4, 0x67, 0x98, 0x0E, 0xC4,
+   0x8D, 0x83, 0x68, 0x69, 0xC4, 0xD3, 0x94, 0x34,
+   0xC4, 0x5D, 0x60, 0x55, 0x22, 0x87, 0x8F, 0x6F,
+   0x17, 0x8E, 0x75, 0xE4, 0x02, 0xF5, 0x1B, 0x99,
+   0xC8, 0x39, 0xA9, 0xAB, 0x23,

[dpdk-dev] [PATCH v6 2/4] app/test: cryptodev AES tests rework

[Slawomir Mrozowicz]

This patch rework AES tests .
In general - rename AES-named functions to blockcipher functions pattern.

Signed-off-by: Piotr Azarewicz 
Signed-off-by: Fiona Trahe 
---
v6:
- fix checkpatch warnings
---
 app/test/Makefile  |2 +-
 app/test/test_cryptodev.c  |   75 +-
 app/test/test_cryptodev_aes.c  |  687 -
 app/test/test_cryptodev_aes.h  | 1124 
 app/test/test_cryptodev_aes_test_vectors.h |  797 
 app/test/test_cryptodev_blockcipher.c  |  516 +
 app/test/test_cryptodev_blockcipher.h  |  124 +++
 7 files changed, 1486 insertions(+), 1839 deletions(-)
 delete mode 100644 app/test/test_cryptodev_aes.c
 delete mode 100644 app/test/test_cryptodev_aes.h
 create mode 100644 app/test/test_cryptodev_aes_test_vectors.h
 create mode 100644 app/test/test_cryptodev_blockcipher.c
 create mode 100644 app/test/test_cryptodev_blockcipher.h

diff --git a/app/test/Makefile b/app/test/Makefile
index 611d77a..5be023a 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -193,7 +193,7 @@ endif
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
 SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring_perf.c

-SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_aes.c
+SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_blockcipher.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_perf.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev.c

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 9d7caba..8f03157 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -43,7 +43,8 @@
 #include "test.h"
 #include "test_cryptodev.h"

-#include "test_cryptodev_aes.h"
+#include "test_cryptodev_blockcipher.h"
+#include "test_cryptodev_aes_test_vectors.h"
 #include "test_cryptodev_kasumi_test_vectors.h"
 #include "test_cryptodev_kasumi_hash_test_vectors.h"
 #include "test_cryptodev_snow3g_test_vectors.h"
@@ -86,12 +87,16 @@ struct crypto_unittest_params {
  */
 static int
 test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(
-   struct crypto_unittest_params *ut_params);
+   struct crypto_unittest_params *ut_params, uint8_t *cipher_key,
+   uint8_t *hmac_key);

 static int
 test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session 
*sess,
struct crypto_unittest_params *ut_params,
-   struct crypto_testsuite_params *ts_param);
+   struct crypto_testsuite_params *ts_param,
+   const uint8_t *cipher,
+   const uint8_t *digest,
+   const uint8_t *iv);

 static struct rte_mbuf *
 setup_test_string(struct rte_mempool *mpool,
@@ -313,7 +318,7 @@ testsuite_setup(void)

nb_devs = rte_cryptodev_count();
if (nb_devs < 1) {
-   RTE_LOG(ERR, USER1, "No crypto devices found?");
+   RTE_LOG(ERR, USER1, "No crypto devices found?\n");
return TEST_FAILED;
}

@@ -872,7 +877,6 @@ static const uint8_t 
catch_22_quote_2_512_bytes_AES_CBC_HMAC_SHA1_digest[] = {
0x18, 0x8c, 0x1d, 0x32
 };

-
 static int
 test_AES_CBC_HMAC_SHA1_encrypt_digest(void)
 {
@@ -1003,17 +1007,24 @@ static const uint8_t 
catch_22_quote_2_512_bytes_AES_CBC_HMAC_SHA512_digest[] = {

 static int
 test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(
-   struct crypto_unittest_params *ut_params);
+   struct crypto_unittest_params *ut_params,
+   uint8_t *cipher_key,
+   uint8_t *hmac_key);

 static int
 test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session 
*sess,
struct crypto_unittest_params *ut_params,
-   struct crypto_testsuite_params *ts_params);
+   struct crypto_testsuite_params *ts_params,
+   const uint8_t *cipher,
+   const uint8_t *digest,
+   const uint8_t *iv);


 static int
 test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(
-   struct crypto_unittest_params *ut_params)
+   struct crypto_unittest_params *ut_params,
+   uint8_t *cipher_key,
+   uint8_t *hmac_key)
 {

/* Setup Cipher Parameters */
@@ -1022,7 +1033,7 @@ test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(

ut_params->cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC;
ut_params->cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT;
-   ut_params->cipher_xform.cipher.key.data = aes_cbc_key;
+   ut_params->cipher_xform.cipher.key.data = cipher_key;
ut_params->cipher_xform.cipher.key.length = CIPHER_KEY_LENGTH_AES_CBC;

/* Setup HMAC Parameters */
@@ -1031,7 +1042,7 @@ test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(

ut_params->auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
ut_params->auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA512_HMAC;
- 

[dpdk-dev] [PATCH v6 1/4] libcrypto_pmd: initial implementation of SW crypto device

[Slawomir Mrozowicz]

This code provides the initial implementation of the libcrypto
poll mode driver. All cryptography operations are using Openssl
library crypto API. Each algorithm uses EVP_ interface from
openssl API - which is recommended by Openssl maintainers.

This patch adds libcrypto poll mode driver support to librte_cryptodev
library.

Signed-off-by: Slawomir Mrozowicz 
Signed-off-by: Michal Kobylinski 
Signed-off-by: Tomasz Kulasek 
Signed-off-by: Daniel Mrzyglod 
---
v2:
- add gcm crypto cipher and authentication algorithm
- rework gmac crypto authentication algorithm

v3:
- fix pmd according to negative verification tests
- change gmac aad max size
- update documentation

v6:
- fix checkpatch warnings
---
 MAINTAINERS|4 +
 config/common_base |6 +
 doc/guides/cryptodevs/index.rst|1 +
 doc/guides/cryptodevs/libcrypto.rst|  116 +++
 doc/guides/rel_notes/release_16_11.rst |   23 +-
 drivers/crypto/Makefile|1 +
 drivers/crypto/libcrypto/Makefile  |   60 ++
 drivers/crypto/libcrypto/rte_libcrypto_pmd.c   | 1062 
 drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c   |  708 +
 .../crypto/libcrypto/rte_libcrypto_pmd_private.h   |  174 
 .../crypto/libcrypto/rte_pmd_libcrypto_version.map |3 +
 lib/librte_cryptodev/rte_cryptodev.h   |5 +-
 mk/rte.app.mk  |   23 +-
 13 files changed, 2173 insertions(+), 13 deletions(-)
 create mode 100644 doc/guides/cryptodevs/libcrypto.rst
 create mode 100644 drivers/crypto/libcrypto/Makefile
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd.c
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_private.h
 create mode 100644 drivers/crypto/libcrypto/rte_pmd_libcrypto_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index 58a10b8..1e9d1f8 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -439,6 +439,10 @@ M: Declan Doherty 
 F: drivers/crypto/null/
 F: doc/guides/cryptodevs/null.rst

+LibCrypto Crypto PMD
+M: Declan Doherty 
+F: drivers/crypto/libcrypto/
+F: doc/guides/cryptodevs/libcrypto.rst

 Packet processing
 -
diff --git a/config/common_base b/config/common_base
index 3a412ee..87b8646 100644
--- a/config/common_base
+++ b/config/common_base
@@ -376,6 +376,12 @@ CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
 CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n

 #
+# Compile PMD for Software backed device
+#
+CONFIG_RTE_LIBRTE_PMD_LIBCRYPTO=n
+CONFIG_RTE_LIBRTE_PMD_LIBCRYPTO_DEBUG=n
+
+#
 # Compile PMD for AESNI GCM device
 #
 CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n
diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst
index 906f1b4..bae8e53 100644
--- a/doc/guides/cryptodevs/index.rst
+++ b/doc/guides/cryptodevs/index.rst
@@ -39,6 +39,7 @@ Crypto Device Drivers
 aesni_mb
 aesni_gcm
 kasumi
+libcrypto
 null
 snow3g
 qat
diff --git a/doc/guides/cryptodevs/libcrypto.rst 
b/doc/guides/cryptodevs/libcrypto.rst
new file mode 100644
index 000..77eff95
--- /dev/null
+++ b/doc/guides/cryptodevs/libcrypto.rst
@@ -0,0 +1,116 @@
+..  BSD LICENSE
+Copyright(c) 2016 Intel Corporation. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+notice, this list of conditions and the following disclaimer in
+the documentation and/or other materials provided with the
+distribution.
+* Neither the name of Intel Corporation nor the names of its
+contributors may be used to endorse or promote products derived
+from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+LibCrypto Crypto Poll Mode Driver
+
+This code provides the initial implementation 

[dpdk-dev] [PATCH v6 0/4] new crypto software based device

[Slawomir Mrozowicz]

This code provides the initial implementation of the libcrypto poll mode driver.
All cryptography operations are using Openssl library crypto API.
Each algorithm uses EVP_ interface from openssl API - which is recommended by
Openssl maintainers.

For more information about how to use this driver, go to:
doc/guides/cryptodevs/libcrypto.rst

Changes in V6:
- fix checkpatch warnings

Changes in V5:
- reduce source of big data test

Changes in V4:
- move aes test rework to another patch
- move big data test to another patch
- checking if libcrypto pmd is available

Changes in V3:
- add nagative verification tests
- add big data test
- fix pmd according to negative verification tests
- change gmac aad max size
- update documentation and commits comments

Changes in V2:
- add gcm/gmac algorithm correction
- unit test rework

Slawomir Mrozowicz (1):
  libcrypto_pmd: initial implementation of SW crypto device

Piotr Azarewicz (2)
  app/test: cryptodev AES tests rework
  app/test: added tests for libcrypto PMD

Daniel Mrzyglod (1)
  examples/l2fwd-crypto: updated example for libcrypto PMD

 MAINTAINERS|4 +
 app/test/Makefile  |2 +-
 app/test/test_cryptodev.c  | 1584 ++--
 app/test/test_cryptodev.h  |1 +
 app/test/test_cryptodev_aes.c  |  687 -
 app/test/test_cryptodev_aes.h  | 1124 --
 app/test/test_cryptodev_aes_test_vectors.h | 1097 ++
 app/test/test_cryptodev_blockcipher.c  |  538 +++
 app/test/test_cryptodev_blockcipher.h  |  125 ++
 app/test/test_cryptodev_des_test_vectors.h |  955 
 app/test/test_cryptodev_gcm_test_vectors.h |   36 +-
 app/test/test_cryptodev_hash_test_vectors.h|  491 ++
 app/test/test_cryptodev_perf.c |  712 -
 config/common_base |6 +
 doc/guides/cryptodevs/index.rst|1 +
 doc/guides/cryptodevs/libcrypto.rst|  116 ++
 doc/guides/rel_notes/release_16_11.rst |   23 +-
 drivers/crypto/Makefile|1 +
 drivers/crypto/libcrypto/Makefile  |   60 +
 drivers/crypto/libcrypto/rte_libcrypto_pmd.c   | 1062 +
 drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c   |  708 +
 .../crypto/libcrypto/rte_libcrypto_pmd_private.h   |  174 +++
 .../crypto/libcrypto/rte_pmd_libcrypto_version.map |3 +
 examples/l2fwd-crypto/main.c   |9 +
 lib/librte_cryptodev/rte_cryptodev.h   |5 +-
 mk/rte.app.mk  |   23 +-
 26 files changed, 7621 insertions(+), 1926 deletions(-)
 delete mode 100644 app/test/test_cryptodev_aes.c
 delete mode 100644 app/test/test_cryptodev_aes.h
 create mode 100644 app/test/test_cryptodev_aes_test_vectors.h
 create mode 100644 app/test/test_cryptodev_blockcipher.c
 create mode 100644 app/test/test_cryptodev_blockcipher.h
 create mode 100644 app/test/test_cryptodev_des_test_vectors.h
 create mode 100644 app/test/test_cryptodev_hash_test_vectors.h
 create mode 100644 doc/guides/cryptodevs/libcrypto.rst
 create mode 100644 drivers/crypto/libcrypto/Makefile
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd.c
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_ops.c
 create mode 100644 drivers/crypto/libcrypto/rte_libcrypto_pmd_private.h
 create mode 100644 drivers/crypto/libcrypto/rte_pmd_libcrypto_version.map

-- 
2.5.0

[dpdk-dev] [PATCH v2] examples: fix ip_pipeline to load PMD driver correctly

[Gowrishankar]

From: Gowrishankar Muthukrishnan 

There is typo in init.c of ip_pipeline example due to which,
invalid file path is added to -d option of EAL i.e path starting
with =.

Signed-off-by: Gowrishankar Muthukrishnan 
---
 examples/ip_pipeline/init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/ip_pipeline/init.c b/examples/ip_pipeline/init.c
index 0dbc332..d580ddf 100644
--- a/examples/ip_pipeline/init.c
+++ b/examples/ip_pipeline/init.c
@@ -236,7 +236,7 @@ app_init_eal(struct app_params *app)
}

if (p->add_driver) {
-   snprintf(buffer, sizeof(buffer), "-d=%s", p->add_driver);
+   snprintf(buffer, sizeof(buffer), "-d%s", p->add_driver);
app->eal_argv[n_args++] = strdup(buffer);
}

-- 
1.9.1

[dpdk-dev] [PATCH v2] examples: fix ip_pipeline to load PMD driver correctly

[Gowrishankar]

From: Gowrishankar Muthukrishnan 

v2: minor correction in patch to avoid space between -d option and driver path

Gowrishankar Muthukrishnan (1):
  examples: fix ip_pipeline to load PMD driver correctly

 examples/ip_pipeline/init.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
1.9.1

[dpdk-dev] [PATCH] doc: arm64: document DPDK application profiling methods

[Jerin Jacob]

Signed-off-by: Jerin Jacob 
---
 doc/guides/prog_guide/profile_app.rst | 58 +++
 1 file changed, 58 insertions(+)

diff --git a/doc/guides/prog_guide/profile_app.rst 
b/doc/guides/prog_guide/profile_app.rst
index 3226187..bb78623 100644
--- a/doc/guides/prog_guide/profile_app.rst
+++ b/doc/guides/prog_guide/profile_app.rst
@@ -31,6 +31,14 @@
 Profile Your Application
 

+Introduction
+
+
+The following sections describe the methods to profile DPDK applications on
+different architectures.
+
+x86
+~~~
 Intel processors provide performance counters to monitor events.
 Some tools provided by Intel can be used to profile and benchmark an 
application.
 See the *VTune Performance Analyzer Essentials* publication from Intel Press 
for more information.
@@ -50,3 +58,53 @@ The main situations that should be monitored through event 
counters are:
 Refer to the
 `Intel Performance Analysis Guide 
`_
 for details about application profiling.
+
+ARM64
+~
+
+Perf
+
+ARM64 architecture provide performance counters to monitor events.
+The Linux perf tool can be used to profile and benchmark an application.
+In addition to the standard events, perf can be used to profile arm64 specific
+PMU events through raw events(-e -rXX)
+
+Refer to the
+`ARM64 specific PMU events enumeration 
`_
+
+High-resolution cycle counter
+^
+The default cntvct_el0 based rte_rdtsc() provides portable means to get wall
+clock counter at user space. Typically it runs at <= 100MHz.
+
+The alternative method to enable rte_rdtsc() for high resolution
+wall clock counter is through armv8 PMU subsystem.
+The PMU cycle counter runs at CPU frequency, However, access to PMU cycle
+counter from user space is not enabled by default in the arm64 linux kernel.
+It is possible to enable cycle counter at user space access
+by configuring the PMU from the privileged mode (kernel space).
+
+by default rte_rdtsc() implementation uses portable cntvct_el0 scheme.
+Application can choose the PMU based implementation with
+CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU
+
+The PMU based scheme useful for high accuracy performance profiling.
+Find below the example steps to configure the PMU based cycle counter on an
+armv8 machine.
+
+.. code-block:: console
+
+git clone https://github.com/jerinjacobk/armv8_pmu_cycle_counter_el0
+cd armv8_pmu_cycle_counter_el0
+make
+sudo insmod pmu_el0_cycle_counter.ko
+cd $DPDK_DIR
+make config T=arm64-armv8a-linuxapp-gcc
+echo "CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU=y" >> build/.config
+make
+
+.. warning::
+
+This method can not be used in production systems as this may alter PMU
+state used by standard Linux user space tool like perf.
+
-- 
2.5.5

[dpdk-dev] [PATCH v4 2/2] net/ixgbe: add callback to user app on VF to PF mbox msg

[Bernard Iremonger]

call _rte_eth_dev_callback_process_vf from ixgbe_rcv_msg_from_vf function.

The callback asks the user application if it is allowed to perform
the function.
If the cb_param.retval is RTE_PMD_IXGBE_MB_EVENT_PROCEED then continue,
if 0, do nothing and send ACK to VF
if > 1, do nothing and send NAK to VF.

Signed-off-by: Alex Zelezniak 
Signed-off-by: Bernard Iremonger 
---
 drivers/net/ixgbe/ixgbe_pf.c  | 42 +--
 drivers/net/ixgbe/rte_pmd_ixgbe.h | 17 
 2 files changed, 53 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_pf.c b/drivers/net/ixgbe/ixgbe_pf.c
index 56393ff..5277517 100644
--- a/drivers/net/ixgbe/ixgbe_pf.c
+++ b/drivers/net/ixgbe/ixgbe_pf.c
@@ -1,7 +1,7 @@
 /*-
  *   BSD LICENSE
  *
- *   Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ *   Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
  *   All rights reserved.
  *
  *   Redistribution and use in source and binary forms, with or without
@@ -51,6 +51,7 @@

 #include "base/ixgbe_common.h"
 #include "ixgbe_ethdev.h"
+#include "rte_pmd_ixgbe.h"

 #define IXGBE_MAX_VFTA (128)
 #define IXGBE_VF_MSG_SIZE_DEFAULT 1
@@ -660,6 +661,7 @@ ixgbe_rcv_msg_from_vf(struct rte_eth_dev *dev, uint16_t vf)
struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
struct ixgbe_vf_info *vfinfo =
*IXGBE_DEV_PRIVATE_TO_P_VFDATA(dev->data->dev_private);
+   struct rte_pmd_ixgbe_mb_event_param cb_param;

retval = ixgbe_read_mbx(hw, msgbuf, mbx_size, vf);
if (retval) {
@@ -674,27 +676,54 @@ ixgbe_rcv_msg_from_vf(struct rte_eth_dev *dev, uint16_t 
vf)
/* flush the ack before we write any messages back */
IXGBE_WRITE_FLUSH(hw);

+   /**
+* initialise structure to send to user application
+* will return response from user in retval field
+*/
+   cb_param.retval = RTE_PMD_IXGBE_MB_EVENT_PROCEED;
+   cb_param.vfid = vf;
+   cb_param.msg_type = msgbuf[0] & 0x;
+   cb_param.userdata = (void *)msgbuf;
+
/* perform VF reset */
if (msgbuf[0] == IXGBE_VF_RESET) {
int ret = ixgbe_vf_reset(dev, vf, msgbuf);

vfinfo[vf].clear_to_send = true;
+
+   /* notify application about VF reset */
+   _rte_eth_dev_callback_process_vf(dev, RTE_ETH_EVENT_VF_MBOX, 
&cb_param);
return ret;
}

+   /**
+* ask user application if we allowed to perform those functions
+* if we get cb_param.retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED
+* then business as usual,
+* if 0, do nothing and send ACK to VF
+* if cb_param.retval > 1, do nothing and send NAK to VF
+*/
+   _rte_eth_dev_callback_process_vf(dev, RTE_ETH_EVENT_VF_MBOX, &cb_param);
+
+   retval = cb_param.retval;
+
/* check & process VF to PF mailbox message */
switch ((msgbuf[0] & 0x)) {
case IXGBE_VF_SET_MAC_ADDR:
-   retval = ixgbe_vf_set_mac_addr(dev, vf, msgbuf);
+   if (retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED)
+   retval = ixgbe_vf_set_mac_addr(dev, vf, msgbuf);
break;
case IXGBE_VF_SET_MULTICAST:
-   retval = ixgbe_vf_set_multicast(dev, vf, msgbuf);
+   if (retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED)
+   retval = ixgbe_vf_set_multicast(dev, vf, msgbuf);
break;
case IXGBE_VF_SET_LPE:
-   retval = ixgbe_set_vf_lpe(dev, vf, msgbuf);
+   if (retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED)
+   retval = ixgbe_set_vf_lpe(dev, vf, msgbuf);
break;
case IXGBE_VF_SET_VLAN:
-   retval = ixgbe_vf_set_vlan(dev, vf, msgbuf);
+   if (retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED)
+   retval = ixgbe_vf_set_vlan(dev, vf, msgbuf);
break;
case IXGBE_VF_API_NEGOTIATE:
retval = ixgbe_negotiate_vf_api(dev, vf, msgbuf);
@@ -704,7 +733,8 @@ ixgbe_rcv_msg_from_vf(struct rte_eth_dev *dev, uint16_t vf)
msg_size = IXGBE_VF_GET_QUEUE_MSG_SIZE;
break;
case IXGBE_VF_UPDATE_XCAST_MODE:
-   retval = ixgbe_set_vf_mc_promisc(dev, vf, msgbuf);
+   if (retval == RTE_PMD_IXGBE_MB_EVENT_PROCEED)
+   retval = ixgbe_set_vf_mc_promisc(dev, vf, msgbuf);
break;
default:
PMD_DRV_LOG(DEBUG, "Unhandled Msg %8.8x", (unsigned)msgbuf[0]);
diff --git a/drivers/net/ixgbe/rte_pmd_ixgbe.h 
b/drivers/net/ixgbe/rte_pmd_ixgbe.h
index 33b5b2d..2f6cf46 100644
--- a/drivers/net/ixgbe/rte_pmd_ixgbe.h
+++ b/drivers/net/ixgbe/rte_pmd_ixgbe.h
@@ -181,4 +181,21 @@ int rte_pmd_ixgbe_set_vf_split_drop_en(uint8_t port, 
uint16_t vf, uint8_t on);
  */
 int
 rte_pmd_ixgbe_set_vf_vlan_stripq(uint8_t port, uint

[dpdk-dev] [PATCH v4 1/2] librte_ether: add internal callback functions

[Bernard Iremonger]

add _rte_eth_dev_callback_process_vf function.
add _rte_eth_dev_callback_process_generic function

Adding a callback to the user application on VF to PF mailbox message,
allows passing information to the application controlling the PF
when a VF mailbox event message is received, such as VF reset.

Signed-off-by: Alex Zelezniak 
Signed-off-by: Bernard Iremonger 
---
 lib/librte_ether/rte_ethdev.c  | 17 +
 lib/librte_ether/rte_ethdev.h  | 44 ++
 lib/librte_ether/rte_ether_version.map |  2 ++
 3 files changed, 63 insertions(+)

diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
index c517e88..e850d88 100644
--- a/lib/librte_ether/rte_ethdev.c
+++ b/lib/librte_ether/rte_ethdev.c
@@ -2510,6 +2510,20 @@ void
 _rte_eth_dev_callback_process(struct rte_eth_dev *dev,
enum rte_eth_event_type event)
 {
+   return _rte_eth_dev_callback_process_generic(dev, event, NULL);
+}
+
+void
+_rte_eth_dev_callback_process_vf(struct rte_eth_dev *dev,
+   enum rte_eth_event_type event, void *param)
+{
+   return _rte_eth_dev_callback_process_generic(dev, event, param);
+}
+
+void
+_rte_eth_dev_callback_process_generic(struct rte_eth_dev *dev,
+   enum rte_eth_event_type event, void *param)
+{
struct rte_eth_dev_callback *cb_lst;
struct rte_eth_dev_callback dev_cb;

@@ -2519,6 +2533,9 @@ _rte_eth_dev_callback_process(struct rte_eth_dev *dev,
continue;
dev_cb = *cb_lst;
cb_lst->active = 1;
+   if (param != NULL)
+   dev_cb.cb_arg = (void *) param;
+
rte_spinlock_unlock(&rte_eth_dev_cb_lock);
dev_cb.cb_fn(dev->data->port_id, dev_cb.event,
dev_cb.cb_arg);
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 7218b6f..b418c5e 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -3026,6 +3026,7 @@ enum rte_eth_event_type {
/**< queue state event (enabled/disabled) */
RTE_ETH_EVENT_INTR_RESET,
/**< reset interrupt event, sent to VF on PF reset */
+   RTE_ETH_EVENT_VF_MBOX,  /**< PF mailbox processing callback */
RTE_ETH_EVENT_MAX   /**< max value of this enum */
 };

@@ -3093,6 +3094,49 @@ void _rte_eth_dev_callback_process(struct rte_eth_dev 
*dev,
enum rte_eth_event_type event);

 /**
+ * @internal Executes all the user application registered callbacks for
+ * the specific device where parameter have to be passed to user application.
+ * It is for DPDK internal user only. User application should not call it
+ * directly.
+ *
+ * @param dev
+ *  Pointer to struct rte_eth_dev.
+ * @param event
+ *  Eth device interrupt event type.
+ *
+ * @param param
+ *  parameters to pass back to user application.
+ *
+ * @return
+ *  void
+ */
+
+void
+_rte_eth_dev_callback_process_vf(struct rte_eth_dev *dev,
+   enum rte_eth_event_type event, void *param);
+
+/**
+ * @internal Executes all the user application registered callbacks. Used by:
+ * _rte_eth_dev_callback_process and _rte_eth_dev_callback_process_vf
+ * It is for DPDK internal user only. User application should not call it
+ * directly.
+ *
+ * @param dev
+ *  Pointer to struct rte_eth_dev.
+ * @param event
+ *  Eth device interrupt event type.
+ *
+ * @param param
+ *  parameters to pass back to user application.
+ *
+ * @return
+ *  void
+ */
+void
+_rte_eth_dev_callback_process_generic(struct rte_eth_dev *dev,
+   enum rte_eth_event_type event, void *param);
+
+/**
  * When there is no rx packet coming in Rx Queue for a long time, we can
  * sleep lcore related to RX Queue for power saving, and enable rx interrupt
  * to be triggered when rx packect arrives.
diff --git a/lib/librte_ether/rte_ether_version.map 
b/lib/librte_ether/rte_ether_version.map
index 72be66d..6f97a8f 100644
--- a/lib/librte_ether/rte_ether_version.map
+++ b/lib/librte_ether/rte_ether_version.map
@@ -143,6 +143,8 @@ DPDK_16.07 {
 DPDK_16.11 {
global:

+   _rte_eth_dev_callback_process_generic;
+   _rte_eth_dev_callback_process_vf;
rte_eth_dev_pci_probe;
rte_eth_dev_pci_remove;

-- 
2.9.0

[dpdk-dev] [PATCH v4 0/2] add callbacks for VF management

[Bernard Iremonger]

This patchset contains new callback functions intended for VF management.

Two new callback functions have been added.
Changes have been made to the ixgbe_rcv_msg_from_vf function to
use the callback functions.

This patchset depends on the following patch.
http://dpdk.org/dev/patchwork/patch/16296/
[dpdk-dev,v6,1/2] net/ixgbe: add API's for VF management

These patches were part of the RFC PATCH v2 of the above patchset,
but were dropped from the v3 patchset.

Changes in v4:
Rebased to latest master.
Moved the callback parameter structure from the ethdev to the ixgbe PMD.

Changes in v3:
Rebased to latest master.
Submitted as a seperate patchset.
Moved the response enums from the ethdev to the ixgbe PMD.

Changes in v2:
Rebased to latest master.

Bernard Iremonger (2):
  librte_ether: add internal callback functions
  net/ixgbe: add callback to user app on VF to PF mbox msg

 drivers/net/ixgbe/ixgbe_pf.c   | 42 +++-
 drivers/net/ixgbe/rte_pmd_ixgbe.h  | 17 +
 lib/librte_ether/rte_ethdev.c  | 17 +
 lib/librte_ether/rte_ethdev.h  | 44 ++
 lib/librte_ether/rte_ether_version.map |  2 ++
 5 files changed, 116 insertions(+), 6 deletions(-)

-- 
2.9.0

[dpdk-dev] Problem Generating Traffic

[Mauricio Vasquez]

Hello,

While performing a series of throughput testing I found a limitation 
while generating traffic.

I have a server equipped with two 10G NICs that are connected using a 
Ethernet wire. MoonGen is used to generate traffic on these interfaces, 
it shows a performance of 22.52 Mpps. Theoretically it should be 29.76 
Mpps (14.88x2) while using 64 bytes long packets.

I tried to implemente a silly traffic generator by myself [1], It uses 4 
cores, 2 for sending and 2 for receiving, however in this case the 
throughput is still 22.52 Mpps.

I tried many different things, change the number of descriptors in the 
NIC, use separated mempools, run two separated DPDK processes, change 
the burst size, change the mempool parameters, however the maximum 
throughput I can get is always 22.52 Mpps.

My question is, what could be the bottleneck in this case?, is the PCI-e 
bus an option?

Any other cue?

Just in case, the server's characteristics:

- Intel Xeon E5-2690 v2 @ 3 GHz (ten physical cores plus hyperthreading)
- 64 GB RAM, Ubuntu 15.04, equipped with two 10G Intel 82599ES NICs.
- DPDK 16.07

Thanks in Advance,

Mauricio V.

[1] http://pastebin.com/k565gW6x

[dpdk-dev] [PATCH] doc: arm64: document DPDK application profiling methods

[Jan Viktorin]

On Tue, 04 Oct 2016 14:40:47 +0200
Thomas Monjalon  wrote:

> Thanks for providing a patch so quickly :)
> 
> 2016-10-04 16:10, Jerin Jacob:
> > +The PMU based scheme useful for high accuracy performance profiling.  
> 
> A verb is missing.
> 
> > +Find below the example steps to configure the PMU based cycle counter on an
> > +armv8 machine.
> > +
> > +.. code-block:: console
> > +
> > +git clone https://github.com/jerinjacobk/armv8_pmu_cycle_counter_el0
> > +cd armv8_pmu_cycle_counter_el0
> > +make
> > +sudo insmod pmu_el0_cycle_counter.ko
> > +cd $DPDK_DIR
> > +make config T=arm64-armv8a-linuxapp-gcc
> > +echo "CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU=y" >> build/.config
> > +make  
> 
> What about the ARM 32 code that Jan is using?

Hi, I didn't have time for this yet. The basic description is here:

 lib/librte_eal/common/include/arch/arm/rte_cycles_32.h

In the Linux Kernel, it is used here:

 arch/arm/kernel/perf_event_v7.c (see registers c12, c13 and c14)

Regards
Jan

> 
> > +.. warning::
> > +
> > +This method can not be used in production systems as this may alter PMU
> > +state used by standard Linux user space tool like perf.  
> 
> More details please?
> 



-- 
   Jan Viktorin  E-mail: Viktorin at RehiveTech.com
   System Architect  Web:www.RehiveTech.com
   RehiveTech
   Brno, Czech Republic

[dpdk-dev] [PATCH] doc: arm64: document DPDK application profiling methods

[Thomas Monjalon]

Thanks for providing a patch so quickly :)

2016-10-04 16:10, Jerin Jacob:
> +The PMU based scheme useful for high accuracy performance profiling.

A verb is missing.

> +Find below the example steps to configure the PMU based cycle counter on an
> +armv8 machine.
> +
> +.. code-block:: console
> +
> +git clone https://github.com/jerinjacobk/armv8_pmu_cycle_counter_el0
> +cd armv8_pmu_cycle_counter_el0
> +make
> +sudo insmod pmu_el0_cycle_counter.ko
> +cd $DPDK_DIR
> +make config T=arm64-armv8a-linuxapp-gcc
> +echo "CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU=y" >> build/.config
> +make

What about the ARM 32 code that Jan is using?

> +.. warning::
> +
> +This method can not be used in production systems as this may alter PMU
> +state used by standard Linux user space tool like perf.

More details please?

[dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if primary is missing tailqs

[Sergio Gonzalez Monroy]

Hi Jean,

As with your other patch, commit title needs fixing and also missing 
Signed-off-by line.

On 22/09/2016 22:17, Jean Tourrilhes wrote:
>   lib/librte_eal/common/eal_common_tailqs.c | 15 ---
>   1 file changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/lib/librte_eal/common/eal_common_tailqs.c 
> b/lib/librte_eal/common/eal_common_tailqs.c
> index bb08ec8..6960d06 100644
> --- a/lib/librte_eal/common/eal_common_tailqs.c
> +++ b/lib/librte_eal/common/eal_common_tailqs.c
> @@ -143,6 +143,8 @@ rte_eal_tailq_update(struct rte_tailq_elem *t)
>   t->head = rte_eal_tailq_create(t->name);
>   } else {
>   t->head = rte_eal_tailq_lookup(t->name);
> + if (t->head != NULL)
> + rte_tailqs_count++;
>   }
>   }
>   
> @@ -188,9 +190,16 @@ rte_eal_tailqs_init(void)
>   if (t->head == NULL) {
>   RTE_LOG(ERR, EAL,
>   "Cannot initialize tailq: %s\n", t->name);
> - /* no need to TAILQ_REMOVE, we are going to panic in
> -  * rte_eal_init() */
> - goto fail;
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + /* no need to TAILQ_REMOVE, we are going
> +  * to panic in rte_eal_init() */
> + goto fail;
> + } else {
> + /* This means our list of constructor is
> +  * no the same as primary. Just remove
> +  * that missing tailq and continue */
> + TAILQ_REMOVE(&rte_tailq_elem_head, t, next);
> + }
>   }
>   }
>   
I might be missing something here so bear with me.
The case you are trying to fix is, as an example, when your secondary 
app is using LPM but your primary is not.
So basically with this patch, you are removing the tailq for LPM on 
secondary and continuing as normal, is that the case?

I am not convinced about this approach.
I guess the assumption here is that all the TAILQ infrastructure works 
even when the tailq list itself is NULL?
I say assumption because I don't have an easy way to trigger this use 
case with default DPDK sample/test apps.

What about letting the secondary process create a tailq if it doesn't 
exists?
We would need to lock protect at least the register function to avoid 
race conditions when having multiple secondary processes.

David, what do you think?

Sergio

[dpdk-dev] [PATCH v2 2/2] app/testpmd/txonly: Reset headroom after raw packet allocation

[Maxime Coquelin]

This patch fixes txonly raw packets allocations by resetting the
available headroom.

Indeed, some PMDs such as Virtio might prepend some data to the
packet, resulting in mbuf's data_off field to be decremented each
time the mbuf gets re-allocated.

For Virtio PMD, it means that we use only single descriptors for the
first times mbufs get allocated, as at some point there is not
enough headroom to store the header.

Other alternative would be use standard API to allocate the packets,
which does reset the headroom, but the impact on performance is too
big to consider this an option.

Signed-off-by: Maxime Coquelin 
---
 app/test-pmd/txonly.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/app/test-pmd/txonly.c b/app/test-pmd/txonly.c
index d2736b7..8513a06 100644
--- a/app/test-pmd/txonly.c
+++ b/app/test-pmd/txonly.c
@@ -222,6 +222,14 @@ pkt_burst_transmit(struct fwd_stream *fs)
return;
break;
}
+
+   /*
+* Using raw alloc is good to improve performance,
+* but some consumers may use the headroom and so
+* decrement data_off. We need to make sure it is
+* reset to default value.
+*/
+   rte_pktmbuf_reset_headroom(pkt);
pkt->data_len = tx_pkt_seg_lengths[0];
pkt_seg = pkt;
if (tx_pkt_split == TX_PKT_SPLIT_RND)
-- 
2.7.4

[dpdk-dev] [PATCH v2 1/2] mbuf: add rte_pktmbuff_reset_headroom function

[Maxime Coquelin]

Some application use rte_mbuf_raw_alloc() function to improve
performance by not resetting mbuf's fields to their default state.

This can be however problematic for mbuf consumers that need some
headroom, meaning that data_off field gets decremented after
allocation. When the mbuf is re-used afterwards, there might not
be enough room for the consumer to prepend anything, if the data_off
field is not reset to its default value.

This patch adds a new rte_pktmbuf_reset_headroom() function that
applications can call to reset the data_off field.
This patch also replaces current data_off affectations in the mbuf
lib with a call to this function.

Signed-off-by: Maxime Coquelin 
---
Changes since v2:
=
 - Specify headroom may be reset only if segment is empty.

 lib/librte_mbuf/rte_mbuf.h | 18 +++---
 1 file changed, 15 insertions(+), 3 deletions(-)

diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 23b7bf8..85a653c 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -1387,6 +1387,19 @@ rte_pktmbuf_priv_size(struct rte_mempool *mp)
 }

 /**
+ * Reset the data_off field of a packet mbuf to its default value.
+ *
+ * The given mbuf must have only one segment, which should be empty.
+ *
+ * @param m
+ *   The packet mbuf's data_off field has to be reset.
+ */
+static inline void rte_pktmbuf_reset_headroom(struct rte_mbuf *m)
+{
+   m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len);
+}
+
+/**
  * Reset the fields of a packet mbuf to their default values.
  *
  * The given mbuf must have only one segment.
@@ -1406,8 +1419,7 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf *m)

m->ol_flags = 0;
m->packet_type = 0;
-   m->data_off = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ?
-   RTE_PKTMBUF_HEADROOM : m->buf_len;
+   rte_pktmbuf_reset_headroom(m);

m->data_len = 0;
__rte_mbuf_sanity_check(m, 1);
@@ -1571,7 +1583,7 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m)
m->buf_addr = (char *)m + mbuf_size;
m->buf_physaddr = rte_mempool_virt2phy(mp, m) + mbuf_size;
m->buf_len = (uint16_t)buf_len;
-   m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len);
+   rte_pktmbuf_reset_headroom(m);
m->data_len = 0;
m->ol_flags = 0;

-- 
2.7.4

[dpdk-dev] [PATCH 1/2] mbuf: add rte_pktmbuff_reset_headroom function

[Maxime Coquelin]

On 10/03/2016 06:11 PM, Olivier Matz wrote:
> Hi Maxime,
>
> On 09/29/2016 02:20 PM, Maxime Coquelin wrote:
>> Some application use rte_mbuf_raw_alloc() function to improve
>> performance by not resetting mbuf's fields to their default state.
>>
>> This can be however problematic for mbuf consumers that need some
>> headroom, meaning that data_off field gets decremented after
>> allocation. When the mbuf is re-used afterwards, there might not
>> be enough room for the consumer to prepend anything, if the data_off
>> field is not reset to its default value.
>>
>> This patch adds a new rte_pktmbuf_reset_headroom() function that
>> applications can call to reset the data_off field.
>> This patch also replaces current data_off affectations in the mbuf
>> lib with a call to this function.
>>
>> Signed-off-by: Maxime Coquelin 
>
> Sounds like a good idea. Just one small comment below.
>
>>
>>  /**
>> + * Reset the data_off field of a packet mbuf to its default value.
>> + *
>> + * The given mbuf must have only one segment.
>> + *
>> + * @param m
>> + *   The packet mbuf's data_off field has to be reset.
>> + */
>> +static inline void rte_pktmbuf_reset_headroom(struct rte_mbuf *m)
>> +{
>> +m->data_off = RTE_MIN(RTE_PKTMBUF_HEADROOM, (uint16_t)m->buf_len);
>> +}
>
> Maybe we should also highlight in the API comment that the segment
> should be empty.

Good point. I'll change to:

The given mbuf must have only one segment, which should be empty.

Thanks,
Maxime

[dpdk-dev] Proposal: enable redirection of DPDK logs from the user app

[Olivier Matz]

Hi Francesco,

On 10/04/2016 12:24 PM, Montorsi, Francesco wrote:
> Hi all,
> I've not been following closely latest DPDK activity but my company is using 
> DPDK and we recently upgraded to 16.07. 
> We apply several patches to DPDK sources, to make it more similar to a 
> "standard library" (currently it is quite intrusive: calls abort() at will, 
> writes its own log, etc etc)... I think that it may be useful to give back to 
> the community some of these (small) "enhancements".
> 
> One of them is about logging: as the application where we embed DPDK already 
> has its log file, we want DPDK to log in our log facility framework. 
> My "fix" is simple: I just put a callback function in RTE logging system 
> that, by default, points to the existing rte_vlog() function. If needed the 
> library user can provide its own callback function to do what he likes.
> 
> The attached patch is what we use right now. I totally understand it needs 
> some rework to put it in a better shape... but first of all: are you 
> interested in such patch? 

It seems the mailing list stripped your patch sent as attachment.
Can you please resend it again in the body of the mail?

I think we can already redirect logs to a file by using fopencookie() +
rte_openlog_stream(). Did you already check these functions?

Regards,
Olivier

[dpdk-dev] Proposal: enable redirection of DPDK logs from the user app

[Montorsi, Francesco]

Hi Olivier,

> It seems the mailing list stripped your patch sent as attachment.
> Can you please resend it again in the body of the mail?
You're right sorry. It's attached at the end of this mail.

> I think we can already redirect logs to a file by using fopencookie() +
> rte_openlog_stream(). Did you already check these functions?

Yes, but to be honest, that seems a troublesome solution for something as easy 
as logging a string; e.g. by using fopencookie() approach, you don't have the 
concept of "log message", you just provide a function that must write a block 
of bytes somewhere.  Typically instead, you need to know where a log message 
starts and ends, to e.g., add prefixes/postfixes to it.

Indeed, most of the C/C++ (open source) libraries have some simple hook that 
allows the user to have more control on logging... I think DPDK should be no 
exception... :)

Thanks,
Francesco



>From 52d4fdccee4de3787e85589ff8f666028ad9ea7b Mon Sep 17 00:00:00 2001
From: Francesco Montorsi 
Date: Tue, 4 Oct 2016 12:08:34 +0200
Subject: [PATCH] Enable custom log sink implementations

---
 lib/librte_eal/common/eal_common_log.c  | 23 ---
 lib/librte_eal/common/include/rte_log.h |  8 ++--
 lib/librte_eal/linuxapp/eal/eal_debug.c | 16 ++--
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  2 +-
 4 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/lib/librte_eal/common/eal_common_log.c 
b/lib/librte_eal/common/eal_common_log.c
index 967991a..5e86309 100644
--- a/lib/librte_eal/common/eal_common_log.c
+++ b/lib/librte_eal/common/eal_common_log.c
@@ -41,15 +41,25 @@

 #include "eal_private.h"

+
+/* forward declaration */
+int
+rte_vlog_to_FILE(uint32_t level, uint32_t logtype, const char *formattedstr);
+
 /* global log structure */
 struct rte_logs rte_logs = {
.type = ~0,
.level = RTE_LOG_DEBUG,
.file = NULL,
+   .log_callback = rte_vlog_to_FILE
 };

 static FILE *default_log_stream;

+#define LOG_BUFFER_SIZE4095
+static char log_buffer[LOG_BUFFER_SIZE+1];
+
+
 /**
  * This global structure stores some informations about the message
  * that is currently beeing processed by one lcore
@@ -123,7 +133,7 @@ int rte_log_cur_msg_logtype(void)
  * defined by the previous call to rte_openlog_stream().
  */
 int
-rte_vlog(uint32_t level, uint32_t logtype, const char *format, va_list ap)
+rte_vlog_to_FILE(uint32_t level, uint32_t logtype, const char *formattedstr)
 {
int ret;
FILE *f = rte_logs.file;
@@ -135,11 +145,16 @@ rte_vlog(uint32_t level, uint32_t logtype, const char 
*format, va_list ap)
RTE_PER_LCORE(log_cur_msg).loglevel = level;
RTE_PER_LCORE(log_cur_msg).logtype = logtype;

-   ret = vfprintf(f, format, ap);
+   ret = fprintf(f, "%s", formattedstr);
fflush(f);
return ret;
 }

+void rte_set_custom_vlog(rte_vlog_func_t callback)
+{
+   rte_logs.log_callback = callback;
+}
+
 /*
  * Generates a log message The message will be sent in the stream
  * defined by the previous call to rte_openlog_stream().
@@ -152,8 +167,10 @@ rte_log(uint32_t level, uint32_t logtype, const char 
*format, ...)
int ret;

va_start(ap, format);
-   ret = rte_vlog(level, logtype, format, ap);
+   vsnprintf(log_buffer, LOG_BUFFER_SIZE, format, ap);
va_end(ap);
+
+   ret = rte_logs.log_callback(level, logtype, log_buffer);
return ret;
 }

diff --git a/lib/librte_eal/common/include/rte_log.h 
b/lib/librte_eal/common/include/rte_log.h
index 919563c..3dcb135 100644
--- a/lib/librte_eal/common/include/rte_log.h
+++ b/lib/librte_eal/common/include/rte_log.h
@@ -50,11 +50,15 @@ extern "C" {
 #include 
 #include 

+/** The backend used for logging. Can be user-defined. */
+typedef int (*rte_vlog_func_t)(uint32_t level, uint32_t logtype, const char 
*formattedstr);
+
 /** The rte_log structure. */
 struct rte_logs {
uint32_t type;  /**< Bitfield with enabled logs. */
uint32_t level; /**< Log level. */
FILE *file; /**< Pointer to current FILE* for logs. */
+   rte_vlog_func_t log_callback;
 };

 /** Global log informations */
@@ -236,8 +240,8 @@ int rte_log(uint32_t level, uint32_t logtype, const char 
*format, ...)
  *   - 0: Success.
  *   - Negative on error.
  */
-int rte_vlog(uint32_t level, uint32_t logtype, const char *format, va_list ap)
-   __attribute__((format(printf,3,0)));
+void rte_set_custom_vlog(rte_vlog_func_t callback);
+

 /**
  * Generates a log message.
diff --git a/lib/librte_eal/linuxapp/eal/eal_debug.c 
b/lib/librte_eal/linuxapp/eal/eal_debug.c
index 5fbc17c..f411d96 100644
--- a/lib/librte_eal/linuxapp/eal/eal_debug.c
+++ b/lib/librte_eal/linuxapp/eal/eal_debug.c
@@ -78,9 +78,16 @@ void __rte_panic(const char *funcname, const char *format, 
...)
va_list ap;

rte_log(RTE_LOG_CRIT, RTE_LOGTYPE_EAL, "PANIC in %s():\n", funcname);
+
+#define LOG_BUFFER_S

[dpdk-dev] [PATCH v11 00/24] Introducing rte_driver/rte_device generalization

[Shreyansh Jain]

Hi Thomas,

On Monday 03 October 2016 07:58 PM, Thomas Monjalon wrote:
> Applied, thanks everybody for the great (re)work!

Thanks!

>
> 2016-09-20 18:11, Shreyansh Jain:
>> Future Work/Pending:
>> ===
>>  - Presently eth_driver, rte_eth_dev are not aligned to the rte_driver/
>>rte_device model. eth_driver still is a PCI specific entity. This
>>has been highlighted by comments from Ferruh in [9].
>>  - Some variables, like drv_name (as highlighted by Ferruh), are getting
>>duplicated across rte_xxx_driver/device and rte_driver/device.

Both the above are already part of my todo list.

>
> What about those pending work?
>
> I would add more remaining issues:
> - probe/remove naming could be applied to vdev for consistency

Is that for uniformity reasons? I still feel 'probe/remove' are not 
appropriate for a virtual device. init/deinit are more appropriate. As 
for PCI, probe/remove are standard parlance and hence suit it better 
than init/deinit.

Nevertheless, uniform naming convention can have its benefits - ease of 
code understanding being one.

Change is simple once we come to a conclusion.

> - rte_eal_device_insert must be called in vdev

Ok.

> - REGISTER macros should be prefixed with RTE_

That would include:
  DRIVER_REGISTER_VDEV
  DRIVER_REGISTER_PCI_TABLE
  DRIVER_REGISTER_PCI

I will publish a patch soon. This would be fairly straightforward change.

> - Some functions in EAL does not need eal_ in their prefix:
>   rte_eal_pci_   -> rte_pci_
>   rte_eal_dev_   -> rte_dev_
>   rte_eal_vdev_  -> rte_vdev_
>   rte_eal_driver -> rte_drv_
>   rte_eal_vdrv   -> rte_vdrv_
>
>

It can be merged with changes for:
  - drv_name
  - EAL_ before _REGISTER_ macros
  - eth_driver => rte_driver naming

-- 
-
Shreyansh

[dpdk-dev] [PATCH v3] tools: fix issue with virtio interfaces

[Thomas Monjalon]

2016-08-26 07:35, souvikdey33:
> 
> This change is required to have the interface name for virtio interfaces.
> When we execute the status command the for virtio inerfaces we get
> Sample output without the change:
> :00:04.0 'Virtio network device' if= drv=virtio-pci 
> unused=virtio_pci,igb_uio
> Though for other drivers this works.
> Sample output with the change:
> :00:04.0 'Virtio network device' if=eth0 drv=virtio-pci 
> unused=virtio_pci,igb_uio
> 
> souvikdey33 (1):
>   Signed-off-by: Souvik Dey 

The patch from Gary - which do not use subprocess - has been preferred:
http://dpdk.org/patch/15595

[dpdk-dev] [PATCH] tools: fix json output of pmdinfo

[Thomas Monjalon]

2016-08-26 15:15, Olivier Matz:
> Using dpdk-pmdinfo with the '-r' flag does not produce a json output as
> documented. Instead, the python representation of the json object is
> shown, which is nearly the same, but cannot be properly parsed by a json
> parser.
> 
> python repr (before):
>   {u'pci_ids': [[5549, 1968, 65535, 65535]], u'name': u'vmxnet3'}
> json (after):
>   {"pci_ids": [[5549, 1968, 65535, 65535]], "name": "vmxnet3"}
> 
> Fixes: c67c9a5c646a ("tools: query binaries for HW and other support 
> information")
> 
> Signed-off-by: Olivier Matz 

Applied, thanks

[dpdk-dev] [PATCH] pmdinfogen: fix clang build error

[Thomas Monjalon]

2016-09-28 11:05, Ferruh Yigit:
> Compile error:
>   CC mlx5.o.pmd.o
> mlx5.o.pmd.c:1:227:
> error: no newline at end of file [-Werror,-Wnewline-eof]
>   ...__attribute__((used)) = "PMD_INFO_STRING= {...}";
>   ^
> 
> Produced with clang 3.8.0 and MLX5_PMD and MLX5_DEBUG
> config options enabled.
> 
> Fixes: 98b0fdb0ffc6 ("pmdinfogen: add buildtools and pmdinfogen utility")
> 
> Signed-off-by: Ferruh Yigit 

Applied, thanks

[dpdk-dev] [PATCH v11 07/24] driver: probe/remove common wrappers for PCI drivers

[Shreyansh Jain]

On Monday 03 October 2016 07:51 PM, Thomas Monjalon wrote:
> 2016-09-20 18:11, Shreyansh Jain:
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -4372,6 +4372,19 @@ rte_eth_dev_get_port_by_name(const char *name, 
>> uint8_t *port_id);
>>  int
>>  rte_eth_dev_get_name_by_port(uint8_t port_id, char *name);
>>
>> +/**
>> + * Wrapper for use by pci drivers as a .probe function to attach to a ethdev
>> + * interface.
>> + */
>> +int rte_eth_dev_pci_probe(struct rte_pci_driver *pci_drv,
>> +  struct rte_pci_device *pci_dev);
>> +
>> +/**
>> + * Wrapper for use by pci drivers as a .remove function to detach a ethdev
>> + * interface.
>> + */
>> +int rte_eth_dev_pci_remove(struct rte_pci_device *pci_dev);
>
> These functions are used by the drivers only (as helpers).
> So they should be marked @internal (added after applying the patch).
>

Ok.
Thanks!

-
Shreyansh

[dpdk-dev] [PATCH] hash: fix incorrect free slot check

[Thomas Monjalon]

2016-10-04 10:34, Bruce Richardson:
> On Tue, Oct 04, 2016 at 08:16:34AM +0100, Pablo de Lara wrote:
> > In function rte_hash_cuckoo_insert_mw_tm, while looking for
> > an empty slot, only the first entry in the bucket was being checked,
> > as key_idx array was not being iterated.
> > 
> > Fixes: 5fc74c2e146d ("hash: check if slot is empty with key index")
> > 
> > Reported-by: Bruce Richardson 
> > Signed-off-by: Pablo de Lara 
> Acked-by: Bruce Richardson 

Applied, thanks

[dpdk-dev] [PATCH] ethdev: clarify API comment for imissed stats

[Thomas Monjalon]

2016-09-09 10:15, Olivier Matz:
> The "imissed" stats represent RX packets dropped by the HW,
> so we should not talk about mbufs as the hardware is not aware
> of this structure. Buffer seems to be a better word.
> 
> Fixes: 4eadb8ba11b7 ("ethdev: do not deprecate imissed counter")
> 
> Signed-off-by: Olivier Matz 

Applied, thanks

[dpdk-dev] [PATCH] ethdev: fix statistics description

[Thomas Monjalon]

2016-08-26 18:08, Wei Dai:
>  /**
>   * A structure used to retrieve statistics for an Ethernet port.
> + * Not all statistics fields in struct rte_eth_stats are supported
> + * by any type of network interface card (NIC). If any statistics
> + * field is not supported, its value is 0 .
>   */
>  struct rte_eth_stats {

I'm missing the point of this patch.
Why do you think it is a fix?

John, any opinion?

[dpdk-dev] [PATCH 1/1 v2] eal: Fix misleading error messages, errno can't be trusted.

[Thomas Monjalon]

2016-10-04 09:03, Sergio Gonzalez Monroy:
> On 03/10/2016 21:46, Thomas Monjalon wrote:
> > 2016-10-03 08:55, Jean Tourrilhes:
> >> On Mon, Oct 03, 2016 at 02:25:40PM +0100, Sergio Gonzalez Monroy wrote:
> >>> Hi Jean,
> >>>
> >>> There are some format issues with the patch:
> >>>
> >>> You can run scripts/check-git-log.sh to check them:
> >>> Wrong headline format:
> >>>  eal: Fix misleading error messages, errno can't be trusted.
> >>> Wrong headline uppercase:
> >>>  eal: Fix misleading error messages, errno can't be trusted.
> >>> Missing 'Fixes' tag:
> >>>  eal: Fix misleading error messages, errno can't be trusted.
> >>>
> >>> The script's output highlights the different issues.
> >>SOrry about that, I casually read the page on
> >> http://dpdk.org/dev, but obviously I need to look at it again.
> > No problem. This guide is more oriented towards regular contributors.
> > You come with a bug and its fix, we can make some effort to format
> > the patch :)
> >
> > The title could be "mem: fix hugepage mapping error messages"
[...]
> > Sergio, do you have more comments?
> 
> Nop.
> 
> > Should we wait another version or is it OK?
> > Maybe you'd prefer to rework it yourself?
> 
> AFAICS it really is just the commit message formatting, so maybe you can 
> fix that when applying it?

Yes but I've just noticed that the Signed-off-by line is missing.
Please Jean, could you add this required line with your explanations,
rebase on fresh head and resend please?

[dpdk-dev] [PATCH v2 1/2] mempool: fix comments for mempool create functions

[Thomas Monjalon]

2016-10-03 17:06, Olivier Matz:
> On 09/28/2016 03:59 PM, Ferruh Yigit wrote:
> > Fixes: 85226f9c526b ("mempool: introduce a function to create an empty 
> > pool")
> > Fixes: d1d914ebbc25 ("mempool: allocate in several memory chunks by 
> > default")
> > 
> > Signed-off-by: Ferruh Yigit 
> 
> Series:
> Acked-by: Olivier Matz 

Applied, thanks

[dpdk-dev] [PATCH v4 0/4] Cuckoo hash enhancements

[Bruce Richardson]

On Tue, Oct 04, 2016 at 08:17:28AM +0100, De Lara Guarch, Pablo wrote:
> 
> 
> > -Original Message-
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of De Lara Guarch,
> > Pablo
> > Sent: Monday, October 03, 2016 11:51 PM
> > To: Richardson, Bruce
> > Cc: dev at dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v4 0/4] Cuckoo hash enhancements
> > 
> > Hi Bruce,
> > 
> > > -Original Message-
> > > From: Richardson, Bruce
> > > Sent: Monday, October 03, 2016 2:59 AM
> > > To: De Lara Guarch, Pablo
> > > Cc: dev at dpdk.org
> > > Subject: Re: [PATCH v4 0/4] Cuckoo hash enhancements
> > >
> > > On Fri, Sep 30, 2016 at 08:38:52AM +0100, Pablo de Lara wrote:
> > > > This patchset improves lookup performance on the current hash library
> > > > by changing the existing lookup bulk pipeline, with an improved 
> > > > pipeline,
> > > > based on a loop-and-jump model, instead of the current 4-stage 2-entry
> > > pipeline.
> > > > Also, x86 vectorized intrinsics are used to improve performance when
> > > comparing signatures.
> > > >
> > > > First patch reorganizes the order of the hash structure.
> > > > The structure takes more than one 64-byte cache line, but not all
> > > > the fields are used in the lookup operation (the most common operation).
> > > > Therefore, all these fields have been moved to the first part of the
> > structure,
> > > > so they all fit in one cache line, improving slightly the performance in
> > some
> > > > scenarios.
> > > >
> > > > Second patch modifies the order of the bucket structure.
> > > > Currently, the buckets store all the signatures together (current and
> > > alternative).
> > > > In order to be able to perform a vectorized signature comparison,
> > > > all current signatures have to be together, so the order of the bucket 
> > > > has
> > > been changed,
> > > > having separated all the current signatures from the alternative
> > signatures.
> > > >
> > > > Third patch introduces x86 vectorized intrinsics.
> > > > When performing a lookup bulk operation, all current signatures in a
> > bucket
> > > > are compared against the signature of the key being looked up.
> > > > Now that they all are together, a vectorized comparison can be
> > performed,
> > > > which takes less instructions to be carried out.
> > > > In case of having a machine with AVX2, number of entries per bucket are
> > > > increased from 4 to 8, as AVX2 allows comparing two 256-bit values, with
> > > 8x32-bit integers,
> > > > which are the 8 signatures on the bucket.
> > > >
> > > > Fourth (and last) patch modifies the current pipeline of the lookup bulk
> > > function.
> > > > The new pipeline is based on a loop-and-jump model. The two key
> > > improvements are:
> > > >
> > > > - Better prefetching: in this case, first 4 keys to be looked up are
> > prefetched,
> > > >   and after that, the rest of the keys are prefetched at the time the
> > > calculation
> > > >   of the signatures are being performed. This gives more time for the 
> > > > CPU
> > to
> > > >   prefetch the data requesting before actually need it, which result in 
> > > > less
> > > >   cache misses and therefore, higher throughput.
> > > >
> > > > - Lower performance penalty when using fallback: the lookup bulk
> > > algorithm
> > > >   assumes that most times there will not be a collision in a bucket, 
> > > > but it
> > > might
> > > >   happen that two or more signatures are equal, which means that more
> > > than one
> > > >   key comparison might be necessary. In that case, only the key of the 
> > > > first
> > > hit is prefetched,
> > > >   like in the current implementation. The difference now is that if this
> > > comparison
> > > >   results in a miss, the information of the other keys to be compared 
> > > > has
> > > been stored,
> > > >   unlike the current implementation, which needs to perform an entire
> > > simple lookup again.
> > > >
> > > > Changes in v4:
> > > > - Reordered hash structure, so alt signature is at the start
> > > >   of the next cache line, and explain in the commit message
> > > >   why it has been moved
> > > > - Reordered hash structure, so name field is on top of the structure,
> > > >   leaving all the fields used in lookup in the next cache line
> > > >   (instead of the first cache line)
> > > >
> > > > Changes in v3:
> > > > - Corrected the cover letter (wrong number of patches)
> > > >
> > > > Changes in v2:
> > > > - Increased entries per bucket from 4 to 8 for all cases,
> > > >   so it is not architecture dependent any longer.
> > > > - Replaced compile-time signature comparison function election
> > > >   with run-time election, so best optimization available
> > > >   will be used from a single binary.
> > > > - Reordered the hash structure, so all the fields used by lookup
> > > >   are in the same cache line (first).
> > > >
> > > > Byron Marohn (3):
> > > >   hash: reorganize bucket structure
> > > >   hash: add vectorized comparison
> > > >   hash: modify lookup bulk 

[dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter

[Thomas Monjalon]

> > Existing cntvct_el0 based rte_rdtsc() provides portable means to get wall 
> > clock
> > counter at user space. Typically it runs at <= 100MHz.
> > 
> > The alternative method to enable rte_rdtsc() for high resolution wall clock
> > counter is through armv8 PMU subsystem.
> > The PMU cycle counter runs at CPU frequency, However, access to PMU cycle
> > counter from user space is not enabled by default in the arm64 linux kernel.
> > It is possible to enable cycle counter at user space access by configuring 
> > the
> > PMU from the privileged mode (kernel space).
> > 
> > by default rte_rdtsc() implementation uses portable
> > cntvct_el0 scheme. Application can choose the PMU based implementation with
> > CONFIG_RTE_ARM_EAL_RDTSC_USE_PMU
> > 
> > Signed-off-by: Jerin Jacob 
> 
> Acked-by: Hemant Agrawal 

Applied, thanks

Please do not forget documentation and upstreaming efforts.

[dpdk-dev] [PATCH] eal/armv8: high-resolution cycle counter

[Thomas Monjalon]

2016-08-19 18:22, Jerin Jacob:
> On Fri, Aug 19, 2016 at 02:24:58PM +0200, Jan Viktorin wrote:
> > On Fri, 19 Aug 2016 17:16:12 +0530
> > Jerin Jacob  wrote:
> > 
> > 
> > I've got a private kernel driver enabling and disabling (hopefully) properly
> > this for ARMv7. If we'd like to merge it, I'd like to have a single module
> > or at least single module with 2 implementations...
> > 
> > I can post it if it would be helpful.
> 
> I don't think we can use this in production as this may alter PMU state used
> by 'perf' etc.I think let it be a debug interface for armv7 and armv8
> and disable it by default.

Please could you document the use of PMU for debug and how it alters
usage of kernel counters?
A patch in doc/guides/prog_guide/profile_app.rst would be welcome.

Ideally, it would be a lot better to have a sysfs entry to enable PMU
counter with an upstream kernel.

[dpdk-dev] [PATCH] hash: fix incorrect free slot check

[Bruce Richardson]

On Tue, Oct 04, 2016 at 08:16:34AM +0100, Pablo de Lara wrote:
> In function rte_hash_cuckoo_insert_mw_tm, while looking for
> an empty slot, only the first entry in the bucket was being checked,
> as key_idx array was not being iterated.
> 
> Fixes: 5fc74c2e146d ("hash: check if slot is empty with key index")
> 
> Reported-by: Bruce Richardson 
> Signed-off-by: Pablo de Lara 
Acked-by: Bruce Richardson 

[dpdk-dev] [PATCH] log: do not drop debug logs at compile time

[David Marchand]

On Mon, Oct 3, 2016 at 6:27 PM, Wiles, Keith  wrote:
>> On Oct 3, 2016, at 10:37 AM, Olivier Matz  wrote:
>> What makes you feel it's easier to add a log level instead of adding a
>> new RTE_LOG_DP() function?
>
> It seems to me the log levels are for displaying logs at different levels 
> adding a new macro to not log is just a hack because we do not have a log 
> level for data path. This is why I would like to see a log level added and 
> not a new macro.
>
> It also appears the new RTE_LOG() will always be in the code as you moved the 
> test to the RTE_LOG_DP() macro. This would mean all RTE_LOG() in the code 
> will always call rte_log(), correct?
>
> If using a new DEBUG_DP (maybe DATAPATH is a better log level name) level we 
> can use the same macro as before and modify the level only. This way we can 
> remove via the compiler any log that is below the default RTE_LOG_LEVEL. I 
> see keeping the rte_log() could be a performance problem or code blot when 
> you really want to remove them all.
>
> The DATAPATH log level would be above (smaller number) then DEBUG in the enum 
> list. To remove all debug logs just set the RTE_LOG_LEVEL to RTE_LOG_DATAPATH.

If I try to draw a parrallel to syslog (well, the log subsystem in eal
has always been bound to syslog ...), what you propose here is like
adding a new level in syslog while you have syslog facilities.

With the current log api, we have types and levels, can't we filter at
build time depending on the log "type" ?
Here we would strip PMD type logs > INFO.


-- 
David Marchand

[dpdk-dev] [PATCH] log: do not drop debug logs at compile time

[David Marchand]

On Fri, Sep 16, 2016 at 9:43 AM, Olivier Matz  wrote:
> Today, all logs whose level is lower than INFO are dropped at
> compile-time. This prevents from enabling debug logs at runtime using
> --log-level=8.
>
> The rationale was to remove debug logs from the data path at
> compile-time, avoiding a test at run-time.
>
> This patch changes the behavior of RTE_LOG() to avoid the compile-time
> optimization, and introduces the RTE_LOG_DP() macro that has the same
> behavior than the previous RTE_LOG(), for the rare cases where debug
> logs are in the data path.
>
> So it is now possible to enable debug logs at run-time by just
> specifying --log-level=8. Some drivers still have special compile-time
> options to enable more debug log. Maintainers may consider to
> remove/reduce them.
>
> Signed-off-by: Olivier Matz 
> ---
>  config/common_base  |  1 +
>  doc/guides/faq/faq.rst  |  2 +-
>  drivers/net/bnxt/bnxt_txr.c |  2 +-
>  drivers/net/nfp/nfp_net.c   |  8 +++---
>  examples/distributor/main.c |  4 +--
>  examples/ipsec-secgw/esp.c  |  2 +-
>  examples/ipsec-secgw/ipsec.c|  4 +--
>  examples/packet_ordering/main.c |  6 ++--
>  examples/quota_watermark/qw/main.c  |  2 +-
>  examples/tep_termination/main.c |  4 +--
>  examples/vhost/main.c   | 14 +-
>  examples/vhost_xen/main.c   | 20 +++---
>  lib/librte_eal/common/include/rte_log.h | 49 
> +
>  13 files changed, 67 insertions(+), 51 deletions(-)
>
> diff --git a/config/common_base b/config/common_base
> index 7830535..04b71e9 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -89,6 +89,7 @@ CONFIG_RTE_MAX_MEMSEG=256
>  CONFIG_RTE_MAX_MEMZONE=2560
>  CONFIG_RTE_MAX_TAILQ=32
>  CONFIG_RTE_LOG_LEVEL=RTE_LOG_INFO
> +CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_INFO
>  CONFIG_RTE_LOG_HISTORY=256
>  CONFIG_RTE_LIBEAL_USE_HPET=n
>  CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n

[snip]

> diff --git a/lib/librte_eal/common/include/rte_log.h 
> b/lib/librte_eal/common/include/rte_log.h
> index 919563c..76b198f 100644
> --- a/lib/librte_eal/common/include/rte_log.h
> +++ b/lib/librte_eal/common/include/rte_log.h

[snip]

> @@ -266,6 +257,30 @@ int rte_vlog(uint32_t level, uint32_t logtype, const 
> char *format, va_list ap)
>   *   - Negative on error.
>   */
>  #define RTE_LOG(l, t, ...) \
> +rte_log(RTE_LOG_ ## l, \
> +RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__)
> +
> +/**
> + * Generates a log message for data path.
> + *
> + * Similar to RTE_LOG(), except that it is removed at compilation time
> + * if the RTE_LOG_DP_LEVEL configuration option is lower than the log
> + * level argument.
> + *
> + * @param l
> + *   Log level. A value between EMERG (1) and DEBUG (8). The short name is
> + *   expanded by the macro, so it cannot be an integer value.
> + * @param t
> + *   The log type, for example, EAL. The short name is expanded by the
> + *   macro, so it cannot be an integer value.
> + * @param ...
> + *   The fmt string, as in printf(3), followed by the variable arguments
> + *   required by the format.
> + * @return
> + *   - 0: Success.
> + *   - Negative on error.
> + */
> +#define RTE_LOG_DP(l, t, ...)  \
> (void)((RTE_LOG_ ## l <= RTE_LOG_LEVEL) ?   \
>  rte_log(RTE_LOG_ ## l, \
>  RTE_LOGTYPE_ ## t, # t ": " __VA_ARGS__) : \
> --
> 2.8.1

Hum, I suppose RTE_LOG_DP should look at RTE_LOG_DP_LEVEL.


-- 
David Marchand

[dpdk-dev] [PATCH 0/5] i40e: vector poll-mode driver on ARM64

[Thomas Monjalon]

2016-09-19 17:25, Bruce Richardson:
> On Wed, Aug 24, 2016 at 03:23:40PM +0530, Jianbo Liu wrote:
> > This patch set is to implement i40e vector PMD on ARM64.
> > For x86, vPMD is only reorganized, there should be no performance loss.
> > 
> > Jianbo Liu (5):
> >   i40e: extract non-x86 specific code from vector driver
> >   i40e: implement vector PMD for ARM architecture
> >   i40e: enable i40e vector PMD on ARMv8a platform
> >   i40e: make vector driver filenames consistent
> >   maintainers: claim i40e vector PMD on ARM
> > 
> Ping for comment/ack - especially from i40e maintainers.

Why is there no review of these patches for acceleration on ARM?

[dpdk-dev] Proposal: enable redirection of DPDK logs from the user app

[Montorsi, Francesco]

Hi all,
I've not been following closely latest DPDK activity but my company is using 
DPDK and we recently upgraded to 16.07. 
We apply several patches to DPDK sources, to make it more similar to a 
"standard library" (currently it is quite intrusive: calls abort() at will, 
writes its own log, etc etc)... I think that it may be useful to give back to 
the community some of these (small) "enhancements".

One of them is about logging: as the application where we embed DPDK already 
has its log file, we want DPDK to log in our log facility framework. 
My "fix" is simple: I just put a callback function in RTE logging system that, 
by default, points to the existing rte_vlog() function. If needed the library 
user can provide its own callback function to do what he likes.

The attached patch is what we use right now. I totally understand it needs some 
rework to put it in a better shape... but first of all: are you interested in 
such patch? 

Thanks,
Francesco Montorsi

[dpdk-dev] [PATCH v2 1/3] mem: fix hugepage mapping error messages

[Jean Tourrilhes]

Running secondary is tricky due to the need to map the memory region
at the right place in VM, which is whatever primary has chosen. If the
base address for primary happens to by already mapped in the
secondary, we will hit precisely these error messages (depending if we
fail on the config region or the hugepages). This is why there is
already a comment about ASLR.

The issue is that in most cases, remapping does not happen and "errno"
is not changed and therefore stale. In our case, we got a "permission
denied", which sent us down the wrong track. It's such a common error
for secondary that I feel this error message should be unambiguous and
helpful.
The call to close was also moved because close() may override errno.

Signed-off-by: Jean Tourrilhes 
---
 lib/librte_eal/linuxapp/eal/eal.c| 14 +++---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 16 
 2 files changed, 23 insertions(+), 7 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal.c 
b/lib/librte_eal/linuxapp/eal/eal.c
index 3fb2188..5df9f6a 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -238,7 +238,8 @@ rte_eal_config_attach(void)
mem_config = (struct rte_mem_config *) mmap(NULL, sizeof(*mem_config),
PROT_READ, MAP_SHARED, mem_cfg_fd, 0);
if (mem_config == MAP_FAILED)
-   rte_panic("Cannot mmap memory for rte_config\n");
+   rte_panic("Cannot mmap memory for rte_config! error %i (%s)\n",
+ errno, strerror(errno));

rte_config.mem_config = mem_config;
 }
@@ -263,9 +264,16 @@ rte_eal_config_reattach(void)
mem_config = (struct rte_mem_config *) mmap(rte_mem_cfg_addr,
sizeof(*mem_config), PROT_READ | PROT_WRITE, MAP_SHARED,
mem_cfg_fd, 0);
+   if (mem_config == MAP_FAILED || mem_config != rte_mem_cfg_addr) {
+   if (mem_config != MAP_FAILED)
+   /* errno is stale, don't use */
+   rte_panic("Cannot mmap memory for rte_config at [%p], 
got [%p] - please use '--base-virtaddr' option\n",
+ rte_mem_cfg_addr, mem_config);
+   else
+   rte_panic("Cannot mmap memory for rte_config! error %i 
(%s)\n",
+ errno, strerror(errno));
+   }
close(mem_cfg_fd);
-   if (mem_config == MAP_FAILED || mem_config != rte_mem_cfg_addr)
-   rte_panic("Cannot mmap memory for rte_config\n");

rte_config.mem_config = mem_config;
 }
diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c 
b/lib/librte_eal/linuxapp/eal/eal_memory.c
index 41e0a92..b036ffc 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -1615,10 +1615,18 @@ rte_eal_hugepage_attach(void)
 PROT_READ, MAP_PRIVATE, fd_zero, 0);
if (base_addr == MAP_FAILED ||
base_addr != mcfg->memseg[s].addr) {
-   RTE_LOG(ERR, EAL, "Could not mmap %llu bytes "
-   "in /dev/zero to requested address [%p]: 
'%s'\n",
-   (unsigned long long)mcfg->memseg[s].len,
-   mcfg->memseg[s].addr, strerror(errno));
+   if (base_addr != MAP_FAILED)
+   /* errno is stale, don't use */
+   RTE_LOG(ERR, EAL, "Could not mmap %llu bytes "
+   "in /dev/zero at [%p], got [%p] - "
+   "please use '--base-virtaddr' option\n",
+   (unsigned long long)mcfg->memseg[s].len,
+   mcfg->memseg[s].addr, base_addr);
+   else
+   RTE_LOG(ERR, EAL, "Could not mmap %llu bytes "
+   "in /dev/zero at [%p]: '%s'\n",
+   (unsigned long long)mcfg->memseg[s].len,
+   mcfg->memseg[s].addr, strerror(errno));
if (aslr_enabled() > 0) {
RTE_LOG(ERR, EAL, "It is recommended to "
"disable ASLR in the kernel "

[dpdk-dev] [PATCH] ethdev: support PCI domains

[Thomas Monjalon]

2016-07-22 18:56, Sinan Kaya:
> On 7/22/2016 5:12 PM, Stephen Hemminger wrote:
> > On Fri, 22 Jul 2016 11:34:10 -0400
> > Sinan Kaya  wrote:
> > 
> >> The current code is enumerating devices based on bus, device and function
> >> pairs. This does not work well for architectures with multiple PCI
> >> segments/domains. Multiple PCI devices will have the same BDF value but
> >> different segment numbers (01:01:01.0 and 02:01:01.0) for instance.
> >>
> >> Adding segment numbers to device naming so that we can uniquely identify
> >> devices.
> >>
> >> Signed-off-by: Sinan Kaya 
> > 
> > I ran into this yes. There is a small risk of breaking some application that
> > assumed something about names though.
> > 
> > Acked-by: Stephen Hemminger 
> > 
> 
> Thanks, hopefully the change is minor and can be contained until next release.

It is part of the EAL rework.
The function has been moved in EAL and includes the PCI domain:
http://dpdk.org/commit/affe1cdc

[dpdk-dev] [PATCH] pci:fix missing free

[Thomas Monjalon]

2016-09-30 17:27, David Marchand:
> On Fri, Sep 30, 2016 at 5:19 PM, David Marchand
>  wrote:
> > Hello,
> >
> > On Thu, Sep 29, 2016 at 3:41 AM, Yangchao Zhou  
> > wrote:
> >> Signed-off-by: Yangchao Zhou 
> >
> > For the title, how about:
> > pci: fix memory leak when detaching devices
> >
> > Afaics the fixes tag would be :
> >
> > Fixes: dbe6b4b61b0e ("pci: probe or close device")
> >
> > + CC stable
> 
> Thomas, can you apply this with my ack ?
> Thanks.

Applied, thanks

[dpdk-dev] [PATCH] ethdev: support PCI domains

[Sinan Kaya]

On 10/4/2016 4:15 AM, Thomas Monjalon wrote:
> 2016-07-22 18:56, Sinan Kaya:
>> On 7/22/2016 5:12 PM, Stephen Hemminger wrote:
>>> On Fri, 22 Jul 2016 11:34:10 -0400
>>> Sinan Kaya  wrote:
>>>
 The current code is enumerating devices based on bus, device and function
 pairs. This does not work well for architectures with multiple PCI
 segments/domains. Multiple PCI devices will have the same BDF value but
 different segment numbers (01:01:01.0 and 02:01:01.0) for instance.

 Adding segment numbers to device naming so that we can uniquely identify
 devices.

 Signed-off-by: Sinan Kaya 
>>>
>>> I ran into this yes. There is a small risk of breaking some application that
>>> assumed something about names though.
>>>
>>> Acked-by: Stephen Hemminger 
>>>
>>
>> Thanks, hopefully the change is minor and can be contained until next 
>> release.
> 
> It is part of the EAL rework.
> The function has been moved in EAL and includes the PCI domain:
>   http://dpdk.org/commit/affe1cdc
> 

Thanks for taking care of it.

-- 
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm 
Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux 
Foundation Collaborative Project.

[dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if primary is missing tailqs

[Jean Tourrilhes]

On Tue, Oct 04, 2016 at 02:11:39PM +0100, Sergio Gonzalez Monroy wrote:
> Hi Jean,
> 
> As with your other patch, commit title needs fixing and also missing
> Signed-off-by line.

I'll do that, no worries...

> I might be missing something here so bear with me.

Yes, I know I was terse. Sorry.

> The case you are trying to fix is, as an example, when your secondary app is
> using LPM but your primary is not.
> So basically with this patch, you are removing the tailq for LPM on
> secondary and continuing as normal, is that the case?

The secondary can't use tailq types that the primary does not
have, because they are shared across the shared memory.

What happens is that the primary and secondary did not compile
in the same list of tailq. See previous e-mail :
http://dpdk.org/ml/archives/dev/2016-September/047329.html
The reason it's happening is that the secondary was not
compiled with the DPDK build system, but with the build system of the
application (in this case, Snort). Oubviously, porting the application
to the DPDK build system is not practical, so we need to live with
this case.
The build system of the application does not have all the
subtelties of the DPDK build system, and ends up including *all* the
constructors, wether they are used or not in the code. Moreover, they
are included in a different order. Actually, by default the builds
include no constructors at all (which is a big fail), so the library
needs to be included with --whole-archive (see Snort DPDK
instructions).

> I am not convinced about this approach.

I agree that the whole constructor approach is flaky and my
patch is only a band aid. Constructors should be entirely removed
IMHO, and a more deterministic init method should be used instead of
depending on linker magic.
Note that the other constructors happen to work right in my
case, but that's probably pure luck. The list of mempool constructors
happen to be the same and in the same order (order matters for mempool
constructors). The app is not using spinlock, hash, crc and acl, so
I did not look if the lists did match.

> I guess the assumption here is that all the TAILQ infrastructure works even
> when the tailq list itself is NULL?

If tailq are not used in the primary, I assume it would work.

> I say assumption because I don't have an easy way to trigger this use case
> with default DPDK sample/test apps.

I know. I'll see what I can do to release the code.

> What about letting the secondary process create a tailq if it doesn't
> exists?

Primary owns the shared memory, and I don't see how primary
could handle an unknown tailq. Secondary can't do much without
primary. So, I don't see how adding the missing tailqs helps.

> We would need to lock protect at least the register function to avoid race
> conditions when having multiple secondary processes.
> 
> David, what do you think?
> 
> Sergio

Regards,

Jean

[dpdk-dev] [PATCH v4] drivers/net:new PMD using tun/tap host interface

[Keith Wiles]

The rte_eth_tap.c PMD creates a device using TUN/TAP interfaces
on the local host. The PMD allows for DPDK and the host to
communicate using a raw device interface on the host and in
the DPDK application. The device created is a Tap device with
a L2 packet header.

v4 - merge with latest driver changes
v3 - fix includes by removing ifdef for other type besides Linux.
 Fix the copyright notice in the Makefile
v2 - merge all of the patches into one patch.
 Fix a typo on naming the tap device.
 Update the maintainers list

Signed-off-by: Keith Wiles 
---
 MAINTAINERS |   5 +
 config/common_linuxapp  |   2 +
 doc/guides/nics/tap.rst |  84 
 drivers/net/Makefile|   1 +
 drivers/net/tap/Makefile|  57 +++
 drivers/net/tap/rte_eth_tap.c   | 866 
 drivers/net/tap/rte_pmd_tap_version.map |   4 +
 mk/rte.app.mk   |   1 +
 8 files changed, 1020 insertions(+)
 create mode 100644 doc/guides/nics/tap.rst
 create mode 100644 drivers/net/tap/Makefile
 create mode 100644 drivers/net/tap/rte_eth_tap.c
 create mode 100644 drivers/net/tap/rte_pmd_tap_version.map

diff --git a/MAINTAINERS b/MAINTAINERS
index 7c33ad4..fad74e4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -392,6 +392,11 @@ F: doc/guides/nics/pcap_ring.rst
 F: app/test/test_pmd_ring.c
 F: app/test/test_pmd_ring_perf.c

+Tap PMD
+M: Keith Wiles 
+F: drivers/net/tap
+F: doc/guides/nics/tap.rst
+
 Null Networking PMD
 M: Tetsuya Mukawa 
 F: drivers/net/null/
diff --git a/config/common_linuxapp b/config/common_linuxapp
index 2483dfa..59a2053 100644
--- a/config/common_linuxapp
+++ b/config/common_linuxapp
@@ -44,3 +44,5 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y
 CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y
 CONFIG_RTE_LIBRTE_POWER=y
 CONFIG_RTE_VIRTIO_USER=y
+CONFIG_RTE_LIBRTE_PMD_TAP=y
+CONFIG_RTE_PMD_TAP_MAX_QUEUES=32
diff --git a/doc/guides/nics/tap.rst b/doc/guides/nics/tap.rst
new file mode 100644
index 000..072def8
--- /dev/null
+++ b/doc/guides/nics/tap.rst
@@ -0,0 +1,84 @@
+..  BSD LICENSE
+Copyright(c) 2016 Intel Corporation. All rights reserved.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright
+notice, this list of conditions and the following disclaimer in
+the documentation and/or other materials provided with the
+distribution.
+* Neither the name of Intel Corporation nor the names of its
+contributors may be used to endorse or promote products derived
+from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Tun/Tap Poll Mode Driver
+
+
+The rte_eth_tap.c PMD creates a device using TUN/TAP interfaces on the local
+host. The PMD allows for DPDK and the host to communicate using a raw device
+interface on the host and in the DPDK application.
+
+The device created is a TAP device, which sends/receives packet in a raw format
+with a L2 header. The usage for a TAP PMD is for connectivity to the local host
+using a TAP interface. When the TAP PMD is initialized it will create a number
+of tap devices in the host accessed via 'ifconfig -a' or 'ip' command. The
+commands can be used to assign and query the virtual like device.
+
+These TAP interfaces can be used with wireshark or tcpdump or Pktgen-DPDK along
+with being able to be used as a network connection to the DPDK application. The
+method enable one or more interfaces is to use the --vdev=eth_tap option on the
+DPDK application  command line. Each --vdev=eth_tap option give will create an
+interface named dtap0, dtap1, ... and so forth.
+
+.. code-block:: console
+
+   The interfaced name can be changed by adding the iface=foo0
+   e.g. --vedv=eth_tap,iface=foo0 --vdev=eth_tap,iface=foo1, ...
+
+.. code-block:: console
+
+   Also the 

[dpdk-dev] [PATCH v11 00/24] Introducing rte_driver/rte_device generalization

[Thomas Monjalon]

2016-10-04 12:21, Shreyansh Jain:
> Hi Thomas,
> 
> On Monday 03 October 2016 07:58 PM, Thomas Monjalon wrote:
> > Applied, thanks everybody for the great (re)work!
> 
> Thanks!
> 
> >
> > 2016-09-20 18:11, Shreyansh Jain:
> >> Future Work/Pending:
> >> ===
> >>  - Presently eth_driver, rte_eth_dev are not aligned to the rte_driver/
> >>rte_device model. eth_driver still is a PCI specific entity. This
> >>has been highlighted by comments from Ferruh in [9].
> >>  - Some variables, like drv_name (as highlighted by Ferruh), are getting
> >>duplicated across rte_xxx_driver/device and rte_driver/device.
> 
> Both the above are already part of my todo list.
> 
> > What about those pending work?
> >
> > I would add more remaining issues:
> > - probe/remove naming could be applied to vdev for consistency
> 
> Is that for uniformity reasons? I still feel 'probe/remove' are not 
> appropriate for a virtual device. init/deinit are more appropriate. As 
> for PCI, probe/remove are standard parlance and hence suit it better 
> than init/deinit.

PCI probe is "scan + checks + init".
The vdev requires "args parsing + checks + init".
The device will not be initialized if checks fail,
e.g. missing support or name conflict.
I think it could fit in "probe" rather than "init".

The remove word looks appropriate in both cases.

> Nevertheless, uniform naming convention can have its benefits - ease of 
> code understanding being one.

Yes that's the other pro for "probe/remove".

> Change is simple once we come to a conclusion.
> 
> > - rte_eal_device_insert must be called in vdev
> 
> Ok.
> 
> > - REGISTER macros should be prefixed with RTE_
> 
> That would include:
>   DRIVER_REGISTER_VDEV
>   DRIVER_REGISTER_PCI_TABLE
>   DRIVER_REGISTER_PCI
> 
> I will publish a patch soon. This would be fairly straightforward change.
> 
> > - Some functions in EAL does not need eal_ in their prefix:
> > rte_eal_pci_   -> rte_pci_
> > rte_eal_dev_   -> rte_dev_
> > rte_eal_vdev_  -> rte_vdev_
> > rte_eal_driver -> rte_drv_
> > rte_eal_vdrv   -> rte_vdrv_
> >
> >
> 
> It can be merged with changes for:
>   - drv_name
>   - EAL_ before _REGISTER_ macros
>   - eth_driver => rte_driver naming

Good.
Could you make it this week, please?

[dpdk-dev] [PATCH 1/1 v2] eal: Fix misleading error messages, errno can't be trusted.

[Sergio Gonzalez Monroy]

On 03/10/2016 21:46, Thomas Monjalon wrote:
> 2016-10-03 08:55, Jean Tourrilhes:
>> On Mon, Oct 03, 2016 at 02:25:40PM +0100, Sergio Gonzalez Monroy wrote:
>>> Hi Jean,
>>>
>>> There are some format issues with the patch:
>>>
>>> You can run scripts/check-git-log.sh to check them:
>>> Wrong headline format:
>>>  eal: Fix misleading error messages, errno can't be trusted.
>>> Wrong headline uppercase:
>>>  eal: Fix misleading error messages, errno can't be trusted.
>>> Missing 'Fixes' tag:
>>>  eal: Fix misleading error messages, errno can't be trusted.
>>>
>>> The script's output highlights the different issues.
>>  SOrry about that, I casually read the page on
>> http://dpdk.org/dev, but obviously I need to look at it again.
> No problem. This guide is more oriented towards regular contributors.
> You come with a bug and its fix, we can make some effort to format
> the patch :)
>
> The title could be "mem: fix hugepage mapping error messages"
>
>>> On 21/09/2016 22:10, Jean Tourrilhes wrote:
 @@ -263,9 +264,16 @@ rte_eal_config_reattach(void)
mem_config = (struct rte_mem_config *) mmap(rte_mem_cfg_addr,
sizeof(*mem_config), PROT_READ | PROT_WRITE, MAP_SHARED,
mem_cfg_fd, 0);
 +  if (mem_config == MAP_FAILED || mem_config != rte_mem_cfg_addr) {
 +  if (mem_config != MAP_FAILED)
 +  /* errno is stale, don't use */
 +  rte_panic("Cannot mmap memory for rte_config at [%p], 
 got [%p] - please use '--base-virtaddr' option\n",
 +rte_mem_cfg_addr, mem_config);
 +  else
 +  rte_panic("Cannot mmap memory for rte_config! error %i 
 (%s)\n",
 +errno, strerror(errno));
 +  }
close(mem_cfg_fd);
 -  if (mem_config == MAP_FAILED || mem_config != rte_mem_cfg_addr)
 -  rte_panic("Cannot mmap memory for rte_config\n");
>>> NIT but any reason you moved the check before closing the file descriptor?
>>> (not that it matters with current code as we panic anyway)
>>  "close()" may change "errno" according to its man page.
> Sergio, do you have more comments?

Nop.

> Should we wait another version or is it OK?
> Maybe you'd prefer to rework it yourself?

AFAICS it really is just the commit message formatting, so maybe you can 
fix that when applying it?

Sergio

[dpdk-dev] [PATCH] hash: fix incorrect free slot check

[Pablo de Lara]

In function rte_hash_cuckoo_insert_mw_tm, while looking for
an empty slot, only the first entry in the bucket was being checked,
as key_idx array was not being iterated.

Fixes: 5fc74c2e146d ("hash: check if slot is empty with key index")

Reported-by: Bruce Richardson 
Signed-off-by: Pablo de Lara 
---
 lib/librte_hash/rte_cuckoo_hash_x86.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_hash/rte_cuckoo_hash_x86.h 
b/lib/librte_hash/rte_cuckoo_hash_x86.h
index e16d69c..7ffa56f 100644
--- a/lib/librte_hash/rte_cuckoo_hash_x86.h
+++ b/lib/librte_hash/rte_cuckoo_hash_x86.h
@@ -53,7 +53,7 @@ rte_hash_cuckoo_insert_mw_tm(struct rte_hash_bucket *prim_bkt,
*/
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
/* Check if slot is available */
-   if (likely(prim_bkt->key_idx == EMPTY_SLOT)) {
+   if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) 
{
prim_bkt->signatures[i].current = sig;
prim_bkt->signatures[i].alt = alt_hash;
prim_bkt->key_idx[i] = new_idx;
-- 
2.7.4

[dpdk-dev] [PATCH v4 0/4] Cuckoo hash enhancements

[De Lara Guarch, Pablo]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of De Lara Guarch,
> Pablo
> Sent: Monday, October 03, 2016 11:51 PM
> To: Richardson, Bruce
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4 0/4] Cuckoo hash enhancements
> 
> Hi Bruce,
> 
> > -Original Message-
> > From: Richardson, Bruce
> > Sent: Monday, October 03, 2016 2:59 AM
> > To: De Lara Guarch, Pablo
> > Cc: dev at dpdk.org
> > Subject: Re: [PATCH v4 0/4] Cuckoo hash enhancements
> >
> > On Fri, Sep 30, 2016 at 08:38:52AM +0100, Pablo de Lara wrote:
> > > This patchset improves lookup performance on the current hash library
> > > by changing the existing lookup bulk pipeline, with an improved pipeline,
> > > based on a loop-and-jump model, instead of the current 4-stage 2-entry
> > pipeline.
> > > Also, x86 vectorized intrinsics are used to improve performance when
> > comparing signatures.
> > >
> > > First patch reorganizes the order of the hash structure.
> > > The structure takes more than one 64-byte cache line, but not all
> > > the fields are used in the lookup operation (the most common operation).
> > > Therefore, all these fields have been moved to the first part of the
> structure,
> > > so they all fit in one cache line, improving slightly the performance in
> some
> > > scenarios.
> > >
> > > Second patch modifies the order of the bucket structure.
> > > Currently, the buckets store all the signatures together (current and
> > alternative).
> > > In order to be able to perform a vectorized signature comparison,
> > > all current signatures have to be together, so the order of the bucket has
> > been changed,
> > > having separated all the current signatures from the alternative
> signatures.
> > >
> > > Third patch introduces x86 vectorized intrinsics.
> > > When performing a lookup bulk operation, all current signatures in a
> bucket
> > > are compared against the signature of the key being looked up.
> > > Now that they all are together, a vectorized comparison can be
> performed,
> > > which takes less instructions to be carried out.
> > > In case of having a machine with AVX2, number of entries per bucket are
> > > increased from 4 to 8, as AVX2 allows comparing two 256-bit values, with
> > 8x32-bit integers,
> > > which are the 8 signatures on the bucket.
> > >
> > > Fourth (and last) patch modifies the current pipeline of the lookup bulk
> > function.
> > > The new pipeline is based on a loop-and-jump model. The two key
> > improvements are:
> > >
> > > - Better prefetching: in this case, first 4 keys to be looked up are
> prefetched,
> > >   and after that, the rest of the keys are prefetched at the time the
> > calculation
> > >   of the signatures are being performed. This gives more time for the CPU
> to
> > >   prefetch the data requesting before actually need it, which result in 
> > > less
> > >   cache misses and therefore, higher throughput.
> > >
> > > - Lower performance penalty when using fallback: the lookup bulk
> > algorithm
> > >   assumes that most times there will not be a collision in a bucket, but 
> > > it
> > might
> > >   happen that two or more signatures are equal, which means that more
> > than one
> > >   key comparison might be necessary. In that case, only the key of the 
> > > first
> > hit is prefetched,
> > >   like in the current implementation. The difference now is that if this
> > comparison
> > >   results in a miss, the information of the other keys to be compared has
> > been stored,
> > >   unlike the current implementation, which needs to perform an entire
> > simple lookup again.
> > >
> > > Changes in v4:
> > > - Reordered hash structure, so alt signature is at the start
> > >   of the next cache line, and explain in the commit message
> > >   why it has been moved
> > > - Reordered hash structure, so name field is on top of the structure,
> > >   leaving all the fields used in lookup in the next cache line
> > >   (instead of the first cache line)
> > >
> > > Changes in v3:
> > > - Corrected the cover letter (wrong number of patches)
> > >
> > > Changes in v2:
> > > - Increased entries per bucket from 4 to 8 for all cases,
> > >   so it is not architecture dependent any longer.
> > > - Replaced compile-time signature comparison function election
> > >   with run-time election, so best optimization available
> > >   will be used from a single binary.
> > > - Reordered the hash structure, so all the fields used by lookup
> > >   are in the same cache line (first).
> > >
> > > Byron Marohn (3):
> > >   hash: reorganize bucket structure
> > >   hash: add vectorized comparison
> > >   hash: modify lookup bulk pipeline
> > >
> >
> > Hi,
> >
> > Firstly, checkpatches is reporting some style errors in these patches.
> >
> > Secondly, when I run the "hash_multiwriter_autotest" I get what I assume
> to
> > be
> > an error after applying this patchset. Before this set is applied, running
> > that test shows the cycles per insert with/without 

[dpdk-dev] [PATCH v4 0/4] Cuckoo hash enhancements

[De Lara Guarch, Pablo]

Hi Bruce,

> -Original Message-
> From: Richardson, Bruce
> Sent: Monday, October 03, 2016 2:59 AM
> To: De Lara Guarch, Pablo
> Cc: dev at dpdk.org
> Subject: Re: [PATCH v4 0/4] Cuckoo hash enhancements
> 
> On Fri, Sep 30, 2016 at 08:38:52AM +0100, Pablo de Lara wrote:
> > This patchset improves lookup performance on the current hash library
> > by changing the existing lookup bulk pipeline, with an improved pipeline,
> > based on a loop-and-jump model, instead of the current 4-stage 2-entry
> pipeline.
> > Also, x86 vectorized intrinsics are used to improve performance when
> comparing signatures.
> >
> > First patch reorganizes the order of the hash structure.
> > The structure takes more than one 64-byte cache line, but not all
> > the fields are used in the lookup operation (the most common operation).
> > Therefore, all these fields have been moved to the first part of the 
> > structure,
> > so they all fit in one cache line, improving slightly the performance in 
> > some
> > scenarios.
> >
> > Second patch modifies the order of the bucket structure.
> > Currently, the buckets store all the signatures together (current and
> alternative).
> > In order to be able to perform a vectorized signature comparison,
> > all current signatures have to be together, so the order of the bucket has
> been changed,
> > having separated all the current signatures from the alternative signatures.
> >
> > Third patch introduces x86 vectorized intrinsics.
> > When performing a lookup bulk operation, all current signatures in a bucket
> > are compared against the signature of the key being looked up.
> > Now that they all are together, a vectorized comparison can be performed,
> > which takes less instructions to be carried out.
> > In case of having a machine with AVX2, number of entries per bucket are
> > increased from 4 to 8, as AVX2 allows comparing two 256-bit values, with
> 8x32-bit integers,
> > which are the 8 signatures on the bucket.
> >
> > Fourth (and last) patch modifies the current pipeline of the lookup bulk
> function.
> > The new pipeline is based on a loop-and-jump model. The two key
> improvements are:
> >
> > - Better prefetching: in this case, first 4 keys to be looked up are 
> > prefetched,
> >   and after that, the rest of the keys are prefetched at the time the
> calculation
> >   of the signatures are being performed. This gives more time for the CPU to
> >   prefetch the data requesting before actually need it, which result in less
> >   cache misses and therefore, higher throughput.
> >
> > - Lower performance penalty when using fallback: the lookup bulk
> algorithm
> >   assumes that most times there will not be a collision in a bucket, but it
> might
> >   happen that two or more signatures are equal, which means that more
> than one
> >   key comparison might be necessary. In that case, only the key of the first
> hit is prefetched,
> >   like in the current implementation. The difference now is that if this
> comparison
> >   results in a miss, the information of the other keys to be compared has
> been stored,
> >   unlike the current implementation, which needs to perform an entire
> simple lookup again.
> >
> > Changes in v4:
> > - Reordered hash structure, so alt signature is at the start
> >   of the next cache line, and explain in the commit message
> >   why it has been moved
> > - Reordered hash structure, so name field is on top of the structure,
> >   leaving all the fields used in lookup in the next cache line
> >   (instead of the first cache line)
> >
> > Changes in v3:
> > - Corrected the cover letter (wrong number of patches)
> >
> > Changes in v2:
> > - Increased entries per bucket from 4 to 8 for all cases,
> >   so it is not architecture dependent any longer.
> > - Replaced compile-time signature comparison function election
> >   with run-time election, so best optimization available
> >   will be used from a single binary.
> > - Reordered the hash structure, so all the fields used by lookup
> >   are in the same cache line (first).
> >
> > Byron Marohn (3):
> >   hash: reorganize bucket structure
> >   hash: add vectorized comparison
> >   hash: modify lookup bulk pipeline
> >
> 
> Hi,
> 
> Firstly, checkpatches is reporting some style errors in these patches.
> 
> Secondly, when I run the "hash_multiwriter_autotest" I get what I assume to
> be
> an error after applying this patchset. Before this set is applied, running
> that test shows the cycles per insert with/without lock elision. Now, though
> I'm getting an error about a key being dropped or failing to insert in the 
> lock
> elision case, e.g.
> 
>   Core #2 inserting 1572864: 0 - 1,572,864
>   key 1497087 is lost
>   1 key lost
> 
> I've run the test a number of times, and there is a single key lost each time.
> Please check on this, is it expected or is it a problem?

I am seeing that error even without the patchset. I am still investigating it,
but using "git bisect" looks like the problem is in