Re: cannot trx when use testpmd

[Ferruh Yigit]

On 4/12/2024 8:51 AM, hao wang wrote:
> I have a pmd that I am upstreaming to DPDK and currently testing. When I
> use testpmd to test my pmd, I found that my pmd could trx in DPDK 23.07,
> but it cannot in DPDK 23.11. After comparing the atlantic pmd between
> DPDK 23.07 and 23.11, I see nearly no changes. I see the testpmd
> difference between DPDK 23.07 and DPDK 23.11 and think these changes may
> not cause the problem.
> 
> But when I use pktgen to test the pmd, it's ok.
> 
> Any suggestions?
> 
> 

Hi Hao,

I am not aware of any known issues in testpmd that may cause this
specific problem.
And without more details it is not possible to know what the issue is.

What is the PMD you are working on? Are you planning to upstream it for
v24.07 release?


Also I assume you are checking 'atlantic' driver code only as sample, is
this correct?


Thanks,
ferruh

Re: flow create with queue range not working

[Ferruh Yigit]

On 3/11/2024 10:32 AM, Vajith Raghman wrote:
> Hi Team,
> 
> We are validating the fdir with test-pmd tool.  We are getting below
> error while trying to add the flow create rule for the same.
> 
> Syntax:
> 
> testpmd>flow create 0ingress pattern eth / ipv4 src *is*ipv4
> dst *is*/tcp src *is* dst *is*/end
> actions rss queues 23end /end
> 
> rule:
> 
> flow create 0 ingress pattern eth / ipv4 src is 20.20.20.2 dst is
> 20.20.20.4 / tcp src is 80 dst is 1501 / end actions rss queues 0 1 2 3
> end / end
> 
> port_flow_complain(): Caught PMD error type 16 (specific action): cause:
> 0x7fff9495c648, RSS types must be empty while configuring queue region:
> Operation not supported
> 
> Please help us on this.
> 

I guess the error from i40e, cc'ing relevant maintainers.

Re: help

[Ferruh Yigit]

On 7/21/2023 12:39 PM, Igor de Paula wrote:
> I am trying to use virtio_user for an interface with the
> kernel: https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html 
> <https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html>
> I think this requires IOVA as va.
>

I am not sure if virtio-user has IOVA as VA requirement, cc'ed Chenbo,
he may know better.

Meanwhile can you give a try to 'enable_unsafe_noiommu_mode' and
'--iova-mode=pa'?


> It does work with Intel host and IOMMU
> enabled. Part of the negotiation when setting it up is getting the IOMMU
> number so I thought it has to have IOMMU.
>

Yes, issue looks like related to the IOMMU, and it may be either related
to HW support, or ESXi iommu driver support, we will check using below
information you provided.

> I tried disabling IOMMU and enabling enable_unsafe_noiommu flag but
> again, that didn't work.
> ESXI version - VMware ESXi, 7.0.0, 16324942
> AMD:  AMD EPYC 7452 32-Core Processo
> 
> On an Intel host which worked: Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
> Regarding the logs I will try and attach it soon.
> 

Thanks for info, waiting for logs.

> 
> On Fri, Jul 21, 2023 at 12:21 PM Ferruh Yigit  <mailto:ferruh.yi...@amd.com>> wrote:
> 
> Hi Igor,
> 
> VM doesn't have IOMMU, and vmxnet3 requires PA mode, for this can you
> please try with:
> - enable 'enable_unsafe_noiommu_mode' flag
> - Force PA mode via '--iova-mode=pa' eal parameter
> 
> 
> Also to be able to figure out AMD IOMMU support level, can you please
> provide:
> - AMD part number
> - ESXi/hypervisor version
> - ESXi & VM kernel logs
> 
> 
> Thanks,
> Ferruh
> 
> On 7/20/2023 5:21 PM, Jochen Behrens wrote:
> > +Ronak from the ESX team
> >
> >  
> >
> >  
> >
> > In our usage, we do set amd_iommu=off in the boot command line from
> > grub. (Or intel_iommu=off for Intel processors.)
> >
> >  
> >
> >     Jochen
> >
> >  
> >
> > *From: *Thomas Monjalon  <mailto:tho...@monjalon.net>>
> > *Date: *Thursday, July 20, 2023 at 6:00 AM
> > *To: *Igor de Paula mailto:igord...@gmail.com>>
> > *Cc: *users@dpdk.org <mailto:users@dpdk.org>  <mailto:users@dpdk.org>>, Jochen Behrens
> > mailto:jbehr...@vmware.com>>, Nipun Gupta
> mailto:nipun.gu...@amd.com>>, Nikhil Agarwal
> > mailto:nikhil.agar...@amd.com>>, Ferruh
> Yigit mailto:ferruh.yi...@amd.com>>
> > *Subject: *Re: help
> >
> > !! External Email
> >
> > +Cc some AMD maintainers, they can have an idea about IOMMU settings.
> >
> >
> > 20/07/2023 14:44, Igor de Paula:
> >> I have enabled it in the host and in the BIOS for AMD...
> >> In the Bios I changed to amd_iommu=on and in the host it's the
> same for
> >> either.
> >>
> >> On Thu, Jul 20, 2023 at 1:31 PM Thomas Monjalon
> mailto:tho...@monjalon.net>> wrote:
> >>
> >> > 20/07/2023 11:35, Igor de Paula:
> >> > > The weird thing is that it only happens when I am using a
> host with an
> >> > AMD
> >> > > processor. It doesn't happen when I use a host with an Intel
> processor.
> >> >
> >> > So it's probably a matter of BIOS settings for the IOMMU?
> >> >
> >> >
> >> > > On Thu, Jul 20, 2023 at 10:32 AM Thomas Monjalon
> mailto:tho...@monjalon.net>>
> >> > > wrote:
> >> > >
> >> > > > +Cc the vmxnet3 maintainer.
> >> > > >
> >> > > > Please Jochen, do you have an idea what's wrong below?
> >> > > >
> >> > > >
> >> > > > 20/07/2023 11:25, Igor de Paula:
> >> > > > > This is because it can't negotiate the IOMMU type with
> any port.
> >> > > > >
> >> > > > > On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon
> mailto:tho...@monjalon.net>
> >> > >
> >> > > > wrote:
> >> > > > >
> >> > > > > > Hello,
> >> > > > > >
> >> > > > > > The first error is &

Re: help

[Ferruh Yigit]

Hi Igor,

VM doesn't have IOMMU, and vmxnet3 requires PA mode, for this can you
please try with:
- enable 'enable_unsafe_noiommu_mode' flag
- Force PA mode via '--iova-mode=pa' eal parameter


Also to be able to figure out AMD IOMMU support level, can you please
provide:
- AMD part number
- ESXi/hypervisor version
- ESXi & VM kernel logs


Thanks,
Ferruh

On 7/20/2023 5:21 PM, Jochen Behrens wrote:
> +Ronak from the ESX team
> 
>  
> 
>  
> 
> In our usage, we do set amd_iommu=off in the boot command line from
> grub. (Or intel_iommu=off for Intel processors.)
> 
>  
> 
>     Jochen
> 
>  
> 
> *From: *Thomas Monjalon 
> *Date: *Thursday, July 20, 2023 at 6:00 AM
> *To: *Igor de Paula 
> *Cc: *users@dpdk.org , Jochen Behrens
> , Nipun Gupta , Nikhil Agarwal
> , Ferruh Yigit 
> *Subject: *Re: help
> 
> !! External Email
> 
> +Cc some AMD maintainers, they can have an idea about IOMMU settings.
> 
> 
> 20/07/2023 14:44, Igor de Paula:
>> I have enabled it in the host and in the BIOS for AMD...
>> In the Bios I changed to amd_iommu=on and in the host it's the same for
>> either.
>>
>> On Thu, Jul 20, 2023 at 1:31 PM Thomas Monjalon  wrote:
>>
>> > 20/07/2023 11:35, Igor de Paula:
>> > > The weird thing is that it only happens when I am using a host with an
>> > AMD
>> > > processor. It doesn't happen when I use a host with an Intel processor.
>> >
>> > So it's probably a matter of BIOS settings for the IOMMU?
>> >
>> >
>> > > On Thu, Jul 20, 2023 at 10:32 AM Thomas Monjalon 
>> > > wrote:
>> > >
>> > > > +Cc the vmxnet3 maintainer.
>> > > >
>> > > > Please Jochen, do you have an idea what's wrong below?
>> > > >
>> > > >
>> > > > 20/07/2023 11:25, Igor de Paula:
>> > > > > This is because it can't negotiate the IOMMU type with any port.
>> > > > >
>> > > > > On Thu, Jul 20, 2023 at 5:08 AM Thomas Monjalon > > >
>> > > > wrote:
>> > > > >
>> > > > > > Hello,
>> > > > > >
>> > > > > > The first error is "Cause: Error: number of ports must be even"
>> > > > > >
>> > > > > >
>> > > > > > 03/05/2023 18:13, Igor de Paula:
>> > > > > > > I am running a VM inside a VMWARE server (vSphere).
>> > > > > > > My goal it to set up DPDK with two HW ports, and set up a
>> > > > virtio_user to
>> > > > > > > interact with the kernel stack.
>> > > > > > > In another app I have it working but instead of virtio_user I am
>> > > > running
>> > > > > > > KNI, it works in IOVA-PA mode.
>> > > > > > > I am looking to replace the KNI.
>> > > > > > >
>> > > > > > > When I try to set up virtio_user port as in the doc:
>> > > > > > >
>> > > > > >
>> > > >
>> > https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path
>> >  
>> > <https://doc.dpdk.org/guides/howto/virtio_user_as_exception_path.html#virtio-user-as-exception-path>
>> > > > > > > I get a error it can't run in PA mode.
>> > > > > > >
>> > > > > > >
>> > > > > > > When I try to run as VA mode from a parameter, I get the
>> > following
>> > > > > > errors:
>> > > > > > > EAL: lib.eal log level changed from info to debug
>> > > > > > > EAL: Detected lcore 0 as core 0 on socket 0
>> > > > > > > EAL: Detected lcore 1 as core 0 on socket 0
>> > > > > > > EAL: Support maximum 128 logical core(s) by configuration.
>> > > > > > > EAL: Detected 2 lcore(s)
>> > > > > > > EAL: Detected 1 NUMA nodes
>> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21.3'
>> > > > > > > EAL: Checking presence of .so 'librte_eal.so.21'
>> > > > > > > EAL: Checking presence of .so 'librte_eal.so'
>> > > > > > > EAL: Detected static linkage of DPDK
>> > > > > > > EAL: Ask a virtual area of 0x7000 bytes
>> > 

Re: DPDK 22.11 - How to fix memory leak for KNI - How to debug

[Ferruh Yigit]

30                    
> 2.59% [|]
> MBUF_POOL                      302            10,097                    
> 2.90% [|]
> MBUF_POOL                      88             10,311                    
> 0.85% [|]
> MBUF_POOL                      305            10,094                    
> 2.93% [|]
> MBUF_POOL                      88             10,311                    
> 0.85% [|]
> MBUF_POOL                      290            10,109                    
> 2.79% [|....]
> MBUF_POOL                      84             10,315                    
> 0.81% [|]
> MBUF_POOL                      85             10,314                    
> 0.82% [|]
> MBUF_POOL                      291            10,108                    
> 2.80% [|]
> MBUF_POOL                      303            10,096                    
> 2.91% [|]
> MBUF_POOL                      92             10,307                    
> 0.88% [|]
> 
> 
> Best regards.
> 
> 
> Ferruh Yigit mailto:ferruh.yi...@amd.com>>, 18
> May 2023 Per, 17:56 tarihinde şunu yazdı:
> 
> On 5/18/2023 9:14 AM, Yasin CANER wrote:
> > Hello Ferruh,
> >
> > Thanks for your kind response. Also thanks to Stephen.
> >
> > Even if 1 packet is consumed from the kernel , each time rx_kni
> > allocates another 32 units. After a while all mempool is used in
> alloc_q
> > from kni. there is not any room for it.
> >
> 
> What you described continues until 'alloc_q' is full, by default fifo
> length is 1024 (KNI_FIFO_COUNT_MAX), do you allocate less mbuf in your
> mempool?
> 
> You can consider either increasing mempool size, or decreasing 'alloc_q'
> fifo length, but reducing fifo size may cause performance issues so you
> need to evaluate that option.
> 
> > Do you think my mistake is using one and common mempool usage both kni
> > and eth?
> >
> 
> Using same mempool for both is fine.
> 
> > If it needs a separate mempool , i'd like to note in docs.
> >
> > Best regards.
> >
> > Ferruh Yigit mailto:ferruh.yi...@amd.com>
> <mailto:ferruh.yi...@amd.com <mailto:ferruh.yi...@amd.com>>>, 17
> > May 2023 Çar, 20:53 tarihinde şunu yazdı:
> >
> >     On 5/9/2023 12:13 PM, Yasin CANER wrote:
> >     > Hello,
> >     >
> >     > I draw a flow via asciiflow to explain myself better.
> Problem is after
> >     > transmitting packets(mbufs) , it never puts in the
> kni->free_q to back
> >     > to the original pool. Each cycle, it allocates another 32
> units that
> >     > cause leaks. Or I am missing something.
> >     >
> >     > I already tried the rte_eth_tx_done_cleanup() function but it
> >     didn't fix
> >     > anything.
> >     >
> >     > I am working on a patch to fix this issue but I am not sure
> if there
> >     > is another way.
> >     >
> >     > Best regards.
> >     >
> >     > https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>
> >     <https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>>
> >     > <https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>
> >     <https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>>>
> >     >
> >     >
> >     > unsigned
> >     > rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs,
> >     unsigned
> >     > int num)
> >     > {
> >     > unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
> >     >
> >     > /* If buffers removed, allocate mbufs and then put them into
> >     alloc_q */
> >     > /* Question, how to test buffers is removed or not?*/
> >     > if (ret)
> >     >     kni_allocate_mbufs(kni);
> >     >
> >     > return ret;
> >     > }
> >     >
> >
> >     Selam Yasin,
> >
> >
> >     You can expect 'kni->alloc_q' fif

Re: DPDK 22.11 - How to fix memory leak for KNI - How to debug

[Ferruh Yigit]

On 5/18/2023 9:14 AM, Yasin CANER wrote:
> Hello Ferruh,
> 
> Thanks for your kind response. Also thanks to Stephen.
> 
> Even if 1 packet is consumed from the kernel , each time rx_kni
> allocates another 32 units. After a while all mempool is used in alloc_q
> from kni. there is not any room for it.
> 

What you described continues until 'alloc_q' is full, by default fifo
length is 1024 (KNI_FIFO_COUNT_MAX), do you allocate less mbuf in your
mempool?

You can consider either increasing mempool size, or decreasing 'alloc_q'
fifo length, but reducing fifo size may cause performance issues so you
need to evaluate that option.

> Do you think my mistake is using one and common mempool usage both kni
> and eth?
> 

Using same mempool for both is fine.

> If it needs a separate mempool , i'd like to note in docs.
> 
> Best regards.
> 
> Ferruh Yigit mailto:ferruh.yi...@amd.com>>, 17
> May 2023 Çar, 20:53 tarihinde şunu yazdı:
> 
> On 5/9/2023 12:13 PM, Yasin CANER wrote:
> > Hello,
> >
> > I draw a flow via asciiflow to explain myself better. Problem is after
> > transmitting packets(mbufs) , it never puts in the kni->free_q to back
> > to the original pool. Each cycle, it allocates another 32 units that
> > cause leaks. Or I am missing something.
> >
> > I already tried the rte_eth_tx_done_cleanup() function but it
> didn't fix
> > anything.
> >
> > I am working on a patch to fix this issue but I am not sure if there
> > is another way.
> >
> > Best regards.
> >
> > https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>
> > <https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> <https://pastebin.ubuntu.com/p/s4h5psqtgZ/>>
> >
> >
> > unsigned
> > rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs,
> unsigned
> > int num)
> > {
> > unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
> >
> > /* If buffers removed, allocate mbufs and then put them into
> alloc_q */
> > /* Question, how to test buffers is removed or not?*/
> > if (ret)
> >     kni_allocate_mbufs(kni);
> >
> > return ret;
> > }
> >
> 
> Selam Yasin,
> 
> 
> You can expect 'kni->alloc_q' fifo to be full, this is not a memory
> leak.
> 
> As you pointed out, number of mbufs consumed by kernel from 'alloc_q'
> and number of mbufs added to 'alloc_q' is not equal and this is
> expected.
> 
> Target here is to prevent buffer underflow from kernel perspective, so
> it will always have available mbufs for new packets.
> That is why new mbufs are added to 'alloc_q' at worst same or sometimes
> higher rate than it is consumed.
> 
> You should calculate your mbuf requirement with the assumption that
> 'kni->alloc_q' will be full of mbufs.
> 
> 
> 'kni->alloc_q' is freed when kni is removed.
> Since 'alloc_q' holds physical address of the mbufs, it is a little
> challenging to free them in the userspace, that is why first kernel
> tries to move mbufs to 'kni->free_q' fifo, please check
> 'kni_net_release_fifo_phy()' for it.
> 
> If all moved to 'free_q' fifo, nothing left to in 'alloc_q', but if not,
> userspace frees 'alloc_q' in 'rte_kni_release()', with following call:
> `kni_free_fifo_phy(kni->pktmbuf_pool, kni->alloc_q);`
> 
> 
> I can see you have submitted fixes for this issue, although as I
> explained above I don't think a defect exist, I will review them
> today/tomorrow.
> 
> Regards,
> Ferruh
> 
> 
> > Stephen Hemminger  <mailto:step...@networkplumber.org>
> > <mailto:step...@networkplumber.org
> <mailto:step...@networkplumber.org>>>, 8 May 2023 Pzt, 19:18 tarihinde
> > şunu yazdı:
> >
> >     On Mon, 8 May 2023 09:01:41 +0300
> >     Yasin CANER  <mailto:yasinnca...@gmail.com> <mailto:yasinnca...@gmail.com
> <mailto:yasinnca...@gmail.com>>>
> >     wrote:
> >
> >     > Hello Stephen,
> >     >
> >     > Thank you for response, it helps me a lot. I understand problem
> >     better.
> >     >
> >     > After reading mbuf library (
> 

Re: DPDK 22.11 - How to fix memory leak for KNI - How to debug

[Ferruh Yigit]

On 5/9/2023 12:13 PM, Yasin CANER wrote:
> Hello,
> 
> I draw a flow via asciiflow to explain myself better. Problem is after
> transmitting packets(mbufs) , it never puts in the kni->free_q to back
> to the original pool. Each cycle, it allocates another 32 units that
> cause leaks. Or I am missing something.
> 
> I already tried the rte_eth_tx_done_cleanup() function but it didn't fix
> anything.
> 
> I am working on a patch to fix this issue but I am not sure if there
> is another way.
> 
> Best regards.
> 
> https://pastebin.ubuntu.com/p/s4h5psqtgZ/
> 
> 
> 
> unsigned
> rte_kni_rx_burst(struct rte_kni *kni, struct rte_mbuf **mbufs, unsigned
> int num)
> {
> unsigned int ret = kni_fifo_get(kni->tx_q, (void **)mbufs, num);
> 
> /* If buffers removed, allocate mbufs and then put them into alloc_q */
> /* Question, how to test buffers is removed or not?*/
> if (ret)
>     kni_allocate_mbufs(kni);
> 
> return ret;
> }
> 

Selam Yasin,


You can expect 'kni->alloc_q' fifo to be full, this is not a memory leak.

As you pointed out, number of mbufs consumed by kernel from 'alloc_q'
and number of mbufs added to 'alloc_q' is not equal and this is expected.

Target here is to prevent buffer underflow from kernel perspective, so
it will always have available mbufs for new packets.
That is why new mbufs are added to 'alloc_q' at worst same or sometimes
higher rate than it is consumed.

You should calculate your mbuf requirement with the assumption that
'kni->alloc_q' will be full of mbufs.


'kni->alloc_q' is freed when kni is removed.
Since 'alloc_q' holds physical address of the mbufs, it is a little
challenging to free them in the userspace, that is why first kernel
tries to move mbufs to 'kni->free_q' fifo, please check
'kni_net_release_fifo_phy()' for it.

If all moved to 'free_q' fifo, nothing left to in 'alloc_q', but if not,
userspace frees 'alloc_q' in 'rte_kni_release()', with following call:
`kni_free_fifo_phy(kni->pktmbuf_pool, kni->alloc_q);`


I can see you have submitted fixes for this issue, although as I
explained above I don't think a defect exist, I will review them
today/tomorrow.

Regards,
Ferruh


> Stephen Hemminger  >, 8 May 2023 Pzt, 19:18 tarihinde
> şunu yazdı:
> 
> On Mon, 8 May 2023 09:01:41 +0300
> Yasin CANER mailto:yasinnca...@gmail.com>>
> wrote:
> 
> > Hello Stephen,
> >
> > Thank you for response, it helps me a lot. I understand problem
> better.
> >
> > After reading mbuf library (
> > https://doc.dpdk.org/guides/prog_guide/mempool_lib.html
> )  i
> realized that
> > 31 units allocation memory slot doesn't return to pool!
> 
> If receive burst returns 1 mbuf, the other 31 pointers in the array
> are not valid. They do not point to mbufs.
> 
> > 1 unit mbuf can be freed via rte_pktmbuf_free so it can back to pool.
> >
> > Main problem is that allocation doesn't return to original pool,
> act as
> > used. So, after following rte_pktmbuf_free
> >
> 
>  >
> > function,
> > i realized that there is 2 function to helps to mbufs back to pool.
> >
> > These are rte_mbuf_raw_free
> >
> 
>  >
> >  and rte_pktmbuf_free_seg
> >
> 
>  >.
> > I will focus on them.
> >
> > If there is another suggestion, I will be very pleased.
> >
> > Best regards.
> >
> > Yasin CANER
> > Ulak
> 

Re: Anonymous structs in DPDK

[Ferruh Yigit]

On 12/13/2022 12:51 PM, Antonio Di Bacco wrote:
> I noticed that DPDK include files have a number of anonymous/unnamed struct:
> 
> For example:
> 
> /**
>  * The rte_spinlock_t type.
>  */
> typedef struct {
> volatile int locked; /**< lock status 0 = unlocked, 1 = locked */
> } rte_spinlock_t;
> 
> This choice doesn't allow to use forward declaration. I need forward
> declaration because I'm using a rte_spinlock_t pointer in a C++ class
> and I don't want to include rte_spinlock.h to prevent my application
> to include it as well.
> 
> Is there any reason to use unnamed structures?
> 

Hi Antonio Di,

I don't think there is a specific reason to not use named struct, I
assume that is only because there was no need to have it.

So if you need, you can send a simple patch to convert anonymous struct
to named struct, although I am not clear why you can't include
'rte_spinlock.h' in the file you declare your class.

Cheers,
ferruh

Re: Shared memory between two primary DPDK processes

[Ferruh Yigit]

On 4/8/2022 2:26 PM, Dmitry Kozlyuk wrote:

CAUTION: This message has originated from an External Source. Please use proper 
judgment and caution when opening attachments, clicking links, or responding to 
this email.


2022-04-08 14:31 (UTC+0200), Antonio Di Bacco:

I know that it is possible to share memory between a primary and secondary
process using rte_memzone_reserve_aligned to allocate memory in primary
that is "seen" also by the secondary. If we have two primary processes
(started with different file-prefix) the same approach is not feasible. I
wonder how to share a chunk of memory hosted on a hugepage between two
primaries.

Regards.


Hi Antonio,

Correction: all hugepages allocated by DPDK are shared
between primary and secondary processes, not only memzones.

I assume we're talking about processes within one host,
because your previous similar question was about sharing memory between hosts
(as we have discussed offline), which is out of scope for DPDK.

As for the question directly, you need to map the same part of the same file
in the second primary as the hugepage is mapped from in the first primary.
I don't recommend to work with file paths, because their management
is not straightforward (--single-file-segments, for one) and is undocumented.

There is a way to share DPDK memory segment file descriptors.
Although public, this DPDK API is dangerous in the sense that you must
clearly understand what you're doing and how DPDK works.
Hence the question: what is the task you need this sharing for?
Maybe there is a simpler way.

1. In the first primary:

 mz = rte_memzone_reserve()
 ms = rte_mem_virt2memseg(mz->addr)
 fd = rte_memseg_get_fd(ms)
 offset = rte_memseg_get_fd_offset(ms)

2. Use Unix domain sockets with SCM_RIGHTS
to send "fd" and "offset" to the second primary.

3. In the second primary, after receiving "fd" and "offset":

 flags = MAP_SHARED | MAP_HUGE | (30 << MAP_HUGE_SHIFT)
 addr = mmap(fd, offset, flags)

Note that "mz" may consist of multiple "ms" depending on the sizes
of the zone and hugepages, and on the zone alignment.
Also "addr" may (and probably will) differ from "mz->addr".
It is possible to pass "mz->addr" and try to force it,
like DPDK does for primary/secondary.



Also 'net/memif' driver can be used:
https://doc.dpdk.org/guides/nics/memif.html

Re: [dpdk-users] [dpdk-dev] Intel FW 8.15 with DPDK 20.11 & 21.02

[Ferruh Yigit]

On 9/2/2021 10:40 PM, Korolevskiy, Igor wrote:
> Dear DPDK community,
> 
> Would you please help us to understand if we can use Intel 8.15 firmware with 
> DPDK 20.11 and 21.02 versions?
> 
> We have huge demand on our servers with usage of Nokia SBC and DPDK, but 
> stuck on validation.
> 
> Thank you in advance!
> 

Hi Igor,

Please clarify which Intel device are you talking about?

cc'ed Intel NIC maintainers (assuming the device you mentioned is a NIC) for
more details.

Regards,
ferruh

Re: [dpdk-users] [DISCUSSION] code snippet documentation

[Ferruh Yigit]

On 7/23/2021 1:02 AM, Ajit Khaparde wrote:
> On Thu, Jul 22, 2021 at 1:29 PM Thomas Monjalon  wrote:
>>
>> 15/07/2021 09:01, Asaf Penso:
>>> Hello DPDK community,
>>>
>>> I would like to bring up a discussion about a way to have code snippets as 
>>> an example for proper usage.
>>> The DPDK tree is filled with great pieces of code that are well documented 
>>> and maintained in high quality.
>>> I feel we are a bit behind when we talk about usage examples.
>>>
>>> One way, whenever we implement a new feature, is to extend one of the 
>>> test-* under the "app" folder.
>>> This, however, provides means to test but doesn't provide a good usage 
>>> example.
>>>
>>> Another way is to check the content of the "example" folder and whenever we 
>>> have a BIG new feature it seems like a good place.
>>> This, however, doesn't provide a good option when we talk about small 
>>> features.
>>> If, for example, we extend rte_flow with an extra action then providing a 
>>> full-blown example application is somewhat an entry barrier.
>>>
>>> A third option could be to document it in one of the .rst files we have.
>>> Obviously, this requires high maintenance and no option to assure it still 
>>> compiles.
>>>
>>> I'd like to propose another approach that will address the main two issues: 
>>> remove the entry barrier and assure compilation.
>>> In this approach, inside the "examples" folder we'll create another folder 
>>> for "snippets".
>>> Inside "snippets" we'll have several files per category, for example, 
>>> rte_flow_snippets.c
>>> Each .c file will include a main function that calls the different use 
>>> cases we want to give as an example.
>>> The purpose is not to generate traffic nor read rx/tx packets from the DPDK 
>>> ports.
>>> The purpose is to have a good example that compiles properly.
>>>
>>> Taking the rte_flow_snippets.c as an example its main function would look 
>>> like this:
>>>
>>> int
>>> main(int argc, char **argv)
>>> {
>>>   rte_flow_snippet_match_5tuple_and_drop();
>>>   rte_flow_snippet_match_geneve_ope_and_rss();
>>>   ...
>>>   Return 0;
>>> }
>>
>> I think we need to have a policy or justification about which snippet
>> is worth to have.
>> My thought is to avoid creating snippets which have no other value
>> than showing a function call.
>> I think there is a value if the context is not simple.
> 
> I agree. Otherwise it will be cluttered with snippets.
> 

I sometimes use DTS code as sample for an API, that has limited context (which I
believe sometimes needed to understand API properly) and of course compiles 
fine.

What about making mandatory to add a test to DTS for each API/feature, that
ensures a reasonable sample for the API call.

And if the intent is just to provide sample for API call without context, I
believe test-* can help on that, it clarifies good & bad input for an API.

>>
>>
>> Please could you provide a more complex example?
>>
>>

Re: [dpdk-users] A simpler question - does DPDK run over virtual interfaces?

[Ferruh Yigit]

On 4/13/2021 12:39 PM, Templin (US), Fred L wrote:
> Let me backtrack and start by asking a simpler question - can DPDK run over
> virtual interfaces such as a loopback?
> 

It can, using DPDK virtual interfaces, like using 'af_packet' or 'pcap'. Like:
"./build/app/dpdk-testpmd --vdev net_af_packet0,iface=lo --no-pci -- -i"

I don't know about CORE vnode, but if it is seen as another Linux virtual
interface, above should work same.

This may work to test/check some functionality, but for performance you will
need the physical interfaces.

Btw, binding a physical interface to vfio driver is to enable driving it by
DPDK, you don't need this step for Linux virtual interfaces.

> Thanks - Fred
> 
>> -Original Message-
>> From: Templin (US), Fred L
>> Sent: Monday, April 12, 2021 10:42 AM
>> To: 'users@dpdk.org' 
>> Subject: Problems using DPDK within CORE emulations running on Ubuntu 18.04 
>> VMs
>>
>> Hi, I am running Ubuntu 18.04 in a VM running on VirtualBox. I have built and
>> installed DPDK-20.11 from sources and had no troubles building by following
>> the Getting Started Guide for Linux instructions:
>>
>> http://doc.dpdk.org/guides/linux_gsg/intro.html
>>
>> Next, within the Ubuntu VM I run the CORE network emulator:
>>
>> https://www.nrl.navy.mil/Our-Work/Areas-of-Research/Information-Technology/NCS/CORE/
>>
>> I have a simple two-node network setup with two CORE vnodes connected
>> via a network switch, and verified that I can ping between the two nodes.
>> Now, I want to experiment with the DPDK-20.11 "ip_fragmentation" and
>> "ip_reassembly" example programs (which I was able to build successfully)
>> but it appears that these example programs require ports to be mapped.
>>
>> So, I skipped ahead to Section 5 of the Getting Started Guide for Linux
>> ("Linux Drivers") and tried to follow the instructions in Section 5.5. on
>> "Binding and Unbinding Network Ports to/from the Kernel Modules" by
>> typing commands into one of the CORE vnode shell windows. The text
>> at the end of this message shows the commands I typed and the output
>> I was shown in response. In particular, the "dpdk-devbind.py --status"
>> script does not appear to show a usable map of my CORE vnode
>> network interfaces, and attempts to bind were unsuccessful.
>>
>> Has anyone ever run DPDK out of a CORE vnode before and/or can
>> you tell me what steps are needed to be able to bind CORE vnode
>> interfaces so that they can be used by DPDK? Or, is DPDK simply
>> incompatible with virtualization environments.
>>
>> Another question - can DPDK be run over loopback interfaces?
>>
>> Thanks - Fred
>>
>> ---
>>
>> Script started on 2021-04-12 09:54:19-0700
>> root@n1: pwd
>> /home/fltemplin/src/DPDK/dpdk-20.11/usertools
>> root@n1: ip link sho
>> 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
>> DEFAULT group default qlen 1000
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> 5: eth0@if6:  mtu 1500 qdisc noqueue state 
>> UP mode DEFAULT group default qlen 1000
>> link/ether 00:00:00:aa:00:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0
>> root@n1: sudo modprobe vfio-pci
>> root@n1: ip link sho
>> 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
>> DEFAULT group default qlen 1000
>> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>> 5: eth0@if6:  mtu 1500 qdisc noqueue state 
>> UP mode DEFAULT group default qlen 1000
>> link/ether 00:00:00:aa:00:00 brd ff:ff:ff:ff:ff:ff link-netnsid 0
>> root@n1: ./dpdk-devbind.py --status
>>
>> Network devices using kernel driver
>> ===
>> :00:03.0 '82540EM Gigabit Ethernet Controller 100e' if= drv=e1000 
>> unused=vfio-pci
>> :00:08.0 '82540EM Gigabit Ethernet Controller 100e' if= drv=e1000 
>> unused=vfio-pci
>>
>> No 'Baseband' devices detected
>> ==
>>
>> No 'Crypto' devices detected
>> 
>>
>> No 'Eventdev' devices detected
>> ==
>>
>> No 'Mempool' devices detected
>> =
>>
>> No 'Compress' devices detected
>> ==
>>
>> No 'Misc (rawdev)' devices detected
>> ===
>>
>> No 'Regex' devices detected
>> ===
>> root@n1: exit
>>
>> Script done on 2021-04-12 09:55:41-0700
>>
> 

Re: [dpdk-users] What is the 'unit of timestamp' assigned to mbuf packet in DPDK?

[Ferruh Yigit]

On 11/18/2019 2:29 PM, Gokul Bargaje wrote:
> Hi,
> 
> The timestamp assigned to packet at the time of enqueue (value of timestamp
> field in mbuf), is it in milliseconds or microseconds or in cpu cycles?

The unit is not defined [1]. 'rte_eth_read_clock()' was added [2] to help
converting it to time when it is clock counter.

[1]
918ae9dc775e ("mbuf: add a timestamp field")

[2]
5e741377657c ("ethdev: add API to read device clock")

> 
> How this timestamp is calculated? Is it calculated using the *rte_cycles.h*?
> 

Re: [dpdk-users] [dpdk-dev] Intel XXV710 SR-IOV packet loss

[Ferruh Yigit]

On 8/20/2019 4:36 PM, Miroslav Kováč wrote:
> Hello,
> 
> 
> We are trying a setup with intel 25 GB card XXV710 and sr-iov. We need sr-iov 
> to sort packets based on vlan in between the VFs. We are using trex on one 
> machine to generate packets and multiple VPPs (each in docker container, 
> using one VF) on another one. Trex machine contains the exact same hardware.
> 
> 
> Each VF contains one vlan with spoof checking off and trust on and specific 
> MAC address. For example ->
> 
> 
> vf 0 MAC ba:dc:0f:fe:ed:00, vlan 1537, spoof checking off, link-state auto, 
> trust on
> 
> 
> 
> We are generating packets with VF destination MACs with the corresponding 
> VLAN. When sending packets to 3 VFs trex shows 35 million tx-packets and Dpdk 
> stats on the trex machine show that 35 million were in fact sent out:
> 
> 
> # DPDK Statistics port0 #
> {
> "tx_good_bytes": 2142835740,
> "tx_good_packets": 35713929,
> "tx_size_64_packets": 35713929,
> "tx_unicast_packets": 35713929
> }
> 
> 
> rate= '96%'; pktSize=   64; frameLoss%=51.31%; bytesReceived/s=
> 1112966528.00; totalReceived=   17390102; totalSent=   35713929; frameLoss=   
> 18323827; bytesReceived=1112966528; targetDuration=1.0
> 
> 
> However VPP shows only 33 million rx-packets:
> 
> VirtualFunctionEthernet17/a/0 2  up  9000/0/0/0
> rx packets   5718196
> rx bytes   343091760
> rx-miss  5572089
> 
> VirtualFunctionEthernet17/a/1 2  up  9000/0/0/0
> rx packets   5831396
> rx bytes   349883760
> rx-miss  5459089
> 
> VirtualFunctionEthernet17/a/2 2  up  9000/0/0/0
> rx packets   5840512
> rx bytes   350430720
> rx-miss  5449466
> 
> Sum of rx packets and rx-miss is 33,870,748. About 2 million is missing.
> 
> 
> 
> Even when I check VFs stats I see only 33 million to come (out of which 9.9 
> million are rx-missed):
> 
> 
> root@protonet:/home/protonet# for f in $(ls 
> /sys/class/net/enp23s0f1/device/sriov/*/stats/rx_packets); do echo "$f: $(cat 
> $f)"; done | grep -v ' 0$'
> 
> /sys/class/net/enp23s0f1/device/sriov/0/stats/rx_packets: 11290290
> /sys/class/net/enp23s0f1/device/sriov/1/stats/rx_packets: 11290485
> /sys/class/net/enp23s0f1/device/sriov/2/stats/rx_packets: 11289978
> 
> 
> 
> When increasing the number of VFs the number of rx-packets on VPP is actually 
> decreasing. Up to 6 or 7 VFs I still receive somewhere around 28-33 million 
> packets, but when I use 8 VFs all the sudden it drops to 16 million packets 
> (no rx-miss any more). The same goes with trunk mode:
> 
> 
> VirtualFunctionEthernet17/a/0 2  up  9000/0/0/0
> rx packets   1959110
> rx bytes   117546600
> 
> 
> VirtualFunctionEthernet17/a/1 2  up  9000/0/0/0
> rx packets   1959181
> rx bytes   117550860
> 
> VirtualFunctionEthernet17/a/2 2  up  9000/0/0/0
> rx packets   1956242
> rx bytes   117374520
> .
> .
> .
> Approximately the same amount of packets for each VPP instance which is 2 
> million packets * 8 = 16 million packets out of 35 million sent. Almost 20 
> million are gone
> 
> 
> 
> We are using vfio-pci driver.
> 
> 
> The strange thing is that when I use only PF, no sr-iov VFs are on and I try 
> the same vpp setup I can see all 35 million packets to come across.
> 
> 
> This leads us to believe that there could be something wrong the sr-iov on 
> XXV710 but we don't know how to debug this any further - The packets seem to 
> be lost somewhere in the NIC when using sr-iov and we don't know of any dpdk 
> or linux tool that could help us with locating the lost packets.
> 
> 
> Regards,
> 
> Miroslav Kovac
> 

+i40e maintainers.

Re: [dpdk-users] QAT cryto PMD queue pair and Ring Libary

[Ferruh Yigit]

On 3/11/2019 4:05 PM, Changchun Zhang wrote:
> I found i40e, ifc, and ixgbe under ./drivers/net/. Which folder is used to 
> build the igb_uio?

searching for it in the repo should yield some info:

$ find . -name "*igb_uio*"
./kernel/linux/igb_uio


> 
> Changchun (Alex)
> 
> 
> -Original Message-
> From: Ferruh Yigit [mailto:ferruh.yi...@intel.com] 
> Sent: Monday, March 11, 2019 7:08 AM
> To: Changchun Zhang ; Trahe, Fiona 
> ; users@dpdk.org
> Subject: Re: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
> 
> On 3/8/2019 9:14 PM, Changchun Zhang wrote:
>> Thanks.
>> The related question, for Ethernet device, if it has been bound to igb_uio 
>> driver, who is going to actually implement the rx_ring and tx_ring for the 
>> Ethernet PMD? The igb_uio driver?
> 
> DPDK drivers does it, please check ./drivers/net/* igb_uio just for exposing 
> the device registers to be able to use with DPDK drivers.
> 
>>
>> Changchun (Alex)
>>
>>
>> -Original Message-
>> From: Trahe, Fiona [mailto:fiona.tr...@intel.com]
>> Sent: Friday, March 08, 2019 5:56 AM
>> To: Changchun Zhang ; users@dpdk.org
>> Cc: Trahe, Fiona 
>> Subject: RE: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
>>
>>
>>> -Original Message-
>>> From: users [mailto:users-boun...@dpdk.org] On Behalf Of Changchun 
>>> Zhang
>>> Sent: Thursday, March 7, 2019 9:29 PM
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
>>>
>>> Hi,
>>>
>>>
>>>
>>> QAT PMD driver uses the queue pair to communication with QAT device. 
>>> My question is, whether or this QAP queue pair is implemented over DPDK 
>>> ring library or just self-defined queue implementation?
>> [Fiona] This doesn't use the DPDK ring library. They are QAT-specific, 
>> hw-supported tx and rx rings.
>>
> 

Re: [dpdk-users] QAT cryto PMD queue pair and Ring Libary

[Ferruh Yigit]

On 3/11/2019 3:59 PM, Changchun Zhang wrote:
> Thanks!
> So igb_uio driver is actually part of DPDK, instead of the general Ethernet 
> driver. This is very important. 

igb_uio driver is delivered by DPDK. igb_uio and drivers are separated.

> 
> Changchun (Alex)
> 
> 
> -Original Message-----
> From: Ferruh Yigit [mailto:ferruh.yi...@intel.com] 
> Sent: Monday, March 11, 2019 7:08 AM
> To: Changchun Zhang ; Trahe, Fiona 
> ; users@dpdk.org
> Subject: Re: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
> 
> On 3/8/2019 9:14 PM, Changchun Zhang wrote:
>> Thanks.
>> The related question, for Ethernet device, if it has been bound to igb_uio 
>> driver, who is going to actually implement the rx_ring and tx_ring for the 
>> Ethernet PMD? The igb_uio driver?
> 
> DPDK drivers does it, please check ./drivers/net/* igb_uio just for exposing 
> the device registers to be able to use with DPDK drivers.
> 
>>
>> Changchun (Alex)
>>
>>
>> -Original Message-
>> From: Trahe, Fiona [mailto:fiona.tr...@intel.com]
>> Sent: Friday, March 08, 2019 5:56 AM
>> To: Changchun Zhang ; users@dpdk.org
>> Cc: Trahe, Fiona 
>> Subject: RE: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
>>
>>
>>> -Original Message-
>>> From: users [mailto:users-boun...@dpdk.org] On Behalf Of Changchun 
>>> Zhang
>>> Sent: Thursday, March 7, 2019 9:29 PM
>>> To: users@dpdk.org
>>> Subject: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
>>>
>>> Hi,
>>>
>>>
>>>
>>> QAT PMD driver uses the queue pair to communication with QAT device. 
>>> My question is, whether or this QAP queue pair is implemented over DPDK 
>>> ring library or just self-defined queue implementation?
>> [Fiona] This doesn't use the DPDK ring library. They are QAT-specific, 
>> hw-supported tx and rx rings.
>>
> 

Re: [dpdk-users] QAT cryto PMD queue pair and Ring Libary

[Ferruh Yigit]

On 3/8/2019 9:14 PM, Changchun Zhang wrote:
> Thanks.
> The related question, for Ethernet device, if it has been bound to igb_uio 
> driver, who is going to actually implement the rx_ring and tx_ring for the 
> Ethernet PMD? The igb_uio driver?

DPDK drivers does it, please check ./drivers/net/*
igb_uio just for exposing the device registers to be able to use with DPDK 
drivers.

> 
> Changchun (Alex)
> 
> 
> -Original Message-
> From: Trahe, Fiona [mailto:fiona.tr...@intel.com] 
> Sent: Friday, March 08, 2019 5:56 AM
> To: Changchun Zhang ; users@dpdk.org
> Cc: Trahe, Fiona 
> Subject: RE: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
> 
> 
>> -Original Message-
>> From: users [mailto:users-boun...@dpdk.org] On Behalf Of Changchun 
>> Zhang
>> Sent: Thursday, March 7, 2019 9:29 PM
>> To: users@dpdk.org
>> Subject: [dpdk-users] QAT cryto PMD queue pair and Ring Libary
>>
>> Hi,
>>
>>
>>
>> QAT PMD driver uses the queue pair to communication with QAT device. 
>> My question is, whether or this QAP queue pair is implemented over DPDK ring 
>> library or just self-defined queue implementation?
> [Fiona] This doesn't use the DPDK ring library. They are QAT-specific, 
> hw-supported tx and rx rings.
> 

Re: [dpdk-users] KNI jumbo support

[Ferruh Yigit]

On 12/20/2018 3:27 PM, Liron Himi wrote:
> Hi,
> 
> I'm using DPDK18.11, and I launch a KNI flow.
> We need to pass jumbo frames via KNI interface but when we try to change its 
> MTU (using ifconfig) above 1500 we get an error.
> 
> is it possible?
> I noticed that the 'rte_kni' support some network callbacks (e.g. changing 
> MTU) but the rte_eth_kni doesn't define them.
> Is it possible to define them from outside? i.e. part of application code.

Hi Liron,

KNI PMD is not using callbacks right now, you can have them if you use kni
library directly, as kni sample application does.

And as far as I remember someone pointed in the list previously that in kernel
side, for Rx mbuf segmentation is implemented but for Tx it is not. So if kni
interface tries to send a packet which is greater than mbuf size, it will be
dropped, you may need to implement that part as well.

Regards,
ferruh

Re: [dpdk-users] [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs

[Ferruh Yigit]

On 10/20/2018 7:30 AM, Shyam Shrivastav wrote:
> Yes you are right, I misread, following code (ipv4 case) assumes no ip
> options while calculating pseudo hdr length field
> 
> psd_hdr.len = rte_cpu_to_be_16(
> (uint16_t)(rte_be_to_cpu_16(ipv4_hdr->total_length)
> - sizeof(struct ipv4_hdr)));
> 
> should be
> 
>  psd_hdr.len = rte_cpu_to_be_16(
> (uint16_t)(rte_be_to_cpu_16(ipv4_hdr->total_length)
> - (ipv4_hdr->version_ihl & 0x0f)*4)));

cc'ed maintainer of the that part, Olivier.

> 
> On Sat, Oct 20, 2018 at 11:44 AM lidejun  wrote:
> 
>> I mean DPDK APIs do not exclude ipv4 options or ipv6 extension headers, it
>> is bug?
>>
>>
>>
>> *发件人:* Shyam Shrivastav [mailto:shrivastav.sh...@gmail.com]
>> *发送时间:* 2018年10月20日 13:23
>> *收件人:* lidejun 
>> *抄送:* users ; d...@dpdk.org; Lichunhe (Cloud Networking) <
>> lichu...@huawei.com>; Wangliefeng 
>> *主题:* Re: [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs
>>
>>
>>
>> Realized my answer is confusing, I meant to say that code is correct as
>> pseudo ipv4/ipv6 headers for the purpose of checksum calculations doesn't
>> include options or extension headers, see udp wiki or corresponding rfcs
>>
>>
>>
>> https://en.wikipedia.org/wiki/User_Datagram_Protocol
>>
>>
>>
>> On Sat, Oct 20, 2018 at 10:42 AM Shyam Shrivastav <
>> shrivastav.sh...@gmail.com> wrote:
>>
>> that is correct , pseudo header doesn't include ipv4 options or ipv6
>> extension headers ..
>>
>>
>>
>> On Sat, Oct 20, 2018 at 9:02 AM lidejun  wrote:
>>
>> Has anybody used the following two APIs calculating ipv4&ipv6 tcp/udp
>> pseudo header checksum？
>>
>> 1.rte_ipv4_phdr_cksum
>>
>> 2.rte_ipv6_phdr_cksum
>> The ipv4 version does not exclude ip options and ipv6 version does not
>> exclude extersion headers.
>>
>>

[dpdk-users] reviewathon is today!(?)

[Ferruh Yigit]

Hi,

An reviewathon is happening in Intel, please feel free to join us.

Indeed this was planned to be community review day, but we were late to
communicate this, sorry. I hope we can do a better organized one next week.

Please join to IRC [1], fell free to ask questions, communicate with other
developers to boost consuming patch backlog.

Cheers,
ferruh

[1]
freenode.net #dpdk

Re: [dpdk-users] Suggestion: DPDK "latest" Download Links

[Ferruh Yigit]

On 8/20/2018 8:03 PM, Justin Parus wrote:
> Hi,
> 
> I would like to propose adding DPDK "latest" download links. For example 
> dpdk-latest-major and dpdk-latest-stable, would automatically update and 
> point to the most recent date major and stable releases respectively. This 
> would be useful for our tests as they would always be testing the latest 
> packages automatically.

+1, looks good idea

cc'ed Thomas & Trishan.


> 
> Thanks,
> Justin
> 

Re: [dpdk-users] Support for more RSS hash types in vmxnet3

[Ferruh Yigit]

On 8/21/2018 6:57 PM, Jay Miller wrote:
> It's clear that the vmxnet3 driver (even as of 18.08) supports just a 
> subset of RSS hash types:
> 
> #define VMXNET3_RSS_OFFLOAD_ALL ( \
>      ETH_RSS_IPV4 | \
>      ETH_RSS_NONFRAG_IPV4_TCP | \
>      ETH_RSS_IPV6 | \
>      ETH_RSS_NONFRAG_IPV6_TCP)
> 
> Are there plans to add support for other hash types (like 
> ETH_RSS_NONFRAG_IPV4_UDP), or is this an architectural limitation of 
> vmxnet3?

Hi Yong,

Can you please double check if the driver reports all supported hash functions
correctly.

On v18.08, the RSS hf request from application changed from best effort to
strict requirement, meaning if an application request a hash function but driver
doesn't report it as supported API will return an error, that is why it is
important for PMD to report supported hf properly.

Thanks,
ferruh

> 
> The documentation could be more explicit about these limitations:
> 
>>>
>>> 36.2. Features and Limitations of VMXNET3 PMD
>>>
>>> In release 1.6.0, the VMXNET3 PMD provides the basic functionality of 
>>> packet reception and transmission. There are several options 
>>> available for filtering packets at VMXNET3 device level including:
>>>
>>>  1. *MAC Address based filtering*:
>>>   * Unicast, Broadcast, All Multicast modes - SUPPORTED BY DEFAULT
>>>   * Multicast with Multicast Filter table - NOT SUPPORTED
>>>   * Promiscuous mode - SUPPORTED
>>>   * *RSS based load balancing between queues - SUPPORTED*
>>>
>>
> 
> Thanks,
> J
> 

Re: [dpdk-users] [dpdk-dev] Dpdk eventdev performance testing

[Ferruh Yigit]

On 8/18/2018 4:12 PM, Guru Prasad wrote:
> Hi All,
> 
> While testing performance with eventdev, I am seeing a crash in dpdk
> service core. Test scenario is given below
> 1) DPDK 1805
> 2 Eventdev ordered Queues & 2 Ports
> 2) 2 Workers threads + 1 Service core for running scheduling.
> 3) 1st worker is getting packet from NIC and enqueueing to Q1 * [PLEASE
> FILL THIS]*
> 4) Packet rate of 5.6Mpps (Pkt size 64 bytes)

Thanks Guru for reporting.

Would you mind submitting this to https://bugs.dpdk.org/, it helps us trace it
better.

Thanks,
ferruh

Re: [dpdk-users] [dpdk-stable] librte_pmd_bnxt library backporting from 17.11 to 16.11

[Ferruh Yigit]

On 11/29/2017 7:59 AM, Raman, Sandeep wrote:
> Hi,
> 
> With 17.11 DPDK, Broadcom NIC binds successfully. However with 16.11 it fails.
> Is there a way to backport the bnxt library from 17.11 to 16.11?
> 
> Any other ways to get this to work with 16.11.

Can Long term stable tree works for you (DPDK 16.11.3) [1], not sure how much of
the bnxt driver back-ported though.

And providing more details abut the failure may be helpful too.

[1]
http://dpdk.org/rel

> 
> Thanks,
> Sandeep.
> 

Re: [dpdk-users] using the KNI on Amazon EC2

[Ferruh Yigit]

On 11/27/2017 11:11 AM, Cody Doucette wrote:
> Is there any way to bring up a KNI on EC2? I seem to be able to create the
> interface, assign IP addresses to it, etc, but can't actually activate the
> interface by bringing it up using either ifconfig or with a netlink socket
> directly.

Bringing interface up via ifconfig should be working.
Are you testing kni sample app or kni pmd?

> 
> This line from the KNI guide gives me doubt: "Ethtool is not supported in
> i40e and VMs."

This is valid for ethtool support only, and related how kni ethtool support
implemented.

> 
> Is there any work around to this?
> 

Re: [dpdk-users] KNI vhost support has been removed from 17.05

[Ferruh Yigit]

On 11/20/2017 1:34 AM, Zhao, Bing wrote:
> Hello Yigit,
>  I searched from the Internet about the KNI vhost support. And from 
> your slides "Interworking with the Linux Kernel" in Dublin, 2016, it is 
> said that the support was planned to be removed because no user use it. 
> And from the release notes of 17.05, it is mentioned, as title, the 
> support is removed. I searched the archive mails and only limited 
> information could be found. So it there any technical discussion about 
> this? And will this be added back in the future or removed forever?

Hi Bing,

There was no discussion, but also there was no comment to the patch that removes
the support, nobody objected it.

The KNI vhost is from times there was no vhost support exists, now it is
possible to use vhost.

And there is no plan to put it back. Are you an user of that feature?

Thanks,
ferruh


> 
> Many thanks in advance~
> 
> BR. Bing
> 

Re: [dpdk-users] Only building certain DPDK components?

[Ferruh Yigit]

On 3/14/2017 2:18 PM, Alex Forster wrote:
> Hi all, my google-fu is failing me.
> 
> Is it possible (supported) in the current build system to skip building
> apps and tests, and to only build certain libs – skipping
> librte_cmdline/librte_cfgfile, for instance?

It is possible to use %_sub target to build specific component, like:
"make lib/librte_eal_sub" will only compile eal library.
This requires config done, and dependent libraries built in advance.
Can be used for all components, like "make drivers/net/null_sub"

A component can be excluded from build via config file:
"make T=... config" will create config file (build/.config by default)
it is possible to edit it and disable components, via setting "=n"

> 
> *Alex Forster*
> Network Engineer, Linode
> 

Re: [dpdk-users] [dpdk-dev] Reg DPDK with unsupported NIC

[Ferruh Yigit]

On 3/14/2017 9:28 AM, raman geetha gopalakrishnan wrote:
> Hi ,
> 
> Please find my query, Currently i am planning to develop DPDK APP (linux
> env). I do not have DPDK supported NIC. Till then i would still like to
> develop DPDK APP and want DPDK  to use OS interface to TX/RX packets from
> NIC. How can i make it? 

> I went through KNI and my understanding is you
> cannot use it - is this correct?

You can use it. KNI does not require a physical device at all.

But with the KNI support in main tree, you need to use KNI specific
APIs, - which KNI sample app shows.

With the KNI PMD in next-net tree, it is possible to use KNI as any
other device.

> 
> In what way i can still develop DPDK APP with non supported NIC till get
> the DPDK NIC.
> 
> Thanks
> Raman
> 

Re: [dpdk-users] Unable to bind intel NIC 82599ES to vfio-pci, uio_pci_generic

[Ferruh Yigit]

On 2/8/2017 7:19 AM, Shyam Shrivastav wrote:
> Hi All
> 
> I am not able to bind 82599 to either uio_pci_generic or vfio-pci
> successfully. Any help greatly appreciated, I am completely stuck at this
> initial step.
> 
> *1) uio_pci_generic* : tools/dpdk-devbind.py script reports success but it
> is not detected by EAL on initialisation, still ixgbe driver is detected.
> here is relevant console output

It is detected, but according following log ports are not enabled by
proper application (not eal) command line argument:
"   Cause: All available ports are disabled. Please set portmask."

./examples/l2fwd/build/l2fwd [EAL options] -- -p PORTMASK [-q NQ]
  -p PORTMASK: hexadecimal bitmask of ports to configure
  -q NQ: number of queue (=ports) per lcore (default is 1)
  -T PERIOD: statistics will be refreshed each PERIOD seconds (0 to
disable, 10 default, 86400 maximum)
  --[no-]mac-updating: Enable or disable MAC addresses updating (enabled
by default)
  When enabled:
   - The source MAC address is replaced by the TX port MAC address
   - The destination MAC address is replaced by
02:00:00:00:00:TX_PORT_ID

<...>

> 
> 
> 
> *2) vfio-pci :* Configured vfio permissions using setup.sh, bind script
> reports error in this case as under

Not sure about this one, below looks correct. Perhaps dmesg can tell more.

<...>

> 
> Thanks
> Shyam
> 

Re: [dpdk-users] NIC NETXtreme II BCM57810 10GE

[Ferruh Yigit]

On 2/13/2017 10:03 AM, Avi Cohen (A) wrote:
> HI,

Hi Avi,

> I don't find this in the list of supported NIC 
> So I get errors when setting it as dpdk type interface in the OVS 
> I understand that it is not supported by DPDK - correct ?

List of supported NICs in DPDK:
http://dpdk.org/doc/nics


And for supported PCI dev IDs by Broadcom driver:
http://dpdk.org/browse/dpdk/tree/drivers/net/bnxt/bnxt_ethdev.c#n95


Above I can't see the mentioned device, it looks like not supported as
you state. Broadcom driver maintainer CC'ed for more accurate answer.
Cc: Stephen Hurd 


> Regards avi
> 

Re: [dpdk-users] KNI drive without supported NICs?

[Ferruh Yigit]

On 2/6/2017 3:01 PM, Lazarenko, Vlad (WorldQuant) wrote:
> Dumitru,
> 
> This clear things up a big time! Much appreciated. Thank you!

KNI PMD patch [1], which is not upstream yet, can be used to create any
number of KNI interfaces independent from hardware.

It is a virtual PMD implementation, so can be used via testpmd or any
DPDK application.

[1]
http://dpdk.org/dev/patchwork/patch/20092/

> 
> - Vlad
> 
>> -Original Message-
>> From: Take Ceara [mailto:dumitru.ce...@gmail.com]
>> Sent: Monday, February 06, 2017 6:05 AM
>> To: Lazarenko, Vlad (WorldQuant)
>> Cc: users@dpdk.org
>> Subject: Re: [dpdk-users] KNI drive without supported NICs?
>>
>> Hi Vlad,
>>
>> On Sun, Feb 5, 2017 at 6:34 PM, Lazarenko, Vlad (WorldQuant)
>>  wrote:
>>> Gentlemen and gentleladies,
>>>
>>> I am trying to get the KNI working on my dev box. So I've loaded the
>> rte_kni.ko module and made sure /dev/kni has correct permissions. But
>> when I am trying to run the KNI example, it fails with "No supported
>> Ethernet device found" error:
>>>
>>> $ build/kni -n 4 -c 0xf0 -- -P -p 0x3 --config="(0,4,6),(1,5,7)"
>>> EAL: Detected 24 lcore(s)
>>> EAL: PCI device :01:00.0 on NUMA socket -1
>>> EAL:   probe driver: 8086:10d3 net_e1000_em
>>> EAL: Error - exiting with code: 1
>>>   Cause: No supported Ethernet device found
>>>
>>> My dev box does not in fact have supported NICs. But my understanding
>> about KNI was that it allows to access Linux network stack trough the KNI
>> kernel module, where applications create a virtual Ethernet devices (i.e.
>> /sys/class/net/vEth*), thus there is no need for a real supported NIC or
>> taking over the entire NIC.
>>>
>>> On the other hand, the example application calls rte_eth_dev_count()
>> first, which in my case returns 0, and exits.
>>>
>>> Is my understanding about what KNI actually is wrong? Or am I missing
>> something else?
>>>
>>
>> As far as I know the KNI sample app actually forwards traffic to/from the
>> kernel and a physical port therefore the check for physical interfaces. If 
>> you
>> want to use KNI for something else there's nothing stopping you from
>> starting a DPDK app and adding a KNI interface without having a real NIC
>> bound to DPDK.
>>
>> I don't have a better example right now but, if you want, you can take a look
>> at how we do it in warp17 (using DPDK) when you start it without any
>> physical interfaces:
>>
>> https://github.com/Juniper/warp17#using-kernel-network-interface-kni-
>> interfaces
>>
>> If you look at the HTTP example there we actually start the application
>> without any physical interfaces (only one KNI):
>>
>> ./build/warp17 -c FC3 -n 4  -m 32768 -- --kni-ifs 1
>>
>>
>>> Thanks,
>>> Vlad
>>
>> Hope this helps..
>>
>> Regards,
>> Dumitru
>>
>>>
>>>
>>>
>>>
>> ###
>> ###
>>> #
>>>
>>> The information contained in this communication is confidential, may
>>> be
>>>
>>> subject to legal privilege, and is intended only for the individual named.
>>>
>>> If you are not the named addressee, please notify the sender
>>> immediately and
>>>
>>> delete this email from your system.  The views expressed in this email
>>> are
>>>
>>> the views of the sender only.  Outgoing and incoming electronic
>>> communications
>>>
>>> to this address are electronically archived and subject to review
>>> and/or disclosure
>>>
>>> to someone other than the recipient.
>>>
>>>
>> ###
>> ###
>>> #
>>
>>
>>
>> --
>> Dumitru Ceara
> 
> 
> ###
> 
> The information contained in this communication is confidential, may be
> 
> subject to legal privilege, and is intended only for the individual named.
> 
> If you are not the named addressee, please notify the sender immediately and
> 
> delete this email from your system.  The views expressed in this email are
> 
> the views of the sender only.  Outgoing and incoming electronic communications
> 
> to this address are electronically archived and subject to review and/or 
> disclosure
> 
> to someone other than the recipient.
> 
> ###
> 

Re: [dpdk-users] Is PF-only driver allowed to upstream into dpdk?

[Ferruh Yigit]

On 2/13/2017 9:43 AM, Netala, Sameera wrote:
> Hi,

Hi Sameera,

> 
> I am currently working on a non-nic mode user space PCI driver which 
> processes custom packets from OCTEON 7xxx processor. As per the requirements, 
> DPDK is the best way to achieve this.

Can you please give more information about hardware? Is this an offload
engine?

> 
> Though I have seen few designs and sources that talk about dpdk pf drivers, I 
> am interested to know if such a driver with pf-only support and which doesn't 
> involve virtual functions will be allowed to upstream into DPDK sources.

There is no problem with PF only drivers, and there are already samples
of it in the project.

As long as driver is open source and for data path, it is good for
upstream. Please feel free to upstream your driver when it is ready.

The discussion about DPDK PF drivers was about if DPDK PF driver should
be in control path or not, this is high level project scope discussion.

Thanks,
ferruh

> 
> Thanks,
> Sameera
> 

Re: [dpdk-users] Compilation errors on Mellanox Poll Mode Driver

[Ferruh Yigit]

On 2/2/2017 6:10 PM, nirmoy das wrote:
> Hi,
> 
> I am trying to compile mellanox pmd but getting lots of undeclared macro
> error,
> 
> 
> e.g.
> 
> /dpdk-16.11/drivers/net/mlx4/mlx4.c:3010:7: error:
> 'IBV_EXP_CQ_RX_OUTER_TCP_UDP_CSUM_OK' undeclared (first use in this
> function)
> [   76s]IBV_EXP_CQ_RX_OUTER_TCP_UDP_CSUM_OK,

For me it comes from /usr/include/infiniband/verbs_exp.h, do you have
that file?

> 
> I do have libibverbs-devel installed, I am wondering what all library and
> what version I need to compile the pmd.
> 
> Regards,
> Nirmoy
> 

Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management

[Ferruh Yigit]

On 1/13/2017 11:10 AM, Alejandro Lucero wrote:
> I completely misread the patch, and I wrongly thought that code was
> linked to module removal, but I see this is not about that, but about
> releasing the /dev/uio file calling release function, what is done by
> the kernel when the process exits.
> 
> So yes, the patch avoids the problem I talked about.
> 
> However, calling that specific ixgbe driver function will break other
> devices relying on igb_uio. What about implementing a notifier chain for
> this? The igb_uio code would be agnostic and each interested driver,
> like ixgbe or nfp_net, could execute the specific port close code when
> the notifier chain triggers.

Can you please elaborate, how can this be done?
This is kernel code, and interested drivers are in userspace. This patch
benefit from Linux kernel driver of same device.

> 
> 
> On Fri, Jan 13, 2017 at 5:33 AM, Tan, Jianfeng  <mailto:jianfeng@intel.com>> wrote:
> 
> 
> 
> > -Original Message-
> > From: Yigit, Ferruh
> > Sent: Friday, January 13, 2017 10:05 AM
> > To: Tan, Jianfeng; Alejandro Lucero
> > Cc: Gregory Etelson; dev; users@dpdk.org <mailto:users@dpdk.org>
> > Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management
> >
> > On 1/13/2017 1:51 AM, Tan, Jianfeng wrote:
> > >
> > >
> > >> -Original Message-
> > >> From: users [mailto:users-boun...@dpdk.org
> <mailto:users-boun...@dpdk.org>] On Behalf Of Ferruh Yigit
> > >> Sent: Thursday, January 12, 2017 8:22 PM
> > >> To: Alejandro Lucero
> > >> Cc: Gregory Etelson; dev; users@dpdk.org <mailto:users@dpdk.org>
> > >> Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources
> > Management
> > >>
> > >> On 1/12/2017 12:12 PM, Alejandro Lucero wrote:
> > >>>
> > >>>
> > >>> On Thu, Jan 12, 2017 at 11:55 AM, Ferruh Yigit
> mailto:ferruh.yi...@intel.com>
> > >>> <mailto:ferruh.yi...@intel.com
> <mailto:ferruh.yi...@intel.com>>> wrote:
> > >>>
> > >>> On 12/9/2016 8:54 AM, Gregory Etelson wrote:
> > >>> > Hello,
> > >>> >
> > >>> > IGB_UIO driver does not close port PCI activities after
> DPDK process
> > >> exits.
> > >>> > DPDK API provides rte_eth_dev_close() to manage port PCI,
> > >>> > but it can be skipped if process receives SIGKILL signal
> > >>>
> > >>> I guess I understand the problem.
> > >>>
> > >>>
> > >>> This is a known problem, but it is not just a UIO problem, and
> this
> > >>> patch does not solve it, maybe it just solves part of it.
> > >>>
> > >>> In fact, a DPDK program crashing could imply the NIC DMAing
> after that
> > >>> and after that memory was assigned to another program.
> > >>
> > >> Yes.
> > >> Can there be a way to stop NIC DMA, (or prevent it access to mem
> > >> anymore) when app crashes?
> > >> I think that is what this patch is looking for.
> > >
> > > If I understand it correctly, you are looking for this patch?
> > > http://dpdk.org/dev/patchwork/patch/17495/
> <http://dpdk.org/dev/patchwork/patch/17495/>
> > >
> >
> > That is good, thanks Jianfeng, I will check it.
> >
> > btw, patch's current state is rejected, which is by mistake, it
> seems I
> > confused it with "iomem and ioport mapping" patch, sorry about it, I
> > will update its status immediately.
> 
> No problem at all. This patch is rejected as it's based on "iomem
> and ioport mapping" patch. As "iomem and ioport mapping" patch has
> backward compatibility issue, we need to figure out a way to
> resubmit this patch without changing the original "iomem and ioport
> mapping" in igb_uio.
> 
> Thanks,
> Jianfeng
> 
> 

Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management

[Ferruh Yigit]

On 1/13/2017 5:33 AM, Tan, Jianfeng wrote:
> 
> 
>> -Original Message-
>> From: Yigit, Ferruh
>> Sent: Friday, January 13, 2017 10:05 AM
>> To: Tan, Jianfeng; Alejandro Lucero
>> Cc: Gregory Etelson; dev; users@dpdk.org
>> Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management
>>
>> On 1/13/2017 1:51 AM, Tan, Jianfeng wrote:
>>>
>>>
>>>> -Original Message-
>>>> From: users [mailto:users-boun...@dpdk.org] On Behalf Of Ferruh Yigit
>>>> Sent: Thursday, January 12, 2017 8:22 PM
>>>> To: Alejandro Lucero
>>>> Cc: Gregory Etelson; dev; users@dpdk.org
>>>> Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources
>> Management
>>>>
>>>> On 1/12/2017 12:12 PM, Alejandro Lucero wrote:
>>>>>
>>>>>
>>>>> On Thu, Jan 12, 2017 at 11:55 AM, Ferruh Yigit >>>> <mailto:ferruh.yi...@intel.com>> wrote:
>>>>>
>>>>> On 12/9/2016 8:54 AM, Gregory Etelson wrote:
>>>>> > Hello,
>>>>> >
>>>>> > IGB_UIO driver does not close port PCI activities after DPDK process
>>>> exits.
>>>>> > DPDK API provides rte_eth_dev_close() to manage port PCI,
>>>>> > but it can be skipped if process receives SIGKILL signal
>>>>>
>>>>> I guess I understand the problem.
>>>>>
>>>>>
>>>>> This is a known problem, but it is not just a UIO problem, and this
>>>>> patch does not solve it, maybe it just solves part of it.
>>>>>
>>>>> In fact, a DPDK program crashing could imply the NIC DMAing after that
>>>>> and after that memory was assigned to another program.
>>>>
>>>> Yes.
>>>> Can there be a way to stop NIC DMA, (or prevent it access to mem
>>>> anymore) when app crashes?
>>>> I think that is what this patch is looking for.
>>>
>>> If I understand it correctly, you are looking for this patch?
>>> http://dpdk.org/dev/patchwork/patch/17495/
>>>
>>
>> That is good, thanks Jianfeng, I will check it.
>>
>> btw, patch's current state is rejected, which is by mistake, it seems I
>> confused it with "iomem and ioport mapping" patch, sorry about it, I
>> will update its status immediately.
> 
> No problem at all. This patch is rejected as it's based on "iomem and ioport 
> mapping" patch. As "iomem and ioport mapping" patch has backward 
> compatibility issue, we need to figure out a way to resubmit this patch 
> without changing the original "iomem and ioport mapping" in igb_uio.

I thinks implementing uio_info->release and uio_info.open is good idea,
but I have a few questions:

1- What is the the dependency to "iomem and ioport mapping" patch?

2- If we keep pci_enable_device() in probe() can this prevent moving
registering/freeing interrupts in open()/release()

3- And is pci_disable_device() done in release is enough to stop NIC DMA
to access memory?


I did a simple test, implemented simple uio_info->release and
uio_info.open, which only does pci_disable_device() and
pci_enable_device(),
but this prevent app receiving packets in its second run, independent
from app terminated gracefully or not. Any idea why this is not working?


btw, I can produce the problematic case, as George Prekas described in:
http://dpdk.org/ml/archives/users/2016-September/001026.html

CC'ed George, since he also seems interested in issue.

> 
> Thanks,
> Jianfeng
> 

Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management

[Ferruh Yigit]

On 1/13/2017 1:51 AM, Tan, Jianfeng wrote:
> 
> 
>> -Original Message-
>> From: users [mailto:users-boun...@dpdk.org] On Behalf Of Ferruh Yigit
>> Sent: Thursday, January 12, 2017 8:22 PM
>> To: Alejandro Lucero
>> Cc: Gregory Etelson; dev; users@dpdk.org
>> Subject: Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management
>>
>> On 1/12/2017 12:12 PM, Alejandro Lucero wrote:
>>>
>>>
>>> On Thu, Jan 12, 2017 at 11:55 AM, Ferruh Yigit >> <mailto:ferruh.yi...@intel.com>> wrote:
>>>
>>> On 12/9/2016 8:54 AM, Gregory Etelson wrote:
>>> > Hello,
>>> >
>>> > IGB_UIO driver does not close port PCI activities after DPDK process
>> exits.
>>> > DPDK API provides rte_eth_dev_close() to manage port PCI,
>>> > but it can be skipped if process receives SIGKILL signal
>>>
>>> I guess I understand the problem.
>>>
>>>
>>> This is a known problem, but it is not just a UIO problem, and this
>>> patch does not solve it, maybe it just solves part of it.
>>>
>>> In fact, a DPDK program crashing could imply the NIC DMAing after that
>>> and after that memory was assigned to another program.
>>
>> Yes.
>> Can there be a way to stop NIC DMA, (or prevent it access to mem
>> anymore) when app crashes?
>> I think that is what this patch is looking for.
> 
> If I understand it correctly, you are looking for this patch?
> http://dpdk.org/dev/patchwork/patch/17495/
> 

That is good, thanks Jianfeng, I will check it.

btw, patch's current state is rejected, which is by mistake, it seems I
confused it with "iomem and ioport mapping" patch, sorry about it, I
will update its status immediately.

Re: [dpdk-users] [dpdk-dev] IGB_UIO: PCI Resources Management

[Ferruh Yigit]

On 1/12/2017 12:12 PM, Alejandro Lucero wrote:
> 
> 
> On Thu, Jan 12, 2017 at 11:55 AM, Ferruh Yigit  <mailto:ferruh.yi...@intel.com>> wrote:
> 
> On 12/9/2016 8:54 AM, Gregory Etelson wrote:
> > Hello,
> >
> > IGB_UIO driver does not close port PCI activities after DPDK process 
> exits.
> > DPDK API provides rte_eth_dev_close() to manage port PCI,
> > but it can be skipped if process receives SIGKILL signal
> 
> I guess I understand the problem.
> 
> 
> This is a known problem, but it is not just a UIO problem, and this
> patch does not solve it, maybe it just solves part of it.
> 
> In fact, a DPDK program crashing could imply the NIC DMAing after that
> and after that memory was assigned to another program.

Yes.
Can there be a way to stop NIC DMA, (or prevent it access to mem
anymore) when app crashes?
I think that is what this patch is looking for.

> 
>  
> 
> 
> > The patches below provide IGB_UIO release callback and IXGBEVF release 
> function
> 
> But adding ixgbe specific code into igb_uio may not be good idea.
> Can be anything done one upper layer, pci layer, generic to all drivers?
> 
> 
> This  module is not just being used for Intel cards, so this addition
> will break, at least, the NFP PMD support.
> 
> I was told to use igb_uio instead of adding a new NFP uio driver, so I
> guess that implies this igb_uio driver should be considered not only a
> igb driver.

No it is generic, I think names has igb_ just for historical reasons.

>  
> 
> > With the patches, each time DPDK process terminates,
> > UIO release callback will trigger port PCI close.
> > On the down side, patched IGB_UIO can be bound to a single adapter type
> >
> > Regards,
> > Gregory
> 
> <...>
> 
> 

Re: [dpdk-users] [dpdk-dev] DPDK NIC is unreachable

[Ferruh Yigit]

Hi Priyanka,

On 12/10/2016 4:46 PM, Priyanka wrote:
> Hi,
> 
> We have a SRIOV+ DPDK 16.04 setup for a VM using KVM-qemu hypervisor. We 
> are unable to ping the VM. We are assigning the VM to a SRIOV vf and for 
> the DPDK enabled NIC on the VM  to detect this VF, we are using KNI.
> 
> We used the KNI app available in the examples folder.  Although the non 
> DPDK NIC is reachable in the VM but once we assign an IP using KNI to 
> the DPDK NIC of the VM, it is unreachable.

Removing dev mail list.

Can you please give some information related setup, is following correct:

Host: Interface controlled by Linux (H1)
VM: There are two VF's, one controlled by DPDK (G1), other by Linux (G2)
NIC: ?

KNI sample application ruun on VM.

You can ping from H1 to G2, but not H1 to G1?

> 
> Please help us solve this issue.
> 
> Thanks,
> 
> Priyanka
> 
> 

Re: [dpdk-users] IGB_UIO: PCI Resources Management

[Ferruh Yigit]

On 12/9/2016 8:54 AM, Gregory Etelson wrote:
> Hello,
> 
> IGB_UIO driver does not close port PCI activities after DPDK process exits.
> DPDK API provides rte_eth_dev_close() to manage port PCI,
> but it can be skipped if process receives SIGKILL signal

I guess I understand the problem.

> The patches below provide IGB_UIO release callback and IXGBEVF release 
> function

But adding ixgbe specific code into igb_uio may not be good idea.
Can be anything done one upper layer, pci layer, generic to all drivers?

> With the patches, each time DPDK process terminates,
> UIO release callback will trigger port PCI close.
> On the down side, patched IGB_UIO can be bound to a single adapter type
> 
> Regards,
> Gregory

<...>

[dpdk-users] rte_kni.ko code

[Ferruh Yigit]

On 8/31/2016 2:31 PM, jagadeesh N wrote:
> Hi,
> 
> quick question Where can i find the rte_kni.ko code?

lib/librte_eal/linuxapp/kni/*

[dpdk-users] Using KNI with virtio-net-pci

[Ferruh Yigit]

On 6/28/2016 4:24 AM, Ruslan Osmanov wrote:
> Hi,
> 
> yes, the problem with running the KNI application was that I skipped
> binding NIC. Now the application is running on two NICs based on host
> TAP devices using igb_uio driver.
Good to hear that it is working now.

> 
> Actually, I've figured this out a couple of weeks ago. However, I still
> have difficulties testing the KNI app. Particularly, I can't see
> traffic going through NICs via `tcpdump -i vEth0`. The latter command
> displays only BOOT/DHCP requests from 0.0.0.0. For example, it doesn't
> show ICMP packets, when I `ping 192.168.1.10`(vEth0).
> 

I use netns if all interfaces are in same box, it should work with
updating routing tables but it is confusing and I am usually doing it
wrong J, can you please try with netns?

Regards,
ferruh

[dpdk-users] Using KNI with virtio-net-pci

[Ferruh Yigit]

On 6/2/2016 4:43 AM, Ruslan Osmanov wrote:
> Hi,
> 
> I'm going to develop a DPDK application on a laptop, but the laptop's
> hardware is not supported by DPDK. Furtunately, DPDK supports
> paravirtualized devices(http://www.dpdk.org/doc/nics) including
> virtio-net.  
> 
> So I'm trying to configure a QEMU guest for running the Kernel NIC
> Interface(KNI) on a virtio-net-pci device. The problem is that the KNI
> sample application doesn't accept the virtio-pci driver.
> 
> 
> QEMU command:
> 
> eth_device=virtio-net-pci
> exec qemu-system-x86_64 -enable-kvm \
>   -cpu host -smp 2 \
>   -vga std \
>   -mem-prealloc -mem-path /dev/hugepages \
>   -drive file=GentooVM.img,if=virtio \
>   -netdev user,id=vmnic,hostname=gentoo \
>   -device $eth_device,netdev=vmnic \
>   -m 1024M \
>   -monitor stdio \
>   -name "Gentoo VM"
> 
> Running the KNI sample application in the guest:
> 
> sudo ./examples/kni/build/app/kni -c 0x3 -n 4 -- \
> -P -p 0x1 --config="(0,0,1)"
> 
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 0 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 2 lcore(s)
> EAL: Probing VFIO support...
> EAL:   IOMMU type 1 (Type 1) is supported
> EAL:   IOMMU type 8 (No-IOMMU) is not supported
> EAL: VFIO support initialized
> EAL: Setting up physically contiguous memory...
> ...
> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> unreliable clock cycles !
> EAL: Master lcore 0 is ready (tid=657d58c0;cpuset=[0])
> PMD: rte_igbvf_pmd_init():  >>
> EAL: lcore 1 is ready (tid=305ff700;cpuset=[1])
> EAL: PCI device :00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> EAL:   Not managed by a supported kernel driver(0), skipped
> PMD: virtio_read_caps(): failed to map pci device!
> PMD: vtpci_init(): trying with legacy virtio pci.
> Segmentation fault
> 
> $ lspci
> ...
> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
> 
> I've noticed that pci_scan_one() function sets dev->kdrv =
> RTE_KDRV_NONE in lib/librte_eal/linuxapp/eal/eal_pci.c, while the
> driver is detected as virtio-pci
> (from /sys/bus/pci/devices/:00:03.0/driver).
> 
> Is it even possible to run KNI with virtio-net-pci device?
> 
> If it's impossible, then are there other options?
> 

Hi Ruslan,

I tried with same QEMU command with you and able to run KNI sample
application. I am using dpdk on latest master branch.

A few questions:

> EAL: PCI device :00:03.0 on NUMA socket -1
> EAL:   probe driver: 1af4:1000 rte_virtio_pmd
> EAL:   Not managed by a supported kernel driver(0), skipped
1)
According above log, device not bind to a supported driver, this looks
like your problem. Can you please send output of:
dpdk_nic_bind.py -s

2)
Can you run testpmd? Or problem is just KNI?

3)
Are you using dpdk16.04?

Regards,
ferruh

[dpdk-users] my problem about ixgbe_rx_vec_dev_conf_condition_check is called by ixgbe_set_rx_function in ixgbe

[Ferruh Yigit]

On 5/27/2016 2:32 PM,  wrote:
> hello,
> 
> 
> firstly ,gcc version is 3.4.3.
> 
> 
> "PMD: ixgbe_set_rx_function(): Port[0] doesn't meet Vector Rx preconditions 
> or RTE_IXGBE_INC_VECTOR is not enabled" is outputed while running l2fwd  and 
> l3fwd ,i GDB the l2fwd and l3fwd.
> i found ixgbe_rx_vec_dev_conf_condition_check has two definition ,one is:
> int __attribute__((weak))
> ixgbe_rx_vec_dev_conf_condition_check(struct rte_eth_dev __rte_unused *dev)
> {
>   return -1;
> }
> 
> 
> the other is:
> int __attribute__((cold))
> ixgbe_rx_vec_dev_conf_condition_check(struct rte_eth_dev *dev)
> {
> 
> }
> when i GDB the EXE, port_conf.rxmode.hw_ip_checksum was set to 0 .
> ixgbe_set_rx_function call the former at every turn ,i want to know why .
> 
> 
> any help would be appreciated.
> 

I don't know if this is from same person/team, but if not please check
following mail thread for same issue:

http://dpdk.org/ml/archives/dev/2016-June/042323.html

If issue is still valid for you:
Briefly, weak function is defined for the case RTE_IXGBE_INC_VECTOR
config option not enabled.

Which one to link against will be chosen on build time, if app linked
against first one, I would guess RTE_IXGBE_INC_VECTOR is not enabled,
can you please share the output of:
objdump -x librte_pmd_i40e.a| grep i40e_rx_vec_dev_conf_condition_check

Thanks,
ferruh

[dpdk-users] KNI Request handler not called for 10G cards

[Ferruh Yigit]

On 6/20/2016 10:59 AM, Gadre Nayan wrote:
> The request RTE_KNI_REQ_CFG_NETWORK_IF does not get generated on an
> admin link up/down command for 10G interfaces.
> 
> Thanks
> Nayan
> 
> On Mon, Jun 20, 2016 at 3:14 PM, Gadre Nayan  wrote:
>> Hi dpdk users,
>>
>> The callback rte_kni_handle_request does not get called for 10G
>> interfaces on Admin up/down commands (ifconfig vEth0 up/down), but
>> works fine with 1G interfaces.
>>
>> Is this a known issue, if there any patch for this to work with 10G 
>> interfaces.
>>
>> Thanks

Hi Nayan,

That part should be independent from hardware,
RTE_KNI_REQ_CFG_NETWORK_IF sent whenever kni_net_open() called. And
ifconfig xxx up should cause kni_net_open() to be called.

KNI is virtual interface, and rte_kni is driver for it, the
request/response part is handled by rte_kni driver, so shouldn't differ
if DPDK port is 1G or 10G.

Do you debug and observe kni_net_open() not called or do you observe
ifconfig up is not working? If observed second, perhaps the reason can
be something else?

Thanks,
ferruh

[dpdk-users] KNI Threads/Cores

[Ferruh Yigit]

On 6/8/2016 8:48 PM, Cliff Burdick wrote:
> Hi Yigit, when you say "I guess it's possible" is that not common? I
> would think that the amount of traffic people would want to forward to
> Linux for normal DPDK applications would be quite small. If you had to
> have a core dedicated for KNI on each interface that would get very
> wasteful.
> 

Hi Cliff,

I am not aware of any similar usage or I didn't test myself, but
according source code, it should work as described for what you request.

Regards,
ferruh

[dpdk-users] KNI Threads/Cores

[Ferruh Yigit]

On 6/8/2016 5:30 PM, Cliff Burdick wrote:
> Hi, I have an application with two sockets where each core I'm planning to
> transmit and receive a fairly large amount of traffic per core. Each core
> right now handles a single queue of either TX or RX of a given port. Across
> all the cores, I may be processing up to 12 ports. I also need to handle
> things like ARP and ping, so I'm going to add in the KNI driver to handle
> that. Since the amount of traffic I'm expecting that I'll need to forward
> to Linux is very small, it seems like I should be able to dedicate one
> lcore per socket to handle this functionality and have the dataplane cores
> pass the traffic off to this core using rte_kni_tx_burst().
> 
> My question is, first of all, is this possible? It seems like I can
> configure the KNI driver to start in "single thread" mode. From that point,
> I want to initialize one KNI device for each port, and have each kernel
> lcore on each processor handle that traffic. I believe if I call
> rte_kni_alloc with core_id set to the kernel lcore for each device, then in
> the end I'll have something like 6 KNI devices on socket one being handled
> by lcore 0, and 6 KNI devices on socket 2 being handled by lcore 31 as an
> example. Then my threads that are handling the dataplane tx/rx can simply
> be passed a pointer to their respective rte_kni device. Does this sound
> correct?

If rte_kni module used "single thread" mode, kernel core_id is not used
at all. For single thread mode, a single thread created, this is used to
for all kni devices and not able to pin to any specific lcore.

For what you have described, first need to insert module with
kthread_mode=multiple param. This will create a kernel thread per kni
interface. But I guess it is possible to provide same
rte_kni_conf->core_id for some of them, and yes rte_kni_conf->force_pin
is required, otherwise core_id is not useful. According your sample,
first 6 kni devices will have core_id value 0, and other 6 kni devices
will have core_id value 31, with all have force_bind set. This will
create 12 kernel threads, will bind 6 of them to core 0 and other 6 to
core 31.

> 
> Also, the sample says the core affinity needs to be set using taskset. Is
> that already taken care of with conf.core_id in rte_kni_alloc or do I still
> need to set it?
> 
> Thanks
> 

[dpdk-users] Reading DPDK and OVS compatibility

[Ferruh Yigit]

On 5/31/2016 3:18 PM, THEEYAGU S wrote:
> Hi all,
>  I am trying to configure OVS(version 2.5.0) with DPDK(version 16.04),
> but failing with some errors.
>  Can anyone tell me the correct version of DPDK which will be
> compatible with OVS(version 2.5.0)?
>  Thanks in advance.
> 
> 
> Thanks & Regards,
>   Theeyagu S
> 

Hi Theeyagu,

AFAIK, the current master branch of OVS supports DPDK16.04 release, but
released OVS2.5 supports DPDK2.2.

Regards,
ferruh

[dpdk-users] RH 7 Support

[Ferruh Yigit]

On 6/8/2016 2:53 PM, Dorsett, Michal wrote:
> 16.04 states support on RH 6.5.
> However, the errata at https://rhn.redhat.com/errata/RHEA-2015-1136.html 
> announces the addition of the dpdk packages to RH 7.
> 
> 
> 1.   Do RH include latest DPDK releases?
RHEL 7.2 has DPDK2.2 release package

> 
> 2.   Would I get support at DPDK if I use RH 7?
> 
> Thanks,
> 
> Michal
> 

[dpdk-users] Unable to see incoming packets with example KNI application

[Ferruh Yigit]

On 5/11/2016 3:31 PM, Pavey, Nicholas wrote:
> Hi Sakthivel, Andriy,
> 
> Thanks for the email.
> 
> I?m still having trouble, although things do seem to be working better than 
> before.
> 
> It seems that the MAC address suggestion and also the IPs on the same class B 
> network aren?t the root cause.
> 
> 
> Current symptoms
> 
> 
> I rebooted my machine, and took a note of the MAC addresses as the machine 
> started up. It turns out that the KNI virtual interfaces actually did 
> initialize with the correct MAC addresses.
> 
> I also reconfigured the virtual interfaces to be using different network 
> classes, so that should no longer be a problem.
> 
> Something has improved because I?m able to see traffic on the KNI virtual 
> interface with tcpdump, which I was not able to do previously.
> 
> Unfortunately, even though tcpdump is able to see the traffic correctly, it 
> seems that the networking stack isn?t working as I would expect.
> 
> 
> 
> Outbound
> 
> 
> Ping
> 
> 
> I can see outbound ?pings? (with initial ARP requests) with ?tcpdump', and I 
> can see a response coming back. However, the ?ping? application reports 100% 
> packet loss.
> 
> The ARP traffic is definitely getting out, because I see the IP address 
> registered in the router?s ARP cache. Likewise, I see a response to the 
> original ?ping? packet, so the outbound direction seems to be working.
> 
> 
> Inbound
> ===
> 
> The problems seem to be on the inbound side. As we saw above, the outbound 
> side appears to be working reasonably, but I don?t appear to be able to 
> capture inbound packets.
> 
> TCP
> ---
> 
> For example, if I set up a simple ?netcat? listener (using TCP for transport) 
> on the target server:
> 
>   nc ?l 172.25.48.200 9876
> 
> And then attempt to connect to it from another machine, as follows:
> 
>   nc 172.25.48.200 9876
> 
> 
> ?tcpdump? on the target server will show me the incoming ?syn? and a ?syn? 
> retransmission, but there are no outbound ?ack? packets. 
> 
> 
> UDP
> ---
> 
> Similarly, inbound UDP traffic never appears to be routed to the user space 
> application.
> 
> I can counters incrementing on the virtual interface with ?sar ?n DEV 1 100?. 
> ?tcpdump? also shows me the incoming data.
> 
> However, if I look at the UDP stats with ?sar ?n UDP 1 100?, I?m not seeing 
> any packets arriving, even with ?no port? or ?idgmerr?, which I?d normally 
> expect if there?s no listening application bound to the IP address.
> 
> 
> Next steps
> ==
> 
> It almost seems as if the receive side of the network stack simply isn?t 
> seeing the inbound data (regardless of whether it?s ICMP, TCP or UDP) and 
> therefore isn?t sending responses.
> 
> 
> The thing I?m confused about here is how ?tcpdump? is able to see the traffic 
> - after all, if it?s able to see the inbound traffic, then a good part of the 
> RX side of the stack must be working. I?d have thought that if ?tcpdump? can 
> see the traffic, then the rest of the stack should be working too.
> 
> It makes me wondering whether perhaps I?m misunderstanding the purpose of the 
> KNI system?
> 
> My interpretation is that it?s supposed to route traffic from the DPDK into 
> the regular Linux network stack, where it can be used as if it were regular 
> traffic. Do I have that right?
> 
> 
> 
> Do you have any ideas? 
> 
> Thanks,
> 
> 
> Nick
> 
> 
> From:  SAKTHIVEL ANAND S 
> Date:  Wednesday, May 11, 2016 at 3:30 AM
> To:  "Pavey, Nicholas" 
> Subject:  Re: [dpdk-users] Unable to see incoming packets with example KNI 
> application
> 
> 
> Hi
> 
> When you try to send echo packets(outbound), your PC will try to resolve ARP, 
> which it could not complete properly .. due to random mac generation for KNI 
> interface(your KNI interface having different MAC than actual interface).
> 
> 
> After starting KNI app write your hardware address on KNI by doing, "ifconfig 
>  hw ether " and try ping. Let me know the results.
> 
> 
> use  "tcpdump -n -i vEth** -e | grep " 
> 
> Regards
> 
> Sakthivel S OM
> 
> 

Hi Nick,

This may not help for your problem but can work as a reference:

I did test two things with latest code and both worked well:
1- Two NICs , as in your case, with packet generator each end.
Internally created an bridge and added KNI interfaces to bridge, I
observe received packets in generator.

2- Tested with netcat, KNI interface able to send/receive packets, both
for udp and tcp.

I checked for your case, not able to find any obvious error.

Regards,
ferruh

[dpdk-users] DPDK logs enable

[Ferruh Yigit]

On 5/10/2016 3:10 PM, SAKTHIVEL ANAND S wrote:
> Hi
> 
> Any idea, how to enable debug logs for DPDK applications.i have added these
> lines one by one in the main file.
> 1. #define DEBUG
> 2.rte_set_log_level(DEBUG)
> 3.rte_set_log_type(PMOD,1)
> 
> also while run the APP, i have added " --log-type 8 ", but i didnt get
> expected logs. Can some one point me, what did i missed. or how can enable
> DEBUG LOGS.
> 

There is a build time configuration option, CONFIG_RTE_LOG_LEVEL, it
became default to RTE_LOG_INFO recently, please update it to
RTE_LOG_DEBUG to get debug messages.

With optimization enabled, CONFIG_RTE_LOG_LEVEL causes log code lower
than defined value optimized out, so setting runtime log level lower
than CONFIG_RTE_LOG_LEVEL doesn't work.

Regards,
ferruh

[dpdk-users] DPDK: KNI Random MAC generation

[Ferruh Yigit]

On 5/10/2016 8:22 AM, SAKTHIVEL ANAND S wrote:
> Hi
> 
> I know by default KNI will generate random MAC address for newly created
> interfaces. But Is there any way to disable random MAC generation? i am
> using VMXNET3 type Ethernet device.
> 

Hi Sakthivel,

For supported drivers (igb, ixgbe) KNI gets the MAC address from device,
but for rest it generates random address, this is not configurable.

Regards,
ferruh

[dpdk-users] EAL: No probed ethernet devices

[Ferruh Yigit]

On 5/4/2016 3:47 PM, Upendra Pathrikar wrote:
> I am new to DPDK and running 2.2.0,I have bound Intel 82579V nic with
> igb_uio and was working on testpmd application and i am not getting any
> ports for testpmd probably beacuse of EAL: No probed ethernet devices
> error. Please let me know what i'm doing wrong.
> 

Hi Upendra,

You may get this if you are using dpdk as shared library. If so please
try with -d option:

"-d "

Regards,
ferruh

[dpdk-users] mbuf free cnt not decreasing

[Ferruh Yigit]

On 4/18/2016 9:41 AM, Javier Coleto Fern?ndez wrote:
> Hi Andriy,
> 
> Thank you for your answer.
> I've tried what you said and determined it's an mbuf leak, because the
> number of free mbufs doesn't stabilize, even when I increase the number of
> mbufs in the mempool.
> Is there any way to easily debug this mbuf leak? I didn't make the full
> code, so I don't really know where to start looking for leaks.
> 

Can you please check free_q?
kni_fifo_free_count(kni->free_q)

If you call rte_kni_tx_burst() for more than 32 packets at a time, mbufs
may stuck in free_q.

Regards,
ferruh