Re: Accessing TLS for EAL threads

[Pavel Vazharov]

Hi there,

I think the DPDK uses the pthread_ functions to manages its threads. You
can see in `rte_eal_init` in `lib/librte_eal/linux/eal.c` that it "spawns"
the threads using
```
/* create a thread for each lcore */

ret = pthread_create(_config[i].thread_id, NULL,

 eal_thread_loop, NULL);
```
So I think you should be able to use other pthread_ functions without
problems from inside the EAL threads.
Hope that helps.

Regards,
Pavel.

On Thu, Nov 18, 2021 at 9:24 AM Antonio Di Bacco <
antonio.di-ba...@keysight.com> wrote:

> I need to emulate the pthread_setspecific and pthread_getspecific for EAL
> threads. I don’t find any suitable APIs in the DPDK to access the TLS and
> get and set keys.
>
>
>
> I launched a number of threads using the rte_eal_remote launch but I don’t
> find any API that allows me to access the TLS for those threads.
>
>
>
>
>
>
>
>
>

Accessing TLS for EAL threads

[Antonio Di Bacco]

I need to emulate the pthread_setspecific and pthread_getspecific for EAL 
threads. I don't find any suitable APIs in the DPDK to access the TLS and get 
and set keys.

I launched a number of threads using the rte_eal_remote launch but I don't find 
any API that allows me to access the TLS for those threads.

Re: Can the DPDK multi-process feature be used in Docker containers?

[Staffan Wiklund]

Cliff,

Many thanks for your help!
Mapping /var/run/dpdk into both containers as you said made it possible to
use the multi-process feature in Docker containers.

Regards
Staffan

Den tors 18 nov. 2021 kl 07:47 skrev Li Feng :

> On Thu, Nov 18, 2021 at 2:23 PM Cliff Burdick  wrote:
> >
> > I don't remember that being a problem. Are you starting them with two
> different file prefixes and mounting the hugepages directory into both
> containers?
> >
> I'm running the spdk processes, so the hugepage files are created with
> prefix "spdk-", both containers are mounting the same
> /dev/hugepages.
>
>
> >
> > On Wed, Nov 17, 2021, 22:20 Li Feng  wrote:
> >>
> >> On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick 
> wrote:
> >> >
> >> > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into
> both containers from the host. You should be able to do the normal dual
> process methods.
> >> >
> >> By the way, if I want to run two separate dpdk processes in two
> >> different dockers, what should I do?
> >> e.g. Running two processes: examples/dpdk-mp_server
> >> These two dockers have mapped the same /dev/hugepages directories.
> >> I have tested, the dpdk will crash because the hugepages are mixed.
> >>
> >> Thanks.
> >> >
> >> > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund 
> wrote:
> >> >>
> >> >> Hello
> >> >>
> >> >> I wonder if it is possible to use the DPDK multi-process feature in
> Docker containers?
> >> >>
> >> >> That is, can a DPDK application execute in a Docker container and
> share its
> >> >> DPDK memory with another DPDK application executing in another
> Docker container
> >> >> using the DPDK multi-process feature?
> >> >>
> >> >> For example if the DPDK example mp_server executes in one Docker
> container:
> >> >> /examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2
> >> >>
> >> >> and the DPDK example mp_client executes in another Docker container:
> >> >>
> >> >> /examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n
> 0
> >> >>
> >> >> Is this possible to implement?
> >> >>
> >> >> Thanks
> >> >> Staffan
> >> >>
> >> >>
>

Re: Can the DPDK multi-process feature be used in Docker containers?

[Cliff Burdick]

I don't remember that being a problem. Are you starting them with two
different file prefixes and mounting the hugepages directory into both
containers?


On Wed, Nov 17, 2021, 22:20 Li Feng  wrote:

> On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick  wrote:
> >
> > Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both
> containers from the host. You should be able to do the normal dual process
> methods.
> >
> By the way, if I want to run two separate dpdk processes in two
> different dockers, what should I do?
> e.g. Running two processes: examples/dpdk-mp_server
> These two dockers have mapped the same /dev/hugepages directories.
> I have tested, the dpdk will crash because the hugepages are mixed.
>
> Thanks.
> >
> > On Wed, Nov 17, 2021, 05:58 Staffan Wiklund 
> wrote:
> >>
> >> Hello
> >>
> >> I wonder if it is possible to use the DPDK multi-process feature in
> Docker containers?
> >>
> >> That is, can a DPDK application execute in a Docker container and share
> its
> >> DPDK memory with another DPDK application executing in another Docker
> container
> >> using the DPDK multi-process feature?
> >>
> >> For example if the DPDK example mp_server executes in one Docker
> container:
> >> /examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2
> >>
> >> and the DPDK example mp_client executes in another Docker container:
> >>
> >> /examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0
> >>
> >> Is this possible to implement?
> >>
> >> Thanks
> >> Staffan
> >>
> >>
>

Re: Can the DPDK multi-process feature be used in Docker containers?

[Li Feng]

On Thu, Nov 18, 2021 at 2:11 PM Cliff Burdick  wrote:
>
> Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both 
> containers from the host. You should be able to do the normal dual process 
> methods.
>
By the way, if I want to run two separate dpdk processes in two
different dockers, what should I do?
e.g. Running two processes: examples/dpdk-mp_server
These two dockers have mapped the same /dev/hugepages directories.
I have tested, the dpdk will crash because the hugepages are mixed.

Thanks.
>
> On Wed, Nov 17, 2021, 05:58 Staffan Wiklund  wrote:
>>
>> Hello
>>
>> I wonder if it is possible to use the DPDK multi-process feature in Docker 
>> containers?
>>
>> That is, can a DPDK application execute in a Docker container and share its
>> DPDK memory with another DPDK application executing in another Docker 
>> container
>> using the DPDK multi-process feature?
>>
>> For example if the DPDK example mp_server executes in one Docker container:
>> /examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2
>>
>> and the DPDK example mp_client executes in another Docker container:
>>
>> /examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0
>>
>> Is this possible to implement?
>>
>> Thanks
>> Staffan
>>
>>

Re: Can the DPDK multi-process feature be used in Docker containers?

[Cliff Burdick]

Yes. Try mapping the dpdk metadata directory (/var/run/dpdk) into both
containers from the host. You should be able to do the normal dual process
methods.


On Wed, Nov 17, 2021, 05:58 Staffan Wiklund  wrote:

> Hello
>
> I wonder if it is possible to use the DPDK multi-process feature in Docker
> containers?
>
> That is, can a DPDK application execute in a Docker container and share its
> DPDK memory with another DPDK application executing in another Docker
> container
> using the DPDK multi-process feature?
>
> For example if the DPDK example mp_server executes in one Docker container:
> /examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2
>
> and the DPDK example mp_client executes in another Docker container:
>
> /examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0
>
> Is this possible to implement?
>
> Thanks
> Staffan
>
>
>

Re: release schedule change proposal

[Ajit Khaparde]

On Mon, Nov 15, 2021 at 6:58 AM Thomas Monjalon  wrote:
>
> For the last 5 years, DPDK was doing 4 releases per year,
> in February, May, August and November (the LTS one):
> .02   .05   .08   .11 (LTS)
>
> This schedule has multiple issues:
> - clash with China's Spring Festival
> - too many rushes, impacting maintainers & testers
> - not much buffer, impacting proposal period
>
> I propose to switch to a new schedule with 3 releases per year:
> .03  .07  .11 (LTS)
>
> New LTS branch would start at the same time of the year as before.
> There would be one less intermediate release during spring/summer:
> .05 and .08 intermediate releases would become a single .07.
> I think it has almost no impact for the users.
> This change could be done starting next year.
>
> In details, this is how we could extend some milestones:
>
> ideal schedule so far (in 13 weeks):
> proposal deadline: 4
> rc1 - API freeze: 5
> rc2 - PMD features freeze: 2
> rc3 - app features freeze: 1
> rc4 - last chance to fix: 1
> release: 0
>
> proposed schedule (in 17 weeks):
> proposal deadline: 4
> rc1 - API freeze: 7
> rc2 - PMD features freeze: 3
> rc3 - app features freeze: 1
> rc4 - more fixes: 1
> rc5 - last chance buffer: 1
> release: 0
+1

>
> Opinions?
>
>

Can the DPDK multi-process feature be used in Docker containers?

[Staffan Wiklund]

Hello

I wonder if it is possible to use the DPDK multi-process feature in Docker
containers?

That is, can a DPDK application execute in a Docker container and share its
DPDK memory with another DPDK application executing in another Docker
container
using the DPDK multi-process feature?

For example if the DPDK example mp_server executes in one Docker container:
/examples/dpdk-mp_server -l 1-2 -n 4 -- -p 3 -n 2

and the DPDK example mp_client executes in another Docker container:

/examples/dpdk-mp_client -l 3 -n 4 --proc-type=auto -- -n 0

Is this possible to implement?

Thanks
Staffan

RE: Pdump Didn't capture the packet

[Pattan, Reshma]

From: 廖書華 
Sent: Wednesday, November 17, 2021 3:11 AM
To: Pattan, Reshma 
Cc: users@dpdk.org
Subject: Re: Pdump Didn't capture the packet

Dear,

Thank you for the information !
I rerun the primary application with the other terminal running pdump, however 
it still can't capture the packet.
As for the logs you mentioned that is printed after the primary application 
terminates. Also, for the primary application log, I can make sure it transmits 
and receives the packet, but pdump still can't capture it.

[Reshma]: Are the pcap captured files empty?  That might need some debugging I 
guess. When you stop the pdump application you should see below stats.
If you see 0 “packets dequeued”, that means there is no Packets received from 
primary to pdump application.
So,  nee to  debug the pdump library what is happening , why primary 
application packets are not send to pdump application. You can try to debugging 
the pdump library and pdump
application.


packets dequeued:

packets transmitted to vdev:

packets freed:
- For the log of pdump
[oran@localhost pdump]$ sudo ./dpdk-pdump --file-prefix wls_1 -- --pdump 
'port=0,queue=*,tx-dev=/home/oran/Music/tx.pcap,rx-dev=/home/oran/Music/rx.pcap'

Also, do you know whether pdump can capture the virtual function packet or not 
? (We bind to vfio-pci)
[Reshma]: I have not tested with VFs ,  I always used PFs.
Best Regards,
Shu-hua, Liao

Pattan, Reshma mailto:reshma.pat...@intel.com>> 於 
2021年11月15日 週一 下午4:58寫道：


From: 廖書華 mailto:sim860...@gmail.com>>
Sent: Monday, November 15, 2021 5:39 AM
To: users@dpdk.org
Cc: 林庭安 mailto:lingwan...@gmail.com>>
Subject: Pdump Didn't capture the packet

Dear all,

Currently, I want to use pdump to capture our DPDK application, however for the 
pdump side, unfortunately, didn't capture any packet, also pdump didn't print 
any error. While for our application, it also didn't print any log related to 
pdump.
- Here's the log of pdump
[oran@localhost pdump]$ sudo ./dpdk-pdump --file-prefix wls_1 -- --pdump 
'port=0,queue=*,tx-dev=/home/oran/Music/tx.pcap,rx-dev=/home/oran/Music/rx.pcap'



Primary process is no longer active, exiting...

[Reshma]: From this log it is clear that, primary application is not running.  
Rerun the primary application and in other terminal run the pdump application.

Best Regards,
Shu-hua, Liao

Accessing EAL threads TLS

[Antonio Di Bacco]

I need to emulate the pthread_setspecific and pthread_getspecific for EAL 
threads. I don't find any suitable APIs in the DPDK to access the TLS and get 
and set keys.

I launched a number of threads using the rte_eal_remote launch but I don't find 
any API that allows me to access the TLS for those threads.