Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Kevin Traynor]

On 10/25/2016 03:15 PM, Bodireddy, Bhanuprakash wrote:
>> -Original Message-
>> From: Aaron Conole [mailto:acon...@redhat.com]
>> Sent: Tuesday, October 25, 2016 2:36 PM
>> To: Bodireddy, Bhanuprakash <bhanuprakash.bodire...@intel.com>
>> Cc: Christian Ehrhardt <christian.ehrha...@canonical.com>; ychen
>> <ychen103...@163.com>; dev@openvswitch.org
>> Subject: Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master
>> lcore?
>>
>> "Bodireddy, Bhanuprakash" <bhanuprakash.bodire...@intel.com> writes:
>>
>>>>
>>>> ++ pidof ovs-vswitchd
>>>> + ps -To tid,pid,psr,comm -p 25932
>>>>  TID   PID PSR COMMAND
>>>> 25932 25932   1 ovs-vswitchd
>>>> 25934 25932   0 eal-intr-thread
>>>> 25935 25932   1 dpdk_watchdog1
>>>> 25936 25932   2 vhost_thread2
>>>> 25937 25932   1 pdump-thread
>>>> 25938 25932   8 urcu3
>>>> 25939 25932   2 ovs-vswitchd
>>>> 25959 25932   1 ct_clean4
>>>> 26013 25932   3 handler18
>>>> 26014 25932   9 handler17
>>>> 26015 25932   3 handler20
>>>> 26016 25932  11 handler19
>>>> 26022 25932   4 handler21
>>>> 26028 25932   9 handler22
>>>> 26033 25932  10 handler23
>>>> 26037 25932  10 handler24
>>>> 26042 25932   9 revalidator25
>>>> 26045 25932  10 revalidator26
>>>> 26046 25932   0 revalidator27
>>>> 26054 25932   2 revalidator28
>>>> 26143 25932   5 pmd29
>>>> 26144 25932   4 pmd30
>>>
>>> This is for the reason that 'dpdk-lcore-mask' hasn't been specified in
>>> the above case.
>>> If one specify dpdk-lcore-mask=, all the
>>> threads(revalidator/handler/ovs-vswitchd/urc) will automatically be
>>> pinned to the lowest core of the 'mask'.
>>>
>>> For example with 'ovs-vsctl --no-wait set Open_vSwitch .
>>> other_config:dpdk-lcore-mask=f00', I would expect the threads to float
>>> between cores 8 and 11, but all the threads get pinned to lowest core
>>> of the core-mask which is core 8.  Hope this clears the confusion.
>>
>> I'm sorry, I don't understand this behaviors.  I don't see that this is 
>> anything to
>> do with ovs, specifically.  Meaning, when that option is specified, we pass 
>> it to
>> the dpdk library directly, and take no actions as far as thread binding is
>> concerned.  I have to admit, I've not delved that deeply into the dpdk side.
> 
> Completely agree Aaron. This isn't anything to do with OVS.  This is the 
> current behavior of DPDK and more to do with how
> it parses the coremask. For the given coremask, DPDK assumes the lowest core 
> of the mask as 'mastercore/control core'
> and pins the control threads to this core, leaving the remaining cores for 
> actual packet forwarding. 

I take the opposite view :) this is everything to do with OVS as it
doesn't use DPDK for multi-threading, it creates its own threads. We
strip the other bits from the mask before it is sent to EAL init as
otherwise they will create worker threads which OVS will never assign
anything to. Handler/Revalidator threads then get create and pinned to
the master thread.

But the above is *not* the default behavior and the default behavior is
much nicer IMO. If dpdk-lcore-mask is not specified the
handler/revalidator type threads will float on the cores that vswitchd
runs on - similar to OVS without DPDK.

> 
> This behavior is reproducible with DPDK sample app.
> 
> Regards,
> Bhanu Prakash. 
> 
>>
>> Do you have the output from the EAL logs?  There are logs which state what
>> dpdk thinks it should have set for an affinity.  Perhaps those shed some 
>> light?
>>
>> -Aaron
> ___
> dev mailing list
> dev@openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
> 

___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Bodireddy, Bhanuprakash]

>-Original Message-
>From: Aaron Conole [mailto:acon...@redhat.com]
>Sent: Tuesday, October 25, 2016 2:36 PM
>To: Bodireddy, Bhanuprakash <bhanuprakash.bodire...@intel.com>
>Cc: Christian Ehrhardt <christian.ehrha...@canonical.com>; ychen
><ychen103...@163.com>; dev@openvswitch.org
>Subject: Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master
>lcore?
>
>"Bodireddy, Bhanuprakash" <bhanuprakash.bodire...@intel.com> writes:
>
>>>
>>>++ pidof ovs-vswitchd
>>>+ ps -To tid,pid,psr,comm -p 25932
>>>  TID   PID PSR COMMAND
>>>25932 25932   1 ovs-vswitchd
>>>25934 25932   0 eal-intr-thread
>>>25935 25932   1 dpdk_watchdog1
>>>25936 25932   2 vhost_thread2
>>>25937 25932   1 pdump-thread
>>>25938 25932   8 urcu3
>>>25939 25932   2 ovs-vswitchd
>>>25959 25932   1 ct_clean4
>>>26013 25932   3 handler18
>>>26014 25932   9 handler17
>>>26015 25932   3 handler20
>>>26016 25932  11 handler19
>>>26022 25932   4 handler21
>>>26028 25932   9 handler22
>>>26033 25932  10 handler23
>>>26037 25932  10 handler24
>>>26042 25932   9 revalidator25
>>>26045 25932  10 revalidator26
>>>26046 25932   0 revalidator27
>>>26054 25932   2 revalidator28
>>>26143 25932   5 pmd29
>>>26144 25932   4 pmd30
>>
>> This is for the reason that 'dpdk-lcore-mask' hasn't been specified in
>> the above case.
>> If one specify dpdk-lcore-mask=, all the
>> threads(revalidator/handler/ovs-vswitchd/urc) will automatically be
>> pinned to the lowest core of the 'mask'.
>>
>> For example with 'ovs-vsctl --no-wait set Open_vSwitch .
>> other_config:dpdk-lcore-mask=f00', I would expect the threads to float
>> between cores 8 and 11, but all the threads get pinned to lowest core
>> of the core-mask which is core 8.  Hope this clears the confusion.
>
>I'm sorry, I don't understand this behaviors.  I don't see that this is 
>anything to
>do with ovs, specifically.  Meaning, when that option is specified, we pass it 
>to
>the dpdk library directly, and take no actions as far as thread binding is
>concerned.  I have to admit, I've not delved that deeply into the dpdk side.

Completely agree Aaron. This isn't anything to do with OVS.  This is the 
current behavior of DPDK and more to do with how
it parses the coremask. For the given coremask, DPDK assumes the lowest core of 
the mask as 'mastercore/control core'
and pins the control threads to this core, leaving the remaining cores for 
actual packet forwarding. 

This behavior is reproducible with DPDK sample app.

Regards,
Bhanu Prakash. 

>
>Do you have the output from the EAL logs?  There are logs which state what
>dpdk thinks it should have set for an affinity.  Perhaps those shed some light?
>
>-Aaron
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Aaron Conole]

"Bodireddy, Bhanuprakash"  writes:

>>
>>++ pidof ovs-vswitchd
>>+ ps -To tid,pid,psr,comm -p 25932
>>  TID   PID PSR COMMAND
>>25932 25932   1 ovs-vswitchd
>>25934 25932   0 eal-intr-thread
>>25935 25932   1 dpdk_watchdog1
>>25936 25932   2 vhost_thread2
>>25937 25932   1 pdump-thread
>>25938 25932   8 urcu3
>>25939 25932   2 ovs-vswitchd
>>25959 25932   1 ct_clean4
>>26013 25932   3 handler18
>>26014 25932   9 handler17
>>26015 25932   3 handler20
>>26016 25932  11 handler19
>>26022 25932   4 handler21
>>26028 25932   9 handler22
>>26033 25932  10 handler23
>>26037 25932  10 handler24
>>26042 25932   9 revalidator25
>>26045 25932  10 revalidator26
>>26046 25932   0 revalidator27
>>26054 25932   2 revalidator28
>>26143 25932   5 pmd29
>>26144 25932   4 pmd30
>
> This is for the reason that 'dpdk-lcore-mask' hasn't been specified in
> the above case.
> If one specify dpdk-lcore-mask=, all the
> threads(revalidator/handler/ovs-vswitchd/urc) will automatically be
> pinned to the lowest core of the 'mask'.
>
> For example with 'ovs-vsctl --no-wait set Open_vSwitch
> . other_config:dpdk-lcore-mask=f00', I would expect the threads to
> float between cores 8 and 11, but all the threads get pinned to lowest
> core of the core-mask which is core 8.  Hope this clears the
> confusion.

I'm sorry, I don't understand this behaviors.  I don't see that this is
anything to do with ovs, specifically.  Meaning, when that option is
specified, we pass it to the dpdk library directly, and take no actions
as far as thread binding is concerned.  I have to admit, I've not delved
that deeply into the dpdk side.

Do you have the output from the EAL logs?  There are logs which state
what dpdk thinks it should have set for an affinity.  Perhaps those shed
some light?

-Aaron
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Bodireddy, Bhanuprakash]

>
>sorry, I am still confused.
>when I launched vswitchd with dpdk-init=false, all the threads changed their
>pinned cores as time goes by.
>The following output is from ovs version 2.6 with dpdk-init=false ps -To
>tid,pid,psr,comm -p 5922
>   TIDPID PSR COMMAND
>  5922   5922  12 ovs-vswitchd
>  5934   5922  24 urcu5 ==>does not follow main the ovs-vswitchd
>  6917   5922   0 handler98
>  6918   5922  24 handler100
>  6919   5922   0 handler99
>  6920   5922   0 handler101
>  6921   5922   0 handler102
>  6922   5922   2 handler103
>6952   5922   0 revalidator133
>  6953   5922  10 revalidator134
>  6954   5922  38 revalidator135  ==>seems these threads can be pinned on any
>cores, just like CPU auto scheduled
>  6955   5922  26 revalidator136
>  6956   5922  40 revalidator137
>  6957   5922  22 revalidator138
>
>
>after somtime,
>5922   5922   7 ovs-vswitchd   ==>changed from core 12 to core 7
>  5934   5922  18 urcu5
>  6917   5922   0 handler98
>  6918   5922   0 handler100
>  6919   5922   0 handler99
>  6920   5922   0 handler101
>  6921   5922   0 handler102
>  6922   5922   0 handler103
>6952   5922   3 revalidator133
>  6953   5922  35 revalidator134
>  6954   5922  27 revalidator135
>  6955   5922  21 revalidator136
>  6956   5922   9 revalidator137
>  6957   5922  33 revalidator138
>
>
>When launch vswitchd with dpdk-init as true, all the handler threads and
>revalidator threads are pinned to master lcore and can not change their
>pinned cores when time goes by.
>so here is my problem:
>1. with the same ovs code, there is different phenomenon for ovs threads
>with dpdk inited have different values?

Yes, with dpdk_init=true, the dpdk initialization will be done and there is a 
mechanism to determine and pin
the threads to the cores. If 'dpdk-lcore-mask' is specified the threads would 
be pinned to the lowest core of 
the mask. This is more dpdk library behavior as DPDK wants to use the lowest 
core of the mask as control core
and the rest of the cores as packet forwarding cores. 

If one doesn't specifiy the dpdk-lcore-mask, the thread will not be pinned 
explicitly to any core. 
To check this run taskset on the respective threads to retrieve the affinity In 
this case you can see
the threads are free to float on any core between 0 and 27. For example.

$ ps -eLo tid,psr,comm | grep -e revalidator -e handler -e ovs -e pmd -e urc -e 
eal   
   99078  17 ovs-vswitchd
   99137  11 handler33
   99157   6 revalidator53

$taskset -cp 99078
pid 99078's current affinity list: 0-27

$ taskset -cp 99137
pid 99137's current affinity list: 0-27

$ taskset -cp 99157
pid 99157's current affinity list: 0-27

Hope this clears your confusion.

Regards,
Bhanu Prakash.

>2.with dpdk-init=true, if all the other threads share the same logical core, 
>then
>for the performance meaning, is it not friendly?
>
>
>
>
>
>
>
>
>
>
>At 2016-10-25 01:39:13, "Kevin Traynor"  wrote:
>>On 10/24/2016 11:55 AM, ychen wrote:
>>> hi, I am a freshman to ovs DPDK, when I tried to launch ovs with dpdk
>>> inited, I found that all the ovs threads are pinned to master lcore, but I
>can't find any code for setting the affinity of the specified thread.
>>
>>On older versions of OVS you you can set the affinity via the -c
>>0x vswitchd dpdk cmd line arg. It will use the lsb only. For the
>>latest versions, you can set this through OVSDB - alternatively you can
>>not set it and by default the non-pmd threads will float on the cores
>>that vswitchd runs on.
>>
>>> Here is my configuration:
>>> lscpu
>>> Architecture:  x86_64
>>> CPU op-mode(s):32-bit, 64-bit
>>> Byte Order:Little Endian
>>> CPU(s):48
>>> On-line CPU(s) list:   0-47
>>> Thread(s) per core:2
>>> Core(s) per socket:12
>>> Socket(s): 2
>>> NUMA node(s):  2
>>> Vendor ID: GenuineIntel
>>> CPU family:6
>>> Model: 63
>>> Stepping:  2
>>> CPU MHz:   2599.988
>>> BogoMIPS:  4600.75
>>> Virtualization:VT-x
>>> L1d cache: 32K
>>> L1i cache: 32K
>>> L2 cache:  256K
>>> L3 cache:  30720K
>>> NUMA node0 CPU(s):
>0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46
>>> NUMA node1 CPU(s):
>1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47
>>>
>>>
>>> ovs-vsctl  list open_vswitch
>>> _uuid   : 61d9066b-178e-4672-b8b3-9dcb5587565d
>>> bridges : [cc2605fb-fcbb-4627-8834-080c43534119]
>>> cur_cfg : 68
>>> datapath_types  : [netdev, system]
>>> db_version  : []
>>> external_ids: {}
>>> iface_types : [dpdk, dpdkr, dpdkvhostuser, dpdkvhostuserclient,
>geneve, gre, internal, ipsec_gre, lisp, patch, stt, system, tap, vxlan]
>>> manager_options : []
>>> next_cfg: 68
>>> other_config: {dpdk-init="true", dpdk-lcore-mask="0xf", dpdk-socket-

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Bodireddy, Bhanuprakash]

>
>++ pidof ovs-vswitchd
>+ ps -To tid,pid,psr,comm -p 25932
>  TID   PID PSR COMMAND
>25932 25932   1 ovs-vswitchd
>25934 25932   0 eal-intr-thread
>25935 25932   1 dpdk_watchdog1
>25936 25932   2 vhost_thread2
>25937 25932   1 pdump-thread
>25938 25932   8 urcu3
>25939 25932   2 ovs-vswitchd
>25959 25932   1 ct_clean4
>26013 25932   3 handler18
>26014 25932   9 handler17
>26015 25932   3 handler20
>26016 25932  11 handler19
>26022 25932   4 handler21
>26028 25932   9 handler22
>26033 25932  10 handler23
>26037 25932  10 handler24
>26042 25932   9 revalidator25
>26045 25932  10 revalidator26
>26046 25932   0 revalidator27
>26054 25932   2 revalidator28
>26143 25932   5 pmd29
>26144 25932   4 pmd30

This is for the reason that 'dpdk-lcore-mask' hasn't been specified in the 
above case. 
If one specify dpdk-lcore-mask=, all the 
threads(revalidator/handler/ovs-vswitchd/urc) will automatically be pinned to 
the lowest core of the 'mask'. 

For example with 'ovs-vsctl --no-wait set Open_vSwitch . 
other_config:dpdk-lcore-mask=f00', I would expect the threads to float between 
cores 8 and 11, but all the threads get pinned to lowest core of the core-mask 
which is core 8.  Hope this clears the confusion. 

$ ps -eLo tid,psr,comm | grep -e revalidator -e handler -e ovs -e pmd -e urc -e 
eal
 98443  22 ovsdb-server
 98454   8 ovs-vswitchd
 98465   8 eal-intr-thread
 98471   8 urcu3
 98520   8 handler60
 98521   8 handler59
 98522   8 handler58
 98523   8 handler57
 98524   8 handler43
 98525   8 handler44
 98526   8 handler46
 98527   8 handler47
 98528   8 handler48
 98529   8 handler41
 98530   8 handler55
 98531   8 handler49
 98532   8 handler33
 98533   8 handler40
 98534   8 handler34
 98535   8 handler35
 98536   8 handler50
 98537   8 handler36
 98538   8 handler51
 98539   8 handler56
 98540   8 revalidator37
 98541   8 revalidator52
 98542   8 revalidator42
 98543   8 revalidator38
 98544   8 revalidator39
 98545   8 revalidator45
 98546   8 revalidator53
 98547   8 revalidator54
 98549   4 pmd61

Regards,
Bhanu Prakash. 
>
>
>--
>Christian Ehrhardt
>Software Engineer, Ubuntu Server
>Canonical Ltd
>___
>dev mailing list
>dev@openvswitch.org
>http://openvswitch.org/mailman/listinfo/dev
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Christian Ehrhardt]

On Tue, Oct 25, 2016 at 4:08 AM, ychen  wrote:

> When launch vswitchd with dpdk-init as true, all the handler threads and
> revalidator threads are pinned to master lcore and can not change their
> pinned cores when time goes by.
> so here is my problem:
> 1. with the same ovs code, there is different phenomenon for ovs threads
> with dpdk inited have different value?
> 2.with dpdk-init=true, if all the other threads share the same logical
> core, then for the performance meaning, is it not friendly?
>

Hi,
since it is easy for me to pick up the same data for my tests I run anyway
I just wanted to share what I see for OVS-DPDK in my cases.
I'm initializing dpdk (other_config:dpdk-init=true), but otherwise use
mostly defaults.
There are more complex cases, but here already I think I don't see your
"all on the lcore cpu" case.
Using OVS 2.6 and DPDK 16.07 as provided in Ubuntu Yakkety.


+ ovs-vsctl show
dbf1f7af-c368-4fac-9151-e9cf6af4a15b
Bridge "ovsdpdkbr0"
Port "dpdk0"
Interface "dpdk0"
type: dpdk
Port "ovsdpdkbr0"
Interface "ovsdpdkbr0"
type: internal
Port "vhost-user-2"
Interface "vhost-user-2"
type: dpdkvhostuser
Port "vhost-user-1"
Interface "vhost-user-1"
type: dpdkvhostuser
ovs_version: "2.6.0"

++ pidof ovs-vswitchd
+ ps -To tid,pid,psr,comm -p 25932
  TID   PID PSR COMMAND
25932 25932   1 ovs-vswitchd
25934 25932   0 eal-intr-thread
25935 25932   1 dpdk_watchdog1
25936 25932   2 vhost_thread2
25937 25932   1 pdump-thread
25938 25932   8 urcu3
25939 25932   2 ovs-vswitchd
25959 25932   1 ct_clean4
26013 25932   3 handler18
26014 25932   9 handler17
26015 25932   3 handler20
26016 25932  11 handler19
26022 25932   4 handler21
26028 25932   9 handler22
26033 25932  10 handler23
26037 25932  10 handler24
26042 25932   9 revalidator25
26045 25932  10 revalidator26
26046 25932   0 revalidator27
26054 25932   2 revalidator28
26143 25932   5 pmd29
26144 25932   4 pmd30



-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd
___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[ychen]

sorry, I am still confused.
when I launched vswitchd with dpdk-init=false, all the threads changed their 
pinned cores as time goes by.
The following output is from ovs version 2.6 with dpdk-init=false
ps -To tid,pid,psr,comm -p 5922
   TIDPID PSR COMMAND
  5922   5922  12 ovs-vswitchd
  5934   5922  24 urcu5 ==>does not follow main the ovs-vswitchd
  6917   5922   0 handler98
  6918   5922  24 handler100
  6919   5922   0 handler99
  6920   5922   0 handler101
  6921   5922   0 handler102
  6922   5922   2 handler103
6952   5922   0 revalidator133
  6953   5922  10 revalidator134
  6954   5922  38 revalidator135  ==>seems these threads can be pinned on any 
cores, just like CPU auto scheduled 
  6955   5922  26 revalidator136
  6956   5922  40 revalidator137
  6957   5922  22 revalidator138


after somtime,
5922   5922   7 ovs-vswitchd   ==>changed from core 12 to core 7
  5934   5922  18 urcu5
  6917   5922   0 handler98
  6918   5922   0 handler100
  6919   5922   0 handler99
  6920   5922   0 handler101
  6921   5922   0 handler102
  6922   5922   0 handler103
6952   5922   3 revalidator133
  6953   5922  35 revalidator134
  6954   5922  27 revalidator135
  6955   5922  21 revalidator136
  6956   5922   9 revalidator137
  6957   5922  33 revalidator138


When launch vswitchd with dpdk-init as true, all the handler threads and 
revalidator threads are pinned to master lcore and can not change their pinned 
cores when time goes by.
so here is my problem:
1. with the same ovs code, there is different phenomenon for ovs threads with 
dpdk inited have different value?
2.with dpdk-init=true, if all the other threads share the same logical core, 
then for the performance meaning, is it not friendly?










At 2016-10-25 01:39:13, "Kevin Traynor"  wrote:
>On 10/24/2016 11:55 AM, ychen wrote:
>> hi, I am a freshman to ovs DPDK, when I tried to launch ovs with dpdk 
>> inited, I found that all the ovs threads are pinned to master lcore,
>> but I can't find any code for setting the affinity of the specified thread.
>
>On older versions of OVS you you can set the affinity via the -c
>0x vswitchd dpdk cmd line arg. It will use the lsb only. For the
>latest versions, you can set this through OVSDB - alternatively you can
>not set it and by default the non-pmd threads will float on the cores
>that vswitchd runs on.
>
>> Here is my configuration:
>> lscpu
>> Architecture:  x86_64
>> CPU op-mode(s):32-bit, 64-bit
>> Byte Order:Little Endian
>> CPU(s):48
>> On-line CPU(s) list:   0-47
>> Thread(s) per core:2
>> Core(s) per socket:12
>> Socket(s): 2
>> NUMA node(s):  2
>> Vendor ID: GenuineIntel
>> CPU family:6
>> Model: 63
>> Stepping:  2
>> CPU MHz:   2599.988
>> BogoMIPS:  4600.75
>> Virtualization:VT-x
>> L1d cache: 32K
>> L1i cache: 32K
>> L2 cache:  256K
>> L3 cache:  30720K
>> NUMA node0 CPU(s): 
>> 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46
>> NUMA node1 CPU(s): 
>> 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47
>> 
>> 
>> ovs-vsctl  list open_vswitch
>> _uuid   : 61d9066b-178e-4672-b8b3-9dcb5587565d
>> bridges : [cc2605fb-fcbb-4627-8834-080c43534119]
>> cur_cfg : 68
>> datapath_types  : [netdev, system]
>> db_version  : []
>> external_ids: {}
>> iface_types : [dpdk, dpdkr, dpdkvhostuser, dpdkvhostuserclient, 
>> geneve, gre, internal, ipsec_gre, lisp, patch, stt, system, tap, vxlan]
>> manager_options : []
>> next_cfg: 68
>> other_config: {dpdk-init="true", dpdk-lcore-mask="0xf", 
>> dpdk-socket-mem="1024,1024", pmd-cpu-mask="f0"}
>> ovs_version : []
>> ssl : []
>> statistics  : {}
>> system_type : []
>> system_version  : []
>> 
>> 
>>  ps -To tid,pid,psr,comm -p 28262
>>TIDPID PSR COMMAND
>>  28262  28262   0 ovs-vswitchd
>>  28263  28262  39 vfio-sync
>>  28297  28262   0 eal-intr-thread
>>  28298  28262   1 lcore-slave-1
>>  28299  28262   2 lcore-slave-2
>>  28300  28262   3 lcore-slave-3
>>  28301  28262   0 dpdk_watchdog2
>>  28302  28262   0 vhost_thread1
>>  28303  28262   0 pdump-thread
>>  28304  28262   0 ct_clean3
>>  28305  28262   0 urcu4
>>  28744  28262   0 handler101
>>  28745  28262   0 handler100
>>  28746  28262   0 handler99
>>  28747  28262   0 handler98
>>  28748  28262   0 handler95
>>  28749  28262   0 handler77
>>  28750  28262   0 handler79
>>  28751  28262   0 handler80
>>  28752  28262   0 handler81
>>  28753  28262   0 handler73
>>  28756  28262   0 handler92
>>  28757  28262   0 handler82
>>  28758  28262   0 handler96
>>  28759  28262   0 handler71
>>  28760  28262   0 handler61
>>  28761  28262   0 handler62
>>  28762  28262   0 handler83
>>  28763  28262   0 

Re: [ovs-dev] [ovs dpdk] why all the ovs threads pinned to master lcore?

[Kevin Traynor]

On 10/24/2016 11:55 AM, ychen wrote:
> hi, I am a freshman to ovs DPDK, when I tried to launch ovs with dpdk inited, 
> I found that all the ovs threads are pinned to master lcore,
> but I can't find any code for setting the affinity of the specified thread.

On older versions of OVS you you can set the affinity via the -c
0x vswitchd dpdk cmd line arg. It will use the lsb only. For the
latest versions, you can set this through OVSDB - alternatively you can
not set it and by default the non-pmd threads will float on the cores
that vswitchd runs on.

> Here is my configuration:
> lscpu
> Architecture:  x86_64
> CPU op-mode(s):32-bit, 64-bit
> Byte Order:Little Endian
> CPU(s):48
> On-line CPU(s) list:   0-47
> Thread(s) per core:2
> Core(s) per socket:12
> Socket(s): 2
> NUMA node(s):  2
> Vendor ID: GenuineIntel
> CPU family:6
> Model: 63
> Stepping:  2
> CPU MHz:   2599.988
> BogoMIPS:  4600.75
> Virtualization:VT-x
> L1d cache: 32K
> L1i cache: 32K
> L2 cache:  256K
> L3 cache:  30720K
> NUMA node0 CPU(s): 
> 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46
> NUMA node1 CPU(s): 
> 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47
> 
> 
> ovs-vsctl  list open_vswitch
> _uuid   : 61d9066b-178e-4672-b8b3-9dcb5587565d
> bridges : [cc2605fb-fcbb-4627-8834-080c43534119]
> cur_cfg : 68
> datapath_types  : [netdev, system]
> db_version  : []
> external_ids: {}
> iface_types : [dpdk, dpdkr, dpdkvhostuser, dpdkvhostuserclient, 
> geneve, gre, internal, ipsec_gre, lisp, patch, stt, system, tap, vxlan]
> manager_options : []
> next_cfg: 68
> other_config: {dpdk-init="true", dpdk-lcore-mask="0xf", 
> dpdk-socket-mem="1024,1024", pmd-cpu-mask="f0"}
> ovs_version : []
> ssl : []
> statistics  : {}
> system_type : []
> system_version  : []
> 
> 
>  ps -To tid,pid,psr,comm -p 28262
>TIDPID PSR COMMAND
>  28262  28262   0 ovs-vswitchd
>  28263  28262  39 vfio-sync
>  28297  28262   0 eal-intr-thread
>  28298  28262   1 lcore-slave-1
>  28299  28262   2 lcore-slave-2
>  28300  28262   3 lcore-slave-3
>  28301  28262   0 dpdk_watchdog2
>  28302  28262   0 vhost_thread1
>  28303  28262   0 pdump-thread
>  28304  28262   0 ct_clean3
>  28305  28262   0 urcu4
>  28744  28262   0 handler101
>  28745  28262   0 handler100
>  28746  28262   0 handler99
>  28747  28262   0 handler98
>  28748  28262   0 handler95
>  28749  28262   0 handler77
>  28750  28262   0 handler79
>  28751  28262   0 handler80
>  28752  28262   0 handler81
>  28753  28262   0 handler73
>  28756  28262   0 handler92
>  28757  28262   0 handler82
>  28758  28262   0 handler96
>  28759  28262   0 handler71
>  28760  28262   0 handler61
>  28761  28262   0 handler62
>  28762  28262   0 handler83
>  28763  28262   0 handler63
>  28764  28262   0 handler84
>  28765  28262   0 handler93
>  28766  28262   0 handler64
>  28767  28262   0 handler85
>  28768  28262   0 handler74
>  28769  28262   0 handler65
>  28770  28262   0 handler66
>  28771  28262   0 handler78
>  28772  28262   0 handler86
>  28773  28262   0 handler87
>  28774  28262   0 handler97
>  28775  28262   0 handler88
>  28776  28262   0 handler56
>  28777  28262   0 handler76
>  28778  28262   0 handler67
>  28779  28262   0 handler60
>  28780  28262   0 handler68
>  28781  28262   0 revalidator75
>  28782  28262   0 revalidator57
>  28783  28262   0 revalidator89
>  28784  28262   0 revalidator69
>  28785  28262   0 revalidator54
>  28786  28262   0 revalidator90
>  28787  28262   0 revalidator55
>  28788  28262   0 revalidator58
>  28789  28262   0 revalidator59
>  28790  28262   0 revalidator70
>  28791  28262   0 revalidator94
>  28792  28262   0 revalidator91
>  28793  28262   0 revalidator72
>  28827  28262   4 pmd103
>  28829  28262   6 pmd102
> ___
> dev mailing list
> dev@openvswitch.org
> http://openvswitch.org/mailman/listinfo/dev
> 

___
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev