HI Chao,
Found a mmap MAP_SHARED issue in multi-process, the address returned seems to
be a local malloc address, conflicts with other variable in secondary process,
code:
Mmap in primary process initiates a shared address:
addr = mmap(NULL, page_size, PROT_WRITE, MAP_SHARED, cmd_fd, offset);
At secondary process, the remapping of primary address with same offset and
length succeed, but it corrupted a local variable allocated by calloc():
addr = mmap((void *)pri_addr, page_size, PROT_WRITE, MAP_FIXED |
MAP_SHARED, fd, offset);
Gdb log of secondary process, comments inside. Hope I'm wrong somewhere:
Thread 1 "symmetric_mp" hit Breakpoint 1, mlx5_init_context (vdev=0x10430c00,
ctx=0x7ffff72200b8, cmd_fd=7)
at /home/xueming/rdma-core/providers/mlx5/mlx5.c:888
888 gross_uuars = context->tot_uuars /
MLX5_NUM_NON_FP_BFREGS_PER_UAR * NUM_BFREGS_PER_UAR;
(gdb) p context
$1 = (struct mlx5_context *) 0x7ffff72200b8
(gdb) p *context
$2 = {ibv_ctx = {device = 0x0, ops = {query_device = 0x0, query_port = 0x0,
alloc_pd = 0x0, dealloc_pd = 0x0, reg_mr = 0x0, rereg_mr = 0x0, dereg_mr = 0x0,
...
/////////// the local variable "context" allocated by calloc() works well
before mmap ///////////
(gdb) c
Continuing.
Thread 1 "symmetric_mp" received signal SIGSEGV, Segmentation fault.
0x00007ffff7d06340 in free () from /lib/powerpc64le-linux-gnu/libc.so.6
(gdb) bt
#0 0x00007ffff7d06340 in free () from /lib/powerpc64le-linux-gnu/libc.so.6
#1 0x00007ffff7efd92c in mlx5_cleanup_context (device=0x10430c00,
ibctx=0x7ffff72200b8) at /home/xueming/rdma-core/providers/mlx5/mlx5.c:1008
#2 0x00007ffff7f3cc74 in __ibv_close_device_1_1 (context=0x7ffff72200b8) at
/home/xueming/rdma-core/libibverbs/device.c:285
#3 0x00000000101d8118 in mlx5_pci_probe (pci_drv=0x103345a8 <mlx5_driver>,
pci_dev=0x10440510) at /home/xueming/mlnx-dpdk.org/drivers/net/mlx5/mlx5.c:925
...
(gdb) f 1
#1 0x00007ffff7efd92c in mlx5_cleanup_context (device=0x10430c00,
ibctx=0x7ffff72200b8) at /home/xueming/rdma-core/providers/mlx5/mlx5.c:1008
1008 free(context->bfs);
(gdb) p context
$3 = (struct mlx5_context *) 0x7ffff72200b8
////////////same address///////////////
(gdb) p *context
Cannot access memory at address 0x7ffff7250000
////////// This is the address returned by mmap() of primary address,
conflicted here. /////////////////
(gdb) p sizeof(struct mlx5_context)
$5 = 199408
(gdb) p/x 0x7ffff72200b8 + 199408
$7 = 0x7ffff7250ba8
Best Regards,
Xueming
> -----Original Message-----
> From: Xueming(Steven) Li
> Sent: Wednesday, September 13, 2017 2:11 AM
> To: Xueming(Steven) Li <xuemi...@mellanox.com>; Chao Zhu
> <chao...@linux.vnet.ibm.com>
> Cc: dev@dpdk.org
> Subject: RE: multi-process shared memory on PPC
>
> It works for me after setting nr_overcommit_hugepages correctly, thanks again.
>
> > -----Original Message-----
> > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Xueming(Steven)
> > Li
> > Sent: Tuesday, September 12, 2017 4:13 PM
> > To: Chao Zhu <chao...@linux.vnet.ibm.com>
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] multi-process shared memory on PPC
> >
> > Great, that might be something missing, will update later. Thanks!
> >
> > > -----Original Message-----
> > > From: Chao Zhu [mailto:chao...@linux.vnet.ibm.com]
> > > Sent: Tuesday, September 12, 2017 4:02 PM
> > > To: Xueming(Steven) Li <xuemi...@mellanox.com>
> > > Cc: dev@dpdk.org
> > > Subject: RE: multi-process shared memory on PPC
> > >
> > > Xueming,
> > >
> > > I tried it on my local server, it works fine.
> > > Did you set this value
> > > "/sys/kernel/mm/hugepages/hugepages-
> > 16384kB/nr_overcommit_hugepages" ?
> > > It should be the same value as "
> > > /sys/kernel/mm/hugepages/hugepages-16384kB/nr_hugepages" on
> ppc64le.
> > >
> > > > -----Original Message-----
> > > > From: Xueming(Steven) Li [mailto:xuemi...@mellanox.com]
> > > > Sent: 2017年8月31日 22:40
> > > > To: Chao Zhu <chao...@linux.vnet.ibm.com>
> > > > Cc: dev@dpdk.org
> > > > Subject: multi-process shared memory on PPC
> > > >
> > > > Hi all,
> > > >
> > > > I'm testing multi-process example on PowerPC system, looks like
> > > > shared memory not working properly.
> > > > Below are the error and debug info of the first multi-process
> > > > example 'simple_mp' (works as expected on X86):
> > > >
> > > > // Build DPDK, master branch
> > > > # make T=ppc_64-power8-linuxapp-gcc -j160 -C
> > > > examples/multi_process/simple_mp/ #
> > > > examples/multi_process/simple_mp/build/simple_mp -w 0:0.0
> > > > --proc-type=primary -l 1-2 --socket-mem
> > > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > > APP: Finished Process Init.
> > > > Starting core 2
> > > > simple_mp >
> > > >
> > > > // Open another terminal and run secondary process # gdb --args
> > > > examples/multi_process/simple_mp/build/simple_mp -n 4 -w 0:0.0
> > > > --proc-type=secondary -l 3-4 --socket-mem
> > > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > > GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1 Copyright (C) 2016
> > > > Free Software Foundation, Inc.
> > > > License GPLv3+: GNU GPL version 3 or later
> > > >
> > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > ur
> > > ldef
> > > ense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > >
> >
> 3A__gnu.org_licenses_gpl&data=02%7C01%7Cxuemingl%40mellanox.com%7C
> > >
> >
> 9cd11fda7a7f48df7cbd08d4f9b486d8%7Ca652971c7d2e4d9ba6a4d149256f461
> > >
> >
> b%7C0%7C0%7C636408001138751301&sdata=bb3hwYz7ehCqt4QA6GGqTDMY
> > > kY1qAJV%2BCMDBVbXmWzE%3D&reserved=0.
> > > h
> > > > tml&d=DwIFAg&c=jf_iaSHvJObTbx-
> > > siA1ZOg&r=4TYe4NpqlgVi0ArYpZ76FrWp7IE
> > > > w8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> > 6aTY_jEvDZtMZR0o&s
> > > > =SbDQBN7qJ9z8fWCSePyvm5FFncp2P_9XZm-qs1lH5PE&e= > This is free
> > > > software: you are free to change and redistribute it.
> > > > There is NO WARRANTY, to the extent permitted by law. Type "show
> > > copying"
> > > > and "show warranty" for details.
> > > > This GDB was configured as "powerpc64le-linux-gnu".
> > > > Type "show configuration" for configuration details.
> > > > For bug reporting instructions, please see:
> > > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > ur
> > > > ldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > > 3A__www.gnu.org_softwar&
> > > >
> > >
> >
> data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d4f
> > > 9b48
> > > >
> > >
> >
> 6d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63640800113875
> > > 1301&sd
> > > >
> > >
> >
> ata=4DNDTyIcHpTIU60K7FM%2F1zuASSB1IdZDRyOaiVp1Y%2BY%3D&reserved=
> > > 0
> > > > e_gdb_bugs_&d=DwIFAg&c=jf_iaSHvJObTbx-
> > > siA1ZOg&r=4TYe4NpqlgVi0ArYpZ
> > > > 76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> > > 6aTY_jEvDZt
> > > > MZR0o&s=GAFKK3jSahAPFXMGcPGbVNTl0CesJy_qfs5TxXfGOX0&e= >.
> > > > Find the GDB manual and other documentation resources online at:
> > > > <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%
> > > > 2F
> > > > ur
> > > > ldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > > 3A__www.gnu.org_softwar&
> > > >
> > >
> >
> data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d4f
> > > 9b48
> > > >
> > >
> >
> 6d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C63640800113875
> > > 1301&sd
> > > >
> > >
> >
> ata=4DNDTyIcHpTIU60K7FM%2F1zuASSB1IdZDRyOaiVp1Y%2BY%3D&reserved=
> > > 0
> > > > e_gdb_documentation_&d=DwIFAg&c=jf_iaSHvJObTbx-
> > > siA1ZOg&r=4TYe4Npql
> > > >
> > gVi0ArYpZ76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwuuryoN7QPg9N-
> > > 6a
> > > > TY_jEvDZtMZR0o&s=JdkMMjGGl-
> > > 3Z3BGnjCkOFP5lI5BIbRNBxH4EVI70YFc&e= > .
> > > > For help, type "help".
> > > > Type "apropos word" to search for commands related to "word"...
> > > > Reading symbols from
> > > > examples/multi_process/simple_mp/build/simple_mp...done.
> > > > (gdb) r
> > > > Starting program:
> > > > /home/xueming/mlnx-
> > > dpdk.org/examples/multi_process/simple_mp/build/sim
> > > > ple_mp -n 4 -w 0:0.0 --proc-type=secondary -l 3-4 --socket-mem
> > > > 256,256,0,0,0,0,0,0,0,0,0,0,0,0,0,0,256,256
> > > > [Thread debugging using libthread_db enabled] Using host
> > > > libthread_db
> > > library
> > > > "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
> > > > EAL: Detected 144 lcore(s)
> > > > EAL: Probing VFIO support...
> > > > [New Thread 0x7ffff7a6eff0 (LWP 58811)] [New Thread 0x7ffff726eff0
> > > > (LWP 58812)]
> > > >
> > > > Thread 1 "simple_mp" received signal SIGSEGV, Segmentation fault.
> > > > 0x00007ffff7d2f3ec in ?? () from
> > > > /lib/powerpc64le-linux-gnu/libc.so.6
> > > > (gdb) bt full
> > > > #0 0x00007ffff7d2f3ec in ?? () from
> > > > /lib/powerpc64le-linux-gnu/libc.so.6
> > > > No symbol table info available.
> > > > #1 0x0000000010077d54 in rte_ring_lookup (name=0x102a4e28
> > > > "PRI_2_SEC")
> > > > at /home/xueming/mlnx-dpdk.org/lib/librte_ring/rte_ring.c:326
> > > > te = 0x7ef3eefd5e00
> > > > r = 0x0
> > > > ring_list = 0x7ffff7aa250c
> > > > #2 0x0000000010032640 in main (argc=10, argv=0x7ffffffff3e8)
> > > > at
> > > > /home/xueming/mlnx-
> > > dpdk.org/examples/multi_process/simple_mp/main.c:12
> > > > 8
> > > > flags = 0
> > > > ring_size = 64
> > > > pool_size = 1024
> > > > pool_cache = 32
> > > > priv_data_sz = 0
> > > > ret = 9
> > > > lcore_id = 32767
> > > > cl = 0x7ffff7fef718 <_dl_argv>
> > > > (gdb) f 1
> > > > #1 0x0000000010077d54 in rte_ring_lookup (name=0x102a4e28
> > > > "PRI_2_SEC")
> > > > at /home/xueming/mlnx-dpdk.org/lib/librte_ring/rte_ring.c:326
> > > > 326 if (strncmp(name, r->name,
> > > > RTE_RING_NAMESIZE) == 0)
> > > > (gdb) l
> > > > 321
> > > > 322 rte_rwlock_read_lock(RTE_EAL_TAILQ_RWLOCK);
> > > > 323
> > > > 324 TAILQ_FOREACH(te, ring_list, next) {
> > > > 325 r = (struct rte_ring *) te->data;
> > > > 326 if (strncmp(name, r->name,
> > > > RTE_RING_NAMESIZE) == 0)
> > > > 327 break;
> > > > 328 }
> > > > 329
> > > > 330 rte_rwlock_read_unlock(RTE_EAL_TAILQ_RWLOCK);
> > > > (gdb) p *te
> > > > $1 = {next = {tqe_next = 0x0, tqe_prev = 0x0}, data = 0x0}
> > > > (gdb) p *ring_list
> > > > $2 = {tqh_first = 0x7ef3eefd5e00, tqh_last = 0x7ef3eec6d100}
> > > >
> > > > # uname -a
> > > > Linux dragon-power-01 4.13.0-rc2 #1 SMP Tue Aug 29 17:27:05 IDT
> > > > 2017 ppc64le ppc64le ppc64le GNU/Linux
> > > >
> > > > # cat /etc/issue
> > > > Ubuntu 16.04.2 LTS \n \l
> > > >
> > > > # gcc --version
> > > > gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 Copyright
> > > > (C)
> > > > 2015 Free Software Foundation, Inc.
> > > > This is free software; see the source for copying conditions.
> > > > There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR
> > > > A PARTICULAR PURPOSE.
> > > >
> > > > # lscpu
> > > > Architecture: ppc64le
> > > > Byte Order: Little Endian
> > > > CPU(s): 144
> > > > On-line CPU(s) list: 0-87,96-103,112-159
> > > > Thread(s) per core: 8
> > > > Core(s) per socket: 4
> > > > Socket(s): 4
> > > > NUMA node(s): 4
> > > > Model: 2.1 (pvr 004b 0201)
> > > > Model name: POWER8E (raw), altivec supported
> > > > CPU max MHz: 3690.0000
> > > > CPU min MHz: 2061.0000
> > > > L1d cache: 64K
> > > > L1i cache: 32K
> > > > L2 cache: 512K
> > > > L3 cache: 8192K
> > > > NUMA node0 CPU(s): 0-39
> > > > NUMA node1 CPU(s): 40-79
> > > > NUMA node16 CPU(s): 80-87,96-103,112-119
> > > > NUMA node17 CPU(s): 120-159
> > > >
> > > > # cat build/.config
> > > > # RTE_EXEC_ENV values are the directories in mk/exec-env/
> > > > CONFIG_RTE_EXEC_ENV="linuxapp"
> > > > # RTE_ARCH values are architecture we compile for. directories in
> > > > mk/arch/ CONFIG_RTE_ARCH="ppc_64"
> > > > # machine can define specific variables or action for a specific
> > > > board # RTE_MACHINE values are architecture we compile for.
> > > > directories in mk/machine/ CONFIG_RTE_MACHINE="power8"
> > > > # The compiler we use.
> > > > # RTE_TOOLCHAIN values are architecture we compile for.
> > > > directories in mk/toolchain/ CONFIG_RTE_TOOLCHAIN="gcc"
> > > > # Use intrinsics or assembly code for key routines
> > > > CONFIG_RTE_FORCE_INTRINSICS=n # Machine forces strict alignment
> > > > constraints.
> > > > CONFIG_RTE_ARCH_STRICT_ALIGN=n
> > > > # Compile to share library
> > > > CONFIG_RTE_BUILD_SHARED_LIB=n
> > > > # Use newest code breaking previous ABI CONFIG_RTE_NEXT_ABI=y #
> > > > Major ABI to overwrite library specific LIBABIVER
> > > > CONFIG_RTE_MAJOR_ABI= # Machine's cache line size
> > > > CONFIG_RTE_CACHE_LINE_SIZE=128
> > > > # Compile Environment Abstraction Layer CONFIG_RTE_LIBRTE_EAL=y
> > > > CONFIG_RTE_MAX_LCORE=256
> > > > CONFIG_RTE_MAX_NUMA_NODES=32
> > > > CONFIG_RTE_MAX_MEMSEG=256
> > > > CONFIG_RTE_MAX_MEMZONE=2560
> > > > CONFIG_RTE_MAX_TAILQ=32
> > > > CONFIG_RTE_LOG_LEVEL=RTE_LOG_INFO
> > > > CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_INFO
> > > > CONFIG_RTE_LOG_HISTORY=256
> > > > CONFIG_RTE_BACKTRACE=y
> > > > CONFIG_RTE_LIBEAL_USE_HPET=n
> > > > CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n
> > > > CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n
> > > > CONFIG_RTE_EAL_IGB_UIO=y
> > > > CONFIG_RTE_EAL_VFIO=y
> > > > CONFIG_RTE_MALLOC_DEBUG=n
> > > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> > > > # Recognize/ignore architecture we compile for. AVX/AVX512 CPU
> > > > flags for performance/power testing.
> > > > # AVX512 is marked as experimental for now, will enable it after
> > > > enough #
> > > field
> > > > test and possible optimization.
> > > > CONFIG_RTE_ENABLE_AVX=y
> > > > CONFIG_RTE_ENABLE_AVX512=n
> > > > # Default driver path (or "" to disable) CONFIG_RTE_EAL_PMD_PATH=""
> > > > # Compile Environment Abstraction Layer to support Vmware TSC map
> > > > CONFIG_RTE_LIBRTE_EAL_VMWARE_TSC_MAP_SUPPORT=n
> > > > # Compile architecture we compile for. argument parser library
> > > > CONFIG_RTE_LIBRTE_KVARGS=y # Compile generic ethernet library
> > > > CONFIG_RTE_LIBRTE_ETHER=y CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=n
> > > > CONFIG_RTE_MAX_ETHPORTS=32
> > > > CONFIG_RTE_MAX_QUEUES_PER_PORT=1024
> > > > CONFIG_RTE_LIBRTE_IEEE1588=n
> > > > CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS=16
> > > > CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y
> > > > # Turn off Tx preparation stage
> > > > # Warning: rte_eth_tx_prepare() can be safely disabled only if
> > > > using a #
> > > driver
> > > > which do not implement any Tx preparation.
> > > > CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=n
> > > > # Compile burst-oriented Amazon ENA PMD driver
> > > > CONFIG_RTE_LIBRTE_ENA_PMD=y
> > CONFIG_RTE_LIBRTE_ENA_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_ENA_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_ENA_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_ENA_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
> > > > # Compile burst-oriented IGB & EM PMD drivers
> > > > CONFIG_RTE_LIBRTE_EM_PMD=y CONFIG_RTE_LIBRTE_IGB_PMD=y
> > > > CONFIG_RTE_LIBRTE_E1000_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_E1000_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_E1000_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_E1000_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_E1000_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_E1000_PF_DISABLE_STRIP_CRC=n
> > > > # Compile burst-oriented IXGBE PMD driver
> > > > CONFIG_RTE_LIBRTE_IXGBE_PMD=n
> > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_IXGBE_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_IXGBE_PF_DISABLE_STRIP_CRC=n
> > > > CONFIG_RTE_IXGBE_INC_VECTOR=y
> > > > CONFIG_RTE_LIBRTE_IXGBE_BYPASS=n
> > > > # Compile burst-oriented I40E PMD driver
> > > > CONFIG_RTE_LIBRTE_I40E_PMD=y
> > CONFIG_RTE_LIBRTE_I40E_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_I40E_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_I40E_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_I40E_RX_ALLOW_BULK_ALLOC=y
> > > > CONFIG_RTE_LIBRTE_I40E_INC_VECTOR=y
> > > > CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n
> > > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF=64
> > > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF=4
> > > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM=4
> > > > # interval up to 8160 us, aligned to 2 (or default value)
> > > > CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL=-1
> > > > # Compile burst-oriented FM10K PMD
> CONFIG_RTE_LIBRTE_FM10K_PMD=n
> > > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_FM10K_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_FM10K_RX_OLFLAGS_ENABLE=y
> > > > CONFIG_RTE_LIBRTE_FM10K_INC_VECTOR=y
> > > > # Compile burst-oriented Mellanox ConnectX-3 (MLX4) PMD
> > > > CONFIG_RTE_LIBRTE_MLX4_PMD=n
> CONFIG_RTE_LIBRTE_MLX4_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_MLX4_DEBUG_BROKEN_VERBS=n
> > > > CONFIG_RTE_LIBRTE_MLX4_SGE_WR_N=4
> > > > CONFIG_RTE_LIBRTE_MLX4_MAX_INLINE=0
> > > > CONFIG_RTE_LIBRTE_MLX4_TX_MP_CACHE=8
> > > > CONFIG_RTE_LIBRTE_MLX4_SOFT_COUNTERS=1
> > > > # Compile burst-oriented Mellanox ConnectX-4 & ConnectX-5 (MLX5)
> > > > PMD CONFIG_RTE_LIBRTE_MLX5_PMD=y
> CONFIG_RTE_LIBRTE_MLX5_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_MLX5_TX_MP_CACHE=8
> > > > # Compile burst-oriented Broadcom PMD driver
> > > > CONFIG_RTE_LIBRTE_BNX2X_PMD=n
> > CONFIG_RTE_LIBRTE_BNX2X_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_BNX2X_MF_SUPPORT=n
> > > > CONFIG_RTE_LIBRTE_BNX2X_DEBUG_PERIODIC=n
> > > > # Compile burst-oriented Chelsio Terminator (CXGBE) PMD
> > > > CONFIG_RTE_LIBRTE_CXGBE_PMD=y
> > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_REG=n
> > > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_MBOX=n
> > > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_CXGBE_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_CXGBE_TPUT=y
> > > > # Compile burst-oriented Cisco ENIC PMD driver
> > > > CONFIG_RTE_LIBRTE_ENIC_PMD=n CONFIG_RTE_LIBRTE_ENIC_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_ENIC_DEBUG_FLOW=n
> > > > # Compile burst-oriented Netronome NFP PMD driver
> > > > CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_NFP_DEBUG=n #
> > > Compile
> > > > burst-oriented Broadcom BNXT PMD driver
> > > CONFIG_RTE_LIBRTE_BNXT_PMD=y #
> > > > Compile burst-oriented Solarflare libefx-based PMD
> > > > CONFIG_RTE_LIBRTE_SFC_EFX_PMD=n
> > > CONFIG_RTE_LIBRTE_SFC_EFX_DEBUG=n #
> > > > Compile software PMD backed by
> > > > SZEDATA2 device CONFIG_RTE_LIBRTE_PMD_SZEDATA2=n # Defines
> > > firmware
> > > > type address space.
> > > > # See documentation for supported values.
> > > > # Other values raise compile time error.
> > > > CONFIG_RTE_LIBRTE_PMD_SZEDATA2_AS=0
> > > > # Compile burst-oriented Cavium Thunderx NICVF PMD driver
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD=y
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_THUNDERX_NICVF_DEBUG_MBOX=n
> > > > # Compile burst-oriented Cavium LiquidIO PMD driver
> > > > CONFIG_RTE_LIBRTE_LIO_PMD=y
> > > CONFIG_RTE_LIBRTE_LIO_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_LIO_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_LIO_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_LIO_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_LIO_DEBUG_MBOX=n
> > > > CONFIG_RTE_LIBRTE_LIO_DEBUG_REGS=n
> > > > # Compile NXP DPAA2 FSL-MC Bus
> > > > CONFIG_RTE_LIBRTE_FSLMC_BUS=n
> > > > # Compile Support Libraries for NXP DPAA2
> > > > CONFIG_RTE_LIBRTE_DPAA2_MEMPOOL=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA=y
> > > > # Compile burst-oriented NXP DPAA2 PMD driver
> > > > CONFIG_RTE_LIBRTE_DPAA2_PMD=n
> > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_DEBUG_TX_FREE=n
> > > > # Compile burst-oriented VIRTIO PMD driver
> > > > CONFIG_RTE_LIBRTE_VIRTIO_PMD=y
> > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DUMP=n
> > > > # Compile virtio device emulation inside virtio PMD driver
> > > > CONFIG_RTE_VIRTIO_USER=y # Compile burst-oriented VMXNET3 PMD
> > > driver
> > > > CONFIG_RTE_LIBRTE_VMXNET3_PMD=n
> > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n
> > > > CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_DRIVER=n
> > > > # Compile example software rings based PMD
> > > > CONFIG_RTE_LIBRTE_PMD_RING=y
> > > > CONFIG_RTE_PMD_RING_MAX_RX_RINGS=16
> > > > CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16
> > > > # Compile software PMD backed by PCAP files
> > > > CONFIG_RTE_LIBRTE_PMD_PCAP=n # Compile link bonding PMD library
> > > > CONFIG_RTE_LIBRTE_PMD_BOND=y
> > > CONFIG_RTE_LIBRTE_BOND_DEBUG_ALB=n
> > > > CONFIG_RTE_LIBRTE_BOND_DEBUG_ALB_L1=n
> > > > # QLogic 10G/25G/40G/50G/100G PMD
> > > > CONFIG_RTE_LIBRTE_QEDE_PMD=y
> > > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO=n
> > > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX=n
> > > > #Provides abs path/name of architecture we compile for. firmware file.
> > > > #Empty string denotes driver will use default firmware
> > > > CONFIG_RTE_LIBRTE_QEDE_FW=""
> > > > # Compile software PMD backed by AF_PACKET sockets (Linux only)
> > > > CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y # Compile ARK PMD
> > > > CONFIG_RTE_LIBRTE_ARK_PMD=y CONFIG_RTE_LIBRTE_ARK_PAD_TX=y
> > > > CONFIG_RTE_LIBRTE_ARK_DEBUG_RX=n
> > > CONFIG_RTE_LIBRTE_ARK_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_ARK_DEBUG_STATS=n
> > > > CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE=n
> > > > # Compile WRS accelerated virtual port (AVP) guest PMD driver
> > > > CONFIG_RTE_LIBRTE_AVP_PMD=n
> > CONFIG_RTE_LIBRTE_AVP_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_AVP_DEBUG_TX=n
> > > CONFIG_RTE_LIBRTE_AVP_DEBUG_DRIVER=y
> > > > CONFIG_RTE_LIBRTE_AVP_DEBUG_BUFFERS=n
> > > > # Compile architecture we compile for. TAP PMD # It is enabled by
> > > > default
> > > for
> > > > Linux only.
> > > > CONFIG_RTE_LIBRTE_PMD_TAP=y
> > > > # Compile Xen PMD
> > > > CONFIG_RTE_LIBRTE_PMD_XENVIRT=n
> > > > # Compile null PMD
> > > > CONFIG_RTE_LIBRTE_PMD_NULL=y
> > > > # Compile fail-safe PMD
> > > > CONFIG_RTE_LIBRTE_PMD_FAILSAFE=y
> > > > # Do prefetch of packet data within PMD driver receive function
> > > > CONFIG_RTE_PMD_PACKET_PREFETCH=y # Compile generic crypto
> device
> > > > library CONFIG_RTE_LIBRTE_CRYPTODEV=y
> > > > CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=n
> > > > CONFIG_RTE_CRYPTO_MAX_DEVS=64
> > > > CONFIG_RTE_CRYPTODEV_NAME_LEN=64
> > > > # Compile PMD for ARMv8 Crypto device
> > > > CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO=n
> > > > CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO_DEBUG=n
> > > > # Compile NXP DPAA2 crypto sec driver for CAAM HW
> > > > CONFIG_RTE_LIBRTE_PMD_DPAA2_SEC=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_DRIVER=n
> > > > CONFIG_RTE_LIBRTE_DPAA2_SEC_DEBUG_RX=n
> > > > # Compile PMD for QuickAssist based devices
> > > > CONFIG_RTE_LIBRTE_PMD_QAT=n
> > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=n
> > > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=n
> > > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=n
> > > > CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=n
> > > > # Number of sessions to create in architecture we compile for.
> > > > session
> > > memory
> > > > pool # on a single QuickAssist device.
> > > > CONFIG_RTE_QAT_PMD_MAX_NB_SESSIONS=2048
> > > > # Compile PMD for AESNI backed device
> > > > CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n
> > > > CONFIG_RTE_LIBRTE_PMD_AESNI_MB_DEBUG=n
> > > > # Compile PMD for Software backed device
> > > > CONFIG_RTE_LIBRTE_PMD_OPENSSL=n
> > > CONFIG_RTE_LIBRTE_PMD_OPENSSL_DEBUG=n
> > > > # Compile PMD for AESNI GCM device
> > > > CONFIG_RTE_LIBRTE_PMD_AESNI_GCM=n
> > > > CONFIG_RTE_LIBRTE_PMD_AESNI_GCM_DEBUG=n
> > > > # Compile PMD for SNOW 3G device
> > > > CONFIG_RTE_LIBRTE_PMD_SNOW3G=n
> > > > CONFIG_RTE_LIBRTE_PMD_SNOW3G_DEBUG=n
> > > > # Compile PMD for KASUMI device
> > > > CONFIG_RTE_LIBRTE_PMD_KASUMI=n
> > > > CONFIG_RTE_LIBRTE_PMD_KASUMI_DEBUG=n
> > > > # Compile PMD for ZUC device
> > > > CONFIG_RTE_LIBRTE_PMD_ZUC=n
> > > > CONFIG_RTE_LIBRTE_PMD_ZUC_DEBUG=n
> > > > # Compile PMD for Crypto Scheduler device
> > > > CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER=y
> > > > CONFIG_RTE_LIBRTE_PMD_CRYPTO_SCHEDULER_DEBUG=n
> > > > # Compile PMD for NULL Crypto device
> > > > CONFIG_RTE_LIBRTE_PMD_NULL_CRYPTO=y
> > > > # Compile generic event device library
> > > > CONFIG_RTE_LIBRTE_EVENTDEV=y
> CONFIG_RTE_LIBRTE_EVENTDEV_DEBUG=n
> > > > CONFIG_RTE_EVENT_MAX_DEVS=16
> > > > CONFIG_RTE_EVENT_MAX_QUEUES_PER_DEV=64
> > > > # Compile PMD for skeleton event device
> > > > CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV=y
> > > > CONFIG_RTE_LIBRTE_PMD_SKELETON_EVENTDEV_DEBUG=n
> > > > # Compile PMD for software event device
> > > > CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV=y
> > > > CONFIG_RTE_LIBRTE_PMD_SW_EVENTDEV_DEBUG=n
> > > > # Compile PMD for octeontx sso event device
> > > > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF=y
> > > > CONFIG_RTE_LIBRTE_PMD_OCTEONTX_SSOVF_DEBUG=n
> > > > # Compile librte_ring
> > > > CONFIG_RTE_LIBRTE_RING=y
> > > > # Compile librte_mempool
> > > > CONFIG_RTE_LIBRTE_MEMPOOL=y
> > > > CONFIG_RTE_MEMPOOL_CACHE_MAX_SIZE=512
> > > > CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG=n
> > > > # Compile Mempool drivers
> > > > CONFIG_RTE_DRIVER_MEMPOOL_RING=y
> > > > CONFIG_RTE_DRIVER_MEMPOOL_STACK=y
> > > > # Compile librte_mbuf
> > > > CONFIG_RTE_LIBRTE_MBUF=y
> > > > CONFIG_RTE_LIBRTE_MBUF_DEBUG=n
> > > > CONFIG_RTE_MBUF_DEFAULT_MEMPOOL_OPS="ring_mp_mc"
> > > > CONFIG_RTE_MBUF_REFCNT_ATOMIC=y
> > > > CONFIG_RTE_PKTMBUF_HEADROOM=128
> > > > # Compile librte_timer
> > > > CONFIG_RTE_LIBRTE_TIMER=y
> > > > CONFIG_RTE_LIBRTE_TIMER_DEBUG=n
> > > > # Compile librte_cfgfile
> > > > CONFIG_RTE_LIBRTE_CFGFILE=y
> > > > # Compile librte_cmdline
> > > > CONFIG_RTE_LIBRTE_CMDLINE=y
> > > > CONFIG_RTE_LIBRTE_CMDLINE_DEBUG=n
> > > > # Compile librte_hash
> > > > CONFIG_RTE_LIBRTE_HASH=y
> > > > CONFIG_RTE_LIBRTE_HASH_DEBUG=n
> > > > # Compile librte_efd
> > > > CONFIG_RTE_LIBRTE_EFD=y
> > > > # Compile librte_jobstats
> > > > CONFIG_RTE_LIBRTE_JOBSTATS=y
> > > > # Compile architecture we compile for. device metrics library
> > > > CONFIG_RTE_LIBRTE_METRICS=y # Compile architecture we compile for.
> > > > bitrate statistics library CONFIG_RTE_LIBRTE_BITRATE=y # Compile
> > > > architecture we compile for. latency statistics library
> > > > CONFIG_RTE_LIBRTE_LATENCY_STATS=y # Compile librte_lpm
> > > > CONFIG_RTE_LIBRTE_LPM=y CONFIG_RTE_LIBRTE_LPM_DEBUG=n #
> > > Compile
> > > > librte_acl CONFIG_RTE_LIBRTE_ACL=y
> > CONFIG_RTE_LIBRTE_ACL_DEBUG=n
> > > #
> > > > Compile librte_power CONFIG_RTE_LIBRTE_POWER=y
> > > > CONFIG_RTE_LIBRTE_POWER_DEBUG=n
> > > > CONFIG_RTE_MAX_LCORE_FREQS=64
> > > > # Compile librte_net
> > > > CONFIG_RTE_LIBRTE_NET=y
> > > > # Compile librte_ip_frag
> > > > CONFIG_RTE_LIBRTE_IP_FRAG=y
> > > > CONFIG_RTE_LIBRTE_IP_FRAG_DEBUG=n
> > > > CONFIG_RTE_LIBRTE_IP_FRAG_MAX_FRAG=4
> > > > CONFIG_RTE_LIBRTE_IP_FRAG_TBL_STAT=n
> > > > # Compile GRO library
> > > > CONFIG_RTE_LIBRTE_GRO=y
> > > > # Compile librte_meter
> > > > CONFIG_RTE_LIBRTE_METER=y
> > > > # Compile librte_sched
> > > > CONFIG_RTE_LIBRTE_SCHED=y
> > > > CONFIG_RTE_SCHED_DEBUG=n
> > > > CONFIG_RTE_SCHED_RED=n
> > > > CONFIG_RTE_SCHED_COLLECT_STATS=n
> > > > CONFIG_RTE_SCHED_SUBPORT_TC_OV=n
> > > > CONFIG_RTE_SCHED_PORT_N_GRINDERS=8
> > > > CONFIG_RTE_SCHED_VECTOR=n
> > > > # Compile architecture we compile for. distributor library
> > > > CONFIG_RTE_LIBRTE_DISTRIBUTOR=y # Compile architecture we compile
> > for.
> > > > reorder library CONFIG_RTE_LIBRTE_REORDER=y # Compile librte_port
> > > > CONFIG_RTE_LIBRTE_PORT=y CONFIG_RTE_PORT_STATS_COLLECT=n
> > > > CONFIG_RTE_PORT_PCAP=n # Compile librte_table
> > > > CONFIG_RTE_LIBRTE_TABLE=y CONFIG_RTE_TABLE_STATS_COLLECT=n #
> > > Compile
> > > > librte_pipeline CONFIG_RTE_LIBRTE_PIPELINE=y
> > > > CONFIG_RTE_PIPELINE_STATS_COLLECT=n
> > > > # Compile librte_kni
> > > > CONFIG_RTE_LIBRTE_KNI=y
> > > > CONFIG_RTE_LIBRTE_PMD_KNI=y
> > > > CONFIG_RTE_KNI_KMOD=y
> > > > CONFIG_RTE_KNI_KMOD_ETHTOOL=n
> > > > CONFIG_RTE_KNI_PREEMPT_DEFAULT=y
> > > > # Compile architecture we compile for. pdump library
> > > > CONFIG_RTE_LIBRTE_PDUMP=y # Compile vhost user library
> > > > CONFIG_RTE_LIBRTE_VHOST=y CONFIG_RTE_LIBRTE_VHOST_NUMA=y
> > > > CONFIG_RTE_LIBRTE_VHOST_DEBUG=n # Compile vhost PMD # To
> compile,
> > > > CONFIG_RTE_LIBRTE_VHOST should be enabled.
> > > > CONFIG_RTE_LIBRTE_PMD_VHOST=y
> > > > #Compile Xen domain0 support
> > > > CONFIG_RTE_LIBRTE_XEN_DOM0=n
> > > > # Compile architecture we compile for. test application
> > > > CONFIG_RTE_APP_TEST=y CONFIG_RTE_APP_TEST_RESOURCE_TAR=n
> > > > # Compile architecture we compile for. PMD test application
> > > > CONFIG_RTE_TEST_PMD=y
> > > CONFIG_RTE_TEST_PMD_RECORD_CORE_CYCLES=n
> > > > CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n
> > > > # Compile architecture we compile for. crypto performance
> > > > application CONFIG_RTE_APP_CRYPTO_PERF=y # Compile architecture
> we
> > compile for.
> > > > eventdev application CONFIG_RTE_APP_EVENTDEV=y
> > > > CONFIG_RTE_EXEC_ENV_LINUXAPP=y CONFIG_RTE_ARCH_PPC_64=y
> > > > CONFIG_RTE_ARCH_64=y CONFIG_RTE_TOOLCHAIN_GCC=y # Note:
> Power
> > > doesn't
> > > > have this support # Note: Initially, all of architecture we
> > > compile for.
> > > > PMD drivers compilation are turned off on Power # Will turn on
> > > > them only
> > > after
> > > > architecture we compile for. successful testing on Power
> > > >
> > > > Please refer here for more information on running DPDK
> > > > multi-process
> > > > examples:
> > > >
> > > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2
> > > > Fu
> > > > rl
> > > > defense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-
> > > 3A__dpdk.org_doc_guides_s
> > > >
> > >
> >
> &data=02%7C01%7Cxuemingl%40mellanox.com%7C9cd11fda7a7f48df7cbd08d
> > > 4f9b4
> > > >
> > >
> >
> 86d8%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C6364080011387
> > > 51301&s
> > > >
> > data=4n76ha0PmSr3wP2c0MAo3PEVYRMKjJVcU9o5xETqhBc%3D&reserved=0
> > > > ample-5Fapp-5Fug_multi-5Fprocess.html&d=DwIFAg&c=jf_iaSHvJObTbx-
> > > siA1Z
> > > >
> > >
> >
> Og&r=4TYe4NpqlgVi0ArYpZ76FrWp7IEw8gfzq7k4itCX_Mg&m=lz_1z0LzwjqjIwu
> > > > uryoN7QPg9N-6aTY_jEvDZtMZR0o&s=gupwe5WWQ-
> > > ap2jObFko6UJymrcy4jTmF
> > > > 2r1dZ2fQN9E&e=
> > > >
> > > > Best Regards,
> > > > Xueming
> > >
