> -----原始邮件-----
> 发件人: "Stephen Hemminger" <step...@networkplumber.org>
> 发送时间: 2024-07-02 23:08:37 (星期二)
> 收件人: "洪全" <hongq...@iie.ac.cn>
> 抄送: users@dpdk.org
> 主题: Re: Issue with Cannot allocate memory when using 32-bit DPDK
application
>
> On Tue, 2 Jul 2024 18:17:08 +0800 (GMT+08:00)
> 洪全 <hongq...@iie.ac.cn> wrote:
>
> > Dear DPDK community,
> >
> >
> > I am encountering an issue when attempting to run a 32-bit DPDK
application on Linux. Specifically, I am facing a "Cannot allocate memory"
error during initialization. While I can mitigate this issue by using the
`--no-huge` option, it adversely affects the performance of my application.
> >
> >
> > Here is the error output I receive:
> >
> >
> > ```
> > sudo ./app -l 0-1 --proc-type=primary --file-prefix=pmd1
--vdev=net_tap001,iface=tap001 --no-pci
> > EAL: Detected CPU lcores: 2
> > EAL: Detected NUMA nodes: 1
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/pmd1/mp_socket
> > EAL: Selected IOVA mode 'PA'
> > EAL: Cannot get a virtual area: Cannot allocate memory
> > EAL: Cannot allocate VA space for memseg list, retrying with
different page size
> > EAL: Cannot allocate VA space on socket 0
> > EAL: FATAL: Cannot init memory
> > EAL: Cannot init memory
> > app: main.c:284: main: Assertion `(ret = rte_eal_init(argc, (char **)
argv)) >= 0' failed.
> > Aborted
> > ```
> >
> >
> > When debugging with `--log-level=eal,8`, the relevant portion of the
output indicates attempts to allocate memory:
> >
> >
> > ```
> > EAL: Attempting to preallocate 2048M on socket 0
> > EAL: Ask a virtual area of 0xc000 bytes
> > EAL: Virtual area found at 0xeb077000 (size = 0xc000)
> > EAL: Memseg list allocated at socket 0, page size 0x800kB
> > EAL: Ask a virtual area of 0x80000000 bytes
> > EAL: Cannot mmap((nil), 0x80200000, 0x0, 0x22, -1, 0x0): Cannot
allocate memory
> > EAL: Cannot get a virtual area: Cannot allocate memory
> > EAL: Cannot allocate VA space for memseg list, retrying with
different page size
> > EAL: Cannot allocate VA space on socket 0
> > EAL: FATAL: Cannot init memory
> > EAL: Cannot init memory
> > app: main.c:284: main: Assertion `(ret = rte_eal_init(argc, (char **)
argv)) >= 0' failed.
> > Aborted
> > ```
> >
> >
> > System information:
> > - Hugepages configured: `echo 1024 >
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages`
> > - Hugepages mounted: `mount -t hugetlbfs hugetlbfs /dev/hugepages`
> > - NUMA node information: `numactl --hardware`
> > - available: 1 nodes (0)
> > - node 0 cpus: 0 1
> > - node 0 size: 7896 MB
> > - node 0 free: 3915 MB
> > - node distances:
> > - node 0: 10
> >
> >
> > DPDK version: 22.03
> > Distribution: Ubuntu 22.04
> > Kernel information: Linux hq-virtual-machine 6.5.0-35-generic
#35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May 7 09:00:52 UTC 2 x86_64 x86_64
x86_64 GNU/Linux
> >
> >
> > I believe the issue stems from the attempt to preallocate 2048M on
socket 0, but using `-m` or `--socket-mem` options did not resolve the problem.
> >
> >
> > Could you please provide guidance on how to properly configure DPDK
to avoid this memory allocation issue while maximizing performance?
>
> What CPU architecture? Traditionally on 32 bit x86 has 3GB for userspace
and 1GB is the
> shadow kernel. Some other architecture may do the same thing.
>
> In userspace, you then have some memory for programs code, data and stack.
The
> DPDK EAL init then tries to map all of available huge pages (2G) and fails
to find
> enough contiguous virtual address space to fit.
>
> Try smaller amount of huge pages.
My CPU architecture is x86-64.
Does trying a small number of huge pages mean allocating several 1GB huge pages?
According to the output of EAL: mmap((nil), 0x80200000, 0x0, 0x22, -1, 0x0). Do
I need to allocate 3 1GB huge pages to satisfy this request? As you said, 32
bit x86 has 3GB for userspace. If I allocate all 3GB to huge pages, will there
be an error?
In addition, I tried to reduce the amount of memory in the above mmap by using
the --socket-mem or -m option, but it did not
work.</hongq...@iie.ac.cn></hongq...@iie.ac.cn></step...@networkplumber.org>
