On 09-Nov-18 12:20 PM, Burakov, Anatoly wrote:
On 09-Nov-18 9:23 AM, jianmingfan wrote:
--- fix coding style of the previous patch
During procless startup, dpdk invokes clear_hugedir() to unlink all
hugepage files under /dev/hugepages. Then in map_all_hugepages(),
it invokes mmap to allocate and zero all the huge pages as configured
in /sys/kernel/mm/hugepages/xxx/nr_hugepages.
This cause startup process extreamly slow with large size of huge page
configured.
In our use case, we usually configure as large as 200GB hugepages in our
router. It takes more than 50s each time dpdk process startup to clear
the pages.
To address this issue, user can turn on --reuse-map switch. With it,
dpdk will check the validity of the exiting page cache under
/dev/hugespages. If valid, the cache will be reused not deleted,
so that the os doesn't need to zero the pages again.
However, as there are a lot of users ,e.g. rte_kni_alloc, rely on the
os zeor page behavior. To keep things work, I add memset during
malloc_heap_alloc(). This makes sense due to the following reason.
1) user often configure hugepage size too large to be used by the
program.
In our router, 200GB is configured, but less than 2GB is actually used.
2) dpdk users don't call heap allocation in performance-critical path.
They alloc memory during process bootup.
Signed-off-by: Jianming Fan <fanjianm...@jd.com>
---
I believe this issue is better solved by actually fixing all of the
memory that DPDK leaves behind. We already have rte_eal_cleanup() call
which will deallocate any EAL-allocated memory that have been reserved,
and an exited application should free any memory it was using so that
memory subsystem could free it back to the system, thereby not needing
any cleaning of hugepages at startup.
If your application does not e.g. free its mempools on exit, it should
:) Chances are, the problem will go away. The only circumstance where
this may not work is if you preallocated your memory using
-m/--socket-mem flag.
To clarify - all of the above is only applicable to 18.05 and beyond.
The map_all_hugepages() function only gets called in the legacy mem
init, so this patch solves a problem that does not exist on recent DPDK
versions in the first place - faster initialization is one of the key
reasons why the new memory subsystem was developed.
--
Thanks,
Anatoly
