Hi Ben, I have tested it with your modifications and it works well.
vpp# show buffers Pool Name Index NUMA Size Data Size Total Avail Cached Used default-numa-0 0 0 3968 3072 127999 126460 0 1539 default-numa-1 1 1 3968 3072 127999 127999 0 0 default-numa-2 2 2 3968 3072 127999 127999 0 0 default-numa-3 3 3 3968 3072 127999 127999 0 0 Thanks & Regards Amelia Benoit Ganne (bganne) via lists.fd.io <[email protected]> 于2026年6月30日周二 17:04写道： > Hello Amelia, > > thank you for your analysis, good catch. What do you think of > https://gerrit.fd.io/r/c/vpp/+/46182 ? > > Best > ben > > ________________________________________ > From: [email protected] <[email protected]> on behalf of amelia yin > via lists.fd.io <[email protected]> > Sent: Friday, June 26, 2026 10:39 > To: [email protected] > Subject: [vpp-dev] 26.06: Page boundary check in vlib_buffer_pool_create > causes severe buffer underutilization with 4KB pages > > Hi Team, > > We recently upgraded from VPP 23.02 to 26.06 and observed a significant > regression in buffer pool allocation. > We'd like to understand the rationale behind the page boundary check > introduced in vlib_buffer_pool_create (in src/vlib/buffer.c). > > The problem: > With the same configuration that has worked since 23.02: > > buffers { > buffers-per-numa 128000 > default data-size 3072 > } > > On VPP 23.02, buffer allocation works as expected: > vpp# show buffers > Pool Name Index NUMA Size Data Size Total Avail Cached > Used > default-numa-0 0 0 3968 3072 128000 127968 32 > 0 > default-numa-1 1 1 3968 3072 128000 128000 0 > 0 > default-numa-2 2 2 3968 3072 128000 128000 0 > 0 > default-numa-3 3 3 3968 3072 128000 128000 0 > 0 > > On VPP 26.06, only ~2032 buffers are allocated per NUMA instead of 128000: > vpp# show buffers > Pool Name Index NUMA Size Data Size Total Avail Cached > Used > default-numa-0 0 0 3968 3072 2031 460 32 > 1539 > default-numa-1 1 1 3968 3072 2032 2032 0 > 0 > default-numa-2 2 2 3968 3072 2032 2032 0 > 0 > default-numa-3 3 3 3968 3072 2031 2031 0 > 0 > > Root cause analysis: > The new code in vlib_buffer_pool_create skips any buffer that would span > across a page boundary: > > /* skip if buffer spans across page boundary */ > if (((uword) p & page_mask) != ((uword) (p + alloc_size) & page_mask)) > continue; > > With 4KB pages and alloc_size = 3968, only 1 buffer fits per 4KB page > (3968 < 4096), but the linear iteration with p += alloc_size causes the > buffer start offset to drift across pages. > The check then rejects ~94% of candidate slots because most will cross a > page boundary when stepping by 3968 through a 4096-aligned address space. > The memory is allocated (physmem region is sized for 128000 buffers), but > most of it goes unused — an effective utilization rate of only ~1.6%. > > my questions: > 1. Was the impact on non-hugepage deployments (4KB pages) considered? With > hugepages (e.g., 2MB), the check is essentially a no-op since many buffers > fit within a single page, but with 4KB pages and large > buffer sizes (>2048), the utilization drops drastically. > 2. Would it be acceptable upstream to use a per-page iteration approach > instead — placing buffers at page-aligned offsets to guarantee no > cross-page spanning while maintaining full utilization? > > Thanks & Regards > Amelia > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#27097): https://lists.fd.io/g/vpp-dev/message/27097 Mute This Topic: https://lists.fd.io/mt/119987044/21656 Group Owner: [email protected] Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
