Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
On Thu, Jun 26, 2025 at 8:23 AM Jakub Kicinski wrote: > > On Wed, 25 Jun 2025 17:22:56 -0700 Mina Almasry wrote: > > What I'm hoping to do is: > > > > 1. Have nipa run the benchmark always (or at least on patches that > > touch pp code, if that's possible), and always succeed. > > 2. The pp reviewers can always check the contest results to manually > > see if there is a regression. That's still great because it saves us > > the time of cherry-pick series and running the tests ourselves (or > > asking submitters to do that). > > 3. If we notice that the results between runs are stable, then we can > > change the test to actually fail/warn if it detects a regression (if > > fast path is > # of instructions, fail). > > That's fine. I don't think putting the data on a graphs would be much > work, and clicking old results out of old runs will be a PITA. Just a > little parsing in the runner to propagate it into JSON. And a fairly > trivial bit of charts.js to fetch the runs and render UI. > > > 4. If we notice that the results have too much noise, then we can > > improve the now merged benchmark to somehow make it more consistent. > > > > FWIW, when I run the benchmark, I get very repeatable results across > > runs, especially when measuring the fast path, but nipa's mileage may > > vary. > > 100% on board. But someone with Meta credentials needs to add a runner > and babysit it, I have enough CI wrangling as is. > Of course! If someone with the credentials volunteers that would be great, if not, no big deal really. We can always get the runs manually in the meantime. The volume of pp patches isn't that much anyway. > Or we wait a couple of months until we migrate to a more public setup. Yes, I'll take a look when/if that happens (I'll watch out for an announcement). Thanks! -- Thanks, Mina
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
On Wed, 25 Jun 2025 17:22:56 -0700 Mina Almasry wrote: > What I'm hoping to do is: > > 1. Have nipa run the benchmark always (or at least on patches that > touch pp code, if that's possible), and always succeed. > 2. The pp reviewers can always check the contest results to manually > see if there is a regression. That's still great because it saves us > the time of cherry-pick series and running the tests ourselves (or > asking submitters to do that). > 3. If we notice that the results between runs are stable, then we can > change the test to actually fail/warn if it detects a regression (if > fast path is > # of instructions, fail). That's fine. I don't think putting the data on a graphs would be much work, and clicking old results out of old runs will be a PITA. Just a little parsing in the runner to propagate it into JSON. And a fairly trivial bit of charts.js to fetch the runs and render UI. > 4. If we notice that the results have too much noise, then we can > improve the now merged benchmark to somehow make it more consistent. > > FWIW, when I run the benchmark, I get very repeatable results across > runs, especially when measuring the fast path, but nipa's mileage may > vary. 100% on board. But someone with Meta credentials needs to add a runner and babysit it, I have enough CI wrangling as is. Or we wait a couple of months until we migrate to a more public setup.
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
On Wed, 25 Jun 2025 16:45:49 -0700 Mina Almasry wrote: > Thank you for merging this. Kinda of a noob question: does this merge > mean that nipa will run this on new submitted patches already? Or do > I/someone need to do something to enable that? I've been clicking on > the contest for new patches like so: > > https://netdev.bots.linux.dev/contest.html?pw-n=0&branch=net-next-2025-06-25--21-00 > > But I don't see this benchmark being run anywhere. I looked for docs > that already cover this but I couldn't find any. Right now to add a new TARGET one needs to have SSH access to the systems that run the tests :( The process of adding a runner is not automated. But this will probably need even more work because it's a performance test. We'd need some way of tracking numerical values and detecting regressions? We have a vague plan of moving NIPA out of Meta owned systems. We may want to wait for that before we tackle performance-like tests.
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
On Mon, Jun 23, 2025 at 6:19 PM wrote: > > Hello: > > This patch was applied to netdev/net-next.git (main) > by Jakub Kicinski : > > On Thu, 19 Jun 2025 18:15:18 + you wrote: > > From: Jesper Dangaard Brouer > > > > We frequently consult with Jesper's out-of-tree page_pool benchmark to > > evaluate page_pool changes. > > > > Import the benchmark into the upstream linux kernel tree so that (a) > > we're all running the same version, (b) pave the way for shared > > improvements, and (c) maybe one day integrate it with nipa, if possible. > > > > [...] > > Here is the summary with links: > - [net-next,v5] page_pool: import Jesper's page_pool benchmark > https://git.kernel.org/netdev/net-next/c/cccfe0982208 > > You are awesome, thank you! > -- > Deet-doot-dot, I am a bot. > https://korg.docs.kernel.org/patchwork/pwbot.html > > Thank you for merging this. Kinda of a noob question: does this merge mean that nipa will run this on new submitted patches already? Or do I/someone need to do something to enable that? I've been clicking on the contest for new patches like so: https://netdev.bots.linux.dev/contest.html?pw-n=0&branch=net-next-2025-06-25--21-00 But I don't see this benchmark being run anywhere. I looked for docs that already cover this but I couldn't find any. -- Thanks, Mina
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
On Wed, Jun 25, 2025 at 5:03 PM Jakub Kicinski wrote: > > On Wed, 25 Jun 2025 16:45:49 -0700 Mina Almasry wrote: > > Thank you for merging this. Kinda of a noob question: does this merge > > mean that nipa will run this on new submitted patches already? Or do > > I/someone need to do something to enable that? I've been clicking on > > the contest for new patches like so: > > > > https://netdev.bots.linux.dev/contest.html?pw-n=0&branch=net-next-2025-06-25--21-00 > > > > But I don't see this benchmark being run anywhere. I looked for docs > > that already cover this but I couldn't find any. > > Right now to add a new TARGET one needs to have SSH access to the > systems that run the tests :( The process of adding a runner is not > automated. But this will probably need even more work because it's > a performance test. We'd need some way of tracking numerical values > and detecting regressions? > I actually did what you suggested earlier, I have the test report the perf numbers but succeed always. What I'm hoping to do is: 1. Have nipa run the benchmark always (or at least on patches that touch pp code, if that's possible), and always succeed. 2. The pp reviewers can always check the contest results to manually see if there is a regression. That's still great because it saves us the time of cherry-pick series and running the tests ourselves (or asking submitters to do that). 3. If we notice that the results between runs are stable, then we can change the test to actually fail/warn if it detects a regression (if fast path is > # of instructions, fail). 4. If we notice that the results have too much noise, then we can improve the now merged benchmark to somehow make it more consistent. FWIW, when I run the benchmark, I get very repeatable results across runs, especially when measuring the fast path, but nipa's mileage may vary. -- Thanks, Mina
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
Hello: This patch was applied to netdev/net-next.git (main) by Jakub Kicinski : On Thu, 19 Jun 2025 18:15:18 + you wrote: > From: Jesper Dangaard Brouer > > We frequently consult with Jesper's out-of-tree page_pool benchmark to > evaluate page_pool changes. > > Import the benchmark into the upstream linux kernel tree so that (a) > we're all running the same version, (b) pave the way for shared > improvements, and (c) maybe one day integrate it with nipa, if possible. > > [...] Here is the summary with links: - [net-next,v5] page_pool: import Jesper's page_pool benchmark https://git.kernel.org/netdev/net-next/c/cccfe0982208 You are awesome, thank you! -- Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Re: [PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
Mina Almasry writes:
> From: Jesper Dangaard Brouer
>
> We frequently consult with Jesper's out-of-tree page_pool benchmark to
> evaluate page_pool changes.
>
> Import the benchmark into the upstream linux kernel tree so that (a)
> we're all running the same version, (b) pave the way for shared
> improvements, and (c) maybe one day integrate it with nipa, if possible.
>
> Import bench_page_pool_simple from commit 35b1716d0c30 ("Add
> page_bench06_walk_all"), from this repository:
> https://github.com/netoptimizer/prototype-kernel.git
>
> Changes done during upstreaming:
> - Fix checkpatch issues.
> - Remove the tasklet logic not needed.
> - Move under tools/testing
> - Create ksft for the benchmark.
> - Changed slightly how the benchmark gets build. Out of tree, time_bench
> is built as an independent .ko. Here it is included in
> bench_page_pool.ko
>
> Steps to run:
>
> ```
> mkdir -p /tmp/run-pp-bench
> make -C ./tools/testing/selftests/net/bench
> make -C ./tools/testing/selftests/net/bench install
> INSTALL_PATH=/tmp/run-pp-bench
> rsync --delete -avz --progress /tmp/run-pp-bench mina@$SERVER:~/
> ssh mina@$SERVER << EOF
> cd ~/run-pp-bench && sudo ./test_bench_page_pool.sh
> EOF
> ```
>
> Note that by default, the Makefile will build the benchmark for the
> currently installed kernel in /lib/modules/$(shell uname -r)/build. To
> build against the current tree, do:
>
> make KDIR=$(pwd) -C ./tools/testing/selftests/net/bench
>
> Output (from Jesper):
>
> ```
> sudo ./test_bench_page_pool.sh
> (benchmark dmesg logs snipped)
>
> Fast path results:
> no-softirq-page_pool01 Per elem: 23 cycles(tsc) 6.571 ns
>
> ptr_ring results:
> no-softirq-page_pool02 Per elem: 60 cycles(tsc) 16.862 ns
>
> slow path results:
> no-softirq-page_pool03 Per elem: 265 cycles(tsc) 73.739 ns
> ```
>
> Output (from me):
>
> ```
> sudo ./test_bench_page_pool.sh
> (benchmark dmesg logs snipped)
>
> Fast path results:
> no-softirq-page_pool01 Per elem: 11 cycles(tsc) 4.177 ns
>
> ptr_ring results:
> no-softirq-page_pool02 Per elem: 51 cycles(tsc) 19.117 ns
>
> slow path results:
> no-softirq-page_pool03 Per elem: 168 cycles(tsc) 62.469 ns
> ```
>
> Results of course will vary based on hardware/kernel/configs, and some
> variance may be there from run to run due to some noise.
>
> Cc: Jesper Dangaard Brouer
> Cc: Ilias Apalodimas
> Cc: Jakub Kicinski
> Cc: Toke Høiland-Jørgensen
>
> Signed-off-by: Mina Almasry
> Acked-by: Ilias Apalodimas
> Signed-off-by: Jesper Dangaard Brouer
Acked-by: Toke Høiland-Jørgensen
[PATCH net-next v5] page_pool: import Jesper's page_pool benchmark
From: Jesper Dangaard Brouer
We frequently consult with Jesper's out-of-tree page_pool benchmark to
evaluate page_pool changes.
Import the benchmark into the upstream linux kernel tree so that (a)
we're all running the same version, (b) pave the way for shared
improvements, and (c) maybe one day integrate it with nipa, if possible.
Import bench_page_pool_simple from commit 35b1716d0c30 ("Add
page_bench06_walk_all"), from this repository:
https://github.com/netoptimizer/prototype-kernel.git
Changes done during upstreaming:
- Fix checkpatch issues.
- Remove the tasklet logic not needed.
- Move under tools/testing
- Create ksft for the benchmark.
- Changed slightly how the benchmark gets build. Out of tree, time_bench
is built as an independent .ko. Here it is included in
bench_page_pool.ko
Steps to run:
```
mkdir -p /tmp/run-pp-bench
make -C ./tools/testing/selftests/net/bench
make -C ./tools/testing/selftests/net/bench install
INSTALL_PATH=/tmp/run-pp-bench
rsync --delete -avz --progress /tmp/run-pp-bench mina@$SERVER:~/
ssh mina@$SERVER << EOF
cd ~/run-pp-bench && sudo ./test_bench_page_pool.sh
EOF
```
Note that by default, the Makefile will build the benchmark for the
currently installed kernel in /lib/modules/$(shell uname -r)/build. To
build against the current tree, do:
make KDIR=$(pwd) -C ./tools/testing/selftests/net/bench
Output (from Jesper):
```
sudo ./test_bench_page_pool.sh
(benchmark dmesg logs snipped)
Fast path results:
no-softirq-page_pool01 Per elem: 23 cycles(tsc) 6.571 ns
ptr_ring results:
no-softirq-page_pool02 Per elem: 60 cycles(tsc) 16.862 ns
slow path results:
no-softirq-page_pool03 Per elem: 265 cycles(tsc) 73.739 ns
```
Output (from me):
```
sudo ./test_bench_page_pool.sh
(benchmark dmesg logs snipped)
Fast path results:
no-softirq-page_pool01 Per elem: 11 cycles(tsc) 4.177 ns
ptr_ring results:
no-softirq-page_pool02 Per elem: 51 cycles(tsc) 19.117 ns
slow path results:
no-softirq-page_pool03 Per elem: 168 cycles(tsc) 62.469 ns
```
Results of course will vary based on hardware/kernel/configs, and some
variance may be there from run to run due to some noise.
Cc: Jesper Dangaard Brouer
Cc: Ilias Apalodimas
Cc: Jakub Kicinski
Cc: Toke Høiland-Jørgensen
Signed-off-by: Mina Almasry
Acked-by: Ilias Apalodimas
Signed-off-by: Jesper Dangaard Brouer
---
v5:
https://lore.kernel.org/netdev/[email protected]/
- Update results in the commit message
- Update to build against current tree
v4: https://lore.kernel.org/netdev/[email protected]/
- Fix more checkpatch and coccicheck issues (Jakub)
v3:
- Non RFC
- Collect Signed-off-by from Jesper and Acked-by Ilias.
- Move test_bench_page_pool.sh to address nipa complaint.
- Remove `static inline` in .c files to address nipa complaint.
v2:
- Move under tools/selftests (Jakub)
- Create ksft for it.
- Remove the tasklet logic no longer needed (Jesper + Toke)
RFC discussion points:
- Desirable to import it?
- Can the benchmark be imported as-is for an initial version? Or needs
lots of modifications?
- Code location. I retained the location in Jesper's tree, but a path
like net/core/bench/ may make more sense.
---
tools/testing/selftests/net/bench/Makefile| 7 +
.../selftests/net/bench/page_pool/Makefile| 17 +
.../bench/page_pool/bench_page_pool_simple.c | 276
.../net/bench/page_pool/time_bench.c | 394 ++
.../net/bench/page_pool/time_bench.h | 238 +++
.../net/bench/test_bench_page_pool.sh | 32 ++
6 files changed, 964 insertions(+)
create mode 100644 tools/testing/selftests/net/bench/Makefile
create mode 100644 tools/testing/selftests/net/bench/page_pool/Makefile
create mode 100644
tools/testing/selftests/net/bench/page_pool/bench_page_pool_simple.c
create mode 100644 tools/testing/selftests/net/bench/page_pool/time_bench.c
create mode 100644 tools/testing/selftests/net/bench/page_pool/time_bench.h
create mode 100755 tools/testing/selftests/net/bench/test_bench_page_pool.sh
diff --git a/tools/testing/selftests/net/bench/Makefile
b/tools/testing/selftests/net/bench/Makefile
new file mode 100644
index ..2546c45e42f7
--- /dev/null
+++ b/tools/testing/selftests/net/bench/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+TEST_GEN_MODS_DIR := page_pool
+
+TEST_PROGS += test_bench_page_pool.sh
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/net/bench/page_pool/Makefile
b/tools/testing/selftests/net/bench/page_pool/Makefile
new file mode 100644
index ..0549a16ba275
--- /dev/null
+++ b/tools/testing/selftests/net/bench/page_pool/Makefile
@@ -0,0 +1,17 @@
+BENCH_PAGE_POOL_SIMPLE_TEST_DIR := $(realpath $(dir $(abspath $(lastword
$(MAKEFILE_LIST)
+KDIR ?= /lib/modules/$(shell uname -r)/build
+
+ifeq ($(V),1)
+Q =
+else
+Q = @
+endif
+
+obj-m += bench_page_pool.o
+bench_page_pool-y += bench_page_pool_simple.o time_bench.o
+
+a
