On 5/12/22 10:35, Stanisław Kardach wrote:
On Thu, May 12, 2022 at 10:04 AM Heinrich Schuchardt
<heinrich.schucha...@canonical.com> wrote:
On 5/5/22 19:29, Stanislaw Kardach wrote:
This patchset adds support for building and running DPDK on 64bit RISC-V
architecture. The initial support targets rv64gc (rv64imafdc) ISA and
was tested on SiFive Unmatched development board with the Freedom U740
SoC running Linux (freedom-u-sdk based kernel).
I have tested this codebase using DPDK unit and perf tests as well as
test-pmd, l2fwd and l3fwd examples.
The NIC attached to the DUT was Intel X520-DA2 which uses ixgbe PMD.
On the UIO side, since U740 does not have an IOMMU, I've used igb_uio,
uio_pci_generic and vfio-pci noiommu drivers.
Commits 1-2 fix small issues which are encountered if a given platform
does not support any vector operations (which is the case with U740).
Commit 3 introduces EAL and build system support for RISC-V architecture
as well as documentation updates.
Commits 4-7 add missing defines and stubs to enable RISC-V operation in
non-EAL parts.
Commit 8 adds RISC-V specific cpuflags test.
Commit 9 works around a bug in the current GCC in test_ring compiled
with -O0 or -Og.
Commit 10 adds RISC-V testing to test-meson-builds.sh automatically
iterating over cross-compile config files (currently present for
generic rv64gc and SiFive U740).
Commit 11 extends hash r/w perf test by displaying both HTM and non-HTM
measurements. This is an extraneous commit which is not directly
needed for RISC-V support but was noticed when we have started
gathering test results. If needed, I can submit it separately.
I appreciate Your comments and feedback.
Best Regards,
Stanislaw Kardach
On an SiFive Unmatched board with Ubuntu Jammy I have been compiling
DPDK origin/main (36db4a1ad464) with this series and the test patches
that you split off.
Thank you for testing!
The fast tests are either skipped or succeed. But I have seen errors for
the performance tests:
4 1GiB hugepages defined
no network devices bound
vfio_pci loaded (no IOMMU)
sudo meson -t 30
129/173 DPDK:perf-tests / pmd_perf_autotest FAIL 7.85s (exit status 255
or signal 127 SIGinvalid)
>>> DPDK_TEST=pmd_perf_autotest MALLOC_PERTURB_=135
build/app/test/dpdk-test
This test I believe requires at least a single NIC port. Conversely
fast-tests seem to fail if a device is bound. DPDK CI doesn't bind any
on fast-tests, I'm not sure if perf-tests suite is run there.
On my local x86 machine perf-tests suite fails when no device is bound too.
136/173 DPDK:perf-tests / ipsec_perf_autotest FAIL 4.47s (exit status
255 or signal 127 SIGinvalid)
>>> MALLOC_PERTURB_=181 DPDK_TEST=ipsec_perf_autotest
build/app/test/dpdk-test
On my unmatched I don't have openssl compiled so most crypto tests are
skipped (since no openssl is installed), aside of this one and it
indeed fails. I'm not sure whether it's a matter of a lack of a guard
to skip this test if there's no crypto backend built or if it's a real
issue.
I admit I haven't tried crypto libraries with RISC-V openssl as
there's no real HW offloading on Unmatched.
160/173 DPDK:driver-tests / link_bonding_mode4_autotest FAIL 31.80s
killed by signal 11 SIGSEGV
>>> MALLOC_PERTURB_=62 DPDK_TEST=link_bonding_mode4_autotest
build/app/test/dpdk-test
I use -t40 when testing on Unmatched. Unfortunately lots of the tests
hit timeouts.
These are the patches used:
# [PATCH 1/1] lpm: add a scalar version of lookupx4 function
https://inbox.dpdk.org/dev/20220510115824.457885-1-...@semihalf.com/
# [PATCH 1/1] examples/l3fwd: fix scalar LPM compilation
https://inbox.dpdk.org/dev/20220510115844.458009-1-...@semihalf.com/
# [PATCH 1/1] test/hash: report non HTM numbers for single r/w
https://inbox.dpdk.org/dev/20220510115734.457718-1-...@semihalf.com/
# [PATCH 1/1] test/ring: remove excessive inlining
https://inbox.dpdk.org/dev/20220510115758.457794-1-...@semihalf.com/
# [PATCH v3 0/8] Introduce support for RISC-V architecture
https://inbox.dpdk.org/dev/20220510154849.530872-1-...@semihalf.com/
# [PATCH 1/1] drivers: define OPENSSL_API_COMPAT
https://inbox.dpdk.org/dev/20220510150635.61975-1-heinrich.schucha...@canonical.com/
Running sudo app/test/dpdk-test does not show the errors:
RTE>>pmd_perf_autotest
Start PMD RXTX cycles cost test.
At least 1 port(s) used for perf. test
Test Failed
RTE>>ipsec_perf_autotest
USER1: rte_ipsec_pkt_crypto_prepare fail
Test Failed
RTE>>link_bonding_mode4_autotest
...
Test OK
RTE>
Best regards
Heinrich+
The above failures for performance tests without a bound NIC can be
reproduced on x86_64. So this is not RISC-V specific.
With both ports of an Intel Corporation Ethernet Controller 10-Gigabit
X540-AT2 bound to the sfio-pci driver some fast tests fail on the
Unmatched board:
16/173 DPDK:fast-tests / eal_flags_n_opt_autotest FAIL 5.54s (exit
status 255 or signal 127 SIGinvalid)
>>> DPDK_TEST=eal_flags_n_opt_autotest MALLOC_PERTURB_=31
build/app/test/dpdk-test --file-prefix=eal_flags_n_opt_autotest
18/173 DPDK:fast-tests / eal_flags_no_huge_autotest FAIL 5.31s (exit
status 255 or signal 127 SIGinvalid)
>>> DPDK_TEST=eal_flags_no_huge_autotest MALLOC_PERTURB_=150
build/app/test/dpdk-test --file-prefix=eal_flags_no_huge_autotest
21/173 DPDK:fast-tests / eal_flags_vdev_opt_autotest FAIL 5.38s (exit
status 255 or signal 127 SIGinvalid)
>>> MALLOC_PERTURB_=71 DPDK_TEST=eal_flags_vdev_opt_autotest
build/app/test/dpdk-test --file-prefix=eal_flags_vdev_opt_autotest
25/173 DPDK:fast-tests / eal_flags_misc_autotest FAIL 5.58s (exit
status 255 or signal 127 SIGinvalid)
>>> DPDK_TEST=eal_flags_misc_autotest MALLOC_PERTURB_=130
build/app/test/dpdk-test --file-prefix=eal_flags_misc_autotest
The eal* tests succeed on x86_64 with a bound Intel I211 NIC.
If the eal* tests are run depends on the installed Linux packages. I
have used 'apt build-dep dpdk' on Ubuntu Jammy to install prerequisite
packages before building DPDK.
Best regards
Heinrich
NOTE: This work was sponsored by StarFive and SiFive which is signified by
"Sponsored-by:" sign-offs in each commit message. After discussing it
with Thomas Monjalon it seemed a better choice than "Suggested-by" which
does not fully convey the nature of involvement. However it makes
Linux checkpatch unhappy so I'm not sure if I shouldn't change the
sign-offs.
NOTE2: I have added maintainers for each commit based on MAINTAINERS file.
However some modules (l3fwd, net/tap and cpuflags unit tests) do not have
any maintainers assigned, hence I've targeted dev@dpdk.org mailing list as
if it was a commit adding new files.
Michal Mazurek (3):
lpm: add a scalar version of lookupx4 function
eal: add initial support for RISC-V architecture
test/cpuflags: add test for RISC-V cpu flag
Stanislaw Kardach (8):
examples/l3fwd: fix scalar LPM compilation
net/ixgbe: enable vector stubs for RISC-V
net/memif: set memfd syscall ID on RISC-V
net/tap: set BPF syscall ID for RISC-V
examples/l3fwd: enable RISC-V operation
test/ring: disable problematic tests for RISC-V
devtools: add RISC-V to test-meson-builds.sh
test/hash: report non HTM numbers for single r/w
MAINTAINERS | 6 +
app/test/test_cpuflags.c | 81 ++++++++++
app/test/test_hash_readwrite.c | 8 +-
app/test/test_ring.c | 8 +
app/test/test_xmmt_ops.h | 16 ++
config/meson.build | 2 +
config/riscv/meson.build | 148 ++++++++++++++++++
config/riscv/riscv64_linux_gcc | 17 ++
config/riscv/riscv64_sifive_u740_linux_gcc | 19 +++
devtools/test-meson-builds.sh | 6 +
doc/guides/contributing/design.rst | 2 +-
.../linux_gsg/cross_build_dpdk_for_riscv.rst | 125 +++++++++++++++
doc/guides/linux_gsg/index.rst | 1 +
doc/guides/nics/features.rst | 5 +
doc/guides/nics/features/default.ini | 1 +
doc/guides/nics/features/ixgbe.ini | 1 +
doc/guides/rel_notes/release_22_07.rst | 29 ++++
drivers/net/i40e/meson.build | 6 +
drivers/net/ixgbe/ixgbe_rxtx.c | 4 +-
drivers/net/memif/rte_eth_memif.h | 2 +
drivers/net/tap/tap_bpf.h | 2 +
examples/l3fwd/l3fwd_em.c | 8 +
examples/l3fwd/l3fwd_fib.c | 2 +
examples/l3fwd/l3fwd_lpm.c | 2 +-
lib/eal/riscv/include/meson.build | 23 +++
lib/eal/riscv/include/rte_atomic.h | 52 ++++++
lib/eal/riscv/include/rte_byteorder.h | 44 ++++++
lib/eal/riscv/include/rte_cpuflags.h | 55 +++++++
lib/eal/riscv/include/rte_cycles.h | 103 ++++++++++++
lib/eal/riscv/include/rte_io.h | 21 +++
lib/eal/riscv/include/rte_mcslock.h | 18 +++
lib/eal/riscv/include/rte_memcpy.h | 63 ++++++++
lib/eal/riscv/include/rte_pause.h | 31 ++++
lib/eal/riscv/include/rte_pflock.h | 17 ++
lib/eal/riscv/include/rte_power_intrinsics.h | 22 +++
lib/eal/riscv/include/rte_prefetch.h | 50 ++++++
lib/eal/riscv/include/rte_rwlock.h | 44 ++++++
lib/eal/riscv/include/rte_spinlock.h | 67 ++++++++
lib/eal/riscv/include/rte_ticketlock.h | 21 +++
lib/eal/riscv/include/rte_vect.h | 55 +++++++
lib/eal/riscv/meson.build | 11 ++
lib/eal/riscv/rte_cpuflags.c | 122 +++++++++++++++
lib/eal/riscv/rte_cycles.c | 77 +++++++++
lib/eal/riscv/rte_hypervisor.c | 13 ++
lib/eal/riscv/rte_power_intrinsics.c | 56 +++++++
lib/lpm/meson.build | 1 +
lib/lpm/rte_lpm.h | 4 +-
lib/lpm/rte_lpm_scalar.h | 122 +++++++++++++++
meson.build | 2 +
49 files changed, 1588 insertions(+), 7 deletions(-)
create mode 100644 config/riscv/meson.build
create mode 100644 config/riscv/riscv64_linux_gcc
create mode 100644 config/riscv/riscv64_sifive_u740_linux_gcc
create mode 100644 doc/guides/linux_gsg/cross_build_dpdk_for_riscv.rst
create mode 100644 lib/eal/riscv/include/meson.build
create mode 100644 lib/eal/riscv/include/rte_atomic.h
create mode 100644 lib/eal/riscv/include/rte_byteorder.h
create mode 100644 lib/eal/riscv/include/rte_cpuflags.h
create mode 100644 lib/eal/riscv/include/rte_cycles.h
create mode 100644 lib/eal/riscv/include/rte_io.h
create mode 100644 lib/eal/riscv/include/rte_mcslock.h
create mode 100644 lib/eal/riscv/include/rte_memcpy.h
create mode 100644 lib/eal/riscv/include/rte_pause.h
create mode 100644 lib/eal/riscv/include/rte_pflock.h
create mode 100644 lib/eal/riscv/include/rte_power_intrinsics.h
create mode 100644 lib/eal/riscv/include/rte_prefetch.h
create mode 100644 lib/eal/riscv/include/rte_rwlock.h
create mode 100644 lib/eal/riscv/include/rte_spinlock.h
create mode 100644 lib/eal/riscv/include/rte_ticketlock.h
create mode 100644 lib/eal/riscv/include/rte_vect.h
create mode 100644 lib/eal/riscv/meson.build
create mode 100644 lib/eal/riscv/rte_cpuflags.c
create mode 100644 lib/eal/riscv/rte_cycles.c
create mode 100644 lib/eal/riscv/rte_hypervisor.c
create mode 100644 lib/eal/riscv/rte_power_intrinsics.c
create mode 100644 lib/lpm/rte_lpm_scalar.h
