On 9/7/23 08:22, Nicolson Ken (ニコルソン ケン) wrote:
Hi David,
Hmm, maybe the issue is at my end - to configure QEMU, I followed the tutorial
at https://www.redhat.com/en/blog/hands-vhost-user-warm-welcome-dpdk
Are you seeing the vhost port getting initialised in testpmd output?
Yes, I get a lot of VHOST_CONFIG messages, ending with:
VHOST_CONFIG: (/tmp/sock0) virtio is now ready for processing.
Rx csum will be done in SW, may impact performance.
***
UPDATE: I've been reading more of the manual while doing the troubleshooting
below, and I think I've found a major issue while checking the Guest OS.
According to https://doc.dpdk.org/guides-21.11/linux_gsg/linux_drivers.html
dmesg | tail
...
[ 1297.875090] vfio-pci: probe of 0000:31:00.0 failed with error -22
I get the above in the Guest when trying to do devbind, and I also get this on
both Host and Guest:
cat /boot/config-$(uname -r) | grep NOIOMMU
CONFIG_VFIO_NOIOMMU=y
Should that actually be "N"? Does "is not set" equal to no IOMMU? I should
follow the grubby settings given on that RedHat page, I think. If that is off, then I would guess
that that could very well be a source of all my issues.
CONFIG_VFIO_NOIOMMU=y is valid but it just build noiommu support, what
you need is to enable it at probe time:
# modprobe vfio enable_unsafe_noiommu_mode=y
# cat /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
Y
But I understood you wanted to inject packets in the guest kernel. If
this is true it should not be necessary, the Virtio device has to be
bound to Kernel Virtio-net driver in the guest, not to VFIO/Virtio PMD.
Maxime
Back to the previous contents:
***
Now, I try your example - in another terminal on the Host, I'll use "ping -v -I dtap0 -6
fe80::5054:ff:fe01:7d00", which is the address of the interface created in the Guest OS by QEMU
(hmm, my <mac address=> seems to have been ignored), after I do "sudo ip link set enp9s0
up" there.
First, ping output (if I don't use "-I dtap0", nothing happens):
$ ping -v -I dtap0 -6 fe80::5054:ff:fe01:7d00
ping: Warning: source address might be selected on device other than: dtap0
PING fe80::5054:ff:fe01:7d00(fe80::5054:ff:fe01:7d00) from :: dtap0: 56 data
bytes
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=1 Destination unreachable:
Address unreachable
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=2 Destination unreachable:
Address unreachable
From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=3 Destination unreachable:
Address unreachable
^C
--- fe80::5054:ff:fe01:7d00 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3055ms
testpmd> set verbose 3
Change verbose level from 0 to 3
testpmd> start
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP
allocation mode: native
Logical Core 1 (socket 0) forwards packets on 2 streams:
RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
Then for each for the 6 attempted pings I see:
port 0/queue 0: received 1 packets
src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd
- length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw ptype: L2_ETHER
L3_IPV6 - l2_len=14 - l3_len=40 - Receive queue=0x0
ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN
RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
port 1/queue 0: sent 1 packets
src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd
- length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw ptype: L2_ETHER
L3_IPV6 - l2_len=14 - l3_len=40 - Send queue=0x0
ol_flags: RTE_MBUF_F_TX_L4_NO_CKSUM
Thanks for the help,
Ken
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Wednesday, September 6, 2023 5:41 PM
To: Nicolson Ken (ニコルソン ケン) <[email protected]>
Cc: [email protected]; Maxime Coquelin <[email protected]>; Xia, Chenbo
<[email protected]>
Subject: Re: Implementing a simple TAP PMD to dpdk-vhost structure
On Wed, Sep 6, 2023 at 9:53 AM Nicolson Ken (ニコルソン ケン)
<[email protected]> wrote:
Alternatively, did you consider using testpmd with the vhost pmd instead ?
I've tried that before, but as far as I can see from net/vhost/rte_eth_host.c
it uses rte_vost_enqueue/dequeue_burst() to basically act as a loopback for the
Guest OS. I use:
$ sudo dpdk-testpmd -l 0-3 -n 4 --vdev 'net_tap0' --vdev
'net_vhost1,iface=/tmp/sock0,client=1' -- -i
But if I feed data in using "tcpreplay -I dtap0 ...", "show port stats all"
shows everything going into the TAP but nothing is forward to vhost.
Well, pinging from a dtap0 netdev in the host to a virtio-net netdev in a guest
works me.
testpmd> set verbose 3
Change verbose level from 0 to 3
testpmd> start
io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP
allocation mode: native Logical Core 1 (socket 1) forwards packets on 2 streams:
RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01
RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00
port 0/queue 0: received 16 packets
src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd
- length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT
- sw ptype: L2_ETHER L3_IPV6_EXT - l2_len=14 - l3_len=48 - Receive
queue=0x0
ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN
RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd
- length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT
- sw ptype: L2_ETHER L3_IPV6_EXT - l2_len=14 - l3_len=48 - Receive
queue=0x0
ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN
RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
src=26:9B:E2:29:7E:C6 - dst=33:33:FF:29:7E:C6 - pool=mb_pool_0 - type=0x86dd
- length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw
ptype: L2_ETHER L3_IPV6 - l2_len=14 - l3_len=40 - Receive queue=0x0
ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN
RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN
Are you seeing the vhost port getting initialised in testpmd output?
How are you sure that nothing is forwarded?
--
David Marchand
