Using VPP+DPDK virtio to connect with VPP + vhost-user is not actively
maintained. I got it working couple years ago by committing some changes to the
DPDK virtio code. Since then, I’ve not been playing with it anymore. Breakage
is possible. I could spend a whole week on it to get it working again (maybe).
However, there is no telling when it will break again. Here is my
recommendation:
If you are using containers, use memif interface.
If you are not using containers, use VPP native virtio to connect to VPP
vhost-user.
Steven
From: <vpp-dev@lists.fd.io> on behalf of Sharon Enoch <shar...@amzetta.com>
Date: Monday, August 12, 2019 at 11:32 AM
To: "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] VPP with DPDK vhost ---- VPP with DPDK virtio
Tried the same with the latest VPP version master which has the DPDK 19.05
version and still faced the same issue..
VPP + DPDK vhost mode
DBGvpp# show version
vpp v20.01-rc0~20-g6b53fd516 built by root on kickseed at Tue Aug 13 02:12:38
JST 2019
DBGvpp# show int
Name Idx State MTU (L3/IP4/IP6/MPLS)
Counter Count
VhostEthernet0 1 up 1500/0/0/0
local0 0 down 0/0/0/0
DBGvpp# show hardware-interfaces
Name Idx Link Hardware
VhostEthernet0 1 up VhostEthernet0
Link speed: 10 Gbps
Ethernet address 56:48:4f:53:54:00
VhostEthernet
carrier up full duplex mtu 1500
flags: admin-up pmd maybe-multiseg
rx: queues 1 (max 1), desc 1024 (min 0 max 65535 align 1)
tx: queues 1 (max 1), desc 1024 (min 0 max 65535 align 1)
max rx packet len: -1
promiscuous: unicast off all-multicast off
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip
rx offload active: none
tx offload avail: vlan-insert multi-segs
tx offload active: multi-segs
rss avail: none
rss active: none
tx burst function: (nil)
rx burst function: (nil)
local0 0 down local0
Link speed: unknown
local
VPP + DPDK VIRTIO
DBGvpp# show int
Name Idx State MTU (L3/IP4/IP6/MPLS)
Counter Count
VirtioUser0 1 up 1500/0/0/0 tx packets
5
tx bytes
550
local0 0 down 0/0/0/0
DBGvpp# show hardware-interfaces
Name Idx Link Hardware
VirtioUser0 1 up VirtioUser0
Link speed: 10 Gbps
Ethernet address e2:84:b1:74:4d:f2
Virtio User
carrier up full duplex mtu 1500
flags: admin-up pmd maybe-multiseg
rx: queues 1 (max 1), desc 1024 (min 0 max 65535 align 1)
tx: queues 1 (max 1), desc 1024 (min 0 max 65535 align 1)
max rx packet len: 9728
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip udp-cksum tcp-cksum tcp-lro jumbo-frame
rx offload active: jumbo-frame
tx offload avail: vlan-insert udp-cksum tcp-cksum tcp-tso multi-segs
tx offload active: multi-segs
rss avail: none
rss active: none
tx burst function: virtio_xmit_pkts_inorder
rx burst function: virtio_recv_pkts_inorder
tx frames ok 5
tx bytes ok 550
local0 0 down local0
Link speed: unknown
local
I noticed the following new logs though in the new version in the show log
output
The below on the virtio side
DBGvpp# show log
2019/08/13 03:10:43:956 err perfmon No table for cpuid 306e4
2019/08/13 03:10:43:956 err perfmon model 3e, stepping 4
2019/08/13 03:10:43:984 warn dpdk EAL init args: -c 2 -n 4
--in-memory --no-pci --log-level 8 --huge-dir /dev/hugepages --vdev
virtio_user0,path=/opt/sock/sock2.sock --file-prefix vpp --master-lcore 1
2019/08/13 03:10:46:666 notice dpdk DPDK drivers found 1 ports...
2019/08/13 03:10:46:666 warn dpdk unsupported rx offloads requested
on port 0: scatter
2019/08/13 03:10:46:669 notice dpdk EAL: Detected 24 lcore(s)
2019/08/13 03:10:46:669 notice dpdk EAL: Detected 2 NUMA nodes
2019/08/13 03:10:46:669 notice dpdk EAL: Probing VFIO support...
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0xa80001000 != 0x7f7c00000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0xc00002000 != 0x7f73c0000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0xd80003000 != 0x7f6b80000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0xf00004000 != 0x7f6340000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0x1080005000 != 0x7f5b00000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0x10c000a000 != 0x7f8475b21000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0x120000b000 != 0x7f52c0000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0x1240010000 != 0x7f8475b20000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: WARNING! Base virtual
address hint (0x1380011000 != 0x7f4a80000000) not respected!
2019/08/13 03:10:46:669 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:10:46:669 notice dpdk EAL: VFIO support not
initialized
2019/08/13 03:10:46:670 notice dpdk EAL: Couldn't map new region for
DMA
2019/08/13 03:10:46:670 notice dpdk EAL: VFIO support not
initialized
2019/08/13 03:10:46:670 notice dpdk EAL: Couldn't map new region for
DMA
The below on the vhost side
DBGvpp# show log
2019/08/13 03:08:43:037 err perfmon No table for cpuid 306e4
2019/08/13 03:08:43:037 err perfmon model 3e, stepping 4
2019/08/13 03:08:43:049 warn dpdk EAL init args: -c 2 -n 4
--in-memory --no-pci --log-level 8 --huge-dir /dev/hugepages --vdev
eth_vhost0,iface=/opt/sock/sock2.sock --file-prefix vpp --master-lcore 1
2019/08/13 03:08:44:461 notice dpdk DPDK drivers found 1 ports...
2019/08/13 03:08:44:461 warn dpdk unsupported rx offloads requested
on port 0: jumbo-frame scatter
2019/08/13 03:08:44:464 notice dpdk EAL: Detected 24 lcore(s)
2019/08/13 03:08:44:464 notice dpdk EAL: Detected 2 NUMA nodes
2019/08/13 03:08:44:464 notice dpdk EAL: Probing VFIO support...
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0xa80001000 != 0x7f1440000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0xc00002000 != 0x7f0c00000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0xd80003000 != 0x7f03c0000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0xf00004000 != 0x7efb80000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0x1080005000 != 0x7ef340000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0x10c000a000 != 0x7f1cd1365000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0x120000b000 != 0x7eeb00000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0x1240010000 != 0x7f1cd1364000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk EAL: WARNING! Base virtual
address hint (0x1380011000 != 0x7ee2c0000000) not respected!
2019/08/13 03:08:44:464 notice dpdk EAL: This may cause issues
with mapping memory into secondary processes
2019/08/13 03:08:44:464 notice dpdk VHOST_CONFIG: vhost-user server:
socket created, fd: 18
2019/08/13 03:08:44:464 notice dpdk VHOST_CONFIG: bind to
/opt/sock/sock2.sock
2019/08/13 03:08:44:464 notice dpdk EAL: VFIO support not
initialized
2019/08/13 03:08:44:464 notice dpdk EAL: Couldn't map new region for
DMA
2019/08/13 03:08:44:464 notice dpdk EAL: VFIO support not
initialized
2019/08/13 03:08:44:464 notice dpdk EAL: Couldn't map new region for
DMA
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: new vhost user
connection is 26
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: new device, handle
is 0
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_OWNER
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_GET_FEATURES
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_CALL
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: vring call idx:0
file:27
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_CALL
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: vring call idx:1
file:28
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_FEATURES
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_MEM_TABLE
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: guest memory region
0, size: 0x40000000
2019/08/13 03:10:44:585 notice dpdk guest physical addr:
0x140000000
2019/08/13 03:10:44:585 notice dpdk guest virtual addr:
0x140000000
2019/08/13 03:10:44:585 notice dpdk host virtual addr:
0x7ee280000000
2019/08/13 03:10:44:585 notice dpdk mmap addr : 0x7ee280000000
2019/08/13 03:10:44:585 notice dpdk mmap size : 0x40000000
2019/08/13 03:10:44:585 notice dpdk mmap align: 0x40000000
2019/08/13 03:10:44:585 notice dpdk mmap off : 0x0
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_NUM
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_BASE
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ADDR
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_KICK
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: vring kick idx:0
file:30
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_NUM
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_BASE
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ADDR
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_KICK
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: vring kick idx:1
file:31
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: virtio is now ready
for processing.
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: set queue enable: 1
to qp idx: 0
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:44:585 notice dpdk VHOST_CONFIG: set queue enable: 1
to qp idx: 1
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: set queue enable: 0
to qp idx: 0
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: set queue enable: 0
to qp idx: 1
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_MEM_TABLE
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: guest memory region
0, size: 0x40000000
2019/08/13 03:10:45:820 notice dpdk guest physical addr:
0x140000000
2019/08/13 03:10:45:820 notice dpdk guest virtual addr:
0x140000000
2019/08/13 03:10:45:820 notice dpdk host virtual addr:
0x7ee280000000
2019/08/13 03:10:45:820 notice dpdk mmap addr : 0x7ee280000000
2019/08/13 03:10:45:820 notice dpdk mmap size : 0x40000000
2019/08/13 03:10:45:820 notice dpdk mmap align: 0x40000000
2019/08/13 03:10:45:820 notice dpdk mmap off : 0x0
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: guest memory region
1, size: 0x40000000
2019/08/13 03:10:45:820 notice dpdk guest physical addr:
0x7f6340000000
2019/08/13 03:10:45:820 notice dpdk guest virtual addr:
0x7f6340000000
2019/08/13 03:10:45:820 notice dpdk host virtual addr:
0x7ee240000000
2019/08/13 03:10:45:820 notice dpdk mmap addr : 0x7ee240000000
2019/08/13 03:10:45:820 notice dpdk mmap size : 0x40000000
2019/08/13 03:10:45:820 notice dpdk mmap align: 0x40000000
2019/08/13 03:10:45:820 notice dpdk mmap off : 0x0
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: set queue enable: 1
to qp idx: 0
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: read message
VHOST_USER_SET_VRING_ENABLE
2019/08/13 03:10:45:820 notice dpdk VHOST_CONFIG: set queue enable: 1
to qp idx: 1
testpmd VIRTIO 19.05 to VHOST 19.05 is working though
root@kickseed:~/vppnew/dpdk/build/app# ./testpmd -l 0-1 -n 4 --socket-mem
1024,1024 --vdev='virtio_user0,path=/opt/sock/sock2.sock' --no-pci --no-shconf
--huge-dir=/dev/hugepages --log-level=8 -- -i
testpmd> show port stats 0
######################## NIC statistics for port 0 ########################
RX-packets: 0 RX-missed: 0 RX-bytes: 0
RX-errors: 0
RX-nombuf: 0
TX-packets: 124448 TX-errors: 0 TX-bytes: 7964672
Throughput (since last show)
Rx-pps: 0
Tx-pps: 3625
############################################################################
root@kickseed:~/vppnew/dpdk/build/app# ./testpmd -l 0-1 -n 4 --socket-mem
1024,1024 --vdev='eth_vhost0,iface=/opt/sock/sock2.sock' --no-pci --in-memory
--huge-dir=/dev/hugepages --log-level=8 -- -i
testpmd> show port stats 0
######################## NIC statistics for port 0 ########################
RX-packets: 124448 RX-missed: 0 RX-bytes: 7964672
RX-errors: 0
RX-nombuf: 0
TX-packets: 0 TX-errors: 0 TX-bytes: 0
Throughput (since last show)
Rx-pps: 2485
Tx-pps: 0
############################################################################
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#13713): https://lists.fd.io/g/vpp-dev/message/13713
Mute This Topic: https://lists.fd.io/mt/32843106/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
