Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Thanks Alec for your help.
Now I can start it but it fails to connect to the server.
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,092 INFO Loading configuration file:
/tmp/nfvbench/nfvbench.cfg
2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)...
2020-08-31 21:43:33,175 INFO Starting TRex ...
2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0
--no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh
--mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown
2020-08-31 21:43:33,181 INFO TRex server is running...
nohup: ignoring input and appending output to 'nohup.out'
2020-08-31 21:43:37,189 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:41,196 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:45,203 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
I tried to bind the devices to vfio-pci manually but it still cannot connect to
the server.
Regards,
Charlie Li
From: Alec Hothan (ahothan)
Sent: Monday, August 31, 2020 1:45 PM
To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
Hi Charlie,
This is likely due to the file "/tmp/nfvbench/nfvbench.cfg" missing inside the
container.
Can you check the following:
* Check where you stored the nfvbench.cfg file on the host (the quickstart
guide suggests to put the file under $HOME/nfvbench)
* Check that that directory on host is mapped to /tmp/nfvbench in the
container
To verify you can also try to cat that file from inside the container (docker
exec cat /tmp/nfvbench/nfvbench.cfg)
For further interaction, you can continue on this email thread or you can join
opnfv.slack.com #nfvbench if you prefer.
Thanks
Alec
From:
mailto:opnfv-tech-discuss@lists.opnfv.org>>
on behalf of "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 9:59 AM
To:
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hello,
I am new to NFVbench and is following the link below to get started
https://opnfv-nfvbench.readthedocs.io/en/latest/testing/user/userguide/quickstart_docker.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopnfv-nfvbench.readthedocs.io%2Fen%2Flatest%2Ftesting%2Fuser%2Fuserguide%2Fquickstart_docker.html=02%7C01%7CCharlie.Li%40amd.com%7C800cb9821b534c2bca7c08d84dde0318%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637344963677109059=gxmMMAgWOXfByQ8tGE4o%2FSYtDWB4A0PllGM8L4nblnw%3D=0>
The NFVbench container is started successfully
$ nfvbench --version
4.3.1.dev4
But when I try the example of run, it fails
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-27 20:10:45,424 INFO Loading configuration string:
/tmp/nfvbench/nfvbench.cfg
2020-08-27 20:10:45,425 ERROR String /tmp/nfvbench/nfvbench.cfg is not well
formatted. Please verify your yaml/json string. If string is a file path, file
was not found. Please use correct path and verify it is visible to container if
you run nfvbench in container.
2020-08-27 20:10:45,425 ERROR {'status': 'ERROR', 'error_message': 'Traceback
(most recent call last):\n File "/nfvbench/nfvbench/config.py", line 46, in
config_loads\ncfg = AttrDict(yaml.safe_load(cfg_text))\n File
"/usr/local/lib/python3.6/dist-packages/attrdict/dictionary.py", line 17, in
__init__\nsuper(AttrDict, self).__init__(*args, **kwargs)\nValueError:
dictionary update sequence element #0 has length 1; 2 is required\n\nDuring
handling of the above exception, another exception occurred:\n\nTraceback (most
recent call last):\n File "/nfvbench/nfvbench/nfvbench.py", line 611, in
main\nconfig = config_loads(opts.config, config, whitelist_keys)\n File
"/nfvbench/nfvbench/config.py", line 55, in config_loads\nraise
Exception(e)\nException: dictionary update sequence element #0 has length 1; 2
is required\n'}
dictionary update sequence element #0 has length 1; 2 is required
The /tmp/nfvbench/nfvbench.cfg is cut and paste (OpenStack is not used and only
PCIe addresses are modified)
traffic_generator:
generator_profile:
- name: trex-local
tool: TRex
ip: 127.0.0.1
cores: 3
software_mode: false
interfaces:
- port: 0
pci: "04:00.0"
- port: 1
pci: "04:00.1"
intf_speed:
Any help is apprec
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use] Hi Alec, I am using Ubuntu 20.04.1 with kernel 5.4.0-42. Hugepage is configured in the grub $ grep HugePages_ /proc/meminfo HugePages_Total:4096 HugePages_Free: 4096 HugePages_Rsvd:0 HugePages_Surp:0 In the host, I can manually bind the devices to vfio-pci, but in the container, it cannot bind the devices to vfio-pci or igb_uio. Please see the logs in the attached file. Regards, Charlie Li From: Alec Hothan (ahothan) Sent: Monday, August 31, 2020 5:36 PM To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails [CAUTION: External Email] This likely indicates that Trex failed to start properly, possibly pci address issue or other dpdk related issue. Can you check/share the trex config and trex log which are located inside the container at: /etc/trex_cfg.yaml /tmp/trex.log (you can use docker cp or simply docker exec cat /tmp/trex.log) Make sure you have your server is set properly to run dpdk (you need huge pages configured). Alec From: "Li, Charlie" mailto:charlie...@amd.com>> Date: Monday, August 31, 2020 at 3:07 PM To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>, "opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>" mailto:opnfv-tech-discuss@lists.opnfv.org>> Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails [AMD Public Use] Thanks Alec for your help. Now I can start it but it fails to connect to the server. $ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps 2020-08-31 21:43:30,092 INFO Loading configuration file: /tmp/nfvbench/nfvbench.cfg 2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps 2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)... 2020-08-31 21:43:33,175 INFO Starting TRex ... 2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0 --no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh --mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown 2020-08-31 21:43:33,181 INFO TRex server is running... nohup: ignoring input and appending output to 'nohup.out' 2020-08-31 21:43:37,189 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... 2020-08-31 21:43:41,196 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... 2020-08-31 21:43:45,203 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... I tried to bind the devices to vfio-pci manually but it still cannot connect to the server. Regards, Charlie Li From: Alec Hothan (ahothan) mailto:ahot...@cisco.com>> Sent: Monday, August 31, 2020 1:45 PM To: Li, Charlie mailto:charlie...@amd.com>>; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails [CAUTION: External Email] Hi Charlie, This is likely due to the file "/tmp/nfvbench/nfvbench.cfg" missing inside the container. Can you check the following: * Check where you stored the nfvbench.cfg file on the host (the quickstart guide suggests to put the file under $HOME/nfvbench) * Check that that directory on host is mapped to /tmp/nfvbench in the container To verify you can also try to cat that file from inside the container (docker exec cat /tmp/nfvbench/nfvbench.cfg) For further interaction, you can continue on this email thread or you can join opnfv.slack.com #nfvbench if you prefer. Thanks Alec From: mailto:opnfv-tech-discuss@lists.opnfv.org>> on behalf of "Li, Charlie" mailto:charlie...@amd.com>> Date: Monday, August 31, 2020 at 9:59 AM To: "opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>" mailto:opnfv-tech-discuss@lists.opnfv.org>> Subject: [opnfv-tech-discuss] #nfvbench - Example of run fails [AMD Public Use] Hello, I am new to NFVbench and is following the link below to get started https://opnfv-nfvbench.readthedocs.io/en/latest/testing/user/userguide/quickstart_docker.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopnfv-nfvbench.readthedocs.io%2Fen%2Flatest%2Ftesting%2Fuser%2Fuserguide%2Fquickstart_docker.html=02%7C01%7CCharlie.Li%40amd.com%7Ca8b345a1f5134d2f91f508d84dfe3b03%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C63734510157618=Yh8wvZoPBaBXqCGaHLZbZvidWTg%2B%2BFQvqfG1TJ9x3vg%3D=0> The NFVbench container is started successfully $ nfvbench --version 4.3.1.dev4 But when I try the example of run, it fails $ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps 2020-08-27 20:10:45,424 INFO Loading configuration string: /tmp/nfvbench/nfvbench.cfg 2020-08-27 20:10:45,425 ERROR String /tmp/nfvbench/nfvbench.cfg is not well formatted. Please ver
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hi Alec,
I was trying to load VFIO as below but it failed to bind the devices to
vfio-pci in the container.
sudo modprobe vfio-pci
sudo chmod a+x /dev/vfio
sudo chmod 0666 /dev/vfio/*
When I load UIO, the devices can successfully bind to uio in the container
sudo modprobe uio
sudo insmod ${RTE_SDK}/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
However, it still cannot connect to the server
$ sudo docker exec nfvbench cat /tmp/trex.log
Trying to bind to igb_uio ...
/usr/local/bin/python3 dpdk_nic_bind.py --bind=igb_uio :04:00.4 :04:00.5
The ports are bound/configured.
Starting TRex v2.79 please wait ...
ERROR in DPDK map
Could not find requested interface 04:00.4
I will try it on an Intel NIC and see if it is NIC related.
Regards,
Charlie Li
From: Alec Hothan (ahothan)
Sent: Monday, August 31, 2020 7:05 PM
To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
Ok you're missing the kernel driver that DPDK needs to work on your host. Since
Trex cannot find any suitable driver it tries to build one (hence make
complaining ity can't find tools to recompile in the container).
http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Flinux_drivers.html=02%7C01%7CCharlie.Li%40amd.com%7C232a7ec100284996448708d84e0aa839%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637345154970820466=4Rt4kYdPpOnSlj0EU51OVk9UulPl%2FRui7S4ExCBYfAA%3D=0>
You have the choice of few drivers, we use vfio for nfvbench so you may want to
install and try that one first.
Alec
From: "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 4:35 PM
To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>,
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hi Alec,
I am using Ubuntu 20.04.1 with kernel 5.4.0-42.
Hugepage is configured in the grub
$ grep HugePages_ /proc/meminfo
HugePages_Total:4096
HugePages_Free: 4096
HugePages_Rsvd:0
HugePages_Surp:0
In the host, I can manually bind the devices to vfio-pci, but in the container,
it cannot bind the devices to vfio-pci or igb_uio. Please see the logs in the
attached file.
Regards,
Charlie Li
From: Alec Hothan (ahothan) mailto:ahot...@cisco.com>>
Sent: Monday, August 31, 2020 5:36 PM
To: Li, Charlie mailto:charlie...@amd.com>>;
opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
This likely indicates that Trex failed to start properly, possibly pci address
issue or other dpdk related issue.
Can you check/share the trex config and trex log which are located inside the
container at:
/etc/trex_cfg.yaml
/tmp/trex.log
(you can use docker cp or simply docker exec cat /tmp/trex.log)
Make sure you have your server is set properly to run dpdk (you need huge pages
configured).
Alec
From: "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 3:07 PM
To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>,
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Thanks Alec for your help.
Now I can start it but it fails to connect to the server.
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,092 INFO Loading configuration file:
/tmp/nfvbench/nfvbench.cfg
2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)...
2020-08-31 21:43:33,175 INFO Starting TRex ...
2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0
--no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh
--mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown
2020-08-31 21:43:33,181 INFO TRex server is running...
nohup: ignoring input and appending output to 'nohup.out'
2020-08-31 21:43:37,189 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:41,196 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:45,203 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
I tried to bind the devices to vfio-pci manually but it still cannot connect to
the server.
Regards,
Charlie Li
From: Alec Hothan (ahothan) ma
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
Charlie,
Pls make sure you have intel_iommu=on in your grub boot as it is required for
using vfio. For AMD it might be different (amd_iommu=on ?)
You should also add iommu=pt if supported.
I have not used uio so can't say if it will work or not.
Thanks
Alec
From: "Li, Charlie"
Date: Tuesday, September 1, 2020 at 9:55 AM
To: "Alec Hothan (ahothan)" ,
"opnfv-tech-discuss@lists.opnfv.org"
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hi Alec,
I was trying to load VFIO as below but it failed to bind the devices to
vfio-pci in the container.
sudo modprobe vfio-pci
sudo chmod a+x /dev/vfio
sudo chmod 0666 /dev/vfio/*
When I load UIO, the devices can successfully bind to uio in the container
sudo modprobe uio
sudo insmod ${RTE_SDK}/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
However, it still cannot connect to the server
$ sudo docker exec nfvbench cat /tmp/trex.log
Trying to bind to igb_uio ...
/usr/local/bin/python3 dpdk_nic_bind.py --bind=igb_uio :04:00.4 :04:00.5
The ports are bound/configured.
Starting TRex v2.79 please wait ...
ERROR in DPDK map
Could not find requested interface 04:00.4
I will try it on an Intel NIC and see if it is NIC related.
Regards,
Charlie Li
From: Alec Hothan (ahothan)
Sent: Monday, August 31, 2020 7:05 PM
To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
Ok you’re missing the kernel driver that DPDK needs to work on your host. Since
Trex cannot find any suitable driver it tries to build one (hence make
complaining ity can't find tools to recompile in the container).
http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Flinux_drivers.html=02%7C01%7CCharlie.Li%40amd.com%7C232a7ec100284996448708d84e0aa839%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637345154970820466=4Rt4kYdPpOnSlj0EU51OVk9UulPl%2FRui7S4ExCBYfAA%3D=0>
You have the choice of few drivers, we use vfio for nfvbench so you may want to
install and try that one first.
Alec
From: "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 4:35 PM
To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>,
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hi Alec,
I am using Ubuntu 20.04.1 with kernel 5.4.0-42.
Hugepage is configured in the grub
$ grep HugePages_ /proc/meminfo
HugePages_Total:4096
HugePages_Free: 4096
HugePages_Rsvd:0
HugePages_Surp:0
In the host, I can manually bind the devices to vfio-pci, but in the container,
it cannot bind the devices to vfio-pci or igb_uio. Please see the logs in the
attached file.
Regards,
Charlie Li
From: Alec Hothan (ahothan) mailto:ahot...@cisco.com>>
Sent: Monday, August 31, 2020 5:36 PM
To: Li, Charlie mailto:charlie...@amd.com>>;
opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
This likely indicates that Trex failed to start properly, possibly pci address
issue or other dpdk related issue.
Can you check/share the trex config and trex log which are located inside the
container at:
/etc/trex_cfg.yaml
/tmp/trex.log
(you can use docker cp or simply docker exec cat /tmp/trex.log)
Make sure you have your server is set properly to run dpdk (you need huge pages
configured).
Alec
From: "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 3:07 PM
To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>,
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Thanks Alec for your help.
Now I can start it but it fails to connect to the server.
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,092 INFO Loading configuration file:
/tmp/nfvbench/nfvbench.cfg
2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)...
2020-08-31 21:43:33,175 INFO Starting TRex ...
2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0
--no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh
--mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown
2020-08-31 21:43:33,181 INFO TRex server is running...
nohup: ignoring input and appending output to 'nohup.out'
2020-08-31 21:43:37,189 INFO Retrying conn
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
Ok you’re missing the kernel driver that DPDK needs to work on your host. Since Trex cannot find any suitable driver it tries to build one (hence make complaining ity can't find tools to recompile in the container). http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html You have the choice of few drivers, we use vfio for nfvbench so you may want to install and try that one first. Alec From: "Li, Charlie" Date: Monday, August 31, 2020 at 4:35 PM To: "Alec Hothan (ahothan)" , "opnfv-tech-discuss@lists.opnfv.org" Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails [AMD Public Use] Hi Alec, I am using Ubuntu 20.04.1 with kernel 5.4.0-42. Hugepage is configured in the grub $ grep HugePages_ /proc/meminfo HugePages_Total:4096 HugePages_Free: 4096 HugePages_Rsvd:0 HugePages_Surp:0 In the host, I can manually bind the devices to vfio-pci, but in the container, it cannot bind the devices to vfio-pci or igb_uio. Please see the logs in the attached file. Regards, Charlie Li From: Alec Hothan (ahothan) Sent: Monday, August 31, 2020 5:36 PM To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails [CAUTION: External Email] This likely indicates that Trex failed to start properly, possibly pci address issue or other dpdk related issue. Can you check/share the trex config and trex log which are located inside the container at: /etc/trex_cfg.yaml /tmp/trex.log (you can use docker cp or simply docker exec cat /tmp/trex.log) Make sure you have your server is set properly to run dpdk (you need huge pages configured). Alec From: "Li, Charlie" mailto:charlie...@amd.com>> Date: Monday, August 31, 2020 at 3:07 PM To: "Alec Hothan (ahothan)" mailto:ahot...@cisco.com>>, "opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>" mailto:opnfv-tech-discuss@lists.opnfv.org>> Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails [AMD Public Use] Thanks Alec for your help. Now I can start it but it fails to connect to the server. $ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps 2020-08-31 21:43:30,092 INFO Loading configuration file: /tmp/nfvbench/nfvbench.cfg 2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps 2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)... 2020-08-31 21:43:33,175 INFO Starting TRex ... 2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0 --no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh --mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown 2020-08-31 21:43:33,181 INFO TRex server is running... nohup: ignoring input and appending output to 'nohup.out' 2020-08-31 21:43:37,189 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... 2020-08-31 21:43:41,196 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... 2020-08-31 21:43:45,203 INFO Retrying connection to TRex (*** [RPC] - Failed to get server response from tcp://127.0.0.1:4501)... I tried to bind the devices to vfio-pci manually but it still cannot connect to the server. Regards, Charlie Li From: Alec Hothan (ahothan) mailto:ahot...@cisco.com>> Sent: Monday, August 31, 2020 1:45 PM To: Li, Charlie mailto:charlie...@amd.com>>; opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails [CAUTION: External Email] Hi Charlie, This is likely due to the file "/tmp/nfvbench/nfvbench.cfg" missing inside the container. Can you check the following: · Check where you stored the nfvbench.cfg file on the host (the quickstart guide suggests to put the file under $HOME/nfvbench) · Check that that directory on host is mapped to /tmp/nfvbench in the container To verify you can also try to cat that file from inside the container (docker exec cat /tmp/nfvbench/nfvbench.cfg) For further interaction, you can continue on this email thread or you can join opnfv.slack.com #nfvbench if you prefer. Thanks Alec From: mailto:opnfv-tech-discuss@lists.opnfv.org>> on behalf of "Li, Charlie" mailto:charlie...@amd.com>> Date: Monday, August 31, 2020 at 9:59 AM To: "opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>" mailto:opnfv-tech-discuss@lists.opnfv.org>> Subject: [opnfv-tech-discuss] #nfvbench - Example of run fails [AMD Public Use] Hello, I am new to NFVbench and is following the link below to get started https://opnfv-nfvbench.readthedocs.io/en/latest/testing/user/userguide/quickstart_docker.html<https://nam11.safelinks.protection.outlook.com/?url=https%3
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
This likely indicates that Trex failed to start properly, possibly pci address
issue or other dpdk related issue.
Can you check/share the trex config and trex log which are located inside the
container at:
/etc/trex_cfg.yaml
/tmp/trex.log
(you can use docker cp or simply docker exec cat /tmp/trex.log)
Make sure you have your server is set properly to run dpdk (you need huge pages
configured).
Alec
From: "Li, Charlie"
Date: Monday, August 31, 2020 at 3:07 PM
To: "Alec Hothan (ahothan)" ,
"opnfv-tech-discuss@lists.opnfv.org"
Subject: RE: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Thanks Alec for your help.
Now I can start it but it fails to connect to the server.
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,092 INFO Loading configuration file:
/tmp/nfvbench/nfvbench.cfg
2020-08-31 21:43:30,101 INFO -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-31 21:43:30,104 INFO Connecting to TRex (127.0.0.1)...
2020-08-31 21:43:33,175 INFO Starting TRex ...
2020-08-31 21:43:33,178 INFO nohup /bin/bash -c ./t-rex-64 -i -c 4 --iom 0
--no-scapy-server --unbind-unused-ports --close-at-end --vlan --hdrh
--mbuf-factor 0.2 --cfg /etc/trex_cfg.yaml &> /tmp/trex.log & disown
2020-08-31 21:43:33,181 INFO TRex server is running...
nohup: ignoring input and appending output to 'nohup.out'
2020-08-31 21:43:37,189 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:41,196 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
2020-08-31 21:43:45,203 INFO Retrying connection to TRex (*** [RPC] - Failed to
get server response from tcp://127.0.0.1:4501)...
I tried to bind the devices to vfio-pci manually but it still cannot connect to
the server.
Regards,
Charlie Li
From: Alec Hothan (ahothan)
Sent: Monday, August 31, 2020 1:45 PM
To: Li, Charlie ; opnfv-tech-discuss@lists.opnfv.org
Subject: Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
[CAUTION: External Email]
Hi Charlie,
This is likely due to the file "/tmp/nfvbench/nfvbench.cfg" missing inside the
container.
Can you check the following:
· Check where you stored the nfvbench.cfg file on the host (the
quickstart guide suggests to put the file under $HOME/nfvbench)
· Check that that directory on host is mapped to /tmp/nfvbench in the
container
To verify you can also try to cat that file from inside the container (docker
exec cat /tmp/nfvbench/nfvbench.cfg)
For further interaction, you can continue on this email thread or you can join
opnfv.slack.com #nfvbench if you prefer.
Thanks
Alec
From:
mailto:opnfv-tech-discuss@lists.opnfv.org>>
on behalf of "Li, Charlie" mailto:charlie...@amd.com>>
Date: Monday, August 31, 2020 at 9:59 AM
To:
"opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>"
mailto:opnfv-tech-discuss@lists.opnfv.org>>
Subject: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hello,
I am new to NFVbench and is following the link below to get started
https://opnfv-nfvbench.readthedocs.io/en/latest/testing/user/userguide/quickstart_docker.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fopnfv-nfvbench.readthedocs.io%2Fen%2Flatest%2Ftesting%2Fuser%2Fuserguide%2Fquickstart_docker.html=02%7C01%7CCharlie.Li%40amd.com%7C800cb9821b534c2bca7c08d84dde0318%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637344963677109059=gxmMMAgWOXfByQ8tGE4o%2FSYtDWB4A0PllGM8L4nblnw%3D=0>
The NFVbench container is started successfully
$ nfvbench --version
4.3.1.dev4
But when I try the example of run, it fails
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-27 20:10:45,424 INFO Loading configuration string:
/tmp/nfvbench/nfvbench.cfg
2020-08-27 20:10:45,425 ERROR String /tmp/nfvbench/nfvbench.cfg is not well
formatted. Please verify your yaml/json string. If string is a file path, file
was not found. Please use correct path and verify it is visible to container if
you run nfvbench in container.
2020-08-27 20:10:45,425 ERROR {'status': 'ERROR', 'error_message': 'Traceback
(most recent call last):\n File "/nfvbench/nfvbench/config.py", line 46, in
config_loads\ncfg = AttrDict(yaml.safe_load(cfg_text))\n File
"/usr/local/lib/python3.6/dist-packages/attrdict/dictionary.py", line 17, in
__init__\nsuper(AttrDict, self).__init__(*args, **kwargs)\nValueError:
dictionary update sequence element #0 has length 1; 2 is required\n\nDuring
handling of the above exception, another exception occurred:\n\nTraceback (most
recent call last):\n File "/nfvbench/nfvbench/nfvbench.py", line 611, in
main\nconfig = config_loads(opts.config, config, whitelist_keys)\n File
"/nfvbench/nfvbench/config.py", li
Re: [opnfv-tech-discuss] #nfvbench - Example of run fails
Hi Charlie,
This is likely due to the file "/tmp/nfvbench/nfvbench.cfg" missing inside the
container.
Can you check the following:
* Check where you stored the nfvbench.cfg file on the host (the quickstart
guide suggests to put the file under $HOME/nfvbench)
* Check that that directory on host is mapped to /tmp/nfvbench in the
container
To verify you can also try to cat that file from inside the container (docker
exec cat /tmp/nfvbench/nfvbench.cfg)
For further interaction, you can continue on this email thread or you can join
opnfv.slack.com #nfvbench if you prefer.
Thanks
Alec
From: on behalf of "Li, Charlie"
Date: Monday, August 31, 2020 at 9:59 AM
To: "opnfv-tech-discuss@lists.opnfv.org"
Subject: [opnfv-tech-discuss] #nfvbench - Example of run fails
[AMD Public Use]
Hello,
I am new to NFVbench and is following the link below to get started
https://opnfv-nfvbench.readthedocs.io/en/latest/testing/user/userguide/quickstart_docker.html
The NFVbench container is started successfully
$ nfvbench --version
4.3.1.dev4
But when I try the example of run, it fails
$ nfvbench -c /tmp/nfvbench/nfvbench.cfg --rate 10kpps
2020-08-27 20:10:45,424 INFO Loading configuration string:
/tmp/nfvbench/nfvbench.cfg
2020-08-27 20:10:45,425 ERROR String /tmp/nfvbench/nfvbench.cfg is not well
formatted. Please verify your yaml/json string. If string is a file path, file
was not found. Please use correct path and verify it is visible to container if
you run nfvbench in container.
2020-08-27 20:10:45,425 ERROR {'status': 'ERROR', 'error_message': 'Traceback
(most recent call last):\n File "/nfvbench/nfvbench/config.py", line 46, in
config_loads\ncfg = AttrDict(yaml.safe_load(cfg_text))\n File
"/usr/local/lib/python3.6/dist-packages/attrdict/dictionary.py", line 17, in
__init__\nsuper(AttrDict, self).__init__(*args, **kwargs)\nValueError:
dictionary update sequence element #0 has length 1; 2 is required\n\nDuring
handling of the above exception, another exception occurred:\n\nTraceback (most
recent call last):\n File "/nfvbench/nfvbench/nfvbench.py", line 611, in
main\nconfig = config_loads(opts.config, config, whitelist_keys)\n File
"/nfvbench/nfvbench/config.py", line 55, in config_loads\nraise
Exception(e)\nException: dictionary update sequence element #0 has length 1; 2
is required\n'}
dictionary update sequence element #0 has length 1; 2 is required
The /tmp/nfvbench/nfvbench.cfg is cut and paste (OpenStack is not used and only
PCIe addresses are modified)
traffic_generator:
generator_profile:
- name: trex-local
tool: TRex
ip: 127.0.0.1
cores: 3
software_mode: false
interfaces:
- port: 0
pci: "04:00.0"
- port: 1
pci: "04:00.1"
intf_speed:
Any help is appreciated.
Regards,
Charlie Li
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#24352):
https://lists.opnfv.org/g/opnfv-tech-discuss/message/24352
Mute This Topic: https://lists.opnfv.org/mt/76538156/21656
Mute #nfvbench:
https://lists.opnfv.org/g/OPNFV+opnfv-tech-discuss/mutehashtag/nfvbench
Group Owner: opnfv-tech-discuss+ow...@lists.opnfv.org
Unsubscribe: https://lists.opnfv.org/g/opnfv-tech-discuss/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
