I am seeing below error when I try to spawn a instance on VPP with DPDK and openstack
openstack server create --flavor CSR-small --image csr-1000v-17-03-05 --nic net-id=suresh-nw1 CSR1k-VPP-suresh-nw1 Oct 13 23:04:02 gna200405 nova-compute[2579]: DEBUG nova.virt.libvirt.vif [None req-4ad4c67d-06ea-4c9c-970e-1c27afb5616e demo demo] vif_type=vhostuser instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,config_drive='',created_at=2022-10-13T22:58:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='CSR1k-VPP-suresh-nw1',display_name='CSR1k-VPP-suresh-nw1',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(11),hidden=False,host='gna200405',hostname='csr1k-vpp-suresh-nw1',id=21,image_ref='3a9571aa-ce4e-4161-a5f8-471bac4ae6a0',info_cache=InstanceInfoCache,instance_type_id=11,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='gna200405',locked=False,locked_by=None,memory_mb=4096,metadata={},migration_context=None,new_flavor=None,node='gna200405',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7195336fe4b449ebb3728460df45f326',ramdisk_id='',reservation_id='r-y3vxxleo',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,anotherrole,member',clean_attempts='1',image_base_image_ref='3a9571aa-ce4e-4161-a5f8-471bac4ae6a0',image_container_format='bare',image_disk_format='qcow2',image_hw_vif_multiqueue_enabled='True',image_min_disk='20',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/csr-1000v-17-03-05',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='demo',owner_user_name='demo'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2022-10-13T22:58:58Z,user_data=None,user_id='8bebe070d9c9482a9a23d1e968eaad61',uuid=b5c83897-19b9-470c-9781-91ca9e067976,vcpu_model=VirtCPUModel,vcpus=2,vm_mode=None,vm_state='building') vif={"id": "99383ca9-69bc-423c-be9f-d6ba6ba2b644", "address": "fa:16:3e:6d:2f:6f", "network": {"id": "74a60fc6-fd8d-414a-99b8-3974cea73094", "bridge": null, "label": "suresh-nw1", "subnets": [{"cidr": "192.168.1.0/24", "dns": [], "gateway": {"address": "192.168.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.1.42", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.1.2"}}], "meta": {"injected": false, "tenant_id": "7195336fe4b449ebb3728460df45f326", "mtu": 1500, "physical_network": "physnet", "tunneled": false}}, "type": "vhostuser", "details": {"vhostuser_socket": "/tmp/99383ca9-69bc-423c-be9f-d6ba6ba2b644", "vhostuser_mode": "server"}, "devname": "tap99383ca9-69", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "meta": {}} {{(pid=2579) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:810}} Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [None req-4ad4c67d-06ea-4c9c-970e-1c27afb5616e demo demo] [instance: b5c83897-19b9-470c-9781-91ca9e067976] *Failed to allocate network(s): nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed* Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] Traceback (most recent call last): Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 6748, in _create_guest_with_network Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] guest = self._create_guest( Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] next(self.gen) Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/opt/stack/nova/nova/compute/manager.py", line 484, in wait_for_instance_event Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] actual_event = event.wait() Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/usr/local/lib/python3.8/dist-packages/eventlet/event.py", line 125, in wait Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] result = hub.switch() Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 298, in switch Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] return self.greenlet.switch() Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] eventlet.timeout.Timeout: 300 seconds Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] During handling of the above exception, another exception occurred: Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] Traceback (most recent call last): Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/opt/stack/nova/nova/compute/manager.py", line 2442, in _build_and_run_instance Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] self.driver.spawn(context, instance, image_meta, Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3787, in spawn Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] self._create_guest_with_network( Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 6774, in _create_guest_with_network Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] raise exception.VirtualInterfaceCreateException() Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] nova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed Oct 13 23:04:03 gna200405 nova-compute[2579]: ERROR nova.compute.manager [instance: b5c83897-19b9-470c-9781-91ca9e067976] stack@gna200405:~/devstack$ sudo vppctl show dpdk phy Segment 4-0: IOVA:0x7f683fe00000, len:2097152, virt:0x7f683fe00000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:365 Segment 4-1: IOVA:0x7f6840000000, len:2097152, virt:0x7f6840000000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:370 Segment 4-2: IOVA:0x7f6840200000, len:2097152, virt:0x7f6840200000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:371 Segment 4-3: IOVA:0x7f6840400000, len:2097152, virt:0x7f6840400000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:376 Segment 4-4: IOVA:0x7f6840600000, len:2097152, virt:0x7f6840600000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:377 Segment 4-5: IOVA:0x7f6840800000, len:2097152, virt:0x7f6840800000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:382 Segment 4-6: IOVA:0x7f6840a00000, len:2097152, virt:0x7f6840a00000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:383 Segment 4-7: IOVA:0x7f6840c00000, len:2097152, virt:0x7f6840c00000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:388 Segment 4-8: IOVA:0x7f6840e00000, len:2097152, virt:0x7f6840e00000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:389 Segment 4-9: IOVA:0x7f6841000000, len:2097152, virt:0x7f6841000000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:390 Segment 4-10: IOVA:0x7f6841200000, len:2097152, virt:0x7f6841200000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:393 Segment 4-11: IOVA:0x7f6841400000, len:2097152, virt:0x7f6841400000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:394 Segment 4-12: IOVA:0x7f6841600000, len:2097152, virt:0x7f6841600000, socket_id:0, hugepage_sz:2097152, nchannel:0, nrank:0 fd:395 Segment 8-0: IOVA:0x7f583f600000, len:2097152, virt:0x7f583f600000, socket_id:1, hugepage_sz:2097152, nchannel:0, nrank:0 fd:391 stack@gna200405:~/devstack$ cat /etc/vpp/startup.conf unix { nodaemon log /tmp/vpp.log full-coredump startup-config /etc/vpp-startup.conf cli-listen /run/vpp/cli.sock gid vpp } api-trace { on } api-segment { gid vpp } socksvr { default } dpdk { dev default { num-rx-queues 8 num-tx-queues 8 } uio-driver vfio-pci dev 0000:02:00.0 dev 0000:02:00.1 dev 0000:06:00.0 dev 0000:06:00.1 # socket-mem 128 } memory { main-heap-size 512M main-heap-page-size default-hugepage } statseg { size 1G page-size default-hugepage per-node-counters off } stack@gna200405:~/devstack$ grep Huge /proc/meminfo AnonHugePages: 0 kB ShmemHugePages: 0 kB FileHugePages: 0 kB HugePages_Total: 1763 HugePages_Free: 941 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 3610624 kB
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22024): https://lists.fd.io/g/vpp-dev/message/22024 Mute This Topic: https://lists.fd.io/mt/94317064/21656 Mute #vpp-dev:https://lists.fd.io/g/vpp-dev/mutehashtag/vpp-dev Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-