I used the instance uuid to find the request id in the nova-conductor logs and then looked for that request id in the nova-scheduler logs and found this:
2017-05-25 21:12:36.102 2562 WARNING nova.scheduler.client.report [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] Placement service credentials do not work. Placement is optional in Newton, but required in Ocata. Please enable the placement service before upgrading. 2017-05-25 21:12:36.104 2562 INFO nova.scheduler.host_manager [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] No hosts matched due to not matching 'force_hosts' value of 'controlNode' 2017-05-25 21:12:36.104 2562 DEBUG nova.scheduler.filter_scheduler [req-8359c011-587a-4c60-95a1-ded9e50b26af 15d62a6cd8db41f48d9a85d9aaff958f f85c91e401304cb8a0b78654352d13f6 - - -] There are 0 hosts available but 1 instances requested to build. select_destinations /usr/lib/python2.7/dist-packages/nova/scheduler/filter_scheduler.py:75 You don't have placement running or configured correctly for nova-scheduler to ask placement for hosts. So this is an invalid bug as you have an incomplete setup. See: https://docs.openstack.org/ocata/install-guide-ubuntu/nova-controller- install.html https://docs.openstack.org/developer/nova/placement.html ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1693481 Title: VM launch is failing with message "No valid host was found. There are not enough hosts available.", "code": 500 Status in OpenStack Compute (nova): Invalid Bug description: Description =========== I have installed multinode node setup using (sudo add-apt-repository ppa:openstack-ubuntu-testing/pike) where controller & compute on one node and neutron on other node When i tried to launch VM using cirros image with flavour m1.tiny gettig message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 915, in schedule_and_build_instances | | | request_specs[0].to_legacy_filter_properties_dict()) Steps to reproduce ================== 1. Install the multinode setup on virutalbox using the procedure mentioned in https://docs.openstack.org/newton/install-guide-ubuntu/keystone- verify.html VM1: ControlNode & Computenode [ RAM- 8GB, Hardisck - 100GB] VM2: NeutronS1 - neutron server [RAM - 8GB, Hardisck - 10GB] Note: Used below repository sudo add-apt-repository ppa:openstack-ubuntu-testing/pike 2.create image cirros and flavor m1.tiny #openstack image create "cirros" \ --file cirros-0.3.4-x86_64-disk.img \ --disk-format qcow2 --container-format bare \ --public # openstack flavor create --public m1.tiny --id auto --ram 512 --disk 1 --vcpus 1 --rxtx-factor 1 3.Once all the nodes are up, create network, subnet and launch VM openstack network create net3 openstack subnet create subnet3 --network net3 --subnet-range 30.30.30.0/24 nova boot --nic net-id=5d51d10b-9776-4617-8963-018639ed78aa --image cirros --flavor m1.tiny VM5 --availability-zone nova:controlNode Expected result =============== Creation of VMs should be successful and able to communicate between two VMs Actual result ============= VM launch is getting failed always with message "No valid host was found. There are not enough hosts available." root@controlNode:~# nova show 8ff61689-11cd-46d6-a88a-fb26a15d7b83 /usr/lib/python2.7/dist-packages/novaclient/client.py:278: UserWarning: The 'tenant_id' argument is deprecated in Ocata and its use may result in errors in future releases. As 'project_id' is provided, the 'tenant_id' argument will be ignored. warnings.warn(msg) +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | - | | OS-EXT-SRV-ATTR:hostname | vm5 | | OS-EXT-SRV-ATTR:hypervisor_hostname | - | | OS-EXT-SRV-ATTR:instance_name | instance-0000000c | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-iijln3t0 | | OS-EXT-SRV-ATTR:root_device_name | - | | OS-EXT-SRV-ATTR:user_data | - | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | - | | OS-EXT-STS:vm_state | error | | OS-SRV-USG:launched_at | - | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | config_drive | | | created | 2017-05-25T15:42:35Z | | description | - | | fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 915, in schedule_and_build_instances | | | request_specs[0].to_legacy_filter_properties_dict()) | | | File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 637, in _schedule_instances | | | hosts = self.scheduler_client.select_destinations(context, spec_obj) | | | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/utils.py\", line 371, in wrapped | | | return func(*args, **kwargs) | | | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py\", line 51, in select_destinations | | | return self.queryclient.select_destinations(context, spec_obj) | | | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/__init__.py\", line 37, in __run_method | | | return getattr(self.instance, __name)(*args, **kwargs) | | | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/client/query.py\", line 32, in select_destinations | | | return self.scheduler_rpcapi.select_destinations(context, spec_obj) | | | File \"/usr/lib/python2.7/dist-packages/nova/scheduler/rpcapi.py\", line 130, in select_destinations | | | return cctxt.call(ctxt, 'select_destinations', **msg_args) | | | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py\", line 169, in call | | | retry=self.retry) | | | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/transport.py\", line 98, in _send | | | timeout=timeout, retry=retry) | | | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 505, in send | | | retry=retry) | | | File \"/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py\", line 496, in _send | | | raise result | | | ", "created": "2017-05-25T15:42:36Z"} | | flavor | m1.tiny (e5bf6e30-0d88-49d0-be26-2fc0418bb7c9) | | hostId | | | host_status | | | id | 8ff61689-11cd-46d6-a88a-fb26a15d7b83 | | image | cirros (302e6c08-e0b0-43df-a6e8-4abad517d01f) | | key_name | - | | locked | False | | metadata | {} | | name | VM5 | | os-extended-volumes:volumes_attached | [] | | status | ERROR | | tags | [] | | tenant_id | f85c91e401304cb8a0b78654352d13f6 | | updated | 2017-05-25T15:42:36Z | | user_id | 15d62a6cd8db41f48d9a85d9aaff958f | +--------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Environment =========== 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ root@controlNode:~# dpkg -l |grep nova ii nova-api 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - API frontend ii nova-common 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - common files ii nova-compute 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node base ii nova-compute-kvm 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node libvirt support ii nova-conductor 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - conductor service ii nova-consoleauth 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - Console Authenticator ii nova-novncproxy 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - NoVNC proxy ii nova-placement-api 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - placement API frontend ii nova-scheduler 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - virtual machine scheduler ii python-nova 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute Python libraries ii python-novaclient 2:7.1.0-0ubuntu1~cloud0 all client library for OpenStack Compute API - Python 2.7 2. Which hypervisor did you use? root@controlNode:~# dpkg -l |grep libvirt ii libvirt-bin 2.5.0-3ubuntu7~cloud0 amd64 programs for the libvirt library ii libvirt-clients 2.5.0-3ubuntu7~cloud0 amd64 Programs for the libvirt library ii libvirt-daemon 2.5.0-3ubuntu7~cloud0 amd64 Virtualization daemon ii libvirt-daemon-system 2.5.0-3ubuntu7~cloud0 amd64 Libvirt daemon configuration files ii libvirt0:amd64 2.5.0-3ubuntu7~cloud0 amd64 library for interfacing with different virtualization systems ii nova-compute-libvirt 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node libvirt support ii python-libvirt 3.0.0-2build1~cloud0 amd64 libvirt Python bindings root@controlNode:~# root@controlNode:~# root@controlNode:~# dpkg -l |grep qemu ii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1 all PXE boot firmware - ROM images for qemu ii qemu-block-extra:amd64 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 extra block backend modules for qemu-system and qemu-utils ii qemu-kvm 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU Full virtualization ii qemu-system-common 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU full system emulation binaries (common files) ii qemu-system-x86 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU full system emulation binaries (x86) ii qemu-utils 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU utilities root@controlNode:~# root@controlNode:~# root@controlNode:~# dpkg -l |grep kvm ii nova-compute-kvm 2:16.0.0.0a1.201704201428.xenial-0ubuntu1 all OpenStack Compute - compute node (KVM) ii qemu-kvm 1:2.8+dfsg-3ubuntu2.1~cloud0 amd64 QEMU Full virtualization 2. Which storage type did you use? LVM root@controlNode:~# dpkg -l |grep lvm ii liblvm2app2.2:amd64 2.02.133-1ubuntu10 amd64 LVM2 application library ii liblvm2cmd2.02:amd64 2.02.133-1ubuntu10 amd64 LVM2 command library ii lvm2 2.02.133-1ubuntu10 amd64 Linux Logical Volume Manager 3. Which networking type did you use? Neutron with openvswitch root@controlNode:~# sudo ovs-vsctl show 8afa9151-19dd-4a76-b77c-3b2e87fc3c3d Manager "tcp:192.168.56.102:6640" is_connected: true Bridge br-int Controller "tcp:192.168.56.102:6653" is_connected: true fail_mode: secure Port br-int Interface br-int type: internal ovs_version: "2.7.0" Logs & Configs ============== Executed following command on compute node (enabled DEBUG mode) to collect the reports and same is attached $ sudo sosreport -o openstack_nova --batch File name: sosreport-controlNode-20170526000241.tar.xz To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1693481/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
