Public bug reported: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException.
Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() +migration.dest_compute = migration['source_compute'] +migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Affects: nova Importance: Undecided Status: New ** Description changed: Short version: - Do rebuild instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. - + Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. - Solution: nova/compute/manager.py - def finish_revert_resize(self, context, instance, reservations, migration): - instance.system_metadata = sys_meta - instance.memory_mb = instance_type['memory_mb'] - instance.vcpus = instance_type['vcpus'] - instance.root_gb = instance_type['root_gb'] - instance.ephemeral_gb = instance_type['ephemeral_gb'] - instance.instance_type_id = instance_type['id'] - instance.host = migration['source_compute'] - instance.node = migration['source_node'] - instance.save() + def finish_revert_resize(self, context, instance, reservations, migration): + instance.system_metadata = sys_meta + instance.memory_mb = instance_type['memory_mb'] + instance.vcpus = instance_type['vcpus'] + instance.root_gb = instance_type['root_gb'] + instance.ephemeral_gb = instance_type['ephemeral_gb'] + instance.instance_type_id = instance_type['id'] + instance.host = migration['source_compute'] + instance.node = migration['source_node'] + instance.save() + migration.dest_compute = migration['source_compute'] + migration.save() + - self.network_api.setup_networks_on_host(context, instance, - migration['source_compute']) + self.network_api.setup_networks_on_host(context, instance, + migration['source_compute']) - block_device_info = self._get_instance_block_device_info( - context, instance, refresh_conn_info=True) + block_device_info = self._get_instance_block_device_info( + context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() - + + + migration.dest_compute = migration['source_compute'] + + migration.save() + + self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() - + + + migration.dest_compute = migration['source_compute'] + + migration.save() + + self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py + def finish_revert_resize(self, context, instance, reservations, migration): + instance.system_metadata = sys_meta + instance.memory_mb = instance_type['memory_mb'] + instance.vcpus = instance_type['vcpus'] + instance.root_gb = instance_type['root_gb'] + instance.ephemeral_gb = instance_type['ephemeral_gb'] + instance.instance_type_id = instance_type['id'] + instance.host = migration['source_compute'] + instance.node = migration['source_node'] + instance.save() - def finish_revert_resize(self, context, instance, reservations, migration): - instance.system_metadata = sys_meta - instance.memory_mb = instance_type['memory_mb'] - instance.vcpus = instance_type['vcpus'] - instance.root_gb = instance_type['root_gb'] - instance.ephemeral_gb = instance_type['ephemeral_gb'] - instance.instance_type_id = instance_type['id'] - instance.host = migration['source_compute'] - instance.node = migration['source_node'] - instance.save() + + migration.dest_compute = migration['source_compute'] + + migration.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() - + - self.network_api.setup_networks_on_host(context, instance, - migration['source_compute']) + self.network_api.setup_networks_on_host(context, instance, + migration['source_compute']) - block_device_info = self._get_instance_block_device_info( - context, instance, refresh_conn_info=True) + block_device_info = self._get_instance_block_device_info( + context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): - instance.system_metadata = sys_meta - instance.memory_mb = instance_type['memory_mb'] - instance.vcpus = instance_type['vcpus'] - instance.root_gb = instance_type['root_gb'] - instance.ephemeral_gb = instance_type['ephemeral_gb'] - instance.instance_type_id = instance_type['id'] - instance.host = migration['source_compute'] - instance.node = migration['source_node'] - instance.save() + instance.system_metadata = sys_meta + instance.memory_mb = instance_type['memory_mb'] + instance.vcpus = instance_type['vcpus'] + instance.root_gb = instance_type['root_gb'] + instance.ephemeral_gb = instance_type['ephemeral_gb'] + instance.instance_type_id = instance_type['id'] + instance.host = migration['source_compute'] + instance.node = migration['source_node'] + instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() + + migration.dest_compute = migration['source_compute'] + + migration.save() - self.network_api.setup_networks_on_host(context, instance, - migration['source_compute']) + self.network_api.setup_networks_on_host(context, instance, + migration['source_compute']) - block_device_info = self._get_instance_block_device_info( - context, instance, refresh_conn_info=True) + block_device_info = self._get_instance_block_device_info( + context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() + + migration.dest_compute = migration['source_compute'] + + migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() + + migration.dest_compute = migration['source_compute'] + + migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] + + migration.dest_compute = migration['source_compute'] + migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() - + migration.dest_compute = migration['source_compute'] - + migration.save() + migration.dest_compute = migration['source_compute'] + migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) ** Description changed: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): - instance.system_metadata = sys_meta - instance.memory_mb = instance_type['memory_mb'] - instance.vcpus = instance_type['vcpus'] - instance.root_gb = instance_type['root_gb'] - instance.ephemeral_gb = instance_type['ephemeral_gb'] - instance.instance_type_id = instance_type['id'] - instance.host = migration['source_compute'] - instance.node = migration['source_node'] - instance.save() + instance.system_metadata = sys_meta + instance.memory_mb = instance_type['memory_mb'] + instance.vcpus = instance_type['vcpus'] + instance.root_gb = instance_type['root_gb'] + instance.ephemeral_gb = instance_type['ephemeral_gb'] + instance.instance_type_id = instance_type['id'] + instance.host = migration['source_compute'] + instance.node = migration['source_node'] + instance.save() - migration.dest_compute = migration['source_compute'] - migration.save() + +migration.dest_compute = migration['source_compute'] + +migration.save() - self.network_api.setup_networks_on_host(context, instance, - migration['source_compute']) + self.network_api.setup_networks_on_host(context, instance, + migration['source_compute']) - block_device_info = self._get_instance_block_device_info( - context, instance, refresh_conn_info=True) + block_device_info = self._get_instance_block_device_info( + context, instance, refresh_conn_info=True) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1380934 Title: failure rebuilding instance after resize-revert Status in OpenStack Compute (Nova): New Bug description: Short version: Rebuilding instance after resize, vm become ERROR cause of VirtualInterfaceCreateException. Long version: Reproduce: master(0bbf6301b1886f501713df8c13a5f8b92c9b0c2c) 1. create instanse 2. do resize(succcess and active in destination host/status is VERIFY_RESIZE) 3. do resize-revert(success and active in source host/status is ACTIVE/port is DOWN) 4. do rebuild After timeout(value of vif_plugging_timeout), vm became ERROR cause of VirtualInterfaceCreateException. If you set vif_plugging_is_fatal to True, vm is removed. Otherwise, if you set vif_plugging_is_fatal to False, vm become ACTIVE. Cause: When I do resize-revert, instance come back to source host, but host information is set to destination node. (host information is 'host' of ml2_port_bindings table of neutron DB) This lead to mismatch of actual neutron port and DB information of port. So this port become to DOWN. Do rebuild in this situation, port don't become to ACTIVE, so results in timuout. Solution: nova/compute/manager.py def finish_revert_resize(self, context, instance, reservations, migration): instance.system_metadata = sys_meta instance.memory_mb = instance_type['memory_mb'] instance.vcpus = instance_type['vcpus'] instance.root_gb = instance_type['root_gb'] instance.ephemeral_gb = instance_type['ephemeral_gb'] instance.instance_type_id = instance_type['id'] instance.host = migration['source_compute'] instance.node = migration['source_node'] instance.save() +migration.dest_compute = migration['source_compute'] +migration.save() self.network_api.setup_networks_on_host(context, instance, migration['source_compute']) block_device_info = self._get_instance_block_device_info( context, instance, refresh_conn_info=True) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1380934/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp