Public bug reported: charm bundle: openstack-2023.2 nova-compute: 28.0.1 channel 2023.2/stable ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors). ubuntu@juju:~/juju/controller$ openstack hypervisor list +----+---------------------+-----------------+--------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+--------------+-------+ | 7 | op1.maas | QEMU | xxxxxxxxxxxx | up | | 8 | op4.maas | QEMU | xxxxxxxxxxxx | up | +----+---------------------+-----------------+--------------+-------+ 1)microcode is the same: root@op1:~# virsh capabilities |grep microcode <microcode version='134222446'/> root@op4:~# virsh capabilities |grep microcode <microcode version='134222446'/> 2) libvirt capabilities features are identical: root@op1:~# virsh capabilities |grep feature <feature name='ht'/> <feature name='monitor'/> <feature name='osxsave'/> <feature name='xsaves'/> <feature name='cmp_legacy'/> <feature name='extapic'/> <feature name='skinit'/> <feature name='wdt'/> <feature name='tce'/> <feature name='topoext'/> <feature name='perfctr_core'/> <feature name='perfctr_nb'/> <feature name='invtsc'/> <feature name='clzero'/> <feature name='xsaveerptr'/> <feature name='npt'/> <feature name='lbrv'/> <feature name='svm-lock'/> <feature name='nrip-save'/> <feature name='tsc-scale'/> <feature name='vmcb-clean'/> <feature name='flushbyasid'/> <feature name='decodeassists'/> <feature name='pause-filter'/> <feature name='pfthreshold'/> <migration_features> </migration_features> <features> </features> <features> </features> root@op4:~# virsh capabilities |grep feature <feature name='ht'/> <feature name='monitor'/> <feature name='osxsave'/> <feature name='xsaves'/> <feature name='cmp_legacy'/> <feature name='extapic'/> <feature name='skinit'/> <feature name='wdt'/> <feature name='tce'/> <feature name='topoext'/> <feature name='perfctr_core'/> <feature name='perfctr_nb'/> <feature name='invtsc'/> <feature name='clzero'/> <feature name='xsaveerptr'/> <feature name='npt'/> <feature name='lbrv'/> <feature name='svm-lock'/> <feature name='nrip-save'/> <feature name='tsc-scale'/> <feature name='vmcb-clean'/> <feature name='flushbyasid'/> <feature name='decodeassists'/> <feature name='pause-filter'/> <feature name='pfthreshold'/> <migration_features> </migration_features> <features> </features> <features> </features> 3) NUMA idem root@op1:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 root@op4:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 ) live-migration openstack server list --all-project +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini | #openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398 No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2) log from nova-cloud-controller (debug mode) here: https://paste.ubuntu.com/p/kF3kmcXtQH/ where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. ** Affects: nova Importance: Undecided Status: New ** Description changed: charm bundle: openstack-2023.2 nova-compute: 28.0.1 channel 2023.2/stable ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors). ubuntu@juju:~/juju/controller$ openstack hypervisor list +----+---------------------+-----------------+--------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+--------------+-------+ | 7 | op1.maas | QEMU | xxxxxxxxxxxx | up | | 8 | op4.maas | QEMU | xxxxxxxxxxxx | up | +----+---------------------+-----------------+--------------+-------+ - 1)microcode is the same: root@op1:~# virsh capabilities |grep microcode - <microcode version='134222446'/> + <microcode version='134222446'/> root@op4:~# virsh capabilities |grep microcode - <microcode version='134222446'/> + <microcode version='134222446'/> 2) libvirt capabilities features are identical: root@op1:~# virsh capabilities |grep feature - <feature name='ht'/> - <feature name='monitor'/> - <feature name='osxsave'/> - <feature name='xsaves'/> - <feature name='cmp_legacy'/> - <feature name='extapic'/> - <feature name='skinit'/> - <feature name='wdt'/> - <feature name='tce'/> - <feature name='topoext'/> - <feature name='perfctr_core'/> - <feature name='perfctr_nb'/> - <feature name='invtsc'/> - <feature name='clzero'/> - <feature name='xsaveerptr'/> - <feature name='npt'/> - <feature name='lbrv'/> - <feature name='svm-lock'/> - <feature name='nrip-save'/> - <feature name='tsc-scale'/> - <feature name='vmcb-clean'/> - <feature name='flushbyasid'/> - <feature name='decodeassists'/> - <feature name='pause-filter'/> - <feature name='pfthreshold'/> - <migration_features> - </migration_features> - <features> - </features> - <features> - </features> + <feature name='ht'/> + <feature name='monitor'/> + <feature name='osxsave'/> + <feature name='xsaves'/> + <feature name='cmp_legacy'/> + <feature name='extapic'/> + <feature name='skinit'/> + <feature name='wdt'/> + <feature name='tce'/> + <feature name='topoext'/> + <feature name='perfctr_core'/> + <feature name='perfctr_nb'/> + <feature name='invtsc'/> + <feature name='clzero'/> + <feature name='xsaveerptr'/> + <feature name='npt'/> + <feature name='lbrv'/> + <feature name='svm-lock'/> + <feature name='nrip-save'/> + <feature name='tsc-scale'/> + <feature name='vmcb-clean'/> + <feature name='flushbyasid'/> + <feature name='decodeassists'/> + <feature name='pause-filter'/> + <feature name='pfthreshold'/> + <migration_features> + </migration_features> + <features> + </features> + <features> + </features> root@op4:~# virsh capabilities |grep feature - <feature name='ht'/> - <feature name='monitor'/> - <feature name='osxsave'/> - <feature name='xsaves'/> - <feature name='cmp_legacy'/> - <feature name='extapic'/> - <feature name='skinit'/> - <feature name='wdt'/> - <feature name='tce'/> - <feature name='topoext'/> - <feature name='perfctr_core'/> - <feature name='perfctr_nb'/> - <feature name='invtsc'/> - <feature name='clzero'/> - <feature name='xsaveerptr'/> - <feature name='npt'/> - <feature name='lbrv'/> - <feature name='svm-lock'/> - <feature name='nrip-save'/> - <feature name='tsc-scale'/> - <feature name='vmcb-clean'/> - <feature name='flushbyasid'/> - <feature name='decodeassists'/> - <feature name='pause-filter'/> - <feature name='pfthreshold'/> - <migration_features> - </migration_features> - <features> - </features> - <features> - </features> + <feature name='ht'/> + <feature name='monitor'/> + <feature name='osxsave'/> + <feature name='xsaves'/> + <feature name='cmp_legacy'/> + <feature name='extapic'/> + <feature name='skinit'/> + <feature name='wdt'/> + <feature name='tce'/> + <feature name='topoext'/> + <feature name='perfctr_core'/> + <feature name='perfctr_nb'/> + <feature name='invtsc'/> + <feature name='clzero'/> + <feature name='xsaveerptr'/> + <feature name='npt'/> + <feature name='lbrv'/> + <feature name='svm-lock'/> + <feature name='nrip-save'/> + <feature name='tsc-scale'/> + <feature name='vmcb-clean'/> + <feature name='flushbyasid'/> + <feature name='decodeassists'/> + <feature name='pause-filter'/> + <feature name='pfthreshold'/> + <migration_features> + </migration_features> + <features> + </features> + <features> + </features> 3) NUMA idem root@op1:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 root@op4:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 - ) live-migration openstack server list --all-project +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini | - - #openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398 + #openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398 + No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2) log from nova-cloud-controller (debug mode) here: https://paste.ubuntu.com/p/kF3kmcXtQH/ where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2061569 Title: Live migration fails despite identical CPUs with Host filter ignoring hosts Status in OpenStack Compute (nova): New Bug description: charm bundle: openstack-2023.2 nova-compute: 28.0.1 channel 2023.2/stable ubuntu: jammy .15.0-102-generic #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux I have two identical host, op1 and op4, DELL PowerEdge R7425 acting as nova-compute nodes (hypervisors). ubuntu@juju:~/juju/controller$ openstack hypervisor list +----+---------------------+-----------------+--------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+--------------+-------+ | 7 | op1.maas | QEMU | xxxxxxxxxxxx | up | | 8 | op4.maas | QEMU | xxxxxxxxxxxx | up | +----+---------------------+-----------------+--------------+-------+ 1)microcode is the same: root@op1:~# virsh capabilities |grep microcode <microcode version='134222446'/> root@op4:~# virsh capabilities |grep microcode <microcode version='134222446'/> 2) libvirt capabilities features are identical: root@op1:~# virsh capabilities |grep feature <feature name='ht'/> <feature name='monitor'/> <feature name='osxsave'/> <feature name='xsaves'/> <feature name='cmp_legacy'/> <feature name='extapic'/> <feature name='skinit'/> <feature name='wdt'/> <feature name='tce'/> <feature name='topoext'/> <feature name='perfctr_core'/> <feature name='perfctr_nb'/> <feature name='invtsc'/> <feature name='clzero'/> <feature name='xsaveerptr'/> <feature name='npt'/> <feature name='lbrv'/> <feature name='svm-lock'/> <feature name='nrip-save'/> <feature name='tsc-scale'/> <feature name='vmcb-clean'/> <feature name='flushbyasid'/> <feature name='decodeassists'/> <feature name='pause-filter'/> <feature name='pfthreshold'/> <migration_features> </migration_features> <features> </features> <features> </features> root@op4:~# virsh capabilities |grep feature <feature name='ht'/> <feature name='monitor'/> <feature name='osxsave'/> <feature name='xsaves'/> <feature name='cmp_legacy'/> <feature name='extapic'/> <feature name='skinit'/> <feature name='wdt'/> <feature name='tce'/> <feature name='topoext'/> <feature name='perfctr_core'/> <feature name='perfctr_nb'/> <feature name='invtsc'/> <feature name='clzero'/> <feature name='xsaveerptr'/> <feature name='npt'/> <feature name='lbrv'/> <feature name='svm-lock'/> <feature name='nrip-save'/> <feature name='tsc-scale'/> <feature name='vmcb-clean'/> <feature name='flushbyasid'/> <feature name='decodeassists'/> <feature name='pause-filter'/> <feature name='pfthreshold'/> <migration_features> </migration_features> <features> </features> <features> </features> 3) NUMA idem root@op1:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 root@op4:~# lscpu | grep -i numa NUMA node(s): 8 NUMA node0 CPU(s): 0,8,16,24,32,40,48,56 NUMA node1 CPU(s): 2,10,18,26,34,42,50,58 NUMA node2 CPU(s): 4,12,20,28,36,44,52,60 NUMA node3 CPU(s): 6,14,22,30,38,46,54,62 NUMA node4 CPU(s): 1,9,17,25,33,41,49,57 NUMA node5 CPU(s): 3,11,19,27,35,43,51,59 NUMA node6 CPU(s): 5,13,21,29,37,45,53,61 NUMA node7 CPU(s): 7,15,23,31,39,47,55,63 ) live-migration openstack server list --all-project +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+------+--------+-------------------+--------------------------+---------+ | 6b75b329-bda5-4348-a25e-ec1b9102b398 | u1 | ACTIVE | hub=xxxxxxxxxxxxx | N/A (booted from volume) | f1.mini | #openstack server migrate --live-migration 6b75b329-bda5-4348-a25e-ec1b9102b398 No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-a3489211-cb92-4744-a3b1-1df96bbcb3f2) log from nova-cloud-controller (debug mode) here: https://paste.ubuntu.com/p/kF3kmcXtQH/ where op4.maas is the source hypervisor and op1.maas is the destination one: they have the same hardware, RAM and ceph storage. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2061569/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
