Public bug reported:
Description
===========
I am trying to enable live migration feature post copy using below config,
seeing post-copy is not supported error
block_migration_flag =
VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_NON_SHARED_INC
cpu_mode = custom
cpu_model_extra_flags =
-ds,-acpi,+ss,-ht,-tm,-pbe,-dtes64,-monitor,-ds_cpl,+vmx,-smx,-est,-tm2,-xtpr,+pdcm,-dca,+tsc_adjust,-intel-pt,+md-clear,+stibp,+ssbd,+pdpe1gb,-invtsc,-hle,-rtm,-mpx,-xsavec,-xgetbv1
cpu_models = Skylake-Client-IBRS
live_migration_bandwidth = 900
live_migration_downtime = 100
live_migration_flag =
VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE
live_migration_permit_post_copy = True
live_migration_timeout_action=force_complete
Steps to reproduce
==================
KVM hypervisor
Using Openstack Antelope base version
qemu-system-x86_64 --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.16)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 8.0.0
#Create general vm once config is set
#perform live migration either with block-migration or without
After Pre-migration phase, able to see migration got trigger
successfully & while copying memory , seeing postcopy is not supported
Expected result
===============
Migration should be sucessfull
Actual result
=============
compute.log
Feb 05 22:16:02 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:02.988
1156821 INFO nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Increasing downtime to 10 ms after 0 sec
elapsed time
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:03.069
1156821 INFO nova.virt.libvirt.driver [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Migration running for 0 secs, memory 100%
remaining (bytes processed=0, remaining=0, total=0); disk 100% remaining (bytes
processed=0, remaining=0, total=0).
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:03.571
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 1 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:03.572
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:04.074
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 1 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:04.075
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:04.577
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 2 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:04.577
1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05 22:16:04.638
1156821 ERROR nova.virt.libvirt.driver [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Live Migration failure: internal error:
unable to execute QEMU command 'migrate-set-capabilities': Postcopy is not
supported: libvirt.libvirtError: internal error: unable to execute QEMU command
'migrate-set-capabilities': Postcopy is not supported
libvirtd.log
-------
2024-02-05 21:27:56.254+0000: 2177204: debug :
qemuMonitorSetMigrationCapabilities:3689 : mon:0x7f8d68080460 vm:0x7f8d6804b930
fd:82
2024-02-05 21:27:56.254+0000: 2177204: info : qemuMonitorSend:914 :
QEMU_MONITOR_SEND_MSG: mon=0x7f8d68080460
msg={"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}^M
fd=-1
2024-02-05 21:27:56.254+0000: 16270: info : qemuMonitorIOWrite:402 :
QEMU_MONITOR_IO_WRITE: mon=0x7f8d68080460
buf={"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}^M
len=531 ret=531 errno=0
2024-02-05 21:27:56.258+0000: 16270: debug : qemuMonitorJSONIOProcessLine:220 :
Line [{"id": "libvirt-402", "error": {"class": "GenericError", "desc":
"Postcopy is not supported"}}]
2024-02-05 21:27:56.258+0000: 16270: info : qemuMonitorJSONIOProcessLine:239 :
QEMU_MONITOR_RECV_REPLY: mon=0x7f8d68080460 reply={"id": "libvirt-402",
"error": {"class": "GenericError", "desc": "Postcopy is not supported"}}
2024-02-05 21:27:56.258+0000: 2177204: debug :
qemuMonitorJSONCheckErrorFull:385 : unable to execute QEMU command
{"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}:
{"id":"libvirt-402","error":{"class":"GenericError","desc":"Postcopy is not
supported"}}
2024-02-05 21:27:56.258+0000: 2177204: error :
qemuMonitorJSONCheckErrorFull:397 : internal error: unable to execute QEMU
command 'migrate-set-capabilities': Postcopy is not supported
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/2052473
Title:
Live migration post-copy not working as Expected
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
I am trying to enable live migration feature post copy using below config,
seeing post-copy is not supported error
block_migration_flag =
VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_NON_SHARED_INC
cpu_mode = custom
cpu_model_extra_flags =
-ds,-acpi,+ss,-ht,-tm,-pbe,-dtes64,-monitor,-ds_cpl,+vmx,-smx,-est,-tm2,-xtpr,+pdcm,-dca,+tsc_adjust,-intel-pt,+md-clear,+stibp,+ssbd,+pdpe1gb,-invtsc,-hle,-rtm,-mpx,-xsavec,-xgetbv1
cpu_models = Skylake-Client-IBRS
live_migration_bandwidth = 900
live_migration_downtime = 100
live_migration_flag =
VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE
live_migration_permit_post_copy = True
live_migration_timeout_action=force_complete
Steps to reproduce
==================
KVM hypervisor
Using Openstack Antelope base version
qemu-system-x86_64 --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.16)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 8.0.0
#Create general vm once config is set
#perform live migration either with block-migration or without
After Pre-migration phase, able to see migration got trigger
successfully & while copying memory , seeing postcopy is not supported
Expected result
===============
Migration should be sucessfull
Actual result
=============
compute.log
Feb 05 22:16:02 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:02.988 1156821 INFO nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Increasing downtime to 10 ms after 0 sec
elapsed time
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:03.069 1156821 INFO nova.virt.libvirt.driver [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Migration running for 0 secs, memory 100%
remaining (bytes processed=0, remaining=0, total=0); disk 100% remaining (bytes
processed=0, remaining=0, total=0).
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:03.571 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 1 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:03 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:03.572 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:04.074 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 1 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:04.075 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:04.577 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Current 10 elapsed 2 steps [(0, 10),
(960, 19), (1920, 28), (2880, 37), (3840, 46), (4800, 55), (5760, 64), (6720,
73), (7680, 82), (8640, 91), (9600, 100)] update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:512
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:04.577 1156821 DEBUG nova.virt.libvirt.migration [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Downtime does not need to change
update_downtime
/openstack/venvs/nova-27.4.0/lib/python3.10/site-packages/nova/virt/libvirt/migration.py:525
Feb 05 22:16:04 cdc-appblx095-37 nova-compute[1156821]: 2024-02-05
22:16:04.638 1156821 ERROR nova.virt.libvirt.driver [None
req-f6650a9f-9465-40ca-9981-fff470751fc7 4807f132b7bb47bbabbe50de9bd974c8
b61fc56101024f498d4d95e863c7333f - - default default] [instance:
31fcf3ba-c0b1-4c74-afdd-685ba45a11f0] Live Migration failure: internal error:
unable to execute QEMU command 'migrate-set-capabilities': Postcopy is not
supported: libvirt.libvirtError: internal error: unable to execute QEMU command
'migrate-set-capabilities': Postcopy is not supported
libvirtd.log
-------
2024-02-05 21:27:56.254+0000: 2177204: debug :
qemuMonitorSetMigrationCapabilities:3689 : mon:0x7f8d68080460 vm:0x7f8d6804b930
fd:82
2024-02-05 21:27:56.254+0000: 2177204: info : qemuMonitorSend:914 :
QEMU_MONITOR_SEND_MSG: mon=0x7f8d68080460
msg={"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}^M
fd=-1
2024-02-05 21:27:56.254+0000: 16270: info : qemuMonitorIOWrite:402 :
QEMU_MONITOR_IO_WRITE: mon=0x7f8d68080460
buf={"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}^M
len=531 ret=531 errno=0
2024-02-05 21:27:56.258+0000: 16270: debug : qemuMonitorJSONIOProcessLine:220
: Line [{"id": "libvirt-402", "error": {"class": "GenericError", "desc":
"Postcopy is not supported"}}]
2024-02-05 21:27:56.258+0000: 16270: info : qemuMonitorJSONIOProcessLine:239
: QEMU_MONITOR_RECV_REPLY: mon=0x7f8d68080460 reply={"id": "libvirt-402",
"error": {"class": "GenericError", "desc": "Postcopy is not supported"}}
2024-02-05 21:27:56.258+0000: 2177204: debug :
qemuMonitorJSONCheckErrorFull:385 : unable to execute QEMU command
{"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"xbzrle","state":false},{"capability":"auto-converge","state":false},{"capability":"rdma-pin-all","state":false},{"capability":"postcopy-ram","state":true},{"capability":"compress","state":false},{"capability":"pause-before-switchover","state":false},{"capability":"late-block-activate","state":true},{"capability":"multifd","state":false},{"capability":"dirty-bitmaps","state":false},{"capability":"return-path","state":true}]},"id":"libvirt-402"}:
{"id":"libvirt-402","error":{"class":"GenericError","desc":"Postcopy is not
supported"}}
2024-02-05 21:27:56.258+0000: 2177204: error :
qemuMonitorJSONCheckErrorFull:397 : internal error: unable to execute QEMU
command 'migrate-set-capabilities': Postcopy is not supported
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/2052473/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
