Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Wed, Nov 19, 2014 at 01:01:14PM +0530, Amit Shah wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. This already requires somewhat messy compatibility support in acpi generation code, and it looks like there'll be more to come. Rather than try to guess the correct size once and for all, this patchset tries to make code future-proof, by adding support for resizeable ram blocks. A (possibly very high) amount of space in ram_addr_t space is reserved for each block, but never allocated. If incoming block size differs from current size, block is reallocated. FW CFG is also notified and updated accordingly. To simplify things, I didn't add support for resizing actual RAM: device RAM such as fw cfg ROMs are never mapped into guests directly, so instead I added an API to flag device RAM explicitly, and manage them using simple alloc/free/realloc Considering this promises to rid us of worries about ROM size considerations once and for all, I thinking about pushing this as a kind of bugfix before 2.2, so we don't need to maintain more band-aids in 2.3 and on. I'd rather wait for 2.3; we've done this for a couple of releases already, so what's one more. And we're at rc2 already.. Paolo feels the same, and I agree. Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Amit Well - guest never sees the resizing. It happens before we start the VM. So I don't see the issue - could you clarify please? -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
Michael S. Tsirkin m...@redhat.com writes: On Tue, Nov 18, 2014 at 03:47:16PM +0100, Markus Armbruster wrote: Michael S. Tsirkin m...@redhat.com writes: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. Pardon my ignorance... changing ROM in migration? I would expect migration to be as transparent for ROM as it is for RAM. What am I missing? [...] Nothing really. We migrate RAM size too - but if there's a mismatch, migration fails. RAM size is user configurable. ROM is used internally so we have to figure out the size, and it turned out to be a hard problem, so we end up maintaining logic for ROM size like we did in 1.7 like we did in 2.0 etc. I don't want to add any more: let's just accept whatever is migrated, and stick to it. Are you proposing to change ROM size on the target from whatever was configured during startup to whatever is configured on the source?
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On (Wed) 19 Nov 2014 [10:15:16], Michael S. Tsirkin wrote: On Wed, Nov 19, 2014 at 01:01:14PM +0530, Amit Shah wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Well - guest never sees the resizing. It happens before we start the VM. So I don't see the issue - could you clarify please? Before we start the VM? That's a really odd corner case (ie not worth bothering about?). I took this to mean that the guest was running while migration happened. Amit
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Wed, Nov 19, 2014 at 01:52:35PM +0530, Amit Shah wrote: On (Wed) 19 Nov 2014 [10:15:16], Michael S. Tsirkin wrote: On Wed, Nov 19, 2014 at 01:01:14PM +0530, Amit Shah wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Well - guest never sees the resizing. It happens before we start the VM. So I don't see the issue - could you clarify please? Before we start the VM? That's a really odd corner case (ie not worth bothering about?). I took this to mean that the guest was running while migration happened. Amit At the moment you get old ROM before reboot, new ROM after reboot. Anyway, let's argue about this when someone proposes this. Guys, please drop responding to patch 0. There's no code there. Please review the actual code. -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Wed, Nov 19, 2014 at 09:16:09AM +0100, Markus Armbruster wrote: Michael S. Tsirkin m...@redhat.com writes: On Tue, Nov 18, 2014 at 03:47:16PM +0100, Markus Armbruster wrote: Michael S. Tsirkin m...@redhat.com writes: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. Pardon my ignorance... changing ROM in migration? I would expect migration to be as transparent for ROM as it is for RAM. What am I missing? [...] Nothing really. We migrate RAM size too - but if there's a mismatch, migration fails. RAM size is user configurable. ROM is used internally so we have to figure out the size, and it turned out to be a hard problem, so we end up maintaining logic for ROM size like we did in 1.7 like we did in 2.0 etc. I don't want to add any more: let's just accept whatever is migrated, and stick to it. Are you proposing to change ROM size on the target from whatever was configured during startup to whatever is configured on the source? Exactly. ROMs are running within guest, they should just migrate together with VM. *They already do* except for their size. Which kind of mostly does not create problems anyway because we round size up. -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On 19 November 2014 07:31, Amit Shah amit.s...@redhat.com wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: Considering this promises to rid us of worries about ROM size considerations once and for all, I thinking about pushing this as a kind of bugfix before 2.2, so we don't need to maintain more band-aids in 2.3 and on. I'd rather wait for 2.3; we've done this for a couple of releases already, so what's one more. And we're at rc2 already.. It certainly seems pretty risky to introduce this change with only two weeks to go til release; I wouldn't want to merge it without a strong consensus from everybody involved that it really needed to go in for 2.2. thanks -- PMM
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
Michael S. Tsirkin m...@redhat.com wrote: On Wed, Nov 19, 2014 at 01:52:35PM +0530, Amit Shah wrote: On (Wed) 19 Nov 2014 [10:15:16], Michael S. Tsirkin wrote: On Wed, Nov 19, 2014 at 01:01:14PM +0530, Amit Shah wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Well - guest never sees the resizing. It happens before we start the VM. So I don't see the issue - could you clarify please? Before we start the VM? That's a really odd corner case (ie not worth bothering about?). I took this to mean that the guest was running while migration happened. Amit At the moment you get old ROM before reboot, new ROM after reboot. Anyway, let's argue about this when someone proposes this. Guys, please drop responding to patch 0. There's no code there. Please review the actual code. How can we answer: The code does what it tell it does, we are not happy with the whole idea? Later, Juan.
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On 19/11/2014 14:52, Peter Maydell wrote: It certainly seems pretty risky to introduce this change with only two weeks to go til release; I wouldn't want to merge it without a strong consensus from everybody involved that it really needed to go in for 2.2. I think there's consensus (you, Amit, me) that it should wait for 2.3. Paolo
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Wed, Nov 19, 2014 at 03:41:23PM +0100, Paolo Bonzini wrote: On 19/11/2014 14:52, Peter Maydell wrote: It certainly seems pretty risky to introduce this change with only two weeks to go til release; I wouldn't want to merge it without a strong consensus from everybody involved that it really needed to go in for 2.2. I think there's consensus (you, Amit, me) that it should wait for 2.3. Paolo I agree too. We can still iterate on it, and I'd like to merge early in 2.3 cycle. So please send comments.
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Wed, Nov 19, 2014 at 02:52:43PM +0100, Juan Quintela wrote: Michael S. Tsirkin m...@redhat.com wrote: On Wed, Nov 19, 2014 at 01:52:35PM +0530, Amit Shah wrote: On (Wed) 19 Nov 2014 [10:15:16], Michael S. Tsirkin wrote: On Wed, Nov 19, 2014 at 01:01:14PM +0530, Amit Shah wrote: On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Well - guest never sees the resizing. It happens before we start the VM. So I don't see the issue - could you clarify please? Before we start the VM? That's a really odd corner case (ie not worth bothering about?). I took this to mean that the guest was running while migration happened. Amit At the moment you get old ROM before reboot, new ROM after reboot. Anyway, let's argue about this when someone proposes this. Guys, please drop responding to patch 0. There's no code there. Please review the actual code. How can we answer: The code does what it tell it does, we are not happy with the whole idea? Later, Juan. My point is you seem to be unhappy with the general idea that ROMs are migrated, because you think all ROMs should be bitwise identical to what we shipped years ago, depending on machine type. But that ship sailed in 1.7 I think. If you want to revert that decision, start a new thread, no need to respond to this patch. All this patch does is remove the artificial same size limitation, which - assuming we migrate ROMs with guests - just makes sense. -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
Paolo Bonzini pbonz...@redhat.com wrote: On 19/11/2014 14:52, Peter Maydell wrote: It certainly seems pretty risky to introduce this change with only two weeks to go til release; I wouldn't want to merge it without a strong consensus from everybody involved that it really needed to go in for 2.2. I think there's consensus (you, Amit, me) that it should wait for 2.3. +1
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Tue, Nov 18, 2014 at 03:47:16PM +0100, Markus Armbruster wrote: Michael S. Tsirkin m...@redhat.com writes: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. Pardon my ignorance... changing ROM in migration? I would expect migration to be as transparent for ROM as it is for RAM. What am I missing? [...] Nothing really. We migrate RAM size too - but if there's a mismatch, migration fails. RAM size is user configurable. ROM is used internally so we have to figure out the size, and it turned out to be a hard problem, so we end up maintaining logic for ROM size like we did in 1.7 like we did in 2.0 etc. I don't want to add any more: let's just accept whatever is migrated, and stick to it. -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
Michael S. Tsirkin m...@redhat.com writes: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. Pardon my ignorance... changing ROM in migration? I would expect migration to be as transparent for ROM as it is for RAM. What am I missing? [...]
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On (Mon) 17 Nov 2014 [22:11:13], Michael S. Tsirkin wrote: And to make it clear, I'd like to see review from migration maintainers on this. The patchset crosses into pc so it's probably best to merge through my tree to avoic conflicts. Fine by me. Amit
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On (Mon) 17 Nov 2014 [22:08:46], Michael S. Tsirkin wrote: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. This already requires somewhat messy compatibility support in acpi generation code, and it looks like there'll be more to come. Rather than try to guess the correct size once and for all, this patchset tries to make code future-proof, by adding support for resizeable ram blocks. A (possibly very high) amount of space in ram_addr_t space is reserved for each block, but never allocated. If incoming block size differs from current size, block is reallocated. FW CFG is also notified and updated accordingly. To simplify things, I didn't add support for resizing actual RAM: device RAM such as fw cfg ROMs are never mapped into guests directly, so instead I added an API to flag device RAM explicitly, and manage them using simple alloc/free/realloc Considering this promises to rid us of worries about ROM size considerations once and for all, I thinking about pushing this as a kind of bugfix before 2.2, so we don't need to maintain more band-aids in 2.3 and on. I'd rather wait for 2.3; we've done this for a couple of releases already, so what's one more. And we're at rc2 already.. Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. I think that'll be too risky; unless we do S4 before / after migration to ensure the kernel realises things might be changing beneath its feet. Amit
[Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. This already requires somewhat messy compatibility support in acpi generation code, and it looks like there'll be more to come. Rather than try to guess the correct size once and for all, this patchset tries to make code future-proof, by adding support for resizeable ram blocks. A (possibly very high) amount of space in ram_addr_t space is reserved for each block, but never allocated. If incoming block size differs from current size, block is reallocated. FW CFG is also notified and updated accordingly. To simplify things, I didn't add support for resizing actual RAM: device RAM such as fw cfg ROMs are never mapped into guests directly, so instead I added an API to flag device RAM explicitly, and manage them using simple alloc/free/realloc Considering this promises to rid us of worries about ROM size considerations once and for all, I thinking about pushing this as a kind of bugfix before 2.2, so we don't need to maintain more band-aids in 2.3 and on. Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. Note: cross version migration testing still ongoing, I'll definitely complete that before pushing it out. Michael S. Tsirkin (5): cpu: add cpu_physical_memory_clear_dirty_range_nocode exec: qemu_ram_alloc_device, qemu_ram_resize arch_init: support resizing on incoming migration memory: interface to allocate device ram acpi-build: make ROMs device RAM, make them resizeable hw/lm32/lm32_hwsetup.h | 3 +- include/exec/cpu-all.h | 8 +++- include/exec/memory.h | 22 ++ include/exec/ram_addr.h | 15 +++ include/hw/loader.h | 4 +- arch_init.c | 13 +++--- exec.c | 113 +++- hw/core/loader.c| 18 ++-- hw/i386/acpi-build.c| 19 +--- memory.c| 17 10 files changed, 202 insertions(+), 30 deletions(-) -- MST
Re: [Qemu-devel] [PATCH 0/5] pc: make ROMs resizeable
On Mon, Nov 17, 2014 at 10:08:46PM +0200, Michael S. Tsirkin wrote: At the moment we migrate ROMs which reside in fw cfg, which allows changing ROM code at will, and supports migrating largish blocks early, with good performance. However, we are running into a problem: changing size breaks migration every time. This already requires somewhat messy compatibility support in acpi generation code, and it looks like there'll be more to come. Rather than try to guess the correct size once and for all, this patchset tries to make code future-proof, by adding support for resizeable ram blocks. A (possibly very high) amount of space in ram_addr_t space is reserved for each block, but never allocated. If incoming block size differs from current size, block is reallocated. FW CFG is also notified and updated accordingly. To simplify things, I didn't add support for resizing actual RAM: device RAM such as fw cfg ROMs are never mapped into guests directly, so instead I added an API to flag device RAM explicitly, and manage them using simple alloc/free/realloc Considering this promises to rid us of worries about ROM size considerations once and for all, I thinking about pushing this as a kind of bugfix before 2.2, so we don't need to maintain more band-aids in 2.3 and on. Note: migration stream is unaffected by these patches. This makes it possible to enable this functionality unconditionally, for all machine types. In the future, this might be handy for other things, such as changing kernels loaded on command line across migrations. Note: cross version migration testing still ongoing, I'll definitely complete that before pushing it out. And to make it clear, I'd like to see review from migration maintainers on this. The patchset crosses into pc so it's probably best to merge through my tree to avoic conflicts. Michael S. Tsirkin (5): cpu: add cpu_physical_memory_clear_dirty_range_nocode exec: qemu_ram_alloc_device, qemu_ram_resize arch_init: support resizing on incoming migration memory: interface to allocate device ram acpi-build: make ROMs device RAM, make them resizeable hw/lm32/lm32_hwsetup.h | 3 +- include/exec/cpu-all.h | 8 +++- include/exec/memory.h | 22 ++ include/exec/ram_addr.h | 15 +++ include/hw/loader.h | 4 +- arch_init.c | 13 +++--- exec.c | 113 +++- hw/core/loader.c| 18 ++-- hw/i386/acpi-build.c| 19 +--- memory.c| 17 10 files changed, 202 insertions(+), 30 deletions(-) -- MST
