Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Avi Kivity writes: > On 01/07/2010 02:33 PM, Anthony Liguori wrote: >> >> There's another option. >> >> Make cpuid information part of live migration protocol, and then >> support something like -cpu Xeon-3550. We would remember the exact >> cpuid mask we present to the guest and then we could validate that >> we can obtain the same mask on the destination. > > Currently, our policy is to only migrate dynamic (from the guest's > point of view) state, and specify static state on the command line > [1]. > > I think your suggestion makes a lot of sense, but I'd like to expand > it to move all guest state, whether dynamic or static. So '-m 1G' > would be migrated as well (but not -mem-path). Similarly, in -drive > file=...,if=ide,index=1, everything but file=... would be migrated. Becomes a bit clearer with the new way to configure stuff: -drive if=none,id=DRIVE-ID,...<--- host, don't migrate -device ide=drive,drive=DRIVE-ID,... <--- guest, do migrate [...]
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 03:14 PM, Anthony Liguori wrote: On 01/07/2010 06:40 AM, Avi Kivity wrote: On 01/07/2010 02:33 PM, Anthony Liguori wrote: There's another option. Make cpuid information part of live migration protocol, and then support something like -cpu Xeon-3550. We would remember the exact cpuid mask we present to the guest and then we could validate that we can obtain the same mask on the destination. It solves controlling the destination qemu execution all right but does not change the initial spawning of the original guest - to know whether ,-syscall is needed or not. Anyway, I'm in favor of it too. Currently, our policy is to only migrate dynamic (from the guest's point of view) state, and specify static state on the command line [1]. I think your suggestion makes a lot of sense, but I'd like to expand it to move all guest state, whether dynamic or static. So '-m 1G' would be migrated as well (but not -mem-path). Similarly, in -drive file=...,if=ide,index=1, everything but file=... would be migrated. Yes, I agree with this and it should be in the form of an fdt. This means we need full qdev conversion. But I think cpuid is somewhere in the middle with respect to static vs. dynamic. For instance, -cpu host is very dynamic in that you get very difficult results on different systems. Likewise, because of kvm filtering, even -cpu qemu64 can be dynamic. So if we didn't have filtering and -cpu host, I'd agree that it's totally static but I think in the current state, it's dynamic. This has an advantage wrt hotplug: since qemu is responsible for migrating all guest visible information, the migrator is no longer responsible for replaying hotplug events in the exact sequence they happened. Yup, 100% in agreement as a long term goal. In short, I think we should apply your suggestion as broadly as possible. [1] cpuid state is actually dynamic; repeated cpuid instruction execution with the same operands can return different results. kvm supports querying and setting this state. Yes, and we save some cpuid state in cpu. We just don't save all of it. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 06:40 AM, Avi Kivity wrote: On 01/07/2010 02:33 PM, Anthony Liguori wrote: There's another option. Make cpuid information part of live migration protocol, and then support something like -cpu Xeon-3550. We would remember the exact cpuid mask we present to the guest and then we could validate that we can obtain the same mask on the destination. Currently, our policy is to only migrate dynamic (from the guest's point of view) state, and specify static state on the command line [1]. I think your suggestion makes a lot of sense, but I'd like to expand it to move all guest state, whether dynamic or static. So '-m 1G' would be migrated as well (but not -mem-path). Similarly, in -drive file=...,if=ide,index=1, everything but file=... would be migrated. Yes, I agree with this and it should be in the form of an fdt. This means we need full qdev conversion. But I think cpuid is somewhere in the middle with respect to static vs. dynamic. For instance, -cpu host is very dynamic in that you get very difficult results on different systems. Likewise, because of kvm filtering, even -cpu qemu64 can be dynamic. So if we didn't have filtering and -cpu host, I'd agree that it's totally static but I think in the current state, it's dynamic. This has an advantage wrt hotplug: since qemu is responsible for migrating all guest visible information, the migrator is no longer responsible for replaying hotplug events in the exact sequence they happened. Yup, 100% in agreement as a long term goal. In short, I think we should apply your suggestion as broadly as possible. [1] cpuid state is actually dynamic; repeated cpuid instruction execution with the same operands can return different results. kvm supports querying and setting this state. Yes, and we save some cpuid state in cpu. We just don't save all of it. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 02:47 PM, Daniel P. Berrange wrote: With the introduction of the new -device spport, there's no need to replay hotplug events in order any more. Instead just use static PCI addresses when starting the guest, and the same addresses after migration. You could argue that QEMU should preserve the addressing automatically during migration, but apps need to do it manually already to keep addreses stable across power-offs, so doing it manually across migration too is no extra burden. That's true - shutdown and startup are an equivalent problem to live migration from that point of view. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Thu, Jan 07, 2010 at 02:40:34PM +0200, Avi Kivity wrote: > On 01/07/2010 02:33 PM, Anthony Liguori wrote: > > > >There's another option. > > > >Make cpuid information part of live migration protocol, and then > >support something like -cpu Xeon-3550. We would remember the exact > >cpuid mask we present to the guest and then we could validate that we > >can obtain the same mask on the destination. > > Currently, our policy is to only migrate dynamic (from the guest's point > of view) state, and specify static state on the command line [1]. > > I think your suggestion makes a lot of sense, but I'd like to expand it > to move all guest state, whether dynamic or static. So '-m 1G' would be > migrated as well (but not -mem-path). Similarly, in -drive > file=...,if=ide,index=1, everything but file=... would be migrated. > > This has an advantage wrt hotplug: since qemu is responsible for > migrating all guest visible information, the migrator is no longer > responsible for replaying hotplug events in the exact sequence they > happened. With the introduction of the new -device spport, there's no need to replay hotplug events in order any more. Instead just use static PCI addresses when starting the guest, and the same addresses after migration. You could argue that QEMU should preserve the addressing automatically during migration, but apps need to do it manually already to keep addreses stable across power-offs, so doing it manually across migration too is no extra burden. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 02:33 PM, Anthony Liguori wrote: There's another option. Make cpuid information part of live migration protocol, and then support something like -cpu Xeon-3550. We would remember the exact cpuid mask we present to the guest and then we could validate that we can obtain the same mask on the destination. Currently, our policy is to only migrate dynamic (from the guest's point of view) state, and specify static state on the command line [1]. I think your suggestion makes a lot of sense, but I'd like to expand it to move all guest state, whether dynamic or static. So '-m 1G' would be migrated as well (but not -mem-path). Similarly, in -drive file=...,if=ide,index=1, everything but file=... would be migrated. This has an advantage wrt hotplug: since qemu is responsible for migrating all guest visible information, the migrator is no longer responsible for replaying hotplug events in the exact sequence they happened. In short, I think we should apply your suggestion as broadly as possible. [1] cpuid state is actually dynamic; repeated cpuid instruction execution with the same operands can return different results. kvm supports querying and setting this state. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 06:20 AM, Dor Laor wrote: On 01/07/2010 02:00 PM, Avi Kivity wrote: On 01/07/2010 01:44 PM, Dor Laor wrote: So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary to say: (2.6.33) qemu -cpu Nehalem,-syscall (2.6.18) qemu -cpu Nehalem Or let qemu do it automatically for you. qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on another node. We can live with it, either have qemu realize the kernel version out of another existing feature or query uname. Alternatively, the matching libvirt package can be the one adding or removing it in the right distribution. There's another option. Make cpuid information part of live migration protocol, and then support something like -cpu Xeon-3550. We would remember the exact cpuid mask we present to the guest and then we could validate that we can obtain the same mask on the destination. Regards, Anthony Liguori -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 02:00 PM, Avi Kivity wrote: On 01/07/2010 01:44 PM, Dor Laor wrote: So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary to say: (2.6.33) qemu -cpu Nehalem,-syscall (2.6.18) qemu -cpu Nehalem Or let qemu do it automatically for you. qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on another node. We can live with it, either have qemu realize the kernel version out of another existing feature or query uname. Alternatively, the matching libvirt package can be the one adding or removing it in the right distribution.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 01:59 PM, Avi Kivity wrote: On 01/07/2010 11:40 AM, Dor Laor wrote: There's no such thing as Nehalem. Intel were ok with it. Again, you can name is corei7 or xeon34234234234, I don't care, the principle remains the same. There are several processors belonging to the Nehalem family and each have different features. We can start with the older one, and once it get's that important add the newer ones. Until that happens users can either use -host or have nehalem,+sse4_2,+newFeature What's not simple about the above 4 options? If a qemu/kvm/processor combo doesn't support a feature (say, nx) we have to remove it from the migration pool even if the Nehalem processor class says it's included. Or else not admit that combination into the migration pool in the first place. It still management role to compute least common denominator for live migration sets. btw: for nx disabled bios, we should just ignore it from product. Qemu direct users should add ,-nx What's a better alternative (that insures users understand it and use it and guest msi and even skype application is happy about it)? Have management scan new nodes and classify them. Of course. Just don't let management control stepping automatically, it should only be made for very advanced users and only manually.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 01:44 PM, Dor Laor wrote: So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary to say: (2.6.33) qemu -cpu Nehalem,-syscall (2.6.18) qemu -cpu Nehalem Or let qemu do it automatically for you. qemu on 2.6.33 doesn't know that you're running qemu on 2.6.18 on another node. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 11:40 AM, Dor Laor wrote: There's no such thing as Nehalem. Intel were ok with it. Again, you can name is corei7 or xeon34234234234, I don't care, the principle remains the same. There are several processors belonging to the Nehalem family and each have different features. What's not simple about the above 4 options? If a qemu/kvm/processor combo doesn't support a feature (say, nx) we have to remove it from the migration pool even if the Nehalem processor class says it's included. Or else not admit that combination into the migration pool in the first place. What's a better alternative (that insures users understand it and use it and guest msi and even skype application is happy about it)? Have management scan new nodes and classify them. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 01:39 PM, Anthony Liguori wrote: On 01/07/2010 03:40 AM, Dor Laor wrote: There's no simple solution except to restrict features to what was available on the first processors. What's not simple about the above 4 options? What's a better alternative (that insures users understand it and use it and guest msi and even skype application is happy about it)? Even if you have -cpu Nehalem, different versions of the KVM kernel module may additionally filter cpuid flags. So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary to say: (2.6.33) qemu -cpu Nehalem,-syscall (2.6.18) qemu -cpu Nehalem Or let qemu do it automatically for you. In order to be compatible. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 03:40 AM, Dor Laor wrote: There's no simple solution except to restrict features to what was available on the first processors. What's not simple about the above 4 options? What's a better alternative (that insures users understand it and use it and guest msi and even skype application is happy about it)? Even if you have -cpu Nehalem, different versions of the KVM kernel module may additionally filter cpuid flags. So if you had a 2.6.18 kernel and a 2.6.33 kernel, it may be necessary to say: (2.6.33) qemu -cpu Nehalem,-syscall (2.6.18) qemu -cpu Nehalem In order to be compatible. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 11:24 AM, Avi Kivity wrote: On 01/07/2010 11:11 AM, Dor Laor wrote: On 01/07/2010 10:18 AM, Avi Kivity wrote: On 01/07/2010 10:03 AM, Dor Laor wrote: We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. AMD and Intel already defined their names (in cat /proc/cpuinfo). They don't define families, the whole idea is to segment the market. The idea here is to minimize the number of models we should have the following range for Intel for example: pentium3 - merom - penry - Nehalem - host - kvm/qemu64 So we're supplying wide range of cpus, p3 for maximum flexibility and migration, nehalem for performance and migration, host for maximum performance and qemu/kvm64 for custom maid. There's no such thing as Nehalem. Intel were ok with it. Again, you can name is corei7 or xeon34234234234, I don't care, the principle remains the same. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 They don't have to deal with different qemu and kvm versions. Both our customers - the end users. It's not their problem. IMO what's missing today is a safe and sound cpu emulation that is simply and friendly to represent. qemu64,+popcount is not simple for the end user. There is no reason to through it on higher level mgmt. There's no simple solution except to restrict features to what was available on the first processors. What's not simple about the above 4 options? What's a better alternative (that insures users understand it and use it and guest msi and even skype application is happy about it)?
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 11:11 AM, Dor Laor wrote: On 01/07/2010 10:18 AM, Avi Kivity wrote: On 01/07/2010 10:03 AM, Dor Laor wrote: We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. AMD and Intel already defined their names (in cat /proc/cpuinfo). They don't define families, the whole idea is to segment the market. The idea here is to minimize the number of models we should have the following range for Intel for example: pentium3 - merom - penry - Nehalem - host - kvm/qemu64 So we're supplying wide range of cpus, p3 for maximum flexibility and migration, nehalem for performance and migration, host for maximum performance and qemu/kvm64 for custom maid. There's no such thing as Nehalem. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 They don't have to deal with different qemu and kvm versions. Both our customers - the end users. It's not their problem. IMO what's missing today is a safe and sound cpu emulation that is simply and friendly to represent. qemu64,+popcount is not simple for the end user. There is no reason to through it on higher level mgmt. There's no simple solution except to restrict features to what was available on the first processors. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 10:24 AM, Daniel P. Berrange wrote: On Thu, Jan 07, 2010 at 10:03:28AM +0200, Dor Laor wrote: On 01/06/2010 05:16 PM, Anthony Liguori wrote: On 01/06/2010 08:48 AM, Dor Laor wrote: On 01/06/2010 04:32 PM, Avi Kivity wrote: On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. Hardware upgrades or software upgrades? Yes. I just want to remind all the the main motivation for using -cpu realModelThatWasOnceShiped is to provide correct cpu emulation for the guest. Using a random qemu|kvm64+flag1-flag2 might really cause trouble for the guest OS or guest apps. On top of -cpu nehalem we can always add fancy features like x2apic, etc. I think it boils down to, how are people going to use this. For individuals, code names like Nehalem are too obscure. From my own personal experience, even power users often have no clue whether there processor is a Nehalem or not. For management tools, Nehalem is a somewhat imprecise target because it covers a wide range of potential processors. In general, I think what we really need to do is simplify the process of going from, here's the output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu so that migration always works for these systems. I don't think -cpu nehalem really helps with that problem. -cpu none helps a bit, but I hope we can find something nicer. We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. There are two main motivations behind the above approach: 1. Sound guest cpu definition. Using a predefined model should automatically set all the relevant vendor/stepping/cpuid flags/cache sizes/etc. We just can let every management application deal with it. It breaks guest OS/apps. For instance there are MSI support in windows guest relay on the stepping. 2. Simplifying end user and mgmt tools. qemu/kvm have the best knowledge about these low levels. If we push it up in the stack, eventually it reaches the user. The end user, not a 'qemu-devel user' which is actually far better from the average user. This means that such users will have to know what is popcount and whether or not to limit migration on one host by adding sse4.2 or not. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 Why should we invent the wheel (qemu64..)? Let's learn from their experience. NB, be careful to distinguish the different levels of VMwares mgmt stack. In terms of guest configuration, VMWare ESX APIs require the management app to specify the raw CPUID masks. With VirtualCenter VMotion they defined this handful of common Intel/AMD CPU sets, and will automatically classify hosts Live migration is the prime motivation for it. In addition, as we all know windows guest do not like to find a new cpu every time they boot. into one of these sets and use that to specify a default CPUID mask, in the case that the guest does not have an explicit one in its config. This gives them good default, out-of-the-box behaviour, while also allowing mgmt apps 100% control over each guest's CPUID should they want it. That's exactly what we need. Regards, Daniel
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 10:18 AM, Avi Kivity wrote: On 01/07/2010 10:03 AM, Dor Laor wrote: We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. AMD and Intel already defined their names (in cat /proc/cpuinfo). They don't define families, the whole idea is to segment the market. The idea here is to minimize the number of models we should have the following range for Intel for example: pentium3 - merom - penry - Nehalem - host - kvm/qemu64 So we're supplying wide range of cpus, p3 for maximum flexibility and migration, nehalem for performance and migration, host for maximum performance and qemu/kvm64 for custom maid. There are two main motivations behind the above approach: 1. Sound guest cpu definition. Using a predefined model should automatically set all the relevant vendor/stepping/cpuid flags/cache sizes/etc. We just can let every management application deal with it. It breaks guest OS/apps. For instance there are MSI support in windows guest relay on the stepping. 2. Simplifying end user and mgmt tools. qemu/kvm have the best knowledge about these low levels. If we push it up in the stack, eventually it reaches the user. The end user, not a 'qemu-devel user' which is actually far better from the average user. This means that such users will have to know what is popcount and whether or not to limit migration on one host by adding sse4.2 or not. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 They don't have to deal with different qemu and kvm versions. Both our customers - the end users. It's not their problem. IMO what's missing today is a safe and sound cpu emulation that is simply and friendly to represent. qemu64,+popcount is not simple for the end user. There is no reason to through it on higher level mgmt.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Thu, Jan 07, 2010 at 10:03:28AM +0200, Dor Laor wrote: > On 01/06/2010 05:16 PM, Anthony Liguori wrote: > >On 01/06/2010 08:48 AM, Dor Laor wrote: > >>On 01/06/2010 04:32 PM, Avi Kivity wrote: > >>>On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: > >We can probably default -enable-kvm to -cpu host, as long as we > >explain > >very carefully that if users wish to preserve cpu features across > >upgrades, they can't depend on the default. > Hardware upgrades or software upgrades? > >>> > >>>Yes. > >>> > >> > >>I just want to remind all the the main motivation for using -cpu > >>realModelThatWasOnceShiped is to provide correct cpu emulation for the > >>guest. Using a random qemu|kvm64+flag1-flag2 might really cause > >>trouble for the guest OS or guest apps. > >> > >>On top of -cpu nehalem we can always add fancy features like x2apic, etc. > > > >I think it boils down to, how are people going to use this. > > > >For individuals, code names like Nehalem are too obscure. From my own > >personal experience, even power users often have no clue whether there > >processor is a Nehalem or not. > > > >For management tools, Nehalem is a somewhat imprecise target because it > >covers a wide range of potential processors. In general, I think what we > >really need to do is simplify the process of going from, here's the > >output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu > >so that migration always works for these systems. > > > >I don't think -cpu nehalem really helps with that problem. -cpu none > >helps a bit, but I hope we can find something nicer. > > We can debate about the exact name/model to represent the Nehalem > family, I don't have an issue with that and actually Intel and Amd > should define it. > > There are two main motivations behind the above approach: > 1. Sound guest cpu definition. >Using a predefined model should automatically set all the relevant >vendor/stepping/cpuid flags/cache sizes/etc. >We just can let every management application deal with it. It breaks >guest OS/apps. For instance there are MSI support in windows guest >relay on the stepping. > > 2. Simplifying end user and mgmt tools. >qemu/kvm have the best knowledge about these low levels. If we push >it up in the stack, eventually it reaches the user. The end user, >not a 'qemu-devel user' which is actually far better from the >average user. > >This means that such users will have to know what is popcount and >whether or not to limit migration on one host by adding sse4.2 or >not. > > This is exactly what vmware are doing: > - Intel CPUs : > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 > - AMD CPUs : > http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 > > Why should we invent the wheel (qemu64..)? Let's learn from their > experience. NB, be careful to distinguish the different levels of VMwares mgmt stack. In terms of guest configuration, VMWare ESX APIs require the management app to specify the raw CPUID masks. With VirtualCenter VMotion they defined this handful of common Intel/AMD CPU sets, and will automatically classify hosts into one of these sets and use that to specify a default CPUID mask, in the case that the guest does not have an explicit one in its config. This gives them good default, out-of-the-box behaviour, while also allowing mgmt apps 100% control over each guest's CPUID should they want it. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/07/2010 10:03 AM, Dor Laor wrote: We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. AMD and Intel already defined their names (in cat /proc/cpuinfo). They don't define families, the whole idea is to segment the market. There are two main motivations behind the above approach: 1. Sound guest cpu definition. Using a predefined model should automatically set all the relevant vendor/stepping/cpuid flags/cache sizes/etc. We just can let every management application deal with it. It breaks guest OS/apps. For instance there are MSI support in windows guest relay on the stepping. 2. Simplifying end user and mgmt tools. qemu/kvm have the best knowledge about these low levels. If we push it up in the stack, eventually it reaches the user. The end user, not a 'qemu-devel user' which is actually far better from the average user. This means that such users will have to know what is popcount and whether or not to limit migration on one host by adding sse4.2 or not. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 They don't have to deal with different qemu and kvm versions. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 05:16 PM, Anthony Liguori wrote: On 01/06/2010 08:48 AM, Dor Laor wrote: On 01/06/2010 04:32 PM, Avi Kivity wrote: On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. Hardware upgrades or software upgrades? Yes. I just want to remind all the the main motivation for using -cpu realModelThatWasOnceShiped is to provide correct cpu emulation for the guest. Using a random qemu|kvm64+flag1-flag2 might really cause trouble for the guest OS or guest apps. On top of -cpu nehalem we can always add fancy features like x2apic, etc. I think it boils down to, how are people going to use this. For individuals, code names like Nehalem are too obscure. From my own personal experience, even power users often have no clue whether there processor is a Nehalem or not. For management tools, Nehalem is a somewhat imprecise target because it covers a wide range of potential processors. In general, I think what we really need to do is simplify the process of going from, here's the output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu so that migration always works for these systems. I don't think -cpu nehalem really helps with that problem. -cpu none helps a bit, but I hope we can find something nicer. We can debate about the exact name/model to represent the Nehalem family, I don't have an issue with that and actually Intel and Amd should define it. There are two main motivations behind the above approach: 1. Sound guest cpu definition. Using a predefined model should automatically set all the relevant vendor/stepping/cpuid flags/cache sizes/etc. We just can let every management application deal with it. It breaks guest OS/apps. For instance there are MSI support in windows guest relay on the stepping. 2. Simplifying end user and mgmt tools. qemu/kvm have the best knowledge about these low levels. If we push it up in the stack, eventually it reaches the user. The end user, not a 'qemu-devel user' which is actually far better from the average user. This means that such users will have to know what is popcount and whether or not to limit migration on one host by adding sse4.2 or not. This is exactly what vmware are doing: - Intel CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1991 - AMD CPUs : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1992 Why should we invent the wheel (qemu64..)? Let's learn from their experience. This is the test description of the original patch by John: # Intel # - # Management layers remove pentium3 by default. # It primarily remains here for testing of 32-bit migration. # [0:Pentium 3 Intel :vmx :pentium3;] # Core 2, 65nm # possible option sets: (+nx,+cx16), (+nx,+cx16,+ssse3) # 1:Merom :vmx,sse2 :qemu64,-nx,+sse2; # Core2 45nm # 2:Penryn :vmx,sse2,nx,cx16,ssse3,sse4_1 :qemu64,+sse2,+cx16,+ssse3,+sse4_1; # Core i7 45/32nm # 3:Nehalem :vmx,sse2,nx,cx16,ssse3,sse4_1,sse4_2,popcnt :qemu64,+sse2,+cx16,+ssse3,+sse4_1,+sse4_2,+popcnt; # AMD # --- # Management layers remove pentium3 by default. # It primarily remains here for testing of 32-bit migration. # [0:Pentium 3 AMD :svm :pentium3;] # Opteron 90nm stepping E1/E4/E6 # possible option sets: (-nx) for 130nm # 1:Opteron G1 :svm,sse2,nx :qemu64,+sse2; # Opteron 90nm stepping F2/F3 # 2:Opteron G2 :svm,sse2,nx,cx16,rdtscp :qemu64,+sse2,+cx16,+rdtscp; # Opteron 65/45nm # 3:Opteron G3 :svm,sse2,nx,cx16,sse4a,misalignsse,popcnt,abm :qemu64,+sse2,+cx16,+sse4a,+misalignsse,+popcnt,+abm; Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Tue, Jan 05, 2010 at 06:10:10PM -0600, Anthony Liguori wrote: > Typically, there is at least a little sanity naming for these cases. > For instance, any Xeon W35xx should have the same features. A Xeon > W55xx may be different. It doesn't work that way for intel. For example: Core 2 Duo E7400 AT80571PH0723M does not have VT support. Core 2 Duo E7400 AT80571PH0723ML does have VT support. Good luck buying the variant you want. So model names do not define the feature set. > It's not going to be easy to include every possible model. It's a hard > problem for management tools too. The thing is, I imagine most > management tools are going to cat /proc/cpuinfo to get what the > processor is and that's going to be a Xeon YY type name so I really > believe that's the thing that makes sense to expose in QEMU. > > Maybe we could name models like IntelXeonW35xx. > > Clever use of the preprocessor will make this effort much, much saner. > Also, we really need some sort of human readable document that explains > the differences between processor classes and makes recommendations > about what management tools should take into consideration. -- Len Sorensen
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 08:48 AM, Dor Laor wrote: On 01/06/2010 04:32 PM, Avi Kivity wrote: On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. Hardware upgrades or software upgrades? Yes. I just want to remind all the the main motivation for using -cpu realModelThatWasOnceShiped is to provide correct cpu emulation for the guest. Using a random qemu|kvm64+flag1-flag2 might really cause trouble for the guest OS or guest apps. On top of -cpu nehalem we can always add fancy features like x2apic, etc. I think it boils down to, how are people going to use this. For individuals, code names like Nehalem are too obscure. From my own personal experience, even power users often have no clue whether there processor is a Nehalem or not. For management tools, Nehalem is a somewhat imprecise target because it covers a wide range of potential processors. In general, I think what we really need to do is simplify the process of going from, here's the output of /proc/cpuinfo for a 100 nodes, what do I need to pass to qemu so that migration always works for these systems. I don't think -cpu nehalem really helps with that problem. -cpu none helps a bit, but I hope we can find something nicer. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 07:54 AM, Avi Kivity wrote:
On 01/06/2010 03:49 PM, Anthony Liguori wrote:
I think that's workable but I think there may be some subtle issues
especially across qemu versions. Can you give an example of what
you would expect the output to be?
-> { command: query-cpu-capabalities }
<- { result: { features: [vm, fpu, lm, sse2, sse3, e3,
ssse3, sse3.14 ], cache: { ... }, vendor: { }, etc. } }
Or something. We'd need similar queries for the number of PCI
slots, for example, so the GUI can tell the user when adding
hardware is no longer an option instead of trying it blindly and
returning an error.
Yeah, the trick with this is that we don't have such a thing as a
bare bones CPU in qemu right now.
So you can't really say -cpu qemu64+vm+fpu+lm
Because some extra features might sneak in. What I'm getting at is
that we need a way to make the results of this command translate into
a -cpu invocation that works reliably across multiple versions of qemu.
Well, we can freeze qemu64 if we wish. That's still not 100% accurate
since kvm can remove features from qemu64.
-cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?
Yeah, I'm not sure I can think of a better way.
Regards,
Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Wed, Jan 06, 2010 at 04:32:07PM +0200, Avi Kivity wrote: > On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: >>> We can probably default -enable-kvm to -cpu host, as long as we explain >>> very carefully that if users wish to preserve cpu features across >>> upgrades, they can't depend on the default. >>> >> Hardware upgrades or software upgrades? >> > > Yes. IMO software upgrades should not change anything as long as user uses -M . CPU flags should be whitelisted in machine description. > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 04:32 PM, Avi Kivity wrote: On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. Hardware upgrades or software upgrades? Yes. I just want to remind all the the main motivation for using -cpu realModelThatWasOnceShiped is to provide correct cpu emulation for the guest. Using a random qemu|kvm64+flag1-flag2 might really cause trouble for the guest OS or guest apps. On top of -cpu nehalem we can always add fancy features like x2apic, etc.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 04:22 PM, Michael S. Tsirkin wrote: We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. Hardware upgrades or software upgrades? Yes. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Wed, Jan 06, 2010 at 03:58:01PM +0200, Avi Kivity wrote: > On 01/06/2010 03:55 PM, Alexander Graf wrote: >> >>> Well, we can freeze qemu64 if we wish. That's still not 100% accurate >>> since kvm can remove features from qemu64. >>> >>> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum? >>> >> I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features. >> > > We can probably default -enable-kvm to -cpu host, as long as we explain > very carefully that if users wish to preserve cpu features across > upgrades, they can't depend on the default. Hardware upgrades or software upgrades? > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 03:55 PM, Alexander Graf wrote: Well, we can freeze qemu64 if we wish. That's still not 100% accurate since kvm can remove features from qemu64. -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum? I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features. We can probably default -enable-kvm to -cpu host, as long as we explain very carefully that if users wish to preserve cpu features across upgrades, they can't depend on the default. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 06.01.2010, at 14:54, Avi Kivity wrote:
> On 01/06/2010 03:49 PM, Anthony Liguori wrote:
>>>
I think that's workable but I think there may be some subtle issues
especially across qemu versions. Can you give an example of what you
would expect the output to be?
>>>
>>> -> { command: query-cpu-capabalities }
>>> <- { result: { features: [vm, fpu, lm, sse2, sse3, e3, ssse3,
>>> sse3.14 ], cache: { ... }, vendor: { }, etc. } }
>>>
>>> Or something. We'd need similar queries for the number of PCI slots, for
>>> example, so the GUI can tell the user when adding hardware is no longer an
>>> option instead of trying it blindly and returning an error.
>>
>>
>> Yeah, the trick with this is that we don't have such a thing as a bare bones
>> CPU in qemu right now.
>>
>> So you can't really say -cpu qemu64+vm+fpu+lm
>>
>> Because some extra features might sneak in. What I'm getting at is that we
>> need a way to make the results of this command translate into a -cpu
>> invocation that works reliably across multiple versions of qemu.
>
> Well, we can freeze qemu64 if we wish. That's still not 100% accurate since
> kvm can remove features from qemu64.
>
> -cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?
I'd rather add a "kvm" cpu and leave the qemu64 one to qemu tcg features.
Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 03:49 PM, Anthony Liguori wrote:
I think that's workable but I think there may be some subtle issues
especially across qemu versions. Can you give an example of what
you would expect the output to be?
-> { command: query-cpu-capabalities }
<- { result: { features: [vm, fpu, lm, sse2, sse3, e3, ssse3,
sse3.14 ], cache: { ... }, vendor: { }, etc. } }
Or something. We'd need similar queries for the number of PCI slots,
for example, so the GUI can tell the user when adding hardware is no
longer an option instead of trying it blindly and returning an error.
Yeah, the trick with this is that we don't have such a thing as a bare
bones CPU in qemu right now.
So you can't really say -cpu qemu64+vm+fpu+lm
Because some extra features might sneak in. What I'm getting at is
that we need a way to make the results of this command translate into
a -cpu invocation that works reliably across multiple versions of qemu.
Well, we can freeze qemu64 if we wish. That's still not 100% accurate
since kvm can remove features from qemu64.
-cpu none,+flags,vendor=foo,cache=bar,ad=nauseum?
--
error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 07:47 AM, Avi Kivity wrote:
On 01/06/2010 03:25 PM, Anthony Liguori wrote:
On 01/05/2010 09:25 PM, Avi Kivity wrote:
Typically, there is at least a little sanity naming for these
cases. For instance, any Xeon W35xx should have the same
features. A Xeon W55xx may be different.
It's not going to be easy to include every possible model. It's a
hard problem for management tools too. The thing is, I imagine
most management tools are going to cat /proc/cpuinfo to get what
the processor is and that's going to be a Xeon YY type name so
I really believe that's the thing that makes sense to expose in QEMU.
Maybe we could name models like IntelXeonW35xx.
While a W3501 should be similar to a W3599, we don't know if it
actually will be. You are no longer on a Fully Correct path and
instead you are wandering in Marketing Land.
Note that the processor type is just part of what determines which
features are exposed to the guest. Qemu version, kvm version, host
kernel version, and even kernel command-line parameters all play a
part, so to really determine migratability the management tool
should talk to qemu, not /proc/cpuinfo.
So if I understand correctly, you're advocating to drop the idea of
common model names, and provide a mechanism for a management tool to
query which cpu features are supported both by the processor, but
also filtered by qemu, kvm, etc?
Well, it's nice to have a -cpu X1234, so I wouldn't recommend dropping
it, but certainly a migration pool that doesn't assume anything about
the host qemu and kernel version needs more fine-grained information.
I think that's workable but I think there may be some subtle issues
especially across qemu versions. Can you give an example of what you
would expect the output to be?
-> { command: query-cpu-capabalities }
<- { result: { features: [vm, fpu, lm, sse2, sse3, e3, ssse3,
sse3.14 ], cache: { ... }, vendor: { }, etc. } }
Or something. We'd need similar queries for the number of PCI slots,
for example, so the GUI can tell the user when adding hardware is no
longer an option instead of trying it blindly and returning an error.
Yeah, the trick with this is that we don't have such a thing as a bare
bones CPU in qemu right now.
So you can't really say -cpu qemu64+vm+fpu+lm
Because some extra features might sneak in. What I'm getting at is that
we need a way to make the results of this command translate into a -cpu
invocation that works reliably across multiple versions of qemu.
Regards,
Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 03:25 PM, Anthony Liguori wrote:
On 01/05/2010 09:25 PM, Avi Kivity wrote:
Typically, there is at least a little sanity naming for these
cases. For instance, any Xeon W35xx should have the same features.
A Xeon W55xx may be different.
It's not going to be easy to include every possible model. It's a
hard problem for management tools too. The thing is, I imagine most
management tools are going to cat /proc/cpuinfo to get what the
processor is and that's going to be a Xeon YY type name so I
really believe that's the thing that makes sense to expose in QEMU.
Maybe we could name models like IntelXeonW35xx.
While a W3501 should be similar to a W3599, we don't know if it
actually will be. You are no longer on a Fully Correct path and
instead you are wandering in Marketing Land.
Note that the processor type is just part of what determines which
features are exposed to the guest. Qemu version, kvm version, host
kernel version, and even kernel command-line parameters all play a
part, so to really determine migratability the management tool should
talk to qemu, not /proc/cpuinfo.
So if I understand correctly, you're advocating to drop the idea of
common model names, and provide a mechanism for a management tool to
query which cpu features are supported both by the processor, but also
filtered by qemu, kvm, etc?
Well, it's nice to have a -cpu X1234, so I wouldn't recommend dropping
it, but certainly a migration pool that doesn't assume anything about
the host qemu and kernel version needs more fine-grained information.
I think that's workable but I think there may be some subtle issues
especially across qemu versions. Can you give an example of what you
would expect the output to be?
-> { command: query-cpu-capabalities }
<- { result: { features: [vm, fpu, lm, sse2, sse3, e3, ssse3,
sse3.14 ], cache: { ... }, vendor: { }, etc. } }
Or something. We'd need similar queries for the number of PCI slots,
for example, so the GUI can tell the user when adding hardware is no
longer an option instead of trying it blindly and returning an error.
--
error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Wed, Jan 06, 2010 at 07:25:10AM -0600, Anthony Liguori wrote: > On 01/05/2010 09:25 PM, Avi Kivity wrote: >>> Typically, there is at least a little sanity naming for these cases. >>> For instance, any Xeon W35xx should have the same features. A Xeon >>> W55xx may be different. >>> >>> It's not going to be easy to include every possible model. It's a >>> hard problem for management tools too. The thing is, I imagine most >>> management tools are going to cat /proc/cpuinfo to get what the >>> processor is and that's going to be a Xeon YY type name so I >>> really believe that's the thing that makes sense to expose in QEMU. >>> >>> Maybe we could name models like IntelXeonW35xx. >>> >> >> While a W3501 should be similar to a W3599, we don't know if it >> actually will be. You are no longer on a Fully Correct path and >> instead you are wandering in Marketing Land. >> >> Note that the processor type is just part of what determines which >> features are exposed to the guest. Qemu version, kvm version, host >> kernel version, and even kernel command-line parameters all play a >> part, so to really determine migratability the management tool should >> talk to qemu, not /proc/cpuinfo. > > So if I understand correctly, you're advocating to drop the idea of > common model names, and provide a mechanism for a management tool to > query which cpu features are supported both by the processor, but also > filtered by qemu, kvm, etc? The syscall issue shows there are several other things necessary IMO: - features need to be scoped with Vendor ID as some feature bits might not be 100% compatible. - Some features might be emulated. So it's not yes/no but yes/no/slow and management needs to know IMO. > I think that's workable but I think there may be some subtle issues > especially across qemu versions. Can you give an example of what you > would expect the output to be? > >>> Clever use of the preprocessor will make this effort much, much saner. >> >> I cringe whenever I read something like this. > > :-) > > Regards, > > Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/05/2010 09:25 PM, Avi Kivity wrote: Typically, there is at least a little sanity naming for these cases. For instance, any Xeon W35xx should have the same features. A Xeon W55xx may be different. It's not going to be easy to include every possible model. It's a hard problem for management tools too. The thing is, I imagine most management tools are going to cat /proc/cpuinfo to get what the processor is and that's going to be a Xeon YY type name so I really believe that's the thing that makes sense to expose in QEMU. Maybe we could name models like IntelXeonW35xx. While a W3501 should be similar to a W3599, we don't know if it actually will be. You are no longer on a Fully Correct path and instead you are wandering in Marketing Land. Note that the processor type is just part of what determines which features are exposed to the guest. Qemu version, kvm version, host kernel version, and even kernel command-line parameters all play a part, so to really determine migratability the management tool should talk to qemu, not /proc/cpuinfo. So if I understand correctly, you're advocating to drop the idea of common model names, and provide a mechanism for a management tool to query which cpu features are supported both by the processor, but also filtered by qemu, kvm, etc? I think that's workable but I think there may be some subtle issues especially across qemu versions. Can you give an example of what you would expect the output to be? Clever use of the preprocessor will make this effort much, much saner. I cringe whenever I read something like this. :-) Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 12:21 PM, Daniel P. Berrange wrote: On Wed, Jan 06, 2010 at 11:54:16AM +0200, Avi Kivity wrote: On 01/06/2010 11:44 AM, Daniel P. Berrange wrote: This is all a very long way of saying that mgmt apps based on libvirt won't care about model names exposed in /proc/cpuinfo so there's no particular need to have a direct mapping from them to QEMU for libvirt's needs. There is still a need to query qemu, since it will filter out bits that kvm.ko or the hardware don't support. If there was a more verbose version of '-cpu ?' which also gave the CPUID masks associated with each supported CPU model, that would be useful info avoiding need for libvirt to hardcode the mapping to QEMU Yes, that's what I'm thinking of. Along with enumerating all the other qemu capabilities. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Wed, Jan 06, 2010 at 11:54:16AM +0200, Avi Kivity wrote: > On 01/06/2010 11:44 AM, Daniel P. Berrange wrote: > > > >This is all a very long way of saying that mgmt apps based on libvirt > >won't care about model names exposed in /proc/cpuinfo so there's no > >particular need to have a direct mapping from them to QEMU for libvirt's > >needs. > > > > There is still a need to query qemu, since it will filter out bits that > kvm.ko or the hardware don't support. If there was a more verbose version of '-cpu ?' which also gave the CPUID masks associated with each supported CPU model, that would be useful info avoiding need for libvirt to hardcode the mapping to QEMU Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 11:44 AM, Daniel P. Berrange wrote: This is all a very long way of saying that mgmt apps based on libvirt won't care about model names exposed in /proc/cpuinfo so there's no particular need to have a direct mapping from them to QEMU for libvirt's needs. There is still a need to query qemu, since it will filter out bits that kvm.ko or the hardware don't support. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Tue, Jan 05, 2010 at 06:10:10PM -0600, Anthony Liguori wrote: > >>For instance, Xeon-X5570 should be a least common denominator for > >>Nehalem processors. It's probably better for users too. It's easier > >>for them to answer "do I have anything older than a Xeon-X5570" than > >>to ask "do I have any Woodcrest class processors". > >> > >>I encounter this confusion a lot. I usually ask people whether they > >>have a Nehalem processor when debugging something and their response > >>is always, I have a Xeon-XYZ, is that Nehalem? > > > >This is complicated by the fact that processors don't have > >straight-line development, and that marketing names don't correspond > >to anything. For example Xeon can be any of the Pentium 4, Core, Core > >2, and Nehalem (and perhaps other) microachitectures. A newer Xeon is > >often introduced with an older core (usually for larger machines) than > >previously existing Xeons. > > Typically, there is at least a little sanity naming for these cases. > For instance, any Xeon W35xx should have the same features. A Xeon > W55xx may be different. > > It's not going to be easy to include every possible model. It's a hard > problem for management tools too. The thing is, I imagine most > management tools are going to cat /proc/cpuinfo to get what the > processor is and that's going to be a Xeon YY type name so I really > believe that's the thing that makes sense to expose in QEMU. > > Maybe we could name models like IntelXeonW35xx. For libvirt, we have an XML database that contains a list of CPU model names, and (for x86) associated CPU ID flags. The guest XML can specify a model name, and we'll map that through to the matching QEMU model name if there's an exact match. Otherwise we'll map it to the closest QEMU model name + a set of named flags. The guest XML can also contain a list of named flags ontop of the basic model, and again we'll map that through to whatever QEMU provides. So apps using libvirt don't worry about what QEMU's exact naming scheme is for processors, but rather use libvirt's database of defined names which are mapped to QEMUs. Now for convenience our database happens to match the QEMU database exactly at this time :-) Finally we also use the same CPU model databse to expose what the host CPU is, and provide APIs to allow apps to query whether a desired guest model is compatible with the current host CPU model The reason for libvirt doing it in this way, is that we wanted to have apps use CPU model names primarily, but with option to override flags if they needed. For the Xen and VMWare drivers the CPU model names are in fact changed into CPUID masks, since that's all those two HVs accept. This is all a very long way of saying that mgmt apps based on libvirt won't care about model names exposed in /proc/cpuinfo so there's no particular need to have a direct mapping from them to QEMU for libvirt's needs. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 01/06/2010 02:10 AM, Anthony Liguori wrote: On 12/23/2009 04:32 AM, Avi Kivity wrote: On 12/22/2009 06:12 PM, Anthony Liguori wrote: I think the only two Fully Correct approachs are to support a very specific CPU (e.g. Xeon-X5270) or provide the ability to individually tweak cpu flags. Yes. By a curious coincidence these are what the hardware vendors define (unlike compat classes etc.). Whenever possible, steal from what the hardware vendors do :-) [...] Typically, there is at least a little sanity naming for these cases. For instance, any Xeon W35xx should have the same features. A Xeon W55xx may be different. It's not going to be easy to include every possible model. It's a hard problem for management tools too. The thing is, I imagine most management tools are going to cat /proc/cpuinfo to get what the processor is and that's going to be a Xeon YY type name so I really believe that's the thing that makes sense to expose in QEMU. Maybe we could name models like IntelXeonW35xx. While a W3501 should be similar to a W3599, we don't know if it actually will be. You are no longer on a Fully Correct path and instead you are wandering in Marketing Land. Note that the processor type is just part of what determines which features are exposed to the guest. Qemu version, kvm version, host kernel version, and even kernel command-line parameters all play a part, so to really determine migratability the management tool should talk to qemu, not /proc/cpuinfo. Clever use of the preprocessor will make this effort much, much saner. I cringe whenever I read something like this. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/23/2009 04:32 AM, Avi Kivity wrote: On 12/22/2009 06:12 PM, Anthony Liguori wrote: I think the only two Fully Correct approachs are to support a very specific CPU (e.g. Xeon-X5270) or provide the ability to individually tweak cpu flags. Yes. By a curious coincidence these are what the hardware vendors define (unlike compat classes etc.). Whenever possible, steal from what the hardware vendors do :-) The notion of compatibility classes should probably be left to management tools. We can make it a lot easier for them though by supporting turning point CPU models. Agreed. For instance, Xeon-X5570 should be a least common denominator for Nehalem processors. It's probably better for users too. It's easier for them to answer "do I have anything older than a Xeon-X5570" than to ask "do I have any Woodcrest class processors". I encounter this confusion a lot. I usually ask people whether they have a Nehalem processor when debugging something and their response is always, I have a Xeon-XYZ, is that Nehalem? This is complicated by the fact that processors don't have straight-line development, and that marketing names don't correspond to anything. For example Xeon can be any of the Pentium 4, Core, Core 2, and Nehalem (and perhaps other) microachitectures. A newer Xeon is often introduced with an older core (usually for larger machines) than previously existing Xeons. Typically, there is at least a little sanity naming for these cases. For instance, any Xeon W35xx should have the same features. A Xeon W55xx may be different. It's not going to be easy to include every possible model. It's a hard problem for management tools too. The thing is, I imagine most management tools are going to cat /proc/cpuinfo to get what the processor is and that's going to be a Xeon YY type name so I really believe that's the thing that makes sense to expose in QEMU. Maybe we could name models like IntelXeonW35xx. Clever use of the preprocessor will make this effort much, much saner. Also, we really need some sort of human readable document that explains the differences between processor classes and makes recommendations about what management tools should take into consideration. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Anthony Liguori wrote: > On 12/21/2009 02:28 AM, Dor Laor wrote: >> John's new cpu definitions are the exact solution for this issue - all >> users, whether using mgmt app or direct qemu (this is no user, this is >> a developer/hacker/other, let's do not optimize this case) should use >> the various 'real' cpu definitions like -cpu Merom | Nehalem | Penry | >> Opteron G1, > > Of course, the tricky part is at what level do you define these names. > For instance, do you do just Nehalem, or do you also do Nehalem, > Nehalem-EP, Nehalem-EX? >From a strict technical perspective I'd agree. But this is more an effort to define simplified discrete cpu models which reasonably reflect deployed silicon. An equal (if not greater) motivation is to intentionally dumb-down portions of the spectrum to simplify migration, albeit at the possible loss of features provided by an individual cpu. > Nehalem is really just a code name. Would it be better to use core-i7? I'd inquired whether marketing sanctioned names existed but didn't receive anything conclusive. The resulting monikers at least are in common usage. > I think the only two Fully Correct approachs are to support a very > specific CPU (e.g. Xeon-X5270) or provide the ability to individually > tweak cpu flags. We have the ability today to pass an exhaustive list of feature flags. Offering a sprawling list of fine tuned models is possible although somewhat cumbersome, and doesn't address the need to collapse ranges into a least common denominator for the benefit of migration. > For instance, Xeon-X5570 should be a least common denominator for > Nehalem processors. It's probably better for users too. It's easier > for them to answer "do I have anything older than a Xeon-X5570" than to > ask "do I have any Woodcrest class processors". I'm not particularly biased to a naming scheme. Although the generic labels avoid suggesting a specific instance of a cpu class which is helpful in this context. Engaging the vendors with this scheme was also somewhat of a sanity check to assure it reflects commercially deployed vs. development silicon. -john -- john.coo...@redhat.com
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/22/2009 06:12 PM, Anthony Liguori wrote: I think the only two Fully Correct approachs are to support a very specific CPU (e.g. Xeon-X5270) or provide the ability to individually tweak cpu flags. Yes. By a curious coincidence these are what the hardware vendors define (unlike compat classes etc.). The notion of compatibility classes should probably be left to management tools. We can make it a lot easier for them though by supporting turning point CPU models. Agreed. For instance, Xeon-X5570 should be a least common denominator for Nehalem processors. It's probably better for users too. It's easier for them to answer "do I have anything older than a Xeon-X5570" than to ask "do I have any Woodcrest class processors". I encounter this confusion a lot. I usually ask people whether they have a Nehalem processor when debugging something and their response is always, I have a Xeon-XYZ, is that Nehalem? This is complicated by the fact that processors don't have straight-line development, and that marketing names don't correspond to anything. For example Xeon can be any of the Pentium 4, Core, Core 2, and Nehalem (and perhaps other) microachitectures. A newer Xeon is often introduced with an older core (usually for larger machines) than previously existing Xeons. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Anthony Liguori wrote: > It would be insane to emulate sse3 too. It doesn't sound too ridiculous if TCG is involved, provided the switching between TCG and KVM isn't too rapid. TCG is slower, but it's not ridiculously slow. Though, I don't expect anyone to volunteer to implement it :-) > how likely is it that any given guest is using an obscure feature? Could KVM detect when a guest starts using each feature? For example, by disabling access to those features, enabling each one when the KVM trap occurs and recording that fact? Is it possible to enable them in the guest-visible cpuid, but force a trap on access to them? That might be quite useful information when you want to migrate something that wasn't planned for X years ago. -- Jamie
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Michael S. Tsirkin wrote: > Bootup on different machines has some of the same issues as migration. > Consider a 64 bit guest as an example, I think it can not boot on a 32 > bit host OS with kvm. I think you can use tcg, but it will be slower. > Same thing applies to other CPU features. Alas, perhaps tcg will not work. The impression I got from irqchip threads was if you have a 64 bit KVM guest and only access to a 32 bit host, you are screwed because KVM/QEMU irqchip models are not interchangable. Could be wrong though, it was a bit too in depth for my knowledge. > Many guests reconfigure themselves when you swap harware under them. > This makes it easier to move such guests between machines, or change > qemu configuration and still reuse guest image. Others might record > hardware state on setup (64 bit above is one such example) making such > changes more painful. Windows is notorious for breaking when the hardware changes. Ignoring the activation issue, which doesn't apply to corporate licenses or time-limited free issues, for example replacing IDE on PIIX4 with IDE on PIIX3 was enough to prevent both Windows 2000 and Windows Server 2003 from booting at all. Same when switching between SCSI and IDE, or vice versa. That's because they load the drivers needed for booting from a preassembled list of drivers that it needed list time, basically just the specific IDE or SCSI driver and a few other things, and they look for exact PCI ID matches to load them. I think the behaviour goes back to NT days, and is much worse for their "server" class OSes; their consumer ones like XP try harder to detect and cope with changes. In a nutshell, I have had to take a disk image from a real Windows machine to a working KVM image several times, and I've never been able to do it without either a technical blue screen on boot problem or an activation problem. -- Jamie
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 02:28 AM, Dor Laor wrote: John's new cpu definitions are the exact solution for this issue - all users, whether using mgmt app or direct qemu (this is no user, this is a developer/hacker/other, let's do not optimize this case) should use the various 'real' cpu definitions like -cpu Merom | Nehalem | Penry | Opteron G1, Of course, the tricky part is at what level do you define these names. For instance, do you do just Nehalem, or do you also do Nehalem, Nehalem-EP, Nehalem-EX? Nehalem is really just a code name. Would it be better to use core-i7? I think the only two Fully Correct approachs are to support a very specific CPU (e.g. Xeon-X5270) or provide the ability to individually tweak cpu flags. The notion of compatibility classes should probably be left to management tools. We can make it a lot easier for them though by supporting turning point CPU models. For instance, Xeon-X5570 should be a least common denominator for Nehalem processors. It's probably better for users too. It's easier for them to answer "do I have anything older than a Xeon-X5570" than to ask "do I have any Woodcrest class processors". I encounter this confusion a lot. I usually ask people whether they have a Nehalem processor when debugging something and their response is always, I have a Xeon-XYZ, is that Nehalem? Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Dor Laor wrote: > On 12/22/2009 12:51 AM, john cooper wrote: >> Dor Laor wrote: >> >>> Qemu will check the required cpuid of the cpu model on the host and >>> refuse to load otherwise. When moving to this model, migration can be >>> simplified too since there are fewer combination, and one can choose >>> performance over migration flexibility and wise versa. >>> Due to the above check, the destination qemu won't load if the host >>> does not support its cpu model. >> >> If you're referring to the check in my patch, that's >> currently advisory only. >> >> The existing cpu model encoding of CPUID tosses flags >> in to the soup on speculation they may be available >> on the host. If not it assumes they will be quietly >> disabled on their way to the guest along with whatever > > why not shout loudly and abort? Do you want windows to reactivate itself? This was the existing behavior of that code which expected certain requested flags to be casually dropped. But bailing after the warning is probably fine as well given the warning itself is optional and bailing wouldn't happen by default. That code could probably use a cleanup going forward to iron out such issues. Andre mentioned he had pending patches which in part addressed this so I thought to let those resolve beforehand. Thanks, -john -- john.coo...@redhat.com
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/22/2009 12:51 AM, john cooper wrote: Dor Laor wrote: Qemu will check the required cpuid of the cpu model on the host and refuse to load otherwise. When moving to this model, migration can be simplified too since there are fewer combination, and one can choose performance over migration flexibility and wise versa. Due to the above check, the destination qemu won't load if the host does not support its cpu model. If you're referring to the check in my patch, that's currently advisory only. The existing cpu model encoding of CPUID tosses flags in to the soup on speculation they may be available on the host. If not it assumes they will be quietly disabled on their way to the guest along with whatever why not shout loudly and abort? Do you want windows to reactivate itself? flags were enabled via + on the command line. This mixed treatment for model implied vs. user specified flags could be partly an artifact of trying to adapt the legacy qemu64 model to whatever host silicon it finds itself upon. -john
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Dor Laor wrote: Qemu will check the required cpuid of the cpu model on the host and refuse to load otherwise. When moving to this model, migration can be simplified too since there are fewer combination, and one can choose performance over migration flexibility and wise versa. Due to the above check, the destination qemu won't load if the host does not support its cpu model. If you're referring to the check in my patch, that's currently advisory only. The existing cpu model encoding of CPUID tosses flags in to the soup on speculation they may be available on the host. If not it assumes they will be quietly disabled on their way to the guest along with whatever flags were enabled via + on the command line. This mixed treatment for model implied vs. user specified flags could be partly an artifact of trying to adapt the legacy qemu64 model to whatever host silicon it finds itself upon. -john -- john.coo...@third-harmonic.com
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 02:59 PM, Andre Przywara wrote: KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3. It would be insane to emulate sse3 too. It does expose sse3 support (called pni in Linux IIRC). Not sure if qemu masks it, but the information is there. What about using -cpu kvm64? This has SSE3 as well as CMPXCHG16, both are available in all VMX/SVM capable machines (as far as I could find out) and in TCG. This gives a pretty descent base without loosing many relevant performance features. Sure. This gives more performance to users and preserves compatibility pretty well. The disadvantage is that the baseline will recede as more processor features are introduced. ... This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is. Right. My concern is with casual users upgrading their machine, not enterprise users who should be protected by their tools. Then what about fixing the CPUID bits to those returned by the KVM module the first time the guest is started? Later you would only use those bits (which may have been cropped) for later restarts of the same guest. If you only upgrade your machine, then there should be no problems. First, even an upgrade may cause loss of cpuid bits (moving to a different vendor). Second, where do you store those bits? P.S. What feature bits do we talk about? Maybe we should group them and use aliases for those. SS_S_E3 and SSE4.x come to mind, are any other bits really relevant? (Or do they justify the pain we have when propagating them?) Then we would only need: Athlon64, Core2, Nehalem (and maybe Phenom). Future bits too, for example AVX. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 03:45 PM, David S. Ahern wrote: On 12/21/2009 05:05 AM, Avi Kivity wrote: On 12/21/2009 01:45 PM, Alexander Graf wrote: Well, we have two groups: 1) Casual user w/o management app 2) Enterprise user w/ management app So I clearly belong to the first group. 3) Developer/power user who knows what he's about. You could simply add -cpu qemu64 for those guests that care about it. 4) embedded virtualization where the use of a management app provides little to no added benefit and everything has to be "automated" (ie., no user). My point is there are other use cases besides data center deployments (aka enterprise) and workstation (casual/power user). There are use cases where virtualization is just yet another tool to achieve a product. If there is no user, then a management app exists. It may be as simple as execing qemu from initrd, but there is still something that is not a user that can set the correct parameters. It is up to the author of this one-line management app to decide what the correct parameters are (-cpu host or some other cpu baseline that allows VM portability, if needed). -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 06:51 AM, Michael S. Tsirkin wrote: > On Mon, Dec 21, 2009 at 06:45:22AM -0700, David S. Ahern wrote: >> >> On 12/21/2009 05:05 AM, Avi Kivity wrote: >>> On 12/21/2009 01:45 PM, Alexander Graf wrote: Well, we have two groups: 1) Casual user w/o management app 2) Enterprise user w/ management app So I clearly belong to the first group. >>> >>> 3) Developer/power user who knows what he's about. >>> >>> You could simply add -cpu qemu64 for those guests that care about it. >>> >> >> 4) embedded virtualization where the use of a management app provides >> little to no added benefit and everything has to be >> "automated" (ie., no user). >> >> My point is there are other use cases besides data center deployments >> (aka enterprise) and workstation (casual/power user). There are use >> cases where virtualization is just yet another tool to achieve a product. >> >> David > > Yes, but unless someone runs qemu directly, default value for any flag > does not matter much. > That's what I was getting at - "direct" invocation of qemu in an automated sense without a libvirt layer. Defaults that do the right thing on the host would be nice, with tweaking from the defaults only if something extra is wanted. David
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 06:45:22AM -0700, David S. Ahern wrote: > > On 12/21/2009 05:05 AM, Avi Kivity wrote: > > On 12/21/2009 01:45 PM, Alexander Graf wrote: > >> > >> Well, we have two groups: > >> > >> 1) Casual user w/o management app > >> 2) Enterprise user w/ management app > >> > >> So I clearly belong to the first group. > >> > > > > 3) Developer/power user who knows what he's about. > > > > You could simply add -cpu qemu64 for those guests that care about it. > > > > 4) embedded virtualization where the use of a management app provides > little to no added benefit and everything has to be > "automated" (ie., no user). > > My point is there are other use cases besides data center deployments > (aka enterprise) and workstation (casual/power user). There are use > cases where virtualization is just yet another tool to achieve a product. > > David Yes, but unless someone runs qemu directly, default value for any flag does not matter much. -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 05:05 AM, Avi Kivity wrote: > On 12/21/2009 01:45 PM, Alexander Graf wrote: >> >> Well, we have two groups: >> >> 1) Casual user w/o management app >> 2) Enterprise user w/ management app >> >> So I clearly belong to the first group. >> > > 3) Developer/power user who knows what he's about. > > You could simply add -cpu qemu64 for those guests that care about it. > 4) embedded virtualization where the use of a management app provides little to no added benefit and everything has to be "automated" (ie., no user). My point is there are other use cases besides data center deployments (aka enterprise) and workstation (casual/power user). There are use cases where virtualization is just yet another tool to achieve a product. David
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Avi Kivity wrote: On 12/20/2009 07:59 PM, Anthony Liguori wrote: Gleb Natapov wrote: Windows is a mystery box, so we can speculate as much as we want about it. If you don't like something just say "it may break Windows" :) Losing activation does sound like an issue too. Downgrading seems more likely to cause problems. Considering running mplayer in a guest. If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation. Migration needs preparation beforehand. You can't take two random hosts with two random qemu flag sets and expect it to work. KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3. It would be insane to emulate sse3 too. It does expose sse3 support (called pni in Linux IIRC). Not sure if qemu masks it, but the information is there. What about using -cpu kvm64? This has SSE3 as well as CMPXCHG16, both are available in all VMX/SVM capable machines (as far as I could find out) and in TCG. This gives a pretty descent base without loosing many relevant performance features. ... This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is. Right. My concern is with casual users upgrading their machine, not enterprise users who should be protected by their tools. Then what about fixing the CPUID bits to those returned by the KVM module the first time the guest is started? Later you would only use those bits (which may have been cropped) for later restarts of the same guest. If you only upgrade your machine, then there should be no problems. Regards, Andre. P.S. What feature bits do we talk about? Maybe we should group them and use aliases for those. SS_S_E3 and SSE4.x come to mind, are any other bits really relevant? (Or do they justify the pain we have when propagating them?) Then we would only need: Athlon64, Core2, Nehalem (and maybe Phenom). -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448 3567 12 to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 02:09:31PM +0200, Avi Kivity wrote: > On 12/21/2009 02:04 PM, Michael S. Tsirkin wrote: >> >> I think so - if this does not happen a lot, it's not a problem to phone home, >> right? >> > > I'm sure it's very annoying when it happens. It could well be less annoying than having your guest be slow every day because qemu masked CPU features that guest needs to be fast. At least with a CPU upgrade, casual user can easily disagnose the problem: the system was changed. If qemu is slow by default, there's no way to find out why without reading the manual. And we don't have one :) > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 02:04 PM, Michael S. Tsirkin wrote: I think so - if this does not happen a lot, it's not a problem to phone home, right? I'm sure it's very annoying when it happens. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 02:04:47PM +0200, Avi Kivity wrote: > On 12/21/2009 01:18 PM, Michael S. Tsirkin wrote: >> >>> No, Windows tries to detect changes in your hardware and assumes that if >>> too many things change, you might be a pirate and requires you to phone >>> their offices to re-authenticate. >>> >> How often does a casual user upgrade his host CPU? >> > > I don't know. Does it matter? I think so - if this does not happen a lot, it's not a problem to phone home, right? > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 01:45 PM, Alexander Graf wrote: Well, we have two groups: 1) Casual user w/o management app 2) Enterprise user w/ management app So I clearly belong to the first group. 3) Developer/power user who knows what he's about. You could simply add -cpu qemu64 for those guests that care about it. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 12:45:26PM +0100, Alexander Graf wrote: > > On 21.12.2009, at 12:38, Michael S. Tsirkin wrote: > > > On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote: > >> > >> On 21.12.2009, at 12:18, Michael S. Tsirkin wrote: > >> > >>> On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote: > On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: > > > >>> Hmm, then, shouldn't either kvm or qemu mask features that we do not > >>> emulate well enough to make windows not crash? > >>> > >> -cpu host does that already, no? > >> > >> Alex > >> > > I expected so, but Avi here seems to say windows will crash if you > > use a new CPU with it ... > > > > No, Windows tries to detect changes in your hardware and assumes that if > > too many things change, you might be a pirate and requires you to phone > their offices to re-authenticate. > >>> > >>> How often does a casual user upgrade his host CPU? > >> > >> I tend to have my VM images on an NFS share and expect them to work > >> properly on all PCs I work on. > >> So I guess the answer is "often"? > >> > >> Alex > > > > Yes but you are not a casual user, are you? > > Well, we have two groups: > > 1) Casual user w/o management app > 2) Enterprise user w/ management app > > So I clearly belong to the first group. No, you are in 3) virtualization hacker without management app :) > > Consider a 64 bit guest to > > see why moving a VM across machines needs some planning. > > -ENOPARSE > > We're still talking about bootup on different machines, not migration / > loadvm. > > Alex Translation: your requirement "work on all PCs I work on" might only be satisfied by specifying the set of PCs you work on. Bootup on different machines has some of the same issues as migration. Consider a 64 bit guest as an example, I think it can not boot on a 32 bit host OS with kvm. I think you can use tcg, but it will be slower. Same thing applies to other CPU features. Many guests reconfigure themselves when you swap harware under them. This makes it easier to move such guests between machines, or change qemu configuration and still reuse guest image. Others might record hardware state on setup (64 bit above is one such example) making such changes more painful. I do think it would be good for us to supply management with utilities that analyse system hardware features relevant to qemu in a portable way. Could be some qemu monitor command, even. Management would be able to decide between using least common denominator and not supporting some hosts. -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 01:18 PM, Michael S. Tsirkin wrote: No, Windows tries to detect changes in your hardware and assumes that if too many things change, you might be a pirate and requires you to phone their offices to re-authenticate. How often does a casual user upgrade his host CPU? I don't know. Does it matter? -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 21.12.2009, at 12:38, Michael S. Tsirkin wrote: > On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote: >> >> On 21.12.2009, at 12:18, Michael S. Tsirkin wrote: >> >>> On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote: On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: > >>> Hmm, then, shouldn't either kvm or qemu mask features that we do not >>> emulate well enough to make windows not crash? >>> >> -cpu host does that already, no? >> >> Alex >> > I expected so, but Avi here seems to say windows will crash if you > use a new CPU with it ... > No, Windows tries to detect changes in your hardware and assumes that if too many things change, you might be a pirate and requires you to phone their offices to re-authenticate. >>> >>> How often does a casual user upgrade his host CPU? >> >> I tend to have my VM images on an NFS share and expect them to work properly >> on all PCs I work on. >> So I guess the answer is "often"? >> >> Alex > > Yes but you are not a casual user, are you? Well, we have two groups: 1) Casual user w/o management app 2) Enterprise user w/ management app So I clearly belong to the first group. > Consider a 64 bit guest to > see why moving a VM across machines needs some planning. -ENOPARSE We're still talking about bootup on different machines, not migration / loadvm. Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 12:22:53PM +0100, Alexander Graf wrote: > > On 21.12.2009, at 12:18, Michael S. Tsirkin wrote: > > > On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote: > >> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: > >>> > > Hmm, then, shouldn't either kvm or qemu mask features that we do not > > emulate well enough to make windows not crash? > > > -cpu host does that already, no? > > Alex > > >>> I expected so, but Avi here seems to say windows will crash if you > >>> use a new CPU with it ... > >>> > >> > >> No, Windows tries to detect changes in your hardware and assumes that if > >> too many things change, you might be a pirate and requires you to phone > >> their offices to re-authenticate. > > > > How often does a casual user upgrade his host CPU? > > I tend to have my VM images on an NFS share and expect them to work properly > on all PCs I work on. > So I guess the answer is "often"? > > Alex Yes but you are not a casual user, are you? Consider a 64 bit guest to see why moving a VM across machines needs some planning. -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 1:12 PM, Avi Kivity wrote: On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: Hmm, then, shouldn't either kvm or qemu mask features that we do not emulate well enough to make windows not crash? -cpu host does that already, no? Alex I expected so, but Avi here seems to say windows will crash if you use a new CPU with it ... No, Windows tries to detect changes in your hardware and assumes that if too many things change, you might be a pirate and requires you to phone their offices to re-authenticate. 'Just' the CPU is not big deal. Might hit you with two 'bad' points. Network interface + CPU will require re-activation. ( http://www.aumha.org/win5/a/wpa.php ) Y.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 21.12.2009, at 12:18, Michael S. Tsirkin wrote: > On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote: >> On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: >>> > Hmm, then, shouldn't either kvm or qemu mask features that we do not > emulate well enough to make windows not crash? > -cpu host does that already, no? Alex >>> I expected so, but Avi here seems to say windows will crash if you >>> use a new CPU with it ... >>> >> >> No, Windows tries to detect changes in your hardware and assumes that if >> too many things change, you might be a pirate and requires you to phone >> their offices to re-authenticate. > > How often does a casual user upgrade his host CPU? I tend to have my VM images on an NFS share and expect them to work properly on all PCs I work on. So I guess the answer is "often"? Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 21, 2009 at 01:12:26PM +0200, Avi Kivity wrote: > On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: >> Hmm, then, shouldn't either kvm or qemu mask features that we do not emulate well enough to make windows not crash? >>> -cpu host does that already, no? >>> >>> Alex >>> >> I expected so, but Avi here seems to say windows will crash if you >> use a new CPU with it ... >> > > No, Windows tries to detect changes in your hardware and assumes that if > too many things change, you might be a pirate and requires you to phone > their offices to re-authenticate. How often does a casual user upgrade his host CPU? > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 07:59 PM, Anthony Liguori wrote: Gleb Natapov wrote: Windows is a mystery box, so we can speculate as much as we want about it. If you don't like something just say "it may break Windows" :) Losing activation does sound like an issue too. Downgrading seems more likely to cause problems. Considering running mplayer in a guest. If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation. Migration needs preparation beforehand. You can't take two random hosts with two random qemu flag sets and expect it to work. KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3. It would be insane to emulate sse3 too. It does expose sse3 support (called pni in Linux IIRC). Not sure if qemu masks it, but the information is there. This is a classic problem with live migration. You can detect this scenario and potentially block the migration, but honestly, how likely is it that any given guest is using an obscure feature? 100% likely. This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is. Right. My concern is with casual users upgrading their machine, not enterprise users who should be protected by their tools. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 07:18 PM, Michael S. Tsirkin wrote: Hmm, then, shouldn't either kvm or qemu mask features that we do not emulate well enough to make windows not crash? -cpu host does that already, no? Alex I expected so, but Avi here seems to say windows will crash if you use a new CPU with it ... No, Windows tries to detect changes in your hardware and assumes that if too many things change, you might be a pirate and requires you to phone their offices to re-authenticate. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/21/2009 09:43 AM, Gleb Natapov wrote: On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote: Gleb Natapov wrote: Windows is a mystery box, so we can speculate as much as we want about it. If you don't like something just say "it may break Windows" :) Losing activation does sound like an issue too. Downgrading seems more likely to cause problems. Considering running mplayer in a guest. If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation. KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3. It would be insane to emulate sse3 too. This is a classic problem with live migration. You can detect this scenario and potentially block the migration, but honestly, how likely is it that any given guest is using an obscure feature? This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is. I am confused. We explicitly not talking about migration, why bring it here? We are talking about casual user who can't care less about migration, but he upgrades his home computer and Windows VM stops working for him. John's new cpu definitions are the exact solution for this issue - all users, whether using mgmt app or direct qemu (this is no user, this is a developer/hacker/other, let's do not optimize this case) should use the various 'real' cpu definitions like -cpu Merom | Nehalem | Penry | Opteron G1, Qemu will check the required cpuid of the cpu model on the host and refuse to load otherwise. When moving to this model, migration can be simplified too since there are fewer combination, and one can choose performance over migration flexibility and wise versa. Due to the above check, the destination qemu won't load if the host does not support its cpu model. -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 07:06:03PM +0100, Alexander Graf wrote: > > On 20.12.2009, at 18:59, Anthony Liguori wrote: > > > Gleb Natapov wrote: > >> Windows is a mystery box, so we can speculate as much as we want about it. > >> If you don't like something just say "it may break Windows" :) Losing > >> activation does sound like an issue too. > >> > > > > Downgrading seems more likely to cause problems. Considering running > > mplayer in a guest. If it detects SSE3 in one host and migrate to another > > host that doesn't have SSE3, you'll be running an instruction stream that > > uses instructions the processor doesn't support resulting in the > > application faulting due to an illegal operation. > > Yeah, migration with -cpu host doesn't sound like a sane thing to do. > > > KVM's cpuid filter doesn't help here because it won't attempt to mask > > things like sse3. It would be insane to emulate sse3 too. > > Well, I wouldn't be too sure on that one. Software may use SSE3 instructions > to access MMIO in which case we do have to emulate it. Unfortunately this is already a reality. Look here: http://groups.google.com/group/linux.kernel/browse_thread/thread/ada9fa2d2574a8af -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote: > Gleb Natapov wrote: > >Windows is a mystery box, so we can speculate as much as we want about it. > >If you don't like something just say "it may break Windows" :) Losing > >activation does sound like an issue too. > > Downgrading seems more likely to cause problems. Considering > running mplayer in a guest. If it detects SSE3 in one host and > migrate to another host that doesn't have SSE3, you'll be running an > instruction stream that uses instructions the processor doesn't > support resulting in the application faulting due to an illegal > operation. > > KVM's cpuid filter doesn't help here because it won't attempt to > mask things like sse3. It would be insane to emulate sse3 too. > > This is a classic problem with live migration. You can detect this > scenario and potentially block the migration, but honestly, how > likely is it that any given guest is using an obscure feature? This > is a classic management tool problem and the solution usually is a > set of heuristics depending on how conservative the target audience > is. > I am confused. We explicitly not talking about migration, why bring it here? We are talking about casual user who can't care less about migration, but he upgrades his home computer and Windows VM stops working for him. -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 11:59:43AM -0600, Anthony Liguori wrote: > Gleb Natapov wrote: >> Windows is a mystery box, so we can speculate as much as we want about it. >> If you don't like something just say "it may break Windows" :) Losing >> activation does sound like an issue too. >> > > Downgrading seems more likely to cause problems. Considering running > mplayer in a guest. If it detects SSE3 in one host and migrate to > another host that doesn't have SSE3, you'll be running an instruction > stream that uses instructions the processor doesn't support resulting in > the application faulting due to an illegal operation. > > KVM's cpuid filter doesn't help here because it won't attempt to mask > things like sse3. It would be insane to emulate sse3 too. > > This is a classic problem with live migration. You can detect this > scenario and potentially block the migration, but honestly, how likely > is it that any given guest is using an obscure feature? This is a > classic management tool problem and the solution usually is a set of > heuristics depending on how conservative the target audience is. Yes, but I suspect it is we that need to implement these heuristics, so that management tools only have a small set of knobs to turn. Duplicating knowledge of CPU features in management tools does not make much sense to me. Does not have to be linked into qemu though, we could have a library or a set of utilities for management to use. > Regards, > > Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 20.12.2009, at 18:59, Anthony Liguori wrote: > Gleb Natapov wrote: >> Windows is a mystery box, so we can speculate as much as we want about it. >> If you don't like something just say "it may break Windows" :) Losing >> activation does sound like an issue too. >> > > Downgrading seems more likely to cause problems. Considering running mplayer > in a guest. If it detects SSE3 in one host and migrate to another host that > doesn't have SSE3, you'll be running an instruction stream that uses > instructions the processor doesn't support resulting in the application > faulting due to an illegal operation. Yeah, migration with -cpu host doesn't sound like a sane thing to do. > KVM's cpuid filter doesn't help here because it won't attempt to mask things > like sse3. It would be insane to emulate sse3 too. Well, I wouldn't be too sure on that one. Software may use SSE3 instructions to access MMIO in which case we do have to emulate it. > This is a classic problem with live migration. You can detect this scenario > and potentially block the migration, but honestly, how likely is it that any > given guest is using an obscure feature? This is a classic management tool > problem and the solution usually is a set of heuristics depending on how > conservative the target audience is. I agree. I think the discussion was about setting -cpu host to the default? I was merely wondering what'd keep us from doing that, as management software would just -cpu qemu64 and be good. Btw: I'd actually prefer if it'd be called -cpu safe or so. That way we could differenciate between "tcg could run the code" and "doesn't break migration". The former one is -cpu qemu64 and coincidently it matches with the latter in most cases, but I'm not sure it'd keep that way. Think of default enablement of nested SVM... Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Gleb Natapov wrote: Windows is a mystery box, so we can speculate as much as we want about it. If you don't like something just say "it may break Windows" :) Losing activation does sound like an issue too. Downgrading seems more likely to cause problems. Considering running mplayer in a guest. If it detects SSE3 in one host and migrate to another host that doesn't have SSE3, you'll be running an instruction stream that uses instructions the processor doesn't support resulting in the application faulting due to an illegal operation. KVM's cpuid filter doesn't help here because it won't attempt to mask things like sse3. It would be insane to emulate sse3 too. This is a classic problem with live migration. You can detect this scenario and potentially block the migration, but honestly, how likely is it that any given guest is using an obscure feature? This is a classic management tool problem and the solution usually is a set of heuristics depending on how conservative the target audience is. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 06:29:11PM +0100, Alexander Graf wrote: > > On 20.12.2009, at 18:23, Gleb Natapov wrote: > > > On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote: > >> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote: > >>> > >>> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: > >>> > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: > > On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: > >> > >>> Maybe we should make -cpu host the default. That will give the best > >>> performance for casual users, more testing for newer features, and > >>> will > >>> force management apps to treat migration much more seriously. The > >>> downside is that casual users upgrading their machines might > >>> experience > >>> issues with Windows. Feature compatibility is not just about > >>> migration. > >>> > >> This seems very aggressive. Can't we whitelist features that we know > >> about? Further, doesn't KVM already do this? > >> > >> > > > > It does, but without -cpuid host you're stuck with qemu64 (kvm.ko > > doesn't add features userspace didn't request). > > Hmm, then, shouldn't either kvm or qemu mask features that we do not > emulate well enough to make windows not crash? > >>> > >>> -cpu host does that already, no? > >>> > >>> Alex > >> > >> I expected so, but Avi here seems to say windows will crash if you > >> use a new CPU with it ... > >> > > Windows _may_ crash if you'll _upgrade_ your _host_ CPU. > > Uh. It may lose activation I guess. But apart from that I can't see how it'd > break as long as you don't expect loadvm to work. > > Anybody mind to go into a bit more detail here? :-) > Windows is a mystery box, so we can speculate as much as we want about it. If you don't like something just say "it may break Windows" :) Losing activation does sound like an issue too. -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 20.12.2009, at 18:23, Gleb Natapov wrote: > On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote: >> On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote: >>> >>> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: >>> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: > On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: >> >>> Maybe we should make -cpu host the default. That will give the best >>> performance for casual users, more testing for newer features, and will >>> force management apps to treat migration much more seriously. The >>> downside is that casual users upgrading their machines might experience >>> issues with Windows. Feature compatibility is not just about migration. >>> >> This seems very aggressive. Can't we whitelist features that we know >> about? Further, doesn't KVM already do this? >> >> > > It does, but without -cpuid host you're stuck with qemu64 (kvm.ko > doesn't add features userspace didn't request). Hmm, then, shouldn't either kvm or qemu mask features that we do not emulate well enough to make windows not crash? >>> >>> -cpu host does that already, no? >>> >>> Alex >> >> I expected so, but Avi here seems to say windows will crash if you >> use a new CPU with it ... >> > Windows _may_ crash if you'll _upgrade_ your _host_ CPU. Uh. It may lose activation I guess. But apart from that I can't see how it'd break as long as you don't expect loadvm to work. Anybody mind to go into a bit more detail here? :-) Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 07:18:22PM +0200, Michael S. Tsirkin wrote: > On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote: > > > > On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: > > > > > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: > > >> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: > > >>> > > Maybe we should make -cpu host the default. That will give the best > > performance for casual users, more testing for newer features, and will > > force management apps to treat migration much more seriously. The > > downside is that casual users upgrading their machines might experience > > issues with Windows. Feature compatibility is not just about > > migration. > > > > >>> This seems very aggressive. Can't we whitelist features that we know > > >>> about? Further, doesn't KVM already do this? > > >>> > > >>> > > >> > > >> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko > > >> doesn't add features userspace didn't request). > > > > > > Hmm, then, shouldn't either kvm or qemu mask features that we do not > > > emulate well enough to make windows not crash? > > > > -cpu host does that already, no? > > > > Alex > > I expected so, but Avi here seems to say windows will crash if you > use a new CPU with it ... > Windows _may_ crash if you'll _upgrade_ your _host_ CPU. -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 20.12.2009, at 18:18, Michael S. Tsirkin wrote: > On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote: >> >> On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: >> >>> On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: > >> Maybe we should make -cpu host the default. That will give the best >> performance for casual users, more testing for newer features, and will >> force management apps to treat migration much more seriously. The >> downside is that casual users upgrading their machines might experience >> issues with Windows. Feature compatibility is not just about migration. >> > This seems very aggressive. Can't we whitelist features that we know > about? Further, doesn't KVM already do this? > > It does, but without -cpuid host you're stuck with qemu64 (kvm.ko doesn't add features userspace didn't request). >>> >>> Hmm, then, shouldn't either kvm or qemu mask features that we do not >>> emulate well enough to make windows not crash? >> >> -cpu host does that already, no? >> >> Alex > > I expected so, but Avi here seems to say windows will crash if you > use a new CPU with it ... Avi, have you narrowed it down to a specific feature bit? Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 06:17:02PM +0100, Alexander Graf wrote: > > On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: > > > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: > >> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: > >>> > Maybe we should make -cpu host the default. That will give the best > performance for casual users, more testing for newer features, and will > force management apps to treat migration much more seriously. The > downside is that casual users upgrading their machines might experience > issues with Windows. Feature compatibility is not just about migration. > > >>> This seems very aggressive. Can't we whitelist features that we know > >>> about? Further, doesn't KVM already do this? > >>> > >>> > >> > >> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko > >> doesn't add features userspace didn't request). > > > > Hmm, then, shouldn't either kvm or qemu mask features that we do not > > emulate well enough to make windows not crash? > > -cpu host does that already, no? > > Alex I expected so, but Avi here seems to say windows will crash if you use a new CPU with it ... -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 20.12.2009, at 17:56, Michael S. Tsirkin wrote: > On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: >> On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: >>> Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. >>> This seems very aggressive. Can't we whitelist features that we know >>> about? Further, doesn't KVM already do this? >>> >>> >> >> It does, but without -cpuid host you're stuck with qemu64 (kvm.ko >> doesn't add features userspace didn't request). > > Hmm, then, shouldn't either kvm or qemu mask features that we do not > emulate well enough to make windows not crash? -cpu host does that already, no? Alex
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 05:59:33PM +0200, Avi Kivity wrote: > On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: >> >>> Maybe we should make -cpu host the default. That will give the best >>> performance for casual users, more testing for newer features, and will >>> force management apps to treat migration much more seriously. The >>> downside is that casual users upgrading their machines might experience >>> issues with Windows. Feature compatibility is not just about migration. >>> >> This seems very aggressive. Can't we whitelist features that we know >> about? Further, doesn't KVM already do this? >> >> > > It does, but without -cpuid host you're stuck with qemu64 (kvm.ko > doesn't add features userspace didn't request). Hmm, then, shouldn't either kvm or qemu mask features that we do not emulate well enough to make windows not crash? > -- > error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 05:51 PM, Michael S. Tsirkin wrote: Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. This seems very aggressive. Can't we whitelist features that we know about? Further, doesn't KVM already do this? It does, but without -cpuid host you're stuck with qemu64 (kvm.ko doesn't add features userspace didn't request). -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 11:49:40AM +0200, Avi Kivity wrote: > On 12/14/2009 10:18 PM, Anthony Liguori wrote: >> Michael S. Tsirkin wrote: >>> This might help 32 bit guests, but not guests with 64 bit >>> kernel and 32 bit userspace (my case) because all 64 bit >>> CPUs advertise syscall bit in cpuid. Thus 64 bit guests >>> do not seem to even bother checking this bit: >>> AMD + 64 bit -> syscall. >> >> Okay, I don't see a great option other than migrating the vendor_id >> string. >> > > That's not strictly necessary since the guest cannot change the vendor > string; all the user has to do is to launch both guests with explicit > vendor ids. Of course that imposes more on the user (or the management > application). > >> Otherwise, cross vendor migration will not work by default. Maybe >> that's not a problem but if so, we really should change the default >> cpu model to be much more aggressive about exposing host features. > > Maybe we should make -cpu host the default. That will give the best > performance for casual users, more testing for newer features, and will > force management apps to treat migration much more seriously. The > downside is that casual users upgrading their machines might experience > issues with Windows. Feature compatibility is not just about migration. This seems very aggressive. Can't we whitelist features that we know about? Further, doesn't KVM already do this? -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 05:49 PM, Michael S. Tsirkin wrote: That needs a config file anyway to let the host qemu know which machine level (e.g. -M pc-0.11) to use. Hmm, yes, but will qemu ship config files which configure host cpu as well? If it doesn't, the management app will have to to this itself. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 05:40:53PM +0200, Avi Kivity wrote: > On 12/20/2009 05:38 PM, Gleb Natapov wrote: >> >>> I was thinking about upgrading their host cpu. I doubt you'd live >>> migrate without a management app. >>> >>> >> And what about VM on disk-on-key or VM image on NFS that can be started >> from different locations? >> > > That needs a config file anyway to let the host qemu know which machine > level (e.g. -M pc-0.11) to use. Hmm, yes, but will qemu ship config files which configure host cpu as well? -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 05:38 PM, Gleb Natapov wrote: I was thinking about upgrading their host cpu. I doubt you'd live migrate without a management app. And what about VM on disk-on-key or VM image on NFS that can be started from different locations? That needs a config file anyway to let the host qemu know which machine level (e.g. -M pc-0.11) to use. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Sun, Dec 20, 2009 at 05:36:57PM +0200, Avi Kivity wrote: > On 12/20/2009 05:33 PM, Anthony Liguori wrote: > >>- casual (non-management-app-using) users will start seeing > >>problems with Windows guests unless they change their command > >>lines > > > > > >Assuming their migrating across different CPU types. > > > > I was thinking about upgrading their host cpu. I doubt you'd live > migrate without a management app. > And what about VM on disk-on-key or VM image on NFS that can be started from different locations? -- Gleb.
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 05:33 PM, Anthony Liguori wrote: - casual (non-management-app-using) users will start seeing problems with Windows guests unless they change their command lines Assuming their migrating across different CPU types. I was thinking about upgrading their host cpu. I doubt you'd live migrate without a management app. - management apps really have to take cpuid into account now (previously, it was just a good idea) Yes. I think we probably should have a management app writers guide that spells out all of the quirks and best practices that we often discuss here and assume that management app writers are going to follow. Definitely. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Avi Kivity wrote: On 12/20/2009 04:48 PM, Anthony Liguori wrote: Avi Kivity wrote: Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. Yes, I'd much rather do this than mucking with vendor_id. If we're going to give up on cross vendor migration by default, I think we should go for best performance. If we do this, we need to advertise it clearly: - casual (non-management-app-using) users will start seeing problems with Windows guests unless they change their command lines Assuming their migrating across different CPU types. - management apps really have to take cpuid into account now (previously, it was just a good idea) Yes. I think we probably should have a management app writers guide that spells out all of the quirks and best practices that we often discuss here and assume that management app writers are going to follow. Sounds like a good thing to live in the proposes docs directory :-) Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/20/2009 04:48 PM, Anthony Liguori wrote: Avi Kivity wrote: Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. Yes, I'd much rather do this than mucking with vendor_id. If we're going to give up on cross vendor migration by default, I think we should go for best performance. If we do this, we need to advertise it clearly: - casual (non-management-app-using) users will start seeing problems with Windows guests unless they change their command lines - management apps really have to take cpuid into account now (previously, it was just a good idea) -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Avi Kivity wrote: Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. Yes, I'd much rather do this than mucking with vendor_id. If we're going to give up on cross vendor migration by default, I think we should go for best performance. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/14/2009 10:18 PM, Anthony Liguori wrote: Michael S. Tsirkin wrote: This might help 32 bit guests, but not guests with 64 bit kernel and 32 bit userspace (my case) because all 64 bit CPUs advertise syscall bit in cpuid. Thus 64 bit guests do not seem to even bother checking this bit: AMD + 64 bit -> syscall. Okay, I don't see a great option other than migrating the vendor_id string. That's not strictly necessary since the guest cannot change the vendor string; all the user has to do is to launch both guests with explicit vendor ids. Of course that imposes more on the user (or the management application). Otherwise, cross vendor migration will not work by default. Maybe that's not a problem but if so, we really should change the default cpu model to be much more aggressive about exposing host features. Maybe we should make -cpu host the default. That will give the best performance for casual users, more testing for newer features, and will force management apps to treat migration much more seriously. The downside is that casual users upgrading their machines might experience issues with Windows. Feature compatibility is not just about migration. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On 12/15/2009 07:56 PM, Michael S. Tsirkin wrote: I see. But unfortunately this bit has multiple meanings for 64/32 bit, kvm does not know whether you will run a 32 bit or a 64 bit program. This is a cpu architecture bug. Correct. One bit is used for two separate features (syscall-32 and syscall-64). syscall-32 rarely matters because i386 guests use sysenter. It's only 32-on-64 guests that attempt to use syscall. -- error compiling committee.c: too many arguments to function
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Anthony Liguori wrote: Gleb Natapov wrote: I thought KVM emulates the syscall instruction? I swear I've seen patches for that. It is. Starting from 2.6.32. Okay, so this is a performance vs. migration compatibility issue then? BTW, couldn't we just not advertise syscall in cpuid? Looks like a good idea, but at least Linux does not care ;-) In 64-on-64 bit mode syscall is unconditionally used, which is OK, since it is architectural to AMD64. Intel integrated this, too. In 32bit legacy mode AMD supports both syscall and sysenter, but mostly sysenter is used when the SEP CPUID bit is on. Intel does not support syscall, but this also is not advertised when CPUID is executed in 32bit. In 32-on-64 compat mode the support is exclusive: AMD does not support sysenter, Intel does not support syscall. Since the AMD CPUID advertising is (at least) a bit "unfortunate" here ;-), Linux does not care about the bits, but unconditionally uses syscall on AMD and sysenter on Intel. See arch/x86/kernel/cpu/intel.c:early_init_intel(): #ifdef CONFIG_X86_64 set_cpu_cap(c, X86_FEATURE_SYSENTER32); #else A similiar line is in cpu/amd.c:early_init_amd(). Then in arch/x86/vdso/vdso32-setup.c:sysenter_setup() these bit are used to populate the VDSO page. In the whole process CPUID feature bits are not honored, so you cannot get around it. The only solution would be to inject neither AMD or Intel (I used VirtualLinux) in the past. This works fine for Linux (even better than the other two, since no vendor-specific workarounds are triggered), but 64bit Windows refuses to boot on anything other than AuthenticAMD, GenuineIntel or CentaurHauls. Linux would then use int80h, which is slower than syscall or sysenter, but still much faster than emulation. So given the whole mess I'd recommend to copy over Avi's "propagate local vendor" solution, as this solves the problem for 99.9+% of the cases. The remaining cases are cross-vendor migration, where we are fine with the 2.6.32 syscall/sysenter emulation. Although this is not as slow as one thinks (I measured less than 5% slowdown in kernbench), I'd not recommend it for common usage except cross-vendor migration. BTW: Windows is not affected at all with this compat mode issue, because 32bit applications switch to 64bit mode before doing the actual system call (which uses the 64bit syscall instruction). That should fix it too without sacrificing migration compatibility. We get a slight slowdown on AMD hosts I think there would be no slowdown. At least Linux only cares about this bit if in legacy 32bit mode, but here it uses sysenter anyway if this is present: From arch/x86/vdso/vdso32-setup.c: #else /* CONFIG_X86_32 */ #define vdso32_sysenter() (boot_cpu_has(X86_FEATURE_SEP)) #define vdso32_syscall()(0) but that's probably minor compared to the cost of using emulated syscall on Intel hosts. Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448 3567 12 to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Tue, Dec 15, 2009 at 11:37:34AM -0600, Anthony Liguori wrote: > Michael S. Tsirkin wrote: >> I don't think it does: cpuid is an unpriveledged operation, >> is it not? >> > > We (qemu) ask the kernel (kvm.ko) to filter out the features bits from > the cpuid we expose to the guest in order to remove feature bits it > (kvm.ko) does not support. > > Regards, > > Anthony Liguori I see. But unfortunately this bit has multiple meanings for 64/32 bit, kvm does not know whether you will run a 32 bit or a 64 bit program. This is a cpu architecture bug. -- MST
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Michael S. Tsirkin wrote: I don't think it does: cpuid is an unpriveledged operation, is it not? We (qemu) ask the kernel (kvm.ko) to filter out the features bits from the cpuid we expose to the guest in order to remove feature bits it (kvm.ko) does not support. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 14, 2009 at 03:49:39PM -0600, Anthony Liguori wrote: > Michael S. Tsirkin wrote: >> On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote: >> >>> Michael S. Tsirkin wrote: >>> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote: > Michael S. Tsirkin wrote: > >> This might help 32 bit guests, but not guests with 64 bit >> kernel and 32 bit userspace (my case) because all 64 bit >> CPUs advertise syscall bit in cpuid. Thus 64 bit guests >> do not seem to even bother checking this bit: >> AMD + 64 bit -> syscall. >> > Okay, I don't see a great option other than migrating the vendor_id > string. > This won't help with kernels <2.6.32 though. I guess we can switch default vendor to Intel, this likely has some other side effects. >>> That's a kernel bug. If we think it effects a lot of users, we >>> should introduce a CAP such that we can detect this in userspace and >>> fail gracefully. >>> >> >> Not emulating feature host CPU does not have is a kernel bug? >> Okay ... >> Yes, almost no one runs 2.6.32 yet. >> > > The kernel has the ability to filter feature bits from cpuid. I don't think it does: cpuid is an unpriveledged operation, is it not? > We assume > it's going to filter out things it doesn't support. It's tricky: this bit has multiple meanings. This is a cpu architecture bug, not a kernel bug. > Regards, > > Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Michael S. Tsirkin wrote: On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote: Michael S. Tsirkin wrote: On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote: Michael S. Tsirkin wrote: This might help 32 bit guests, but not guests with 64 bit kernel and 32 bit userspace (my case) because all 64 bit CPUs advertise syscall bit in cpuid. Thus 64 bit guests do not seem to even bother checking this bit: AMD + 64 bit -> syscall. Okay, I don't see a great option other than migrating the vendor_id string. This won't help with kernels <2.6.32 though. I guess we can switch default vendor to Intel, this likely has some other side effects. That's a kernel bug. If we think it effects a lot of users, we should introduce a CAP such that we can detect this in userspace and fail gracefully. Not emulating feature host CPU does not have is a kernel bug? Okay ... Yes, almost no one runs 2.6.32 yet. The kernel has the ability to filter feature bits from cpuid. We assume it's going to filter out things it doesn't support. Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 14, 2009 at 02:54:49PM -0600, Anthony Liguori wrote: > Michael S. Tsirkin wrote: >> On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote: >> >>> Michael S. Tsirkin wrote: >>> This might help 32 bit guests, but not guests with 64 bit kernel and 32 bit userspace (my case) because all 64 bit CPUs advertise syscall bit in cpuid. Thus 64 bit guests do not seem to even bother checking this bit: AMD + 64 bit -> syscall. >>> Okay, I don't see a great option other than migrating the vendor_id string. >>> >> >> This won't help with kernels <2.6.32 though. I guess we can switch >> default vendor to Intel, this likely has some other side effects. >> > > That's a kernel bug. If we think it effects a lot of users, we should > introduce a CAP such that we can detect this in userspace and fail > gracefully. Not emulating feature host CPU does not have is a kernel bug? Okay ... Yes, almost no one runs 2.6.32 yet. >>> Otherwise, cross vendor migration will not work by default. Maybe >>> that's not a problem but if so, we really should change the default >>> cpu model to be much more aggressive about exposing host features. >>> >> It's a tradeoff, but yes. We also need more sanity checks and >> management commands giving management tools to understand whether >> emulating guest CPU X on host CPU Y is safe/possible, and which guests >> this might break. >> > > I'd like to see Avi weigh in as historically, he's been one of the > strongest advocates of a default migration policy of maximum > compatibility. Personally, I think cross vendor migration is extremely > uncommon in production and I would strongly recommend against it. > > My own experience has been that hardware homogeneity is pretty common in > deployments, particularly in the processor space. There's such a > different between Nehalem and non-Nehalem class processors that you > really can't run the same workloads across the two without seeing a > significant impact. > > So I'd actually be in favor of changing the default cpu to something > that exposes a lot more of the underlying processor features. I think > increased performance would be better for most consumers vs. increased > migration flexibility. > > Then again, I'm certainly bias based on being employed by a hardware > vendor :-) > > Regards, > > Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
Michael S. Tsirkin wrote: On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote: Michael S. Tsirkin wrote: This might help 32 bit guests, but not guests with 64 bit kernel and 32 bit userspace (my case) because all 64 bit CPUs advertise syscall bit in cpuid. Thus 64 bit guests do not seem to even bother checking this bit: AMD + 64 bit -> syscall. Okay, I don't see a great option other than migrating the vendor_id string. This won't help with kernels <2.6.32 though. I guess we can switch default vendor to Intel, this likely has some other side effects. That's a kernel bug. If we think it effects a lot of users, we should introduce a CAP such that we can detect this in userspace and fail gracefully. Otherwise, cross vendor migration will not work by default. Maybe that's not a problem but if so, we really should change the default cpu model to be much more aggressive about exposing host features. It's a tradeoff, but yes. We also need more sanity checks and management commands giving management tools to understand whether emulating guest CPU X on host CPU Y is safe/possible, and which guests this might break. I'd like to see Avi weigh in as historically, he's been one of the strongest advocates of a default migration policy of maximum compatibility. Personally, I think cross vendor migration is extremely uncommon in production and I would strongly recommend against it. My own experience has been that hardware homogeneity is pretty common in deployments, particularly in the processor space. There's such a different between Nehalem and non-Nehalem class processors that you really can't run the same workloads across the two without seeing a significant impact. So I'd actually be in favor of changing the default cpu to something that exposes a lot more of the underlying processor features. I think increased performance would be better for most consumers vs. increased migration flexibility. Then again, I'm certainly bias based on being employed by a hardware vendor :-) Regards, Anthony Liguori
Re: [Qemu-devel] cpuid problem in upstream qemu with kvm
On Mon, Dec 14, 2009 at 02:18:33PM -0600, Anthony Liguori wrote: > Michael S. Tsirkin wrote: >> This might help 32 bit guests, but not guests with 64 bit >> kernel and 32 bit userspace (my case) because all 64 bit >> CPUs advertise syscall bit in cpuid. Thus 64 bit guests >> do not seem to even bother checking this bit: >> AMD + 64 bit -> syscall. >> > > Okay, I don't see a great option other than migrating the vendor_id string. This won't help with kernels <2.6.32 though. I guess we can switch default vendor to Intel, this likely has some other side effects. > Otherwise, cross vendor migration will not work by default. Maybe > that's not a problem but if so, we really should change the default cpu > model to be much more aggressive about exposing host features. > > Regards, > > Anthony Liguori It's a tradeoff, but yes. We also need more sanity checks and management commands giving management tools to understand whether emulating guest CPU X on host CPU Y is safe/possible, and which guests this might break. -- MST
