Re: KVM on IBM PowerEN processor
Hi Kun, On 08/29/2011 11:31 AM, Kun Wang wrote: Hi, everyone, This is Kun Wang from IBM Research China. I and my team have been working on IBM PowerEN processor in recent years, including its simulation, lib/runtime optimization and etc. Now we start the work to enable KVM on PowerEN processor. Since the A2 core of PowerEN follows Power ISA v2.06 (more specifically, book3e and 64-bit), I believe 99% of our work will stick to the ISA, and hence can be leveraged by others. As the new one to this KVM world, I and my team definitely need your help. Looking forward to talking and working with you guys in the future. Welcome to the PowerPC KVM world! I'm looking very much forward to working with you there. Please always CC k...@vger.kernel.org for emails you send to kvm-ppc@vger, unless you think of it as completely off-generic-kvm discussions. I like the idea of having more clever people involved in the PPC KVM process and by CC'ing kvm@vger, we get more exposure. The Freescale e500 cores are basically BookE 2.06 compliant with a few specialties here and there. Have you looked at their code? Sure, it's 32-bit as is now but there is definitely work going on at Freescale to get it 64-bit too. Maybe it makes sense to take the current code as a starting point and slowly move towards an architecture that is more generic. Maybe it makes sense to start from scratch and actually design something flexible that splits guest and host TLB code, so we could potentially run cross-book KVM in the future (which I would love to see happen! Just imagine running a PowerEN guest on a POWER7 system). Either way, please make sure to coordinate any efforts with me and Scott so we don't walk off in different directions and only get to realize we went down the wrong path when there's already a 100 patches patch set on the mailing list :) Again, welcome on board! Alex -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Avoid soft lockup message when KVM is stopped by host
* Marcelo Tosatti mtosa...@redhat.com [2011-08-30 07:35]: On Mon, Aug 29, 2011 at 05:27:11PM -0600, Eric B Munson wrote: Currently, when qemu stops a guest kernel that guest will issue a soft lockup message when it resumes. This set provides the ability for qemu to comminucate to the guest that it has been stopped. When the guest hits the watchdog on resume it will check if it was suspended before issuing the warning. Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/include/asm/pvclock.h |2 ++ arch/x86/kernel/kvmclock.c | 14 ++ include/asm-generic/pvclock.h | 14 ++ kernel/watchdog.c | 12 5 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/pvclock.h -- 1.7.4.1 How is the host supposed to set this flag? As mentioned previously, if you save save/restore the offset added to kvmclock on stop/cont (and the TSC MSR, forgot to mention that), no paravirt infrastructure is required. Which means the issue is also fixed for older guests. How is saving the offset going to prevent a large jump from triggering the softlock message? Won't we still have not touched the watchdog for that long period of time? -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM on IBM PowerEN processor
On Tue, Aug 30, 2011 at 4:45 AM, Alexander Graf ag...@suse.de wrote: Hi Kun, On 08/29/2011 11:31 AM, Kun Wang wrote: Hi, everyone, This is Kun Wang from IBM Research China. I and my team have been working on IBM PowerEN processor in recent years, including its simulation, lib/runtime optimization and etc. Now we start the work to enable KVM on PowerEN processor. Since the A2 core of PowerEN follows Power ISA v2.06 (more specifically, book3e and 64-bit), I believe 99% of our work will stick to the ISA, and hence can be leveraged by others. As the new one to this KVM world, I and my team definitely need your help. Looking forward to talking and working with you guys in the future. Welcome to the PowerPC KVM world! I'm looking very much forward to working with you there. Please always CC k...@vger.kernel.org for emails you send to kvm-ppc@vger, unless you think of it as completely off-generic-kvm discussions. I like the idea of having more clever people involved in the PPC KVM process and by CC'ing kvm@vger, we get more exposure. The Freescale e500 cores are basically BookE 2.06 compliant with a few specialties here and there. Have you looked at their code? Sure, it's 32-bit as is now but there is definitely work going on at Freescale to get it 64-bit too. Maybe it makes sense to take the current code as a starting point and slowly move towards an architecture that is more generic. Maybe it makes sense to start from scratch and actually design something flexible that splits guest and host TLB code, so we could potentially run cross-book KVM in the future (which I would love to see happen! Just imagine running a PowerEN guest on a POWER7 system). Either way, please make sure to coordinate any efforts with me and Scott so we don't walk off in different directions and only get to realize we went down the wrong path when there's already a 100 patches patch set on the mailing list :) Also, note-- we (Freescale) do have a patchset coming soon for the e500mc which is a 2.06 embedded 32-bit CPU with category E.HV.We've had this running for a while and are close to finishing some cleanup before posting upstream. We are are also starting to look at 64-bit support for our e5500 CPU we should be a delta on top of the e500mc support. Just want to make you aware of some current work that may overlap your A2 KVM support... Stuart -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Avoid soft lockup message when KVM is stopped by host
On Tue, Aug 30, 2011 at 09:12:17AM -0500, Ryan Harper wrote: * Marcelo Tosatti mtosa...@redhat.com [2011-08-30 07:35]: On Mon, Aug 29, 2011 at 05:27:11PM -0600, Eric B Munson wrote: Currently, when qemu stops a guest kernel that guest will issue a soft lockup message when it resumes. This set provides the ability for qemu to comminucate to the guest that it has been stopped. When the guest hits the watchdog on resume it will check if it was suspended before issuing the warning. Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/include/asm/pvclock.h |2 ++ arch/x86/kernel/kvmclock.c | 14 ++ include/asm-generic/pvclock.h | 14 ++ kernel/watchdog.c | 12 5 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/pvclock.h -- 1.7.4.1 How is the host supposed to set this flag? As mentioned previously, if you save save/restore the offset added to kvmclock on stop/cont (and the TSC MSR, forgot to mention that), no paravirt infrastructure is required. Which means the issue is also fixed for older guests. How is saving the offset going to prevent a large jump from triggering the softlock message? Won't we still have not touched the watchdog for that long period of time? The idea is to adjust the offset and the TSC value so that the guest does not notice the actual elapsed time. This is what happens when a guest is migrated or saved/restore, for example. -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Avoid soft lockup message when KVM is stopped by host
* Marcelo Tosatti mtosa...@redhat.com [2011-08-30 09:40]: On Tue, Aug 30, 2011 at 09:12:17AM -0500, Ryan Harper wrote: * Marcelo Tosatti mtosa...@redhat.com [2011-08-30 07:35]: On Mon, Aug 29, 2011 at 05:27:11PM -0600, Eric B Munson wrote: Currently, when qemu stops a guest kernel that guest will issue a soft lockup message when it resumes. This set provides the ability for qemu to comminucate to the guest that it has been stopped. When the guest hits the watchdog on resume it will check if it was suspended before issuing the warning. Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/include/asm/pvclock.h |2 ++ arch/x86/kernel/kvmclock.c | 14 ++ include/asm-generic/pvclock.h | 14 ++ kernel/watchdog.c | 12 5 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/pvclock.h -- 1.7.4.1 How is the host supposed to set this flag? As mentioned previously, if you save save/restore the offset added to kvmclock on stop/cont (and the TSC MSR, forgot to mention that), no paravirt infrastructure is required. Which means the issue is also fixed for older guests. How is saving the offset going to prevent a large jump from triggering the softlock message? Won't we still have not touched the watchdog for that long period of time? The idea is to adjust the offset and the TSC value so that the guest does not notice the actual elapsed time. This is what happens when a guest is migrated or saved/restore, for example. but that's within a rather short period of time... if we stop the guest for a day, I don't think what you are suggesting can happen without time being wrong in the guest, and as soon as it's corrected and we jump forward larger than the watchdog timeout, won't we trigger it again? -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM on IBM PowerEN processor
于 2011-8-30 22:15, Stuart Yoder 写道: On Tue, Aug 30, 2011 at 4:45 AM, Alexander Grafag...@suse.de wrote: Hi Kun, On 08/29/2011 11:31 AM, Kun Wang wrote: Hi, everyone, This is Kun Wang from IBM Research China. I and my team have been working on IBM PowerEN processor in recent years, including its simulation, lib/runtime optimization and etc. Now we start the work to enable KVM on PowerEN processor. Since the A2 core of PowerEN follows Power ISA v2.06 (more specifically, book3e and 64-bit), I believe 99% of our work will stick to the ISA, and hence can be leveraged by others. As the new one to this KVM world, I and my team definitely need your help. Looking forward to talking and working with you guys in the future. Welcome to the PowerPC KVM world! I'm looking very much forward to working with you there. Please always CC k...@vger.kernel.org for emails you send to kvm-ppc@vger, unless you think of it as completely off-generic-kvm discussions. I like the idea of having more clever people involved in the PPC KVM process and by CC'ing kvm@vger, we get more exposure. The Freescale e500 cores are basically BookE 2.06 compliant with a few specialties here and there. Have you looked at their code? Sure, it's 32-bit as is now but there is definitely work going on at Freescale to get it 64-bit too. Maybe it makes sense to take the current code as a starting point and slowly move towards an architecture that is more generic. Maybe it makes sense to start from scratch and actually design something flexible that splits guest and host TLB code, so we could potentially run cross-book KVM in the future (which I would love to see happen! Just imagine running a PowerEN guest on a POWER7 system). Either way, please make sure to coordinate any efforts with me and Scott so we don't walk off in different directions and only get to realize we went down the wrong path when there's already a 100 patches patch set on the mailing list :) Also, note-- we (Freescale) do have a patchset coming soon for the e500mc which is a 2.06 embedded 32-bit CPU with category E.HV.We've had this running for a while and are close to finishing some cleanup before posting upstream. We are are also starting to look at 64-bit support for our e5500 CPU we should be a delta on top of the e500mc support. Just want to make you aware of some current work that may overlap your A2 KVM support... Stuart -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Stuart, Thank you very much for your information. That's really helpful to us. I do think there is some overlap between your work and ours. Do you have any suggestions for us to avoid re-inventing the wheel? Best regards, Kun Wang -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Avoid soft lockup message when KVM is stopped by host
On Tue, Aug 30, 2011 at 06:26, Marcelo Tosatti mtosa...@redhat.com wrote: On Mon, Aug 29, 2011 at 05:27:11PM -0600, Eric B Munson wrote: Currently, when qemu stops a guest kernel that guest will issue a soft lockup message when it resumes. This set provides the ability for qemu to comminucate to the guest that it has been stopped. When the guest hits the watchdog on resume it will check if it was suspended before issuing the warning. Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector arch/x86/include/asm/pvclock-abi.h | 1 + arch/x86/include/asm/pvclock.h | 2 ++ arch/x86/kernel/kvmclock.c | 14 ++ include/asm-generic/pvclock.h | 14 ++ kernel/watchdog.c | 12 5 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/pvclock.h -- 1.7.4.1 How is the host supposed to set this flag? The hypervisor will set the flag (the second bit in the struct pvclock_vcpu_time_info.flags field) when it stops the guest. I could be reading pvclock-abi.h incorrectly, but the way I understand it is that this structure is used to communicate between hypervisor and guest. As mentioned previously, if you save save/restore the offset added to kvmclock on stop/cont (and the TSC MSR, forgot to mention that), no paravirt infrastructure is required. Which means the issue is also fixed for older guests. -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: KVM on IBM PowerEN processor
于 2011-8-30 17:45, Alexander Graf 写道: Hi Kun, On 08/29/2011 11:31 AM, Kun Wang wrote: Hi, everyone, This is Kun Wang from IBM Research China. I and my team have been working on IBM PowerEN processor in recent years, including its simulation, lib/runtime optimization and etc. Now we start the work to enable KVM on PowerEN processor. Since the A2 core of PowerEN follows Power ISA v2.06 (more specifically, book3e and 64-bit), I believe 99% of our work will stick to the ISA, and hence can be leveraged by others. As the new one to this KVM world, I and my team definitely need your help. Looking forward to talking and working with you guys in the future. Welcome to the PowerPC KVM world! I'm looking very much forward to working with you there. Please always CC k...@vger.kernel.org for emails you send to kvm-ppc@vger, unless you think of it as completely off-generic-kvm discussions. I like the idea of having more clever people involved in the PPC KVM process and by CC'ing kvm@vger, we get more exposure. The Freescale e500 cores are basically BookE 2.06 compliant with a few specialties here and there. Have you looked at their code? Sure, it's 32-bit as is now but there is definitely work going on at Freescale to get it 64-bit too. Maybe it makes sense to take the current code as a starting point and slowly move towards an architecture that is more generic. Maybe it makes sense to start from scratch and actually design something flexible that splits guest and host TLB code, so we could potentially run cross-book KVM in the future (which I would love to see happen! Just imagine running a PowerEN guest on a POWER7 system). Either way, please make sure to coordinate any efforts with me and Scott so we don't walk off in different directions and only get to realize we went down the wrong path when there's already a 100 patches patch set on the mailing list :) Again, welcome on board! Alex Thanks, Alex. Yes, we are aware of the e500 work done by Freescale folks. I agree with you that we should coordinate together to achieve as more synergy. Best regards, Kun Wang -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/4] Avoid soft lockup message when KVM is stopped by host
On Tue, Aug 30, 2011 at 09:43:19AM -0500, Ryan Harper wrote: * Marcelo Tosatti mtosa...@redhat.com [2011-08-30 09:40]: On Tue, Aug 30, 2011 at 09:12:17AM -0500, Ryan Harper wrote: * Marcelo Tosatti mtosa...@redhat.com [2011-08-30 07:35]: On Mon, Aug 29, 2011 at 05:27:11PM -0600, Eric B Munson wrote: Currently, when qemu stops a guest kernel that guest will issue a soft lockup message when it resumes. This set provides the ability for qemu to comminucate to the guest that it has been stopped. When the guest hits the watchdog on resume it will check if it was suspended before issuing the warning. Eric B Munson (4): Add flag to indicate that a vm was stopped by the host Add functions to check if the host has stopped the vm Add generic stubs for kvm stop check functions Add check for suspended vm in softlockup detector arch/x86/include/asm/pvclock-abi.h |1 + arch/x86/include/asm/pvclock.h |2 ++ arch/x86/kernel/kvmclock.c | 14 ++ include/asm-generic/pvclock.h | 14 ++ kernel/watchdog.c | 12 5 files changed, 43 insertions(+), 0 deletions(-) create mode 100644 include/asm-generic/pvclock.h -- 1.7.4.1 How is the host supposed to set this flag? As mentioned previously, if you save save/restore the offset added to kvmclock on stop/cont (and the TSC MSR, forgot to mention that), no paravirt infrastructure is required. Which means the issue is also fixed for older guests. How is saving the offset going to prevent a large jump from triggering the softlock message? Won't we still have not touched the watchdog for that long period of time? The idea is to adjust the offset and the TSC value so that the guest does not notice the actual elapsed time. This is what happens when a guest is migrated or saved/restore, for example. but that's within a rather short period of time... if we stop the guest for a day, I don't think what you are suggesting can happen without time being wrong in the guest, Right. and as soon as it's corrected and we jump forward larger than the watchdog timeout, won't we trigger it again? The time correction within the guest happens separately from the kvmclock clocksource, so no, watchdog timeout will not trigger. -- To unsubscribe from this list: send the line unsubscribe kvm-ppc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
