Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 12:11 +0200, Alexander Graf wrote: > > So I must add one more ioctl to enable MULTITCE in kernel handling. Is it > > what you are saying? > > > > I can see KVM_CHECK_EXTENSION but I do not see KVM_ENABLE_EXTENSION or > > anything like that. > > KVM_ENABLE_CAP. It's how we enable sPAPR capabilities too. But in that case I don't see the point. Cheers, Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 15:13, Alexey Kardashevskiy wrote: > On 07/11/2013 10:58 PM, Benjamin Herrenschmidt wrote: >> On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: >>> I don't like bloat usually. But Alexey even had an #ifdef DEBUG in >>> there to selectively disable in-kernel handling of multi-TCE. Not >>> calling ENABLE_CAP would give him exactly that without ugly #ifdefs in >>> the kernel. >> >> I don't see much point in disabling it... but ok, if that's a valuable >> feature, then shoot some VM level ENABLE_CAP (please don't iterate all >> VCPUs, that's gross). > > No use for me whatsoever as I only want to disable real more handlers and > keep virtual mode handlers enabled (sometime, for debug only) and this > capability is not about that - I can easily just not enable it in QEMU with > the exactly the same effect. > > So please, fellas, decide whether I should iterate vcpu's or add ENABLE_CAP > per KVM. Thanks. Thinking hard about this it might actually be ok to not have an ENABLE_CAP for this, if kernel code always works properly, because from the guest's point of view nothing changes - it either gets handled by kernel or by user space. And user space either handles it or doesn't, so it's ok. Just leave it out this time. But be very weary of adding new features without an ENABLE_CAP switch. They might be guest visible changes. Alex -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 10:58 PM, Benjamin Herrenschmidt wrote: > On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: >> I don't like bloat usually. But Alexey even had an #ifdef DEBUG in >> there to selectively disable in-kernel handling of multi-TCE. Not >> calling ENABLE_CAP would give him exactly that without ugly #ifdefs in >> the kernel. > > I don't see much point in disabling it... but ok, if that's a valuable > feature, then shoot some VM level ENABLE_CAP (please don't iterate all > VCPUs, that's gross). No use for me whatsoever as I only want to disable real more handlers and keep virtual mode handlers enabled (sometime, for debug only) and this capability is not about that - I can easily just not enable it in QEMU with the exactly the same effect. So please, fellas, decide whether I should iterate vcpu's or add ENABLE_CAP per KVM. Thanks. -- Alexey -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 14:33, Benjamin Herrenschmidt wrote: > On Thu, 2013-07-11 at 15:12 +1000, Alexey Kardashevskiy wrote: Any debug code is prohibited? Ok, I'll remove. >>> >>> Debug code that requires code changes is prohibited, yes. >>> Debug code that is runtime switchable (pr_debug, trace points, etc) >>> are allowed. > > Bollox. > > $ grep DBG\( arch/powerpc/ -r | wc -l > 418 > > Also pr_devel is not runtime switchable in normal kernels either and > still an "official" kernel interface. > >> Is there any easy way to enable just this specific udbg_printf (not all of >> them at once)? Trace points do not work in real mode as we figured out. > > The cleaner way to do it is to use some kind of local macro that you > enable/disable by changing a #define at the top of the function, possibly > several. If you do that in a way that doesn't bitrot, that's acceptable, yes. Something that leaves cpp compile checks working while optimizing out the debug code. Alex -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 15:12 +1000, Alexey Kardashevskiy wrote: > >> Any debug code is prohibited? Ok, I'll remove. > > > > Debug code that requires code changes is prohibited, yes. > > Debug code that is runtime switchable (pr_debug, trace points, etc) > > are allowed. Bollox. $ grep DBG\( arch/powerpc/ -r | wc -l 418 Also pr_devel is not runtime switchable in normal kernels either and still an "official" kernel interface. > Is there any easy way to enable just this specific udbg_printf (not all of > them at once)? Trace points do not work in real mode as we figured out. The cleaner way to do it is to use some kind of local macro that you enable/disable by changing a #define at the top of the function, possibly several. Cheers, Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: > I don't like bloat usually. But Alexey even had an #ifdef DEBUG in > there to selectively disable in-kernel handling of multi-TCE. Not > calling ENABLE_CAP would give him exactly that without ugly #ifdefs in > the kernel. I don't see much point in disabling it... but ok, if that's a valuable feature, then shoot some VM level ENABLE_CAP (please don't iterate all VCPUs, that's gross). Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 10:51 PM, Alexander Graf wrote: > > On 11.07.2013, at 14:39, Benjamin Herrenschmidt wrote: > >> On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: >>> And that's bad. Jeez, seriously. Don't argue this case. We enable new >>> features individually unless we're 100% sure we can keep everything >>> working. In this case an ENABLE_CAP doesn't hurt at all, because user >>> space still needs to handle the hypercalls if it wants them anyways. >>> But you get debugging for free for example. >> >> An ENABLE_CAP is utterly pointless. More bloat. But you seem to like >> it :-) > > I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there > to selectively disable in-kernel handling of multi-TCE. Not calling > ENABLE_CAP would give him exactly that without ugly #ifdefs in the > kernel. No, it would not give m anithing. My ugly debug was to disable realmode only and still leave virtual mode on, not to disable both real and virtual modes. It is a lot easier to disable in kernel handling in QEMU. -- Alexey -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 14:39, Benjamin Herrenschmidt wrote: > On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: >> And that's bad. Jeez, seriously. Don't argue this case. We enable new >> features individually unless we're 100% sure we can keep everything >> working. In this case an ENABLE_CAP doesn't hurt at all, because user >> space still needs to handle the hypercalls if it wants them anyways. >> But you get debugging for free for example. > > An ENABLE_CAP is utterly pointless. More bloat. But you seem to like > it :-) I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. Alex > > Ben. > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: > There are 2 ways of dealing with this: > > 1) Call the ENABLE_CAP on every vcpu. That way one CPU may handle > this hypercall in the kernel while another one may not. The same as we > handle PAPR today. > > 2) Create a new ENABLE_CAP for the vm. > > I think in this case option 1 is fine - it's how we handle everything > else already. So, you are now asking him to chose between a gross horror or adding a new piece of infrastructure for something that is entirely pointless to begin with ? Come on, give him a break. That stuff is fine as it is. Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: > And that's bad. Jeez, seriously. Don't argue this case. We enable new > features individually unless we're 100% sure we can keep everything > working. In this case an ENABLE_CAP doesn't hurt at all, because user > space still needs to handle the hypercalls if it wants them anyways. > But you get debugging for free for example. An ENABLE_CAP is utterly pointless. More bloat. But you seem to like it :-) Ben. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 12:54, Alexey Kardashevskiy wrote: > On 07/11/2013 08:11 PM, Alexander Graf wrote: >> >> On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: >> >>> On 07/10/2013 08:05 PM, Alexander Graf wrote: On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: > On 07/10/2013 03:02 AM, Alexander Graf wrote: >> On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: >>> This adds real mode handlers for the H_PUT_TCE_INDIRECT and >>> H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO >>> devices or emulated PCI. These calls allow adding multiple entries >>> (up to 512) into the TCE table in one call which saves time on >>> transition to/from real mode. >> >> We don't mention QEMU explicitly in KVM code usually. >> >>> This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs >>> (copied from user and verified) before writing the whole list into >>> the TCE table. This cache will be utilized more in the upcoming >>> VFIO/IOMMU support to continue TCE list processing in the virtual >>> mode in the case if the real mode handler failed for some reason. >>> >>> This adds a guest physical to host real address converter >>> and calls the existing H_PUT_TCE handler. The converting function >>> is going to be fully utilized by upcoming VFIO supporting patches. >>> >>> This also implements the KVM_CAP_PPC_MULTITCE capability, >>> so in order to support the functionality of this patch, QEMU >>> needs to query for this capability and set the "hcall-multi-tce" >>> hypertas property only if the capability is present, otherwise >>> there will be serious performance degradation. >> >> Same as above. But really you're only giving recommendations here. What's >> the point? Please describe what the benefit of this patch is, not what >> some >> other random subsystem might do with the benefits it brings. >> >>> >>> Signed-off-by: Paul Mackerras >>> Signed-off-by: Alexey Kardashevskiy >>> >>> --- >>> Changelog: >>> 2013/07/06: >>> * fixed number of wrong get_page()/put_page() calls >>> >>> 2013/06/27: >>> * fixed clear of BUSY bit in kvmppc_lookup_pte() >>> * H_PUT_TCE_INDIRECT does realmode_get_page() now >>> * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 >>> * updated doc >>> >>> 2013/06/05: >>> * fixed mistype about IBMVIO in the commit message >>> * updated doc and moved it to another section >>> * changed capability number >>> >>> 2013/05/21: >>> * added kvm_vcpu_arch::tce_tmp >>> * removed cleanup if put_indirect failed, instead we do not even start >>> writing to TCE table if we cannot get TCEs from the user and they are >>> invalid >>> * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce >>> and kvmppc_emulated_validate_tce (for the previous item) >>> * fixed bug with failthrough for H_IPI >>> * removed all get_user() from real mode handlers >>> * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) >>> >>> Signed-off-by: Alexey Kardashevskiy >>> --- >>> Documentation/virtual/kvm/api.txt | 25 +++ >>> arch/powerpc/include/asm/kvm_host.h | 9 ++ >>> arch/powerpc/include/asm/kvm_ppc.h | 16 +- >>> arch/powerpc/kvm/book3s_64_vio.c| 154 ++- >>> arch/powerpc/kvm/book3s_64_vio_hv.c | 260 >>> >>> arch/powerpc/kvm/book3s_hv.c| 41 - >>> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + >>> arch/powerpc/kvm/book3s_pr_papr.c | 37 - >>> arch/powerpc/kvm/powerpc.c | 3 + >>> 9 files changed, 517 insertions(+), 34 deletions(-) >>> >>> diff --git a/Documentation/virtual/kvm/api.txt >>> b/Documentation/virtual/kvm/api.txt >>> index 6365fef..762c703 100644 >>> --- a/Documentation/virtual/kvm/api.txt >>> +++ b/Documentation/virtual/kvm/api.txt >>> @@ -2362,6 +2362,31 @@ calls by the guest for that service will be >>> passed >>> to userspace to be >>> handled. >>> >>> >>> +4.86 KVM_CAP_PPC_MULTITCE >>> + >>> +Capability: KVM_CAP_PPC_MULTITCE >>> +Architectures: ppc >>> +Type: vm >>> + >>> +This capability means the kernel is capable of handling hypercalls >>> +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user >>> +space. This significanly accelerates DMA operations for PPC KVM guests. >> >> significanly? Please run this through a spell checker. >> >>> +The user space should expect that its handlers for these hypercalls >> >> s/The// >> >>> +are not going to be called. >> >> Is user space guaranteed they will not be called? Or can it still happen? > > ... if user space
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 08:11 PM, Alexander Graf wrote: > > On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: > >> On 07/10/2013 08:05 PM, Alexander Graf wrote: >>> >>> On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: >>> On 07/10/2013 03:02 AM, Alexander Graf wrote: > On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: >> This adds real mode handlers for the H_PUT_TCE_INDIRECT and >> H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO >> devices or emulated PCI. These calls allow adding multiple entries >> (up to 512) into the TCE table in one call which saves time on >> transition to/from real mode. > > We don't mention QEMU explicitly in KVM code usually. > >> This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs >> (copied from user and verified) before writing the whole list into >> the TCE table. This cache will be utilized more in the upcoming >> VFIO/IOMMU support to continue TCE list processing in the virtual >> mode in the case if the real mode handler failed for some reason. >> >> This adds a guest physical to host real address converter >> and calls the existing H_PUT_TCE handler. The converting function >> is going to be fully utilized by upcoming VFIO supporting patches. >> >> This also implements the KVM_CAP_PPC_MULTITCE capability, >> so in order to support the functionality of this patch, QEMU >> needs to query for this capability and set the "hcall-multi-tce" >> hypertas property only if the capability is present, otherwise >> there will be serious performance degradation. > > Same as above. But really you're only giving recommendations here. What's > the point? Please describe what the benefit of this patch is, not what > some > other random subsystem might do with the benefits it brings. > >> >> Signed-off-by: Paul Mackerras >> Signed-off-by: Alexey Kardashevskiy >> >> --- >> Changelog: >> 2013/07/06: >> * fixed number of wrong get_page()/put_page() calls >> >> 2013/06/27: >> * fixed clear of BUSY bit in kvmppc_lookup_pte() >> * H_PUT_TCE_INDIRECT does realmode_get_page() now >> * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 >> * updated doc >> >> 2013/06/05: >> * fixed mistype about IBMVIO in the commit message >> * updated doc and moved it to another section >> * changed capability number >> >> 2013/05/21: >> * added kvm_vcpu_arch::tce_tmp >> * removed cleanup if put_indirect failed, instead we do not even start >> writing to TCE table if we cannot get TCEs from the user and they are >> invalid >> * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce >> and kvmppc_emulated_validate_tce (for the previous item) >> * fixed bug with failthrough for H_IPI >> * removed all get_user() from real mode handlers >> * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) >> >> Signed-off-by: Alexey Kardashevskiy >> --- >> Documentation/virtual/kvm/api.txt | 25 +++ >> arch/powerpc/include/asm/kvm_host.h | 9 ++ >> arch/powerpc/include/asm/kvm_ppc.h | 16 +- >> arch/powerpc/kvm/book3s_64_vio.c| 154 ++- >> arch/powerpc/kvm/book3s_64_vio_hv.c | 260 >> >> arch/powerpc/kvm/book3s_hv.c| 41 - >> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + >> arch/powerpc/kvm/book3s_pr_papr.c | 37 - >> arch/powerpc/kvm/powerpc.c | 3 + >> 9 files changed, 517 insertions(+), 34 deletions(-) >> >> diff --git a/Documentation/virtual/kvm/api.txt >> b/Documentation/virtual/kvm/api.txt >> index 6365fef..762c703 100644 >> --- a/Documentation/virtual/kvm/api.txt >> +++ b/Documentation/virtual/kvm/api.txt >> @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed >> to userspace to be >> handled. >> >> >> +4.86 KVM_CAP_PPC_MULTITCE >> + >> +Capability: KVM_CAP_PPC_MULTITCE >> +Architectures: ppc >> +Type: vm >> + >> +This capability means the kernel is capable of handling hypercalls >> +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user >> +space. This significanly accelerates DMA operations for PPC KVM guests. > > significanly? Please run this through a spell checker. > >> +The user space should expect that its handlers for these hypercalls > > s/The// > >> +are not going to be called. > > Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? >>> >>> How about this? >>> >>> The hypercalls mentioned above may or may not be processed successfully in >>>
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: > On 07/10/2013 08:05 PM, Alexander Graf wrote: >> >> On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: >> >>> On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: > This adds real mode handlers for the H_PUT_TCE_INDIRECT and > H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO > devices or emulated PCI. These calls allow adding multiple entries > (up to 512) into the TCE table in one call which saves time on > transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. > This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs > (copied from user and verified) before writing the whole list into > the TCE table. This cache will be utilized more in the upcoming > VFIO/IOMMU support to continue TCE list processing in the virtual > mode in the case if the real mode handler failed for some reason. > > This adds a guest physical to host real address converter > and calls the existing H_PUT_TCE handler. The converting function > is going to be fully utilized by upcoming VFIO supporting patches. > > This also implements the KVM_CAP_PPC_MULTITCE capability, > so in order to support the functionality of this patch, QEMU > needs to query for this capability and set the "hcall-multi-tce" > hypertas property only if the capability is present, otherwise > there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. > > Signed-off-by: Paul Mackerras > Signed-off-by: Alexey Kardashevskiy > > --- > Changelog: > 2013/07/06: > * fixed number of wrong get_page()/put_page() calls > > 2013/06/27: > * fixed clear of BUSY bit in kvmppc_lookup_pte() > * H_PUT_TCE_INDIRECT does realmode_get_page() now > * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 > * updated doc > > 2013/06/05: > * fixed mistype about IBMVIO in the commit message > * updated doc and moved it to another section > * changed capability number > > 2013/05/21: > * added kvm_vcpu_arch::tce_tmp > * removed cleanup if put_indirect failed, instead we do not even start > writing to TCE table if we cannot get TCEs from the user and they are > invalid > * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce > and kvmppc_emulated_validate_tce (for the previous item) > * fixed bug with failthrough for H_IPI > * removed all get_user() from real mode handlers > * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) > > Signed-off-by: Alexey Kardashevskiy > --- > Documentation/virtual/kvm/api.txt | 25 +++ > arch/powerpc/include/asm/kvm_host.h | 9 ++ > arch/powerpc/include/asm/kvm_ppc.h | 16 +- > arch/powerpc/kvm/book3s_64_vio.c| 154 ++- > arch/powerpc/kvm/book3s_64_vio_hv.c | 260 > > arch/powerpc/kvm/book3s_hv.c| 41 - > arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + > arch/powerpc/kvm/book3s_pr_papr.c | 37 - > arch/powerpc/kvm/powerpc.c | 3 + > 9 files changed, 517 insertions(+), 34 deletions(-) > > diff --git a/Documentation/virtual/kvm/api.txt > b/Documentation/virtual/kvm/api.txt > index 6365fef..762c703 100644 > --- a/Documentation/virtual/kvm/api.txt > +++ b/Documentation/virtual/kvm/api.txt > @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed > to userspace to be > handled. > > > +4.86 KVM_CAP_PPC_MULTITCE > + > +Capability: KVM_CAP_PPC_MULTITCE > +Architectures: ppc > +Type: vm > + > +This capability means the kernel is capable of handling hypercalls > +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user > +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. > +The user space should expect that its handlers for these hypercalls s/The// > +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? >>> >>> ... if user space previously registered LIOBN in KVM (via >>> KVM_CREATE_SPAPR_TCE or similar calls). >>> >>> ok? >> >> How about this? >> >> The hypercalls mentioned above may or may not be processed successfully in >> the kernel based fast path. If they can not be handled by the kernel, they >> will get passed on to user space. So user space still has to have an >>
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: On 07/10/2013 08:05 PM, Alexander Graf wrote: On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, +the user space might have to advertise it for the guest. For example,
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 08:11 PM, Alexander Graf wrote: On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: On 07/10/2013 08:05 PM, Alexander Graf wrote: On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, +the user space might have to advertise it for the
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 12:54, Alexey Kardashevskiy wrote: On 07/11/2013 08:11 PM, Alexander Graf wrote: On 11.07.2013, at 07:12, Alexey Kardashevskiy wrote: On 07/10/2013 08:05 PM, Alexander Graf wrote: On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: And that's bad. Jeez, seriously. Don't argue this case. We enable new features individually unless we're 100% sure we can keep everything working. In this case an ENABLE_CAP doesn't hurt at all, because user space still needs to handle the hypercalls if it wants them anyways. But you get debugging for free for example. An ENABLE_CAP is utterly pointless. More bloat. But you seem to like it :-) Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: There are 2 ways of dealing with this: 1) Call the ENABLE_CAP on every vcpu. That way one CPU may handle this hypercall in the kernel while another one may not. The same as we handle PAPR today. 2) Create a new ENABLE_CAP for the vm. I think in this case option 1 is fine - it's how we handle everything else already. So, you are now asking him to chose between a gross horror or adding a new piece of infrastructure for something that is entirely pointless to begin with ? Come on, give him a break. That stuff is fine as it is. Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 14:39, Benjamin Herrenschmidt wrote: On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: And that's bad. Jeez, seriously. Don't argue this case. We enable new features individually unless we're 100% sure we can keep everything working. In this case an ENABLE_CAP doesn't hurt at all, because user space still needs to handle the hypercalls if it wants them anyways. But you get debugging for free for example. An ENABLE_CAP is utterly pointless. More bloat. But you seem to like it :-) I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. Alex Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 10:51 PM, Alexander Graf wrote: On 11.07.2013, at 14:39, Benjamin Herrenschmidt wrote: On Thu, 2013-07-11 at 13:15 +0200, Alexander Graf wrote: And that's bad. Jeez, seriously. Don't argue this case. We enable new features individually unless we're 100% sure we can keep everything working. In this case an ENABLE_CAP doesn't hurt at all, because user space still needs to handle the hypercalls if it wants them anyways. But you get debugging for free for example. An ENABLE_CAP is utterly pointless. More bloat. But you seem to like it :-) I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. No, it would not give m anithing. My ugly debug was to disable realmode only and still leave virtual mode on, not to disable both real and virtual modes. It is a lot easier to disable in kernel handling in QEMU. -- Alexey -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. I don't see much point in disabling it... but ok, if that's a valuable feature, then shoot some VM level ENABLE_CAP (please don't iterate all VCPUs, that's gross). Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 15:12 +1000, Alexey Kardashevskiy wrote: Any debug code is prohibited? Ok, I'll remove. Debug code that requires code changes is prohibited, yes. Debug code that is runtime switchable (pr_debug, trace points, etc) are allowed. Bollox. $ grep DBG\( arch/powerpc/ -r | wc -l 418 Also pr_devel is not runtime switchable in normal kernels either and still an official kernel interface. Is there any easy way to enable just this specific udbg_printf (not all of them at once)? Trace points do not work in real mode as we figured out. The cleaner way to do it is to use some kind of local macro that you enable/disable by changing a #define at the top of the function, possibly several. Cheers, Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 14:33, Benjamin Herrenschmidt wrote: On Thu, 2013-07-11 at 15:12 +1000, Alexey Kardashevskiy wrote: Any debug code is prohibited? Ok, I'll remove. Debug code that requires code changes is prohibited, yes. Debug code that is runtime switchable (pr_debug, trace points, etc) are allowed. Bollox. $ grep DBG\( arch/powerpc/ -r | wc -l 418 Also pr_devel is not runtime switchable in normal kernels either and still an official kernel interface. Is there any easy way to enable just this specific udbg_printf (not all of them at once)? Trace points do not work in real mode as we figured out. The cleaner way to do it is to use some kind of local macro that you enable/disable by changing a #define at the top of the function, possibly several. If you do that in a way that doesn't bitrot, that's acceptable, yes. Something that leaves cpp compile checks working while optimizing out the debug code. Alex -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/11/2013 10:58 PM, Benjamin Herrenschmidt wrote: On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. I don't see much point in disabling it... but ok, if that's a valuable feature, then shoot some VM level ENABLE_CAP (please don't iterate all VCPUs, that's gross). No use for me whatsoever as I only want to disable real more handlers and keep virtual mode handlers enabled (sometime, for debug only) and this capability is not about that - I can easily just not enable it in QEMU with the exactly the same effect. So please, fellas, decide whether I should iterate vcpu's or add ENABLE_CAP per KVM. Thanks. -- Alexey -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 11.07.2013, at 15:13, Alexey Kardashevskiy wrote: On 07/11/2013 10:58 PM, Benjamin Herrenschmidt wrote: On Thu, 2013-07-11 at 14:51 +0200, Alexander Graf wrote: I don't like bloat usually. But Alexey even had an #ifdef DEBUG in there to selectively disable in-kernel handling of multi-TCE. Not calling ENABLE_CAP would give him exactly that without ugly #ifdefs in the kernel. I don't see much point in disabling it... but ok, if that's a valuable feature, then shoot some VM level ENABLE_CAP (please don't iterate all VCPUs, that's gross). No use for me whatsoever as I only want to disable real more handlers and keep virtual mode handlers enabled (sometime, for debug only) and this capability is not about that - I can easily just not enable it in QEMU with the exactly the same effect. So please, fellas, decide whether I should iterate vcpu's or add ENABLE_CAP per KVM. Thanks. Thinking hard about this it might actually be ok to not have an ENABLE_CAP for this, if kernel code always works properly, because from the guest's point of view nothing changes - it either gets handled by kernel or by user space. And user space either handles it or doesn't, so it's ok. Just leave it out this time. But be very weary of adding new features without an ENABLE_CAP switch. They might be guest visible changes. Alex -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On Thu, 2013-07-11 at 12:11 +0200, Alexander Graf wrote: So I must add one more ioctl to enable MULTITCE in kernel handling. Is it what you are saying? I can see KVM_CHECK_EXTENSION but I do not see KVM_ENABLE_EXTENSION or anything like that. KVM_ENABLE_CAP. It's how we enable sPAPR capabilities too. But in that case I don't see the point. Cheers, Ben. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/10/2013 08:05 PM, Alexander Graf wrote: > > On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: > >> On 07/10/2013 03:02 AM, Alexander Graf wrote: >>> On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. >>> >>> We don't mention QEMU explicitly in KVM code usually. >>> This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the "hcall-multi-tce" hypertas property only if the capability is present, otherwise there will be serious performance degradation. >>> >>> Same as above. But really you're only giving recommendations here. What's >>> the point? Please describe what the benefit of this patch is, not what some >>> other random subsystem might do with the benefits it brings. >>> Signed-off-by: Paul Mackerras Signed-off-by: Alexey Kardashevskiy --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiy --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. >>> >>> significanly? Please run this through a spell checker. >>> +The user space should expect that its handlers for these hypercalls >>> >>> s/The// >>> +are not going to be called. >>> >>> Is user space guaranteed they will not be called? Or can it still happen? >> >> ... if user space previously registered LIOBN in KVM (via >> KVM_CREATE_SPAPR_TCE or similar calls). >> >> ok? > > How about this? > > The hypercalls mentioned above may or may not be processed successfully in > the kernel based fast path. If they can not be handled by the kernel, they > will get passed on to user space. So user space still has to have an > implementation for these despite the in kernel acceleration. > > --- > > The target audience for this documentation is user space KVM API users. > Someone developing kvm tool for
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: > On 07/10/2013 03:02 AM, Alexander Graf wrote: >> On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: >>> This adds real mode handlers for the H_PUT_TCE_INDIRECT and >>> H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO >>> devices or emulated PCI. These calls allow adding multiple entries >>> (up to 512) into the TCE table in one call which saves time on >>> transition to/from real mode. >> >> We don't mention QEMU explicitly in KVM code usually. >> >>> This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs >>> (copied from user and verified) before writing the whole list into >>> the TCE table. This cache will be utilized more in the upcoming >>> VFIO/IOMMU support to continue TCE list processing in the virtual >>> mode in the case if the real mode handler failed for some reason. >>> >>> This adds a guest physical to host real address converter >>> and calls the existing H_PUT_TCE handler. The converting function >>> is going to be fully utilized by upcoming VFIO supporting patches. >>> >>> This also implements the KVM_CAP_PPC_MULTITCE capability, >>> so in order to support the functionality of this patch, QEMU >>> needs to query for this capability and set the "hcall-multi-tce" >>> hypertas property only if the capability is present, otherwise >>> there will be serious performance degradation. >> >> Same as above. But really you're only giving recommendations here. What's >> the point? Please describe what the benefit of this patch is, not what some >> other random subsystem might do with the benefits it brings. >> >>> >>> Signed-off-by: Paul Mackerras >>> Signed-off-by: Alexey Kardashevskiy >>> >>> --- >>> Changelog: >>> 2013/07/06: >>> * fixed number of wrong get_page()/put_page() calls >>> >>> 2013/06/27: >>> * fixed clear of BUSY bit in kvmppc_lookup_pte() >>> * H_PUT_TCE_INDIRECT does realmode_get_page() now >>> * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 >>> * updated doc >>> >>> 2013/06/05: >>> * fixed mistype about IBMVIO in the commit message >>> * updated doc and moved it to another section >>> * changed capability number >>> >>> 2013/05/21: >>> * added kvm_vcpu_arch::tce_tmp >>> * removed cleanup if put_indirect failed, instead we do not even start >>> writing to TCE table if we cannot get TCEs from the user and they are >>> invalid >>> * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce >>> and kvmppc_emulated_validate_tce (for the previous item) >>> * fixed bug with failthrough for H_IPI >>> * removed all get_user() from real mode handlers >>> * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) >>> >>> Signed-off-by: Alexey Kardashevskiy >>> --- >>> Documentation/virtual/kvm/api.txt | 25 +++ >>> arch/powerpc/include/asm/kvm_host.h | 9 ++ >>> arch/powerpc/include/asm/kvm_ppc.h | 16 +- >>> arch/powerpc/kvm/book3s_64_vio.c| 154 ++- >>> arch/powerpc/kvm/book3s_64_vio_hv.c | 260 >>> >>> arch/powerpc/kvm/book3s_hv.c| 41 - >>> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + >>> arch/powerpc/kvm/book3s_pr_papr.c | 37 - >>> arch/powerpc/kvm/powerpc.c | 3 + >>> 9 files changed, 517 insertions(+), 34 deletions(-) >>> >>> diff --git a/Documentation/virtual/kvm/api.txt >>> b/Documentation/virtual/kvm/api.txt >>> index 6365fef..762c703 100644 >>> --- a/Documentation/virtual/kvm/api.txt >>> +++ b/Documentation/virtual/kvm/api.txt >>> @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed >>> to userspace to be >>> handled. >>> >>> >>> +4.86 KVM_CAP_PPC_MULTITCE >>> + >>> +Capability: KVM_CAP_PPC_MULTITCE >>> +Architectures: ppc >>> +Type: vm >>> + >>> +This capability means the kernel is capable of handling hypercalls >>> +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user >>> +space. This significanly accelerates DMA operations for PPC KVM guests. >> >> significanly? Please run this through a spell checker. >> >>> +The user space should expect that its handlers for these hypercalls >> >> s/The// >> >>> +are not going to be called. >> >> Is user space guaranteed they will not be called? Or can it still happen? > > ... if user space previously registered LIOBN in KVM (via > KVM_CREATE_SPAPR_TCE or similar calls). > > ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. > > There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet > and may never get there.
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, +the user space might have to advertise it for the guest. For example, +IBM pSeries guest starts using them if hcall-multi-tce is present in +the ibm,hypertas-functions
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/10/2013 08:05 PM, Alexander Graf wrote: On 10.07.2013, at 07:00, Alexey Kardashevskiy wrote: On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? How about this? The hypercalls mentioned above may or may not be processed successfully in the kernel based fast path. If they can not be handled by the kernel, they will get passed on to user space. So user space still has to have an implementation for these despite the in kernel acceleration. --- The target audience for this documentation is user space KVM API users. Someone developing kvm tool for example. They want to know implications specific CAPs have. There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, +the user space might have to advertise it for the guest. For example, +IBM pSeries guest starts using them if hcall-multi-tce is present in
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/10/2013 03:02 AM, Alexander Graf wrote: > On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: >> This adds real mode handlers for the H_PUT_TCE_INDIRECT and >> H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO >> devices or emulated PCI. These calls allow adding multiple entries >> (up to 512) into the TCE table in one call which saves time on >> transition to/from real mode. > > We don't mention QEMU explicitly in KVM code usually. > >> This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs >> (copied from user and verified) before writing the whole list into >> the TCE table. This cache will be utilized more in the upcoming >> VFIO/IOMMU support to continue TCE list processing in the virtual >> mode in the case if the real mode handler failed for some reason. >> >> This adds a guest physical to host real address converter >> and calls the existing H_PUT_TCE handler. The converting function >> is going to be fully utilized by upcoming VFIO supporting patches. >> >> This also implements the KVM_CAP_PPC_MULTITCE capability, >> so in order to support the functionality of this patch, QEMU >> needs to query for this capability and set the "hcall-multi-tce" >> hypertas property only if the capability is present, otherwise >> there will be serious performance degradation. > > Same as above. But really you're only giving recommendations here. What's > the point? Please describe what the benefit of this patch is, not what some > other random subsystem might do with the benefits it brings. > >> >> Signed-off-by: Paul Mackerras >> Signed-off-by: Alexey Kardashevskiy >> >> --- >> Changelog: >> 2013/07/06: >> * fixed number of wrong get_page()/put_page() calls >> >> 2013/06/27: >> * fixed clear of BUSY bit in kvmppc_lookup_pte() >> * H_PUT_TCE_INDIRECT does realmode_get_page() now >> * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 >> * updated doc >> >> 2013/06/05: >> * fixed mistype about IBMVIO in the commit message >> * updated doc and moved it to another section >> * changed capability number >> >> 2013/05/21: >> * added kvm_vcpu_arch::tce_tmp >> * removed cleanup if put_indirect failed, instead we do not even start >> writing to TCE table if we cannot get TCEs from the user and they are >> invalid >> * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce >> and kvmppc_emulated_validate_tce (for the previous item) >> * fixed bug with failthrough for H_IPI >> * removed all get_user() from real mode handlers >> * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) >> >> Signed-off-by: Alexey Kardashevskiy >> --- >> Documentation/virtual/kvm/api.txt | 25 +++ >> arch/powerpc/include/asm/kvm_host.h | 9 ++ >> arch/powerpc/include/asm/kvm_ppc.h | 16 +- >> arch/powerpc/kvm/book3s_64_vio.c| 154 ++- >> arch/powerpc/kvm/book3s_64_vio_hv.c | 260 >> >> arch/powerpc/kvm/book3s_hv.c| 41 - >> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + >> arch/powerpc/kvm/book3s_pr_papr.c | 37 - >> arch/powerpc/kvm/powerpc.c | 3 + >> 9 files changed, 517 insertions(+), 34 deletions(-) >> >> diff --git a/Documentation/virtual/kvm/api.txt >> b/Documentation/virtual/kvm/api.txt >> index 6365fef..762c703 100644 >> --- a/Documentation/virtual/kvm/api.txt >> +++ b/Documentation/virtual/kvm/api.txt >> @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed >> to userspace to be >> handled. >> >> >> +4.86 KVM_CAP_PPC_MULTITCE >> + >> +Capability: KVM_CAP_PPC_MULTITCE >> +Architectures: ppc >> +Type: vm >> + >> +This capability means the kernel is capable of handling hypercalls >> +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user >> +space. This significanly accelerates DMA operations for PPC KVM guests. > > significanly? Please run this through a spell checker. > >> +The user space should expect that its handlers for these hypercalls > > s/The// > >> +are not going to be called. > > Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. >> +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, >> +the user space might have to advertise it for the guest. For example, >> +IBM pSeries guest starts using them if "hcall-multi-tce" is present in >> +the "ibm,hypertas-functions" device-tree property. > > This paragraph describes sPAPR. That's fine, but please document it as > such. Also please check your grammar. >> + >> +Without this capability, only H_PUT_TCE is handled by the kernel and >> +therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended >> +unless the capability is present as passing hypercalls to the userspace >> +slows
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote:
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
We don't mention QEMU explicitly in KVM code usually.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the "hcall-multi-tce"
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Same as above. But really you're only giving recommendations here.
What's the point? Please describe what the benefit of this patch is, not
what some other random subsystem might do with the benefits it brings.
Signed-off-by: Paul Mackerras
Signed-off-by: Alexey Kardashevskiy
---
Changelog:
2013/07/06:
* fixed number of wrong get_page()/put_page() calls
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
Signed-off-by: Alexey Kardashevskiy
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h | 9 ++
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 154 ++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 260
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c | 3 +
9 files changed, 517 insertions(+), 34 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
significanly? Please run this through a spell checker.
+The user space should expect that its handlers for these hypercalls
s/The//
+are not going to be called.
Is user space guaranteed they will not be called? Or can it still happen?
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if "hcall-multi-tce" is present in
+the "ibm,hypertas-functions" device-tree property.
This paragraph describes sPAPR. That's fine, but please document it as
such. Also please check your grammar.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
Why? Wouldn't that confuse older user space?
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..20d04bd 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -180,6 +180,7 @@ struct kvmppc_spapr_tce_table {
struct kvm *kvm;
u64 liobn;
u32 window_size;
+ struct { struct {
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/10/2013 03:02 AM, Alexander Graf wrote: On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote: This adds real mode handlers for the H_PUT_TCE_INDIRECT and H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO devices or emulated PCI. These calls allow adding multiple entries (up to 512) into the TCE table in one call which saves time on transition to/from real mode. We don't mention QEMU explicitly in KVM code usually. This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs (copied from user and verified) before writing the whole list into the TCE table. This cache will be utilized more in the upcoming VFIO/IOMMU support to continue TCE list processing in the virtual mode in the case if the real mode handler failed for some reason. This adds a guest physical to host real address converter and calls the existing H_PUT_TCE handler. The converting function is going to be fully utilized by upcoming VFIO supporting patches. This also implements the KVM_CAP_PPC_MULTITCE capability, so in order to support the functionality of this patch, QEMU needs to query for this capability and set the hcall-multi-tce hypertas property only if the capability is present, otherwise there will be serious performance degradation. Same as above. But really you're only giving recommendations here. What's the point? Please describe what the benefit of this patch is, not what some other random subsystem might do with the benefits it brings. Signed-off-by: Paul Mackerraspau...@samba.org Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Changelog: 2013/07/06: * fixed number of wrong get_page()/put_page() calls 2013/06/27: * fixed clear of BUSY bit in kvmppc_lookup_pte() * H_PUT_TCE_INDIRECT does realmode_get_page() now * KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64 * updated doc 2013/06/05: * fixed mistype about IBMVIO in the commit message * updated doc and moved it to another section * changed capability number 2013/05/21: * added kvm_vcpu_arch::tce_tmp * removed cleanup if put_indirect failed, instead we do not even start writing to TCE table if we cannot get TCEs from the user and they are invalid * kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce and kvmppc_emulated_validate_tce (for the previous item) * fixed bug with failthrough for H_IPI * removed all get_user() from real mode handlers * kvmppc_lookup_pte() added (instead of making lookup_linux_pte public) Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru --- Documentation/virtual/kvm/api.txt | 25 +++ arch/powerpc/include/asm/kvm_host.h | 9 ++ arch/powerpc/include/asm/kvm_ppc.h | 16 +- arch/powerpc/kvm/book3s_64_vio.c| 154 ++- arch/powerpc/kvm/book3s_64_vio_hv.c | 260 arch/powerpc/kvm/book3s_hv.c| 41 - arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 + arch/powerpc/kvm/book3s_pr_papr.c | 37 - arch/powerpc/kvm/powerpc.c | 3 + 9 files changed, 517 insertions(+), 34 deletions(-) diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index 6365fef..762c703 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to userspace to be handled. +4.86 KVM_CAP_PPC_MULTITCE + +Capability: KVM_CAP_PPC_MULTITCE +Architectures: ppc +Type: vm + +This capability means the kernel is capable of handling hypercalls +H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user +space. This significanly accelerates DMA operations for PPC KVM guests. significanly? Please run this through a spell checker. +The user space should expect that its handlers for these hypercalls s/The// +are not going to be called. Is user space guaranteed they will not be called? Or can it still happen? ... if user space previously registered LIOBN in KVM (via KVM_CREATE_SPAPR_TCE or similar calls). ok? There is also KVM_CREATE_SPAPR_TCE_IOMMU but it is not in the kernel yet and may never get there. +In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest, +the user space might have to advertise it for the guest. For example, +IBM pSeries guest starts using them if hcall-multi-tce is present in +the ibm,hypertas-functions device-tree property. This paragraph describes sPAPR. That's fine, but please document it as such. Also please check your grammar. + +Without this capability, only H_PUT_TCE is handled by the kernel and +therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended +unless the capability is present as passing hypercalls to the userspace +slows operations a lot. + +Unlike other capabilities of this section, this one is always enabled. Why? Wouldn't that confuse older user space? How? Old user space won't check for
Re: [PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
On 07/06/2013 05:07 PM, Alexey Kardashevskiy wrote:
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
We don't mention QEMU explicitly in KVM code usually.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the hcall-multi-tce
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Same as above. But really you're only giving recommendations here.
What's the point? Please describe what the benefit of this patch is, not
what some other random subsystem might do with the benefits it brings.
Signed-off-by: Paul Mackerraspau...@samba.org
Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru
---
Changelog:
2013/07/06:
* fixed number of wrong get_page()/put_page() calls
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
Signed-off-by: Alexey Kardashevskiya...@ozlabs.ru
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h | 9 ++
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 154 ++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 260
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c | 3 +
9 files changed, 517 insertions(+), 34 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
significanly? Please run this through a spell checker.
+The user space should expect that its handlers for these hypercalls
s/The//
+are not going to be called.
Is user space guaranteed they will not be called? Or can it still happen?
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if hcall-multi-tce is present in
+the ibm,hypertas-functions device-tree property.
This paragraph describes sPAPR. That's fine, but please document it as
such. Also please check your grammar.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
Why? Wouldn't that confuse older user space?
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..20d04bd 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -180,6 +180,7 @@ struct kvmppc_spapr_tce_table {
struct kvm *kvm;
u64 liobn;
u32
[PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the "hcall-multi-tce"
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Signed-off-by: Paul Mackerras
Signed-off-by: Alexey Kardashevskiy
---
Changelog:
2013/07/06:
* fixed number of wrong get_page()/put_page() calls
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
Signed-off-by: Alexey Kardashevskiy
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h | 9 ++
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 154 ++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 260
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c | 3 +
9 files changed, 517 insertions(+), 34 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
+The user space should expect that its handlers for these hypercalls
+are not going to be called.
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if "hcall-multi-tce" is present in
+the "ibm,hypertas-functions" device-tree property.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..20d04bd 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -180,6 +180,7 @@ struct kvmppc_spapr_tce_table {
struct kvm *kvm;
u64 liobn;
u32 window_size;
+ struct { struct { unsigned long put, indir, stuff; } rm, vm; } stat;
struct page *pages[0];
};
@@ -609,6 +610,14 @@ struct kvm_vcpu_arch {
spinlock_t tbacct_lock;
u64 busy_stolen;
u64 busy_preempt;
+
+ unsigned long *tce_tmp_hpas;/* TCE cache for TCE_PUT_INDIRECT hcall
*/
+ enum {
+ TCERM_NONE,
+ TCERM_GETPAGE,
+ TCERM_PUTTCE,
+ TCERM_PUTLIST,
+ } tce_rm_fail; /* failed stage of request processing */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h
b/arch/powerpc/include/asm/kvm_ppc.h
index a5287fe..fa722a0 100644
[PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the hcall-multi-tce
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Signed-off-by: Paul Mackerras pau...@samba.org
Signed-off-by: Alexey Kardashevskiy a...@ozlabs.ru
---
Changelog:
2013/07/06:
* fixed number of wrong get_page()/put_page() calls
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
Signed-off-by: Alexey Kardashevskiy a...@ozlabs.ru
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h | 9 ++
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 154 ++-
arch/powerpc/kvm/book3s_64_vio_hv.c | 260
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c | 3 +
9 files changed, 517 insertions(+), 34 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
+The user space should expect that its handlers for these hypercalls
+are not going to be called.
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if hcall-multi-tce is present in
+the ibm,hypertas-functions device-tree property.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..20d04bd 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -180,6 +180,7 @@ struct kvmppc_spapr_tce_table {
struct kvm *kvm;
u64 liobn;
u32 window_size;
+ struct { struct { unsigned long put, indir, stuff; } rm, vm; } stat;
struct page *pages[0];
};
@@ -609,6 +610,14 @@ struct kvm_vcpu_arch {
spinlock_t tbacct_lock;
u64 busy_stolen;
u64 busy_preempt;
+
+ unsigned long *tce_tmp_hpas;/* TCE cache for TCE_PUT_INDIRECT hcall
*/
+ enum {
+ TCERM_NONE,
+ TCERM_GETPAGE,
+ TCERM_PUTTCE,
+ TCERM_PUTLIST,
+ } tce_rm_fail; /* failed stage of request processing */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h
[PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the "hcall-multi-tce"
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Signed-off-by: Paul Mackerras
Signed-off-by: Alexey Kardashevskiy
---
Changelog:
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h |2 +
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 123 ++
arch/powerpc/kvm/book3s_64_vio_hv.c | 270 +++
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S |6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c |3 +
9 files changed, 490 insertions(+), 33 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
+The user space should expect that its handlers for these hypercalls
+are not going to be called.
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if "hcall-multi-tce" is present in
+the "ibm,hypertas-functions" device-tree property.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..3bf407b 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -609,6 +609,8 @@ struct kvm_vcpu_arch {
spinlock_t tbacct_lock;
u64 busy_stolen;
u64 busy_preempt;
+
+ unsigned long *tce_tmp;/* TCE cache for TCE_PUT_INDIRECT hcall */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h
b/arch/powerpc/include/asm/kvm_ppc.h
index a5287fe..e852921b 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -133,8 +133,20 @@ extern int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu);
extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
struct kvm_create_spapr_tce *args);
-extern long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
-unsigned long ioba, unsigned long tce);
+extern struct kvmppc_spapr_tce_table *kvmppc_find_tce_table(
+ struct kvm_vcpu *vcpu, unsigned long liobn);
[PATCH 6/8] KVM: PPC: Add support for multiple-TCE hcalls
This adds real mode handlers for the H_PUT_TCE_INDIRECT and
H_STUFF_TCE hypercalls for QEMU emulated devices such as IBMVIO
devices or emulated PCI. These calls allow adding multiple entries
(up to 512) into the TCE table in one call which saves time on
transition to/from real mode.
This adds a tce_tmp cache to kvm_vcpu_arch to save valid TCEs
(copied from user and verified) before writing the whole list into
the TCE table. This cache will be utilized more in the upcoming
VFIO/IOMMU support to continue TCE list processing in the virtual
mode in the case if the real mode handler failed for some reason.
This adds a guest physical to host real address converter
and calls the existing H_PUT_TCE handler. The converting function
is going to be fully utilized by upcoming VFIO supporting patches.
This also implements the KVM_CAP_PPC_MULTITCE capability,
so in order to support the functionality of this patch, QEMU
needs to query for this capability and set the hcall-multi-tce
hypertas property only if the capability is present, otherwise
there will be serious performance degradation.
Signed-off-by: Paul Mackerras pau...@samba.org
Signed-off-by: Alexey Kardashevskiy a...@ozlabs.ru
---
Changelog:
2013/06/27:
* fixed clear of BUSY bit in kvmppc_lookup_pte()
* H_PUT_TCE_INDIRECT does realmode_get_page() now
* KVM_CAP_SPAPR_MULTITCE now depends on CONFIG_PPC_BOOK3S_64
* updated doc
2013/06/05:
* fixed mistype about IBMVIO in the commit message
* updated doc and moved it to another section
* changed capability number
2013/05/21:
* added kvm_vcpu_arch::tce_tmp
* removed cleanup if put_indirect failed, instead we do not even start
writing to TCE table if we cannot get TCEs from the user and they are
invalid
* kvmppc_emulated_h_put_tce is split to kvmppc_emulated_put_tce
and kvmppc_emulated_validate_tce (for the previous item)
* fixed bug with failthrough for H_IPI
* removed all get_user() from real mode handlers
* kvmppc_lookup_pte() added (instead of making lookup_linux_pte public)
---
Documentation/virtual/kvm/api.txt | 25 +++
arch/powerpc/include/asm/kvm_host.h |2 +
arch/powerpc/include/asm/kvm_ppc.h | 16 +-
arch/powerpc/kvm/book3s_64_vio.c| 123 ++
arch/powerpc/kvm/book3s_64_vio_hv.c | 270 +++
arch/powerpc/kvm/book3s_hv.c| 41 -
arch/powerpc/kvm/book3s_hv_rmhandlers.S |6 +
arch/powerpc/kvm/book3s_pr_papr.c | 37 -
arch/powerpc/kvm/powerpc.c |3 +
9 files changed, 490 insertions(+), 33 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 6365fef..762c703 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2362,6 +2362,31 @@ calls by the guest for that service will be passed to
userspace to be
handled.
+4.86 KVM_CAP_PPC_MULTITCE
+
+Capability: KVM_CAP_PPC_MULTITCE
+Architectures: ppc
+Type: vm
+
+This capability means the kernel is capable of handling hypercalls
+H_PUT_TCE_INDIRECT and H_STUFF_TCE without passing those into the user
+space. This significanly accelerates DMA operations for PPC KVM guests.
+The user space should expect that its handlers for these hypercalls
+are not going to be called.
+
+In order to enable H_PUT_TCE_INDIRECT and H_STUFF_TCE use in the guest,
+the user space might have to advertise it for the guest. For example,
+IBM pSeries guest starts using them if hcall-multi-tce is present in
+the ibm,hypertas-functions device-tree property.
+
+Without this capability, only H_PUT_TCE is handled by the kernel and
+therefore the use of H_PUT_TCE_INDIRECT and H_STUFF_TCE is not recommended
+unless the capability is present as passing hypercalls to the userspace
+slows operations a lot.
+
+Unlike other capabilities of this section, this one is always enabled.
+
+
5. The kvm_run structure
diff --git a/arch/powerpc/include/asm/kvm_host.h
b/arch/powerpc/include/asm/kvm_host.h
index af326cd..3bf407b 100644
--- a/arch/powerpc/include/asm/kvm_host.h
+++ b/arch/powerpc/include/asm/kvm_host.h
@@ -609,6 +609,8 @@ struct kvm_vcpu_arch {
spinlock_t tbacct_lock;
u64 busy_stolen;
u64 busy_preempt;
+
+ unsigned long *tce_tmp;/* TCE cache for TCE_PUT_INDIRECT hcall */
#endif
};
diff --git a/arch/powerpc/include/asm/kvm_ppc.h
b/arch/powerpc/include/asm/kvm_ppc.h
index a5287fe..e852921b 100644
--- a/arch/powerpc/include/asm/kvm_ppc.h
+++ b/arch/powerpc/include/asm/kvm_ppc.h
@@ -133,8 +133,20 @@ extern int kvmppc_pseries_do_hcall(struct kvm_vcpu *vcpu);
extern long kvm_vm_ioctl_create_spapr_tce(struct kvm *kvm,
struct kvm_create_spapr_tce *args);
-extern long kvmppc_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn,
-unsigned long ioba, unsigned long tce);
+extern struct kvmppc_spapr_tce_table *kvmppc_find_tce_table(
+ struct kvm_vcpu *vcpu,
