Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, Sep 29, 2008 at 10:42:37PM +0900, FUJITA Tomonori wrote: > On Mon, 29 Sep 2008 15:26:47 +0200 > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > On Mon, Sep 29, 2008 at 10:16:39PM +0900, FUJITA Tomonori wrote: > > > On Sun, 28 Sep 2008 20:49:26 +0200 > > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > > > On Sun, Sep 28, 2008 at 11:21:26PM +0900, FUJITA Tomonori wrote: > > > > > On Mon, 22 Sep 2008 20:21:12 +0200 > > > > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > this patch series implements stackable dma_ops on x86. This is > > > > > > useful to > > > > > > be able to fall back to a different dma_ops implementation if one > > > > > > can > > > > > > not handle a particular device (as necessary for example with > > > > > > paravirtualized device passthrough or if a hardware IOMMU only > > > > > > handles a > > > > > > subset of available devices). > > > > > > > > > > We already handle the latter. This patchset is more flexible but > > > > > seems to incur more overheads. > > > > > > > > > > This feature will be used for only paravirtualized device passthrough? > > > > > If so, I feel that there is more simpler (and specific) solutions for > > > > > it. > > > > > > > > Its not only for device passthrough. It handles also the cases where a > > > > hardware IOMMU does not handle all devices in the system (like in some > > > > Calgary systems but also possible with AMD IOMMU). With this patchset we > > > > > > I know that. As I wrote in the previous mail, we already solved that > > > problem with per-device-dma-ops. > > > > > > My question is what unsolved problems this patchset can fix? > > > > > > > > > This patchset is named "stackable dma_ops" but it's different from > > > what we discussed as "stackable dma_ops". This patchset provides > > > IOMMUs a generic mechanism to set up "stackable dma_ops". But this > > > patchset doesn't solve the problem that a hardware IOMMU does not > > > handle all devices (it was already solved with per-device-dma-ops). > > > > > > If paravirtualized device passthrough still needs to call multiple > > > dma_ops, then this patchset doesn't solve that issue. > > > > Ok, the name "stackable" is misleading and was a bad choice. I will > > rename it to "multiplexing". This should make it more clear what it is. > > Like you pointed out, the problems are solved with per-device dma_ops, > > but in the current implementation it needs special hacks in the IOMMU > > drivers to use these per-device dma_ops. > > I see this patchset as a continuation of the per-device dma_ops idea. It > > moves the per-device handling out of the specific drivers to a common > > place. So we can avoid or remove special hacks in the IOMMU drivers. > > Basically, I'm not against this patchset. It simplify Calgary and AMD > IOMMUs code to set up per-device-dma-ops (though it makes dma_ops a > bit complicated). Yes. But mind that this patchset adds complexity to one point (at dma_ops initialization) while we can avoid and remove it at many other places (in the dma_ops drivers). > But it doesn't solve any problems including the paravirtualized device > passthrough. When I wrote per-device-dma-ops, I expected that KVM > people want more changes (such as stackable dma_ops) to dma_ops for > the paravirtualized device passthrough. I'd like to hear what they > want first. Sure. Therefore this patchset is RFC and I cc'ed them. Joerg -- | AMD Saxony Limited Liability Company & Co. KG Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany System| Register Court Dresden: HRA 4896 Research | General Partner authorized to represent: Center| AMD Saxony LLC (Wilmington, Delaware, US) | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, 29 Sep 2008 15:26:47 +0200 Joerg Roedel <[EMAIL PROTECTED]> wrote: > On Mon, Sep 29, 2008 at 10:16:39PM +0900, FUJITA Tomonori wrote: > > On Sun, 28 Sep 2008 20:49:26 +0200 > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > On Sun, Sep 28, 2008 at 11:21:26PM +0900, FUJITA Tomonori wrote: > > > > On Mon, 22 Sep 2008 20:21:12 +0200 > > > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > > > > > Hi, > > > > > > > > > > this patch series implements stackable dma_ops on x86. This is useful > > > > > to > > > > > be able to fall back to a different dma_ops implementation if one can > > > > > not handle a particular device (as necessary for example with > > > > > paravirtualized device passthrough or if a hardware IOMMU only > > > > > handles a > > > > > subset of available devices). > > > > > > > > We already handle the latter. This patchset is more flexible but > > > > seems to incur more overheads. > > > > > > > > This feature will be used for only paravirtualized device passthrough? > > > > If so, I feel that there is more simpler (and specific) solutions for > > > > it. > > > > > > Its not only for device passthrough. It handles also the cases where a > > > hardware IOMMU does not handle all devices in the system (like in some > > > Calgary systems but also possible with AMD IOMMU). With this patchset we > > > > I know that. As I wrote in the previous mail, we already solved that > > problem with per-device-dma-ops. > > > > My question is what unsolved problems this patchset can fix? > > > > > > This patchset is named "stackable dma_ops" but it's different from > > what we discussed as "stackable dma_ops". This patchset provides > > IOMMUs a generic mechanism to set up "stackable dma_ops". But this > > patchset doesn't solve the problem that a hardware IOMMU does not > > handle all devices (it was already solved with per-device-dma-ops). > > > > If paravirtualized device passthrough still needs to call multiple > > dma_ops, then this patchset doesn't solve that issue. > > Ok, the name "stackable" is misleading and was a bad choice. I will > rename it to "multiplexing". This should make it more clear what it is. > Like you pointed out, the problems are solved with per-device dma_ops, > but in the current implementation it needs special hacks in the IOMMU > drivers to use these per-device dma_ops. > I see this patchset as a continuation of the per-device dma_ops idea. It > moves the per-device handling out of the specific drivers to a common > place. So we can avoid or remove special hacks in the IOMMU drivers. Basically, I'm not against this patchset. It simplify Calgary and AMD IOMMUs code to set up per-device-dma-ops (though it makes dma_ops a bit complicated). But it doesn't solve any problems including the paravirtualized device passthrough. When I wrote per-device-dma-ops, I expected that KVM people want more changes (such as stackable dma_ops) to dma_ops for the paravirtualized device passthrough. I'd like to hear what they want first. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, Sep 29, 2008 at 10:16:39PM +0900, FUJITA Tomonori wrote: > On Sun, 28 Sep 2008 20:49:26 +0200 > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > On Sun, Sep 28, 2008 at 11:21:26PM +0900, FUJITA Tomonori wrote: > > > On Mon, 22 Sep 2008 20:21:12 +0200 > > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > > > Hi, > > > > > > > > this patch series implements stackable dma_ops on x86. This is useful to > > > > be able to fall back to a different dma_ops implementation if one can > > > > not handle a particular device (as necessary for example with > > > > paravirtualized device passthrough or if a hardware IOMMU only handles a > > > > subset of available devices). > > > > > > We already handle the latter. This patchset is more flexible but > > > seems to incur more overheads. > > > > > > This feature will be used for only paravirtualized device passthrough? > > > If so, I feel that there is more simpler (and specific) solutions for > > > it. > > > > Its not only for device passthrough. It handles also the cases where a > > hardware IOMMU does not handle all devices in the system (like in some > > Calgary systems but also possible with AMD IOMMU). With this patchset we > > I know that. As I wrote in the previous mail, we already solved that > problem with per-device-dma-ops. > > My question is what unsolved problems this patchset can fix? > > > This patchset is named "stackable dma_ops" but it's different from > what we discussed as "stackable dma_ops". This patchset provides > IOMMUs a generic mechanism to set up "stackable dma_ops". But this > patchset doesn't solve the problem that a hardware IOMMU does not > handle all devices (it was already solved with per-device-dma-ops). > > If paravirtualized device passthrough still needs to call multiple > dma_ops, then this patchset doesn't solve that issue. Ok, the name "stackable" is misleading and was a bad choice. I will rename it to "multiplexing". This should make it more clear what it is. Like you pointed out, the problems are solved with per-device dma_ops, but in the current implementation it needs special hacks in the IOMMU drivers to use these per-device dma_ops. I see this patchset as a continuation of the per-device dma_ops idea. It moves the per-device handling out of the specific drivers to a common place. So we can avoid or remove special hacks in the IOMMU drivers. Joerg -- | AMD Saxony Limited Liability Company & Co. KG Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany System| Register Court Dresden: HRA 4896 Research | General Partner authorized to represent: Center| AMD Saxony LLC (Wilmington, Delaware, US) | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Sun, 28 Sep 2008 20:49:26 +0200 Joerg Roedel <[EMAIL PROTECTED]> wrote: > On Sun, Sep 28, 2008 at 11:21:26PM +0900, FUJITA Tomonori wrote: > > On Mon, 22 Sep 2008 20:21:12 +0200 > > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > > > Hi, > > > > > > this patch series implements stackable dma_ops on x86. This is useful to > > > be able to fall back to a different dma_ops implementation if one can > > > not handle a particular device (as necessary for example with > > > paravirtualized device passthrough or if a hardware IOMMU only handles a > > > subset of available devices). > > > > We already handle the latter. This patchset is more flexible but > > seems to incur more overheads. > > > > This feature will be used for only paravirtualized device passthrough? > > If so, I feel that there is more simpler (and specific) solutions for > > it. > > Its not only for device passthrough. It handles also the cases where a > hardware IOMMU does not handle all devices in the system (like in some > Calgary systems but also possible with AMD IOMMU). With this patchset we I know that. As I wrote in the previous mail, we already solved that problem with per-device-dma-ops. My question is what unsolved problems this patchset can fix? This patchset is named "stackable dma_ops" but it's different from what we discussed as "stackable dma_ops". This patchset provides IOMMUs a generic mechanism to set up "stackable dma_ops". But this patchset doesn't solve the problem that a hardware IOMMU does not handle all devices (it was already solved with per-device-dma-ops). If paravirtualized device passthrough still needs to call multiple dma_ops, then this patchset doesn't solve that issue. > can handle these cases in a generic way without hacking it into the > hardware drivers (these hacks are also in the AMD IOMMU code and I plan > to remove them in the case this patchset will be accepted). -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Sun, Sep 28, 2008 at 11:21:26PM +0900, FUJITA Tomonori wrote: > On Mon, 22 Sep 2008 20:21:12 +0200 > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > Hi, > > > > this patch series implements stackable dma_ops on x86. This is useful to > > be able to fall back to a different dma_ops implementation if one can > > not handle a particular device (as necessary for example with > > paravirtualized device passthrough or if a hardware IOMMU only handles a > > subset of available devices). > > We already handle the latter. This patchset is more flexible but > seems to incur more overheads. > > This feature will be used for only paravirtualized device passthrough? > If so, I feel that there is more simpler (and specific) solutions for > it. Its not only for device passthrough. It handles also the cases where a hardware IOMMU does not handle all devices in the system (like in some Calgary systems but also possible with AMD IOMMU). With this patchset we can handle these cases in a generic way without hacking it into the hardware drivers (these hacks are also in the AMD IOMMU code and I plan to remove them in the case this patchset will be accepted). Joerg -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, 22 Sep 2008 20:21:12 +0200 Joerg Roedel <[EMAIL PROTECTED]> wrote: > Hi, > > this patch series implements stackable dma_ops on x86. This is useful to > be able to fall back to a different dma_ops implementation if one can > not handle a particular device (as necessary for example with > paravirtualized device passthrough or if a hardware IOMMU only handles a > subset of available devices). We already handle the latter. This patchset is more flexible but seems to incur more overheads. This feature will be used for only paravirtualized device passthrough? If so, I feel that there is more simpler (and specific) solutions for it. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, 22 Sep 2008 19:41:28 -0700 Jeremy Fitzhardinge <[EMAIL PROTECTED]> wrote: > Joerg Roedel wrote: > > Its implemented using the per-device dma-ops already there. With > > this patches there is a list of available dma_ops implementations > > which are asked in a particular order if they can handle the > > device. The first implementation which returns true is assigned to > > the device as the per-device dma_ops structure. > > > > (Hmm, maybe the name stackable is misleading, is "dma_ops > > multiplexing" better?) > > Is per-device the right level? Wouldn't per-bus make more sense? not really; all DMA functions get a device as argument already anyway; just going to bus makes no sense there. Even if you set it the same for the whole bus almost all of the time... the APIs just work per device. (and device assignment clearly is per device as well) -- Arjan van de VenIntel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
Joerg Roedel wrote: > Its implemented using the per-device dma-ops already there. With this > patches there is a list of available dma_ops implementations which are > asked in a particular order if they can handle the device. The first > implementation which returns true is assigned to the device as the > per-device dma_ops structure. > > (Hmm, maybe the name stackable is misleading, is "dma_ops multiplexing" > better?) Is per-device the right level? Wouldn't per-bus make more sense? How does a dma_ops implementation "know" whether it can handle a particular device? (I haven't had a chance to read the patches yet.) J -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, Sep 22, 2008 at 11:36:19AM -0700, Arjan van de Ven wrote: > On Mon, 22 Sep 2008 20:21:12 +0200 > Joerg Roedel <[EMAIL PROTECTED]> wrote: > > > Hi, > > > > this patch series implements stackable dma_ops on x86. This is useful > > to be able to fall back to a different dma_ops implementation if one > > can not handle a particular device (as necessary for example with > > paravirtualized device passthrough or if a hardware IOMMU only > > handles a subset of available devices). > > isn't the right answer here to have a per device DMA ops instead ? Its implemented using the per-device dma-ops already there. With this patches there is a list of available dma_ops implementations which are asked in a particular order if they can handle the device. The first implementation which returns true is assigned to the device as the per-device dma_ops structure. (Hmm, maybe the name stackable is misleading, is "dma_ops multiplexing" better?) Joerg -- | AMD Saxony Limited Liability Company & Co. KG Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany System| Register Court Dresden: HRA 4896 Research | General Partner authorized to represent: Center| AMD Saxony LLC (Wilmington, Delaware, US) | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/9][RFC] stackable dma_ops for x86
On Mon, 22 Sep 2008 20:21:12 +0200 Joerg Roedel <[EMAIL PROTECTED]> wrote: > Hi, > > this patch series implements stackable dma_ops on x86. This is useful > to be able to fall back to a different dma_ops implementation if one > can not handle a particular device (as necessary for example with > paravirtualized device passthrough or if a hardware IOMMU only > handles a subset of available devices). isn't the right answer here to have a per device DMA ops instead ? -- Arjan van de VenIntel Open Source Technology Centre For development, discussion and tips for power savings, visit http://www.lesswatts.org -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/9][RFC] stackable dma_ops for x86
Hi, this patch series implements stackable dma_ops on x86. This is useful to be able to fall back to a different dma_ops implementation if one can not handle a particular device (as necessary for example with paravirtualized device passthrough or if a hardware IOMMU only handles a subset of available devices). The patch series is split to change the different iommu implementations in different patches. This should make it easier for the specific maintainers to review the part that changes their code. Thanks for review and comments, Joerg diffstat: arch/x86/kernel/amd_iommu.c |5 ++- arch/x86/kernel/pci-calgary_64.c | 21 ++-- arch/x86/kernel/pci-dma.c| 63 +- arch/x86/kernel/pci-gart_64.c|9 + arch/x86/kernel/pci-nommu.c | 12 +++ arch/x86/kernel/pci-swiotlb_64.c | 16 +- drivers/pci/intel-iommu.c|8 + include/asm-x86/device.h |6 ++-- include/asm-x86/dma-mapping.h| 43 + include/asm-x86/swiotlb.h|1 + 10 files changed, 160 insertions(+), 24 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
