Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On Wed, Jun 01, 2011 at 08:45:56AM -0700, Richard Henderson wrote: On 06/01/2011 08:35 AM, Eduard - Gabriel Munteanu wrote: Maybe it's not nice, but you're missing the fact upcasting gives you some type safety. With opaques you have none. Lol. Do you understand what container_of does? This is not dynamic_cast with RTTI. You can put any type name in there that you like, so long as it has a field name to match. The type of the field you give doesn't even have to match the type of the pointer that you pass in. Uh, if that's true, that's a bug in the container_of implementation. The ccan container_of implementation, for example, certainly does check that the given field has type matching the pointer. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On Wed, Jun 01, 2011 at 08:16:44AM -0700, Richard Henderson wrote: On 06/01/2011 07:29 AM, Avi Kivity wrote: On 06/01/2011 05:01 PM, Richard Henderson wrote: +err = dev-mmu-translate(dev, addr,paddr,plen, is_write); I see you didn't take my suggestion for using an opaque callback pointer. Really and truly, I won't be able to use this as-is for Alpha. Rather than opaques, please pass the DMA engine itself and use container_of(). The dma engine object is currently sitting in the PCIBus structure. Which is private, and can't be extended by a host bridge implementation. The entire code could be re-arranged, true, but please suggest something reasonable. We should be removing opaques, not adding them. See my followup elsewhere. Opaques *can* be cleaner than upcasting, particularly if there are too many hoops through which to jump. So, in the meantime, I've also done a version of Eduard's earlier patches, with added support for the PAPR hypervisor managed IOMMU. I have also significantly reworked how the structure lookup works, partly because in my case I'l looking at IOMMU translation for non-PCI devices, but I think it may also address your concerns. I'm still using upcasts, but there are less steps from the device to the IOMMU state. I've been sick and haven't had a chance to merge my stuff with Eduard's changes. I'll post them anyway, as another discussion point. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On 05/31/2011 06:38 PM, Eduard - Gabriel Munteanu wrote: +static inline void dma_memory_rw(DMADevice *dev, + dma_addr_t addr, + void *buf, + dma_addr_t len, + int is_write) I don't think this needs to be inline... +{ +/* + * Fast-path non-iommu. + * More importantly, makes it obvious what this function does. + */ +if (!dev || !dev-mmu) { +cpu_physical_memory_rw(addr, buf, len, is_write); +return; +} ... because you'll never be able to eliminate the if or the calls. You might as well make the overall code smaller by taking the entire function out of line. +#define DEFINE_DMA_LD(prefix, suffix, devtype, dmafield, size)\ +static inline uint##size##_t \ +dma_ld##suffix(DMADevice *dev, dma_addr_t addr) \ +{ \ +int err; \ +dma_addr_t paddr, plen; \ + \ +if (!dev || !dev-mmu) { \ +return ld##suffix##_phys(addr); \ +} \ Similarly for all the ld/st functions. +#define DEFINE_DMA_MEMORY_RW(prefix, devtype, dmafield) +#define DEFINE_DMA_MEMORY_READ(prefix, devtype, dmafield) +#define DEFINE_DMA_MEMORY_WRITE(prefix, devtype, dmafield) + +#define DEFINE_DMA_OPS(prefix, devtype, dmafield) \ I think this is a bit over the top, really. +err = dev-mmu-translate(dev, addr, paddr, plen, is_write); I see you didn't take my suggestion for using an opaque callback pointer. Really and truly, I won't be able to use this as-is for Alpha. r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On 06/01/2011 05:01 PM, Richard Henderson wrote: +err = dev-mmu-translate(dev, addr,paddr,plen, is_write); I see you didn't take my suggestion for using an opaque callback pointer. Really and truly, I won't be able to use this as-is for Alpha. Rather than opaques, please pass the DMA engine itself and use container_of(). We should be removing opaques, not adding them. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On Wed, Jun 01, 2011 at 07:01:42AM -0700, Richard Henderson wrote: On 05/31/2011 06:38 PM, Eduard - Gabriel Munteanu wrote: +static inline void dma_memory_rw(DMADevice *dev, + dma_addr_t addr, + void *buf, + dma_addr_t len, + int is_write) I don't think this needs to be inline... +{ +/* + * Fast-path non-iommu. + * More importantly, makes it obvious what this function does. + */ +if (!dev || !dev-mmu) { +cpu_physical_memory_rw(addr, buf, len, is_write); +return; +} ... because you'll never be able to eliminate the if or the calls. You might as well make the overall code smaller by taking the entire function out of line. +#define DEFINE_DMA_LD(prefix, suffix, devtype, dmafield, size)\ +static inline uint##size##_t \ +dma_ld##suffix(DMADevice *dev, dma_addr_t addr) \ +{ \ +int err; \ +dma_addr_t paddr, plen; \ + \ +if (!dev || !dev-mmu) { \ +return ld##suffix##_phys(addr); \ +} \ Similarly for all the ld/st functions. The idea was to get to the fastpath as soon as possible. I'm not really concerned about the case where there's an IOMMU present, since translation/checking does a lot more work. But other people might be worried about that additional function call when there's no IOMMU. And these functions are quite small anyway. Thoughts, anybody else? +#define DEFINE_DMA_MEMORY_RW(prefix, devtype, dmafield) +#define DEFINE_DMA_MEMORY_READ(prefix, devtype, dmafield) +#define DEFINE_DMA_MEMORY_WRITE(prefix, devtype, dmafield) + +#define DEFINE_DMA_OPS(prefix, devtype, dmafield) \ I think this is a bit over the top, really. Yeah, it's a bit unconventional, but why do you think that? The main selling point is there are more chances to screw up if every bus layer implements these manually. And it's really convenient, especially if we get to add another ld/st. I do have one concern about it, though: it might increase compile time due to additional preprocessing work. I haven't done any benchmarks on that. But apart from this, are there any other objections? +err = dev-mmu-translate(dev, addr, paddr, plen, is_write); I see you didn't take my suggestion for using an opaque callback pointer. Really and truly, I won't be able to use this as-is for Alpha. If I understand correctly you need some sort of shared state between IOMMUs or units residing on different buses. Then you should be able to get to it even with this API, just like I do with my AMD IOMMU state by upcasting. It doesn't seem to matter whether you've got an opaque, that opaque could very well be reachable by upcasting. Did I get this wrong? Eduard r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On 06/01/2011 07:52 AM, Eduard - Gabriel Munteanu wrote: The main selling point is there are more chances to screw up if every bus layer implements these manually. And it's really convenient, especially if we get to add another ld/st. If we drop the ld/st, we're talking about 5 lines for every bus layer. If I recall, there was just the one driver that actually uses the ld/st interface; most used the read/write interface. If I understand correctly you need some sort of shared state between IOMMUs or units residing on different buses. Then you should be able to get to it even with this API, just like I do with my AMD IOMMU state by upcasting. It doesn't seem to matter whether you've got an opaque, that opaque could very well be reachable by upcasting. Did I get this wrong? Can you honestly tell me that +static int amd_iommu_translate(DMADevice *dev, + dma_addr_t addr, + dma_addr_t *paddr, + dma_addr_t *len, + int is_write) +{ +PCIDevice *pci_dev = container_of(dev, PCIDevice, dma); +PCIDevice *iommu_dev = DO_UPCAST(PCIDevice, qdev, dev-mmu-iommu); +AMDIOMMUState *s = DO_UPCAST(AMDIOMMUState, dev, iommu_dev); THREE (3) upcasts is a sane to write maintainable software? The margin for error here is absolutely enormous. If you had just passed in that AMDIOMMUState* as the opaque value, it would be trivial to look at the initialization statement and the callback function to verify that the right value is being passed. r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On 06/01/2011 07:29 AM, Avi Kivity wrote: On 06/01/2011 05:01 PM, Richard Henderson wrote: +err = dev-mmu-translate(dev, addr,paddr,plen, is_write); I see you didn't take my suggestion for using an opaque callback pointer. Really and truly, I won't be able to use this as-is for Alpha. Rather than opaques, please pass the DMA engine itself and use container_of(). The dma engine object is currently sitting in the PCIBus structure. Which is private, and can't be extended by a host bridge implementation. The entire code could be re-arranged, true, but please suggest something reasonable. We should be removing opaques, not adding them. See my followup elsewhere. Opaques *can* be cleaner than upcasting, particularly if there are too many hoops through which to jump. r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On Wed, Jun 01, 2011 at 08:09:29AM -0700, Richard Henderson wrote: On 06/01/2011 07:52 AM, Eduard - Gabriel Munteanu wrote: The main selling point is there are more chances to screw up if every bus layer implements these manually. And it's really convenient, especially if we get to add another ld/st. If we drop the ld/st, we're talking about 5 lines for every bus layer. If I recall, there was just the one driver that actually uses the ld/st interface; most used the read/write interface. Hm, indeed there seem to be far fewer uses of those now, actually my patches don't seem to be using those. What do you guys think? Will these go away completely? If I understand correctly you need some sort of shared state between IOMMUs or units residing on different buses. Then you should be able to get to it even with this API, just like I do with my AMD IOMMU state by upcasting. It doesn't seem to matter whether you've got an opaque, that opaque could very well be reachable by upcasting. Did I get this wrong? Can you honestly tell me that +static int amd_iommu_translate(DMADevice *dev, + dma_addr_t addr, + dma_addr_t *paddr, + dma_addr_t *len, + int is_write) +{ +PCIDevice *pci_dev = container_of(dev, PCIDevice, dma); +PCIDevice *iommu_dev = DO_UPCAST(PCIDevice, qdev, dev-mmu-iommu); +AMDIOMMUState *s = DO_UPCAST(AMDIOMMUState, dev, iommu_dev); THREE (3) upcasts is a sane to write maintainable software? The margin for error here is absolutely enormous. If you had just passed in that AMDIOMMUState* as the opaque value, it would be trivial to look at the initialization statement and the callback function to verify that the right value is being passed. Maybe it's not nice, but you're missing the fact upcasting gives you some type safety. With opaques you have none. Plus you also get the PCI device that made the call while you're at it. Eduard r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 01/13] Generic DMA memory access interface
On 06/01/2011 08:35 AM, Eduard - Gabriel Munteanu wrote: Maybe it's not nice, but you're missing the fact upcasting gives you some type safety. With opaques you have none. Lol. Do you understand what container_of does? This is not dynamic_cast with RTTI. You can put any type name in there that you like, so long as it has a field name to match. The type of the field you give doesn't even have to match the type of the pointer that you pass in. Type safety this is not. r~ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html