On Fri, May 29, 2015 at 06:44:40PM +1000, Alexey Kardashevskiy wrote:
>Modern IBM POWERPC systems support multiple (currently two) TCE tables
>per IOMMU group (a.k.a. PE). This adds a iommu_table_group container
>for TCE tables. Right now just one table is supported.
>
>This defines iommu_table_group struct which stores pointers to
>iommu_group and iommu_table(s). This replaces iommu_table with
>iommu_table_group where iommu_table was used to identify a group:
>- iommu_register_group();
>- iommudata of generic iommu_group;
>
>This removes @data from iommu_table as it_table_group provides
>same access to pnv_ioda_pe.
>
>For IODA, instead of embedding iommu_table, the new iommu_table_group
>keeps pointers to those. The iommu_table structs are allocated
>dynamically.
>
>For P5IOC2, both iommu_table_group and iommu_table are embedded into
>PE struct. As there is no EEH and SRIOV support for P5IOC2,
>iommu_free_table() should not be called on iommu_table struct pointers
>so we can keep it embedded in pnv_phb::p5ioc2.
>
>For pSeries, this replaces multiple calls of kzalloc_node() with a new
>iommu_pseries_alloc_group() helper and stores the table group struct
>pointer into the pci_dn struct. For release, a iommu_table_free_group()
>helper is added.
>
>This moves iommu_table struct allocation from SR-IOV code to
>the generic DMA initialization code in pnv_pci_ioda_setup_dma_pe and
>pnv_pci_ioda2_setup_dma_pe as this is where DMA is actually initialized.
>This change is here because those lines had to be changed anyway.
>
>This should cause no behavioural change.
>
>Signed-off-by: Alexey Kardashevskiy
>[aw: for the vfio related changes]
>Acked-by: Alex Williamson
Reviewed-by: Gavin Shan
>---
>Changes:
>v11:
>* iommu_table_group moved outside #ifdef CONFIG_IOMMU_API as iommu_table
>is dynamically allocated and it needs a pointer to PE and
>iommu_table_group is this pointer
>
>v10:
>* new to the series, separated from
>"powerpc/spapr: vfio: Switch from iommu_table to new iommu_table_group"
>* iommu_table is not embedded into iommu_table_group but allocated
>dynamically in most cases
>* iommu_table allocation is moved to a single place for IODA2's
>pnv_pci_ioda_setup_dma_pe where it belongs to
>* added list of groups into iommu_table; most of the code just looks at
>the first item to keep the patch simpler
>---
> arch/powerpc/include/asm/iommu.h| 19 ++---
> arch/powerpc/include/asm/pci-bridge.h | 2 +-
> arch/powerpc/kernel/iommu.c | 17 ++---
> arch/powerpc/platforms/powernv/pci-ioda.c | 55 +++---
> arch/powerpc/platforms/powernv/pci-p5ioc2.c | 18 +++--
> arch/powerpc/platforms/powernv/pci.h| 3 +-
> arch/powerpc/platforms/pseries/iommu.c | 107 +++-
> drivers/vfio/vfio_iommu_spapr_tce.c | 23 +++---
> 8 files changed, 152 insertions(+), 92 deletions(-)
>
>diff --git a/arch/powerpc/include/asm/iommu.h
>b/arch/powerpc/include/asm/iommu.h
>index e2a45c3..5a7267f 100644
>--- a/arch/powerpc/include/asm/iommu.h
>+++ b/arch/powerpc/include/asm/iommu.h
>@@ -91,14 +91,9 @@ struct iommu_table {
> struct iommu_pool pools[IOMMU_NR_POOLS];
> unsigned long *it_map; /* A simple allocation bitmap for now */
> unsigned long it_page_shift;/* table iommu page size */
>-#ifdef CONFIG_IOMMU_API
>- struct iommu_group *it_group;
>-#endif
>+ struct iommu_table_group *it_table_group;
> struct iommu_table_ops *it_ops;
> void (*set_bypass)(struct iommu_table *tbl, bool enable);
>-#ifdef CONFIG_PPC_POWERNV
>- void *data;
>-#endif
> };
>
> /* Pure 2^n version of get_order */
>@@ -129,14 +124,22 @@ extern void iommu_free_table(struct iommu_table *tbl,
>const char *node_name);
> */
> extern struct iommu_table *iommu_init_table(struct iommu_table * tbl,
> int nid);
>+#define IOMMU_TABLE_GROUP_MAX_TABLES 1
>+
>+struct iommu_table_group {
>+ struct iommu_group *group;
>+ struct iommu_table *tables[IOMMU_TABLE_GROUP_MAX_TABLES];
>+};
>+
Number of TCE tables supported in group would be worthy to be
changed dynamically in long run, but not for now. P7IOC has one
table per group while PHB3 has two tables per group.
Thanks,
Gavin
> #ifdef CONFIG_IOMMU_API
>-extern void iommu_register_group(struct iommu_table *tbl,
>+
>+extern void iommu_register_group(struct iommu_table_group *table_group,
>int pci_domain_number, unsigned long pe_num);
> extern int iommu_add_device(struct device *dev);
> extern void iommu_del_device(struct device *dev);
> extern int __init tce_iommu_bus_notifier_init(void);
> #else
>-static inline void iommu_register_group(struct iommu_table *tbl,
>+static inline void iommu_register_group(struct iommu_table_group *table_group,
> int pci_domain_number,
> unsigned long pe_num)
> {
>diff --git a/arch/powerpc/include/as
