Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive

2018-08-09 Thread Kenneth Lee
On Thu, Aug 09, 2018 at 10:46:13AM -0400, Jerome Glisse wrote: > Date: Thu, 9 Aug 2018 10:46:13 -0400 > From: Jerome Glisse > To: Kenneth Lee > CC: Kenneth Lee , "Tian, Kevin" > , Alex Williamson , > Herbert Xu , "k...@vger.kernel.org" > , Jonathan Corbet , Greg > Kroah-Hartman , Zaibo Xu ,

Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive

2018-08-09 Thread Kenneth Lee
On Thu, Aug 09, 2018 at 08:31:31AM +, Tian, Kevin wrote: > Date: Thu, 9 Aug 2018 08:31:31 + > From: "Tian, Kevin" > To: Kenneth Lee , Jerome Glisse > CC: Kenneth Lee , Alex Williamson > , Herbert Xu , > "k...@vger.kernel.org" , Jonathan Corbet > , Greg Kroah-Hartman , Zaibo > Xu ,

Re: [PATCH] vfio/pci: Some buggy virtual functions incorrectly report 1 for intx.

2018-08-09 Thread Raj, Ashok
On Thu, Aug 09, 2018 at 01:44:17PM -0600, Alex Williamson wrote: > On Thu, 9 Aug 2018 12:37:06 -0700 > Ashok Raj wrote: > > > PCI_INTERRUPT_PIN should always read 0 for SRIOV Virtual Functions. > > > > Some SRIOV devices have some bugs in RTL and VF's end up reading 1 > > instead of 0 for the

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Kit Chow
On 08/09/2018 03:50 PM, Logan Gunthorpe wrote: On 09/08/18 04:48 PM, Kit Chow wrote: Based on Logan's comments, I am very hopeful that the dma_map_resource will make things work on the older platforms... Well, I *think* dma_map_single() would still work. So I'm not that confident that's the

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Logan Gunthorpe
On 09/08/18 04:48 PM, Kit Chow wrote: > Based on Logan's comments, I am very hopeful that the dma_map_resource > will make things work on the older platforms... Well, I *think* dma_map_single() would still work. So I'm not that confident that's the root of your problem. I'd still like to see

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Kit Chow
On 08/09/2018 03:40 PM, Jiang, Dave wrote: -Original Message- From: linux-pci-ow...@vger.kernel.org [mailto:linux-pci-ow...@vger.kernel.org] On Behalf Of Kit Chow Sent: Thursday, August 9, 2018 2:48 PM To: Logan Gunthorpe ; Eric Pilmore ; Bjorn Helgaas Cc: linux-...@vger.kernel.org;

RE: IOAT DMA w/IOMMU

2018-08-09 Thread Jiang, Dave
> -Original Message- > From: linux-pci-ow...@vger.kernel.org > [mailto:linux-pci-ow...@vger.kernel.org] On Behalf Of Kit Chow > Sent: Thursday, August 9, 2018 2:48 PM > To: Logan Gunthorpe ; Eric Pilmore > ; Bjorn Helgaas > Cc: linux-...@vger.kernel.org; David Woodhouse ; Alex >

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Kit Chow
On 08/09/2018 02:11 PM, Logan Gunthorpe wrote: On 09/08/18 02:57 PM, Kit Chow wrote: On 08/09/2018 01:11 PM, Logan Gunthorpe wrote: On 09/08/18 01:47 PM, Kit Chow wrote: I haven't tested this scenario but my guess would be that IOAT would indeed go through the IOMMU and the PCI BAR

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Logan Gunthorpe
On 09/08/18 03:31 PM, Eric Pilmore wrote: > On Thu, Aug 9, 2018 at 12:35 PM, Logan Gunthorpe wrote: >> Hey, >> >> On 09/08/18 12:51 PM, Eric Pilmore wrote: > Was wondering if anybody here has used IOAT DMA engines with an > IOMMU turned on (Xeon based system)? My specific question is

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Eric Pilmore
On Thu, Aug 9, 2018 at 12:35 PM, Logan Gunthorpe wrote: > Hey, > > On 09/08/18 12:51 PM, Eric Pilmore wrote: Was wondering if anybody here has used IOAT DMA engines with an IOMMU turned on (Xeon based system)? My specific question is really whether it is possible to DMA (w/IOAT) to

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Logan Gunthorpe
On 09/08/18 02:57 PM, Kit Chow wrote: > > > On 08/09/2018 01:11 PM, Logan Gunthorpe wrote: >> >> On 09/08/18 01:47 PM, Kit Chow wrote: I haven't tested this scenario but my guess would be that IOAT would indeed go through the IOMMU and the PCI BAR address would need to be

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Kit Chow
On 08/09/2018 01:11 PM, Logan Gunthorpe wrote: On 09/08/18 01:47 PM, Kit Chow wrote: I haven't tested this scenario but my guess would be that IOAT would indeed go through the IOMMU and the PCI BAR address would need to be properly mapped into the IOAT's IOVA. The fact that you see DMAR

Re: [PATCH] iommu/iova: Optimise attempts to allocate iova from 32bit address range

2018-08-09 Thread Robin Murphy
On 2018-08-09 6:49 PM, Ganapatrao Kulkarni wrote: Hi Robin, On Thu, Aug 9, 2018 at 9:54 PM, Robin Murphy wrote: On 07/08/18 09:54, Ganapatrao Kulkarni wrote: As an optimisation for PCI devices, there is always first attempt been made to allocate iova from SAC address range. This will lead

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Logan Gunthorpe
On 09/08/18 01:47 PM, Kit Chow wrote: >> I haven't tested this scenario but my guess would be that IOAT would >> indeed go through the IOMMU and the PCI BAR address would need to be >> properly mapped into the IOAT's IOVA. The fact that you see DMAR errors >> is probably a good indication that

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Logan Gunthorpe
Hey, On 09/08/18 12:51 PM, Eric Pilmore wrote: >>> Was wondering if anybody here has used IOAT DMA engines with an >>> IOMMU turned on (Xeon based system)? My specific question is really >>> whether it is possible to DMA (w/IOAT) to a PCI BAR address as the >>> destination without having to map

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Kit Chow
On 08/09/2018 12:35 PM, Logan Gunthorpe wrote: Hey, On 09/08/18 12:51 PM, Eric Pilmore wrote: Was wondering if anybody here has used IOAT DMA engines with an IOMMU turned on (Xeon based system)? My specific question is really whether it is possible to DMA (w/IOAT) to a PCI BAR address as the

Re: [PATCH] vfio/pci: Some buggy virtual functions incorrectly report 1 for intx.

2018-08-09 Thread Alex Williamson
On Thu, 9 Aug 2018 12:37:06 -0700 Ashok Raj wrote: > PCI_INTERRUPT_PIN should always read 0 for SRIOV Virtual Functions. > > Some SRIOV devices have some bugs in RTL and VF's end up reading 1 > instead of 0 for the PIN. Hi Ashok, One question, can we identify which VFs are known to have

[PATCH] vfio/pci: Some buggy virtual functions incorrectly report 1 for intx.

2018-08-09 Thread Ashok Raj
PCI_INTERRUPT_PIN should always read 0 for SRIOV Virtual Functions. Some SRIOV devices have some bugs in RTL and VF's end up reading 1 instead of 0 for the PIN. Since this is a spec required value, rather than having a device specific quirk, we could fix it permanently in vfio. Reworked

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Eric Pilmore
On Thu, Aug 9, 2018 at 11:43 AM, Bjorn Helgaas wrote: > [+cc David, Logan, Alex, iommu list] > > On Thu, Aug 09, 2018 at 11:14:13AM -0700, Eric Pilmore wrote: >> Didn't get any response on the IRC channel so trying here. >> >> Was wondering if anybody here has used IOAT DMA engines with an >>

Re: IOAT DMA w/IOMMU

2018-08-09 Thread Bjorn Helgaas
[+cc David, Logan, Alex, iommu list] On Thu, Aug 09, 2018 at 11:14:13AM -0700, Eric Pilmore wrote: > Didn't get any response on the IRC channel so trying here. > > Was wondering if anybody here has used IOAT DMA engines with an > IOMMU turned on (Xeon based system)? My specific question is

Re: [PATCH] iommu/iova: Optimise attempts to allocate iova from 32bit address range

2018-08-09 Thread Ganapatrao Kulkarni
Hi Robin, On Thu, Aug 9, 2018 at 9:54 PM, Robin Murphy wrote: > On 07/08/18 09:54, Ganapatrao Kulkarni wrote: >> >> As an optimisation for PCI devices, there is always first attempt >> been made to allocate iova from SAC address range. This will lead >> to unnecessary attempts/function calls,

Re: [PATCH] iommu/iova: Optimise attempts to allocate iova from 32bit address range

2018-08-09 Thread Robin Murphy
On 07/08/18 09:54, Ganapatrao Kulkarni wrote: As an optimisation for PCI devices, there is always first attempt been made to allocate iova from SAC address range. This will lead to unnecessary attempts/function calls, when there are no free ranges available. This patch optimises by adding flag

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Dmitry Osipenko
On Thursday, 9 August 2018 17:52:06 MSK Thierry Reding wrote: > On Thu, Aug 09, 2018 at 05:22:59PM +0300, Dmitry Osipenko wrote: > > On Thursday, 9 August 2018 16:59:24 MSK Thierry Reding wrote: > > > On Thu, Aug 09, 2018 at 02:39:03PM +0300, Dmitry Osipenko wrote: > > > > On Thursday, 9 August

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Thierry Reding
On Thu, Aug 09, 2018 at 05:22:59PM +0300, Dmitry Osipenko wrote: > On Thursday, 9 August 2018 16:59:24 MSK Thierry Reding wrote: > > On Thu, Aug 09, 2018 at 02:39:03PM +0300, Dmitry Osipenko wrote: > > > On Thursday, 9 August 2018 14:17:46 MSK Thierry Reding wrote: > > > > On Sat, Aug 04, 2018 at

Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive

2018-08-09 Thread Jerome Glisse
On Thu, Aug 09, 2018 at 04:03:52PM +0800, Kenneth Lee wrote: > On Wed, Aug 08, 2018 at 11:18:35AM -0400, Jerome Glisse wrote: > > On Wed, Aug 08, 2018 at 09:08:42AM +0800, Kenneth Lee wrote: > > > 在 2018年08月06日 星期一 11:32 下午, Jerome Glisse 写道: > > > > On Mon, Aug 06, 2018 at 11:12:52AM +0800,

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Dmitry Osipenko
On Thursday, 9 August 2018 16:59:24 MSK Thierry Reding wrote: > On Thu, Aug 09, 2018 at 02:39:03PM +0300, Dmitry Osipenko wrote: > > On Thursday, 9 August 2018 14:17:46 MSK Thierry Reding wrote: > > > On Sat, Aug 04, 2018 at 05:29:57PM +0300, Dmitry Osipenko wrote: > > > > GART contain registers

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Thierry Reding
On Thu, Aug 09, 2018 at 02:39:03PM +0300, Dmitry Osipenko wrote: > On Thursday, 9 August 2018 14:17:46 MSK Thierry Reding wrote: > > On Sat, Aug 04, 2018 at 05:29:57PM +0300, Dmitry Osipenko wrote: > > > GART contain registers needed by the Memory Controller driver, provide > > > access to the MC

Re: [PATCH v2 2/2] iommu/arm-smmu-v3: avoid redundant CMD_SYNCs if possible

2018-08-09 Thread Robin Murphy
On 09/08/18 12:48, Zhen Lei wrote: More than two CMD_SYNCs maybe adjacent in the command queue, and the first one has done what others want to do. Drop the redundant CMD_SYNCs can improve IO performance especially under the pressure scene. I did the statistics in my test environment, the number

[PATCH v2 1/2] iommu/arm-smmu-v3: fix unexpected CMD_SYNC timeout

2018-08-09 Thread Zhen Lei
The condition "(int)(VAL - sync_idx) >= 0" to break loop in function __arm_smmu_sync_poll_msi requires that sync_idx must be increased monotonously according to the sequence of the CMDs in the cmdq. But ".msidata = atomic_inc_return_relaxed(>sync_nr)" is not protected by spinlock, so the

[PATCH v2 0/2] bugfix and optimization about CMD_SYNC

2018-08-09 Thread Zhen Lei
v1->v2: 1. move the call to arm_smmu_cmdq_build_cmd into the critical section, and keep itself unchange. 2. Although patch2 can make sure no two CMD_SYNCs will be adjacent, but patch1 is still needed, see below: cpu0cpu1cpu2 msidata=0

[PATCH v2 2/2] iommu/arm-smmu-v3: avoid redundant CMD_SYNCs if possible

2018-08-09 Thread Zhen Lei
More than two CMD_SYNCs maybe adjacent in the command queue, and the first one has done what others want to do. Drop the redundant CMD_SYNCs can improve IO performance especially under the pressure scene. I did the statistics in my test environment, the number of CMD_SYNCs can be reduced about

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Dmitry Osipenko
On Thursday, 9 August 2018 14:17:46 MSK Thierry Reding wrote: > On Sat, Aug 04, 2018 at 05:29:57PM +0300, Dmitry Osipenko wrote: > > GART contain registers needed by the Memory Controller driver, provide > > access to the MC driver by utilizing its GART-integration facility. > > > >

Re: [PATCH v4 3/5] iommu/io-pgtable-arm: add support for non-strict mode

2018-08-09 Thread Leizhen (ThunderTown)
On 2018/8/9 18:54, Robin Murphy wrote: > On 06/08/18 13:27, Zhen Lei wrote: >> To support the non-strict mode, now we only tlbi and sync for the strict >> mode. But for the non-leaf case, always follow strict mode. >> >> Signed-off-by: Zhen Lei >> --- >> drivers/iommu/io-pgtable-arm.c | 27

Re: [PATCH v2 2/8] iommu/tegra: gart: Provide access to Memory Controller driver

2018-08-09 Thread Thierry Reding
On Sat, Aug 04, 2018 at 05:29:57PM +0300, Dmitry Osipenko wrote: > GART contain registers needed by the Memory Controller driver, provide > access to the MC driver by utilizing its GART-integration facility. > > Signed-off-by: Dmitry Osipenko > --- > drivers/iommu/tegra-gart.c | 23

Re: [PATCH v2 1/8] memory: tegra: Provide facility for integration with the GART driver

2018-08-09 Thread Thierry Reding
On Sat, Aug 04, 2018 at 05:29:56PM +0300, Dmitry Osipenko wrote: > In order to report clients name and access direction on GART's page fault, > MC driver need to access GART registers. Add facility that provides access > to the GART. > > Signed-off-by: Dmitry Osipenko > --- >

Re: [PATCH v4 5/5] iommu/arm-smmu-v3: add bootup option "arm_iommu"

2018-08-09 Thread Robin Murphy
On 06/08/18 13:27, Zhen Lei wrote: Add a bootup option to make the system manager can choose which mode to be used. The default mode is strict. Signed-off-by: Zhen Lei --- Documentation/admin-guide/kernel-parameters.txt | 9 + drivers/iommu/arm-smmu-v3.c | 17

Re: [PATCH v4 4/5] iommu/arm-smmu-v3: add support for non-strict mode

2018-08-09 Thread Robin Murphy
On 06/08/18 13:27, Zhen Lei wrote: Dynamically choose strict or non-strict mode for page table config based on the iommu domain type. Signed-off-by: Zhen Lei --- drivers/iommu/arm-smmu-v3.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c

Re: [PATCH v4 2/5] iommu/dma: add support for non-strict mode

2018-08-09 Thread Leizhen (ThunderTown)
On 2018/8/9 18:46, Robin Murphy wrote: > On 06/08/18 13:27, Zhen Lei wrote: >> 1. Save the related domain pointer in struct iommu_dma_cookie, make iovad >> capable call domain->ops->flush_iotlb_all to flush TLB. >> 2. During the iommu domain initialization phase, base on domain->non_strict

Re: [PATCH v4 3/5] iommu/io-pgtable-arm: add support for non-strict mode

2018-08-09 Thread Robin Murphy
On 06/08/18 13:27, Zhen Lei wrote: To support the non-strict mode, now we only tlbi and sync for the strict mode. But for the non-leaf case, always follow strict mode. Signed-off-by: Zhen Lei --- drivers/iommu/io-pgtable-arm.c | 27 ++- drivers/iommu/io-pgtable.h

Re: [PATCH v4 2/5] iommu/dma: add support for non-strict mode

2018-08-09 Thread Robin Murphy
On 06/08/18 13:27, Zhen Lei wrote: 1. Save the related domain pointer in struct iommu_dma_cookie, make iovad capable call domain->ops->flush_iotlb_all to flush TLB. 2. During the iommu domain initialization phase, base on domain->non_strict field to check whether non-strict mode is

Re: [PATCH v4 1/5] iommu/arm-smmu-v3: fix the implementation of flush_iotlb_all hook

2018-08-09 Thread Robin Murphy
On 06/08/18 13:27, Zhen Lei wrote: .flush_iotlb_all can not just wait for previous tlbi operations to be completed, but should also invalid all TLBs of the related domain. I think it was like that because the only caller in practice was iommu_group_create_direct_mappings(), and at that point

Re: [PATCH 1/1] iommu/arm-smmu-v3: fix unexpected CMD_SYNC timeout

2018-08-09 Thread Leizhen (ThunderTown)
On 2018/8/9 16:49, Will Deacon wrote: > On Thu, Aug 09, 2018 at 09:30:51AM +0800, Leizhen (ThunderTown) wrote: >> On 2018/8/8 18:12, Will Deacon wrote: >>> On Mon, Aug 06, 2018 at 08:31:29PM +0800, Zhen Lei wrote: The condition "(int)(VAL - sync_idx) >= 0" to break loop in function

Re: [PATCH 1/1] iommu/arm-smmu-v3: fix unexpected CMD_SYNC timeout

2018-08-09 Thread Will Deacon
On Thu, Aug 09, 2018 at 09:30:51AM +0800, Leizhen (ThunderTown) wrote: > On 2018/8/8 18:12, Will Deacon wrote: > > On Mon, Aug 06, 2018 at 08:31:29PM +0800, Zhen Lei wrote: > >> The condition "(int)(VAL - sync_idx) >= 0" to break loop in function > >> __arm_smmu_sync_poll_msi requires that

RE: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive

2018-08-09 Thread Tian, Kevin
> From: Kenneth Lee [mailto:liguo...@hisilicon.com] > Sent: Thursday, August 9, 2018 4:04 PM > > But we have another requirement which is to combine some device > together to > share the same address space. This is a little like these kinds of solution: > >

Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive

2018-08-09 Thread Kenneth Lee
On Wed, Aug 08, 2018 at 11:18:35AM -0400, Jerome Glisse wrote: > Date: Wed, 8 Aug 2018 11:18:35 -0400 > From: Jerome Glisse > To: Kenneth Lee > CC: Kenneth Lee , "Tian, Kevin" > , Alex Williamson , > Herbert Xu , "k...@vger.kernel.org" > , Jonathan Corbet , Greg > Kroah-Hartman , Zaibo Xu ,