On Mon, Nov 16, 2015 at 12:42 AM, David Woodhouse wrote:
>
> On Sun, 2015-11-15 at 22:54 -0800, Benjamin Serebrin wrote:
> > We looked into Intel IOMMU performance a while ago and learned a few
> > useful things. We generally did a parallel 200 thread TCP_RR test,
> > as this churns through mappi
On Sun, 2015-11-15 at 22:54 -0800, Benjamin Serebrin wrote:
> We looked into Intel IOMMU performance a while ago and learned a few
> useful things. We generally did a parallel 200 thread TCP_RR test,
> as this churns through mappings quickly and uses all available cores.
>
> First, the main bottl
We looked into Intel IOMMU performance a while ago and learned a few
useful things. We generally did a parallel 200 thread TCP_RR test, as
this churns through mappings quickly and uses all available cores.
First, the main bottleneck was software performance[1]. This study
preceded the recent pat
On Thu, Nov 05, 2015 at 04:11:21PM -0500, David Miller wrote:
>
> And for the record Sowmini fixed a lot of the lock contention:
>
> commit ff7d37a502022149655c18035b99a53391be0383
> Author: Sowmini Varadhan
> Date: Thu Apr 9 15:33:30 2015 -0400
>
> Break up monolithic iommu table/lock in
From: Joerg Roedel
Date: Thu, 5 Nov 2015 14:42:06 +0100
> Contended IOMMU locks are not only a problem on SPARC, but on x86 and
> various other IOMMU drivers too. But I have some ideas on how to improve
> the situation there.
And for the record Sowmini fixed a lot of the lock contention:
commit
From: David Woodhouse
Date: Thu, 29 Oct 2015 22:35:25 +
> For the receive side, it shouldn't be beyond the wit of man to
> introduce an API which allocates *and* DMA-maps a skb. Pass it to
> netif_rx() still mapped, with a destructor that just shoves it back in
> a pool for re-use.
For forwa
On Mon, Nov 02, 2015 at 07:32:19PM +0200, Shamir Rabinovitch wrote:
> Correct. This issue is one of the concerns here in the previous replies.
> I will take different approach which will not require the IOMMU bypass
> per mapping. Will try to shift to the x86 'iommu=pt' approach.
Yeah, it doesn't
On Tue, 2015-11-03 at 14:11 +0100, Christoph Hellwig wrote:
> > xHCI for example, vs. something like 10G ethernet... but yes I agree it
> > sucks. I don't like that sort of policy anywhere in drivers. On the
> > other hand the platform doesn't have much information to make that sort
> > of decision
On Tue, Nov 03, 2015 at 10:08:13AM +1100, Benjamin Herrenschmidt wrote:
> On Mon, 2015-11-02 at 22:45 +0100, Arnd Bergmann wrote:
> > > Then I would argue for naming this differently. Make it an optional
> > > hint "DMA_ATTR_HIGH_PERF" or something like that. Whether this is
> > > achieved via usin
On Mon, 2015-11-02 at 22:45 +0100, Arnd Bergmann wrote:
> > Then I would argue for naming this differently. Make it an optional
> > hint "DMA_ATTR_HIGH_PERF" or something like that. Whether this is
> > achieved via using a bypass or other means in the backend not the
> > business of the driver.
> >
On Mon, 2015-11-02 at 22:48 +, David Woodhouse wrote:
>
> In the Intel case, the mapping setup is entirely per-device (except for
> crap devices and devices behind a PCIe-PCI bridge, etc.).
>
> So you can happily have a passthrough mapping for *one* device, without
> making that same mapping
On Mon, 2015-11-02 at 08:10 +1100, Benjamin Herrenschmidt wrote:
> On Sun, 2015-11-01 at 09:45 +0200, Shamir Rabinovitch wrote:
> > Not sure this use case is possible for Infiniband where application hold
> > the data buffers and there is no way to force application to re use the
> > buffer as sug
On Tue, Nov 03, 2015 at 07:13:28AM +1100, Benjamin Herrenschmidt wrote:
>
> Then I would argue for naming this differently. Make it an optional
> hint "DMA_ATTR_HIGH_PERF" or something like that. Whether this is
> achieved via using a bypass or other means in the backend not the
> business of the
On Tuesday 03 November 2015 07:13:28 Benjamin Herrenschmidt wrote:
> On Mon, 2015-11-02 at 14:07 +0200, Shamir Rabinovitch wrote:
> > On Mon, Nov 02, 2015 at 09:00:34PM +1100, Benjamin Herrenschmidt
> > wrote:
> > >
> > > Chosing on a per-mapping basis *in the back end* might still make
> > > some
On Mon, 2015-11-02 at 14:07 +0200, Shamir Rabinovitch wrote:
> On Mon, Nov 02, 2015 at 09:00:34PM +1100, Benjamin Herrenschmidt
> wrote:
> >
> > Chosing on a per-mapping basis *in the back end* might still make
> > some
>
> In my case, choosing mapping based on the hardware that will use this
> m
On Mon, Nov 02, 2015 at 03:44:27PM +0100, Joerg Roedel wrote:
>
> How do you envision this per-mapping by-pass to work? For the DMA-API
> mappings you have only one device address space. For by-pass to work,
> you need to map all physical memory of the machine into this space,
> which leaves the q
On Fri, Oct 30, 2015 at 11:32:06AM +0100, Arnd Bergmann wrote:
> I wonder if the 'iommu=force' attribute is too coarse-grained though,
> and if we should perhaps allow a per-device setting on architectures
> that allow this.
Yeah, definitly. Currently we only have iommu=pt to enable pass-through
m
On Thu, Oct 29, 2015 at 09:32:32AM +0200, Shamir Rabinovitch wrote:
> For the IB case, setting and tearing DMA mappings for the drivers data buffers
> is expensive. But we could for example consider to map all the HW control
> objects
> that validate the HW access to the drivers data buffers as IO
On Mon, Nov 02, 2015 at 09:00:34PM +1100, Benjamin Herrenschmidt wrote:
>
> Chosing on a per-mapping basis *in the back end* might still make some
In my case, choosing mapping based on the hardware that will use this
mappings makes more sense. Most hardware are not that performance sensitive
as t
On Mon, 2015-11-02 at 09:23 +0200, Shamir Rabinovitch wrote:
> To summary -
>
> 1. The whole point of the IOMMU pass through was to get bigger address space
> and faster map/unmap operations for performance critical hardware
> 2. SPARC IOMMU in particular has the ability to DVMA which adre
On Mon, Nov 02, 2015 at 08:10:49AM +1100, Benjamin Herrenschmidt wrote:
> But but but ...
>
> What I don't understand is how that brings you any safety.
Limited safety maybe? If some device DMA mappings are via IOMMU
and this fall to some address range that is far from the bypass /
pass through
On Sun, 2015-11-01 at 09:45 +0200, Shamir Rabinovitch wrote:
> Not sure this use case is possible for Infiniband where application hold
> the data buffers and there is no way to force application to re use the
> buffer as suggested.
>
> This is why I think there will be no easy way to bypass the
On Thu, Oct 29, 2015 at 10:35:25PM +, David Woodhouse wrote:
> >
> > Could this be mitigated using pools? I don't know if the net code
> > would play along easily.
>
> For the receive side, it shouldn't be beyond the wit of man to
> introduce an API which allocates *and* DMA-maps a skb. Pass
On Saturday 31 October 2015 10:17:22 Benjamin Herrenschmidt wrote:
> On Fri, 2015-10-30 at 11:32 +0100, Arnd Bergmann wrote:
> > On Thursday 29 October 2015 10:10:46 Benjamin Herrenschmidt wrote:
> > >
> > > > Maybe we should at least coordinate IOMMU 'paranoid/fast' modes
> > > > across
> > > > a
On Fri, 2015-10-30 at 11:32 +0100, Arnd Bergmann wrote:
> On Thursday 29 October 2015 10:10:46 Benjamin Herrenschmidt wrote:
> >
> > > Maybe we should at least coordinate IOMMU 'paranoid/fast' modes
> > > across
> > > architectures, and then the DMA_ATTR_IOMMU_BYPASS flag would have
> > > a
> > >
On Thursday 29 October 2015 10:10:46 Benjamin Herrenschmidt wrote:
>
> > Maybe we should at least coordinate IOMMU 'paranoid/fast' modes across
> > architectures, and then the DMA_ATTR_IOMMU_BYPASS flag would have a
> > sane meaning in the paranoid mode (and perhaps we'd want an ultra
> > -paranoi
On Thu, 2015-10-29 at 11:31 -0700, Andy Lutomirski wrote:
> On Oct 28, 2015 6:11 PM, "Benjamin Herrenschmidt"
> wrote:
> >
> > On Thu, 2015-10-29 at 09:42 +0900, David Woodhouse wrote:
> > > On Thu, 2015-10-29 at 09:32 +0900, Benjamin Herrenschmidt wrote:
> > >
> > > > On Power, I generally have
On Thu, 2015-10-29 at 11:31 -0700, Andy Lutomirski wrote:
> On Oct 28, 2015 6:11 PM, "Benjamin Herrenschmidt"
> wrote:
> >
> > On Thu, 2015-10-29 at 09:42 +0900, David Woodhouse wrote:
> > > On Thu, 2015-10-29 at 09:32 +0900, Benjamin Herrenschmidt wrote:
> > >
> > > > On Power, I generally have
On Oct 28, 2015 6:11 PM, "Benjamin Herrenschmidt"
wrote:
>
> On Thu, 2015-10-29 at 09:42 +0900, David Woodhouse wrote:
> > On Thu, 2015-10-29 at 09:32 +0900, Benjamin Herrenschmidt wrote:
> >
> > > On Power, I generally have 2 IOMMU windows for a device, one at the
> > > bottom is remapped, and is
On Thu, Oct 29, 2015 at 09:42:12AM +0900, David Woodhouse wrote:
> Aside from the lack of security, the other disadvantage of that is that
> you have to pin *all* pages of a guest in case DMA happens; you don't
> get to pin *only* those pages which are referenced by that guest's
> IOMMU page table
On Thu, 2015-10-29 at 09:42 +0900, David Woodhouse wrote:
> On Thu, 2015-10-29 at 09:32 +0900, Benjamin Herrenschmidt wrote:
>
> > On Power, I generally have 2 IOMMU windows for a device, one at the
> > bottom is remapped, and is generally used for 32-bit devices and the
> > one at the top us setu
On Thu, 2015-10-29 at 09:32 +0900, Benjamin Herrenschmidt wrote:
> On Power, I generally have 2 IOMMU windows for a device, one at the
> bottom is remapped, and is generally used for 32-bit devices and the
> one at the top us setup as a bypass
So in the normal case of decent 64-bit devices (and
On Wed, 2015-10-28 at 22:31 +0900, David Woodhouse wrote:
> We have an option in the Intel IOMMU for pass-through mode too, which
> basically *is* a total bypass. In practice, what's the difference
> between that and a "simple translation that does not require any
> [translation]"? We set up a full
From: David Woodhouse
Date: Wed, 28 Oct 2015 22:57:12 +0900
> On Wed, 2015-10-28 at 07:07 -0700, David Miller wrote:
>> In the sparc64 case, the 64-bit DMA address space is divided into
>> IOMMU translated and non-IOMMU translated.
>>
>> You just set the high bits differently depending upon what
On Wed, 2015-10-28 at 07:07 -0700, David Miller wrote:
> In the sparc64 case, the 64-bit DMA address space is divided into
> IOMMU translated and non-IOMMU translated.
>
> You just set the high bits differently depending upon what you want.
Wait, does that mean a (rogue) device could *always* get
From: David Woodhouse
Date: Wed, 28 Oct 2015 22:31:50 +0900
> On Wed, 2015-10-28 at 13:10 +0200, Shamir Rabinovitch wrote:
>> On Wed, Oct 28, 2015 at 03:30:01PM +0900, David Woodhouse wrote:
>> > > > +For systems with IOMMU it is assumed all DMA translations use the
>> > > > IOMMU.
>> >
>> > No
On Wed, 2015-10-28 at 13:10 +0200, Shamir Rabinovitch wrote:
> On Wed, Oct 28, 2015 at 03:30:01PM +0900, David Woodhouse wrote:
> > > > +For systems with IOMMU it is assumed all DMA translations use the
> > > > IOMMU.
> >
> > Not entirely true. We have per-device dma_ops on a most architectures
>
On Wed, Oct 28, 2015 at 03:30:01PM +0900, David Woodhouse wrote:
> > > +For systems with IOMMU it is assumed all DMA translations use the IOMMU.
>
> Not entirely true. We have per-device dma_ops on a most architectures
> already, and we were just talking about the need to add them to
> POWER/SPARC
On Sun, 2015-10-25 at 09:07 -0700, Shamir Rabinovitch wrote:
>
> +
> +DMA_ATTR_IOMMU_BYPASS
> +-
> +
> > +For systems with IOMMU it is assumed all DMA translations use the IOMMU.
Not entirely true. We have per-device dma_ops on a most architectures
already, and we were just ta
39 matches
Mail list logo