Re: bypass support for iommu on sparc64

2018-10-20 Thread Bryan Steele
This is OpenBSD tech@ On Sat, Oct 20, 2018 at 08:36:33PM +0100, Andrew Grillet wrote: > So, substitute opening and closing the connection to the network? > > Is the IOMMU not used for disk (and all SCSI) access also? > > > > On Sat, 20 Oct 2018 at 20:32, Theo de Raadt wrote: > > > Andrew Gri

Re: bypass support for iommu on sparc64

2018-10-20 Thread Andrew Grillet
So, substitute opening and closing the connection to the network? Is the IOMMU not used for disk (and all SCSI) access also? On Sat, 20 Oct 2018 at 20:32, Theo de Raadt wrote: > Andrew Grillet wrote: > > > Ok, what I am proposing is that the IOMMU is set up when a file is opened > > to provi

Re: bypass support for iommu on sparc64

2018-10-20 Thread Theo de Raadt
Andrew Grillet wrote: > Ok, what I am proposing is that the IOMMU is set up when a file is opened > to provide the address space required for that file's IO. Wow, you keep saying file as if it means something. packets off the network are not associated with any specific "file" activity it isn'

Re: bypass support for iommu on sparc64

2018-10-20 Thread Andrew Grillet
Ok, what I am proposing is that the IOMMU is set up when a file is opened to provide the address space required for that file's IO. This remains set up until the file is closed, avoiding frequent set-up and tear-down for each IO transfer. I assume that there is sufficient IOMMU address space to ha

Re: bypass support for iommu on sparc64

2018-10-20 Thread Theo de Raadt
In this case, what do mbufs have to do with files? I am very confused. > I was assuming that the main objection to allocating mbufs for duration of > file open, > rather than allocating per transfer, this could result in a much higher > number of mbufs > being in use concurrently. I cannot see an

Re: bypass support for iommu on sparc64

2018-10-20 Thread Andrew Grillet
I was assuming that the main objection to allocating mbufs for duration of file open, rather than allocating per transfer, this could result in a much higher number of mbufs being in use concurrently. I cannot see any other downside (which may be due to my not understanding a lot of stuff - I last

Re: bypass support for iommu on sparc64

2018-10-20 Thread Theo de Raadt
Andrew Grillet wrote: > These days we are not so short of memory - would it not be possible to > allocate an mbuf (or two for double-buffered) for each file > when opened, and free when closed? What does this have to do with files??

Re: bypass support for iommu on sparc64

2018-10-20 Thread Andrew Grillet
These days we are not so short of memory - would it not be possible to allocate an mbuf (or two for double-buffered) for each file when opened, and free when closed? I can see the management might be more complex, but the performance benefits might be considerable. Also, for VM disk access (ldom o

Re: bypass support for iommu on sparc64

2018-10-19 Thread David Gwynne
On Sat, Oct 20, 2018 at 02:44:29AM +, Joseph Mayer wrote: > > Last iteration from me on this one. > > Why is this not a problem on some other architectures? It is a problem, it's just that other archs don't have an iommu like sparc64. > I'd have thought DMA and hardware being assigned trans

Re: bypass support for iommu on sparc64

2018-10-19 Thread Joseph Mayer
On Saturday, October 20, 2018 10:14 AM, David Gwynne wrote: > > On 20 Oct 2018, at 11:56 am, Joseph Mayer joseph.ma...@protonmail.com wrote: > > ‐‐‐ Original Message ‐‐‐ > > On Friday, October 19, 2018 5:15 PM, Mark Kettenis mark.kette...@xs4all.nl > > wrote: > > > > > > Date: Fri, 19 Oc

Re: bypass support for iommu on sparc64

2018-10-19 Thread David Gwynne
> On 20 Oct 2018, at 11:56 am, Joseph Mayer wrote: > > ‐‐‐ Original Message ‐‐‐ > On Friday, October 19, 2018 5:15 PM, Mark Kettenis > wrote: > >>> Date: Fri, 19 Oct 2018 10:22:30 +1000 >>> From: David Gwynne da...@gwynne.id.au >>> On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwy

Re: bypass support for iommu on sparc64

2018-10-19 Thread Joseph Mayer
‐‐‐ Original Message ‐‐‐ On Friday, October 19, 2018 5:15 PM, Mark Kettenis wrote: > > Date: Fri, 19 Oct 2018 10:22:30 +1000 > > From: David Gwynne da...@gwynne.id.au > > On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwynne wrote: > > > > > On Mon, May 08, 2017 at 11:03:58AM +1000, Dav

Re: bypass support for iommu on sparc64

2018-10-19 Thread David Gwynne
> On 19 Oct 2018, at 9:59 pm, Andrew Grillet wrote: > > Is the setup and teardown per transfer or when file is opened and closed? > Or is it set up once per context switch of task? > > I am partly interested cos I would like to improve mt one day (as user of > tape > and Sparc64 Txxx) if I ge

Re: bypass support for iommu on sparc64

2018-10-19 Thread David Gwynne
> On 19 Oct 2018, at 7:15 pm, Mark Kettenis wrote: > >> Date: Fri, 19 Oct 2018 10:22:30 +1000 >> From: David Gwynne >> >> On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwynne wrote: >>> On Mon, May 08, 2017 at 11:03:58AM +1000, David Gwynne wrote: on modern sparc64s (think fire or spa

Re: bypass support for iommu on sparc64

2018-10-19 Thread Andrew Grillet
Is the setup and teardown per transfer or when file is opened and closed? Or is it set up once per context switch of task? I am partly interested cos I would like to improve mt one day (as user of tape and Sparc64 Txxx) if I get the time. Andrew On Fri, 19 Oct 2018 at 10:22, Mark Kettenis wro

Re: bypass support for iommu on sparc64

2018-10-19 Thread Mark Kettenis
> Date: Fri, 19 Oct 2018 10:22:30 +1000 > From: David Gwynne > > On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwynne wrote: > > On Mon, May 08, 2017 at 11:03:58AM +1000, David Gwynne wrote: > > > on modern sparc64s (think fire or sparc enterprise Mx000 boxes), > > > setting up and tearing down

Re: bypass support for iommu on sparc64

2018-10-18 Thread David Gwynne
On Wed, May 10, 2017 at 10:09:59PM +1000, David Gwynne wrote: > On Mon, May 08, 2017 at 11:03:58AM +1000, David Gwynne wrote: > > on modern sparc64s (think fire or sparc enterprise Mx000 boxes), > > setting up and tearing down the translation table entries (TTEs) > > is very expensive. so expensive

Re: bypass support for iommu on sparc64

2017-05-10 Thread David Gwynne
On Mon, May 08, 2017 at 11:03:58AM +1000, David Gwynne wrote: > on modern sparc64s (think fire or sparc enterprise Mx000 boxes), > setting up and tearing down the translation table entries (TTEs) > is very expensive. so expensive that the cost of doing it for disk > io has a noticable impact on com

bypass support for iommu on sparc64

2017-05-07 Thread David Gwynne
on modern sparc64s (think fire or sparc enterprise Mx000 boxes), setting up and tearing down the translation table entries (TTEs) is very expensive. so expensive that the cost of doing it for disk io has a noticable impact on compile times. now that there's a BUS_DMA_64BIT flag, we can use that to