Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-25 Thread Andrew Morton
On Wed, 20 Jun 2007 10:06:39 +0200 Peter Zijlstra <[EMAIL PROTECTED]> wrote: > On Tue, 2007-06-19 at 14:37 -0700, Keshavamurthy, Anil S wrote: > > plain text document attachment (intel_iommu_pf_memalloc.patch) > > Intel IOMMU driver needs memory during DMA map calls to setup its internal > > page

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-25 Thread Andrew Morton
On Wed, 20 Jun 2007 10:06:39 +0200 Peter Zijlstra [EMAIL PROTECTED] wrote: On Tue, 2007-06-19 at 14:37 -0700, Keshavamurthy, Anil S wrote: plain text document attachment (intel_iommu_pf_memalloc.patch) Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Thu, Jun 21, 2007 at 09:13:11AM +0200, Peter Zijlstra wrote: > On Wed, 2007-06-20 at 23:37 -0700, Keshavamurthy, Anil S wrote: > > On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: > > > On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: > > > > Peter Zijlstra wrote: > > >

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 23:37 -0700, Keshavamurthy, Anil S wrote: > On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: > > On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: > > > Peter Zijlstra wrote: > > Also, the other iommu code you pointed me to, was happy to fail, it did >

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: > On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: > > Peter Zijlstra wrote: > Also, the other iommu code you pointed me to, was happy to fail, it did > not attempt to use the emergency reserves. Is the same behavior

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Wed, Jun 20, 2007 at 11:11:05PM -0700, Arjan van de Ven wrote: > Peter Zijlstra wrote: > >What I'm saying is that if you do use the reserves, you should ensure > >the use is bounded. I'm not seeing anything like that. > > each mapping takes at most 3 pages With 3 pages(3 level page table),

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: > Peter Zijlstra wrote: > > What I'm saying is that if you do use the reserves, you should ensure > > the use is bounded. I'm not seeing anything like that. > > each mapping takes at most 3 pages That is a start, but the thing I'm

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Arjan van de Ven
Peter Zijlstra wrote: What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that. each mapping takes at most 3 pages This is a generic API, who is to ensure some other non-swap device will not deplete memory and deadlock the

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 16:03 -0700, Keshavamurthy, Anil S wrote: > On Wed, Jun 20, 2007 at 10:08:51PM +0200, Peter Zijlstra wrote: > > On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: > > > Peter Zijlstra wrote: > > > > So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to >

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 16:03 -0700, Keshavamurthy, Anil S wrote: On Wed, Jun 20, 2007 at 10:08:51PM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Arjan van de Ven
Peter Zijlstra wrote: What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that. each mapping takes at most 3 pages This is a generic API, who is to ensure some other non-swap device will not deplete memory and deadlock the

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that. each mapping takes at most 3 pages That is a start, but the thing I'm worried most

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Wed, Jun 20, 2007 at 11:11:05PM -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: What I'm saying is that if you do use the reserves, you should ensure the use is bounded. I'm not seeing anything like that. each mapping takes at most 3 pages With 3 pages(3 level page table), IOMMU can

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: Also, the other iommu code you pointed me to, was happy to fail, it did not attempt to use the emergency reserves. Is the same behavior acceptable

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Peter Zijlstra
On Wed, 2007-06-20 at 23:37 -0700, Keshavamurthy, Anil S wrote: On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: Also, the other iommu code you pointed me to, was happy to fail, it did not

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-21 Thread Keshavamurthy, Anil S
On Thu, Jun 21, 2007 at 09:13:11AM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 23:37 -0700, Keshavamurthy, Anil S wrote: On Thu, Jun 21, 2007 at 08:29:34AM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 23:11 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: Also, the

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Keshavamurthy, Anil S
On Wed, Jun 20, 2007 at 10:08:51PM +0200, Peter Zijlstra wrote: > On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: > > Peter Zijlstra wrote: > > > So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to > > > ensure they themselves will not block on a memory allocation. And

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: > Peter Zijlstra wrote: > > So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to > > ensure they themselves will not block on a memory allocation. And it is > > understood that these code paths have a bounded memory

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Arjan van de Ven
Peter Zijlstra wrote: So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint. that's a too simplistic view though; what happens is that kswapd

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Wed, 2007-06-20 at 10:30 -0700, Siddha, Suresh B wrote: > On Wed, Jun 20, 2007 at 06:03:02AM -0700, Arjan van de Ven wrote: > > Peter Zijlstra wrote: > > > > > > > > >PF_MEMALLOC as is, is meant to salvage the VM from the typical VM > > >deadlock. > > > > .. and this IS the typical VM

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Siddha, Suresh B
On Wed, Jun 20, 2007 at 06:03:02AM -0700, Arjan van de Ven wrote: > Peter Zijlstra wrote: > > > > > >PF_MEMALLOC as is, is meant to salvage the VM from the typical VM > >deadlock. > > .. and this IS the typical VM deadlock.. it is your storage driver > trying to write out a piece of memory on

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Arjan van de Ven
Peter Zijlstra wrote: PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. .. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the VM, and calls the iommu to map it, which then needs a bit of

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Tue, 2007-06-19 at 14:37 -0700, Keshavamurthy, Anil S wrote: > plain text document attachment (intel_iommu_pf_memalloc.patch) > Intel IOMMU driver needs memory during DMA map calls to setup its internal > page tables and for other data structures. As we all know that these DMA > map calls are

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Tue, 2007-06-19 at 14:37 -0700, Keshavamurthy, Anil S wrote: plain text document attachment (intel_iommu_pf_memalloc.patch) Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Arjan van de Ven
Peter Zijlstra wrote: PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. .. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the VM, and calls the iommu to map it, which then needs a bit of

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Siddha, Suresh B
On Wed, Jun 20, 2007 at 06:03:02AM -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. .. and this IS the typical VM deadlock.. it is your storage driver trying to write out a piece of memory on behalf of the

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Wed, 2007-06-20 at 10:30 -0700, Siddha, Suresh B wrote: On Wed, Jun 20, 2007 at 06:03:02AM -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: PF_MEMALLOC as is, is meant to salvage the VM from the typical VM deadlock. .. and this IS the typical VM deadlock.. it is your

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Arjan van de Ven
Peter Zijlstra wrote: So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint. that's a too simplistic view though; what happens is that kswapd

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Peter Zijlstra
On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is understood that these code paths have a bounded memory footprint.

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-20 Thread Keshavamurthy, Anil S
On Wed, Jun 20, 2007 at 10:08:51PM +0200, Peter Zijlstra wrote: On Wed, 2007-06-20 at 12:14 -0700, Arjan van de Ven wrote: Peter Zijlstra wrote: So a reclaim context (kswapd and direct reclaim) set PF_MEMALLOC to ensure they themselves will not block on a memory allocation. And it is

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Arjan van de Ven
On Tue, 2007-06-19 at 16:34 -0700, Christoph Lameter wrote: > On Tue, 19 Jun 2007, Arjan van de Ven wrote: > > > > Otherwise you are locked into the use of GFP_ATOMIC. > > > > all callers pretty much are either in irq context or with spinlocks held. > > Good > > luck. it's also called

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Christoph Lameter
On Tue, 19 Jun 2007, Arjan van de Ven wrote: > > Otherwise you are locked into the use of GFP_ATOMIC. > > all callers pretty much are either in irq context or with spinlocks held. Good > luck. it's also called primarily from the PCI DMA API which doesn't take a > gfp_t argument in the first

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Arjan van de Ven
Christoph Lameter wrote: On Tue, 19 Jun 2007, Keshavamurthy, Anil S wrote: So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls. All these functions should have gfp_t flags passed to them. why? Otherwise you are locked into the use

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Christoph Lameter
On Tue, 19 Jun 2007, Keshavamurthy, Anil S wrote: > So far in our test scenario, we were unable to create > any memory allocation failure inside dma map api calls. All these functions should have gfp_t flags passed to them. Otherwise you are locked into the use of GFP_ATOMIC. If this is a

[Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Keshavamurthy, Anil S
Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to

[Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Keshavamurthy, Anil S
Intel IOMMU driver needs memory during DMA map calls to setup its internal page tables and for other data structures. As we all know that these DMA map calls are mostly called in the interrupt context or with the spinlock held by the upper level drivers(network/storage drivers), so in order to

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Christoph Lameter
On Tue, 19 Jun 2007, Keshavamurthy, Anil S wrote: So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls. All these functions should have gfp_t flags passed to them. Otherwise you are locked into the use of GFP_ATOMIC. If this is a

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Arjan van de Ven
Christoph Lameter wrote: On Tue, 19 Jun 2007, Keshavamurthy, Anil S wrote: So far in our test scenario, we were unable to create any memory allocation failure inside dma map api calls. All these functions should have gfp_t flags passed to them. why? Otherwise you are locked into the use

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Christoph Lameter
On Tue, 19 Jun 2007, Arjan van de Ven wrote: Otherwise you are locked into the use of GFP_ATOMIC. all callers pretty much are either in irq context or with spinlocks held. Good luck. it's also called primarily from the PCI DMA API which doesn't take a gfp_t argument in the first

Re: [Intel IOMMU 06/10] Avoid memory allocation failures in dma map api calls

2007-06-19 Thread Arjan van de Ven
On Tue, 2007-06-19 at 16:34 -0700, Christoph Lameter wrote: On Tue, 19 Jun 2007, Arjan van de Ven wrote: Otherwise you are locked into the use of GFP_ATOMIC. all callers pretty much are either in irq context or with spinlocks held. Good luck. it's also called primarily from the