Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Wed, 15 Aug 2007, Andi Kleen wrote: > > Ok I have a vague idea on how this could but its likely that the > > changes make things worse rather than better. Additional reference to a > > new cacheline (per cpu but still), preempt disable. Lots of code at all > > call sites. Interrupt enable/dis

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
> Ok I have a vague idea on how this could but its likely that the > changes make things worse rather than better. Additional reference to a > new cacheline (per cpu but still), preempt disable. Lots of code at all > call sites. Interrupt enable/disable is quite efficient in recent > processors.

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Wed, 15 Aug 2007, Andi Kleen wrote: > > > The interrupt handler shouldn't touch zone_flag. If it wants > > > to it would need to be converted to a local_t and incremented/decremented > > > (should be about the same cost at least on architectures with sane > > > local_t implementation) > > > >

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
On Tue, Aug 14, 2007 at 03:07:10PM -0700, Christoph Lameter wrote: > There are more spinlocks needed. So we would just check the whole bunch > and fail if any of them are used? Yes zone_flag would apply to all of them. > > > do things with zone locks > > } > > > > The interrup

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > Could you be a bit more specific? Where do you want to place the data? > > DEFINE_PER_CPU(int, zone_flag); > > get_cpu(); // likely already true and then not needed > __get_cpu(zone_flag) = 1; > /* wmb is implied in spin_lock I th

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
On Tue, Aug 14, 2007 at 02:48:31PM -0700, Christoph Lameter wrote: > On Tue, 14 Aug 2007, Andi Kleen wrote: > > > > But that still creates lots of overhead each time we take the lru lock! > > > > A lot of overhead in what way? Setting a flag in a cache hot > > per CPU data variable shouldn't be m

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > But that still creates lots of overhead each time we take the lru lock! > > A lot of overhead in what way? Setting a flag in a cache hot > per CPU data variable shouldn't be more than a few cycles. Could you be a bit more specific? Where do you want to

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
On Tue, Aug 14, 2007 at 02:37:27PM -0700, Christoph Lameter wrote: > On Tue, 14 Aug 2007, Andi Kleen wrote: > > > > We already have such a flag in the zone structure > > > > Zone structure is not strictly CPU local so it's broader > > than needed. But it might work. > > We could convert this int

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > We already have such a flag in the zone structure > > Zone structure is not strictly CPU local so it's broader > than needed. But it might work. We could convert this into a per cpu array? But that still creates lots of overhead each time we take the l

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
On Tue, Aug 14, 2007 at 02:26:43PM -0700, Christoph Lameter wrote: > On Tue, 14 Aug 2007, Andi Kleen wrote: > > > > So every spinlock would have an array of chars sized to NR_CPUS and set > > > the flag when the lock is taken? > > > > I was more thinking of a single per cpu flag for all of page

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > So every spinlock would have an array of chars sized to NR_CPUS and set > > the flag when the lock is taken? > > I was more thinking of a single per cpu flag for all of page reclaim > That keeps it also cache local. We already have such a flag in the z

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
On Tue, Aug 14, 2007 at 02:15:12PM -0700, Christoph Lameter wrote: > On Tue, 14 Aug 2007, Andi Kleen wrote: > > > > H... The spinlock is its own flag. > > > > Yes, but it's not CPU local. Taking the spinlock from another CPU's > > interrupt handler is perfectly safe, just not from the local C

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > H... The spinlock is its own flag. > > Yes, but it's not CPU local. Taking the spinlock from another CPU's > interrupt handler is perfectly safe, just not from the local CPU. > If you use the spinlock as flag you would need to lock out everybody. So

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > > There are other lock interactions that may cause problems. If we do not > > switch to the saving of irq flags then all involved spinlocks must become > > trylocks because the interrupt could have happened while the spinlock is > > held. So interrupts m

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
> H... The spinlock is its own flag. Yes, but it's not CPU local. Taking the spinlock from another CPU's interrupt handler is perfectly safe, just not from the local CPU. If you use the spinlock as flag you would need to lock out everybody. -Andi - To unsubscribe from this list: send the line

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
> A much simpler approach to this seems to use threaded interrupts like > -rt does. Then the interrupt could potentially stay blocked for very long waiting for process context to finish its work. Also not good. Essentially it would be equivalent to cli/sti for interrupts that need to free memory.

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
> There are other lock interactions that may cause problems. If we do not > switch to the saving of irq flags then all involved spinlocks must become > trylocks because the interrupt could have happened while the spinlock is > held. So interrupts must be disabled on locks acquired during an > i

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Peter Zijlstra
On Tue, 2007-08-14 at 12:12 -0700, Christoph Lameter wrote: > On Tue, 14 Aug 2007, Andi Kleen wrote: > > > Christoph Lameter <[EMAIL PROTECTED]> writes: > > > > > Reclaim can be called with interrupts disabled in atomic reclaim. > > > vmscan.c is currently using spinlock_irq(). Switch to spin_loc

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Christoph Lameter
On Tue, 14 Aug 2007, Andi Kleen wrote: > Christoph Lameter <[EMAIL PROTECTED]> writes: > > > Reclaim can be called with interrupts disabled in atomic reclaim. > > vmscan.c is currently using spinlock_irq(). Switch to spin_lock_irqsave(). > > I like the idea in principle. If this fully works out

Re: [RFC 4/9] Atomic reclaim: Save irq flags in vmscan.c

2007-08-14 Thread Andi Kleen
Christoph Lameter <[EMAIL PROTECTED]> writes: > Reclaim can be called with interrupts disabled in atomic reclaim. > vmscan.c is currently using spinlock_irq(). Switch to spin_lock_irqsave(). I like the idea in principle. If this fully works out we could potentially keep less memory free by defaul