Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-15 Thread William Lee Irwin III
On Tue, Mar 13, 2007 at 06:12:44PM -0700, William Lee Irwin III wrote: > There are furthermore distinctions to make between fork() and execve(). > fork() stomps over the entire process address space copying pagetables > en masse. After execve() a process incrementally faults in PTE's one at > a tim

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-15 Thread Christoph Lameter
On Tue, 13 Mar 2007, Andrew Morton wrote: > > 1. We need to support other states of pages other than zeroed. > > What does this mean? pgd are not completely zeroed. They contain mappings that are always present. Thus the state is not a zeroed state. > > 2. Prezeroing does not make much sense i

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-15 Thread Christoph Lameter
On Tue, 13 Mar 2007, Andrew Morton wrote: > > On Tue, 13 Mar 2007 04:20:48 -0700 (PDT) Christoph Lameter <[EMAIL > > PROTECTED]> wrote: > > On Tue, 13 Mar 2007, Andrew Morton wrote: > > > > > Yeah, prezeroing in idle is probably pointless. But I'm not aware of > > > anyone having tried it prope

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread William Lee Irwin III
On Tue, Mar 13, 2007 at 04:47:56AM -0800, Andrew Morton wrote: > I'm trying to remember why we ever would have needed to zero out the > pagetable pages if we're taking down the whole mm? Maybe it's > because "oh, the arch wants to put this page into a quicklist to > recycle it", which is all rathe

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Paul Mackerras
Andrew Morton writes: > Plus, we can get in a situation where take a cache-cold, known-zero page > from the pte quicklist when there is a cache-hot, non-zero page sitting in > the page allocator. I suspect that zeroing the cache-hot page would take a > similar amount of time to a single miss agai

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Peter Chubb
> "Jeremy" == Jeremy Fitzhardinge <[EMAIL PROTECTED]> writes: Jeremy> And do the same in pte pages for actual mapped pages? Or do Jeremy> you think they would be too densely populated for it to be Jeremy> worthwhile? We've been doing some measurements on how densely clumped ptes are. On 32-

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread David Miller
From: Matt Mackall <[EMAIL PROTECTED]> Date: Tue, 13 Mar 2007 16:14:35 -0500 > Well you -could- do this: > > - reuse a long in struct page as a used map that divides the page up > into 32 or 64 segments > - every time you set a PTE, set the corresponding bit in the mask > - when we zap, only vi

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Jeremy Fitzhardinge
Matt Mackall wrote: > Well you -could- do this: > > - reuse a long in struct page as a used map that divides the page up > into 32 or 64 segments > - every time you set a PTE, set the corresponding bit in the mask > - when we zap, only visit the regions set in the mask > > Thus, you avoid visitin

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Matt Mackall
On Tue, Mar 13, 2007 at 02:07:22PM -0700, David Miller wrote: > From: Matt Mackall <[EMAIL PROTECTED]> > Date: Tue, 13 Mar 2007 15:21:25 -0500 > > > Because the fan-out is large, the bulk of the work is bringing the last > > layer of the tree into cache to find all the pages in the address > > spa

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread David Miller
From: Matt Mackall <[EMAIL PROTECTED]> Date: Tue, 13 Mar 2007 15:21:25 -0500 > Because the fan-out is large, the bulk of the work is bringing the last > layer of the tree into cache to find all the pages in the address > space. And there's really no way around that. That's right. And I will note

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Matt Mackall
On Tue, Mar 13, 2007 at 01:17:00PM -0700, Jeremy Fitzhardinge wrote: > Matt Mackall wrote: > > On Tue, Mar 13, 2007 at 10:30:10AM -0700, Jeremy Fitzhardinge wrote: > > > >> Nick Piggin wrote: > >> > >>> However we still have to visit those to-be-unmapped parts of the page > >>> table, > >>>

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Matt Mackall
On Tue, Mar 13, 2007 at 10:30:10AM -0700, Jeremy Fitzhardinge wrote: > Nick Piggin wrote: > > However we still have to visit those to-be-unmapped parts of the page > > table, > > to find the pages and free them. So we still at least need to bring it > > into > > cache for the read... at which point

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Jeremy Fitzhardinge
Matt Mackall wrote: > On Tue, Mar 13, 2007 at 10:30:10AM -0700, Jeremy Fitzhardinge wrote: > >> Nick Piggin wrote: >> >>> However we still have to visit those to-be-unmapped parts of the page >>> table, >>> to find the pages and free them. So we still at least need to bring it >>> into >>>

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Jeremy Fitzhardinge
Nick Piggin wrote: > However we still have to visit those to-be-unmapped parts of the page > table, > to find the pages and free them. So we still at least need to bring it > into > cache for the read... at which point, the store probably isn't a big > burden. Why not try to find a place to stash

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Nick Piggin
Andrew Morton wrote: On Tue, 13 Mar 2007 23:01:11 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: Andrew Morton wrote: It would be interesting to look at a) leave the page full of random garbage if we're releasing the whole mm and b) return it straight to the page allocator. Well we have the 'f

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 23:01:11 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > >>On Tue, 13 Mar 2007 22:30:19 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > >>We don't actually have to zap_pte_range the entire page table in > >>order to free it (IIRC we used to have to, before

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Nick Piggin
Andrew Morton wrote: On Tue, 13 Mar 2007 22:30:19 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: We don't actually have to zap_pte_range the entire page table in order to free it (IIRC we used to have to, before the 4lpt patches). I'm trying to remember why we ever would have needed to zero out

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 22:30:19 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > We don't actually have to zap_pte_range the entire page table in > order to free it (IIRC we used to have to, before the 4lpt patches). I'm trying to remember why we ever would have needed to zero out the pagetable pages

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 04:20:48 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > On Tue, 13 Mar 2007, Andrew Morton wrote: > > > Yeah, prezeroing in idle is probably pointless. But I'm not aware of > > anyone having tried it properly... > > Ok, then what did I do wrong 3 years ago wi

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Nick Piggin
Andrew Morton wrote: On Tue, 13 Mar 2007 22:06:46 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: Andrew Morton wrote: On Tue, 13 Mar 2007 19:03:38 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: ... Page allocator still requires interrupts to be disabled, which this doesn't. it is worthwhil

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 04:17:26 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > On Tue, 13 Mar 2007, Andrew Morton wrote: > > > > On Tue, 13 Mar 2007 00:13:25 -0700 (PDT) Christoph Lameter <[EMAIL > > > PROTECTED]> wrote: > > > Page table pages have the characteristics that they are

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Christoph Lameter
On Tue, 13 Mar 2007, Andrew Morton wrote: > Yeah, prezeroing in idle is probably pointless. But I'm not aware of > anyone having tried it properly... Ok, then what did I do wrong 3 years ago with the prezeroing patchsets? - To unsubscribe from this list: send the line "unsubscribe linux-kernel

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Christoph Lameter
On Tue, 13 Mar 2007, Andrew Morton wrote: > > On Tue, 13 Mar 2007 00:13:25 -0700 (PDT) Christoph Lameter <[EMAIL > > PROTECTED]> wrote: > > Page table pages have the characteristics that they are typically zero > > or in a known state when they are freed. > > Well if they're zero then perhaps th

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 22:06:46 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > >>On Tue, 13 Mar 2007 19:03:38 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > > ... > > >>Page allocator still requires interrupts to be disabled, which this doesn't. > >>it is worthwhile. > > >

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Nick Piggin
Andrew Morton wrote: On Tue, 13 Mar 2007 19:03:38 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: Page allocator still requires interrupts to be disabled, which this doesn't. Bah. How many cli/sti statements fit into a single cachemiss? On a Pentium 4? ;) Sure, that is a minor detail, cons

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 19:03:38 +1100 Nick Piggin <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > >>On Tue, 13 Mar 2007 00:13:25 -0700 (PDT) Christoph Lameter <[EMAIL > >>PROTECTED]> wrote: > >>Page table pages have the characteristics that they are typically zero > >>or in a known state when t

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-13 Thread Nick Piggin
Andrew Morton wrote: On Tue, 13 Mar 2007 00:13:25 -0700 (PDT) Christoph Lameter <[EMAIL PROTECTED]> wrote: Page table pages have the characteristics that they are typically zero or in a known state when they are freed. Well if they're zero then perhaps they should be released to the page allo

Re: [QUICKLIST 0/4] Arch independent quicklists V2

2007-03-12 Thread Andrew Morton
> On Tue, 13 Mar 2007 00:13:25 -0700 (PDT) Christoph Lameter <[EMAIL > PROTECTED]> wrote: > Page table pages have the characteristics that they are typically zero > or in a known state when they are freed. Well if they're zero then perhaps they should be released to the page allocator to satisfy

[QUICKLIST 0/4] Arch independent quicklists V2

2007-03-12 Thread Christoph Lameter
V1->V2 - Add sparch64 patch - Single i386 and x86_64 patch - Update attribution - Update justification - Update approvals - Earlier discussion of V1 was at http://marc.info/?l=linux-kernel&m=117357922219342&w=2 This patchset introduces an arch independent framework to handle lists of recently us