Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Anton Blanchard
Hi Jakub, > That would mean an additional syscall. Furthermore, if you allocate a big > chunk of memory, dirty it, then free (with madvise (MADV_FREE)) it and soon > allocate the same size of memory again, it is better to start that with > non-dirty memory, it might be that this time you e.g.

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Jakub Jelinek
On Mon, Apr 16, 2007 at 11:10:39AM -0500, Anton Blanchard wrote: > > Making the pte clean also needs to clear the hardware writable > > bit on architectures where we do pte dirtying in software. > > > > If we don't, we would have corruption problems all over the VM, > > for example in the code

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Anton Blanchard
Hi, > Making the pte clean also needs to clear the hardware writable > bit on architectures where we do pte dirtying in software. > > If we don't, we would have corruption problems all over the VM, > for example in the code around pte_clean_one :) > > >But as Linus recently said, even

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Anton Blanchard
Hi, Making the pte clean also needs to clear the hardware writable bit on architectures where we do pte dirtying in software. If we don't, we would have corruption problems all over the VM, for example in the code around pte_clean_one :) But as Linus recently said, even hardware

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Jakub Jelinek
On Mon, Apr 16, 2007 at 11:10:39AM -0500, Anton Blanchard wrote: Making the pte clean also needs to clear the hardware writable bit on architectures where we do pte dirtying in software. If we don't, we would have corruption problems all over the VM, for example in the code around

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-16 Thread Anton Blanchard
Hi Jakub, That would mean an additional syscall. Furthermore, if you allocate a big chunk of memory, dirty it, then free (with madvise (MADV_FREE)) it and soon allocate the same size of memory again, it is better to start that with non-dirty memory, it might be that this time you e.g.

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Rik van Riel wrote: Nick Piggin wrote: The lazy freeing is aimed at avoiding page faults on memory that is freed and later realloced, which is quite a common thing in many workloads. I would be interested to see how it performs and what these workloads look like, although we do need to fix

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Rik van Riel
Nick Piggin wrote: The lazy freeing is aimed at avoiding page faults on memory that is freed and later realloced, which is quite a common thing in many workloads. I would be interested to see how it performs and what these workloads look like, although we do need to fix the basic glibc and

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Rik van Riel wrote: Nick Piggin wrote: Nick Piggin wrote: Eric Dumazet wrote: ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it isn't. That is to say, it isn't required for correctness. But if the question was about avoiding a fault, then

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Rik van Riel
Nick Piggin wrote: Nick Piggin wrote: Eric Dumazet wrote: Two things can happen here. If this program used the pages before the kernel needed them, the program will be reusing its old pages. ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Nick Piggin wrote: Eric Dumazet wrote: Two things can happen here. If this program used the pages before the kernel needed them, the program will be reusing its old pages. ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it isn't. That is to

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Eric Dumazet wrote: Rik van Riel a écrit : Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Eric Dumazet wrote: Rik van Riel a écrit : Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Nick Piggin wrote: Eric Dumazet wrote: Two things can happen here. If this program used the pages before the kernel needed them, the program will be reusing its old pages. ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it isn't. That is to

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Rik van Riel
Nick Piggin wrote: Nick Piggin wrote: Eric Dumazet wrote: Two things can happen here. If this program used the pages before the kernel needed them, the program will be reusing its old pages. ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Rik van Riel wrote: Nick Piggin wrote: Nick Piggin wrote: Eric Dumazet wrote: ah ok, this is because accessed/dirty bits are set by hardware and not a page fault. No it isn't. That is to say, it isn't required for correctness. But if the question was about avoiding a fault, then

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Rik van Riel
Nick Piggin wrote: The lazy freeing is aimed at avoiding page faults on memory that is freed and later realloced, which is quite a common thing in many workloads. I would be interested to see how it performs and what these workloads look like, although we do need to fix the basic glibc and

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-12 Thread Nick Piggin
Rik van Riel wrote: Nick Piggin wrote: The lazy freeing is aimed at avoiding page faults on memory that is freed and later realloced, which is quite a common thing in many workloads. I would be interested to see how it performs and what these workloads look like, although we do need to fix

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Eric Dumazet
Rik van Riel a écrit : Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Rik van Riel
Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Eric Dumazet
Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs the memory, not even a

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Eric Dumazet
Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs the memory, not even a

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Rik van Riel
Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs

Re: [PATCH] make MADV_FREE lazily free memory

2007-04-11 Thread Eric Dumazet
Rik van Riel a écrit : Eric Dumazet wrote: Rik van Riel a écrit : Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them

[PATCH] make MADV_FREE lazily free memory

2007-04-10 Thread Rik van Riel
Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs the memory, not even a page fault will happen.

[PATCH] make MADV_FREE lazily free memory

2007-04-10 Thread Rik van Riel
Make it possible for applications to have the kernel free memory lazily. This reduces a repeated free/malloc cycle from freeing pages and allocating them, to just marking them freeable. If the application wants to reuse them before the kernel needs the memory, not even a page fault will happen.