Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-12 Thread Dave Hansen
On 08/09/2013 12:55 AM, Jan Kara wrote: > On Thu 08-08-13 15:58:39, Dave Hansen wrote: >> I was coincidentally tracking down what I thought was a scalability >> problem (turned out to be full disks :). I noticed, though, that ext4 >> is about 20% slower than ext2/3 at doing write page faults

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-12 Thread Dave Hansen
On 08/09/2013 12:55 AM, Jan Kara wrote: On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write page faults (x-axis is

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Jan Kara
On Fri 09-08-13 10:36:41, Andy Lutomirski wrote: > On Fri, Aug 9, 2013 at 12:55 AM, Jan Kara wrote: > > On Thu 08-08-13 15:58:39, Dave Hansen wrote: > >> I was coincidentally tracking down what I thought was a scalability > >> problem (turned out to be full disks :). I noticed, though, that ext4

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Andy Lutomirski
On Fri, Aug 9, 2013 at 10:42 AM, Dave Hansen wrote: > On 08/09/2013 12:55 AM, Jan Kara wrote: >> On Thu 08-08-13 15:58:39, Dave Hansen wrote: >>> > I was coincidentally tracking down what I thought was a scalability >>> > problem (turned out to be full disks :). I noticed, though, that ext4 >>>

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Dave Hansen
On 08/09/2013 12:55 AM, Jan Kara wrote: > On Thu 08-08-13 15:58:39, Dave Hansen wrote: >> > I was coincidentally tracking down what I thought was a scalability >> > problem (turned out to be full disks :). I noticed, though, that ext4 >> > is about 20% slower than ext2/3 at doing write page

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Andy Lutomirski
On Fri, Aug 9, 2013 at 12:55 AM, Jan Kara wrote: > On Thu 08-08-13 15:58:39, Dave Hansen wrote: >> I was coincidentally tracking down what I thought was a scalability >> problem (turned out to be full disks :). I noticed, though, that ext4 >> is about 20% slower than ext2/3 at doing write page

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Jan Kara
On Thu 08-08-13 15:58:39, Dave Hansen wrote: > I was coincidentally tracking down what I thought was a scalability > problem (turned out to be full disks :). I noticed, though, that ext4 > is about 20% slower than ext2/3 at doing write page faults (x-axis is > number of tasks): > >

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Jan Kara
On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write page faults (x-axis is number of tasks):

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Andy Lutomirski
On Fri, Aug 9, 2013 at 12:55 AM, Jan Kara j...@suse.cz wrote: On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Dave Hansen
On 08/09/2013 12:55 AM, Jan Kara wrote: On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write page faults (x-axis

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Andy Lutomirski
On Fri, Aug 9, 2013 at 10:42 AM, Dave Hansen dave.han...@intel.com wrote: On 08/09/2013 12:55 AM, Jan Kara wrote: On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-09 Thread Jan Kara
On Fri 09-08-13 10:36:41, Andy Lutomirski wrote: On Fri, Aug 9, 2013 at 12:55 AM, Jan Kara j...@suse.cz wrote: On Thu 08-08-13 15:58:39, Dave Hansen wrote: I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 12:25 PM, Andy Lutomirski wrote: > > Whoops -- I read your email too quickly. I haven't tried > MADV_WILLNEED, but I think I tried reading each page to fault them in. > Is there any reason to expect MADV_WILLNEED to do any better? I'll > try to do some new tests to see

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Dave Hansen
I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write page faults (x-axis is number of tasks):

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 11:53 AM, Jan Kara wrote: > On Thu 08-08-13 08:56:28, Andy Lutomirski wrote: >> On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara wrote: >> > On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: >> >> On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen >> >> wrote: >> >> > On 08/07/2013

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Jan Kara
On Thu 08-08-13 08:56:28, Andy Lutomirski wrote: > On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara wrote: > > On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: > >> On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: > >> > On 08/07/2013 06:40 AM, Jan Kara wrote: > >> >> One question before I look at

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara wrote: > On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: >> On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: >> > On 08/07/2013 06:40 AM, Jan Kara wrote: >> >> One question before I look at the patches: Why don't you use fallocate() >> >> in your

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Jan Kara
On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: > On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: > > On 08/07/2013 06:40 AM, Jan Kara wrote: > >> One question before I look at the patches: Why don't you use fallocate() > >> in your application? The functionality you require seems to be

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Jan Kara
On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen dave.han...@intel.com wrote: On 08/07/2013 06:40 AM, Jan Kara wrote: One question before I look at the patches: Why don't you use fallocate() in your application? The functionality you require

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara j...@suse.cz wrote: On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen dave.han...@intel.com wrote: On 08/07/2013 06:40 AM, Jan Kara wrote: One question before I look at the patches: Why don't you use

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Jan Kara
On Thu 08-08-13 08:56:28, Andy Lutomirski wrote: On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara j...@suse.cz wrote: On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen dave.han...@intel.com wrote: On 08/07/2013 06:40 AM, Jan Kara wrote: One question

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 11:53 AM, Jan Kara j...@suse.cz wrote: On Thu 08-08-13 08:56:28, Andy Lutomirski wrote: On Thu, Aug 8, 2013 at 3:18 AM, Jan Kara j...@suse.cz wrote: On Wed 07-08-13 11:00:52, Andy Lutomirski wrote: On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen dave.han...@intel.com

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Dave Hansen
I was coincidentally tracking down what I thought was a scalability problem (turned out to be full disks :). I noticed, though, that ext4 is about 20% slower than ext2/3 at doing write page faults (x-axis is number of tasks):

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-08 Thread Andy Lutomirski
On Thu, Aug 8, 2013 at 12:25 PM, Andy Lutomirski l...@amacapital.net wrote: Whoops -- I read your email too quickly. I haven't tried MADV_WILLNEED, but I think I tried reading each page to fault them in. Is there any reason to expect MADV_WILLNEED to do any better? I'll try to do some new

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Andy Lutomirski
On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen wrote: > On 08/07/2013 06:40 AM, Jan Kara wrote: >> One question before I look at the patches: Why don't you use fallocate() >> in your application? The functionality you require seems to be pretty >> similar to it - writing to an already allocated

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Dave Hansen
On 08/07/2013 06:40 AM, Jan Kara wrote: > One question before I look at the patches: Why don't you use fallocate() > in your application? The functionality you require seems to be pretty > similar to it - writing to an already allocated block is usually quick. One problem I've seen is that it

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Andy Lutomirski
On Wed, Aug 7, 2013 at 6:40 AM, Jan Kara wrote: > On Mon 05-08-13 12:43:58, Andy Lutomirski wrote: >> My application fallocates and mmaps (shared, writable) a lot (several >> GB) of data at startup. Those mappings are mlocked, and they live on >> ext4. The first write to any given page is slow

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Jan Kara
On Mon 05-08-13 12:43:58, Andy Lutomirski wrote: > My application fallocates and mmaps (shared, writable) a lot (several > GB) of data at startup. Those mappings are mlocked, and they live on > ext4. The first write to any given page is slow because > ext4_da_get_block_prep can block. This

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Jan Kara
On Mon 05-08-13 12:43:58, Andy Lutomirski wrote: My application fallocates and mmaps (shared, writable) a lot (several GB) of data at startup. Those mappings are mlocked, and they live on ext4. The first write to any given page is slow because ext4_da_get_block_prep can block. This means

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Andy Lutomirski
On Wed, Aug 7, 2013 at 6:40 AM, Jan Kara j...@suse.cz wrote: On Mon 05-08-13 12:43:58, Andy Lutomirski wrote: My application fallocates and mmaps (shared, writable) a lot (several GB) of data at startup. Those mappings are mlocked, and they live on ext4. The first write to any given page is

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Dave Hansen
On 08/07/2013 06:40 AM, Jan Kara wrote: One question before I look at the patches: Why don't you use fallocate() in your application? The functionality you require seems to be pretty similar to it - writing to an already allocated block is usually quick. One problem I've seen is that it

Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-07 Thread Andy Lutomirski
On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen dave.han...@intel.com wrote: On 08/07/2013 06:40 AM, Jan Kara wrote: One question before I look at the patches: Why don't you use fallocate() in your application? The functionality you require seems to be pretty similar to it - writing to an

[RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-05 Thread Andy Lutomirski
My application fallocates and mmaps (shared, writable) a lot (several GB) of data at startup. Those mappings are mlocked, and they live on ext4. The first write to any given page is slow because ext4_da_get_block_prep can block. This means that, to get decent performance, I need to write

[RFC 0/3] Add madvise(..., MADV_WILLWRITE)

2013-08-05 Thread Andy Lutomirski
My application fallocates and mmaps (shared, writable) a lot (several GB) of data at startup. Those mappings are mlocked, and they live on ext4. The first write to any given page is slow because ext4_da_get_block_prep can block. This means that, to get decent performance, I need to write