Re: [patch] speed up single bio_vec allocation

2006-12-14 Thread Jens Axboe
On Fri, Dec 08 2006, Chen, Kenneth W wrote: > > Chen, Kenneth wrote on Wednesday, December 06, 2006 10:20 AM > > > Jens Axboe wrote on Wednesday, December 06, 2006 2:09 AM > > > This is what I had in mind, in case it wasn't completely clear. Not > > > tested, other than it compiles. Basically it el

RE: [patch] speed up single bio_vec allocation

2006-12-08 Thread Chen, Kenneth W
> Chen, Kenneth wrote on Wednesday, December 06, 2006 10:20 AM > > Jens Axboe wrote on Wednesday, December 06, 2006 2:09 AM > > This is what I had in mind, in case it wasn't completely clear. Not > > tested, other than it compiles. Basically it eliminates the small > > bio_vec pool, and grows the b

Re: [patch] speed up single bio_vec allocation

2006-12-08 Thread Jens Axboe
On Thu, Dec 07 2006, Nate Diller wrote: > On 12/7/06, Chen, Kenneth W <[EMAIL PROTECTED]> wrote: > >Nate Diller wrote on Thursday, December 07, 2006 1:46 PM > >> the current code is straightforward and obviously correct. you want > >> to make the alloc/dealloc paths more complex, by special-casing

Re: [patch] speed up single bio_vec allocation

2006-12-07 Thread Andi Kleen
On Friday 08 December 2006 05:23, Chen, Kenneth W wrote: > Andi Kleen wrote on Thursday, December 07, 2006 6:28 PM > > "Chen, Kenneth W" <[EMAIL PROTECTED]> writes: > > > I tried to use cache_line_size() to find out the alignment of struct bio, > > > but > > > stumbled on that it is a runtime func

RE: [patch] speed up single bio_vec allocation

2006-12-07 Thread Chen, Kenneth W
Andi Kleen wrote on Thursday, December 07, 2006 6:28 PM > "Chen, Kenneth W" <[EMAIL PROTECTED]> writes: > > I tried to use cache_line_size() to find out the alignment of struct bio, > > but > > stumbled on that it is a runtime function for x86_64. > > It's a single global variable access: > > #d

Re: [patch] speed up single bio_vec allocation

2006-12-07 Thread Andi Kleen
"Chen, Kenneth W" <[EMAIL PROTECTED]> writes: > > I tried to use cache_line_size() to find out the alignment of struct bio, but > stumbled on that it is a runtime function for x86_64. It's a single global variable access: #define cache_line_size() (boot_cpu_data.x86_cache_alignment) Or do you m

Re: [patch] speed up single bio_vec allocation

2006-12-07 Thread Nate Diller
On 12/7/06, Chen, Kenneth W <[EMAIL PROTECTED]> wrote: Nate Diller wrote on Thursday, December 07, 2006 1:46 PM > the current code is straightforward and obviously correct. you want > to make the alloc/dealloc paths more complex, by special-casing for an > arbitrary limit of "small" I/O, AFAICT.

RE: [patch] speed up single bio_vec allocation

2006-12-07 Thread Chen, Kenneth W
Nate Diller wrote on Thursday, December 07, 2006 1:46 PM > the current code is straightforward and obviously correct. you want > to make the alloc/dealloc paths more complex, by special-casing for an > arbitrary limit of "small" I/O, AFAICT. of *course* you can expect > less overhead when you're

Re: [patch] speed up single bio_vec allocation

2006-12-07 Thread Nate Diller
On 12/7/06, Chen, Kenneth W <[EMAIL PROTECTED]> wrote: Nate Diller wrote on Thursday, December 07, 2006 11:22 AM > > > I still can't help but think we can do better than this, and that this > > > is nothing more than optimizing for a benchmark. For high performance > > > I/O, you will be doing >

RE: [patch] speed up single bio_vec allocation

2006-12-07 Thread Chen, Kenneth W
Nate Diller wrote on Thursday, December 07, 2006 11:22 AM > > > I still can't help but think we can do better than this, and that this > > > is nothing more than optimizing for a benchmark. For high performance > > > I/O, you will be doing > 1 page bio's anyway and this patch wont help > > > you at

Re: [patch] speed up single bio_vec allocation

2006-12-07 Thread Nate Diller
On 12/6/06, Chen, Kenneth W <[EMAIL PROTECTED]> wrote: Jens Axboe wrote on Wednesday, December 06, 2006 2:09 AM > > > I will try that too. I'm a bit touchy about sharing a cache line for > > > different bio. But given that there are 200,000 I/O per second we are > > > currently pushing the kern

RE: [patch] speed up single bio_vec allocation

2006-12-06 Thread Chen, Kenneth W
Jens Axboe wrote on Wednesday, December 06, 2006 2:09 AM > > > I will try that too. I'm a bit touchy about sharing a cache line for > > > different bio. But given that there are 200,000 I/O per second we are > > > currently pushing the kernel, the chances of two cpu working on two > > > bio that

Re: [patch] speed up single bio_vec allocation

2006-12-06 Thread Jens Axboe
On Wed, Dec 06 2006, Jens Axboe wrote: > I still can't help but think we can do better than this, and that this > is nothing more than optimizing for a benchmark. For high performance > I/O, you will be doing > 1 page bio's anyway and this patch wont help > you at all. Perhaps we can just kill bio_

Re: [patch] speed up single bio_vec allocation

2006-12-06 Thread Jens Axboe
On Mon, Dec 04 2006, Jens Axboe wrote: > On Mon, Dec 04 2006, Chen, Kenneth W wrote: > > > [...] > > > > > > Another idea would be to kill SLAB_HWCACHE_ALIGN (it's pretty pointless, > > > I bet), and always alloc sizeof(*bio) + sizeof(*bvl) in one go when a > > > bio is allocated. It doesn't add a

RE: [patch] speed up single bio_vec allocation

2006-12-04 Thread Chen, Kenneth W
Jens Axboe wrote on Monday, December 04, 2006 12:07 PM > On Mon, Dec 04 2006, Chen, Kenneth W wrote: > > On 64-bit arch like x86_64, struct bio is 104 byte. Since bio slab is > > created with SLAB_HWCACHE_ALIGN flag, there are usually spare memory > > available at the end of bio. I think we can u

Re: [patch] speed up single bio_vec allocation

2006-12-04 Thread Jens Axboe
On Mon, Dec 04 2006, Chen, Kenneth W wrote: > > [...] > > > > Another idea would be to kill SLAB_HWCACHE_ALIGN (it's pretty pointless, > > I bet), and always alloc sizeof(*bio) + sizeof(*bvl) in one go when a > > bio is allocated. It doesn't add a lot of overhead even for the case > > where we do

Re: [patch] speed up single bio_vec allocation

2006-12-04 Thread Jens Axboe
On Mon, Dec 04 2006, Chen, Kenneth W wrote: > On 64-bit arch like x86_64, struct bio is 104 byte. Since bio slab is > created with SLAB_HWCACHE_ALIGN flag, there are usually spare memory > available at the end of bio. I think we can utilize that memory for > bio_vec allocation. The purpose is no

[patch] speed up single bio_vec allocation

2006-12-04 Thread Chen, Kenneth W
On 64-bit arch like x86_64, struct bio is 104 byte. Since bio slab is created with SLAB_HWCACHE_ALIGN flag, there are usually spare memory available at the end of bio. I think we can utilize that memory for bio_vec allocation. The purpose is not so much on saving memory consumption for bio_vec,