Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-19 Thread Brian Akins
Colm MacCarthaigh wrote: The optimisations wouldn't be removing buffering, they'd be using a different kind of buffering :-) If MMap is not available/enabled we can fail back to a buffered APR read. I did some tinkering with mod_disk_cache. I encapsulated all the reads of the headers file

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Colm MacCarthaigh wrote: The optimisations wouldn't be removing buffering, they'd be using a different kind of buffering :-) If MMap is not available/enabled we can fail back to a buffered APR read. So it's evolved even further... I would be interested to see in a variety of cases which one i

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Colm MacCarthaigh
On Thu, Aug 18, 2005 at 10:43:45AM -0700, Justin Erenkrantz wrote: > None of the code in mod_disk_cache used buffering before I turned it on. > It gave significant speedups in my performance tests by reducing the > syscall overhead. I also had identified and fixed some bugs in apr's > bufferin

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Justin Erenkrantz wrote: None of the code in mod_disk_cache used buffering before I turned it on. It gave significant speedups in my performance tests by reducing the syscall overhead. I also had identified and fixed some bugs in apr's buffering code to go along with these speedups that went

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Justin Erenkrantz
--On August 18, 2005 1:34:56 PM -0400 Brian Akins <[EMAIL PROTECTED]> wrote: The optimizations that Colm is talking about should be helpful everywhere. Also, that's why we test to make sure it does "kill us." Also, I don't think we "know" it will kill us. None of the code in mod_disk_cache us

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Justin Erenkrantz wrote: But, that's why we do the buffered read. It almost always ends up being one read() for me as the header file is always less than 4k. True. I don't believe we should necessarily optimize based on the results of one platform when we know its going to kill us everywhe

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Justin Erenkrantz
--On August 18, 2005 3:59:05 PM +0100 Colm MacCarthaigh <[EMAIL PROTECTED]> wrote: Thinking about it, everything is going into memory anyway, so why not just stat() the file and read it all in in one go? We know it's never going to be that big anyway. This completely minimises the number of re

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Jeff Trawick
On 8/18/05, Brian Akins <[EMAIL PROTECTED]> wrote: > I didn't mess with the EBCDIC stuff, so alot of the patch is just where > I commented all that out. The EBCDIC stuff came here: http://svn.apache.org/viewcvs.cgi?rev=105236&view=rev Apparently Justin needed logic similar to ap_scan_script_he

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Colm MacCarthaigh wrote: Do you mean as in using MAP_SHARED? I could see that being a race-condition nightmare. For example cache_info_t's pointers to the header tables would have to be local to the process, but the memory for the cache_info_t itself wouldn't be. Definitely to be avoided! True.

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Colm MacCarthaigh
On Thu, Aug 18, 2005 at 12:11:56PM -0400, Brian Akins wrote: > >Yep, but there are definite speed-up's to be had. I'm going to > >canibalise your patch and try some of the above anyway. > > Cool. I'd be willing to help. I'll be putting on-line all of my TODO's and patches-in-progress shortly, I h

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Colm MacCarthaigh wrote: Thinking about it, everything is going into memory anyway, so why not just stat() the file and read it all in in one go? Yes! Brilliant. We need all that data anyway. so: open file. length = get_info read_full(file, length). manipulate pointers for table (apr_t

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Colm MacCarthaigh
On Thu, Aug 18, 2005 at 10:43:41AM -0400, Brian Akins wrote: > >I can understand why this is faster, I'm guessing that you've enough RAM > >that you're retrieving files from the vmcache and that the higher-layer > >buffering is just overhead. This is probably going to be the majority > >case, but w

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Colm MacCarthaigh wrote: I can understand why this is faster, I'm guessing that you've enough RAM that you're retrieving files from the vmcache and that the higher-layer buffering is just overhead. This is probably going to be the majority case, but would do with testing. Not to heap more conf

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Colm MacCarthaigh
On Thu, Aug 18, 2005 at 09:22:13AM -0400, Brian Akins wrote: > Thanks to all who reminded me what a dumb-a## I am this morning... > > I forgot the patch. Here it is. Cool. > diff -ru httpd-trunk.orig/modules/cache/mod_disk_cache.c > httpd-trunk.new2/modules/cache/mod_disk_cache.c > --- httpd-t

Re: [PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
Thanks to all who reminded me what a dumb-a## I am this morning... I forgot the patch. Here it is. Caffiene level much to low... Brian Akins wrote: This one's kinda ugly. Rather than rely on apr_file_gets, this stores the total length of the tables, then the serialized table in store_ta

[PATCH] mod_disk_cache: speed up read_table

2005-08-18 Thread Brian Akins
This one's kinda ugly. Rather than rely on apr_file_gets, this stores the total length of the tables, then the serialized table in store_table. In read_table, it reads this length, allocs that amount, and reads the headers into the buffer. Then it just uses memchr to "parse" it into a table.