On Sun, Oct 15, 2006 at 02:26:12PM -0400, Neil Conway wrote: > On Sun, 2006-10-15 at 19:56 +0200, Martijn van Oosterhout wrote: > > Sure, I even implemented it once. Didn't get any faster. > > Did you just do something akin to s/read/aio_read/ etc., or something > more ambitious? I think that really taking advantage of the ability to > have multiple I/O requests outstanding would take some leg work.
Sure. Basically, at certain strategic points in the code there were extra ReadAsyncBuffer() commands (the IndexScan node and the b-tree scan code). This command was allowed to do nothing, but if there were not too many outstanding requests and a buffer was available, it would allocate a buffer and initiate an AIO request for it. IIRC there was a table of outstanding requests (I think I originally allowed up to 32) and when a normal ReadBuffer() found the block had already been requested, it "waited" on that block. The principle was that the index-scan node would read a page full of tids, submit a ReadAsyncBuffer() on each one, and then proceed as normal. Fairly unintrusive patch all up. ifdeffing it out is safe, and #defineing ReadAsyncBuffer() away causes the compiler to optimise the loop away altogether. The POSIX AIO layer sucks somewhat so it was tricky but it did work. The hardest part is really how to decide if a buffer currently in the buffercache is worth more than an asyncronously loaded buffer that may not be used. I posted the results ot -hackers some time ago, so you can always try that. > At least according to [1], kernel AIO on Linux still doesn't work for > buffered (i.e. non-O_DIRECT) files. There have been patches available > for quite some time that implement this, but I'm not sure when they are > likely to get into the mainline kernel. You can also do it by spawning off threads to do the requests. The glibc emulation uses threads, but only allows one outstanding request per file, which makes it useless for our purposes... Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to > litigate.
signature.asc
Description: Digital signature
