On 10/03/2010 21:19, Timo Sirainen wrote:
On 10.8.2009, at 20.01, Timo Sirainen wrote:
(3.5. Implement async I/O filesystem backend.)
You know what I found out today? Linux doesn't support async IO for regular
buffered files. I had heard there were issues, but I thought it was mainly
about some annoying APIs and such. Anyone know if some project has successfully
figured out some usable way to do async disk IO? The possibilities seem to be:
a) Use Linux's native AIO, which requires direct-io for files. This *might* not
be horribly bad for mail files. After all, same mail is rarely read multiple
times. Except when parsing its headers first and then its body. Maybe the
process could do some internal buffering?..
I guess no one ever tried my posix_fadvise() patch? The idea was that it would
tell the kernel after closing a mail file that it's no longer needed in memory,
so kernel could remove it from page cache. I never heard any positive or
negative comments about how it affected performance..
http://dovecot.org/patches/1.1/fadvise.diff
b) Use threads, either via some library or implement yourself. Each thread of
course uses some extra memory. Also enabling threads causes glibc to start
using a thread-safe version of malloc() (I think?), which slows things down
(unless that can be avoided, maybe by using clone() directly instead of
pthreads?).
c) I read someone's idea about using posix_fadvise() and fincore() functions to somehow
make it "kind of work, usually, maybe". I'm not sure if there's a practical way
to make them work though. And of course I don't think fincore() has even been accepted by
Linus yet.
Perhaps mail this question to the kernel list, stand back and watch it
ignite?
Ed
