Re: Disk Cache, Was: O_DIRECT question
Zan Lynx wrote: > On Sat, 2007-01-13 at 00:03 +0300, Michael Tokarev wrote: > [snip] >> And sure thing, withOUT O_DIRECT, the whole system is almost dead under this >> load - because everything is thrown away from the cache, even caches of /bin >> /usr/bin etc... ;) (For that, fadvise() seems to help a bit, but not alot). > > One thing that I've been using, and seems to work well, is a customized > version of the readahead program several distros use during boot up. [idea to lock some (commonly-used) cache pages in memory] > Something like that could keep your system responsive no matter what the > disk cache is doing otherwise. Unfortunately it's not. Sure, things like libc.so etc will be force-cached and will start fast. But not my data files and other stuff (what an unfortunate thing: memory usually is smaller in size than disks ;) I can do usual work without noticing something's working with the disks intensively, doing O_DIRECT I/O. For example, I can run large report on a database, which requires alot of disk I/O, and run a kernel compile at the same time. Sure, disk access is alot slower, but disk cache helps alot, too. My kernel compile will not be much slower than usual. But if I'll turn O_DIRECT off, the compile will take ages to finish. *And* the report running, too! Because the system tries hard to cache the WRONG pages! (yes I remember fadvise - which aren't used by the database(s) currently, and quite alot of words has been said about that, too; I also noticied it's slower as well, at least currently.) /mjt - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Disk Cache, Was: O_DIRECT question
On Sat, 2007-01-13 at 00:03 +0300, Michael Tokarev wrote: [snip] > And sure thing, withOUT O_DIRECT, the whole system is almost dead under this > load - because everything is thrown away from the cache, even caches of /bin > /usr/bin etc... ;) (For that, fadvise() seems to help a bit, but not alot). One thing that I've been using, and seems to work well, is a customized version of the readahead program several distros use during boot up. Mine starts off doing: mlockall(MCL_CURRENT|MCL_FUTURE); ...yadda, yadda... and for each file listed: ...open, stat stuff... if( NULL == mmap( NULL, stat_buf.st_size, PROT_READ, MAP_SHARED|MAP_LOCKED|MAP_POPULATE, fd, 0) ) { fprintf(stderr, "'%s' ", file); perror("mmap"); } ...more stuff... and then ends with: pause(); and it sits there forever. As far as I can tell, this makes the program and library code stay in RAM. At least, after a drop_caches nautilus doesn't load 12 MB off disk, it just starts. It has to be reloaded after software updates and after prelinking. I find the 250 MB used to be worthwhile, even if its kinda Windowsey. Something like that could keep your system responsive no matter what the disk cache is doing otherwise. -- Zan Lynx <[EMAIL PROTECTED]> signature.asc Description: This is a digitally signed message part
Disk Cache, Was: O_DIRECT question
On Sat, 2007-01-13 at 00:03 +0300, Michael Tokarev wrote: [snip] And sure thing, withOUT O_DIRECT, the whole system is almost dead under this load - because everything is thrown away from the cache, even caches of /bin /usr/bin etc... ;) (For that, fadvise() seems to help a bit, but not alot). One thing that I've been using, and seems to work well, is a customized version of the readahead program several distros use during boot up. Mine starts off doing: mlockall(MCL_CURRENT|MCL_FUTURE); ...yadda, yadda... and for each file listed: ...open, stat stuff... if( NULL == mmap( NULL, stat_buf.st_size, PROT_READ, MAP_SHARED|MAP_LOCKED|MAP_POPULATE, fd, 0) ) { fprintf(stderr, '%s' , file); perror(mmap); } ...more stuff... and then ends with: pause(); and it sits there forever. As far as I can tell, this makes the program and library code stay in RAM. At least, after a drop_caches nautilus doesn't load 12 MB off disk, it just starts. It has to be reloaded after software updates and after prelinking. I find the 250 MB used to be worthwhile, even if its kinda Windowsey. Something like that could keep your system responsive no matter what the disk cache is doing otherwise. -- Zan Lynx [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: Disk Cache, Was: O_DIRECT question
Zan Lynx wrote: On Sat, 2007-01-13 at 00:03 +0300, Michael Tokarev wrote: [snip] And sure thing, withOUT O_DIRECT, the whole system is almost dead under this load - because everything is thrown away from the cache, even caches of /bin /usr/bin etc... ;) (For that, fadvise() seems to help a bit, but not alot). One thing that I've been using, and seems to work well, is a customized version of the readahead program several distros use during boot up. [idea to lock some (commonly-used) cache pages in memory] Something like that could keep your system responsive no matter what the disk cache is doing otherwise. Unfortunately it's not. Sure, things like libc.so etc will be force-cached and will start fast. But not my data files and other stuff (what an unfortunate thing: memory usually is smaller in size than disks ;) I can do usual work without noticing something's working with the disks intensively, doing O_DIRECT I/O. For example, I can run large report on a database, which requires alot of disk I/O, and run a kernel compile at the same time. Sure, disk access is alot slower, but disk cache helps alot, too. My kernel compile will not be much slower than usual. But if I'll turn O_DIRECT off, the compile will take ages to finish. *And* the report running, too! Because the system tries hard to cache the WRONG pages! (yes I remember fadvise Co - which aren't used by the database(s) currently, and quite alot of words has been said about that, too; I also noticied it's slower as well, at least currently.) /mjt - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/