Bill Stoddard wrote:
Why not use mod_file_cache?
On my wimpy 200MHz server, the SPEC file_set contains 5760 files and uses .8G of disk. On more modern servers, the size of the file_set goes up in proportion to the number of conforming connections you hope to push thru it, so figure 57,600 files and 8G of space on a 2GHz server. 8G isn't going to fit into one process's virtual memory on a Pentium; caching 57,600 fd's is going to be a challenge too, as I think Madhu is finding out.
One way to make this easier might be to cache as many of the smaller files in memory as will reasonably fit, then caching the file descriptors for the remaining files. If that still isn't practical, you could fagettabout caching the very biggest files since they will take a long time to serve no matter what.
On the hypothetical 2GHz server, if I cached all the class0_* thru class2_* (smallest) files in memory, that would eat about .8G per process, leaving 14,400 larger class3_* files to deal with somehow.
A disadvantage of caching in memory might be higher CPU usage from copying + checksumming data from user space to NIC buffers vs. using sendfile. That probably doesn't matter for really tiny files.
Greg
