Derek Martin <[EMAIL PROTECTED]> writes: > > Both are clocking in at the same time (1m 5sec for 2.6Gb), are there > > any ways I can optimize either solution?
Getting 40+ MB/sec through a file system is pretty impressive. Sounds like a RAID? > That said, due to normal I/O generally involving double-buffering, you > might be able to speed things up noticably by using Memory-Mapped I/O > (MMIO). It depends on whether or not the implementation of the Python > things you're using already use MMIO under the hood, and whether or > not MMIO happens to be broken in your OS. :) Python has the mmap module and I use it sometimes, but it's not necessarily the right thing for something like this. Each page you try to read from results in own delay while the resulting page fault is serviced, so any overlapped i/o you get comes from the OS being nice enough to do some predictive readahead for you on sequential access if it does that. By coincidence there are a couple other threads mentioning AIO which is a somewhat more powerful mechanism. -- http://mail.python.org/mailman/listinfo/python-list
