On 2009-01-09, Sion Arrowsmith <si...@chiark.greenend.org.uk> wrote: > Grant Edwards <inva...@invalid> wrote: >>On 2009-01-09, Johannes Bauer <dfnsonfsdu...@gmx.de> wrote: >>> I've come from C/C++ and am now trying to code some Python because I >>> absolutely love the language. However I still have trouble getting >>> Python code to run efficiently. Right now I have a easy task: Get a >>> file, >>If I were you, I'd try mmap()ing the file instead of reading it >>into string objects one chunk at a time. > > You've snipped the bit further on in that sentence where the > OP says that the file of interest is 2GB. Do you still want to > try mmap'ing it?
Sure. The larger the file, the more you gain from mmap'ing it. 2GB should easily fit within the process's virtual memory space. When you mmap a file, it doesn't take up any physical memory. As you access different parts of it, pages are swapped in/out by the OS's VM system. If you're using a decent OS, the demand-paged VM system will handle things far more efficiently than creating millions of strings and letting the Python garbage collector clean them up. Or does "mmap" in Python mean something completely different than "mmap" in the C library? -- Grant Edwards grante Yow! Everybody gets free at BORSCHT! visi.com -- http://mail.python.org/mailman/listinfo/python-list