Erin Sheldon writes: [...] > This was why I essentially wrote my own memmap like interface with > recfile, the code I'm converting. It allows working with columns and > rows without loading large chunks of memory. [...]
This sounds like at any point in time you only have one part of the array mapped into the application. My question is then, why would you manually implement the buffering? The OS should already take care of that by unmapping pages when it's short on physical memory, and faulting pages in when you access them. This reminds me of some previous discussion about making the ndarray API more friendly to code that wants to manage the underlying storage, from mmap'ing it to handling compressed storage. Are there any news on that front? Lluis -- "And it's much the same thing with knowledge, for whenever you learn something new, the whole world becomes that much richer." -- The Princess of Pure Reason, as told by Norton Juster in The Phantom Tollbooth _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
