On Wed, Jul 1, 2009 at 6:14 PM, Pauli Virtanen<[email protected]> wrote: > Wed, 01 Jul 2009 10:17:51 +0200, Emmanuelle Gouillart kirjoitti: >> I'm using numpy.memmap to open big 3-D arrays of Xray tomography >> data. After I have created a new array using memmap, I modify the >> contrast of every Z-slice (along the first dimension) inside a for loop, >> for a better visualization of the data. Although I call memmap.flush >> after each modification of a Z-slice, the memory used by Ipython keeps >> increasing at every new iteration. At the end of the loop, the memory >> used by Ipython is of the order of magnitude of the size of the data >> file (1.8Go !). I would have expected that the maximum amount of memory >> used would corresponde to only one Z-slice of the 3D array. See the code >> snapshots below for more details. >> >> Is this an expected behaviour? How can I reduce the amount of >> memory used by Ipython and still process my data? > > How do you measure the memory used? Note that on Linux, "top" includes > the size of OS caches for the memmap in the RSS size of a process. > You can try to monitor "free" instead:
A good tool on linux is exmap: http://ktown.kde.org/~seli/memory/analysis.html Unfortunately, it is static, but more accurate than free or top. David _______________________________________________ Numpy-discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
