guido, On Thu, Aug 30, 2012 at 12:04 PM, Guido Fratesi <fratesi at mater.unimib.it> wrote: > I'm sorry for a previous incomplete message. > >> sed -ri "s/(^ *)(allocate.*$)/\1\2\n\1 CALL mem_whatever()/i" $(find >> /where/is/espresso -name \*.f90) > > That was very useful, thank you. > > I now get 1.9GB out of 2.4, which starts giving some usable estimate, > but I do understand that getting the accurate value is very complex.
quantifying memory usage precisely on unix/linux machines with virtual memory management and memory sharing is almost impossible. you have multiple components to worry about: - address space (memory reserved to be used, but initially all mapped to the same copy-on-write location) - resident set size (actual physical memory used) - shared memory (it is not memory that is shared, but more a measure for how much sharing is going on) - swap space. - device memory (from infiniband cards for example) - pinned memory (allocated memory that cannot be swapped, usually used to back device memory) so what is the real memory usage is difficult to determine. address space (VMEM) is usually too large, resident set size (RSS) does not consider memory that is swapped out, so it is often too small. using many MPI tasks drives up the address space for device memory (which doesn't increase real memory usage, but also requires more pinned memory, which makes swapping more likely). multi-threading results in a lot of sharing. ...and tracking allocations in the code only handles explicit allocations, not those incurred by the fortran language on the stack or otherwise. so for all practical purposes you can say that memory use is usually somewhere between VMEM and RSS, but that can be pretty far apart. axel. > To my purpose, I'll monitor the memory occupancy by a script, which > follows below in case someone finds it useful... > Guido > > ================ > > command=pw.x > > maxsecs=$((60*60*24)) > delay=1 > nsteps=$((maxsecs/delay)) > > echo "#when RAM PID (0 for all $command instances)"} > > for ((i=0;((i<nsteps));i++)); do > #timer=`date +%H:%M:%S` > #timer=`date +"%H:%M:%S %s.%N"` > timer=`date +"%s.%N"` > ps -eo comm,rss,pid | > awk -v comm=$command -v timer=$timer ' > BEGIN {tot=0} > ($1==comm) { > tot+=$2; > print timer, $2, $3; > } > END {if (tot) print timer, tot, 0} > ' > sleep 1 > done > > > > -- > Guido Fratesi > > Dipartimento di Scienza dei Materiali > Universita` degli Studi di Milano-Bicocca > via Cozzi 53, 20125 Milano, Italy > > _______________________________________________ > Pw_forum mailing list > Pw_forum at pwscf.org > http://www.democritos.it/mailman/listinfo/pw_forum -- Dr. Axel Kohlmeyer akohlmey at gmail.com http://goo.gl/1wk0 International Centre for Theoretical Physics, Trieste. Italy.