Sorry I've got no insight on this. On Apr 21, 2016 6:29 AM, "Biddiscombe, John A." <biddi...@cscs.ch> wrote:
Dave So it turns out that aprun/alps won’t let me run a script with dstat to monitor the memory use, and using craypat makes the execution take too long (weeks it seems). So I tried importing part of the dstat python into my paraview python Using this I am able to query memory use on the node and dump it out. Question : If I start a background python thread that polls memory like this ##################### # start a thread that prints mem use ##################### def dump_memory(): # do something here ... global stop_thread line = '' for o in stats: o.extract() line = line + ' ' + o.show() print line + dstat.ansi['reset'] # call f() again in n seconds if stop_thread==0: threading.Timer(0.5, dump_memory).start() and set it going at the start of my paraview python pipeline : Will it actually run and collect info whilst the paraview script is executing the rest of the pipeline. intuition and brief googling tells me that this won’t work. Can it be done? JB *From:* ParaView [mailto:paraview-boun...@paraview.org] *On Behalf Of *Biddiscombe, John A. *Sent:* 18 April 2016 14:53 *To:* David E DeMarle *Cc:* paraview@paraview.org *Subject:* Re: [Paraview] peak memory use on remote nodes Thanks Dave A colleague here suggested I put my aprun pvbatch <blah> into a script and add a ‘dtrace” command at the start of the script. This will then dump out a txt file on each node and after the job completes I can grep the info I need. I’m going to give that a try first as it seems straightforward, but I will revert to your suggestion if I fail and nothing else comes up. cheers JB *From:* David E DeMarle [mailto:dave.dema...@kitware.com <dave.dema...@kitware.com>] *Sent:* 18 April 2016 14:48 *To:* Biddiscombe, John A. *Cc:* paraview@paraview.org *Subject:* Re: [Paraview] peak memory use on remote nodes Perhaps extend the benchmark to glean /proc/<PID>/status's VmHWM entry on Linux at least. Get the number for t1 and at t2 and subtract to see what the maximum was in between those two times. Seems the right place to do that is kwsys/SystemInformation.cxx, but you could prototype in the script. Hope that helps and please post some code. Seems like a very useful thing to have. David E DeMarle Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4909 On Mon, Apr 18, 2016 at 2:42 AM, Biddiscombe, John A. <biddi...@cscs.ch> wrote: The python scripting module “paraview.benchmark” allows one to get the memory use using “paraview.benchmark.get_memuse” - but I presume this is the memory reported as used by the system at the moment when the function is called. Does anyone know a way of recording the peak memory use on remote nodes between t1 and t2 - where t1 and t2 are the start and stop of either a function, job or even paraview python script on the remote node? thanks JB -- John Biddiscombe, email:biddisco @.at.@ cscs.ch http://www.cscs.ch/ CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07 Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82 _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview