I think there is an api in the multi-block writer to disable writing
metadata. Utkarsh knows, he's the one that figured it out. not sure how
it applies to your use case. you may not even be using multi-block format...
On 05/23/2016 06:57 AM, Gallagher, Timothy P wrote:
Interesting -- I'd be willing to try it out if you have more details
(commit hashtag, ticket, etc.), even if I have to figure out how to
backport it from a newer version.
Thanks,
Tim
------------------------------------------------------------------------
*From:* Burlen Loring <burlen.lor...@gmail.com>
*Sent:* Sunday, May 22, 2016 10:30 PM
*To:* Gallagher, Timothy P; Andy Bauer
*Cc:* paraview@paraview.org
*Subject:* Re: [Paraview] Memory leak with Catalyst
btw, we just had similar problem that was entirely explained by vtk
writer attempting to gather a bunch of arrays to the root node.
Utkarsh found a way to disable that behavior. I wonder if you are
hitting the same.
On 5/20/2016 4:36 PM, Gallagher, Timothy P wrote:
Well... not going so well.
If I run a small simulation with massif and the Release build of
paraview, I don't see any growing memory in time. I also get my VTK
files and images.
If I run the same simulation linked against a Debug build with
VTK_DEBUG_LEAKS on, there is a segfault inside one of the
libvtkRendering libraries (it says libvtkRenderingOp but the rest of
the name is cut off). If I take the image rendering out of my
pipeline (comment out the RegisterView call) and leave the VTK file
writing, it runs fine and doesn't report any leaks. I tried to get
the segfault to drop a core file, but it won't do it.
I'm thinking that there is an issue in the rendering part causing the
leak. I have no idea what it could be though -- this is the 4.4.0
version, is anybody aware of any bug fixes between 4.4.0 and current
versions that might be relevant?
Thanks for the help,
Tim
------------------------------------------------------------------------
*From:* ParaView <paraview-boun...@paraview.org> on behalf of
Gallagher, Timothy P <tim.gallag...@gatech.edu>
*Sent:* Friday, May 20, 2016 4:17 PM
*To:* Burlen Loring; Andy Bauer
*Cc:* paraview@paraview.org
*Subject:* Re: [Paraview] Memory leak with Catalyst
Thanks for the advice Burlen -- I haven't used Massif before, so this
is a good opportunity to give it a try.
The initialize/finalize each time approach doesn't work either. I am
calling the Finalize() routine on the vtkCPProcessor class but then
when it tries to initialize again, it throws:
Warning: In
/data4/ParaView/VTK/Parallel/Core/vtkSocketController.cxx, line 50
vtkSocketController (0xa4342f0): Already initialized.
and then crashes. So, hopefully I can sort out the memory leak rather
than trying to debug why a hacked fix doesn't work!
Tim
------------------------------------------------------------------------
*From:* Burlen Loring <burlen.lor...@gmail.com>
*Sent:* Friday, May 20, 2016 3:46 PM
*To:* Gallagher, Timothy P; Andy Bauer
*Cc:* paraview@paraview.org
*Subject:* Re: [Paraview] Memory leak with Catalyst
VTK_DEBUG_LEAKS, although will catch some serious errors, will not
catch all kinds of leaks. For example you can have leaks where data
erroneously accumulates with each time step, but is deleted at
program termination. VTK_DEBUG_LEAKS will not catch that kind of
error. It's probably better to use massif to profile your code on a
small one node run. There's an excellent tool called massif
visualizer to aid in exploring the data generated.
$0.02
On 05/20/2016 11:56 AM, Gallagher, Timothy P wrote:
Hi Andy,
Thanks for the tips. I will get working on the VTK_DEBUG_LEAKS now
and see what it turns up.
The initialize/finalize every time is definitely a hack and not for
long-term use. But, sponsors want a report on Monday and we need to
be able to visualize things for that -- the simulation is too big to
write data files and post-process later. So I modified the code to
do that for now until I can find a proper fix.
I'll let you know if I get stuck with the log file.
Thanks again,
Tim
------------------------------------------------------------------------
*From:* Andy Bauer <andy.ba...@kitware.com>
*Sent:* Friday, May 20, 2016 2:39 PM
*To:* Gallagher, Timothy P
*Cc:* paraview@paraview.org
*Subject:* Re: [Paraview] Memory leak with Catalyst
Hi Tim,
If you build Catalyst with VTK_DEBUG_LEAKS enabled it is pretty good
at finding VTK objects that aren't deleted properly. You should be
able to run this with a small amount of calls to Catalyst as well.
If you try this and want help understanding the output (if an object
like a vtkUnstructuredGrid is leaked it will often give a whole
bunch of false leaks that the unstructured grid is just holding the
references to), just share your output file and I can take a look at
it to try and narrow down the culprit. That may be slightly easier
to use than Valgrind.
Beyond this, you could maybe try the same run but without doing any
Catalyst work to see what happens then. That may be a lot of compute
cycles but I'm not sure how else to try and bisect where the memory
leak is occurring.
Initializing and finalizing Catalyst every time you want output
would probably work but I'd think there may be significant overhead
doing it like this. Plus, it's not really solving the problem --
just avoiding it.
Best,
Andy
On Fri, May 20, 2016 at 12:57 PM, Gallagher, Timothy P
<tim.gallag...@gatech.edu <mailto:tim.gallag...@gatech.edu>> wrote:
Hi,
One of our users is running a very big simulation and writing
out images of two slices (two different views) every 1000
iterations and writing out the data for the two slices (two
different data writers) as VTK files every 5000 iterations. It
is using Paraview 4.4.
After 21000 iterations, the simulation is killed because the
memory on the compute nodes fills up. I usually know how to
track down memory problems in our code using valgrind and
related tools, but is that the right way to go to try and find
this problem?
Are there any tips on how to isolate where the problem may be? I
don't know if it is in the adapter, or in paraview itself. Has
anybody encountered problems with runaway memory using Catalyst
in 4.4 when writing images or VTK files?
I know when we use pvpython to generate images and loop over
many files, sometimes the memory also blows up and so we usually
move the loop over the files outside the pvpython script and
into a driver script that executes a new pvpython for each file.
Is there a way to shut down/start up Catalyst each time it needs
to write something? Is that advisable?
Thanks,
Tim
_______________________________________________
Powered by www.kitware.com <http://www.kitware.com>
Visit other Kitware open-source projects at
<http://www.kitware.com/opensource/opensource.html>http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
<http://paraview.org/Wiki/ParaView>http://paraview.org/Wiki/ParaView
Search the list archives at:
<http://markmail.org/search/?q=ParaView>http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
_______________________________________________
Powered bywww.kitware.com
Visit other Kitware open-source projects
athttp://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki
at:http://paraview.org/Wiki/ParaView
Search the list archives at:http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview