After reading about memory leaks I modified the script at the end by
adding "del" after "Delete":
Delete(DataRepresentation4)
del DataRepresentation4
Delete(Contour2)
del Contour2
Delete(DataRepresentation3)
del DataRepresentation3
Delete(Q_VP_xmf)
del Q_VP_xmf
Delete(DataRepresentation2)
del DataRepresentation2
Delete(Contour1)
del Contour1
Delete(DataRepresentation1)
del DataRepresentation1
Delete(cf_VP_xmf)
del cf_VP_xmf
That should fully delete the objects from the memory and it helped a
little but I still have memory leaks. Before the script memory usage
in the Memory Inspector is 1.4Gb and after one iteration it increases
to 4Gb, compared to 10Gb without the del statements. After 4
iterations the memory usage increases to 8Gb. Any ideas on how to
track and eliminate completely the memory leaks? I am forgetting to
delete an object from the script?
On Wed, Jan 7, 2015 at 9:17 PM, Nikolaos Beratlis
<nikos.berat...@gmail.com <mailto:nikos.berat...@gmail.com>> wrote:
So I did some more tests by running the script interactively. I
used a smaller grid first, 2002 x 20 x 1000 points and ran it only
for 1 iteration. When Paraview loaded the memory usage on the
server was 1.04GB. After doing one iteration the memory usage
spiked to 10.42GB and remained like that when the script finished.
What I found out is the Delete statements make no difference at
all, with or without them the memory usage remained 10.42GB.
Performing more iterations in the script actually worsens this
problem, after 4 iterations the memory used increased to 14.32GB.
With a larger grid of 2002 x 100 x 1000 points the memory after
one iteration increased from 10.42GB before the script to 45.25GB
after the script. How can I properly delete all datasets and clear
the memory in Paraview?
On Wed, Jan 7, 2015 at 2:55 PM, Nikolaos Beratlis
<nikos.berat...@gmail.com <mailto:nikos.berat...@gmail.com>> wrote:
I am trying to run a script on a cluster with Paraview in
parallel. The cluster architecture is 16cores and 64GB per
node. The script reads in two XMF files, one contains a grid
of 2002x100x1000 points along with one variable. There is a
loop to perform two iterations:
try: paraview.simple
except: from paraview.simple import *
paraview.simple._DisableFirstRenderCameraReset()
RenderView1 = GetRenderView()
RenderView1.Background = [1.0, 1.0, 1.0]
view = GetActiveView()
view.ViewSize = [ 800, 500 ]
RenderView1.CenterAxesVisibility = 0
#RenderView1.OrientationAxesVisibility = 0
#Define variables
wdir =
'/lustre/groups/balarasgrp/nikosb/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'
#wdir =
'/Users/nikosb/colone/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm'
#wdir =
'/Users/nikosb/Research/Simulations/SPHERE/Re100/gridC1/run'
Qval = 0.1
cfmin = -0.001
cfmax = 0.005
index = [None]*9999
m = 0
for i in range(0, 9):
for j in range(0, 9):
for k in range(0, 9):
for l in range(0, 9):
index[m] = chr(i+48)+chr(j+48)+chr(k+48)+chr(l+48)
m = m+1
for i in range(1,3):
cf_VP_xmf = XDMFReader(
FileName=wdir+'/VPfield/XDMF/cf_VP'+index[i]+'.xmf' )
cf_VP_xmf.Sets = []
cf_VP_xmf.Grids = ['STL Grid']
cf_VP_xmf.PointArrays = ['Cf']
RenderView1 = GetRenderView()
RenderView1.CenterOfRotation = [6.2286853790283203e-06,
2.5033950805664062e-06, 7.1227550506591797e-06]
DataRepresentation1 = Show()
DataRepresentation1.EdgeColor = [0.0, 0.0, 0.50000762951094835]
DataRepresentation1.SelectionPointFieldDataArrayName = 'Cf'
DataRepresentation1.ColorArrayName = ('POINT_DATA', 'Cf')
DataRepresentation1.ScalarOpacityUnitDistance =
0.033928799990483226
DataRepresentation1.ScaleFactor = 0.099998557567596444
a1_Cf_PVLookupTable = GetLookupTableForArray( "Cf", 1,
RGBPoints=[-0.0099065126851201057, 0.23000000000000001,
0.29899999999999999, 0.754, 0.0097341262735426426,
0.86499999999999999, 0.86499999999999999, 0.86499999999999999,
0.029374765232205391, 0.70599999999999996, 0.016,
0.14999999999999999], VectorMode='Magnitude', NanColor=[0.25,
0.0, 0.0], ColorSpace='Diverging', ScalarRangeInitialized=1.0 )
a1_Cf_PiecewiseFunction = CreatePiecewiseFunction(
Points=[-0.0099065126851201057, 0.0, 0.5, 0.0,
0.029374765232205391, 1.0, 0.5, 0.0] )
DataRepresentation1.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction
DataRepresentation1.LookupTable = a1_Cf_PVLookupTable
a1_Cf_PVLookupTable.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction
a1_Cf_PVLookupTable.RGBPoints = [cfmin, 0.23000000000000001,
0.29899999999999999, 0.754, -0.002, 0.86499999999999999,
0.86499999999999999, 0.86499999999999999, cfmax,
0.70599999999999996, 0.016, 0.14999999999999999]
a1_Cf_PVLookupTable.ScalarOpacityFunction =
a1_Cf_PiecewiseFunction
a1_Cf_PVLookupTable.LockScalarRange = 1
RenderView1.CameraPosition = [6.2286853790283203e-06,
2.5033950805664062e-06, 3.3460035306026636]
RenderView1.CameraFocalPoint = [6.2286853790283203e-06,
2.5033950805664062e-06, 7.1227550506591797e-06]
RenderView1.CameraClippingRange = [2.3175641399614015,
4.6486514901655163]
RenderView1.CameraParallelScale = 0.86600759519558368
Contour1 = Contour( PointMergeMethod="Uniform Binning" )
Contour1.PointMergeMethod = "Uniform Binning"
Contour1.ContourBy = ['POINTS', 'Cf']
Contour1.Isosurfaces = [0.0097341262735426426]
DataRepresentation2 = Show()
DataRepresentation2.ColorArrayName = ('POINT_DATA', '')
DataRepresentation2.ScaleFactor = 0.099971860647201538
DataRepresentation2.SelectionPointFieldDataArrayName = 'Cf'
DataRepresentation2.EdgeColor = [0.0, 0.0, 0.50000762951094835]
DataRepresentation2.Representation = 'Wireframe'
DataRepresentation2.AmbientColor = [0.0, 0.0, 0.0]
DataRepresentation1.Visibility = 0
RenderView1.CameraClippingRange = [2.7470738944403328,
4.2397684828477349]
DataRepresentation1.Visibility = 1
RenderView1.CameraClippingRange = [2.3175641399614015,
4.6486514901655163]
Q_VP_xmf = XDMFReader(
FileName=wdir+'/VPfield/XDMF/Q_VP'+index[i]+'.xmf' )
Contour1.Isosurfaces = [0.0]
Q_VP_xmf.Sets = []
Q_VP_xmf.Grids = ['Eulerian Grid']
Q_VP_xmf.PointArrays = ['Q']
DataRepresentation3 = Show()
DataRepresentation3.EdgeColor = [0.0, 0.0, 0.50000762951094835]
DataRepresentation3.SelectionPointFieldDataArrayName = 'Q'
DataRepresentation3.ScalarOpacityUnitDistance =
0.0045778956118952439
DataRepresentation3.Representation = 'Outline'
DataRepresentation3.ScaleFactor = 0.25585924386978148
RenderView1.CameraClippingRange = [0.30866115510384229,
6.1311800002592065]
Contour2 = Contour( PointMergeMethod="Uniform Binning" )
Contour2.PointMergeMethod = "Uniform Binning"
Contour2.ContourBy = ['POINTS', 'Q']
Contour2.Isosurfaces = [-348374.28125]
DataRepresentation4 = Show()
DataRepresentation4.ScaleFactor = 0.1348903000354767
DataRepresentation4.SelectionPointFieldDataArrayName = 'Normals'
DataRepresentation4.EdgeColor = [0.0, 0.0, 0.50000762951094835]
Contour2.Isosurfaces = [0.5]
DataRepresentation3.Visibility = 0
RenderView1.CameraClippingRange = [0.46635874330917981,
5.5760844897764175]
RenderView1.CameraViewUp = [-1.0, 0.0, 0.0]
RenderView1.CameraPosition = [-5.0, 0.0, 5.0]
RenderView1.CameraClippingRange = [3.0180305778980254,
7.5069525367021557]
RenderView1.CameraFocalPoint = [0.0, 0.0, 5.0]
RenderView1.CameraParallelScale = 1.565457167140391
RenderView1.CenterOfRotation = [0.0, 0.0, 0.0]
RenderView1.CameraFocalPoint = [0.0, 0.0, 0.5]
RenderView1.CameraClippingRange = [0.86197012554183883,
4.2263653610655147]
RenderView1.CameraPosition = [0.0, -2.5, 0.0]
WriteImage(wdir+'/POSTPRO/IMAGES/Q_cf_VP'+index[i]+'.jpg')
Delete(DataRepresentation4)
Delete(Contour2)
Delete(DataRepresentation3)
Delete(Q_VP_xmf)
Delete(DataRepresentation2)
Delete(Contour1)
Delete(DataRepresentation1)
Delete(cf_VP_xmf)
The script was created by running an interactive Paraview
session and recording the commands. I then modified it
slightly to add the loop and some variables.
I have been able to run the script successfully on 1,2,16 and
32 cores successfully and it loops twice. However when I try
to run on 64 cores I get the following error message:
[node116:58372] 63 more processes have sent help message
help-mpi-common-cuda.txt / dlopen failed
[node116:58372] Set MCA parameter "orte_base_help_aggregate"
to 0 to see all help / error messages
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
[node125:37180] *** Process received signal ***
[node125:37180] Signal: Aborted (6)
[node125:37180] Signal code: (-6)
[node125:37180] [ 0] /lib64/libpthread.so.0(+0xf710)
[0x2aaab28f3710]
[node125:37180] [ 1] /lib64/libc.so.6(gsignal+0x35)
[0x2aaab479f625]
[node125:37180] [ 2] /lib64/libc.so.6(abort+0x175)
[0x2aaab47a0e05]
[node125:37180] [ 3]
/usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)
[0x2aaab408aa7d]
[node125:37180] [ 4] /usr/lib64/libstdc++.so.6(+0xbcbd6)
[0x2aaab4088bd6]
[node125:37180] [ 5] /usr/lib64/libstdc++.so.6(+0xbcc03)
[0x2aaab4088c03]
[node125:37180] [ 6] /usr/lib64/libstdc++.so.6(+0xbcd22)
[0x2aaab4088d22]
[node125:37180] [ 7] /usr/lib64/libstdc++.so.6(_Znwm+0x7d)
[0x2aaab408911d]
[node125:37180] [ 8] /usr/lib64/libstdc++.so.6(_Znam+0x9)
[0x2aaab40891d9]
[node125:37180] [ 9]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData10ReadPointsEP12XdmfGeometryPiS2_+0x3a6)
[0x2aaab716310c]
[node125:37180] [10]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData21RequestStructuredGridEP8XdmfGrid+0xfb)
[0x2aaab716292f]
[node125:37180] [11]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData15ReadUniformDataEP8XdmfGrid+0x14c)
[0x2aaab71614ce]
[node125:37180] [12]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEP8XdmfGrid+0xd1)
[0x2aaab7160c91]
[node125:37180] [13]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEv+0x4f)
[0x2aaab7160a0f]
[node125:37180] [14]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader11RequestDataEP14vtkInformationPP20vtkInformationVectorS3_+0x31c)
[0x2aaab716daf0]
[node125:37180] [15]
/c1/apps/paraview/4.1.0/lib/libvtkIOLegacy-pv4.1.so.1(_ZN13vtkDataReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x62)
[0x2aaab0b3b9be]
[node125:37180] [16]
/c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
[0x2aaab716c702]
[node125:37180] [17]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN12vtkExecutive13CallAlgorithmEP14vtkInformationiPP20vtkInformationVectorS3_+0x8d)
[0x2aaab10c63d1]
[node125:37180] [18]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x76)
[0x2aaab10c0b1c]
[node125:37180] [19]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x468)
[0x2aaab10b9950]
[node125:37180] [20]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x522)
[0x2aaab10c0248]
[node125:37180] [21]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
[0x2aaab10e5ace]
[node125:37180] [22]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline15ForwardUpstreamEP14vtkInformation+0x2d6)
[0x2aaab10bbb6e]
[node125:37180] [23]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x48a)
[0x2aaab10c01b0]
[node125:37180] [24]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c)
[0x2aaab10e5ace]
[node125:37180] [25]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline10UpdateDataEi+0x2fb)
[0x2aaab10c0965]
[node125:37180] [26]
/c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline6UpdateEi+0xf8)
[0x2aaab10e5d3e]
[node125:37180] [27]
/c1/apps/paraview/4.1.0/lib/libvtkPVServerImplementationCore-pv4.1.so.1(_ZN16vtkSISourceProxy14UpdatePipelineEidb+0x196)
[0x2aaaac77ef72]
[node125:37180] [28]
/c1/apps/paraview/4.1.0/lib/libvtkPVServerManagerApplication-pv4.1.so.1(_Z23vtkSISourceProxyCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_Pv+0x6ee)
[0x2aaaab7f27bd]
[node125:37180] [29]
/c1/apps/paraview/4.1.0/lib/libvtkClientServer-pv4.1.so.1(_ZN26vtkClientServerInterpreter19CallCommandFunctionEPKcP13vtkObjectBaseS1_RK21vtkClientServerStreamRS4_+0x343)
[0x2aaaafb57aa5]
[node125:37180] *** End of error message ***
I am afraid there is an issue with memory leakage or improper
memory allocation and deallocation. At this point I do not
know how to proceed to debug it. Is there an obvious problem
in in the script and the way datasets are deallocated at the
end of each iteration?
Regards,
Nikos
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
