I am trying to run a script on a cluster with Paraview in parallel. The cluster architecture is 16cores and 64GB per node. The script reads in two XMF files, one contains a grid of 2002x100x1000 points along with one variable. There is a loop to perform two iterations:
try: paraview.simple except: from paraview.simple import * paraview.simple._DisableFirstRenderCameraReset() RenderView1 = GetRenderView() RenderView1.Background = [1.0, 1.0, 1.0] view = GetActiveView() view.ViewSize = [ 800, 500 ] RenderView1.CenterAxesVisibility = 0 #RenderView1.OrientationAxesVisibility = 0 #Define variables wdir = '/lustre/groups/balarasgrp/nikosb/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm' #wdir = '/Users/nikosb/colone/Research/Simulations/golfball/stationary/S120/Re200000/gridC1/run_wm' #wdir = '/Users/nikosb/Research/Simulations/SPHERE/Re100/gridC1/run' Qval = 0.1 cfmin = -0.001 cfmax = 0.005 index = [None]*9999 m = 0 for i in range(0, 9): for j in range(0, 9): for k in range(0, 9): for l in range(0, 9): index[m] = chr(i+48)+chr(j+48)+chr(k+48)+chr(l+48) m = m+1 for i in range(1,3): cf_VP_xmf = XDMFReader( FileName=wdir+'/VPfield/XDMF/cf_VP'+index[i]+'.xmf' ) cf_VP_xmf.Sets = [] cf_VP_xmf.Grids = ['STL Grid'] cf_VP_xmf.PointArrays = ['Cf'] RenderView1 = GetRenderView() RenderView1.CenterOfRotation = [6.2286853790283203e-06, 2.5033950805664062e-06, 7.1227550506591797e-06] DataRepresentation1 = Show() DataRepresentation1.EdgeColor = [0.0, 0.0, 0.50000762951094835] DataRepresentation1.SelectionPointFieldDataArrayName = 'Cf' DataRepresentation1.ColorArrayName = ('POINT_DATA', 'Cf') DataRepresentation1.ScalarOpacityUnitDistance = 0.033928799990483226 DataRepresentation1.ScaleFactor = 0.099998557567596444 a1_Cf_PVLookupTable = GetLookupTableForArray( "Cf", 1, RGBPoints=[-0.0099065126851201057, 0.23000000000000001, 0.29899999999999999, 0.754, 0.0097341262735426426, 0.86499999999999999, 0.86499999999999999, 0.86499999999999999, 0.029374765232205391, 0.70599999999999996, 0.016, 0.14999999999999999], VectorMode='Magnitude', NanColor=[0.25, 0.0, 0.0], ColorSpace='Diverging', ScalarRangeInitialized=1.0 ) a1_Cf_PiecewiseFunction = CreatePiecewiseFunction( Points=[-0.0099065126851201057, 0.0, 0.5, 0.0, 0.029374765232205391, 1.0, 0.5, 0.0] ) DataRepresentation1.ScalarOpacityFunction = a1_Cf_PiecewiseFunction DataRepresentation1.LookupTable = a1_Cf_PVLookupTable a1_Cf_PVLookupTable.ScalarOpacityFunction = a1_Cf_PiecewiseFunction a1_Cf_PVLookupTable.RGBPoints = [cfmin, 0.23000000000000001, 0.29899999999999999, 0.754, -0.002, 0.86499999999999999, 0.86499999999999999, 0.86499999999999999, cfmax, 0.70599999999999996, 0.016, 0.14999999999999999] a1_Cf_PVLookupTable.ScalarOpacityFunction = a1_Cf_PiecewiseFunction a1_Cf_PVLookupTable.LockScalarRange = 1 RenderView1.CameraPosition = [6.2286853790283203e-06, 2.5033950805664062e-06, 3.3460035306026636] RenderView1.CameraFocalPoint = [6.2286853790283203e-06, 2.5033950805664062e-06, 7.1227550506591797e-06] RenderView1.CameraClippingRange = [2.3175641399614015, 4.6486514901655163] RenderView1.CameraParallelScale = 0.86600759519558368 Contour1 = Contour( PointMergeMethod="Uniform Binning" ) Contour1.PointMergeMethod = "Uniform Binning" Contour1.ContourBy = ['POINTS', 'Cf'] Contour1.Isosurfaces = [0.0097341262735426426] DataRepresentation2 = Show() DataRepresentation2.ColorArrayName = ('POINT_DATA', '') DataRepresentation2.ScaleFactor = 0.099971860647201538 DataRepresentation2.SelectionPointFieldDataArrayName = 'Cf' DataRepresentation2.EdgeColor = [0.0, 0.0, 0.50000762951094835] DataRepresentation2.Representation = 'Wireframe' DataRepresentation2.AmbientColor = [0.0, 0.0, 0.0] DataRepresentation1.Visibility = 0 RenderView1.CameraClippingRange = [2.7470738944403328, 4.2397684828477349] DataRepresentation1.Visibility = 1 RenderView1.CameraClippingRange = [2.3175641399614015, 4.6486514901655163] Q_VP_xmf = XDMFReader( FileName=wdir+'/VPfield/XDMF/Q_VP'+index[i]+'.xmf' ) Contour1.Isosurfaces = [0.0] Q_VP_xmf.Sets = [] Q_VP_xmf.Grids = ['Eulerian Grid'] Q_VP_xmf.PointArrays = ['Q'] DataRepresentation3 = Show() DataRepresentation3.EdgeColor = [0.0, 0.0, 0.50000762951094835] DataRepresentation3.SelectionPointFieldDataArrayName = 'Q' DataRepresentation3.ScalarOpacityUnitDistance = 0.0045778956118952439 DataRepresentation3.Representation = 'Outline' DataRepresentation3.ScaleFactor = 0.25585924386978148 RenderView1.CameraClippingRange = [0.30866115510384229, 6.1311800002592065] Contour2 = Contour( PointMergeMethod="Uniform Binning" ) Contour2.PointMergeMethod = "Uniform Binning" Contour2.ContourBy = ['POINTS', 'Q'] Contour2.Isosurfaces = [-348374.28125] DataRepresentation4 = Show() DataRepresentation4.ScaleFactor = 0.1348903000354767 DataRepresentation4.SelectionPointFieldDataArrayName = 'Normals' DataRepresentation4.EdgeColor = [0.0, 0.0, 0.50000762951094835] Contour2.Isosurfaces = [0.5] DataRepresentation3.Visibility = 0 RenderView1.CameraClippingRange = [0.46635874330917981, 5.5760844897764175] RenderView1.CameraViewUp = [-1.0, 0.0, 0.0] RenderView1.CameraPosition = [-5.0, 0.0, 5.0] RenderView1.CameraClippingRange = [3.0180305778980254, 7.5069525367021557] RenderView1.CameraFocalPoint = [0.0, 0.0, 5.0] RenderView1.CameraParallelScale = 1.565457167140391 RenderView1.CenterOfRotation = [0.0, 0.0, 0.0] RenderView1.CameraFocalPoint = [0.0, 0.0, 0.5] RenderView1.CameraClippingRange = [0.86197012554183883, 4.2263653610655147] RenderView1.CameraPosition = [0.0, -2.5, 0.0] WriteImage(wdir+'/POSTPRO/IMAGES/Q_cf_VP'+index[i]+'.jpg') Delete(DataRepresentation4) Delete(Contour2) Delete(DataRepresentation3) Delete(Q_VP_xmf) Delete(DataRepresentation2) Delete(Contour1) Delete(DataRepresentation1) Delete(cf_VP_xmf) The script was created by running an interactive Paraview session and recording the commands. I then modified it slightly to add the loop and some variables. I have been able to run the script successfully on 1,2,16 and 32 cores successfully and it loops twice. However when I try to run on 64 cores I get the following error message: [node116:58372] 63 more processes have sent help message help-mpi-common-cuda.txt / dlopen failed [node116:58372] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc [node125:37180] *** Process received signal *** [node125:37180] Signal: Aborted (6) [node125:37180] Signal code: (-6) [node125:37180] [ 0] /lib64/libpthread.so.0(+0xf710) [0x2aaab28f3710] [node125:37180] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2aaab479f625] [node125:37180] [ 2] /lib64/libc.so.6(abort+0x175) [0x2aaab47a0e05] [node125:37180] [ 3] /usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d) [0x2aaab408aa7d] [node125:37180] [ 4] /usr/lib64/libstdc++.so.6(+0xbcbd6) [0x2aaab4088bd6] [node125:37180] [ 5] /usr/lib64/libstdc++.so.6(+0xbcc03) [0x2aaab4088c03] [node125:37180] [ 6] /usr/lib64/libstdc++.so.6(+0xbcd22) [0x2aaab4088d22] [node125:37180] [ 7] /usr/lib64/libstdc++.so.6(_Znwm+0x7d) [0x2aaab408911d] [node125:37180] [ 8] /usr/lib64/libstdc++.so.6(_Znam+0x9) [0x2aaab40891d9] [node125:37180] [ 9] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData10ReadPointsEP12XdmfGeometryPiS2_+0x3a6) [0x2aaab716310c] [node125:37180] [10] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData21RequestStructuredGridEP8XdmfGrid+0xfb) [0x2aaab716292f] [node125:37180] [11] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData15ReadUniformDataEP8XdmfGrid+0x14c) [0x2aaab71614ce] [node125:37180] [12] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEP8XdmfGrid+0xd1) [0x2aaab7160c91] [node125:37180] [13] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN16vtkXdmfHeavyData8ReadDataEv+0x4f) [0x2aaab7160a0f] [node125:37180] [14] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader11RequestDataEP14vtkInformationPP20vtkInformationVectorS3_+0x31c) [0x2aaab716daf0] [node125:37180] [15] /c1/apps/paraview/4.1.0/lib/libvtkIOLegacy-pv4.1.so.1(_ZN13vtkDataReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x62) [0x2aaab0b3b9be] [node125:37180] [16] /c1/apps/paraview/4.1.0/lib/libvtkIOXdmf2-pv4.1.so.1(_ZN13vtkXdmfReader14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x76) [0x2aaab716c702] [node125:37180] [17] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN12vtkExecutive13CallAlgorithmEP14vtkInformationiPP20vtkInformationVectorS3_+0x8d) [0x2aaab10c63d1] [node125:37180] [18] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x76) [0x2aaab10c0b1c] [node125:37180] [19] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline11ExecuteDataEP14vtkInformationPP20vtkInformationVectorS3_+0x468) [0x2aaab10b9950] [node125:37180] [20] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x522) [0x2aaab10c0248] [node125:37180] [21] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c) [0x2aaab10e5ace] [node125:37180] [22] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN24vtkCompositeDataPipeline15ForwardUpstreamEP14vtkInformation+0x2d6) [0x2aaab10bbb6e] [node125:37180] [23] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0x48a) [0x2aaab10c01b0] [node125:37180] [24] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline14ProcessRequestEP14vtkInformationPP20vtkInformationVectorS3_+0xa6c) [0x2aaab10e5ace] [node125:37180] [25] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN23vtkDemandDrivenPipeline10UpdateDataEi+0x2fb) [0x2aaab10c0965] [node125:37180] [26] /c1/apps/paraview/4.1.0/lib/libvtkCommonExecutionModel-pv4.1.so.1(_ZN32vtkStreamingDemandDrivenPipeline6UpdateEi+0xf8) [0x2aaab10e5d3e] [node125:37180] [27] /c1/apps/paraview/4.1.0/lib/libvtkPVServerImplementationCore-pv4.1.so.1(_ZN16vtkSISourceProxy14UpdatePipelineEidb+0x196) [0x2aaaac77ef72] [node125:37180] [28] /c1/apps/paraview/4.1.0/lib/libvtkPVServerManagerApplication-pv4.1.so.1(_Z23vtkSISourceProxyCommandP26vtkClientServerInterpreterP13vtkObjectBasePKcRK21vtkClientServerStreamRS5_Pv+0x6ee) [0x2aaaab7f27bd] [node125:37180] [29] /c1/apps/paraview/4.1.0/lib/libvtkClientServer-pv4.1.so.1(_ZN26vtkClientServerInterpreter19CallCommandFunctionEPKcP13vtkObjectBaseS1_RK21vtkClientServerStreamRS4_+0x343) [0x2aaaafb57aa5] [node125:37180] *** End of error message *** I am afraid there is an issue with memory leakage or improper memory allocation and deallocation. At this point I do not know how to proceed to debug it. Is there an obvious problem in in the script and the way datasets are deallocated at the end of each iteration? Regards, Nikos
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview