Hi All,
I want to share a performance issue related to PV start up time, which
in general on our cray's is significant usability issue, especially on
Hopper.
Earlier this week NERSC did some upgrades on one of our Crays which
introduced some issues that prevent ParaView from starting. In the end
we found that the issue is a system issue (file system or python
problem) and not ParaView, but this exposed a performance issue in
ParaView. ParaView would connect then hang when creating the render
view. The debugger shows ParaView stuck in a python import call(stack is
below) from vtkMatplotlibMathTextUtilities::CheckMPLAvailability. This
function initializes the python interpreter and tries to "import
matplotlib".
Importing a python module (that's not built into the core of python) is
an extremely expensive and unscalable operation in HPC setting. Software
like python and ParaView, at least at NERSC, lives on slow remote file
systems and in parallel the import hammers these file systems from each
process. Python is essential for batch processing, but when using the
interactive ParaView we should delay/avoid python imports as much as
possible , especially during start up. This is a scalability issue in
that most user aren't going to use the tex feature, but all users
startup time is impacted, especially as job size grows.
I think in the case of ParaView's matplotlib tex features the run time
check for matplotlib could probably be deferred until someone actually
tries to use tex. What do you think? This would help keep startup time fast.
Burlen
proc_set{0}: #0 0x00002aaaae6b8fd0 in __open_nocancel
proc_set{0}: #1 0x00002aaaae65e483 in __GI__IO_file_open
proc_set{0}: #2 0x00002aaaae65e5fb in _IO_new_file_fopen
proc_set{0}: #3 0x00002aaaae6522f4 in __fopen_internal
proc_set{0}: #4 0x00002aaaaddb833c in find_module at Python/import.c:1556
proc_set{0}: #5 0x00002aaaaddba470 in import_submodule at
Python/import.c:2689
proc_set{0}: #6 0x00002aaaaddbb0e3 in load_next at Python/import.c:2515
proc_set{0}: #7 0x00002aaaaddbb0e3 in import_module_level at
Python/import.c:2224
proc_set{0}: #8 0x00002aaaaddbb0e3 in PyImport_ImportModuleLevel at
Python/import.c:2288
proc_set{0}: #9 0x00002aaaadd998cf in builtin___import__ at
Python/bltinmodule.c:49
proc_set{0}: #10 0x00002aaaadceae93 in PyObject_Call at
Objects/abstract.c:2529
proc_set{0}: #11 0x00002aaaadd9b327 in PyEval_CallObjectWithKeywords at
Python/ceval.c:3890
proc_set{0}: #12 0x00002aaaadd9e296 in PyEval_EvalFrameEx at
Python/ceval.c:2333
proc_set{0}: #13 0x00002aaaadda198d in PyEval_EvalCodeEx at
Python/ceval.c:3253
proc_set{0}: #14 0x00002aaaadda1ac2 in PyEval_EvalCode at Python/ceval.c:667
proc_set{0}: #15 0x00002aaaaddb924c in PyImport_ExecCodeModuleEx at
Python/import.c:709
proc_set{0}: #16 0x00002aaaaddb95ae in load_source_module at
Python/import.c:1099
proc_set{0}: #17 0x00002aaaaddba4a9 in import_submodule at
Python/import.c:2700
proc_set{0}: #18 0x00002aaaaddbb0e3 in load_next at Python/import.c:2515
proc_set{0}: #19 0x00002aaaaddbb0e3 in import_module_level at
Python/import.c:2224
proc_set{0}: #20 0x00002aaaaddbb0e3 in PyImport_ImportModuleLevel at
Python/import.c:2288
proc_set{0}: #21 0x00002aaaadd998cf in builtin___import__ at
Python/bltinmodule.c:49
proc_set{0}: #22 0x00002aaaadceae93 in PyObject_Call at
Objects/abstract.c:2529
proc_set{0}: #23 0x00002aaaadd9b327 in PyEval_CallObjectWithKeywords at
Python/ceval.c:3890
proc_set{0}: #24 0x00002aaaadd9e296 in PyEval_EvalFrameEx at
Python/ceval.c:2333
proc_set{0}: #25 0x00002aaaadda198d in PyEval_EvalCodeEx at
Python/ceval.c:3253
proc_set{0}: #26 0x00002aaaadda1ac2 in PyEval_EvalCode at Python/ceval.c:667
proc_set{0}: #27 0x00002aaaaddb924c in PyImport_ExecCodeModuleEx at
Python/import.c:709
proc_set{0}: #28 0x00002aaaaddb95ae in load_source_module at
Python/import.c:1099
proc_set{0}: #29 0x00002aaaaddba4a9 in import_submodule at
Python/import.c:2700
proc_set{0}: #30 0x00002aaaaddbb567 in load_next at Python/import.c:2519
proc_set{0}: #31 0x00002aaaaddbb567 in import_module_level at
Python/import.c:2224
proc_set{0}: #32 0x00002aaaaddbb567 in PyImport_ImportModuleLevel at
Python/import.c:2288
proc_set{0}: #33 0x00002aaaadd998cf in builtin___import__ at
Python/bltinmodule.c:49
proc_set{0}: #34 0x00002aaaadceae93 in PyObject_Call at
Objects/abstract.c:2529
proc_set{0}: #35 0x00002aaaadd9b327 in PyEval_CallObjectWithKeywords at
Python/ceval.c:3890
proc_set{0}: #36 0x00002aaaadd9e296 in PyEval_EvalFrameEx at
Python/ceval.c:2333
proc_set{0}: #37 0x00002aaaadda198d in PyEval_EvalCodeEx at
Python/ceval.c:3253
proc_set{0}: #38 0x00002aaaadda1ac2 in PyEval_EvalCode at Python/ceval.c:667
proc_set{0}: #39 0x00002aaaaddb924c in PyImport_ExecCodeModuleEx at
Python/import.c:709
proc_set{0}: #40 0x00002aaaaddb95ae in load_source_module at
Python/import.c:1099
proc_set{0}: #41 0x00002aaaaddba9e9 in load_package at Python/import.c:1166
proc_set{0}: #42 0x00002aaaaddba4a9 in import_submodule at
Python/import.c:2700
proc_set{0}: #43 0x00002aaaaddbb0e3 in load_next at Python/import.c:2515
proc_set{0}: #44 0x00002aaaaddbb0e3 in import_module_level at
Python/import.c:2224
proc_set{0}: #45 0x00002aaaaddbb0e3 in PyImport_ImportModuleLevel at
Python/import.c:2288
proc_set{0}: #46 0x00002aaaadd998cf in builtin___import__ at
Python/bltinmodule.c:49
proc_set{0}: #47 0x00002aaaadceae93 in PyObject_Call at
Objects/abstract.c:2529
proc_set{0}: #48 0x00002aaaadceafeb in call_function_tail at
Objects/abstract.c:2561
proc_set{0}: #49 0x00002aaaadceafeb in PyObject_CallFunction at
Objects/abstract.c:2585
proc_set{0}: #50 0x00002aaaaddbbeb2 in PyImport_Import at
Python/import.c:2886
proc_set{0}: #51 0x00002aaaaddbc02a in PyImport_ImportModule at
Python/import.c:2129
proc_set{0}: #52 0x0000000004766407 in
vtkMatplotlibMathTextUtilities::CheckMPLAvailability at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/Matplotlib/vtkMatplotlibMathTextUtilities.cxx:75
proc_set{0}: #53 0x000000000476681a in
vtkMatplotlibMathTextUtilities::New at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/Matplotlib/vtkMatplotlibMathTextUtilities.cxx:119
proc_set{0}: #54 0x000000000476abed in
vtkObjectFactoryCreatevtkMatplotlibMathTextUtilities at
/global/common/edison/graphics/ParaView/builds/PV-4.2.0/VTK/Rendering/Matplotlib/vtkRenderingMatplotlibObjectFactory.cxx:27
proc_set{0}: #55 0x0000000007336149 in vtkObjectFactory::CreateObject at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Common/Core/vtkObjectFactory.cxx:488
proc_set{0}: #56 0x0000000007334a8a in vtkObjectFactory::CreateInstance
at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Common/Core/vtkObjectFactory.cxx:62
proc_set{0}: #57 0x0000000005a85a61 in vtkMathTextUtilities::GetInstance
at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/FreeType/vtkMathTextUtilities.cxx:58
proc_set{0}: #58 0x0000000005a85566 in
vtkMathTextFreeTypeTextRenderer::vtkMathTextFreeTypeTextRenderer at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/FreeType/vtkMathTextFreeTypeTextRenderer.cxx:489
proc_set{0}: #59 0x0000000005a81b35 in
vtkMathTextFreeTypeTextRenderer::New at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/FreeType/vtkMathTextFreeTypeTextRenderer.cxx:26
proc_set{0}: #60 0x0000000005a8dc29 in
vtkObjectFactoryCreatevtkMathTextFreeTypeTextRenderer at
/global/common/edison/graphics/ParaView/builds/PV-4.2.0/VTK/Rendering/FreeType/vtkRenderingFreeTypeObjectFactory.cxx:27
proc_set{0}: #61 0x0000000007336149 in vtkObjectFactory::CreateObject at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Common/Core/vtkObjectFactory.cxx:488
proc_set{0}: #62 0x0000000007334a8a in vtkObjectFactory::CreateInstance
at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Common/Core/vtkObjectFactory.cxx:62
proc_set{0}: #63 0x000000000601cec9 in vtkTextRenderer::GetInstance at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/Core/vtkTextRenderer.cxx:79
proc_set{0}: #64 0x0000000005a87f11 in vtkTextActor::vtkTextActor at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/FreeType/vtkTextActor.cxx:105
proc_set{0}: #65 0x0000000005a87a34 in vtkTextActor::New at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Rendering/FreeType/vtkTextActor.cxx:35
proc_set{0}: #66 0x000000000501535d in
vtkTextRepresentation::vtkTextRepresentation at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Interaction/Widgets/vtkTextRepresentation.cxx:62
proc_set{0}: #67 0x00000000050152e0 in vtkTextRepresentation::New at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Interaction/Widgets/vtkTextRepresentation.cxx:54
proc_set{0}: #68 0x000000000386c735 in
vtkNew<vtkTextRepresentation>::vtkNew at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Common/Core/vtkNew.h:66
proc_set{0}: #69 0x0000000003865225 in vtkPVRenderView::vtkPVRenderView
at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:154
proc_set{0}: #70 0x0000000003864eac in vtkPVRenderView::New at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ClientServerCore/Rendering/vtkPVRenderView.cxx:140
proc_set{0}: #71 0x00000000032f9975 in
vtkPVRenderViewClientServerNewCommand at
/global/common/edison/graphics/ParaView/builds/PV-4.2.0/Wrapping/ClientServer/vtkPVRenderViewClientServer.cxx:13
proc_set{0}: #72 0x000000000474760f in
vtkClientServerInterpreter::NewInstance at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ClientServerStream/vtkClientServerInterpreter.cxx:1068
proc_set{0}: #73 0x00000000036a1b80 in vtkSIProxy::CreateVTKObjects at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkSIProxy.cxx:303
proc_set{0}: #74 0x00000000036a04cd in vtkSIProxy::Push at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkSIProxy.cxx:112
proc_set{0}: #75 0x00000000036848bf in
vtkPVSessionCore::PushStateInternal at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:486
proc_set{0}: #76 0x0000000003684b43 in vtkPVSessionCore::PushState at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionCore.cxx:529
proc_set{0}: #77 0x0000000003681f69 in vtkPVSessionBase::PushState at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionBase.cxx:142
proc_set{0}: #78 0x000000000368edf2 in
vtkPVSessionServer::OnClientServerMessageRMI at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:495
proc_set{0}: #79 0x000000000368d29c in at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ServerImplementation/Core/vtkPVSessionServer.cxx:56
proc_set{0}: #80 0x0000000004b0a589 in
vtkMultiProcessController::ProcessRMI at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:774
proc_set{0}: #81 0x0000000004b0a145 in
vtkMultiProcessController::ProcessRMIs at
/global/common/edison/graphics/ParaView/builds/ParaView/VTK/Parallel/Core/vtkMultiProcessController.cxx:720
proc_set{0}: #82 0x00000000039c1f95 in
vtkTCPNetworkAccessManager::ProcessEventsInternal at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:268
proc_set{0}: #83 0x00000000039c1b32 in
vtkTCPNetworkAccessManager::ProcessEvents at
/global/common/edison/graphics/ParaView/builds/ParaView/ParaViewCore/ClientServerCore/Core/vtkTCPNetworkAccessManager.cxx:165
proc_set{0}: #84 0x0000000001e8c2e9 in RealMain at
/global/common/edison/graphics/ParaView/builds/ParaView/CommandLineExecutables/pvserver_common.h:91
proc_set{0}: #85 0x0000000001e8c3b3 in main at
/global/common/edison/graphics/ParaView/builds/ParaView/CommandLineExecutables/pvserver.cxx:27
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
