Hi Andy,
Thanks for your help. You can find the Python script as attachment. BTW,
the "coprocessorinitializewithpython" method is called by subset of
processors that are supposed to do co-processing. I think that the usage
of vtkCPPythonScriptPipeline is correct in this case. It is weird but it
works under Mac OS and allinputsgridwriter.py script works on both Linux
and Mac OS.
extern "C" void coprocessorinitializewithpython_(int *fcomm, const char*
pythonScriptName, const char strarr[][255], int *size) {
if (pythonScriptName != NULL) {
if (!g_coprocessor) {
g_coprocessor = vtkCPProcessor::New();
MPI_Comm handle = MPI_Comm_f2c(*fcomm);
vtkMPICommunicatorOpaqueComm *Comm = new
vtkMPICommunicatorOpaqueComm(&handle);
g_coprocessor->Initialize(*Comm);
vtkSmartPointer<vtkCPPythonScriptPipeline> pipeline =
vtkSmartPointer<vtkCPPythonScriptPipeline>::New();
pipeline->Initialize(pythonScriptName);
g_coprocessor->AddPipeline(pipeline);
//pipeline->FastDelete();
}
if (!g_coprocessorData) {
g_coprocessorData = vtkCPDataDescription::New();
// must be input port for all model components and for all dimensions
for (int i = 0; i < *size; i++) {
g_coprocessorData->AddInput(strarr[i]);
std::cout << "adding input port [" << i << "] = " << strarr[i] <<
std::endl;
}
}
}
}
Regards,
--ufuk
> Can you share your Python script? Another thought is that your Python
> script was added to each process instead of the subset of processes that
> are supposed to do the calculation on it. For example, the Python script
> that is supposed to generate the image should only be added through a
> vtkCPPythonScriptPipeline on those 8 processes.
>
> On Thu, Aug 4, 2016 at 2:48 AM, Ufuk Utku Turuncoglu (BE) <
> u.utku.turunco...@be.itu.edu.tr> wrote:
>
>> Hi,
>>
>> After getting help from the list, i finished the initial implementation
>> of
>> the code but in this case i have a strange experience with Catalyst. The
>> prototype code is working with allinputsgridwriter.py script and could
>> write multi-piece dataset in VTK format without any problem. In this
>> case,
>> the code also handles four different input ports to get data in
>> different
>> grid structure and dimensions (2d/3d).
>>
>> The main problem is that if i try to use the same code to output a png
>> file after creating iso-surface from single 3d field (141x115x14 =
>> 227K),
>> it is hanging. In this case, if i check the utilization of the
>> processors
>> (on Linux, Centos 7.1,
>>
>> 12064 turuncu 20 0 1232644 216400 77388 R 100.0 0.7 10:44.17
>> main.x
>> 12068 turuncu 20 0 1672156 483712 70420 R 100.0 1.5 10:44.17
>> main.x
>> 12069 turuncu 20 0 1660620 266716 70500 R 100.0 0.8 10:44.26
>> main.x
>> 12070 turuncu 20 0 1660412 267204 71204 R 100.0 0.8 10:44.22
>> main.x
>> 12071 turuncu 20 0 1659988 266644 71360 R 100.0 0.8 10:44.18
>> main.x
>> 12065 turuncu 20 0 1220328 202224 77620 R 99.7 0.6 10:44.08
>> main.x
>> 12066 turuncu 20 0 1220236 204696 77444 R 99.7 0.6 10:44.16
>> main.x
>> 12067 turuncu 20 0 1219644 199116 77152 R 99.7 0.6 10:44.18
>> main.x
>> 12078 turuncu 20 0 1704272 286924 102940 S 10.6 0.9 1:12.91 main.x
>> 12074 turuncu 20 0 1704488 287668 103456 S 10.0 0.9 1:08.50 main.x
>> 12072 turuncu 20 0 1704444 287488 103316 S 9.6 0.9 1:09.09 main.x
>> 12076 turuncu 20 0 1704648 287268 102848 S 9.6 0.9 1:10.16 main.x
>> 12073 turuncu 20 0 1704132 284128 103384 S 9.3 0.9 1:05.27 main.x
>> 12077 turuncu 20 0 1706236 286228 103380 S 9.3 0.9 1:05.49 main.x
>> 12079 turuncu 20 0 1699944 278800 102864 S 9.3 0.9 1:05.87 main.x
>> 12075 turuncu 20 0 1704356 284408 103436 S 8.6 0.9 1:07.03 main.x
>>
>> they seems normal because the co-processing component only works on a
>> subset of the resource (8 processor, has utilization around 99 percent).
>> The GPU utilization (from nvidia-smi command) is
>>
>> +------------------------------------------------------+
>> | NVIDIA-SMI 352.79 Driver Version: 352.79 |
>> |-------------------------------+----------------------+----
>> ------------------+
>> | GPU Name Persistence-M| Bus-Id Disp.A | Volatile
>> Uncorr.
>> ECC |
>> | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util
>> Compute
>> M. |
>> |===============================+======================+====
>> ==================|
>> | 0 Quadro K5200 Off | 0000:42:00.0 On |
>> Off |
>> | 26% 42C P8 14W / 150W | 227MiB / 8191MiB | 0% Default
>> |
>> +-------------------------------+----------------------+----
>> ------------------+
>>
>> +-----------------------------------------------------------
>> ------------------+
>> | Processes: GPU Memory |
>> | GPU PID Type Process name Usage |
>> |===========================================================
>> ==================|
>> | 0 1937 G /usr/bin/Xorg
>> 81MiB |
>> | 0 3817 G /usr/bin/gnome-shell
>> 110MiB |
>> | 0 9551 G paraview
>> 16MiB |
>> +-----------------------------------------------------------
>> ------------------+
>>
>> So, the GPU is not overloaded in this case. I tested the code with two
>> different version of ParaView (5.0.0 and 5.1.0). The results are same
>> for
>> both cases even if i create the co-processing Python scripts with same
>> version of the ParaView that is used to compile the code. I also tried
>> to
>> use 2d field (141x115) but the result is also same and the code is still
>> hanging. The different machine (MacOS+ParaView 5.0.0) works without
>> problem. There might be a issue of Linux or installation but i am not
>> sure
>> and it was working before. Is there any flag or tool that allows to
>> analyze
>> Paraview deeply to find the source of the problem.
>>
>> Regards,
>>
>> --ufuk
>>
>>
>>
>>
>>
>> _______________________________________________
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensou
>> rce/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at:
>> http://paraview.org/Wiki/ParaView
>>
>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/paraview
>>
>
from paraview.simple import *
from paraview import coprocessing
#--------------------------------------------------------------
# Code generated from cpstate.py to create the CoProcessor.
# ParaView 5.0.0 64 bits
# ----------------------- CoProcessor definition -----------------------
def CreateCoProcessor():
def _CreatePipeline(coprocessor, datadescription):
class Pipeline:
# state file generated using paraview version 5.0.0
# ----------------------------------------------------------------
# setup views used in the visualization
# ----------------------------------------------------------------
#### disable automatic camera reset on 'Show'
paraview.simple._DisableFirstRenderCameraReset()
# Create a new 'Render View'
renderView1 = CreateView('RenderView')
renderView1.ViewSize = [852, 566]
renderView1.AxesGrid = 'GridAxes3DActor'
renderView1.CenterOfRotation = [20.0, 42.64287233352661, 10.019999999552965]
renderView1.StereoType = 0
renderView1.CameraPosition = [-63.809095722758066, -72.20808641074147, 146.21326787941686]
renderView1.CameraFocalPoint = [56.629437838324165, 92.8394055106853, -49.504360653452316]
renderView1.CameraViewUp = [0.1957804870647042, 0.686981500602347, 0.6998045575122064]
renderView1.CameraParallelScale = 73.22864217229005
renderView1.Background = [0.32, 0.34, 0.43]
# register the view with coprocessor
# and provide it with information such as the filename to use,
# how frequently to write the images, etc.
coprocessor.RegisterView(renderView1,
filename='image_%t.png', freq=1, fittoscreen=0, magnification=1, width=852, height=566, cinema={})
renderView1.ViewTime = datadescription.GetTime()
# ----------------------------------------------------------------
# setup the data processing pipelines
# ----------------------------------------------------------------
# create a new 'XML MultiBlock Data Reader'
# create a producer from a simulation input
atm_input3d_ = coprocessor.CreateProducer(datadescription, 'atm_input3d')
# create a new 'Transform'
transform1 = Transform(Input=atm_input3d_)
transform1.Transform = 'Transform'
# init the 'Transform' selected for 'Transform'
transform1.Transform.Scale = [1.0, 1.0, 0.002]
# create a new 'Contour'
contour1 = Contour(Input=transform1)
contour1.ContourBy = ['POINTS', 'rh']
contour1.Isosurfaces = [90.0]
contour1.PointMergeMethod = 'Uniform Binning'
# ----------------------------------------------------------------
# setup the visualization in view 'renderView1'
# ----------------------------------------------------------------
# show data from contour1
contour1Display = Show(contour1, renderView1)
# trace defaults for the display properties.
contour1Display.ColorArrayName = [None, '']
contour1Display.GlyphType = 'Arrow'
contour1Display.SelectInputVectors = ['POINTS', 'Normals']
contour1Display.WriteLog = ''
return Pipeline()
class CoProcessor(coprocessing.CoProcessor):
def CreatePipeline(self, datadescription):
self.Pipeline = _CreatePipeline(self, datadescription)
coprocessor = CoProcessor()
# these are the frequencies at which the coprocessor updates.
freqs = {'atm_input3d': [1, 1, 1]}
coprocessor.SetUpdateFrequencies(freqs)
return coprocessor
#--------------------------------------------------------------
# Global variables that will hold the pipeline for each timestep
# Creating the CoProcessor object, doesn't actually create the ParaView pipeline.
# It will be automatically setup when coprocessor.UpdateProducers() is called the
# first time.
coprocessor = CreateCoProcessor()
#--------------------------------------------------------------
# Enable Live-Visualizaton with ParaView
coprocessor.EnableLiveVisualization(False, 1)
# ---------------------- Data Selection method ----------------------
def RequestDataDescription(datadescription):
"Callback to populate the request for current timestep"
global coprocessor
if datadescription.GetForceOutput() == True:
# We are just going to request all fields and meshes from the simulation
# code/adaptor.
for i in range(datadescription.GetNumberOfInputDescriptions()):
datadescription.GetInputDescription(i).AllFieldsOn()
datadescription.GetInputDescription(i).GenerateMeshOn()
return
# setup requests for all inputs based on the requirements of the
# pipeline.
coprocessor.LoadRequestedData(datadescription)
# ------------------------ Processing method ------------------------
def DoCoProcessing(datadescription):
"Callback to do co-processing for current timestep"
global coprocessor
# Update the coprocessor by providing it the newly generated simulation data.
# If the pipeline hasn't been setup yet, this will setup the pipeline.
coprocessor.UpdateProducers(datadescription)
# Write output data, if appropriate.
coprocessor.WriteData(datadescription);
# Write image capture (Last arg: rescale lookup table), if appropriate.
coprocessor.WriteImages(datadescription, rescale_lookuptable=False)
# Live Visualization, if enabled.
coprocessor.DoLiveVisualization(datadescription, "localhost", 22222)_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
