Re: [Paraview] LiveInsituLink proxy in pvpython client

[Kolja Petersen]

What is meant by support services? Step-by-step guidance to solve the
problem or a complete solution?

How much would it cost if you developed a python client script that
connects to a pvserver, waits for an insitu connection and writes rendered
images of all data available from catalyst to disk for all time steps?
200$, 2000$, 2$?
Thank you
Kolja

On Fri, Oct 27, 2017 at 8:38 PM, Andy Bauer  wrote:

> Unfortunately there's no comprehensive "developing paraview" guide. It can
> be used as a framework for other applications and has been designed to do
> that but we haven't had any funding to work on a comprehensive guide for
> that.
>
> I haven't looked at the Live parts in depth in a while so my memory is
> probably fuzzy on that and it would take a good bit of time for me to
> familiarize myself with that part of the code. By murky I mean that things
> get a bit confusing when you're looking at the classes for both the client,
> server and in situ side. Kitware does offer support services if this is
> important to get working for you.
>
> Best,
> Andy
>
> On Fri, Oct 27, 2017 at 12:51 PM, Kolja Petersen 
> wrote:
>
>> More questions below...
>>
>> On Thu, Oct 26, 2017 at 5:54 PM, Andy Bauer 
>> wrote:
>>
>>> I inlined some answers below...
>>>
>>> On Wed, Oct 25, 2017 at 1:42 PM, Kolja Petersen >> > wrote:
>>>
 Thank you, Andy,
 unfortunately your answer has shown me how little I understand about
 Catalyst. But maybe it leads in the right direction after some work from my
 side.

 Just to make sure that we are talking about the same thing: My pvpython
 client is supposed to automate some of the visualization functionality of
 the paraview GUI. The coprocessing.py is in charge of the other side of the
 link, namely to provide data to the visualization side, if I understand
 correctly? I rather followed what happens in the constructor
 pqLiveInsituVisualizationManager::pqLiveInsituVisualizationManager(int
 connection_port, pqServer* server), which creates a
 vtkSMLiveInsituLinkProxy, not a vtkLiveInsituLink as coprocessing.py.

>>>
>>> Yes, coprocessing.py should only be used on the simulation side.
>>>
>>
>> So, did you suggest, coprocessing.py could also help to develop the
>> visualization side? If yes, how? Or was my first email too unclear, so that
>> my intention was not obvious?
>>
>>
>>>

 Now I saw that the vtkLiveInsituLink class can have ProcessType LIVE
 (for the visualization side? as server side object of the proxy created by
 paraview?) or INSITU (for the simulation side?). So apparently the same
 class can take both roles? If yes, that'd clarify some of my confusion. But
 still I don't see how coprocessing.py helps me to implement a python
 visualization client?


>>> Yes, the same class does work in both roles with the LIVE ProcessType
>>> being the visualization side and the INSITU being the simulation side.
>>> Things get a little murkier when there is a separate pvserver from the
>>> client that connects to the simulation.
>>>
>>
>> What do you mean by "murkier"? Paraview also uses a proxy (to a
>> vtkLiveInsituLink instance either on a pvserver or on the builtin server),
>> not  the class itself. Do you have an easier solution (C++ or python) by
>> using vtkInsituLink directly?
>>
>> Are there any design documents from Kitware that show the timings and
>> message exchange between Catalyst, pvserver and Paraview (or other
>> clients)? I don't know Kitware's publishing policies. My impression was
>> always that you are not just developing the Paraview application but also
>> an open source visualization programming framework. Maybe I'm wrong, but I
>> am missing some kind of "Paraview Programming Guide" that goes beyond a
>> User's Guide, to enable others understand the concepts behind Paraview and
>> contribute at a deeper level than just adding filters and data processing
>> algorithms.
>> Anyway, thanks for your explanations, although I couldn't make much
>> progress.
>> Kolja
>>
>
>
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

Re: [Paraview] LiveInsituLink proxy in pvpython client

[Andy Bauer]

Unfortunately there's no comprehensive "developing paraview" guide. It can
be used as a framework for other applications and has been designed to do
that but we haven't had any funding to work on a comprehensive guide for
that.

I haven't looked at the Live parts in depth in a while so my memory is
probably fuzzy on that and it would take a good bit of time for me to
familiarize myself with that part of the code. By murky I mean that things
get a bit confusing when you're looking at the classes for both the client,
server and in situ side. Kitware does offer support services if this is
important to get working for you.

Best,
Andy

On Fri, Oct 27, 2017 at 12:51 PM, Kolja Petersen 
wrote:

> More questions below...
>
> On Thu, Oct 26, 2017 at 5:54 PM, Andy Bauer 
> wrote:
>
>> I inlined some answers below...
>>
>> On Wed, Oct 25, 2017 at 1:42 PM, Kolja Petersen 
>> wrote:
>>
>>> Thank you, Andy,
>>> unfortunately your answer has shown me how little I understand about
>>> Catalyst. But maybe it leads in the right direction after some work from my
>>> side.
>>>
>>> Just to make sure that we are talking about the same thing: My pvpython
>>> client is supposed to automate some of the visualization functionality of
>>> the paraview GUI. The coprocessing.py is in charge of the other side of the
>>> link, namely to provide data to the visualization side, if I understand
>>> correctly? I rather followed what happens in the constructor
>>> pqLiveInsituVisualizationManager::pqLiveInsituVisualizationManager(int
>>> connection_port, pqServer* server), which creates a
>>> vtkSMLiveInsituLinkProxy, not a vtkLiveInsituLink as coprocessing.py.
>>>
>>
>> Yes, coprocessing.py should only be used on the simulation side.
>>
>
> So, did you suggest, coprocessing.py could also help to develop the
> visualization side? If yes, how? Or was my first email too unclear, so that
> my intention was not obvious?
>
>
>>
>>>
>>> Now I saw that the vtkLiveInsituLink class can have ProcessType LIVE
>>> (for the visualization side? as server side object of the proxy created by
>>> paraview?) or INSITU (for the simulation side?). So apparently the same
>>> class can take both roles? If yes, that'd clarify some of my confusion. But
>>> still I don't see how coprocessing.py helps me to implement a python
>>> visualization client?
>>>
>>>
>> Yes, the same class does work in both roles with the LIVE ProcessType
>> being the visualization side and the INSITU being the simulation side.
>> Things get a little murkier when there is a separate pvserver from the
>> client that connects to the simulation.
>>
>
> What do you mean by "murkier"? Paraview also uses a proxy (to a
> vtkLiveInsituLink instance either on a pvserver or on the builtin server),
> not  the class itself. Do you have an easier solution (C++ or python) by
> using vtkInsituLink directly?
>
> Are there any design documents from Kitware that show the timings and
> message exchange between Catalyst, pvserver and Paraview (or other
> clients)? I don't know Kitware's publishing policies. My impression was
> always that you are not just developing the Paraview application but also
> an open source visualization programming framework. Maybe I'm wrong, but I
> am missing some kind of "Paraview Programming Guide" that goes beyond a
> User's Guide, to enable others understand the concepts behind Paraview and
> contribute at a deeper level than just adding filters and data processing
> algorithms.
> Anyway, thanks for your explanations, although I couldn't make much
> progress.
> Kolja
>
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

Re: [Paraview] LiveInsituLink proxy in pvpython client

[Kolja Petersen]

More questions below...

On Thu, Oct 26, 2017 at 5:54 PM, Andy Bauer  wrote:

> I inlined some answers below...
>
> On Wed, Oct 25, 2017 at 1:42 PM, Kolja Petersen 
> wrote:
>
>> Thank you, Andy,
>> unfortunately your answer has shown me how little I understand about
>> Catalyst. But maybe it leads in the right direction after some work from my
>> side.
>>
>> Just to make sure that we are talking about the same thing: My pvpython
>> client is supposed to automate some of the visualization functionality of
>> the paraview GUI. The coprocessing.py is in charge of the other side of the
>> link, namely to provide data to the visualization side, if I understand
>> correctly? I rather followed what happens in the constructor
>> pqLiveInsituVisualizationManager::pqLiveInsituVisualizationManager(int
>> connection_port, pqServer* server), which creates a
>> vtkSMLiveInsituLinkProxy, not a vtkLiveInsituLink as coprocessing.py.
>>
>
> Yes, coprocessing.py should only be used on the simulation side.
>

So, did you suggest, coprocessing.py could also help to develop the
visualization side? If yes, how? Or was my first email too unclear, so that
my intention was not obvious?


>
>>
>> Now I saw that the vtkLiveInsituLink class can have ProcessType LIVE (for
>> the visualization side? as server side object of the proxy created by
>> paraview?) or INSITU (for the simulation side?). So apparently the same
>> class can take both roles? If yes, that'd clarify some of my confusion. But
>> still I don't see how coprocessing.py helps me to implement a python
>> visualization client?
>>
>>
> Yes, the same class does work in both roles with the LIVE ProcessType
> being the visualization side and the INSITU being the simulation side.
> Things get a little murkier when there is a separate pvserver from the
> client that connects to the simulation.
>

What do you mean by "murkier"? Paraview also uses a proxy (to a
vtkLiveInsituLink instance either on a pvserver or on the builtin server),
not  the class itself. Do you have an easier solution (C++ or python) by
using vtkInsituLink directly?

Are there any design documents from Kitware that show the timings and
message exchange between Catalyst, pvserver and Paraview (or other
clients)? I don't know Kitware's publishing policies. My impression was
always that you are not just developing the Paraview application but also
an open source visualization programming framework. Maybe I'm wrong, but I
am missing some kind of "Paraview Programming Guide" that goes beyond a
User's Guide, to enable others understand the concepts behind Paraview and
contribute at a deeper level than just adding filters and data processing
algorithms.
Anyway, thanks for your explanations, although I couldn't make much
progress.
Kolja
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

Re: [Paraview] LiveInsituLink proxy in pvpython client

[Andy Bauer]

I inlined some answers below...

On Wed, Oct 25, 2017 at 1:42 PM, Kolja Petersen 
wrote:

> Thank you, Andy,
> unfortunately your answer has shown me how little I understand about
> Catalyst. But maybe it leads in the right direction after some work from my
> side.
>
> Just to make sure that we are talking about the same thing: My pvpython
> client is supposed to automate some of the visualization functionality of
> the paraview GUI. The coprocessing.py is in charge of the other side of the
> link, namely to provide data to the visualization side, if I understand
> correctly? I rather followed what happens in the constructor
> pqLiveInsituVisualizationManager::pqLiveInsituVisualizationManager(int
> connection_port, pqServer* server), which creates a
> vtkSMLiveInsituLinkProxy, not a vtkLiveInsituLink as coprocessing.py.
>

Yes, coprocessing.py should only be used on the simulation side.


>
> Now I saw that the vtkLiveInsituLink class can have ProcessType LIVE (for
> the visualization side? as server side object of the proxy created by
> paraview?) or INSITU (for the simulation side?). So apparently the same
> class can take both roles? If yes, that'd clarify some of my confusion. But
> still I don't see how coprocessing.py helps me to implement a python
> visualization client?
>
>
Yes, the same class does work in both roles with the LIVE ProcessType being
the visualization side and the INSITU being the simulation side. Things get
a little murkier when there is a separate pvserver from the client that
connects to the simulation.


> Then, following the creation of a vtkSMLiveInsituLinkProxy in the
> pqLiveInsituVisualizationManager constructor, this proxy's server side
> vtkLiveInsituLink calls 
> "parallelController->TriggerRMIOnAllChildren(INITIALIZE_CONNECTION);"
> inside vtkLiveInsituLink::InsituConnect(vtkMultiProcessController*
> proc0NodesController). This seems to be the first RMI sent over the link.
> Is there any possibility to log the transmission or reception of this RMI?
> I don't find it in the cslogs.
> I think I could make some progress if I am able to debug where those
> messages are sent to, and why the python client doesn't react to them.
>

I believe the cslogs are only between the client and server. You may be
able to print the cslogs on the simulation side as well but I haven't tried
that to verify that it works.


> Thank you
> Kolja
>
> On Wed, Oct 25, 2017 at 4:05 PM, Andy Bauer 
> wrote:
>
>> Hi Kolja,
>>
>> I can't recall if anyone tried the Catalyst Live connection through
>> pvpython but I don't think anyone has done that yet. The design was meant
>> to go through the GUI. My suggestion would be to start looking at the
>> DoLiveVisualization() method in coprocessing.py.
>>
>> Cheers,
>> Andy
>>
>> On Tue, Oct 24, 2017 at 12:49 PM, Kolja Petersen > > wrote:
>>
>>> Hello,
>>> Are there any examples how to receive extracts from Catalyst simulations
>>> in a batch pvpython script?
>>>
>>> My first attempt looks as attached. I connect to a pvserver running on
>>> localhost. Then I create a LiveInsituLink proxy, set the insitu port to
>>> 2 and initialize the proxy. Finally, I repeat ProcessEvents(), which I
>>> hoped would allow me to interact with the Catalyst simulation.
>>>
>>> However, when I first start pvserver, then the "coproc.py" script
>>> (attached), and finally a Catalyst example (e.g.
>>> Examples/Catalyst/CFullExample/FEDriver.c), nothing useful happens. The
>>> callback, which should react to any LiveInsituLink events, is only called
>>> during initialization (UpdatePropertyEvent and ModifiedEvent), then never
>>> again. lil.GetTimeStep() in the endless loop shows always the same
>>> (uninitialized?) value, there are no Debug messages from the LiveInsituLink
>>> proxy.
>>>
>>> How can I receive a notification, when a Catalyst simulation connects?
>>> How can I get a list of available extracts?
>>> Thank you
>>> Kolja
>>>
>>> ___
>>> Powered by www.kitware.com
>>>
>>> Visit other Kitware open-source projects at
>>> http://www.kitware.com/opensource/opensource.html
>>>
>>> Please keep messages on-topic and check the ParaView Wiki at:
>>> http://paraview.org/Wiki/ParaView
>>>
>>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>>
>>> Follow this link to subscribe/unsubscribe:
>>> http://public.kitware.com/mailman/listinfo/paraview
>>>
>>>
>>
>
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

Re: [Paraview] LiveInsituLink proxy in pvpython client

[Kolja Petersen]

Thank you, Andy,
unfortunately your answer has shown me how little I understand about
Catalyst. But maybe it leads in the right direction after some work from my
side.

Just to make sure that we are talking about the same thing: My pvpython
client is supposed to automate some of the visualization functionality of
the paraview GUI. The coprocessing.py is in charge of the other side of the
link, namely to provide data to the visualization side, if I understand
correctly? I rather followed what happens in the constructor
pqLiveInsituVisualizationManager::pqLiveInsituVisualizationManager(int
connection_port, pqServer* server), which creates a
vtkSMLiveInsituLinkProxy, not a vtkLiveInsituLink as coprocessing.py.

Now I saw that the vtkLiveInsituLink class can have ProcessType LIVE (for
the visualization side? as server side object of the proxy created by
paraview?) or INSITU (for the simulation side?). So apparently the same
class can take both roles? If yes, that'd clarify some of my confusion. But
still I don't see how coprocessing.py helps me to implement a python
visualization client?

Then, following the creation of a vtkSMLiveInsituLinkProxy in the
pqLiveInsituVisualizationManager constructor, this proxy's server side
vtkLiveInsituLink calls
"parallelController->TriggerRMIOnAllChildren(INITIALIZE_CONNECTION);"
inside vtkLiveInsituLink::InsituConnect(vtkMultiProcessController*
proc0NodesController). This seems to be the first RMI sent over the link.
Is there any possibility to log the transmission or reception of this RMI?
I don't find it in the cslogs.
I think I could make some progress if I am able to debug where those
messages are sent to, and why the python client doesn't react to them.
Thank you
Kolja

On Wed, Oct 25, 2017 at 4:05 PM, Andy Bauer  wrote:

> Hi Kolja,
>
> I can't recall if anyone tried the Catalyst Live connection through
> pvpython but I don't think anyone has done that yet. The design was meant
> to go through the GUI. My suggestion would be to start looking at the
> DoLiveVisualization() method in coprocessing.py.
>
> Cheers,
> Andy
>
> On Tue, Oct 24, 2017 at 12:49 PM, Kolja Petersen 
> wrote:
>
>> Hello,
>> Are there any examples how to receive extracts from Catalyst simulations
>> in a batch pvpython script?
>>
>> My first attempt looks as attached. I connect to a pvserver running on
>> localhost. Then I create a LiveInsituLink proxy, set the insitu port to
>> 2 and initialize the proxy. Finally, I repeat ProcessEvents(), which I
>> hoped would allow me to interact with the Catalyst simulation.
>>
>> However, when I first start pvserver, then the "coproc.py" script
>> (attached), and finally a Catalyst example (e.g.
>> Examples/Catalyst/CFullExample/FEDriver.c), nothing useful happens. The
>> callback, which should react to any LiveInsituLink events, is only called
>> during initialization (UpdatePropertyEvent and ModifiedEvent), then never
>> again. lil.GetTimeStep() in the endless loop shows always the same
>> (uninitialized?) value, there are no Debug messages from the LiveInsituLink
>> proxy.
>>
>> How can I receive a notification, when a Catalyst simulation connects?
>> How can I get a list of available extracts?
>> Thank you
>> Kolja
>>
>> ___
>> Powered by www.kitware.com
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Please keep messages on-topic and check the ParaView Wiki at:
>> http://paraview.org/Wiki/ParaView
>>
>> Search the list archives at: http://markmail.org/search/?q=ParaView
>>
>> Follow this link to subscribe/unsubscribe:
>> http://public.kitware.com/mailman/listinfo/paraview
>>
>>
>
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

Re: [Paraview] LiveInsituLink proxy in pvpython client

[Andy Bauer]

Hi Kolja,

I can't recall if anyone tried the Catalyst Live connection through
pvpython but I don't think anyone has done that yet. The design was meant
to go through the GUI. My suggestion would be to start looking at the
DoLiveVisualization() method in coprocessing.py.

Cheers,
Andy

On Tue, Oct 24, 2017 at 12:49 PM, Kolja Petersen 
wrote:

> Hello,
> Are there any examples how to receive extracts from Catalyst simulations
> in a batch pvpython script?
>
> My first attempt looks as attached. I connect to a pvserver running on
> localhost. Then I create a LiveInsituLink proxy, set the insitu port to
> 2 and initialize the proxy. Finally, I repeat ProcessEvents(), which I
> hoped would allow me to interact with the Catalyst simulation.
>
> However, when I first start pvserver, then the "coproc.py" script
> (attached), and finally a Catalyst example (e.g.
> Examples/Catalyst/CFullExample/FEDriver.c), nothing useful happens. The
> callback, which should react to any LiveInsituLink events, is only called
> during initialization (UpdatePropertyEvent and ModifiedEvent), then never
> again. lil.GetTimeStep() in the endless loop shows always the same
> (uninitialized?) value, there are no Debug messages from the LiveInsituLink
> proxy.
>
> How can I receive a notification, when a Catalyst simulation connects?
> How can I get a list of available extracts?
> Thank you
> Kolja
>
> ___
> Powered by www.kitware.com
>
> Visit other Kitware open-source projects at http://www.kitware.com/
> opensource/opensource.html
>
> Please keep messages on-topic and check the ParaView Wiki at:
> http://paraview.org/Wiki/ParaView
>
> Search the list archives at: http://markmail.org/search/?q=ParaView
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/paraview
>
>
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview

[Paraview] LiveInsituLink proxy in pvpython client

[Kolja Petersen]

Hello,
Are there any examples how to receive extracts from Catalyst simulations in
a batch pvpython script?

My first attempt looks as attached. I connect to a pvserver running on
localhost. Then I create a LiveInsituLink proxy, set the insitu port to
2 and initialize the proxy. Finally, I repeat ProcessEvents(), which I
hoped would allow me to interact with the Catalyst simulation.

However, when I first start pvserver, then the "coproc.py" script
(attached), and finally a Catalyst example (e.g.
Examples/Catalyst/CFullExample/FEDriver.c),
nothing useful happens. The callback, which should react to any
LiveInsituLink events, is only called during initialization
(UpdatePropertyEvent and ModifiedEvent), then never again.
lil.GetTimeStep() in the endless loop shows always the same
(uninitialized?) value, there are no Debug messages from the LiveInsituLink
proxy.

How can I receive a notification, when a Catalyst simulation connects?
How can I get a list of available extracts?
Thank you
Kolja


coproc.py
Description: Binary data
___
Powered by www.kitware.com

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Please keep messages on-topic and check the ParaView Wiki at: 
http://paraview.org/Wiki/ParaView

Search the list archives at: http://markmail.org/search/?q=ParaView

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview