Re: [Rd] using Paraview "in-situ" with R?

[Mike Marchywka]

I'm posting here instead of R-package since your response
was the most relevant :) I started using the "in-situ" workflow
and its very nice.  Thew viewer stays up between runs and across
data sources and its the same UI regardless of source :) 
fwiw. 

I started digging through some noisy data, dog diet data
in which the supplements were pulsed ( in essence PWM lol).
This made it hard to find the right average doses to correlate
with outcomes. I had to play with trailing average filtering but
ultimately some trends emerged that made sense with
subjective observations ( maybe they are not real but that is
not the point of this post ).

I did not use R or Visit but  I used my own code 
for filtering and MJMDatascope for viewing and expect
to do more sophisticated stuff in R soon. I can tell you
though that eliminating the annoyance and confusion of
temp files was very helpful. The workflow, and mind flow,
for the "in situ" mode makes you wonder how you ever did without
it. I will probably need to update my R interface package and
MJMDatascope is hardly ideal but improving both seems
worth the effort.

Thanks.


 Mike Marchywka 
44 Crosscreek Trail
Jasper GA 30143
was 306 Charles Cox Drive  Canton, GA 30115
470-758-0799
404-788-1216 





From: George Ostrouchov 
Sent: Wednesday, January 10, 2024 3:06 PM
To: r-devel@r-project.org
Cc: Mike Marchywka
Subject: Re:  [Rd] using Paraview "in-situ" with R?

At ORNL, we worked with VisIt (a sibling of Paraview, both funded largely by 
DOE) around 2016 and made an in situ demo with R. We used packages pbdMPI (on 
CRAN) and pbdDMAT (on GitHub/RbigData), which were in part built for this 
purpose. Later also the package hola (on GitHub/RbigData) was built to connect 
with adios2, which can do buffered in situ connections with various codes.

But the VisIt developers were not interested in R (preferring to roll their 
own), so that direction fizzled. Paraview is a competetive sibling of VisIt, so 
I don’t know if they would be interested. The packages we developed are viable 
for that purpose. There is a lot in R that could benefit Paraview (or VisIt).

George

>
> Message: 1
> Date: Tue, 9 Jan 2024 14:20:17 +
> From: Mike Marchywka 
> To: R-devel 
> Subject: [Rd] using Paraview "in-situ" with R?
> Message-ID:
>   
> 
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I had previously asked about R interfaces to various "other" visualization
> tools specifically lightweights for monitoring progress of
> various codes. I was working on this,
>
> https://github.com/mmarchywka/mjmdatascope
>
> but in the meantime found out that Paraview has an "in-situ"
> capability for similar objectives.
>
> https://discourse.paraview.org/t/does-or-can-paraview-support-streaming-input/13637/9
>
> While R does have a lot of plotting features,
> it seems like an excellent tool to interface to R allowing visualization 
> without
> a bunch of temp files or
>
> Is anyone aware of anyone doing this interface or reasons its  a boondoggle?
>
> Thanks.
>
>
>
>  Mike Marchywka
> 44 Crosscreek Trail
> Jasper GA 30143
> was 306 Charles Cox Drive  Canton, GA 30115
> 470-758-0799
> 404-788-1216
>


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] using Paraview "in-situ" with R?

[George Ostrouchov]

Thanks for adding more explanation. As Ivan Krylov mentioned earlier, this 
sounds like an idea for developing an R package. The viewers and R largely 
operate in communities that so far have little interaction and both can benefit 
from ideas in the other. 

George

> On Jan 11, 2024, at 6:30 AM, Mike Marchywka  wrote:
> 
> Thanks. I take it though you see "R" in this role as adding to the 
> capabilities of 
> the viewers, maybe adding some quick model fits over FEM results or something?
> Right now I was imagining working with freefem and rolling my own c++ code
> with supporting use of R code. Ideally I could easily overlay stuff without
> messing around with temp files.  There are a lot of R things, probably
> optimizations etc, that may be nice to view as they progress
> with more than just a figure of merit. 
> Right now I'm just trying to use Runge-Kutta on a simple orbit 
> and the mjmdatascope output is much more useful on-the-fly 
> than text or after the fact.
> 
> 
>  Mike Marchywka 
> 44 Crosscreek Trail
> Jasper GA 30143
> was 306 Charles Cox Drive  Canton, GA 30115
> 470-758-0799
> 404-788-1216 
> 
> 
> 
> 
> 
> From: George Ostrouchov 
> Sent: Wednesday, January 10, 2024 3:06 PM
> To: r-devel@r-project.org
> Cc: Mike Marchywka
> Subject: Re:  [Rd] using Paraview "in-situ" with R?
> 
> At ORNL, we worked with VisIt (a sibling of Paraview, both funded largely by 
> DOE) around 2016 and made an in situ demo with R. We used packages pbdMPI (on 
> CRAN) and pbdDMAT (on GitHub/RbigData), which were in part built for this 
> purpose. Later also the package hola (on GitHub/RbigData) was built to 
> connect with adios2, which can do buffered in situ connections with various 
> codes.
> 
> But the VisIt developers were not interested in R (preferring to roll their 
> own), so that direction fizzled. Paraview is a competetive sibling of VisIt, 
> so I don’t know if they would be interested. The packages we developed are 
> viable for that purpose. There is a lot in R that could benefit Paraview (or 
> VisIt).
> 
> George
> 
>> 
>> Message: 1
>> Date: Tue, 9 Jan 2024 14:20:17 +
>> From: Mike Marchywka 
>> To: R-devel 
>> Subject: [Rd] using Paraview "in-situ" with R?
>> Message-ID:
>>  
>> 
>> 
>> Content-Type: text/plain; charset="iso-8859-1"
>> 
>> I had previously asked about R interfaces to various "other" visualization
>> tools specifically lightweights for monitoring progress of
>> various codes. I was working on this,
>> 
>> https://github.com/mmarchywka/mjmdatascope
>> 
>> but in the meantime found out that Paraview has an "in-situ"
>> capability for similar objectives.
>> 
>> https://discourse.paraview.org/t/does-or-can-paraview-support-streaming-input/13637/9
>> 
>> While R does have a lot of plotting features,
>> it seems like an excellent tool to interface to R allowing visualization 
>> without
>> a bunch of temp files or
>> 
>> Is anyone aware of anyone doing this interface or reasons its  a boondoggle?
>> 
>> Thanks.
>> 
>> 
>> 
>> Mike Marchywka
>> 44 Crosscreek Trail
>> Jasper GA 30143
>> was 306 Charles Cox Drive  Canton, GA 30115
>> 470-758-0799
>> 404-788-1216
>> 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] using Paraview "in-situ" with R?

[Mike Marchywka]

Thanks. I take it though you see "R" in this role as adding to the capabilities 
of 
the viewers, maybe adding some quick model fits over FEM results or something?
Right now I was imagining working with freefem and rolling my own c++ code
with supporting use of R code. Ideally I could easily overlay stuff without
messing around with temp files.  There are a lot of R things, probably
optimizations etc, that may be nice to view as they progress
with more than just a figure of merit. 
Right now I'm just trying to use Runge-Kutta on a simple orbit 
and the mjmdatascope output is much more useful on-the-fly 
than text or after the fact.


 Mike Marchywka 
44 Crosscreek Trail
Jasper GA 30143
was 306 Charles Cox Drive  Canton, GA 30115
470-758-0799
404-788-1216 





From: George Ostrouchov 
Sent: Wednesday, January 10, 2024 3:06 PM
To: r-devel@r-project.org
Cc: Mike Marchywka
Subject: Re:  [Rd] using Paraview "in-situ" with R?

At ORNL, we worked with VisIt (a sibling of Paraview, both funded largely by 
DOE) around 2016 and made an in situ demo with R. We used packages pbdMPI (on 
CRAN) and pbdDMAT (on GitHub/RbigData), which were in part built for this 
purpose. Later also the package hola (on GitHub/RbigData) was built to connect 
with adios2, which can do buffered in situ connections with various codes.

But the VisIt developers were not interested in R (preferring to roll their 
own), so that direction fizzled. Paraview is a competetive sibling of VisIt, so 
I don’t know if they would be interested. The packages we developed are viable 
for that purpose. There is a lot in R that could benefit Paraview (or VisIt).

George

>
> Message: 1
> Date: Tue, 9 Jan 2024 14:20:17 +
> From: Mike Marchywka 
> To: R-devel 
> Subject: [Rd] using Paraview "in-situ" with R?
> Message-ID:
>   
> 
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I had previously asked about R interfaces to various "other" visualization
> tools specifically lightweights for monitoring progress of
> various codes. I was working on this,
>
> https://github.com/mmarchywka/mjmdatascope
>
> but in the meantime found out that Paraview has an "in-situ"
> capability for similar objectives.
>
> https://discourse.paraview.org/t/does-or-can-paraview-support-streaming-input/13637/9
>
> While R does have a lot of plotting features,
> it seems like an excellent tool to interface to R allowing visualization 
> without
> a bunch of temp files or
>
> Is anyone aware of anyone doing this interface or reasons its  a boondoggle?
>
> Thanks.
>
>
>
>  Mike Marchywka
> 44 Crosscreek Trail
> Jasper GA 30143
> was 306 Charles Cox Drive  Canton, GA 30115
> 470-758-0799
> 404-788-1216
>


__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] using Paraview "in-situ" with R?

[George Ostrouchov]

At ORNL, we worked with VisIt (a sibling of Paraview, both funded largely by 
DOE) around 2016 and made an in situ demo with R. We used packages pbdMPI (on 
CRAN) and pbdDMAT (on GitHub/RbigData), which were in part built for this 
purpose. Later also the package hola (on GitHub/RbigData) was built to connect 
with adios2, which can do buffered in situ connections with various codes.

But the VisIt developers were not interested in R (preferring to roll their 
own), so that direction fizzled. Paraview is a competetive sibling of VisIt, so 
I don’t know if they would be interested. The packages we developed are viable 
for that purpose. There is a lot in R that could benefit Paraview (or VisIt).

George

> 
> Message: 1
> Date: Tue, 9 Jan 2024 14:20:17 +
> From: Mike Marchywka 
> To: R-devel 
> Subject: [Rd] using Paraview "in-situ" with R?
> Message-ID:
>   
> 
>   
> Content-Type: text/plain; charset="iso-8859-1"
> 
> I had previously asked about R interfaces to various "other" visualization
> tools specifically lightweights for monitoring progress of
> various codes. I was working on this,
> 
> https://github.com/mmarchywka/mjmdatascope
> 
> but in the meantime found out that Paraview has an "in-situ"
> capability for similar objectives. 
> 
> https://discourse.paraview.org/t/does-or-can-paraview-support-streaming-input/13637/9
> 
> While R does have a lot of plotting features, 
> it seems like an excellent tool to interface to R allowing visualization 
> without
> a bunch of temp files or 
> 
> Is anyone aware of anyone doing this interface or reasons its  a boondoggle?
> 
> Thanks. 
> 
> 
> 
>  Mike Marchywka 
> 44 Crosscreek Trail
> Jasper GA 30143
> was 306 Charles Cox Drive  Canton, GA 30115
> 470-758-0799
> 404-788-1216 
> 

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

Re: [Rd] using Paraview "in-situ" with R?

[Ivan Krylov via R-devel]

В Tue, 9 Jan 2024 14:20:17 +
Mike Marchywka  пишет:

> it seems like an excellent tool to interface to R allowing
> visualization without a bunch of temp files or 
> 
> Is anyone aware of anyone doing this interface or reasons its  a
> boondoggle?

This sounds like it's better suited for r-package-de...@r-project.org,
not R-devel itself.

In theory, nothing should prevent you from writing C++ code interfacing
with ParaView (via its "adios" streaming library) and with R. The Rcpp
package will likely help you bring the semantics of the two languages
closer together. (Memory allocation and error handling are the two
major topics where R and C++ especially disagree.)

On the R side, make an object with reference semantics (i.e. an
external pointer) and use callbacks to update it with new information
while R code is running. On the R extension side, translate these
callbacks into necessary calls to the adios library to transfer the
data to ParaView.

For more informaion, see Writing R Extensions at
 and Rcpp
documentation at .

-- 
Best regards,
Ivan

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel