Mark,
Thanks for letting us know. We have added it to the PETSc package list. Should any issues come up we'll be glad to help support it. Barry > On Jun 25, 2020, at 4:53 AM, Mark Olesen <mark.ole...@esi-group.com> wrote: > > Late followup, but people may want to know that a PETSc/OpenFOAM interface is > included in the pending OpenFOAM-v2006 release as a git submodule: > > https://develop.openfoam.com/modules/external-solver > > Cheers, > /mark > > On 2019-05-24 12:06, Matthew Knepley via petsc-users wrote: >> On Thu, May 23, 2019 at 10:41 PM Vu Q. Do via petsc-users >> <petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> wrote: >> Hi all, >> Thanks for your previous suggestion, I have been able to >> successfully link Petsc to OpenFOAM. I have written a simple >> interface and it works quite well in serial mode, but cannot run in >> parallel. I have been thinking about this problem for weeks but >> couldn't solve it. So I think maybe you could give me some idea. I >> describe my problem below. >> My interface is just a class named "petscSolver/"/, which is used to >> convert an openfoam's matrix or blocked matrix to Petsc Mat, then >> solve the matrix using Petsc's solver. >> To use Petsc, an Openfoam's solver need to be recompiled after >> adding the following lines to make file: >> EXE_INC = \ >> ... >> -I$(LIB_SRC)/petscSolver \ >> -I$(PETSC_DIR)/include \ >> -I$(PETSC_DIR)/$(PETSC_ARCH)/include >> EXE_LIBS = \ >> ... >> -L$(PETSC_DIR)/$(PETSC_ARCH)/lib -lpetsc >> To run an openfoam's case in parallel, first I need to discretize >> the domain into subdomains (e.g. 2 subdomains ), then use the >> following command: >> mpirun -np 2 mySolver -parallel >> (where mpirun is literally mpiexec) The simulation crashed even >> before doing anything and the error message is as in the attached image. >> Run in the debugger and see where it is crashing. Its possible to screw up >> the MPI linking here, so that you >> link OPENFOAM with one MPI and PETSc with another, or you call MPIInit() >> after you call PetscInitialize(), etc. >> Thanks, >> Matt >> I have tested and realized that the solver can run in parallel as >> normal by removing the two lines: >> -I$(PETSC_DIR)/include \ >> -I$(PETSC_DIR)/$(PETSC_ARCH)/include >> But then it is clearly no longer linked to Petsc. >> I would appreciate any suggestion. >> Screenshot from 2019-05-24 09-22-17.png >> On Thu, Apr 11, 2019 at 1:37 PM Smith, Barry F. <bsm...@mcs.anl.gov >> <mailto:bsm...@mcs.anl.gov>> wrote: >> Mark, >> Thanks for the clarifying email. My google searches didn't >> locate the rheoTool you mention nor "a PRACE project running via >> CINECA (Bologna)". >> It would be nice if someday OpenFOAM had (either directly >> or somehow with the modules directory) an interface to the PETSc >> solvers. This would allow the use of a variety of other solvers >> including hypre BoomerAMG, SuperLU_Dist, MUMPS, and even the use >> of PETSc/ViennaCL GPU based solvers automatically from OpenFOAM. >> Unfortunately the PETSc group doesn't have the resources or >> expertise to develop and support such an interface ourselves. We >> would, of course, try to answer emails about PETSc usage and >> bugs for such an interface. >> Barry >> If OpenFOAM did have such an interface one thing we could >> provide is the CI infrastructure for tracking changes to PETSc >> that may effect OpenFOAM. For example we could automatically >> build OpenFOAM each day with the latest master of PETSc thus >> immediately detecting changes that effect the interface. >> > On Apr 10, 2019, at 4:55 PM, Mark Olesen >> <mark.ole...@esi-group.com <mailto:mark.ole...@esi-group.com>> >> wrote: >> > >> > The paper that Barry mentioned gives some generalities, but >> probably >> > won't help much. There are some PETSc/OpenFOAM interfaces in >> rheoTool >> > that are probably much more helpful. >> > >> > As Barry also rightly noted, there are some config files in >> the OpenFOAM >> > tree that were put in some time ago for helping with setting >> up PETSc >> > and OpenFOAM. Assuming that you have set the appropriate >> values in the >> > etc/config.sh/petsc <http://config.sh/petsc> file you will be >> able to use those when using wmake. >> > For running you will still need to ensure that the >> LD_LIBARY_PATH is set >> > correctly. For example, what some build scripts exhibit: >> > >> > wmake(petsc) : >> > ==> Before running, verify that PETSc libraries can be found >> > >> > Enable in the OpenFOAM etc/bashrc, define manually or try >> with the >> > following (POSIX shell): >> > >> > eval $(foamEtcFile -sh -config petsc -- -force) >> > >> > == >> > >> > >> > There is currently a PRACE project running via CINECA >> (Bologna) with >> > binding in PETSc as a runtime selectable linear solver in >> OpenFOAM. This >> > is still at the stage of early testing and performance >> benchmarking. >> > >> > Cheers, >> > /mark >> > >> > On 4/10/19 6:37 PM, Smith, Barry F. via petsc-users wrote: >> >> >> >> We don't know much about OpenFoam but >> >> >> >> 1) if I do a >> >> >> >> git grep -i petsc >> >> >> >> in the >> https://develop.openfoam.com/Development/OpenFOAM-plus.git >> repository I see various configuration files specifically for PETSc. >> >> >> >> etc/config.csh/petsc etc/config.sh/petsc >> <http://config.sh/petsc> wmake/scripts/have_petsc >> >> >> >> so it appears that OpenFOAM has the tools to be linked >> against PETSc (to me the documentation on how to use them is >> rather terse). Are >> >> you using these? If you have trouble with them perhaps >> you can ask the OpenFOAM user community how to use them. >> >> >> >> >> >> 2) if you are editing the Make/options file directly you >> can try changing >> >> >> >> -L$(PETSC_DIR)/$(PETSC_ARCH)/lib -lpetsc >> >> >> >> to >> >> >> >> -Wl,-rpath,$(PETSC_DIR)/$(PETSC_ARCH)/lib >> -L$(PETSC_DIR)/$(PETSC_ARCH)/lib -lpetsc >> >> >> >> >> >> >> >> Note also that simply including petsc.h into the OpenFoam >> source code and linking against -lpetsc will not immediately >> allow calling the PETSc solvers from OpenFOAM. One needs to >> write all the interface code that sets up and calls the PETSc >> solvers from OpenFOAM. There is a paper >> >> https://www.researchgate.net/publication/319045499_Insertion_of_PETSc_in_the_OpenFOAM_Framework >> that describes at an abstract level how they wrote code that >> calls the PETSc solvers from OpenFOAM but the source code that >> actually does the work does not appear to be available. >> >> >> >> Note that PETSc is now at version 3.11 we recommend >> working with that version (unless you already have a lot of code >> that calls PETSc written with a previous version of PETSc, for >> that we recommend first upgrading to petsc 3.11 and then >> continuing to add code). >> >> >> >> Barry >> >> >> >> >> >> >> >> >> >> >> >>> On Apr 10, 2019, at 8:23 AM, Balay, Satish via petsc-users >> <petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> wrote: >> >>> >> >>> Runtime error? You might have to add the path to >> $PETSC_ARCH/lib in LD_LIBRARY_PATH env variable >> >>> or - to your link command. If linux/gcc - the linker option >> is -Wl,-rpath,$PETSC_ARCH/lib >> >>> >> >>> If not - send detail logs. >> >>> >> >>> Satish >> >>> >> >>> On Wed, 10 Apr 2019, Vu Do Quoc via petsc-users wrote: >> >>> >> >>>> Hi all, >> >>>> >> >>>> I am trying to insert Petsc to OpenFOAM opensource software. >> >>>> I have been successfully compiling Petsc with an available >> solver in >> >>>> OpenFOAM by linking it with the shared library >> libpetsc.so. However, when I >> >>>> call the solver to run a test case, I got an error saying >> that: >> >>>> "libpetsc.so cannot be found", even though the library >> still exists in the >> >>>> $PETSC_ARCH/lib folder. >> >>>> >> >>>> I have been struggling for weeks but still, have not been >> able to figure it >> >>>> out. Therefore I would be very grateful for any suggestion >> to solve this >> >>>> problem. >> >>>> >> >>>> Thanks in advance for your time, >> >>>> >> >>>> Best regards, >> >>>> >> >>>> Vu Do >> >>>> >> -- *Vu Q. Do >> *------------------------------------------------ >> /Student of Aeronautical Engineering/ >> Programme de Formation d'Ingénieurs d'Excellence au Vietnam /- PFIEV/ >> School of Transportation Engineering/ >> / >> Hanoi University of Science and Technology >> 01 Dai Co Viet Avenue, Hanoi, Vietnam >> E-mail: vu.doquoch...@gmail.com <mailto:vu.doquoch...@gmail.com> >> -- >> What most experimenters take for granted before they begin their experiments >> is infinitely more interesting than any results to which their experiments >> lead. >> -- Norbert Wiener >> https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/> > > -- > Dr Mark OLESEN > Principal Engineer, ESI-OpenCFD > Engineering System International GmbH | Einsteinring 24 | 85609 Munich > Mob. +49 171 9710 149 > www.openfoam.com | www.esi-group.com | mark.ole...@esi-group.com