Hi Junchao Apologies for not seeing that. Usually, the fortran90-specific functions have notes on the original C version, and I also can't see PetscSFCreateSectionSFF90 on the function list on the doc site. Thanks so much, and I saw your notes on the merge request.
I don't suppose PetscSFReduceBegin and End are likewise hidden somewhere. I'm moving between distributions, and I can go forward with PetscSFBcastBegin, but I also need to go backward with Reduce. I feel like this is a one-to-one change from Bcast to Reduce, and I've added the relevant lines in src/vec/is/sf/interface/ftn-custom/zsf.c and src/vec/f90-mod/petscvec.h90 and it compiles fine, but I'm still getting a linking error for the Reduce routines. I need some input on what I'm missing here. I hope I didn't miss that this routine exists elsewhere. I've attached the two files, but it's not an ideal way to transmit changes. If I get some instructions on contributing, I can make a merge request for the changes if they are helpful. Thanks Nicholas On Tue, Jan 10, 2023 at 4:42 PM Junchao Zhang <[email protected]> wrote: > Hi, Nicholas, > It seems we have implemented it, but with another name, > PetscSFCreateSectionSFF90, see > https://gitlab.com/petsc/petsc/-/merge_requests/5386 > Try it to see if it works! > > --Junchao Zhang > > > On Tue, Jan 10, 2023 at 11:45 AM Nicholas Arnold-Medabalimi < > [email protected]> wrote: > >> Er to be honest I still can't get my stub to compile properly, and I >> don't know how to go about making a merge request. But here is what I am >> attempting right now. Let me know how best to proceed >> >> >> Its not exactly clear to me how to setup up the remote offset properly. >> >> in src/vec/is/sf/interface/ftn-custom/zsf.c >> >> PETSC_EXTERN void petscsfcreatesectionsf(PetscSF *sf, PetscSection >> *rootSection, F90Array1d *aremoteOffsets, PetscSection *leafSection, >> PetscSF *sectionSF, int * ierr PETSC_F90_2PTR_PROTO(remoteoffsetsd)) >> { >> >> int * remoteOffsets; >> *ierr = F90Array1dAccess(aremoteOffsets, PETSC_INT, (void**) >> &remoteOffsets PETSC_F90_2PTR_PARAM(remoteoffsetsd));if (*ierr) return; >> *ierr = PetscSFCreateSectionSF(*sf,*rootSection, >> &remoteOffsets,*leafSection,*sectionSF);if (*ierr) return; >> >> } >> >> This is the sticking point. >> >> Sincerely >> Nicholas >> >> >> On Tue, Jan 10, 2023 at 12:38 PM Junchao Zhang <[email protected]> >> wrote: >> >>> Hi, Nicholas, >>> Could you make a merge request to PETSc and then our Fortran experts >>> can comment on your MR? >>> Thanks. >>> >>> --Junchao Zhang >>> >>> >>> On Tue, Jan 10, 2023 at 11:10 AM Nicholas Arnold-Medabalimi < >>> [email protected]> wrote: >>> >>>> Hi Junchao >>>> >>>> I think I'm almost there, but I could use some insight into how to use >>>> the PETSC_F90_2PTR_PROTO and F90Array1dAccess for the remoteOffset >>>> parameter input so if another function comes up, I can add it myself >>>> without wasting your time. >>>> I am very grateful for your help and time. >>>> >>>> Sincerely >>>> Nicholas >>>> >>>> On Tue, Jan 10, 2023 at 10:55 AM Junchao Zhang <[email protected]> >>>> wrote: >>>> >>>>> Hi, Nicholas, >>>>> I am not a fortran guy, but I will try to add >>>>> petscsfcreatesectionsf. >>>>> >>>>> Thanks. >>>>> --Junchao Zhang >>>>> >>>>> >>>>> On Tue, Jan 10, 2023 at 12:50 AM Nicholas Arnold-Medabalimi < >>>>> [email protected]> wrote: >>>>> >>>>>> I think it should be something like this, but I'm not very fluent in >>>>>> Fortran C interop syntax. Any advice would be appreciated. Thanks >>>>>> >>>>>> PETSC_EXTERN void petscsfcreatesectionsf(PetscSF *sf, PetscSection * >>>>>> rootSection, F90Array1d *aremoteOffsets, PetscSection *leafSection, >>>>>> PetscSF *sectionSF, int * ierr PETSC_F90_2PTR_PROTO(remoteoffsetsd)) >>>>>> { >>>>>> >>>>>> int * remoteOffsets; >>>>>> *ierr = F90Array1dAccess(aremoteOffsets, PETSC_INT, (void**) & >>>>>> remoteOffsets PETSC_F90_2PTR_PARAM(remoteoffsetsd));if (*ierr) return >>>>>> ; >>>>>> *ierr = PetscSFCreateSectionSF(*sf,*rootSection, &remoteOffsets,* >>>>>> leafSection,*sectionSF);if (*ierr) return; >>>>>> >>>>>> } >>>>>> >>>>>> On Mon, Jan 9, 2023 at 11:41 PM Nicholas Arnold-Medabalimi < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Junchao >>>>>>> >>>>>>> Thanks again for your help in November. I've been using the your >>>>>>> merge request branch quite heavily. Would it be possible to add a >>>>>>> petscsfcreatesectionsf interface as well? >>>>>>> I'm trying to write it myself using your commits as a guide but I >>>>>>> have been struggling with handling the section parameter properly. >>>>>>> >>>>>>> Sincerely >>>>>>> Nicholas >>>>>>> >>>>>>> On Sat, Nov 19, 2022 at 9:44 PM Junchao Zhang < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sat, Nov 19, 2022 at 8:05 PM Nicholas Arnold-Medabalimi < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> Thanks, this is awesome. Thanks for the very prompt fix. Just one >>>>>>>>> question: will the array outputs on the fortran side copies (and need >>>>>>>>> to be >>>>>>>>> deallocated) or direct access to the dmplex? >>>>>>>>> >>>>>>>> Direct access to internal data; no need to deallocate >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Sincerely >>>>>>>>> Nicholas >>>>>>>>> >>>>>>>>> On Sat, Nov 19, 2022 at 8:21 PM Junchao Zhang < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi, Nicholas, >>>>>>>>>> See this MR, >>>>>>>>>> https://gitlab.com/petsc/petsc/-/merge_requests/5860 >>>>>>>>>> It is in testing, but you can try branch >>>>>>>>>> jczhang/add-petscsf-fortran to see if it works for you. >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> --Junchao Zhang >>>>>>>>>> >>>>>>>>>> On Sat, Nov 19, 2022 at 4:16 PM Nicholas Arnold-Medabalimi < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Junchao >>>>>>>>>>> >>>>>>>>>>> Thanks. I was wondering if there is any update on this. I may >>>>>>>>>>> write a small interface for those two routines myself in the >>>>>>>>>>> interim but >>>>>>>>>>> I'd appreciate any insight you have. >>>>>>>>>>> >>>>>>>>>>> Sincerely >>>>>>>>>>> Nicholas >>>>>>>>>>> >>>>>>>>>>> On Wed, Nov 16, 2022 at 10:39 PM Junchao Zhang < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, Nicholas, >>>>>>>>>>>> I will have a look and get back to you. >>>>>>>>>>>> Thanks. >>>>>>>>>>>> --Junchao Zhang >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Nov 16, 2022 at 9:27 PM Nicholas Arnold-Medabalimi < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Petsc Users >>>>>>>>>>>>> >>>>>>>>>>>>> I'm in the process of adding some Petsc for mesh management >>>>>>>>>>>>> into an existing Fortran Solver. It has been relatively >>>>>>>>>>>>> straightforward so >>>>>>>>>>>>> far but I am running into an issue with using PetscSF routines. >>>>>>>>>>>>> Some like >>>>>>>>>>>>> the PetscSFGetGraph work no problem but a few of my routines >>>>>>>>>>>>> require the >>>>>>>>>>>>> use of PetscSFGetLeafRanks and PetscSFGetRootRanks and those >>>>>>>>>>>>> don't seem to >>>>>>>>>>>>> be in the fortran interface and I just get a linking error. I >>>>>>>>>>>>> also don't >>>>>>>>>>>>> seem to see a PetscSF file in the finclude. Any clarification or >>>>>>>>>>>>> assistance >>>>>>>>>>>>> would be appreciated. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Sincerely >>>>>>>>>>>>> Nicholas >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Nicholas Arnold-Medabalimi >>>>>>>>>>>>> >>>>>>>>>>>>> Ph.D. Candidate >>>>>>>>>>>>> Computational Aeroscience Lab >>>>>>>>>>>>> University of Michigan >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Nicholas Arnold-Medabalimi >>>>>>>>>>> >>>>>>>>>>> Ph.D. Candidate >>>>>>>>>>> Computational Aeroscience Lab >>>>>>>>>>> University of Michigan >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Nicholas Arnold-Medabalimi >>>>>>>>> >>>>>>>>> Ph.D. Candidate >>>>>>>>> Computational Aeroscience Lab >>>>>>>>> University of Michigan >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Nicholas Arnold-Medabalimi >>>>>>> >>>>>>> Ph.D. Candidate >>>>>>> Computational Aeroscience Lab >>>>>>> University of Michigan >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Nicholas Arnold-Medabalimi >>>>>> >>>>>> Ph.D. Candidate >>>>>> Computational Aeroscience Lab >>>>>> University of Michigan >>>>>> >>>>> >>>> >>>> -- >>>> Nicholas Arnold-Medabalimi >>>> >>>> Ph.D. Candidate >>>> Computational Aeroscience Lab >>>> University of Michigan >>>> >>> >> >> -- >> Nicholas Arnold-Medabalimi >> >> Ph.D. Candidate >> Computational Aeroscience Lab >> University of Michigan >> > -- Nicholas Arnold-Medabalimi Ph.D. Candidate Computational Aeroscience Lab University of Michigan
#include <petsc/private/f90impl.h>
#include <petsc/private/sfimpl.h>
#if defined(PETSC_HAVE_FORTRAN_CAPS)
#define petscsfview_ PETSCSFVIEW
#define petscsfgetgraph_ PETSCSFGETGRAPH
#define petscsfbcastbegin_ PETSCSFBCASTBEGIN
#define petscsfbcastend_ PETSCSFBCASTEND
#define petscsfreducebegin_ PETSCSFREDUCEBEGIN
#define petscsfreduceend_ PETSCSFREDUCEEND
#define f90arraysfnodecreate_ F90ARRAYSFNODECREATE
#define petscsfviewfromoptions_ PETSCSFVIEWFROMOPTIONS
#define petscsfdestroy_ PETSCSFDESTROY
#define petscsfsetgraph_ PETSCSFSETGRAPH
#define petscsfgetleafranks_ PETSCSFGETLEAFRANKS
#define petscsfgetrootranks_ PETSCSFGETROOTRANKS
#elif !defined(PETSC_HAVE_FORTRAN_UNDERSCORE)
#define petscsfgetgraph_ petscsfgetgraph
#define petscsfview_ petscsfview
#define petscsfbcastbegin_ petscsfbcastbegin
#define petscsfbcastend_ petscsfbcastend
#define petscsfreducebegin_ petscsfreducebegin
#define petscsfreduceend_ petscsfreduceend
#define f90arraysfnodecreate_ f90arraysfnodecreate
#define petscsfviewfromoptions_ petscsfviewfromoptions
#define petscsfdestroy_ petscsfdestroy
#define petscsfsetgraph_ petscsfsetgraph
#define petscsfgetleafranks_ petscsfgetleafranks
#define petscsfgetrootranks_ petscsfgetrootranks
#endif
PETSC_EXTERN void f90arraysfnodecreate_(const PetscInt *,PetscInt *,void *
PETSC_F90_2PTR_PROTO_NOVAR);
PETSC_EXTERN void petscsfsetgraph_(PetscSF *sf,PetscInt *nroots,PetscInt
*nleaves, PetscInt *ilocal,PetscCopyMode *localmode, PetscSFNode
*iremote,PetscCopyMode *remotemode, int *ierr)
{
if (ilocal == PETSC_NULL_INTEGER_Fortran) ilocal = NULL;
*ierr =
PetscSFSetGraph(*sf,*nroots,*nleaves,ilocal,*localmode,iremote,*remotemode);
}
PETSC_EXTERN void petscsfview_(PetscSF *sf, PetscViewer *vin, PetscErrorCode
*ierr)
{
PetscViewer v;
PetscPatchDefaultViewers_Fortran(vin, v);
*ierr = PetscSFView(*sf, v);
}
PETSC_EXTERN void petscsfgetgraph_(PetscSF *sf,PetscInt *nroots,PetscInt
*nleaves, F90Array1d *ailocal, F90Array1d *airemote, PetscErrorCode *ierr
PETSC_F90_2PTR_PROTO(pilocal) PETSC_F90_2PTR_PROTO(piremote))
{
const PetscInt *ilocal;
const PetscSFNode *iremote;
PetscInt nl;
*ierr = PetscSFGetGraph(*sf,nroots,nleaves,&ilocal,&iremote);if (*ierr)
return;
nl = *nleaves;
if (!ilocal) nl = 0;
*ierr = F90Array1dCreate((void*)ilocal,MPIU_INT,1,nl, ailocal
PETSC_F90_2PTR_PARAM(pilocal));
/* this creates a memory leak */
f90arraysfnodecreate_((PetscInt*)iremote,nleaves, airemote
PETSC_F90_2PTR_PARAM(piremote));
}
PETSC_EXTERN void petscsfgetleafranks_(PetscSF *sf, PetscInt *niranks,
F90Array1d *airanks, F90Array1d *aioffset, F90Array1d *airootloc,
PetscErrorCode *ierr PETSC_F90_2PTR_PROTO(piranks)
PETSC_F90_2PTR_PROTO(pioffset) PETSC_F90_2PTR_PROTO(pirootloc))
{
const PetscMPIInt *iranks;
const PetscInt *ioffset;
const PetscInt *irootloc;
*ierr = PetscSFGetLeafRanks(*sf, niranks, &iranks, &ioffset, &irootloc);if
(*ierr) return;
*ierr = F90Array1dCreate((void *)iranks, MPI_INT, 1, *niranks, airanks
PETSC_F90_2PTR_PARAM(piranks));if (*ierr) return;
*ierr = F90Array1dCreate((void*)ioffset, MPIU_INT, 1, *niranks+1, aioffset
PETSC_F90_2PTR_PARAM(pioffset));if (*ierr) return;
*ierr = F90Array1dCreate((void *)irootloc, MPIU_INT, 1, ioffset[*niranks],
airootloc PETSC_F90_2PTR_PARAM(pirootloc));if (*ierr) return;
}
PETSC_EXTERN void petscsfgetrootranks_(PetscSF *sf, PetscInt *nranks,
F90Array1d *aranks, F90Array1d *aroffset, F90Array1d *armine, F90Array1d
*arremote,
PetscErrorCode *ierr PETSC_F90_2PTR_PROTO(pranks)
PETSC_F90_2PTR_PROTO(proffset) PETSC_F90_2PTR_PROTO(prmine)
PETSC_F90_2PTR_PROTO(prremote))
{
const PetscMPIInt *ranks;
const PetscInt *roffset;
const PetscInt *rmine;
const PetscInt *rremote;
*ierr = PetscSFGetRootRanks(*sf, nranks, &ranks, &roffset, &rmine,
&rremote);if (*ierr) return;
*ierr = F90Array1dCreate((void*)ranks, MPI_INT, 1, *nranks, aranks
PETSC_F90_2PTR_PARAM(pranks));if (*ierr) return;
*ierr = F90Array1dCreate((void*)roffset, MPIU_INT, 1, *nranks+1, aroffset
PETSC_F90_2PTR_PARAM(proffset));if (*ierr) return;
*ierr = F90Array1dCreate((void*)rmine, MPIU_INT, 1, roffset[*nranks], armine
PETSC_F90_2PTR_PARAM(prmine));if (*ierr) return;
*ierr = F90Array1dCreate((void*)rremote, MPIU_INT, 1, roffset[*nranks],
arremote PETSC_F90_2PTR_PARAM(prremote));if (*ierr) return;
}
#if defined(PETSC_HAVE_F90_ASSUMED_TYPE_NOT_PTR)
PETSC_EXTERN void petscsfbcastbegin_(PetscSF *sf, MPI_Fint *unit, const void
*rptr, void *lptr, MPI_Fint *op, PetscErrorCode *ierr)
{
MPI_Datatype dtype;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = PetscSFBcastBegin(*sf, dtype, rptr, lptr, cop);
}
PETSC_EXTERN void petscsfbcastend_(PetscSF *sf, MPI_Fint *unit, const void
*rptr, void *lptr, MPI_Fint *op, PetscErrorCode *ierr)
{
MPI_Datatype dtype;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = PetscSFBcastEnd(*sf, dtype, rptr, lptr, cop);
}
PETSC_EXTERN void petscsfreducebegin_(PetscSF *sf, MPI_Fint *unit, const void
*lptr, void *rptr, MPI_Fint *op, PetscErrorCode *ierr)
{
MPI_Datatype dtype;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = PetscSFReduceBegin(*sf, dtype, lptr, rptr, cop);
}
PETSC_EXTERN void petscsfreduceend_(PetscSF *sf, MPI_Fint *unit, const void
*lptr, void *rptr, MPI_Fint *op, PetscErrorCode *ierr)
{
MPI_Datatype dtype;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = PetscSFReduceEnd(*sf, dtype, lptr, rptr, cop);
}
#else
PETSC_EXTERN void petscsfbcastbegin_(PetscSF *sf, MPI_Fint *unit,F90Array1d
*rptr, F90Array1d *lptr, MPI_Fint *op, PetscErrorCode *ierr
PETSC_F90_2PTR_PROTO(rptrd) PETSC_F90_2PTR_PROTO(lptrd))
{
MPI_Datatype dtype;
const void *rootdata;
void *leafdata;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = F90Array1dAccess(rptr, dtype, (void**) &rootdata
PETSC_F90_2PTR_PARAM(rptrd));if (*ierr) return;
*ierr = F90Array1dAccess(lptr, dtype, (void**) &leafdata
PETSC_F90_2PTR_PARAM(lptrd));if (*ierr) return;
*ierr = PetscSFBcastBegin(*sf, dtype, rootdata, leafdata, cop);
}
PETSC_EXTERN void petscsfbcastend_(PetscSF *sf, MPI_Fint *unit,F90Array1d
*rptr, F90Array1d *lptr, MPI_Fint *op, PetscErrorCode *ierr
PETSC_F90_2PTR_PROTO(rptrd) PETSC_F90_2PTR_PROTO(lptrd))
{
MPI_Datatype dtype;
const void *rootdata;
void *leafdata;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = F90Array1dAccess(rptr, dtype, (void**) &rootdata
PETSC_F90_2PTR_PARAM(rptrd));if (*ierr) return;
*ierr = F90Array1dAccess(lptr, dtype, (void**) &leafdata
PETSC_F90_2PTR_PARAM(lptrd));if (*ierr) return;
*ierr = PetscSFBcastEnd(*sf, dtype, rootdata, leafdata, cop);
}
PETSC_EXTERN void petscsfreducebegin_(PetscSF *sf, MPI_Fint *unit,F90Array1d
*lptr, F90Array1d *rptr, MPI_Fint *op, PetscErrorCode *ierr
PETSC_F90_2PTR_PROTO(lptrd) PETSC_F90_2PTR_PROTO(rptrd))
{
MPI_Datatype dtype;
const void *rootdata;
void *leafdata;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = F90Array1dAccess(rptr, dtype, (void**) &rootdata
PETSC_F90_2PTR_PARAM(rptrd));if (*ierr) return;
*ierr = F90Array1dAccess(lptr, dtype, (void**) &leafdata
PETSC_F90_2PTR_PARAM(lptrd));if (*ierr) return;
*ierr = PetscSFReduceBegin(*sf, dtype, rootdata, leafdata, cop);
}
PETSC_EXTERN void petscsfreduceend_(PetscSF *sf, MPI_Fint *unit,F90Array1d
*lptr, F90Array1d *rptr, MPI_Fint *op, PetscErrorCode *ierr
PETSC_F90_2PTR_PROTO(lptrd) PETSC_F90_2PTR_PROTO(rptrd))
{
MPI_Datatype dtype;
const void *rootdata;
void *leafdata;
MPI_Op cop = MPI_Op_f2c(*op);
*ierr = PetscMPIFortranDatatypeToC(*unit,&dtype);if (*ierr) return;
*ierr = F90Array1dAccess(rptr, dtype, (void**) &rootdata
PETSC_F90_2PTR_PARAM(rptrd));if (*ierr) return;
*ierr = F90Array1dAccess(lptr, dtype, (void**) &leafdata
PETSC_F90_2PTR_PARAM(lptrd));if (*ierr) return;
*ierr = PetscSFReduceEnd(*sf, dtype, rootdata, leafdata, cop);
}
PETSC_EXTERN void petscsfviewfromoptions_(PetscSF *ao,PetscObject obj,char*
type,PetscErrorCode *ierr,PETSC_FORTRAN_CHARLEN_T len)
{
char *t;
FIXCHAR(type,len,t);
CHKFORTRANNULLOBJECT(obj);
*ierr = PetscSFViewFromOptions(*ao,obj,t);if (*ierr) return;
FREECHAR(type,t);
}
PETSC_EXTERN void petscsfdestroy_(PetscSF *x,int *ierr)
{
PETSC_FORTRAN_OBJECT_F_DESTROYED_TO_C_NULL(x);
*ierr = PetscSFDestroy(x); if (*ierr) return;
PETSC_FORTRAN_OBJECT_C_NULL_TO_F_DESTROYED(x);
}
#endif
petscvec.h90
Description: Binary data
