I think this requires the political decision to be made on the order of the finalization and rules to enforce the order and timings. In a distributed DOE world this kind of political decision is tough.
I think a system more like just-in-time-packaging has to emerge to give more flexibility for the free-wheeling open source work. I have no idea how one could achieve this. Some kind of "smart" versioning that doe not require one to explicitly manage a bunch of "if package version is x do y else if version is z do w." > On Apr 3, 2022, at 1:29 PM, Satish Balay <ba...@mcs.anl.gov> wrote: > > there is certainly frustration with changes. > > And then there could be real issues. If similar major changes land in sept > release [at the last minite] in any critical packages [that others packages > don't quickly add it to their own sept release ] - that might break things in > a way that xsdk release could not be delivered. > > [and usually triggers hard to resolve discussion of who should address this > failure - that changed package - or dependent packages. And even if patches > are available - they might not get in - due to workflow isues and such]. > > Satish > > On Sun, 3 Apr 2022, Barry Smith wrote: > >> >> We have not updated Sundials because they developed an entirely new code >> with new APIs, it is essentially a new package with tons of new >> functionality. Had they been incrementally changing things over the years we >> would have actually kept up with it; so this is not a good example of how >> small API changes keep us from upgrading, not at all. It is just an example >> of how it takes a lot of work to wrap a new large package like Sundials 3 >> from scratch and someone must make a big effort to do it. Note: I think >> Sundials was right to do a rewrite, their classic design was preventing them >> from making dramatic additions to the old code by doing a complete rewrite >> they could accomplish so much more. >> >> MOAB has not been updated presumably because there are no or very >> unaggressive users. >> >> Side note: I understand the frustration and grumbling that takes place when >> one has to deal with change, especially when from a perspective as an >> outer-sider to a project the change may seem unnecessary, that frustration >> and grumbling is normal, I do it all the time. But it should not dictate >> policy. >> >> >> >> >> >>> On Apr 3, 2022, at 12:58 PM, Satish Balay <ba...@mcs.anl.gov> wrote: >>> >>> On Sun, 3 Apr 2022, Barry Smith wrote: >>> >>>> >>>> >>>>> On Apr 3, 2022, at 12:24 PM, Satish Balay <ba...@mcs.anl.gov> wrote: >>>>> >>>>>> If we had this attitude with the external packages PETSc uses we would >>>>>> have to stop using most of the packages/*.py. >>>>> >>>>> Sure one can take extreme view on both sides. [no change, vs won't >>>>> hesitate to change] - having a manageable (minimal) change is harder to >>>>> do. >>>>> >>>>> I would point out that most externalpackages don't change much and we >>>>> benefit from it - hence we are able to support so may. Some packages had >>>>> major changes - and we haven't upgraded to their new versions. >>>> >>>> What packages are these? We should have a tool that runs through all the >>>> packages/xxx.py and determines the date of release of the version we are >>>> using and if there are any newer versions available. We could run this >>>> tool automatically a month before each PETSc release sending its output to >>>> petsc-dev to see what we should be updating. >>> >>> sundials, moab [,trilinos - ml was only recently updated] - that I can >>> think off right now. >>> >>> Satish >>> >>>> >>>> Note also that some packages we don't update to, not because of API >>>> changes but because the new releases are broken in some way, this is life >>>> in the HPC world. >>>> >>>> >>>> >>>> >>>>> >>>>> [i.e with the current state one can use them only if they completely buy >>>>> into petsc ecosystem i.e use old version - but not any larger one - as in >>>>> use newer features from them] >>>>> >>>>> We did update to newer interfaces in some packages. >>>>> >>>>> But these problems remain - and have to be dealt with - and sometimes the >>>>> complexity increases based on the dependency tree. >>>>> >>>>> [and also results in folk using and requiring help with older petsc >>>>> versions] >>>>> >>>>> Satish >>>>> >>>>> >>>>> On Sun, 3 Apr 2022, Barry Smith wrote: >>>>> >>>>>> >>>>>> I would say it is not reasonable for the package developers in the xsdk >>>>>> ecosystem to expect that they can just continue to use another HPC >>>>>> package for multiple years without doing some minimal amount of work to >>>>>> keep up with the other packages' new releases. If we had this attitude >>>>>> with the external packages PETSc uses we would have to stop using most >>>>>> of the packages/*.py. Yes, it is a constant race to keep up the versions >>>>>> in packages/*.py and requires some effort but if you want to play in >>>>>> this game that is a race you have to remain in. And it goes way beyond >>>>>> HPC, to say you do software development but don't need to manage >>>>>> constant change in everything is an oxymoron. There was never a golden >>>>>> age of computing where things didn't change rapidly, pretending there >>>>>> was or can be is not productive. Of course, we want to minimize public >>>>>> change, but having a goal of no public change is not a realistic or even >>>>>> desirable goal. >>>>>> >>>>>>> Just noticed - CHKERRQ() got removed from fortran interface - breaking >>>>>>> pflotran >>>>>> >>>>>> This was just a oversight, easily fixed. >>>>>> >>>>>>> On Apr 3, 2022, at 11:13 AM, Satish Balay <ba...@mcs.anl.gov> wrote: >>>>>>> >>>>>>> >>>>>>> Note this is not just 'users should update their code' issue. >>>>>>> - all packages (that use petsc) would need to do this update >>>>>>> - and this update doesn't always happen - so pakages will stay at old >>>>>>> release - some might not >>>>>>> - so now we cant build PETSc with both these packages together. >>>>>>> >>>>>>> this type of change causes major issues in xsdk ecosystem (depends on >>>>>>> how many direct/indirect dependencies are on the given package) >>>>>>> >>>>>>> Just noticed - CHKERRQ() got removed from fortran interface - breaking >>>>>>> pflotran >>>>>>> >>>>>>> https://gitlab.com/xsdk-project/spack-xsdk/-/jobs/2285145624 >>>>>>> >>>>>>> [also CHKERRABORT]. Perhaps they can be added back in. >>>>>>> >>>>>>> $ git diff release-3.16..release include/petsc/finclude/petscsys.h >>>>>>> diff --git a/include/petsc/finclude/petscsys.h >>>>>>> b/include/petsc/finclude/petscsys.h >>>>>>> <snip> >>>>>>> #define SETERRABORT(c,ierr,s) call PetscError(c,ierr,0,s); call >>>>>>> MPI_Abort(c,ierr) >>>>>>> -#define CHKERRQ(ierr) if (ierr .ne. 0) then;call >>>>>>> PetscErrorF(ierr);return;endif >>>>>>> +#define PetscCall(ierr) if (ierr .ne. 0) then;call >>>>>>> PetscErrorF(ierr);return;endif >>>>>>> #define CHKERRA(ierr) if (ierr .ne. 0) then;call PetscErrorF(ierr);call >>>>>>> MPIU_Abort(PETSC_COMM_SELF,ierr);endif >>>>>>> -#define CHKERRABORT(c,ierr) if (ierr .ne. 0) then;call >>>>>>> PetscErrorF(ierr);call MPI_Abort(c,ierr);endif >>>>>>> +#define PetscCallAbort(c,ierr) if (ierr .ne. 0) then;call >>>>>>> PetscErrorF(ierr);call MPI_Abort(c,ierr);endif >>>>>>> #define CHKMEMQ call chkmemfortran(__LINE__,__FILE__,ierr) >>>>>>> >>>>>>> Satish >>>>>>> >>>>>>> On Sun, 3 Apr 2022, Barry Smith wrote: >>>>>>> >>>>>>>> >>>>>>>> To use the latest version of PETSc, each user needs to remove the >>>>>>>> error checks on these calls. The resulting code will work with >>>>>>>> previous versions of PETSc as well as the current version of PETSc. >>>>>>>> PETSc has never promised complete backward compatibility in the sense >>>>>>>> of promising that one can use new PETSc releases without any changes >>>>>>>> to their code; the documentation has always stated new releases will >>>>>>>> contain changes in the API. We began using depreciate a few years ago >>>>>>>> to limit the number of changes that needed to be made immediately for >>>>>>>> each release but depreciate is not suitable for all changes and so >>>>>>>> users do need to make some changes for each new release. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Apr 3, 2022, at 7:23 AM, Lisandro Dalcin <dalc...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> The recent PetscUse/TryMethod changes are backward incompatible. >>>>>>>>> Third-party codes cannot compile without modification. Our users >>>>>>>>> deserve better. >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Lisandro Dalcin >>>>>>>>> ============ >>>>>>>>> Senior Research Scientist >>>>>>>>> Extreme Computing Research Center (ECRC) >>>>>>>>> King Abdullah University of Science and Technology (KAUST) >>>>>>>>> http://ecrc.kaust.edu.sa/ <http://ecrc.kaust.edu.sa/> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
