On the hypre versioning - aha. For this project I locked the petsc version a little while ago (3.19.1), but I've been using a fresh clone of hypre, so clearly it's too modern a version. Using the appropriate version of hypre (2.28.0, according to hypre.py) might fix some things. I may have other problems in the form of the confilicting compiler options as Satish mentioned, which I'll have to figure out too.
Thanks, Daniel On Fri, Jul 21, 2023 at 9:32 PM Barry Smith <bsm...@petsc.dev> wrote: > > > hypre_Error was changed from an integer to a struct in hypre. PETSc code > was changed in main to work with that struct and not work if hypre_Error is > an int. This means main cannot work with previous hypre versions (where > hypre_Error is an int). Sure, instead of changing the minimum version one > could potentially change PETSc's main version source code to work with both > old and new hypre, but that would need to be done and has not been done. > main is simply broken, it allows configure to succeed with an older version > of hypre but PETSc main cannot compile with this, thus confusing developers > and users. This does need to be fixed, one way or another. > > > > > On Jul 21, 2023, at 1:32 PM, Satish Balay <ba...@mcs.anl.gov> wrote: > > > > We don't have a good way to verify if the changes continue to work > > with the minver of the pkg [and if minor fixes can get this working - > > without increasing this requirement - sure without access to the new > > features provided by the new version] > > > > Having a single version dependency between pkgs makes the whole > > ecosystem [where many pkgs with all their dependencies mixed in] > > brittle.. > > > > Satish > > > > On Thu, 20 Jul 2023, Barry Smith wrote: > > > >> > >> Satish, > >> > >> I think hypre.py in main needs minversion = 2.29 instead of 2.14 > >> > >> > >> > >>> On Jul 20, 2023, at 11:05 AM, Satish Balay <ba...@mcs.anl.gov> wrote: > >>> > >>> Can check config/BuildSystem/config/packages/hypre.py > >>> > >>> petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc > 'main' branch with 2.29.0 > >>> > >>> Satish > >>> > >>> On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote: > >>> > >>>> > >>>> You cannot use this version of PETSc, 3.19, with the version of hypre > you installed. In hypre they recently changed hypre_Error from an integer > to a struct which completely breaks compatibility with previous versions of > hypre (and hence previous versions of PETSc). You must use the main git > branch of PETSc with the version of hypre you installed. > >>>> > >>>> Barry > >>>> > >>>> > >>>>> On Jul 20, 2023, at 5:10 AM, Daniel Stone < > daniel.st...@opengosim.com> wrote: > >>>>> > >>>>> Hi All, > >>>>> > >>>>> Many thanks for the detailed explainations and ideas! > >>>>> > >>>>> I tried skipping the test. When it came time to do the build itself > (make $PETSC_DIR... all) I get some failures, unsurprisingly: > >>>>> > >>>>> -------------------------------- > >>>>> > >>>>> FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o > >>>>> CC > arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o > >>>>> CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o > >>>>> CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o > >>>>> > C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): > error: assigning to 'hypre_Error' from incompatible type 'int' > >>>>> hypre__global_error = 0; > >>>>> ^ ~ > >>>>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): > note: expanded from macro 'PetscStackCallExternalVoid' > >>>>> __VA_ARGS__; \ > >>>>> ^~~~~~~~~~~ > >>>>> > C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): > error: assigning to 'hypre_Error' from incompatible type 'int' > >>>>> hypre__global_error = 0; > >>>>> ^ ~ > >>>>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): > note: expanded from macro 'PetscStackCallExternalVoid' > >>>>> __VA_ARGS__; \ > >>>>> ^~~~~~~~~~~ > >>>>> 2 errors generated. > >>>>> make[3]: *** [gmakefile:195: > arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 > >>>>> make[3]: *** Waiting for unfinished jobs.... > >>>>> FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o > >>>>> CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o > >>>>> CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o > >>>>> make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' > >>>>> make[2]: *** > [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 > >>>>> make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' > >>>>> **************************ERROR************************************* > >>>>> Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log > >>>>> Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to > petsc-ma...@mcs.anl.gov <mailto:petsc-ma...@mcs.anl.gov> > >>>>> ******************************************************************** > >>>>> Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100 > >>>>> > >>>>> ---------------------------------------- > >>>>> > >>>>> But wait - isn't this the compile stage, not the linking stage? This > seems to imply that I've made a hash of providing include file such that a > definition of "hypre_Error" > >>>>> cannot be seen - unless I'm misinterpreting. Interesting note about > Hypre and include files - if built using configure and make, all the > include files are conviniently copied > >>>>> into hypre/src/hypre/include. This is not done for a cmake build - I > had to do the copying myself. Maybe I missed one. > >>>>> > >>>>> > >>>>> On shared vs. static - if there a clear way of telling which I've > ended up with? I've checked the cmakelists for hypre and this seems to > imply that not-shared is the default, > >>>>> which I didn't change: > >>>>> > >>>>> # Configuration options > >>>>> option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) > >>>>> option(HYPRE_ENABLE_BIGINT "Use long long int for > HYPRE_Int" OFF) > >>>>> option(HYPRE_ENABLE_MIXEDINT "Use long long int for > HYPRE_BigInt, int for HYPRE_INT" OFF) > >>>>> [....] > >>>>> > >>>>> > >>>>> checking again, I've noticed that the way that the stub-test fails > is different depending on whether it's called from the config script or > used in isolation - more details on that soon. > >>>>> > >>>>> > >>>>> > >>>>> Thanks again, > >>>>> > >>>>> Daniel > >>>>> > >>>>> > >>>>> > >>>>> On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < > petsc-users@mcs.anl.gov <mailto:petsc-users@mcs.anl.gov>> wrote: > >>>>>> I think it should work with static libraries and 64bit compilers. > >>>>>> > >>>>>> That's how I think --download-f2cblaslapack [etc] work. > >>>>>> > >>>>>> Also it works with MS-MPI - even-though its a dll install, the > library stub provides this symbol somehow.. > >>>>>> > >>>>>> balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft > SDKs/MPI/Lib/x64 > >>>>>> $ nm -Ao msmpi.lib |grep " MPI_Init" > >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init > >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread > >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized > >>>>>> > >>>>>> However - if the library symbol is somehow mangled - this configure > mode of checking library functions will fail. > >>>>>> > >>>>>> Checking PETSc dll build: > >>>>>> > >>>>>> balay@ps5 ~/petsc/arch-ci-mswin-uni/lib > >>>>>> $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays > >>>>>> libpetsc.lib:libpetsc.dll:0000000000000000 I > __imp_MatCreateSeqAIJWithArrays > >>>>>> libpetsc.lib:libpetsc.dll:0000000000000000 T > MatCreateSeqAIJWithArrays > >>>>>> > >>>>>> It also has the unmangled symbol - so I guess this mode can work > generally with dlls. > >>>>>> > >>>>>> Satish > >>>>>> > >>>>>> > >>>>>> On Wed, 19 Jul 2023, Barry Smith wrote: > >>>>>> > >>>>>>> > >>>>>>> Satish, > >>>>>>> > >>>>>>> So it will always fail on Windows with Windows compilers (both > with static and shared libraries)? Is this true for all PETSc external > packages? If so, why does the installation documentation say that some > external packages can work with Windows compilers? (Presumably PETSc cannot > since the configure tests will fail). > >>>>>>> > >>>>>>> Barry > >>>>>>> > >>>>>>> > >>>>>>>> On Jul 19, 2023, at 11:40 AM, Satish Balay <ba...@mcs.anl.gov > <mailto:ba...@mcs.anl.gov>> wrote: > >>>>>>>> > >>>>>>>> BTW: Some explanation of configure: > >>>>>>>> > >>>>>>>> It attempts the following on linux: > >>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>> Source: > >>>>>>>> #include "confdefs.h" > >>>>>>>> #include "conffix.h" > >>>>>>>> /* Override any gcc2 internal prototype to avoid an error. */ > >>>>>>>> char HYPRE_IJMatrixCreate(); > >>>>>>>> static void _check_HYPRE_IJMatrixCreate() { > HYPRE_IJMatrixCreate(); } > >>>>>>>> > >>>>>>>> int main(void) { > >>>>>>>> _check_HYPRE_IJMatrixCreate(); > >>>>>>>> return 0; > >>>>>>>> } > >>>>>>>> <<<<<<< > >>>>>>>> > >>>>>>>> Note - it does not include 'HYPRE.h' here - but redefines the > prototype as 'char HYPRE_IJMatrixCreate(); > >>>>>>>> > >>>>>>>> Compiling it manually: > >>>>>>>> > >>>>>>>>>>>> > >>>>>>>> [balay@pj01 petsc]$ cat conftest.c > >>>>>>>> char HYPRE_IJMatrixCreate(); > >>>>>>>> static void _check_HYPRE_IJMatrixCreate() { > HYPRE_IJMatrixCreate(); } > >>>>>>>> > >>>>>>>> int main(void) { > >>>>>>>> _check_HYPRE_IJMatrixCreate(); > >>>>>>>> return 0; > >>>>>>>> } > >>>>>>>> [balay@pj01 petsc]$ gcc -c conftest.c > >>>>>>>> [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate > >>>>>>>> conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate > >>>>>>>> conftest.o: U HYPRE_IJMatrixCreate > >>>>>>>> [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so > |grep HYPRE_IJMatrixCreate > >>>>>>>> arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T > HYPRE_IJMatrixCreate > >>>>>>>> [balay@pj01 petsc]$ > >>>>>>>> <<<< > >>>>>>>> > >>>>>>>> Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T > HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure > succeeds! > >>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>> [balay@pj01 petsc]$ gcc -o conftest conftest.o > arch-linux-c-debug/lib/libHYPRE.so > >>>>>>>> [balay@pj01 petsc]$ echo $? > >>>>>>>> 0 > >>>>>>>> <<<<< > >>>>>>>> > >>>>>>>> On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) > etc..] - this might not match - resulting in link failures. > >>>>>>>> > >>>>>>>> Satish > >>>>>>>> > >>>>>>>> > >>>>>>>> On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote: > >>>>>>>> > >>>>>>>>> You could try skipping this test [and assume > --with-hypre-include and --with-hypre-lib options are correct] - and see if > this works. > >>>>>>>>> > >>>>>>>>> diff --git a/config/BuildSystem/config/packages/hypre.py > b/config/BuildSystem/config/packages/hypre.py > >>>>>>>>> index 5bc88322aa2..2d6c7932e17 100644 > >>>>>>>>> --- a/config/BuildSystem/config/packages/hypre.py > >>>>>>>>> +++ b/config/BuildSystem/config/packages/hypre.py > >>>>>>>>> @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): > >>>>>>>>> self.requiresversion = 1 > >>>>>>>>> self.gitcommit = 'v'+self.version > >>>>>>>>> self.download = ['git:// > https://github.com/hypre-space/hypre',' > https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] > >>>>>>>>> - self.functions = ['HYPRE_IJMatrixCreate'] > >>>>>>>>> + self.functions = [] > >>>>>>>>> self.includes = ['HYPRE.h'] > >>>>>>>>> self.liblist = [['libHYPRE.a']] > >>>>>>>>> self.buildLanguages = ['C','Cxx'] > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> Satish > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Wed, 19 Jul 2023, Barry Smith wrote: > >>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> You don't indicate what type of libraries you built hypre with; > static or shared. My guess is you ended up with shared > >>>>>>>>>> > >>>>>>>>>> I think the answer to your difficulty is hidden in __cdecl > (Satish will know much better than me). When you are looking for symbols in > Windows shared libraries you have to prepend something to the function > prototype to have it successfully found. For example the PETSc include > files have these things __declspec(dllimport) The configure test fails > because it does not provide the needed prototype. Likely you built PTScotch > with static libraries so no problem. > >>>>>>>>>> > >>>>>>>>>> The simplest fix would be to build static hypre libraries. I > think it is a major project to get PETSc configure and macro system to work > properly with external packages that are in Windows shared libraries since > more use of __declspec would be needed. > >>>>>>>>>> > >>>>>>>>>> Barry > >>>>>>>>>> > >>>>>>>>>> The PETSc installation instructions should probably say > something about external packages with Windows shared libraries. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> On Jul 19, 2023, at 10:52 AM, Daniel Stone < > daniel.st...@opengosim.com <mailto:daniel.st...@opengosim.com>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> Hello, > >>>>>>>>>>> > >>>>>>>>>>> I'm working on getting a petsc build running on windows. One > necessary package to include is Hypre. I've been able to build Hypre > seperately using cmake, and confirmed that the library works > >>>>>>>>>>> by setting up a VS project to run some of the example programs. > >>>>>>>>>>> > >>>>>>>>>>> My attempted petsc build is being done through cygwin. I've > been able to (with varying degrees of difficulty), build a fairly plain > petsc, and one that downloads and builds ptscotch (after some modifications > >>>>>>>>>>> to both ptscotch and the config script). I am now attempting > to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). > Note that the same compilers are being used for both Hypre and for petsc > >>>>>>>>>>> through cygwin - the new intel oneapi compilers (icx and ifx, > after again varying amounts of pain to work around their awkwardness with > the config script). > >>>>>>>>>>> > >>>>>>>>>>> I'm seeing a problem when the config script does some tests on > the included hypre lib. The source code looks like: > >>>>>>>>>>> > >>>>>>>>>>> #include "confdefs.h" > >>>>>>>>>>> #include "conffix.h" > >>>>>>>>>>> /* Override any gcc2 internal prototype to avoid an error. */ > >>>>>>>>>>> > >>>>>>>>>>> #include "HYPRE.h" > >>>>>>>>>>> > >>>>>>>>>>> char HYPRE_IJMatrixCreate(); > >>>>>>>>>>> static void _check_HYPRE_IJMatrixCreate() { > HYPRE_IJMatrixCreate(); } > >>>>>>>>>>> > >>>>>>>>>>> int main() { > >>>>>>>>>>> _check_HYPRE_IJMatrixCreate();; > >>>>>>>>>>> return 0; > >>>>>>>>>>> } > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> As I understand this is a fairly standard type of stub program > used by the config script to check that it is able to link to certain > symbols in given libraries. Tests like this have succeeded in my builds that > >>>>>>>>>>> include PTScotch. > >>>>>>>>>>> > >>>>>>>>>>> I keep getting a linker error with the above test, including > if I seperate it out and try to build it seperately: > >>>>>>>>>>> > >>>>>>>>>>> unresolved external symbol "char __cdel > HYPRE_IJMatrixCreate(void)" .... > >>>>>>>>>>> > >>>>>>>>>>> Ok, it looks like a problem with either the library or linker > commands. But here's the interesting thing - If I transplant this code into > VS, with the same project setting that allows it to build the much more > >>>>>>>>>>> nontrivial Hypre example programs, I get the same error: > >>>>>>>>>>> > >>>>>>>>>>> Error LNK2001 unresolved external symbol "char __cdecl > HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 > C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1 > >>>>>>>>>>> > >>>>>>>>>>> So it seems like there might be something about this type of > stub program that is not working with my Hypre library. I don't fully > understand this program - it's able to call the function with no arguments, > but > >>>>>>>>>>> it also needs to be linked against a library containing the > function, apparently by wrapping it in a static void function? Not > something I've seen before. > >>>>>>>>>>> > >>>>>>>>>>> Does anyone have any insight into what might be going wrong - > or really just any explaination of how the stub program works so I can > figure out why it isn't in this case? > >>>>>>>>>>> > >>>>>>>>>>> Many thanks, > >>>>>>>>>>> > >>>>>>>>>>> Daniel > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>> > >>>> > >> > >