[petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Hi, I am testing intel 16 compilers under mac OS, and notice that options handling for all past and current versions of petsc is broken in fortran (options are not parsed). I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is used (but taking the right value using intel 15 or gcc 5), but I can’t find my way through the multiple macros to figure out what is really the matter… The problem does not seem to arise under linux with the same compilers. Any idea of what is going on? Any test I can run to help (I assume that you don’t have a test box with the latest intel compilers)? Is it a petsc bug or an intel bug? I use an up to date pets-dev master, and the configure.log file is available at https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 Thanks, Blaise -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Do you have configure.log fr intel-15 compiler - on the same machine? It would be good to compare petscconf.h between intel-15 and intel-16. Satish On Wed, 16 Sep 2015, Blaise A Bourdin wrote: > Hi, > > I am testing intel 16 compilers under mac OS, and notice that options > handling for all past and current versions of petsc is broken in fortran > (options are not parsed). > I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is > used (but taking the right value using intel 15 or gcc 5), but I can’t find > my way through the multiple macros to figure out what is really the matter… > The problem does not seem to arise under linux with the same compilers. > > Any idea of what is going on? Any test I can run to help (I assume that you > don’t have a test box with the latest intel compilers)? Is it a petsc bug or > an intel bug? > > I use an up to date pets-dev master, and the configure.log file is available > at https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 > > Thanks, > > Blaise > >
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Hi Satish, It is at https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/38806CDddLJ > On Sep 16, 2015, at 10:34 AM, Satish Balay wrote: > > Do you have configure.log fr intel-15 compiler - on the same machine? > > It would be good to compare petscconf.h between intel-15 and intel-16. bourdin@galerkin:petsc-dev (master)$ diff Darwin-intel15.0-g//include/petscconf.h Darwin-intel16.0-g//include/petscconf.h 85c85 < #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel15.0-g/lib" --- > #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib" 489c489 < #define PETSC_ARCH "Darwin-intel15.0-g" --- > #define PETSC_ARCH "Darwin-intel16.0-g” Not very exciting! I am attaching the one for intel16 petscconf.h Description: petscconf.h Regards, Blaise > > Satish > > On Wed, 16 Sep 2015, Blaise A Bourdin wrote: > >> Hi, >> >> I am testing intel 16 compilers under mac OS, and notice that options >> handling for all past and current versions of petsc is broken in fortran >> (options are not parsed). >> I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is >> used (but taking the right value using intel 15 or gcc 5), but I canât >> find my way through the multiple macros to figure out what is really the >> matter⦠>> The problem does not seem to arise under linux with the same compilers. >> >> Any idea of what is going on? Any test I can run to help (I assume that you >> donât have a test box with the latest intel compilers)? Is it a petsc bug >> or an intel bug? >> >> I use an up to date pets-dev master, and the configure.log file is available >> at https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 >> >> Thanks, >> >> Blaise >> >> -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Well - the petscconf.h is identicall for both builds. So the change in behavior must be due to some internal changes in ifort on Mac [as you say - it works fine on linux] Can you try using the branch 'balay/ftn-command_argument' - and see if that works for you? Satish On Wed, 16 Sep 2015, Blaise A Bourdin wrote: > Hi Satish, > > It is at > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/38806CDddLJ > > > On Sep 16, 2015, at 10:34 AM, Satish Balay wrote: > > > > Do you have configure.log fr intel-15 compiler - on the same machine? > > > > It would be good to compare petscconf.h between intel-15 and intel-16. > > bourdin@galerkin:petsc-dev (master)$ diff > Darwin-intel15.0-g//include/petscconf.h > Darwin-intel16.0-g//include/petscconf.h > 85c85 > < #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel15.0-g/lib" > --- > > #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib" > 489c489 > < #define PETSC_ARCH "Darwin-intel15.0-g" > --- > > #define PETSC_ARCH "Darwin-intel16.0-g” > Not very exciting! > > I am attaching the one for intel16 > >
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Please report this via premier.intel.com. Jeff On Wed, Sep 16, 2015 at 11:13 AM, Blaise A Bourdin wrote: > Hi, > > I am testing intel 16 compilers under mac OS, and notice that options > handling for all past and current versions of petsc is broken in fortran > (options are not parsed). > I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is > used (but taking the right value using intel 15 or gcc 5), but I can’t find > my way through the multiple macros to figure out what is really the matter… > The problem does not seem to arise under linux with the same compilers. > > Any idea of what is going on? Any test I can run to help (I assume that > you don’t have a test box with the latest intel compilers)? Is it a petsc > bug or an intel bug? > > I use an up to date pets-dev master, and the configure.log file is > available at > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 > > Thanks, > > Blaise > > -- > Department of Mathematics and Center for Computation & Technology > Louisiana State University, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > http://www.math.lsu.edu/~bourdin > > > > > > > > -- Jeff Hammond jeff.scie...@gmail.com http://jeffhammond.github.io/
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
This change is now merged master [Its setup as the prefered mode to get fortran command line arguments] Satish On Wed, 16 Sep 2015, Satish Balay wrote: > Well - the petscconf.h is identicall for both builds. So the change in > behavior must be due to some internal changes in ifort on Mac [as you > say - it works fine on linux] > > Can you try using the branch 'balay/ftn-command_argument' - and see if > that works for you? > > Satish > > On Wed, 16 Sep 2015, Blaise A Bourdin wrote: > > > Hi Satish, > > > > It is at > > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/38806CDddLJ > > > > > On Sep 16, 2015, at 10:34 AM, Satish Balay wrote: > > > > > > Do you have configure.log fr intel-15 compiler - on the same machine? > > > > > > It would be good to compare petscconf.h between intel-15 and intel-16. > > > > bourdin@galerkin:petsc-dev (master)$ diff > > Darwin-intel15.0-g//include/petscconf.h > > Darwin-intel16.0-g//include/petscconf.h > > 85c85 > > < #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel15.0-g/lib" > > --- > > > #define PETSC_LIB_DIR "/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib" > > 489c489 > > < #define PETSC_ARCH "Darwin-intel15.0-g" > > --- > > > #define PETSC_ARCH "Darwin-intel16.0-g” > > Not very exciting! > > > > I am attaching the one for intel16 > > > > >
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
[from the available logs] its not clear what the issue is. PETSc fortran interface attempts to use getarg_() and iargc_() [fortran library functions] - or its variants - directly from C. This usage is non-standard and messy [and some compiler folks - said - its unsupported usage]. Also - configure attempts to guess the fortran link libraries - and constructs a link command cCompilerLibs + cxxCompilerLibs + fortranCompilerLibs. Perhaps there is a bug in this detection code - thats triggering this error. Its verifyable by attempting to use: --with-clib-autodetect=0 --with-clib-autodetect=0 --with-cxxlib-autodetect=0 LIBS='liblist' where 'liblist' should be determined manually - for the linking of C/fortran/CXX to work correctly. My fix attempts to call command_argument_count(), get_command_argument() directly from fortran - and if that doesn't work - then its likely a compiler bug.. [which one can try to replicate with a simple test code] Satish On Thu, 17 Sep 2015, Jeff Hammond wrote: > Please report this via premier.intel.com. > > Jeff > > On Wed, Sep 16, 2015 at 11:13 AM, Blaise A Bourdin wrote: > > > Hi, > > > > I am testing intel 16 compilers under mac OS, and notice that options > > handling for all past and current versions of petsc is broken in fortran > > (options are not parsed). > > I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is > > used (but taking the right value using intel 15 or gcc 5), but I can’t find > > my way through the multiple macros to figure out what is really the matter… > > The problem does not seem to arise under linux with the same compilers. > > > > Any idea of what is going on? Any test I can run to help (I assume that > > you don’t have a test box with the latest intel compilers)? Is it a petsc > > bug or an intel bug? > > > > I use an up to date pets-dev master, and the configure.log file is > > available at > > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 > > > > Thanks, > > > > Blaise > > > > -- > > Department of Mathematics and Center for Computation & Technology > > Louisiana State University, Baton Rouge, LA 70803, USA > > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > > http://www.math.lsu.edu/~bourdin > > > > > > > > > > > > > > > > > > >
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Hi, I was traveling and did not have time to finish investigating, but there seems to be an issue with fortran get_command_argument and friends in mixed language mode. I will continue to investigate and will report back. Blaise > On Sep 19, 2015, at 10:18 AM, Satish Balay wrote: > > [from the available logs] its not clear what the issue is. > > PETSc fortran interface attempts to use getarg_() and iargc_() [fortran > library functions] - or its variants - directly from C. > > This usage is non-standard and messy [and some compiler folks - said - > its unsupported usage]. > > Also - configure attempts to guess the fortran link libraries - and > constructs a link command cCompilerLibs + cxxCompilerLibs + > fortranCompilerLibs. > > Perhaps there is a bug in this detection code - thats triggering this > error. Its verifyable by attempting to use: > > --with-clib-autodetect=0 --with-clib-autodetect=0 --with-cxxlib-autodetect=0 > LIBS='liblist' > > where 'liblist' should be determined manually - for the linking of > C/fortran/CXX to work correctly. > > My fix attempts to call command_argument_count(), > get_command_argument() directly from fortran - and if that doesn't > work - then its likely a compiler bug.. > > [which one can try to replicate with a simple test code] > > Satish > > On Thu, 17 Sep 2015, Jeff Hammond wrote: > >> Please report this via premier.intel.com. >> >> Jeff >> >> On Wed, Sep 16, 2015 at 11:13 AM, Blaise A Bourdin wrote: >> >>> Hi, >>> >>> I am testing intel 16 compilers under mac OS, and notice that options >>> handling for all past and current versions of petsc is broken in fortran >>> (options are not parsed). >>> I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is >>> used (but taking the right value using intel 15 or gcc 5), but I can’t find >>> my way through the multiple macros to figure out what is really the matter… >>> The problem does not seem to arise under linux with the same compilers. >>> >>> Any idea of what is going on? Any test I can run to help (I assume that >>> you don’t have a test box with the latest intel compilers)? Is it a petsc >>> bug or an intel bug? >>> >>> I use an up to date pets-dev master, and the configure.log file is >>> available at >>> https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 >>> >>> Thanks, >>> >>> Blaise >>> >>> -- >>> Department of Mathematics and Center for Computation & Technology >>> Louisiana State University, Baton Rouge, LA 70803, USA >>> Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 >>> http://www.math.lsu.edu/~bourdin >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Hm - I would suggest doing a minimal build build with: --with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0 LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a" And see if that makes a difference. Satish On Sat, 19 Sep 2015, Blaise A Bourdin wrote: > Hi, > > I was traveling and did not have time to finish investigating, but there > seems to be an issue with fortran get_command_argument and friends in mixed > language mode. I will continue to investigate and will report back. > > Blaise > > > On Sep 19, 2015, at 10:18 AM, Satish Balay wrote: > > > > [from the available logs] its not clear what the issue is. > > > > PETSc fortran interface attempts to use getarg_() and iargc_() [fortran > > library functions] - or its variants - directly from C. > > > > This usage is non-standard and messy [and some compiler folks - said - > > its unsupported usage]. > > > > Also - configure attempts to guess the fortran link libraries - and > > constructs a link command cCompilerLibs + cxxCompilerLibs + > > fortranCompilerLibs. > > > > Perhaps there is a bug in this detection code - thats triggering this > > error. Its verifyable by attempting to use: > > > > --with-clib-autodetect=0 --with-clib-autodetect=0 > > --with-cxxlib-autodetect=0 LIBS='liblist' > > > > where 'liblist' should be determined manually - for the linking of > > C/fortran/CXX to work correctly. > > > > My fix attempts to call command_argument_count(), > > get_command_argument() directly from fortran - and if that doesn't > > work - then its likely a compiler bug.. > > > > [which one can try to replicate with a simple test code] > > > > Satish > > > > On Thu, 17 Sep 2015, Jeff Hammond wrote: > > > >> Please report this via premier.intel.com. > >> > >> Jeff > >> > >> On Wed, Sep 16, 2015 at 11:13 AM, Blaise A Bourdin wrote: > >> > >>> Hi, > >>> > >>> I am testing intel 16 compilers under mac OS, and notice that options > >>> handling for all past and current versions of petsc is broken in fortran > >>> (options are not parsed). > >>> I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is > >>> used (but taking the right value using intel 15 or gcc 5), but I can’t > >>> find > >>> my way through the multiple macros to figure out what is really the > >>> matter… > >>> The problem does not seem to arise under linux with the same compilers. > >>> > >>> Any idea of what is going on? Any test I can run to help (I assume that > >>> you don’t have a test box with the latest intel compilers)? Is it a petsc > >>> bug or an intel bug? > >>> > >>> I use an up to date pets-dev master, and the configure.log file is > >>> available at > >>> https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 > >>> > >>> Thanks, > >>> > >>> Blaise > >>> > >>> -- > >>> Department of Mathematics and Center for Computation & Technology > >>> Louisiana State University, Baton Rouge, LA 70803, USA > >>> Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > >>> http://www.math.lsu.edu/~bourdin > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> > >> > >> > >
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
> On Sep 20, 2015, at 9:04 AM, Satish Balay wrote: > > Hm - I would suggest doing a minimal build build with: > > --with-cxx=0 > --with-clib-autodetect=0 --with-fortranlib-autodetect=0 > LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a" > > And see if that makes a difference. Satish, It does help. Turning off auto detect leads to a functional build, turning it back on leads to a non-functioning one. The configure.log without auto detect is here: https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/61967j4XaVp The one with auto detect there: https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/691pPOSRU The petscconf.h are attached petscconf-autodetect.h Description: petscconf-autodetect.h petscconf-noautodetect.h Description: petscconf-noautodetect.h I thought I had found something fishy with command line argument handling in mixed languages, but that does not seem to be the case. Regards, Blaise > > Satish > > On Sat, 19 Sep 2015, Blaise A Bourdin wrote: > >> Hi, >> >> I was traveling and did not have time to finish investigating, but there >> seems to be an issue with fortran get_command_argument and friends in mixed >> language mode. I will continue to investigate and will report back. >> >> Blaise >> >>> On Sep 19, 2015, at 10:18 AM, Satish Balay wrote: >>> >>> [from the available logs] its not clear what the issue is. >>> >>> PETSc fortran interface attempts to use getarg_() and iargc_() [fortran >>> library functions] - or its variants - directly from C. >>> >>> This usage is non-standard and messy [and some compiler folks - said - >>> its unsupported usage]. >>> >>> Also - configure attempts to guess the fortran link libraries - and >>> constructs a link command cCompilerLibs + cxxCompilerLibs + >>> fortranCompilerLibs. >>> >>> Perhaps there is a bug in this detection code - thats triggering this >>> error. Its verifyable by attempting to use: >>> >>> --with-clib-autodetect=0 --with-clib-autodetect=0 >>> --with-cxxlib-autodetect=0 LIBS='liblist' >>> >>> where 'liblist' should be determined manually - for the linking of >>> C/fortran/CXX to work correctly. >>> >>> My fix attempts to call command_argument_count(), >>> get_command_argument() directly from fortran - and if that doesn't >>> work - then its likely a compiler bug.. >>> >>> [which one can try to replicate with a simple test code] >>> >>> Satish >>> >>> On Thu, 17 Sep 2015, Jeff Hammond wrote: >>> Please report this via premier.intel.com. Jeff On Wed, Sep 16, 2015 at 11:13 AM, Blaise A Bourdin wrote: > Hi, > > I am testing intel 16 compilers under mac OS, and notice that options > handling for all past and current versions of petsc is broken in fortran > (options are not parsed). > I narrowed it down to *argc at start.c:168 always being 0 when intel 16 is > used (but taking the right value using intel 15 or gcc 5), but I canât > find > my way through the multiple macros to figure out what is really the > matter⦠> The problem does not seem to arise under linux with the same compilers. > > Any idea of what is going on? Any test I can run to help (I assume that > you donât have a test box with the latest intel compilers)? Is it a > petsc > bug or an intel bug? > > I use an up to date pets-dev master, and the configure.log file is > available at > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/89329uFByT9 > > Thanks, > > Blaise > > -- > Department of Mathematics and Center for Computation & Technology > Louisiana State University, Baton Rouge, LA 70803, USA > Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 > http://www.math.lsu.edu/~bourdin > > > > > > > > >> >> -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Blaise and Satish,
I'm a bit slow to pick up on this thread as I was busy traveling, but since
I use a Mac and work for Intel, I thought I should see if I could reproduce
the problems that Blaise is seeing. I installed the 16.0 compilers and
built a simple configuration ('--with-debugging=1 COPTFLAGS="-g -O0"
FOPTFLAGS="-g -O0" CXXOPTFLAGS="-g -O0"
--with-blas-lapack-dir=/opt/intel/compilers_and_libraries_2016/mac/mkl
--with-mpi-dir=/Users/rtmills/packages/mpich-3.1.4-intel') using a very
recent revision of 'master' (09b4d96fa5749f82a0af9a914729f77a4ef2b2fd, Sun
Sep 20 22:51:31 2015 -0500). When I try running SNES ex5f and passing
various command-line options, everything appears to work fine. Any
suggestions for digging deeping into this to try to determine the
difference between what Blaise and I are seeing?
Best regards,
Richard
On Mon, Sep 21, 2015 at 10:21 AM, Blaise A Bourdin wrote:
>
> > On Sep 20, 2015, at 9:04 AM, Satish Balay wrote:
> >
> > Hm - I would suggest doing a minimal build build with:
> >
> > --with-cxx=0
> > --with-clib-autodetect=0 --with-fortranlib-autodetect=0
> LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a"
> >
> > And see if that makes a difference.
> Satish,
>
> It does help.
> Turning off auto detect leads to a functional build, turning it back on
> leads to a non-functioning one.
>
> The configure.log without auto detect is here:
>
> https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/61967j4XaVp
> The one with auto detect there:
>https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/691pPOSRU
>
> The petscconf.h are attached
>
>
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
I'm not sure what to suggest.
Blaise gets a successful build with:
--with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0
LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a"
The next thing Blaise could try is:
--with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0
LIBS=-lifcore
[and if this fails - see which ifcore the binary is using - via 'otool -l
petscbinary |grep ifore']
If not - what I would do - is try compiling an example with a petsc makefile.
[with the broken build]
And then copy/paste the link line - removing stuff - until the binary
created by that link line works. [and identify the linker option is
making the difference]. Again - this would be for Blaise to try.
Intel compilers (esp on Mac) had workarrounds to system issues encoded
in the link command. [I remember the first generation of intel
compilers on Mac had such issues]. And the way PETSc configure, and
checkFortranLibraries() process/use this 'ifort -v' info can break
these encoded workarrounds.. [Its a hacky piece of code - and not easy
to get right]
Ideally compilers should provide equivalent of 'mpicc -show' - so that
configure tools can grab and use this info for language interoperable
linking.
[Jed suggested we should 'always use FC as linker' and not do any such
detection. But then we still have to worry about C++. Previously we
had machines where we had to use C++ linker - but perhaps such
machines don't exist anymore]
Satish
On Tue, 22 Sep 2015, Richard Mills wrote:
> Blaise and Satish,
>
> I'm a bit slow to pick up on this thread as I was busy traveling, but since
> I use a Mac and work for Intel, I thought I should see if I could reproduce
> the problems that Blaise is seeing. I installed the 16.0 compilers and
> built a simple configuration ('--with-debugging=1 COPTFLAGS="-g -O0"
> FOPTFLAGS="-g -O0" CXXOPTFLAGS="-g -O0"
> --with-blas-lapack-dir=/opt/intel/compilers_and_libraries_2016/mac/mkl
> --with-mpi-dir=/Users/rtmills/packages/mpich-3.1.4-intel') using a very
> recent revision of 'master' (09b4d96fa5749f82a0af9a914729f77a4ef2b2fd, Sun
> Sep 20 22:51:31 2015 -0500). When I try running SNES ex5f and passing
> various command-line options, everything appears to work fine. Any
> suggestions for digging deeping into this to try to determine the
> difference between what Blaise and I are seeing?
>
> Best regards,
> Richard
>
> On Mon, Sep 21, 2015 at 10:21 AM, Blaise A Bourdin wrote:
>
> >
> > > On Sep 20, 2015, at 9:04 AM, Satish Balay wrote:
> > >
> > > Hm - I would suggest doing a minimal build build with:
> > >
> > > --with-cxx=0
> > > --with-clib-autodetect=0 --with-fortranlib-autodetect=0
> > LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a"
> > >
> > > And see if that makes a difference.
> > Satish,
> >
> > It does help.
> > Turning off auto detect leads to a functional build, turning it back on
> > leads to a non-functioning one.
> >
> > The configure.log without auto detect is here:
> >
> > https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/61967j4XaVp
> > The one with auto detect there:
> >https://filestogeaux.lsu.edu/public/download.php?FILE=bourdin/691pPOSRU
> >
> > The petscconf.h are attached
> >
> >
>
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Satish Balay writes: > [Jed suggested we should 'always use FC as linker' and not do any such > detection. But then we still have to worry about C++. Previously we > had machines where we had to use C++ linker - but perhaps such > machines don't exist anymore] We might only need to distinguish -lstdc++ and -lc++. It's certainly simpler and more consistent than finding Fortran run-time libraries. signature.asc Description: PGP signature
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
On Tue, 22 Sep 2015, Jed Brown wrote: > Satish Balay writes: > > [Jed suggested we should 'always use FC as linker' and not do any such > > detection. But then we still have to worry about C++. Previously we > > had machines where we had to use C++ linker - but perhaps such > > machines don't exist anymore] > > We might only need to distinguish -lstdc++ and -lc++. It's certainly > simpler and more consistent than finding Fortran run-time libraries. And then the MPI c++ libraries.. Satish
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Satish Balay writes: > And then the MPI c++ libraries.. The current MPI standard does not contain C++ bindings. signature.asc Description: PGP signature
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
On Sep 22, 2015, at 3:50 PM, Satish Balay mailto:ba...@mcs.anl.gov>> wrote: I'm not sure what to suggest. Blaise gets a successful build with: --with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0 LIBS="/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a" The next thing Blaise could try is: --with-cxx=0 --with-clib-autodetect=0 --with-fortranlib-autodetect=0 LIBS=-lifcore This works. bourdin@galerkin:tutorials (master)$ otool -l ex5f90 |grep ifor name /opt/HPC/mpich-3.1.4-intel16.0/lib/libmpifort.12.dylib (offset 24) In contrast, with the build command suggested with richard, I get a non-functioning build: bourdin@galerkin:petsc-dev (master)$ ./configure --with-debugging=1 COPTFLAGS="-g -O0" FOPTFLAGS="-g -O0” CXXOPTFLAGS="-g -O0" --with-blas-lapack-dir=/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl --with-mpi-dir=$MPI_HOME bourdin@galerkin:tutorials (master)$ otool -l ex5f90 |grep ifor name /opt/HPC/mpich-3.1.4-intel16.0/lib/libmpifort.12.dylib (offset 24) [and if this fails - see which ifcore the binary is using - via 'otool -l petscbinary |grep ifore'] If not - what I would do - is try compiling an example with a petsc makefile. [with the broken build] And then copy/paste the link line - removing stuff - until the binary created by that link line works. [and identify the linker option is making the difference]. Again - this would be for Blaise to try. With the later configuration, it looks like what matters is the order in which -lmpifort and -lifcore are listed: This one works: mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -lifcore -lmpifort This one does not: mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -lmpifort -lifcore Same goes with the lengthy link line from the petsc makefile: mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl -L/opt/intel-16.0/compilers_and_libraries_2016/mac/mkl -lmkl_intel -lmkl_sequential -lmkl_core -lpthread -lm -Wl,-rpath,/opt/X11/lib -L/opt/X11/lib -lX11 -lssl -lcrypto -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.1.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/6.1.0/lib/darwin -lifcore -lmpifort -lifport -limf -lsvml -lipgo -lirc -lpthread -Wl,-rpath,/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -L/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin -lclang_rt.osx -limf -lsvml -lirng -lipgo -ldecimal -lirc -lclang_rt.osx -lmpicxx -limf -lsvml -lirng -lipgo -ldecimal -lirc -lclang_rt.osx -ldl -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -lmpi -lpmpi -Wl,-rpath,/opt/HPC/mpich-3.1.4-intel16.0/lib -L/opt/HPC/mpich-3.1.4-intel16.0/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib -L/opt/intel-16.0/compilers_and_libraries
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
On Tue, 22 Sep 2015, Blaise A Bourdin wrote: > With the later configuration, it looks like what matters is the order in > which -lmpifort and -lifcore are listed: > This one works: > mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > -lifcore -lmpifort > > This one does not: > mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > -lmpifort -lifcore Does linking with -lifcore vs /opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a make a difference? And what do you have for: cd /opt/HPC/mpich-3.1.4-intel16.0/lib ls nm -Ao libmpiifort.* |grep get_command_argument BTW: I'm assuming you have sept 20 or newer petsc master branch. Satish
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
> On Sep 22, 2015, at 11:16 PM, Satish Balay wrote: > > On Tue, 22 Sep 2015, Blaise A Bourdin wrote: > >> With the later configuration, it looks like what matters is the order in >> which -lmpifort and -lifcore are listed: >> This one works: >> mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >> -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 >> -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o >> -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib >> -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib >> -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib >> -lifcore -lmpifort >> >> This one does not: >> mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress >> -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 >> -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o >> -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib >> -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib >> -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib >> -lmpifort -lifcore > > Does linking with -lifcore vs > /opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a > make a difference? It does. When I specify the path explicitly, the build works, regardless of the order in which libmpifort and libifcore are listed. > > And what do you have for: > > cd /opt/HPC/mpich-3.1.4-intel16.0/lib > ls > nm -Ao libmpiifort.* |grep get_command_argument This gives nothing at all: bourdin@galerkin:lib $ nm -Ao lib* |grep -i command bourdin@galerkin:lib $ > > BTW: I'm assuming you have sept 20 or newer petsc master branch. Yes. I have checked out your changes before testing. Out of curiosity: it looks like linking with -lpetsc is enough, so why use this very complicated link command? i.e. this is enough to produce a perfectly fine binary: mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc Blaise -- Department of Mathematics and Center for Computation & Technology Louisiana State University, Baton Rouge, LA 70803, USA Tel. +1 (225) 578 1612, Fax +1 (225) 578 4276 http://www.math.lsu.edu/~bourdin
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
On Wed, 23 Sep 2015, Blaise A Bourdin wrote: > > > On Sep 22, 2015, at 11:16 PM, Satish Balay wrote: > > > > On Tue, 22 Sep 2015, Blaise A Bourdin wrote: > > > >> With the later configuration, it looks like what matters is the order in > >> which -lmpifort and -lifcore are listed: > >> This one works: > >> mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > >> -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > >> -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > >> -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > >> -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > >> -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > >> -lifcore -lmpifort > >> > >> This one does not: > >> mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > >> -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > >> -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > >> -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > >> -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > >> -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > >> -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > >> -lmpifort -lifcore > > > > Does linking with -lifcore vs > > /opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a > > make a difference? > It does. When I specify the path explicitly, the build works, regardless of > the order in which libmpifort and libifcore are listed. > > > > And what do you have for: > > > > cd /opt/HPC/mpich-3.1.4-intel16.0/lib > > ls > > nm -Ao libmpiifort.* |grep get_command_argument > This gives nothing at all: > bourdin@galerkin:lib $ nm -Ao lib* |grep -i command > bourdin@galerkin:lib $ > Here is my guess: Presumably you have libmpifort.dylib When libmpifort.dylib was created 'libifcore.a' was used in the link linke - thus some of the compiler symbols were pulled into libmpifort.dylib. When libpetsc.dylib [or an example binary] is created the duplcate symbols in libmpifort.dylib is causing grief with libifcore.dylib [but not libifcore.a] Such issues sometimes come up when linking to static libraries for creating a shared/dylib. > > > > BTW: I'm assuming you have sept 20 or newer petsc master branch. > > Yes. I have checked out your changes before testing. > > Out of curiosity: it looks like linking with -lpetsc is enough, so why use > this very complicated link command? > i.e. this is enough to produce a perfectly fine binary: > mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc There is no [easy] automatic way of figuring out this reduced link command that might work. Satish
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Satish Balay writes: > There is no [easy] automatic way of figuring out this reduced link > command that might work. Almost all such problems are caused by shared libraries being broken or improperly linked on too many HPC systems. If shared libraries are used correctly, then using PETSc amounts to passing -lpetsc. If you call another library, you also link it directly (even if PETSc also links to it transitively). That's all. Note that almost all HPC machines are based on Linux and that shared libraries have had well-defined fully-functional semantics on Linux for 20+ years, so functionality is not a matter of a new implementation, but rather just not breaking something that works. Facilities and vendors need constant reminders that functional and properly-used shared libraries are important. As for PETSc, I actually attempt a simplified link line (e.g., in FindPETSc.cmake) and only fall back to the unwieldy beast when it fails. (This happens for static libraries, but also when shared libraries are foobared.) signature.asc Description: PGP signature
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
What the specification document says is irrelevant, since every implementation still supports the C++ bindings. They can be disabled explicitly in MPICH (and derivatives) via MPICH_SKIP_MPICXX and in OpenMPI with OMPI_SKIP_MPICXX. Jeff On Tue, Sep 22, 2015 at 2:11 PM, Jed Brown wrote: > Satish Balay writes: > > And then the MPI c++ libraries.. > > The current MPI standard does not contain C++ bindings. > -- Jeff Hammond jeff.scie...@gmail.com http://jeffhammond.github.io/
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
For what it's worth, I have actually used get_command_argument in my own code and just now confirmed that it works just fine with Intel 16 in a small test program. It would be helpful to disambiguate that from all the shared library discussion. The other solution is to use ISO C11* as God intended and stop mucking around with all these flowery languages. Jeff * It is true that no compiler yet supports C11 threads, but that is because C compiler developers know better than to troll the PETSc team with programming atrocities like threads. On Tue, Sep 22, 2015 at 9:16 PM, Satish Balay wrote: > On Tue, 22 Sep 2015, Blaise A Bourdin wrote: > > > With the later configuration, it looks like what matters is the order in > which -lmpifort and -lifcore are listed: > > This one works: > > mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > -lifcore -lmpifort > > > > This one does not: > > mpif90 -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress > -Wl,-commons,use_dylibs -Wl,-search_paths_first -g -O0 > -I/opt/HPC/petsc-dev/include/petsc/finclude -o ex5f90 ex5f90.o > -Wl,-rpath,/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib > -L/opt/HPC/petsc-dev/Darwin-intel16.0-g/lib -lpetsc > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -L/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib > -Wl,-rpath,/opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/mkl/lib > -lmpifort -lifcore > > Does linking with -lifcore vs > > /opt/intel-16.0/compilers_and_libraries_2016.0.083/mac/compiler/lib/libifcore.a > make a difference? > > And what do you have for: > > cd /opt/HPC/mpich-3.1.4-intel16.0/lib > ls > nm -Ao libmpiifort.* |grep get_command_argument > > BTW: I'm assuming you have sept 20 or newer petsc master branch. > > Satish > -- Jeff Hammond jeff.scie...@gmail.com http://jeffhammond.github.io/
Re: [petsc-dev] broken options handling with intel 16.0 compilers on mac OS
Jeff Hammond writes: > What the specification document says is irrelevant, since every > implementation still supports the C++ bindings. They can be disabled > explicitly in MPICH (and derivatives) via MPICH_SKIP_MPICXX and in OpenMPI > with OMPI_SKIP_MPICXX. It's a linking discussion, and you would only need to link libmpicxx.so (or whatever) if you actually used those bindings. We certainly don't do that and any other project that still uses them should be chastised. BTW, PETSc defines the above macros for simplicity and to avoid the unfortunate messes that used to plague the C++ implementations. signature.asc Description: PGP signature
