Re: [petsc-dev] Fwd: Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines
On Wed, 3 Mar 2021, Jacob Faibussowitsch wrote:
> Hello All,
>
> I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago
> that would crash the compiler when compiling petsc fortran interfaces. The
> TL;DR of it is that the xl compiler creates a function dictionary for every
> function imported in fortran modules, and since petsc fortran interfaces seem
> to import entire packages writ-large this exceeds the number of dictionary
> entries (2**21):
>
> > The reason for the Internal Compiler Error is because we can't grow an
> > interal dictionary anymore (ie we hit a 2**21 limit).
> > The file contains many module procedures and interfaces that use the same
> > helper module. As a result, we are importing the dictionary entries for
> > that module repeatedly reaching
> > the limit.
> >
> > Can you please give the following source code workaround a try?
> > Since there is already "use petscvecdefdummy" at the module scope, one
> > workaround might be to remove the unnecessary "use petscvecdefdummy" in
> > vecnotequal and vecequals
> > and all similar procedures.
> >
> > For example, the test case has:
> > module petscvecdef
> > use petscvecdefdummy
> > ...
> > function vecnotequal(A,B)
> > use petscvecdefdummy
> > logical vecnotequal
> > type(tVec), intent(in) :: A,B
> > vecnotequal = (A%v .ne. B%v)
> > end function
> > function vecequals(A,B)
> > use petscvecdefdummy
> > logical vecequals
> > type(tVec), intent(in) :: A,B
> > vecequals = (A%v .eq. B%v)
> > end function
> > ...
> > end module
> > Another workaround would be to put the procedure definitions from this
> > large module into several submodules. Each submodule would be able to
> > accommodate a dictionary with 2**21 entries.
> >
> >
> > Please let us know if one of the above workarounds resolve the issue.
>
>
> The proposed fix from IBM would be to pull “use moduleXXX” out of subroutines
> or to have our auto-fortran interfaces detect which symbols to include from
> the respective modules and only include those in the subroutines. I’m not
> familiar at all with how the interfaces are generated so I don’t even know if
> this is possible.
I'm not sure what would happen if these 'use' statements are removed [whats
required and what can be removed?]
The relevant code that adds this is in
lib/petsc/bin/maint/generatefortranstubs.py
fd.write(' use petsc'+mansec+'def\n')
Satish
> > IBM provided the following additional explanation and example. Can the
> > process used to generate these routines and functions determine the
> > specific symbols required and then use the only keyword or import statement
> > to include them?
> >
> > When factoring out use statements out of module procedures, you can just
> > delete them. But you can't completely remove them from interface blocks.
> > Instead, you can limit them either by using use , only: or
> > import . if the hundreds of use statements in the program are
> > factored out / limited in this way, that should reduce the dictionary size
> > sufficiently for the program to compile.
> >
> > For example
> > Interface
> > Subroutine VecRestoreArrayReadF90(v,array,ierr)
> > use petscvecdef
> > real(kind=selected_real_kind(10)), pointer :: array(:)
> > integer(kind=selected_int_kind(5)) ierr
> > type(tVec) v
> > End Subroutine
> > End Interface
> >
> > imports all symbols from petscvecdef into the dictionary even though we
> > only need tVec . So we can either:
> >
> > Interface
> > Subroutine VecRestoreArrayReadF90(v,array,ierr)
> > use petscvecdef, only: tVec
> > implicit none
> > real(kind=selected_real_kind(10)), pointer :: array(:)
> > integer(kind=selected_int_kind(5)) ierr
> > type(tVec) v
> > End Subroutine
> > End Interface
> >
> > or if use petscvecdef is used in the outer scope, we can:
> > Interface
> > Subroutine VecRestoreArrayReadF90(v,array,ierr)
> > import tVec
> > implicit none
> > real(kind=selected_real_kind(10)), pointer :: array(:)
> > integer(kind=selected_int_kind(5)) ierr
> > type(tVec) v
> > End Subroutine
> > End Interface
> > (The two methods (use, only vs import) are equivalent in terms of impact to
> > the dictionary.)
> >
>
> Is this compiler ~feature~ something that we intend to work around? Thoughts?
>
> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
>
> > Begin forwarded message:
> >
> > From: "Roy Musselman"
> > Subject: Re: Case TS005062693 - XLF: ICE in xlfentry compiling a module
> > with 358 subroutines
> > Date: March 3, 2021 at 08:23:17 CST
> > To: Jacob Faibus
Re: [petsc-dev] Fwd: Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines
On Wed, 3 Mar 2021, Jacob Faibussowitsch wrote: > Hello All, > > I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago > that would crash the compiler when compiling petsc fortran interfaces. The > TL;DR of it is that the xl compiler creates a function dictionary for every > function imported in fortran modules, and since petsc fortran interfaces seem > to import entire packages writ-large this exceeds the number of dictionary > entries (2**21): > > > The reason for the Internal Compiler Error is because we can't grow an > > interal dictionary anymore (ie we hit a 2**21 limit). > > The file contains many module procedures and interfaces that use the same > > helper module. As a result, we are importing the dictionary entries for > > that module repeatedly reaching > > the limit. > > > > Can you please give the following source code workaround a try? > > Since there is already "use petscvecdefdummy" at the module scope, one > > workaround might be to remove the unnecessary "use petscvecdefdummy" in > > vecnotequal and vecequals > > and all similar procedures. This sounds reasonable - but the change might be tedious [to make without breaking some required dependency]. Perhaps it will also help gfortran RAM requirements.. Satish > > > > For example, the test case has: > > module petscvecdef > > use petscvecdefdummy > > ... > > function vecnotequal(A,B) > > use petscvecdefdummy > > logical vecnotequal > > type(tVec), intent(in) :: A,B > > vecnotequal = (A%v .ne. B%v) > > end function > > function vecequals(A,B) > > use petscvecdefdummy > > logical vecequals > > type(tVec), intent(in) :: A,B > > vecequals = (A%v .eq. B%v) > > end function > > ... > > end module > > Another workaround would be to put the procedure definitions from this > > large module into several submodules. Each submodule would be able to > > accommodate a dictionary with 2**21 entries. > > > > > > Please let us know if one of the above workarounds resolve the issue. > > > The proposed fix from IBM would be to pull “use moduleXXX” out of subroutines > or to have our auto-fortran interfaces detect which symbols to include from > the respective modules and only include those in the subroutines. I’m not > familiar at all with how the interfaces are generated so I don’t even know if > this is possible. > > IBM provided the following additional explanation and example. Can the > > process used to generate these routines and functions determine the > > specific symbols required and then use the only keyword or import statement > > to include them? > > > > When factoring out use statements out of module procedures, you can just > > delete them. But you can't completely remove them from interface blocks. > > Instead, you can limit them either by using use , only: or > > import . if the hundreds of use statements in the program are > > factored out / limited in this way, that should reduce the dictionary size > > sufficiently for the program to compile. > > > > For example > > Interface > > Subroutine VecRestoreArrayReadF90(v,array,ierr) > > use petscvecdef > > real(kind=selected_real_kind(10)), pointer :: array(:) > > integer(kind=selected_int_kind(5)) ierr > > type(tVec) v > > End Subroutine > > End Interface > > > > imports all symbols from petscvecdef into the dictionary even though we > > only need tVec . So we can either: > > > > Interface > > Subroutine VecRestoreArrayReadF90(v,array,ierr) > > use petscvecdef, only: tVec > > implicit none > > real(kind=selected_real_kind(10)), pointer :: array(:) > > integer(kind=selected_int_kind(5)) ierr > > type(tVec) v > > End Subroutine > > End Interface > > > > or if use petscvecdef is used in the outer scope, we can: > > Interface > > Subroutine VecRestoreArrayReadF90(v,array,ierr) > > import tVec > > implicit none > > real(kind=selected_real_kind(10)), pointer :: array(:) > > integer(kind=selected_int_kind(5)) ierr > > type(tVec) v > > End Subroutine > > End Interface > > (The two methods (use, only vs import) are equivalent in terms of impact to > > the dictionary.) > > > > Is this compiler ~feature~ something that we intend to work around? Thoughts? > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391 > > > Begin forwarded message: > > > > From: "Roy Musselman" > > Subject: Re: Case TS005062693 - XLF: ICE in xlfentry compiling a module > > with 358 subroutines > > Date: March 3, 2021 at 08:23:17 CST > > To: Jacob Faibussowitsch > > Cc: "Gyllenhaal, John C." > > > > Hi Jacob, > > I tried the first sugge
[petsc-dev] Fwd: Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines
Hello All, I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago that would crash the compiler when compiling petsc fortran interfaces. The TL;DR of it is that the xl compiler creates a function dictionary for every function imported in fortran modules, and since petsc fortran interfaces seem to import entire packages writ-large this exceeds the number of dictionary entries (2**21): > The reason for the Internal Compiler Error is because we can't grow an > interal dictionary anymore (ie we hit a 2**21 limit). > The file contains many module procedures and interfaces that use the same > helper module. As a result, we are importing the dictionary entries for that > module repeatedly reaching > the limit. > > Can you please give the following source code workaround a try? > Since there is already "use petscvecdefdummy" at the module scope, one > workaround might be to remove the unnecessary "use petscvecdefdummy" in > vecnotequal and vecequals > and all similar procedures. > > For example, the test case has: > module petscvecdef > use petscvecdefdummy > ... > function vecnotequal(A,B) > use petscvecdefdummy > logical vecnotequal > type(tVec), intent(in) :: A,B > vecnotequal = (A%v .ne. B%v) > end function > function vecequals(A,B) > use petscvecdefdummy > logical vecequals > type(tVec), intent(in) :: A,B > vecequals = (A%v .eq. B%v) > end function > ... > end module > Another workaround would be to put the procedure definitions from this large > module into several submodules. Each submodule would be able to accommodate > a dictionary with 2**21 entries. > > > Please let us know if one of the above workarounds resolve the issue. The proposed fix from IBM would be to pull “use moduleXXX” out of subroutines or to have our auto-fortran interfaces detect which symbols to include from the respective modules and only include those in the subroutines. I’m not familiar at all with how the interfaces are generated so I don’t even know if this is possible. > IBM provided the following additional explanation and example. Can the > process used to generate these routines and functions determine the specific > symbols required and then use the only keyword or import statement to include > them? > > When factoring out use statements out of module procedures, you can just > delete them. But you can't completely remove them from interface blocks. > Instead, you can limit them either by using use , only: or > import . if the hundreds of use statements in the program are > factored out / limited in this way, that should reduce the dictionary size > sufficiently for the program to compile. > > For example > Interface > Subroutine VecRestoreArrayReadF90(v,array,ierr) > use petscvecdef > real(kind=selected_real_kind(10)), pointer :: array(:) > integer(kind=selected_int_kind(5)) ierr > type(tVec) v > End Subroutine > End Interface > > imports all symbols from petscvecdef into the dictionary even though we only > need tVec . So we can either: > > Interface > Subroutine VecRestoreArrayReadF90(v,array,ierr) > use petscvecdef, only: tVec > implicit none > real(kind=selected_real_kind(10)), pointer :: array(:) > integer(kind=selected_int_kind(5)) ierr > type(tVec) v > End Subroutine > End Interface > > or if use petscvecdef is used in the outer scope, we can: > Interface > Subroutine VecRestoreArrayReadF90(v,array,ierr) > import tVec > implicit none > real(kind=selected_real_kind(10)), pointer :: array(:) > integer(kind=selected_int_kind(5)) ierr > type(tVec) v > End Subroutine > End Interface > (The two methods (use, only vs import) are equivalent in terms of impact to > the dictionary.) > Is this compiler ~feature~ something that we intend to work around? Thoughts? Best regards, Jacob Faibussowitsch (Jacob Fai - booss - oh - vitch) Cell: (312) 694-3391 > Begin forwarded message: > > From: "Roy Musselman" > Subject: Re: Case TS005062693 - XLF: ICE in xlfentry compiling a module with > 358 subroutines > Date: March 3, 2021 at 08:23:17 CST > To: Jacob Faibussowitsch > Cc: "Gyllenhaal, John C." > > Hi Jacob, > I tried the first suggestion and commented out the use statements called > within the functions. However, I hit the following error complaining about > specific symbol dependencies provided by the library. > > .../src/vec/f90-mod/petscvecmod.F90", line 107.37: 1514-084 (S) Identifier a > is being declared with type name tvec which has not been defined in a derived > type definition. > > IBM provided the following additional explanation and example. Can the > process used to gene
