Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Wed, 17 Mar 2021, Blaise A Bourdin wrote:

> 
> 
> On Mar 16, 2021, at 11:17 PM, Barry Smith 
> mailto:bsm...@petsc.dev>> wrote:
> 
> 
>   So it actually does not load everything at the call to use but waits until 
> the subroutine definitions and then only loads what you ask to import?
> 
>   Sorry, this is odd, with C include or python import it immediately loads up 
> everything as soon as it sees the include or import, there is no way later to 
> say "wo hoarsy I didn't really mean you should get everything I asked, for 
> please only load up a subset".
> 
> 
> But then Fortran was always weird, when I saw presentations of the Fortran 
> standards committee members I always felt like they read every third page of 
> some computer science book but never realized they missed many of the pages.
> 
> Which proves that they know their target audience ;)
> 
> >From the standpoint of somebody who also read every third page of a computer 
> >science book, fortran modules sounded like a good idea on the paper (I 
> >basically think of them as auto-generated header files), but they are 
> >miserable in practice, and in particular a major block to cross compiler 
> >compatibility since the format of .mod file was left to the discretion of 
> >the compiler writers.
> 

Not just (a)cross compiler - different versions of gfortran are incompatible 
with each other's .mod format..

Satish

> Blaise
> 
> --
> A.K. & Shirley Barton Professor of  Mathematics
> Adjunct Professor of Mechanical Engineering
> Adjunct of the Center for Computation & Technology
> Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA
> Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 Web 
> http://www.math.lsu.edu/~bourdin
> 
> 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Blaise A Bourdin]

On Mar 16, 2021, at 11:17 PM, Barry Smith 
mailto:bsm...@petsc.dev>> wrote:


  So it actually does not load everything at the call to use but waits until 
the subroutine definitions and then only loads what you ask to import?

  Sorry, this is odd, with C include or python import it immediately loads up 
everything as soon as it sees the include or import, there is no way later to 
say "wo hoarsy I didn't really mean you should get everything I asked, for 
please only load up a subset".


But then Fortran was always weird, when I saw presentations of the Fortran 
standards committee members I always felt like they read every third page of 
some computer science book but never realized they missed many of the pages.

Which proves that they know their target audience ;)

>From the standpoint of somebody who also read every third page of a computer 
>science book, fortran modules sounded like a good idea on the paper (I 
>basically think of them as auto-generated header files), but they are 
>miserable in practice, and in particular a major block to cross compiler 
>compatibility since the format of .mod file was left to the discretion of the 
>compiler writers.

Blaise

--
A.K. & Shirley Barton Professor of  Mathematics
Adjunct Professor of Mechanical Engineering
Adjunct of the Center for Computation & Technology
Louisiana State University, Lockett Hall Room 344, Baton Rouge, LA 70803, USA
Tel. +1 (225) 578 1612, Fax  +1 (225) 578 4276 Web 
http://www.math.lsu.edu/~bourdin

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

  So it actually does not load everything at the call to use but waits until 
the subroutine definitions and then only loads what you ask to import? 

  Sorry, this is odd, with C include or python import it immediately loads up 
everything as soon as it sees the include or import, there is no way later to 
say "wo hoarsy I didn't really mean you should get everything I asked, for 
please only load up a subset".  But then Fortran was always weird, when I saw 
presentations of the Fortran standards committee members I always felt like 
they read every third page of some computer science book but never realized 
they missed many of the pages.

  But, ok, if it works, it works. 

 Barry






> On Mar 16, 2021, at 6:54 PM, Jacob Faibussowitsch  wrote:
> 
>> I am puzzled by the import tXXX, how it can work? How does it know what 
>> module file to look in for the import? With use xxx the xxx gets translated 
>> directly to a file name. But with import tXXX that is impossible.
> 
> Its similar to nested namespace semantics, you “use" the super-namespace at 
> the module level then import the objects at interface of that module. i.e.:
> 
> Module foo
>   Use bar <—— module contains bop
>   Interface 
> Subroutine baz(x)
>   Import bop <—— hoists bop out of bar defined in super-scope
>   Bop x
> End subroutine
>   End interface
> End module
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
>> On Mar 16, 2021, at 18:25, Barry Smith > > wrote:
>> 
>> 
>> 
>> 
>>> On Mar 16, 2021, at 5:03 PM, Satish Balay >> > wrote:
>>> 
>>> On Tue, 16 Mar 2021, Barry Smith wrote:
>>> 
 
 Jacob,
 
 You very well may be right, splitting may not be needed or relevant. I 
 only advocated it since for Mat it did fix a VM gfortran memory issue. How 
 closely the IBM compiler and gfortran issues are I do not know. 
 
 My MR Mat split is read to to go into main.
 
 https://gitlab.com/petsc/petsc/-/merge_requests/3715 
  
 >  The MPI one is 
 still needing some babysitting, hopefully it can pass the CI and get into 
 main soon.
 
 Both should be in main before starting to muck with changing the use stuff 
 for the IBM compiler.
 
 Satish, does your Vec file split pass the CI? If so I think we should also 
 get it into main before messing with the use for the IBM compiler.
>>> 
>>> Barry - if the switch from 'use' to 'import' is sufficient  [for VM builds] 
>>> - should we still split the sources? Right now it changes only the 
>>> auto-fortran stubs, fixing custom stubs should give further improvements.
>> 
>>   If the import solves the VM gfortran problem then I see no reason to split 
>> the source (it is a little easier to maintain as one file). 
>> 
>>   So maybe, for now use the import and later if the VM gfortran problem 
>> continues we split the source.
>> 
>>  Barry
>> 
>>  I am puzzled by the import tXXX, how it can work? How does it know what 
>> module file to look in for the import? With use xxx the xxx gets translated 
>> directly to a file name. But with import tXXX that is impossible.
>> 
>> 
>> 
>>> 
>>> If spliting sources is useful - I'll look into pushing the Vec split into 
>>> this MR.
>>> 
>>> BTW: Matt - can you test balay/reorg-f90-for-xlf branch against  'gfortran 
>>> VM'?
>>> 
>>> Satish
>>> 
 
 
 Who has access to the wonky IBM compiler?  Is there any way to have 
 general access so it can be added to the CI and we can maintain code that 
 works with it? Or does someone need to fix things one at a time, first 
 Vec, then Mat, then  
 
 
 Barry
 
 
> On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
> mailto:petsc-dev@mcs.anl.gov>> wrote:
> 
> On Tue, 16 Mar 2021, Matthew Knepley wrote:
> 
>> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
>> mailto:jacob@gmail.com>>
>> wrote:
>> 
>>> To me the best solution is to first split the files when possible (it is
>>> often not or just too painful) and then adding imports if needed.
>>> 
>>> 
>>> I’m not sure I agree. Bear in mind my familiarity with fortran is very
>>> limited, but to me the “use” statement is similar to #include  
>>> in C
>>> or using namespace xxx in cpp.
>>> 
>>> Our fortran interface is not optimal code atm, it’s as if we put a
>>> "#include ” before every Vec function in the C source
>>> (assuming include guards are not used). Sure you can fix this by 
>>> splitting
>>> the vec src files in two, but at the end of the day that’s a bandaid
>>> solution. I think I can get the import stuff to work reasonably well (a 
>>

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

> I am puzzled by the import tXXX, how it can work? How does it know what 
> module file to look in for the import? With use xxx the xxx gets translated 
> directly to a file name. But with import tXXX that is impossible.

Its similar to nested namespace semantics, you “use" the super-namespace at the 
module level then import the objects at interface of that module. i.e.:

Module foo
  Use bar <—— module contains bop
  Interface 
Subroutine baz(x)
  Import bop <—— hoists bop out of bar defined in super-scope
  Bop x
End subroutine
  End interface
End module

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 18:25, Barry Smith  wrote:
> 
> 
> 
> 
>> On Mar 16, 2021, at 5:03 PM, Satish Balay  wrote:
>> 
>> On Tue, 16 Mar 2021, Barry Smith wrote:
>> 
>>> 
>>> Jacob,
>>> 
>>> You very well may be right, splitting may not be needed or relevant. I 
>>> only advocated it since for Mat it did fix a VM gfortran memory issue. How 
>>> closely the IBM compiler and gfortran issues are I do not know. 
>>> 
>>> My MR Mat split is read to to go into main.
>>> 
>>> https://gitlab.com/petsc/petsc/-/merge_requests/3715 
>>>   The MPI one is 
>>> still needing some babysitting, hopefully it can pass the CI and get into 
>>> main soon.
>>> 
>>> Both should be in main before starting to muck with changing the use stuff 
>>> for the IBM compiler.
>>> 
>>> Satish, does your Vec file split pass the CI? If so I think we should also 
>>> get it into main before messing with the use for the IBM compiler.
>> 
>> Barry - if the switch from 'use' to 'import' is sufficient  [for VM builds] 
>> - should we still split the sources? Right now it changes only the 
>> auto-fortran stubs, fixing custom stubs should give further improvements.
> 
>   If the import solves the VM gfortran problem then I see no reason to split 
> the source (it is a little easier to maintain as one file). 
> 
>   So maybe, for now use the import and later if the VM gfortran problem 
> continues we split the source.
> 
>  Barry
> 
>  I am puzzled by the import tXXX, how it can work? How does it know what 
> module file to look in for the import? With use xxx the xxx gets translated 
> directly to a file name. But with import tXXX that is impossible.
> 
> 
> 
>> 
>> If spliting sources is useful - I'll look into pushing the Vec split into 
>> this MR.
>> 
>> BTW: Matt - can you test balay/reorg-f90-for-xlf branch against  'gfortran 
>> VM'?
>> 
>> Satish
>> 
>>> 
>>> 
>>> Who has access to the wonky IBM compiler?  Is there any way to have general 
>>> access so it can be added to the CI and we can maintain code that works 
>>> with it? Or does someone need to fix things one at a time, first Vec, then 
>>> Mat, then  
>>> 
>>> 
>>> Barry
>>> 
>>> 
 On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
  wrote:
 
 On Tue, 16 Mar 2021, Matthew Knepley wrote:
 
> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
> wrote:
> 
>> To me the best solution is to first split the files when possible (it is
>> often not or just too painful) and then adding imports if needed.
>> 
>> 
>> I’m not sure I agree. Bear in mind my familiarity with fortran is very
>> limited, but to me the “use” statement is similar to #include  
>> in C
>> or using namespace xxx in cpp.
>> 
>> Our fortran interface is not optimal code atm, it’s as if we put a
>> "#include ” before every Vec function in the C source
>> (assuming include guards are not used). Sure you can fix this by 
>> splitting
>> the vec src files in two, but at the end of the day that’s a bandaid
>> solution. I think I can get the import stuff to work reasonably well (a 
>> lot
>> of the infrastructure to capture types is already present in the current
>> generatefortranstubs).
>> 
> 
> If you can get this to work, we should definitely do it. Was there a
> question about getting it to work?
 
 Likely these additional changes should be done starting with Barry's 
 split-up of sources.
 
 Satish
 
> 
> Thanks,
> 
>   Matt
> 
> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
>> 
>> 
>> Jacob,
>> 
>>   I split  the mat definitions in the MR
>> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
>> memory requirements enough to get builds through on some VM that failed
>> previously ran out of memory (with gfortran).
>> 
>>   The petsc*mod.F90 files are all handwritten so it is ok to manually
>> change them with imports if that helps the IBM compiler. To me the best
>> solution is to first split the files when possible (it is

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

> On Mar 16, 2021, at 5:03 PM, Satish Balay  wrote:
> 
> On Tue, 16 Mar 2021, Barry Smith wrote:
> 
>> 
>>  Jacob,
>> 
>>  You very well may be right, splitting may not be needed or relevant. I 
>> only advocated it since for Mat it did fix a VM gfortran memory issue. How 
>> closely the IBM compiler and gfortran issues are I do not know. 
>> 
>>  My MR Mat split is read to to go into main.
>> 
>>  https://gitlab.com/petsc/petsc/-/merge_requests/3715 
>>   The MPI one is still 
>> needing some babysitting, hopefully it can pass the CI and get into main 
>> soon.
>> 
>>  Both should be in main before starting to muck with changing the use stuff 
>> for the IBM compiler.
>> 
>>  Satish, does your Vec file split pass the CI? If so I think we should also 
>> get it into main before messing with the use for the IBM compiler.
> 
> Barry - if the switch from 'use' to 'import' is sufficient  [for VM builds] - 
> should we still split the sources? Right now it changes only the auto-fortran 
> stubs, fixing custom stubs should give further improvements.

   If the import solves the VM gfortran problem then I see no reason to split 
the source (it is a little easier to maintain as one file). 

   So maybe, for now use the import and later if the VM gfortran problem 
continues we split the source.

  Barry

  I am puzzled by the import tXXX, how it can work? How does it know what 
module file to look in for the import? With use xxx the xxx gets translated 
directly to a file name. But with import tXXX that is impossible.



> 
> If spliting sources is useful - I'll look into pushing the Vec split into 
> this MR.
> 
> BTW: Matt - can you test balay/reorg-f90-for-xlf branch against  'gfortran 
> VM'?
> 
> Satish
> 
>> 
>> 
>> Who has access to the wonky IBM compiler?  Is there any way to have general 
>> access so it can be added to the CI and we can maintain code that works with 
>> it? Or does someone need to fix things one at a time, first Vec, then Mat, 
>> then  
>> 
>> 
>>  Barry
>> 
>> 
>>> On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
>>>  wrote:
>>> 
>>> On Tue, 16 Mar 2021, Matthew Knepley wrote:
>>> 
 On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
 wrote:
 
> To me the best solution is to first split the files when possible (it is
> often not or just too painful) and then adding imports if needed.
> 
> 
> I’m not sure I agree. Bear in mind my familiarity with fortran is very
> limited, but to me the “use” statement is similar to #include  in 
> C
> or using namespace xxx in cpp.
> 
> Our fortran interface is not optimal code atm, it’s as if we put a
> "#include ” before every Vec function in the C source
> (assuming include guards are not used). Sure you can fix this by splitting
> the vec src files in two, but at the end of the day that’s a bandaid
> solution. I think I can get the import stuff to work reasonably well (a 
> lot
> of the infrastructure to capture types is already present in the current
> generatefortranstubs).
> 
 
 If you can get this to work, we should definitely do it. Was there a
 question about getting it to work?
>>> 
>>> Likely these additional changes should be done starting with Barry's 
>>> split-up of sources.
>>> 
>>> Satish
>>> 
 
 Thanks,
 
Matt
 
 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
> 
> 
>  Jacob,
> 
>I split  the mat definitions in the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
> memory requirements enough to get builds through on some VM that failed
> previously ran out of memory (with gfortran).
> 
>The petsc*mod.F90 files are all handwritten so it is ok to manually
> change them with imports if that helps the IBM compiler. To me the best
> solution is to first split the files when possible (it is often not or 
> just
> too painful) and then adding imports if needed.
> 
>Note also the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
> a great deal with the IBM compiler or not.
> 
>Thanks
> 
>Barry
> 
> 
> 
> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
> wrote:
> 
> Ok something I have gotten to work, but only doing things by hand in
> petscvecmod.F90:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90
> b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
> module petscisdef
> use petscisdefdummy
> interface operator(.ne.)
> function isnotequal(A,B)
> -  use petscisdefdummy
> +

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

Satish, I’ve managed to get the automatic gathering of petsc types to work 
surprisingly easily (who knew that rigid language standards would make  
definitions easy to look for), so I propose we combine our contributions here. 
Your replacement is far cleaner than mine, so we just supplement it with my 
auto type lookup. 

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 17:03, Satish Balay via petsc-dev  
> wrote:
> 
> On Tue, 16 Mar 2021, Barry Smith wrote:
> 
>> 
>>  Jacob,
>> 
>>  You very well may be right, splitting may not be needed or relevant. I 
>> only advocated it since for Mat it did fix a VM gfortran memory issue. How 
>> closely the IBM compiler and gfortran issues are I do not know. 
>> 
>>  My MR Mat split is read to to go into main.
>> 
>>  https://gitlab.com/petsc/petsc/-/merge_requests/3715 
>>   The MPI one is still 
>> needing some babysitting, hopefully it can pass the CI and get into main 
>> soon.
>> 
>>  Both should be in main before starting to muck with changing the use stuff 
>> for the IBM compiler.
>> 
>>  Satish, does your Vec file split pass the CI? If so I think we should also 
>> get it into main before messing with the use for the IBM compiler.
> 
> Barry - if the switch from 'use' to 'import' is sufficient  [for VM builds] - 
> should we still split the sources? Right now it changes only the auto-fortran 
> stubs, fixing custom stubs should give further improvements.
> 
> If spliting sources is useful - I'll look into pushing the Vec split into 
> this MR.
> 
> BTW: Matt - can you test balay/reorg-f90-for-xlf branch against  'gfortran 
> VM'?
> 
> Satish
> 
>> 
>> 
>> Who has access to the wonky IBM compiler?  Is there any way to have general 
>> access so it can be added to the CI and we can maintain code that works with 
>> it? Or does someone need to fix things one at a time, first Vec, then Mat, 
>> then  
>> 
>> 
>>  Barry
>> 
>> 
>>> On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
>>>  wrote:
>>> 
>>> On Tue, 16 Mar 2021, Matthew Knepley wrote:
>>> 
 On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
 wrote:
 
> To me the best solution is to first split the files when possible (it is
> often not or just too painful) and then adding imports if needed.
> 
> 
> I’m not sure I agree. Bear in mind my familiarity with fortran is very
> limited, but to me the “use” statement is similar to #include  in 
> C
> or using namespace xxx in cpp.
> 
> Our fortran interface is not optimal code atm, it’s as if we put a
> "#include ” before every Vec function in the C source
> (assuming include guards are not used). Sure you can fix this by splitting
> the vec src files in two, but at the end of the day that’s a bandaid
> solution. I think I can get the import stuff to work reasonably well (a 
> lot
> of the infrastructure to capture types is already present in the current
> generatefortranstubs).
> 
 
 If you can get this to work, we should definitely do it. Was there a
 question about getting it to work?
>>> 
>>> Likely these additional changes should be done starting with Barry's 
>>> split-up of sources.
>>> 
>>> Satish
>>> 
 
 Thanks,
 
Matt
 
 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
> 
> 
>  Jacob,
> 
>I split  the mat definitions in the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
> memory requirements enough to get builds through on some VM that failed
> previously ran out of memory (with gfortran).
> 
>The petsc*mod.F90 files are all handwritten so it is ok to manually
> change them with imports if that helps the IBM compiler. To me the best
> solution is to first split the files when possible (it is often not or 
> just
> too painful) and then adding imports if needed.
> 
>Note also the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
> a great deal with the IBM compiler or not.
> 
>Thanks
> 
>Barry
> 
> 
> 
> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
> wrote:
> 
> Ok something I have gotten to work, but only doing things by hand in
> petscvecmod.F90:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90
> b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
> module petscisdef
> use petscisdefdummy
> interface operator(.ne.)
> function isnotequal(A,B)
> -  use petscisdefdummy
> + import tIS
>  logical isnotequal
> 
>

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Tue, 16 Mar 2021, Barry Smith wrote:

> 
>   Jacob,
> 
>   You very well may be right, splitting may not be needed or relevant. I 
> only advocated it since for Mat it did fix a VM gfortran memory issue. How 
> closely the IBM compiler and gfortran issues are I do not know. 
> 
>   My MR Mat split is read to to go into main.
> 
>   https://gitlab.com/petsc/petsc/-/merge_requests/3715 
>   The MPI one is still 
> needing some babysitting, hopefully it can pass the CI and get into main soon.
> 
>   Both should be in main before starting to muck with changing the use stuff 
> for the IBM compiler.
> 
>   Satish, does your Vec file split pass the CI? If so I think we should also 
> get it into main before messing with the use for the IBM compiler.

Barry - if the switch from 'use' to 'import' is sufficient  [for VM builds] - 
should we still split the sources? Right now it changes only the auto-fortran 
stubs, fixing custom stubs should give further improvements.

If spliting sources is useful - I'll look into pushing the Vec split into this 
MR.

BTW: Matt - can you test balay/reorg-f90-for-xlf branch against  'gfortran VM'?

Satish

> 
> 
>  Who has access to the wonky IBM compiler?  Is there any way to have general 
> access so it can be added to the CI and we can maintain code that works with 
> it? Or does someone need to fix things one at a time, first Vec, then Mat, 
> then  
> 
> 
>   Barry
> 
> 
> > On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
> >  wrote:
> > 
> > On Tue, 16 Mar 2021, Matthew Knepley wrote:
> > 
> >> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
> >> wrote:
> >> 
> >>> To me the best solution is to first split the files when possible (it is
> >>> often not or just too painful) and then adding imports if needed.
> >>> 
> >>> 
> >>> I’m not sure I agree. Bear in mind my familiarity with fortran is very
> >>> limited, but to me the “use” statement is similar to #include  in 
> >>> C
> >>> or using namespace xxx in cpp.
> >>> 
> >>> Our fortran interface is not optimal code atm, it’s as if we put a
> >>> "#include ” before every Vec function in the C source
> >>> (assuming include guards are not used). Sure you can fix this by splitting
> >>> the vec src files in two, but at the end of the day that’s a bandaid
> >>> solution. I think I can get the import stuff to work reasonably well (a 
> >>> lot
> >>> of the infrastructure to capture types is already present in the current
> >>> generatefortranstubs).
> >>> 
> >> 
> >> If you can get this to work, we should definitely do it. Was there a
> >> question about getting it to work?
> > 
> > Likely these additional changes should be done starting with Barry's 
> > split-up of sources.
> > 
> > Satish
> > 
> >> 
> >>  Thanks,
> >> 
> >> Matt
> >> 
> >> 
> >>> Best regards,
> >>> 
> >>> Jacob Faibussowitsch
> >>> (Jacob Fai - booss - oh - vitch)
> >>> Cell: (312) 694-3391
> >>> 
> >>> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
> >>> 
> >>> 
> >>>   Jacob,
> >>> 
> >>> I split  the mat definitions in the MR
> >>> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
> >>> memory requirements enough to get builds through on some VM that failed
> >>> previously ran out of memory (with gfortran).
> >>> 
> >>> The petsc*mod.F90 files are all handwritten so it is ok to manually
> >>> change them with imports if that helps the IBM compiler. To me the best
> >>> solution is to first split the files when possible (it is often not or 
> >>> just
> >>> too painful) and then adding imports if needed.
> >>> 
> >>> Note also the MR
> >>> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
> >>> a great deal with the IBM compiler or not.
> >>> 
> >>> Thanks
> >>> 
> >>> Barry
> >>> 
> >>> 
> >>> 
> >>> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
> >>> wrote:
> >>> 
> >>> Ok something I have gotten to work, but only doing things by hand in
> >>> petscvecmod.F90:
> >>> 
> >>> diff --git a/src/vec/f90-mod/petscvecmod.F90
> >>> b/src/vec/f90-mod/petscvecmod.F90
> >>> index 0c447156b9..ef3e2e2e55 100644
> >>> --- a/src/vec/f90-mod/petscvecmod.F90
> >>> +++ b/src/vec/f90-mod/petscvecmod.F90
> >>> module petscisdef
> >>> use petscisdefdummy
> >>> interface operator(.ne.)
> >>> function isnotequal(A,B)
> >>> -  use petscisdefdummy
> >>> + import tIS
> >>>   logical isnotequal
> >>> 
> >>>   type(tIS), intent(in) :: A,B
> >>> end function isnotequal
> >>> 
> >>> This works for everything wrapped in a module which contains an interface
> >>> block. For dangling functions the following works:
> >>> 
> >>> function isnotequal(A,B)
> >>> -  use petscisdefdummy
> >>> + use petscisdef, only: tIs
> >>>   logical isnotequal
> >>>   type(tIS), intent(in) :: A,B
> >>>   isnotequal = (A%v .ne. B%v)
> >>> end function isnotequal
> >>> 
> >>> Do our fortran stub generators have any notion of types? Specifically
> >>> types th

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

Great! I created MR with the current changes

https://gitlab.com/petsc/petsc/-/merge_requests/3723

Satish

On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:

> Satish your branch built successfully.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 16, 2021, at 16:37, Satish Balay  wrote:
> > 
> > hm - generatefortranstubs is not useful for the custom stubs.
> > 
> > From them - I would:
> > - in the first pass - remove all 'use' statements
> > - iterate:
> >  * run a build
> >  * add 'import' for  types that give build errors
> > 
> > But I think its best to do this after the 2 MRs that change fortran 
> > interfaces are (reviewed/)merged in.
> > 
> > Sure - its would be better to automate the search for t
> > declarations in PETSc. bfort reads in lib/petsc/conf/bfort-petsc.txt -
> > so perhaps generatefortranstubs can also use this data..
> > 
> > Satish
> > 
> > On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> > 
> >> Alternatively generatefortranstubs can traipse through 
> >> src//f90-mod/petsc.h90 and look for type t 
> >> definitions and build a list of petsc types that way, but at that point 
> >> we’re halfway to writing our own compiler.
> >> 
> >> Best regards,
> >> 
> >> Jacob Faibussowitsch
> >> (Jacob Fai - booss - oh - vitch)
> >> Cell: (312) 694-3391
> >> 
> >>> On Mar 16, 2021, at 16:23, Jacob Faibussowitsch  
> >>> wrote:
> >>> 
> >>> So I have hit a bit of a wall. I am able to pull out all of the types for 
> >>> a subroutine declaration but I cannot determine which types are classes 
> >>> since those are the ones that need to be imported. For example:
> >>> 
> >>> subroutine PetscMatlabEngineDestroy(a,z)  
> >>> 
> >>>   PetscMatlabEngine a ! PetscMatlabEngine
> >>>PetscErrorCode z
> >>> end subroutine PetscMatlabEngineDestroy
> >>> 
> >>> Is compiles completely fine but
> >>> 
> >>> subroutine PetscRandomDestroy(a,z)
> >>> 
> >>>   PetscRandom a ! PetscRandom
> >>>PetscErrorCode z
> >>> end subroutine PetscRandomDestroy
> >>> 
> >>> Requires an “import tPetscRandom”. Previously both of these had “import 
> >>> petscsys” stuck in them.
> >>> 
> >>> Best regards,
> >>> 
> >>> Jacob Faibussowitsch
> >>> (Jacob Fai - booss - oh - vitch)
> >>> Cell: (312) 694-3391
> >>> 
>  On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev 
>  mailto:petsc-dev@mcs.anl.gov> 
>  >> wrote:
>  
>  And with this change (alone) - the time to compile the f90 modules went 
>  (on pj01 testbox) from:
>  
>  0m48.676s
>  to
>  0m15.053s
>  
>  Satish
>  
>  On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
>  
> > Ah - the issue is 'import IS' vs 'import tIS'
> > 
> > Pushed a fix now. [and reverted my earlier source split change]. My 
> > build goes through fine now.
> > 
> > Satish
> > 
> > On Tue, 16 Mar 2021, Barry Smith wrote:
> > 
> >> 
> >> Satish,
> >> 
> >>   The  import tIS might only work because the IS is already defined in 
> >> the same file so the compiler can pull in just part of the use 
> >> petscisdefdummy ? If the original module that contains the PetscRandom 
> >> is in a different file then I don't see how the compiler can find and 
> >> import PetscRandom. Is there a version of import where you also list 
> >> the module (file) that the beast is from so the compiler can get it 
> >> from that module?
> >> 
> >> Barry
> >> 
> >>   Do both the manually generated petsc*mod.F90 and the automatically 
> >> generated files need to be switched to not use use everywhere? Or is 
> >> it enough just to fix the manual ones for now and not mess with sowing?
> >> 
> >> 
> >> 
> >>> On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
> >>> mailto:petsc-dev@mcs.anl.gov> 
> >>> >> wrote:
> >>> 
> >>> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> >>> 
>  
>  
>  Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>  
> > Ok something I have gotten to work, but only doing things by hand 
> > in petscvecmod.F90:
> > 
> > diff --git a/src/vec/f90-mod/petscvecmod.F90 
> > b/src/vec/f90-mod/petscvecmod.F90
> > index 0c447156b9..ef3e2e2e55 100644
> > --- a/src/vec/f90-mod/petscvecmod.F90
> > +++ b/src/vec/f90-mod/petscvecmod.F90
> > module petscisdef
> > use petscisdefdummy
> > interface operator(.ne.)
> > function isnotequal(A,B)
> > -  use petscisdefdummy
> > + import tIS
> > logical isnotequal  

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

Satish your branch built successfully.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 16:37, Satish Balay  wrote:
> 
> hm - generatefortranstubs is not useful for the custom stubs.
> 
> From them - I would:
> - in the first pass - remove all 'use' statements
> - iterate:
>  * run a build
>  * add 'import' for  types that give build errors
> 
> But I think its best to do this after the 2 MRs that change fortran 
> interfaces are (reviewed/)merged in.
> 
> Sure - its would be better to automate the search for t
> declarations in PETSc. bfort reads in lib/petsc/conf/bfort-petsc.txt -
> so perhaps generatefortranstubs can also use this data..
> 
> Satish
> 
> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> 
>> Alternatively generatefortranstubs can traipse through 
>> src//f90-mod/petsc.h90 and look for type t 
>> definitions and build a list of petsc types that way, but at that point 
>> we’re halfway to writing our own compiler.
>> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>>> On Mar 16, 2021, at 16:23, Jacob Faibussowitsch  wrote:
>>> 
>>> So I have hit a bit of a wall. I am able to pull out all of the types for a 
>>> subroutine declaration but I cannot determine which types are classes since 
>>> those are the ones that need to be imported. For example:
>>> 
>>> subroutine PetscMatlabEngineDestroy(a,z)
>>>   
>>>   PetscMatlabEngine a ! PetscMatlabEngine
>>>PetscErrorCode z
>>> end subroutine PetscMatlabEngineDestroy
>>> 
>>> Is compiles completely fine but
>>> 
>>> subroutine PetscRandomDestroy(a,z)  
>>>   
>>>   PetscRandom a ! PetscRandom
>>>PetscErrorCode z
>>> end subroutine PetscRandomDestroy
>>> 
>>> Requires an “import tPetscRandom”. Previously both of these had “import 
>>> petscsys” stuck in them.
>>> 
>>> Best regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> Cell: (312) 694-3391
>>> 
 On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev 
 mailto:petsc-dev@mcs.anl.gov> 
 >> wrote:
 
 And with this change (alone) - the time to compile the f90 modules went 
 (on pj01 testbox) from:
 
 0m48.676s
 to
 0m15.053s
 
 Satish
 
 On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
 
> Ah - the issue is 'import IS' vs 'import tIS'
> 
> Pushed a fix now. [and reverted my earlier source split change]. My build 
> goes through fine now.
> 
> Satish
> 
> On Tue, 16 Mar 2021, Barry Smith wrote:
> 
>> 
>> Satish,
>> 
>>   The  import tIS might only work because the IS is already defined in 
>> the same file so the compiler can pull in just part of the use 
>> petscisdefdummy ? If the original module that contains the PetscRandom 
>> is in a different file then I don't see how the compiler can find and 
>> import PetscRandom. Is there a version of import where you also list the 
>> module (file) that the beast is from so the compiler can get it from 
>> that module?
>> 
>> Barry
>> 
>>   Do both the manually generated petsc*mod.F90 and the automatically 
>> generated files need to be switched to not use use everywhere? Or is it 
>> enough just to fix the manual ones for now and not mess with sowing?
>> 
>> 
>> 
>>> On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
>>> mailto:petsc-dev@mcs.anl.gov> 
>>> >> wrote:
>>> 
>>> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
>>> 
 
 
 Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
 
> Ok something I have gotten to work, but only doing things by hand in 
> petscvecmod.F90:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
> module petscisdef
> use petscisdefdummy
> interface operator(.ne.)
> function isnotequal(A,B)
> -  use petscisdefdummy
> + import tIS
> logical isnotequal
>  
> type(tIS), intent(in) :: A,B
 
 generatefortranstubs.py has some hakey parsing code. I guess it can be 
 updated to do this.
 
 i.e - look for 'type(tIS)' and add 'import tIS'
>>> 
>>> I pushed changes to generatefortranstubs.py for this (to
>>> balay/reorg-f90-for-xlf branch). But there are errors. (I don't 
>>> completely understa

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

hm - generatefortranstubs is not useful for the custom stubs.

>From them - I would:
- in the first pass - remove all 'use' statements
- iterate:
  * run a build
  * add 'import' for  types that give build errors

But I think its best to do this after the 2 MRs that change fortran interfaces 
are (reviewed/)merged in.

Sure - its would be better to automate the search for t
declarations in PETSc. bfort reads in lib/petsc/conf/bfort-petsc.txt -
so perhaps generatefortranstubs can also use this data..

Satish

On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:

> Alternatively generatefortranstubs can traipse through 
> src//f90-mod/petsc.h90 and look for type t 
> definitions and build a list of petsc types that way, but at that point we’re 
> halfway to writing our own compiler.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 16, 2021, at 16:23, Jacob Faibussowitsch  wrote:
> > 
> > So I have hit a bit of a wall. I am able to pull out all of the types for a 
> > subroutine declaration but I cannot determine which types are classes since 
> > those are the ones that need to be imported. For example:
> > 
> > subroutine PetscMatlabEngineDestroy(a,z)
> >   
> >PetscMatlabEngine a ! PetscMatlabEngine
> > PetscErrorCode z
> > end subroutine PetscMatlabEngineDestroy
> > 
> > Is compiles completely fine but
> > 
> > subroutine PetscRandomDestroy(a,z)  
> >   
> >PetscRandom a ! PetscRandom
> > PetscErrorCode z
> > end subroutine PetscRandomDestroy
> > 
> > Requires an “import tPetscRandom”. Previously both of these had “import 
> > petscsys” stuck in them.
> > 
> > Best regards,
> > 
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > Cell: (312) 694-3391
> > 
> >> On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev 
> >> mailto:petsc-dev@mcs.anl.gov>> wrote:
> >> 
> >> And with this change (alone) - the time to compile the f90 modules went 
> >> (on pj01 testbox) from:
> >> 
> >> 0m48.676s
> >> to
> >> 0m15.053s
> >> 
> >> Satish
> >> 
> >> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> >> 
> >>> Ah - the issue is 'import IS' vs 'import tIS'
> >>> 
> >>> Pushed a fix now. [and reverted my earlier source split change]. My build 
> >>> goes through fine now.
> >>> 
> >>> Satish
> >>> 
> >>> On Tue, 16 Mar 2021, Barry Smith wrote:
> >>> 
>  
>   Satish,
>  
> The  import tIS might only work because the IS is already defined in 
>  the same file so the compiler can pull in just part of the use 
>  petscisdefdummy ? If the original module that contains the PetscRandom 
>  is in a different file then I don't see how the compiler can find and 
>  import PetscRandom. Is there a version of import where you also list the 
>  module (file) that the beast is from so the compiler can get it from 
>  that module?
>  
>   Barry
>  
> Do both the manually generated petsc*mod.F90 and the automatically 
>  generated files need to be switched to not use use everywhere? Or is it 
>  enough just to fix the manual ones for now and not mess with sowing?
>  
>  
>  
> > On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
> > mailto:petsc-dev@mcs.anl.gov>> wrote:
> > 
> > On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> > 
> >> 
> >> 
> >> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> >> 
> >>> Ok something I have gotten to work, but only doing things by hand in 
> >>> petscvecmod.F90:
> >>> 
> >>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> >>> b/src/vec/f90-mod/petscvecmod.F90
> >>> index 0c447156b9..ef3e2e2e55 100644
> >>> --- a/src/vec/f90-mod/petscvecmod.F90
> >>> +++ b/src/vec/f90-mod/petscvecmod.F90
> >>> module petscisdef
> >>> use petscisdefdummy
> >>> interface operator(.ne.)
> >>> function isnotequal(A,B)
> >>> -  use petscisdefdummy
> >>> + import tIS
> >>>  logical isnotequal   
> >>>   
> >>>  type(tIS), intent(in) :: A,B
> >> 
> >> generatefortranstubs.py has some hakey parsing code. I guess it can be 
> >> updated to do this.
> >> 
> >> i.e - look for 'type(tIS)' and add 'import tIS'
> > 
> > I pushed changes to generatefortranstubs.py for this (to
> > balay/reorg-f90-for-xlf branch). But there are errors. (I don't 
> > completely understand this issue yet)
> > 
>  
> > FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
> > /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
> > 
> >   6 |import PetscRandom
> > |   1
> > Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

Alternatively generatefortranstubs can traipse through 
src//f90-mod/petsc.h90 and look for type t 
definitions and build a list of petsc types that way, but at that point we’re 
halfway to writing our own compiler.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 16:23, Jacob Faibussowitsch  wrote:
> 
> So I have hit a bit of a wall. I am able to pull out all of the types for a 
> subroutine declaration but I cannot determine which types are classes since 
> those are the ones that need to be imported. For example:
> 
> subroutine PetscMatlabEngineDestroy(a,z)  
> 
>PetscMatlabEngine a ! PetscMatlabEngine
> PetscErrorCode z
> end subroutine PetscMatlabEngineDestroy
> 
> Is compiles completely fine but
> 
> subroutine PetscRandomDestroy(a,z)
> 
>PetscRandom a ! PetscRandom
> PetscErrorCode z
> end subroutine PetscRandomDestroy
> 
> Requires an “import tPetscRandom”. Previously both of these had “import 
> petscsys” stuck in them.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
>> On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev > > wrote:
>> 
>> And with this change (alone) - the time to compile the f90 modules went (on 
>> pj01 testbox) from:
>> 
>> 0m48.676s
>> to
>> 0m15.053s
>> 
>> Satish
>> 
>> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
>> 
>>> Ah - the issue is 'import IS' vs 'import tIS'
>>> 
>>> Pushed a fix now. [and reverted my earlier source split change]. My build 
>>> goes through fine now.
>>> 
>>> Satish
>>> 
>>> On Tue, 16 Mar 2021, Barry Smith wrote:
>>> 
 
  Satish,
 
The  import tIS might only work because the IS is already defined in 
 the same file so the compiler can pull in just part of the use 
 petscisdefdummy ? If the original module that contains the PetscRandom is 
 in a different file then I don't see how the compiler can find and import 
 PetscRandom. Is there a version of import where you also list the module 
 (file) that the beast is from so the compiler can get it from that module?
 
  Barry
 
Do both the manually generated petsc*mod.F90 and the automatically 
 generated files need to be switched to not use use everywhere? Or is it 
 enough just to fix the manual ones for now and not mess with sowing?
 
 
 
> On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
> mailto:petsc-dev@mcs.anl.gov>> wrote:
> 
> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> 
>> 
>> 
>> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>> 
>>> Ok something I have gotten to work, but only doing things by hand in 
>>> petscvecmod.F90:
>>> 
>>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
>>> b/src/vec/f90-mod/petscvecmod.F90
>>> index 0c447156b9..ef3e2e2e55 100644
>>> --- a/src/vec/f90-mod/petscvecmod.F90
>>> +++ b/src/vec/f90-mod/petscvecmod.F90
>>> module petscisdef
>>> use petscisdefdummy
>>> interface operator(.ne.)
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + import tIS
>>>  logical isnotequal 
>>> 
>>>  type(tIS), intent(in) :: A,B
>> 
>> generatefortranstubs.py has some hakey parsing code. I guess it can be 
>> updated to do this.
>> 
>> i.e - look for 'type(tIS)' and add 'import tIS'
> 
> I pushed changes to generatefortranstubs.py for this (to
> balay/reorg-f90-for-xlf branch). But there are errors. (I don't 
> completely understand this issue yet)
> 
 
> FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
> 
>   6 |import PetscRandom
> |   1
> Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not 
> exist.
> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
> 
>   7 |PetscRandom a ! PetscRandom
> |  1
> Error: Derived type ‘tpetscrandom’ at (1) is being used before it is 
> defined
> 
> 
> 
> Satish
> 
>> 
>>> end function isnotequal
>>> 
>>> This works for everything wrapped in a module which contains an 
>>> interface block. For dangling functions the following works:
>> 
>> Hm - maybe these are only on the custom side [and not ftn-auto side]
>> 
>> Satish
>>> 
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + use petscisdef, only: tIs
>> 
>> 
>>>  logical isnotequal
>>>  type(tIS), intent(in) :: A,B

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

I have this list in my branch [this list is created based on compiler errors - 
from the auto-generated fortran stubs - so perhaps incomplete].

Can you try my branch and see if that change is sufficient for xlf?

Satish

-

+ptypes = [
+  'PetscRandom',
+  'IS',
+  'Vec',
+  'PetscViewer',
+  'PetscOptions',
+  'PetscSubcomm',
+  'ISColoring',
+  'PetscSection',
+  'VecTagger',
+  'MatNullSpace',
+  'Mat',
+  'MatFDColoring',
+  'VecScatter',
+  'DMLabel',
+  'DM',
+  'PetscSectionSym',
+  'PetscSF',
+  'DMPlexCellRefiner',
+  'KSP',
+  'PC',
+  'KSPGuess',
+  'PetscConvEst',
+  'SNES',
+  'TSTrajectory',
+  'TS',
+  'TSAdapt',
+]


On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:

> So I have hit a bit of a wall. I am able to pull out all of the types for a 
> subroutine declaration but I cannot determine which types are classes since 
> those are the ones that need to be imported. For example:
> 
> subroutine PetscMatlabEngineDestroy(a,z)  
> 
>PetscMatlabEngine a ! PetscMatlabEngine
> PetscErrorCode z
> end subroutine PetscMatlabEngineDestroy
> 
> Is compiles completely fine but
> 
> subroutine PetscRandomDestroy(a,z)
> 
>PetscRandom a ! PetscRandom
> PetscErrorCode z
> end subroutine PetscRandomDestroy
> 
> Requires an “import tPetscRandom”. Previously both of these had “import 
> petscsys” stuck in them.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev 
> >  wrote:
> > 
> > And with this change (alone) - the time to compile the f90 modules went (on 
> > pj01 testbox) from:
> > 
> > 0m48.676s
> > to
> > 0m15.053s
> > 
> > Satish
> > 
> > On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> > 
> >> Ah - the issue is 'import IS' vs 'import tIS'
> >> 
> >> Pushed a fix now. [and reverted my earlier source split change]. My build 
> >> goes through fine now.
> >> 
> >> Satish
> >> 
> >> On Tue, 16 Mar 2021, Barry Smith wrote:
> >> 
> >>> 
> >>>  Satish,
> >>> 
> >>>The  import tIS might only work because the IS is already defined in 
> >>> the same file so the compiler can pull in just part of the use 
> >>> petscisdefdummy ? If the original module that contains the PetscRandom is 
> >>> in a different file then I don't see how the compiler can find and import 
> >>> PetscRandom. Is there a version of import where you also list the module 
> >>> (file) that the beast is from so the compiler can get it from that module?
> >>> 
> >>>  Barry
> >>> 
> >>>Do both the manually generated petsc*mod.F90 and the automatically 
> >>> generated files need to be switched to not use use everywhere? Or is it 
> >>> enough just to fix the manual ones for now and not mess with sowing?
> >>> 
> >>> 
> >>> 
>  On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
>   wrote:
>  
>  On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
>  
> > 
> > 
> > Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> > 
> >> Ok something I have gotten to work, but only doing things by hand in 
> >> petscvecmod.F90:
> >> 
> >> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> >> b/src/vec/f90-mod/petscvecmod.F90
> >> index 0c447156b9..ef3e2e2e55 100644
> >> --- a/src/vec/f90-mod/petscvecmod.F90
> >> +++ b/src/vec/f90-mod/petscvecmod.F90
> >> module petscisdef
> >> use petscisdefdummy
> >> interface operator(.ne.)
> >> function isnotequal(A,B)
> >> -  use petscisdefdummy
> >> + import tIS
> >>  logical isnotequal
> >>  
> >>  type(tIS), intent(in) :: A,B
> > 
> > generatefortranstubs.py has some hakey parsing code. I guess it can be 
> > updated to do this.
> > 
> > i.e - look for 'type(tIS)' and add 'import tIS'
>  
>  I pushed changes to generatefortranstubs.py for this (to
>  balay/reorg-f90-for-xlf branch). But there are errors. (I don't 
>  completely understand this issue yet)
>  
> >>> 
>  FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
>  /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
>  
>    6 |import PetscRandom
>  |   1
>  Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not 
>  exist.
>  /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
>  
>    7 |PetscRandom a ! PetscRandom
>  |  1
>  Error: Derived type ‘tpetscrandom’ at (1) is being used before it is 
>  defined
>  
>  
>  
>  Satish
>  
> > 
> >> end function isnotequal
> >> 
> >> This works for everything wrapped in a module which contain

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

So I have hit a bit of a wall. I am able to pull out all of the types for a 
subroutine declaration but I cannot determine which types are classes since 
those are the ones that need to be imported. For example:

subroutine PetscMatlabEngineDestroy(a,z)
  
   PetscMatlabEngine a ! PetscMatlabEngine
PetscErrorCode z
end subroutine PetscMatlabEngineDestroy

Is compiles completely fine but

subroutine PetscRandomDestroy(a,z)  
  
   PetscRandom a ! PetscRandom
PetscErrorCode z
end subroutine PetscRandomDestroy

Requires an “import tPetscRandom”. Previously both of these had “import 
petscsys” stuck in them.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 16:13, Satish Balay via petsc-dev  
> wrote:
> 
> And with this change (alone) - the time to compile the f90 modules went (on 
> pj01 testbox) from:
> 
> 0m48.676s
> to
> 0m15.053s
> 
> Satish
> 
> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> 
>> Ah - the issue is 'import IS' vs 'import tIS'
>> 
>> Pushed a fix now. [and reverted my earlier source split change]. My build 
>> goes through fine now.
>> 
>> Satish
>> 
>> On Tue, 16 Mar 2021, Barry Smith wrote:
>> 
>>> 
>>>  Satish,
>>> 
>>>The  import tIS might only work because the IS is already defined in the 
>>> same file so the compiler can pull in just part of the use petscisdefdummy 
>>> ? If the original module that contains the PetscRandom is in a different 
>>> file then I don't see how the compiler can find and import PetscRandom. Is 
>>> there a version of import where you also list the module (file) that the 
>>> beast is from so the compiler can get it from that module?
>>> 
>>>  Barry
>>> 
>>>Do both the manually generated petsc*mod.F90 and the automatically 
>>> generated files need to be switched to not use use everywhere? Or is it 
>>> enough just to fix the manual ones for now and not mess with sowing?
>>> 
>>> 
>>> 
 On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
  wrote:
 
 On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
 
> 
> 
> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> 
>> Ok something I have gotten to work, but only doing things by hand in 
>> petscvecmod.F90:
>> 
>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
>> b/src/vec/f90-mod/petscvecmod.F90
>> index 0c447156b9..ef3e2e2e55 100644
>> --- a/src/vec/f90-mod/petscvecmod.F90
>> +++ b/src/vec/f90-mod/petscvecmod.F90
>> module petscisdef
>> use petscisdefdummy
>> interface operator(.ne.)
>> function isnotequal(A,B)
>> -  use petscisdefdummy
>> + import tIS
>>  logical isnotequal  
>>
>>  type(tIS), intent(in) :: A,B
> 
> generatefortranstubs.py has some hakey parsing code. I guess it can be 
> updated to do this.
> 
> i.e - look for 'type(tIS)' and add 'import tIS'
 
 I pushed changes to generatefortranstubs.py for this (to
 balay/reorg-f90-for-xlf branch). But there are errors. (I don't completely 
 understand this issue yet)
 
>>> 
 FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
 /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
 
   6 |import PetscRandom
 |   1
 Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not exist.
 /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
 
   7 |PetscRandom a ! PetscRandom
 |  1
 Error: Derived type ‘tpetscrandom’ at (1) is being used before it is 
 defined
 
 
 
 Satish
 
> 
>> end function isnotequal
>> 
>> This works for everything wrapped in a module which contains an 
>> interface block. For dangling functions the following works:
> 
> Hm - maybe these are only on the custom side [and not ftn-auto side]
> 
> Satish
>> 
>> function isnotequal(A,B)
>> -  use petscisdefdummy
>> + use petscisdef, only: tIs
> 
> 
>>  logical isnotequal
>>  type(tIS), intent(in) :: A,B
>>  isnotequal = (A%v .ne. B%v)
>> end function isnotequal
>> 
>> Do our fortran stub generators have any notion of types? Specifically 
>> types that originate from petsc?
>> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>>> On Mar 16, 2021, at 11:11, Satish Balay  wrote:
>>> 
>>> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>>> 
> My [partial] change is in branch balay/reorg-f90-for-xlf
 
 Satish is this branch pushed? I’d like to send it 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

And with this change (alone) - the time to compile the f90 modules went (on 
pj01 testbox) from:

0m48.676s
to
0m15.053s

Satish

On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:

> Ah - the issue is 'import IS' vs 'import tIS'
> 
> Pushed a fix now. [and reverted my earlier source split change]. My build 
> goes through fine now.
> 
> Satish
> 
> On Tue, 16 Mar 2021, Barry Smith wrote:
> 
> > 
> >   Satish,
> > 
> > The  import tIS might only work because the IS is already defined in 
> > the same file so the compiler can pull in just part of the use 
> > petscisdefdummy ? If the original module that contains the PetscRandom is 
> > in a different file then I don't see how the compiler can find and import 
> > PetscRandom. Is there a version of import where you also list the module 
> > (file) that the beast is from so the compiler can get it from that module?
> > 
> >   Barry
> > 
> > Do both the manually generated petsc*mod.F90 and the automatically 
> > generated files need to be switched to not use use everywhere? Or is it 
> > enough just to fix the manual ones for now and not mess with sowing?
> > 
> > 
> > 
> > > On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
> > >  wrote:
> > > 
> > > On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> > > 
> > >> 
> > >> 
> > >> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> > >> 
> > >>> Ok something I have gotten to work, but only doing things by hand in 
> > >>> petscvecmod.F90:
> > >>> 
> > >>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> > >>> b/src/vec/f90-mod/petscvecmod.F90
> > >>> index 0c447156b9..ef3e2e2e55 100644
> > >>> --- a/src/vec/f90-mod/petscvecmod.F90
> > >>> +++ b/src/vec/f90-mod/petscvecmod.F90
> > >>> module petscisdef
> > >>> use petscisdefdummy
> > >>> interface operator(.ne.)
> > >>> function isnotequal(A,B)
> > >>> -  use petscisdefdummy
> > >>> + import tIS
> > >>>   logical isnotequal
> > >>>  
> > >>>   type(tIS), intent(in) :: A,B
> > >> 
> > >> generatefortranstubs.py has some hakey parsing code. I guess it can be 
> > >> updated to do this.
> > >> 
> > >> i.e - look for 'type(tIS)' and add 'import tIS'
> > > 
> > > I pushed changes to generatefortranstubs.py for this (to
> > > balay/reorg-f90-for-xlf branch). But there are errors. (I don't 
> > > completely understand this issue yet)
> > > 
> >  
> > >  FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
> > > /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
> > > 
> > >6 |import PetscRandom
> > >  |   1
> > > Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not 
> > > exist.
> > > /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
> > > 
> > >7 |PetscRandom a ! PetscRandom
> > >  |  1
> > > Error: Derived type ‘tpetscrandom’ at (1) is being used before it is 
> > > defined
> > > 
> > > 
> > > 
> > > Satish
> > > 
> > >> 
> > >>> end function isnotequal
> > >>> 
> > >>> This works for everything wrapped in a module which contains an 
> > >>> interface block. For dangling functions the following works:
> > >> 
> > >> Hm - maybe these are only on the custom side [and not ftn-auto side]
> > >> 
> > >> Satish
> > >>> 
> > >>> function isnotequal(A,B)
> > >>> -  use petscisdefdummy
> > >>> + use petscisdef, only: tIs
> > >> 
> > >> 
> > >>>   logical isnotequal
> > >>>   type(tIS), intent(in) :: A,B
> > >>>   isnotequal = (A%v .ne. B%v)
> > >>> end function isnotequal
> > >>> 
> > >>> Do our fortran stub generators have any notion of types? Specifically 
> > >>> types that originate from petsc?
> > >>> 
> > >>> Best regards,
> > >>> 
> > >>> Jacob Faibussowitsch
> > >>> (Jacob Fai - booss - oh - vitch)
> > >>> Cell: (312) 694-3391
> > >>> 
> >  On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> >  
> >  On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> >  
> > >> My [partial] change is in branch balay/reorg-f90-for-xlf
> > > 
> > > Satish is this branch pushed? I’d like to send it to the ibm folks to 
> > > see if it works for them as well.
> >  
> >  Sorry - pushed now. From what I remember - it didn't work.
> >  
> >  Satish
> >  
> >  
> > > They also added this extra follow up:
> > > 
> > > The change we did in the source files is to replace all the "use 
> > > pet*" statements in all the Interface blocks with IMPORT statement.
> > > 
> > > The nature of this workaround is to reduce the number of symbols that 
> > > the compiler have to create, which exceeded the limitation and caused 
> > > ICE.
> > > 
> > > Every USE statement in an interface block opens up the module symbol 
> > > file and reads all the symbols from it and creates an entity for each 
> > > symbol in compiler. This is unnecessary when the modul

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

Ah - the issue is 'import IS' vs 'import tIS'

Pushed a fix now. [and reverted my earlier source split change]. My build goes 
through fine now.

Satish

On Tue, 16 Mar 2021, Barry Smith wrote:

> 
>   Satish,
> 
> The  import tIS might only work because the IS is already defined in the 
> same file so the compiler can pull in just part of the use petscisdefdummy ? 
> If the original module that contains the PetscRandom is in a different file 
> then I don't see how the compiler can find and import PetscRandom. Is there a 
> version of import where you also list the module (file) that the beast is 
> from so the compiler can get it from that module?
> 
>   Barry
> 
> Do both the manually generated petsc*mod.F90 and the automatically 
> generated files need to be switched to not use use everywhere? Or is it 
> enough just to fix the manual ones for now and not mess with sowing?
> 
> 
> 
> > On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
> >  wrote:
> > 
> > On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> > 
> >> 
> >> 
> >> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> >> 
> >>> Ok something I have gotten to work, but only doing things by hand in 
> >>> petscvecmod.F90:
> >>> 
> >>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> >>> b/src/vec/f90-mod/petscvecmod.F90
> >>> index 0c447156b9..ef3e2e2e55 100644
> >>> --- a/src/vec/f90-mod/petscvecmod.F90
> >>> +++ b/src/vec/f90-mod/petscvecmod.F90
> >>> module petscisdef
> >>> use petscisdefdummy
> >>> interface operator(.ne.)
> >>> function isnotequal(A,B)
> >>> -  use petscisdefdummy
> >>> + import tIS
> >>>   logical isnotequal  
> >>>
> >>>   type(tIS), intent(in) :: A,B
> >> 
> >> generatefortranstubs.py has some hakey parsing code. I guess it can be 
> >> updated to do this.
> >> 
> >> i.e - look for 'type(tIS)' and add 'import tIS'
> > 
> > I pushed changes to generatefortranstubs.py for this (to
> > balay/reorg-f90-for-xlf branch). But there are errors. (I don't completely 
> > understand this issue yet)
> > 
>  
> >  FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
> > /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
> > 
> >6 |import PetscRandom
> >  |   1
> > Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not exist.
> > /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
> > 
> >7 |PetscRandom a ! PetscRandom
> >  |  1
> > Error: Derived type ‘tpetscrandom’ at (1) is being used before it is defined
> > 
> > 
> > 
> > Satish
> > 
> >> 
> >>> end function isnotequal
> >>> 
> >>> This works for everything wrapped in a module which contains an interface 
> >>> block. For dangling functions the following works:
> >> 
> >> Hm - maybe these are only on the custom side [and not ftn-auto side]
> >> 
> >> Satish
> >>> 
> >>> function isnotequal(A,B)
> >>> -  use petscisdefdummy
> >>> + use petscisdef, only: tIs
> >> 
> >> 
> >>>   logical isnotequal
> >>>   type(tIS), intent(in) :: A,B
> >>>   isnotequal = (A%v .ne. B%v)
> >>> end function isnotequal
> >>> 
> >>> Do our fortran stub generators have any notion of types? Specifically 
> >>> types that originate from petsc?
> >>> 
> >>> Best regards,
> >>> 
> >>> Jacob Faibussowitsch
> >>> (Jacob Fai - booss - oh - vitch)
> >>> Cell: (312) 694-3391
> >>> 
>  On Mar 16, 2021, at 11:11, Satish Balay  wrote:
>  
>  On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>  
> >> My [partial] change is in branch balay/reorg-f90-for-xlf
> > 
> > Satish is this branch pushed? I’d like to send it to the ibm folks to 
> > see if it works for them as well.
>  
>  Sorry - pushed now. From what I remember - it didn't work.
>  
>  Satish
>  
>  
> > They also added this extra follow up:
> > 
> > The change we did in the source files is to replace all the "use pet*" 
> > statements in all the Interface blocks with IMPORT statement.
> > 
> > The nature of this workaround is to reduce the number of symbols that 
> > the compiler have to create, which exceeded the limitation and caused 
> > ICE.
> > 
> > Every USE statement in an interface block opens up the module symbol 
> > file and reads all the symbols from it and creates an entity for each 
> > symbol in compiler. This is unnecessary when the module already has the 
> > same USE statement in the module scope. Instead, users can use IMPORT 
> > statement to make the module symbols accessible inside interface face 
> > blocks.
> > 
> > With the change, the compiler would only read the module symbol file 
> > once and create one set of symbols where the old code reads the module 
> > symbol files as many times as the number of the USE statements in 
> > Interface blocks and create

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

> Do both the manually generated petsc*mod.F90 and the automatically generated 
> files need to be switched to not use use everywhere?
It seems both might need to be changed. I changed petscvecmod, petscsysmod and 
petscmatmod, and whilst petscvecmod now compiled petscmatmod still crashed.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 14:41, Barry Smith  wrote:
> 
> 
>  Satish,
> 
>The  import tIS might only work because the IS is already defined in the 
> same file so the compiler can pull in just part of the use petscisdefdummy ? 
> If the original module that contains the PetscRandom is in a different file 
> then I don't see how the compiler can find and import PetscRandom. Is there a 
> version of import where you also list the module (file) that the beast is 
> from so the compiler can get it from that module?
> 
>  Barry
> 
>Do both the manually generated petsc*mod.F90 and the automatically 
> generated files need to be switched to not use use everywhere? Or is it 
> enough just to fix the manual ones for now and not mess with sowing?
> 
> 
> 
>> On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
>>  wrote:
>> 
>> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
>> 
>>> 
>>> 
>>> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>>> 
 Ok something I have gotten to work, but only doing things by hand in 
 petscvecmod.F90:
 
 diff --git a/src/vec/f90-mod/petscvecmod.F90 
 b/src/vec/f90-mod/petscvecmod.F90
 index 0c447156b9..ef3e2e2e55 100644
 --- a/src/vec/f90-mod/petscvecmod.F90
 +++ b/src/vec/f90-mod/petscvecmod.F90
 module petscisdef
 use petscisdefdummy
 interface operator(.ne.)
 function isnotequal(A,B)
 -  use petscisdefdummy
 + import tIS
  logical isnotequal
  
  type(tIS), intent(in) :: A,B
>>> 
>>> generatefortranstubs.py has some hakey parsing code. I guess it can be 
>>> updated to do this.
>>> 
>>> i.e - look for 'type(tIS)' and add 'import tIS'
>> 
>> I pushed changes to generatefortranstubs.py for this (to
>> balay/reorg-f90-for-xlf branch). But there are errors. (I don't completely 
>> understand this issue yet)
>> 
> 
>> FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
>> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
>> 
>>   6 |import PetscRandom
>> |   1
>> Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not exist.
>> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
>> 
>>   7 |PetscRandom a ! PetscRandom
>> |  1
>> Error: Derived type ‘tpetscrandom’ at (1) is being used before it is defined
>> 
>> 
>> 
>> Satish
>> 
>>> 
 end function isnotequal
 
 This works for everything wrapped in a module which contains an interface 
 block. For dangling functions the following works:
>>> 
>>> Hm - maybe these are only on the custom side [and not ftn-auto side]
>>> 
>>> Satish
 
 function isnotequal(A,B)
 -  use petscisdefdummy
 + use petscisdef, only: tIs
>>> 
>>> 
  logical isnotequal
  type(tIS), intent(in) :: A,B
  isnotequal = (A%v .ne. B%v)
 end function isnotequal
 
 Do our fortran stub generators have any notion of types? Specifically 
 types that originate from petsc?
 
 Best regards,
 
 Jacob Faibussowitsch
 (Jacob Fai - booss - oh - vitch)
 Cell: (312) 694-3391
 
> On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> 
> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> 
>>> My [partial] change is in branch balay/reorg-f90-for-xlf
>> 
>> Satish is this branch pushed? I’d like to send it to the ibm folks to 
>> see if it works for them as well.
> 
> Sorry - pushed now. From what I remember - it didn't work.
> 
> Satish
> 
> 
>> They also added this extra follow up:
>> 
>> The change we did in the source files is to replace all the "use pet*" 
>> statements in all the Interface blocks with IMPORT statement.
>> 
>> The nature of this workaround is to reduce the number of symbols that 
>> the compiler have to create, which exceeded the limitation and caused 
>> ICE.
>> 
>> Every USE statement in an interface block opens up the module symbol 
>> file and reads all the symbols from it and creates an entity for each 
>> symbol in compiler. This is unnecessary when the module already has the 
>> same USE statement in the module scope. Instead, users can use IMPORT 
>> statement to make the module symbols accessible inside interface face 
>> blocks.
>> 
>> With the change, the compiler would only read the module symbol file 
>> once and create one set of symbols where the old code reads the module

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

  Satish,

The  import tIS might only work because the IS is already defined in the 
same file so the compiler can pull in just part of the use petscisdefdummy ? If 
the original module that contains the PetscRandom is in a different file then I 
don't see how the compiler can find and import PetscRandom. Is there a version 
of import where you also list the module (file) that the beast is from so the 
compiler can get it from that module?

  Barry

Do both the manually generated petsc*mod.F90 and the automatically 
generated files need to be switched to not use use everywhere? Or is it enough 
just to fix the manual ones for now and not mess with sowing?



> On Mar 16, 2021, at 2:34 PM, Satish Balay via petsc-dev 
>  wrote:
> 
> On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:
> 
>> 
>> 
>> Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>> 
>>> Ok something I have gotten to work, but only doing things by hand in 
>>> petscvecmod.F90:
>>> 
>>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
>>> b/src/vec/f90-mod/petscvecmod.F90
>>> index 0c447156b9..ef3e2e2e55 100644
>>> --- a/src/vec/f90-mod/petscvecmod.F90
>>> +++ b/src/vec/f90-mod/petscvecmod.F90
>>> module petscisdef
>>> use petscisdefdummy
>>> interface operator(.ne.)
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + import tIS
>>>   logical isnotequal
>>>  
>>>   type(tIS), intent(in) :: A,B
>> 
>> generatefortranstubs.py has some hakey parsing code. I guess it can be 
>> updated to do this.
>> 
>> i.e - look for 'type(tIS)' and add 'import tIS'
> 
> I pushed changes to generatefortranstubs.py for this (to
> balay/reorg-f90-for-xlf branch). But there are errors. (I don't completely 
> understand this issue yet)
> 
 
>  FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:
> 
>6 |import PetscRandom
>  |   1
> Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not exist.
> /home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:
> 
>7 |PetscRandom a ! PetscRandom
>  |  1
> Error: Derived type ‘tpetscrandom’ at (1) is being used before it is defined
> 
> 
> 
> Satish
> 
>> 
>>> end function isnotequal
>>> 
>>> This works for everything wrapped in a module which contains an interface 
>>> block. For dangling functions the following works:
>> 
>> Hm - maybe these are only on the custom side [and not ftn-auto side]
>> 
>> Satish
>>> 
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + use petscisdef, only: tIs
>> 
>> 
>>>   logical isnotequal
>>>   type(tIS), intent(in) :: A,B
>>>   isnotequal = (A%v .ne. B%v)
>>> end function isnotequal
>>> 
>>> Do our fortran stub generators have any notion of types? Specifically types 
>>> that originate from petsc?
>>> 
>>> Best regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> Cell: (312) 694-3391
>>> 
 On Mar 16, 2021, at 11:11, Satish Balay  wrote:
 
 On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
 
>> My [partial] change is in branch balay/reorg-f90-for-xlf
> 
> Satish is this branch pushed? I’d like to send it to the ibm folks to see 
> if it works for them as well.
 
 Sorry - pushed now. From what I remember - it didn't work.
 
 Satish
 
 
> They also added this extra follow up:
> 
> The change we did in the source files is to replace all the "use pet*" 
> statements in all the Interface blocks with IMPORT statement.
> 
> The nature of this workaround is to reduce the number of symbols that the 
> compiler have to create, which exceeded the limitation and caused ICE.
> 
> Every USE statement in an interface block opens up the module symbol file 
> and reads all the symbols from it and creates an entity for each symbol 
> in compiler. This is unnecessary when the module already has the same USE 
> statement in the module scope. Instead, users can use IMPORT statement to 
> make the module symbols accessible inside interface face blocks.
> 
> With the change, the compiler would only read the module symbol file once 
> and create one set of symbols where the old code reads the module symbol 
> files as many times as the number of the USE statements in Interface 
> blocks and create that many sets of duplicate symbols. Replacing those 
> USE statements with IMPORT statements also shortens the compile time 
> significantly.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
>> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
>>  wrote:
>> 
>> The only change I can get working (i.e avoid compile errors) is to split 
>> petscvecmod.F90 in

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

  Jacob,

  You very well may be right, splitting may not be needed or relevant. I 
only advocated it since for Mat it did fix a VM gfortran memory issue. How 
closely the IBM compiler and gfortran issues are I do not know. 

  My MR Mat split is read to to go into main.

  https://gitlab.com/petsc/petsc/-/merge_requests/3715 
  The MPI one is still 
needing some babysitting, hopefully it can pass the CI and get into main soon.

  Both should be in main before starting to muck with changing the use stuff 
for the IBM compiler.

  Satish, does your Vec file split pass the CI? If so I think we should also 
get it into main before messing with the use for the IBM compiler.


 Who has access to the wonky IBM compiler?  Is there any way to have general 
access so it can be added to the CI and we can maintain code that works with 
it? Or does someone need to fix things one at a time, first Vec, then Mat, then 
 


  Barry


> On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev 
>  wrote:
> 
> On Tue, 16 Mar 2021, Matthew Knepley wrote:
> 
>> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
>> wrote:
>> 
>>> To me the best solution is to first split the files when possible (it is
>>> often not or just too painful) and then adding imports if needed.
>>> 
>>> 
>>> I’m not sure I agree. Bear in mind my familiarity with fortran is very
>>> limited, but to me the “use” statement is similar to #include  in C
>>> or using namespace xxx in cpp.
>>> 
>>> Our fortran interface is not optimal code atm, it’s as if we put a
>>> "#include ” before every Vec function in the C source
>>> (assuming include guards are not used). Sure you can fix this by splitting
>>> the vec src files in two, but at the end of the day that’s a bandaid
>>> solution. I think I can get the import stuff to work reasonably well (a lot
>>> of the infrastructure to capture types is already present in the current
>>> generatefortranstubs).
>>> 
>> 
>> If you can get this to work, we should definitely do it. Was there a
>> question about getting it to work?
> 
> Likely these additional changes should be done starting with Barry's split-up 
> of sources.
> 
> Satish
> 
>> 
>>  Thanks,
>> 
>> Matt
>> 
>> 
>>> Best regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> Cell: (312) 694-3391
>>> 
>>> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
>>> 
>>> 
>>>   Jacob,
>>> 
>>> I split  the mat definitions in the MR
>>> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
>>> memory requirements enough to get builds through on some VM that failed
>>> previously ran out of memory (with gfortran).
>>> 
>>> The petsc*mod.F90 files are all handwritten so it is ok to manually
>>> change them with imports if that helps the IBM compiler. To me the best
>>> solution is to first split the files when possible (it is often not or just
>>> too painful) and then adding imports if needed.
>>> 
>>> Note also the MR
>>> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
>>> a great deal with the IBM compiler or not.
>>> 
>>> Thanks
>>> 
>>> Barry
>>> 
>>> 
>>> 
>>> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
>>> wrote:
>>> 
>>> Ok something I have gotten to work, but only doing things by hand in
>>> petscvecmod.F90:
>>> 
>>> diff --git a/src/vec/f90-mod/petscvecmod.F90
>>> b/src/vec/f90-mod/petscvecmod.F90
>>> index 0c447156b9..ef3e2e2e55 100644
>>> --- a/src/vec/f90-mod/petscvecmod.F90
>>> +++ b/src/vec/f90-mod/petscvecmod.F90
>>> module petscisdef
>>> use petscisdefdummy
>>> interface operator(.ne.)
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + import tIS
>>>   logical isnotequal
>>> 
>>>   type(tIS), intent(in) :: A,B
>>> end function isnotequal
>>> 
>>> This works for everything wrapped in a module which contains an interface
>>> block. For dangling functions the following works:
>>> 
>>> function isnotequal(A,B)
>>> -  use petscisdefdummy
>>> + use petscisdef, only: tIs
>>>   logical isnotequal
>>>   type(tIS), intent(in) :: A,B
>>>   isnotequal = (A%v .ne. B%v)
>>> end function isnotequal
>>> 
>>> Do our fortran stub generators have any notion of types? Specifically
>>> types that originate from petsc?
>>> 
>>> Best regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> Cell: (312) 694-3391
>>> 
>>> On Mar 16, 2021, at 11:11, Satish Balay  wrote:
>>> 
>>> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>>> 
>>> My [partial] change is in branch balay/reorg-f90-for-xlf
>>> 
>>> 
>>> Satish is this branch pushed? I’d like to send it to the ibm folks to see
>>> if it works for them as well.
>>> 
>>> 
>>> Sorry - pushed now. From what I remember - it didn't work.
>>> 
>>> Satish
>>> 
>>> 
>>> They also added this extra follow up:
>>> 
>>> The change we did in the source files is to replace all the "use pet*"
>>> statements in all the Interface blocks with IMPORT statem

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Tue, 16 Mar 2021, Satish Balay via petsc-dev wrote:

> 
> 
>  Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> 
> > Ok something I have gotten to work, but only doing things by hand in 
> > petscvecmod.F90:
> > 
> > diff --git a/src/vec/f90-mod/petscvecmod.F90 
> > b/src/vec/f90-mod/petscvecmod.F90
> > index 0c447156b9..ef3e2e2e55 100644
> > --- a/src/vec/f90-mod/petscvecmod.F90
> > +++ b/src/vec/f90-mod/petscvecmod.F90
> >  module petscisdef
> >  use petscisdefdummy
> >  interface operator(.ne.)
> >  function isnotequal(A,B)
> > -  use petscisdefdummy
> > + import tIS
> >logical isnotequal   
> >   
> >type(tIS), intent(in) :: A,B
> 
> generatefortranstubs.py has some hakey parsing code. I guess it can be 
> updated to do this.
> 
> i.e - look for 'type(tIS)' and add 'import tIS'

I pushed changes to generatefortranstubs.py for this (to
balay/reorg-f90-for-xlf branch). But there are errors. (I don't completely 
understand this issue yet)

>>>
  FC arch-linux-c-debug/obj/sys/f90-mod/petscsysmod.o
/home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:6:19:

6 |import PetscRandom
  |   1
Error: Cannot IMPORT ‘type’ from host scoping unit at (1) - does not exist.
/home/balay/petsc/include/../src/sys/f90-mod/ftn-auto-interfaces/petscsys.h90:7:26:

7 |PetscRandom a ! PetscRandom
  |  1
Error: Derived type ‘tpetscrandom’ at (1) is being used before it is defined



Satish

> 
> >  end function isnotequal
> > 
> > This works for everything wrapped in a module which contains an interface 
> > block. For dangling functions the following works:
> 
> Hm - maybe these are only on the custom side [and not ftn-auto side]
> 
> Satish
> > 
> >  function isnotequal(A,B)
> > -  use petscisdefdummy
> > + use petscisdef, only: tIs
> 
> 
> >logical isnotequal
> >type(tIS), intent(in) :: A,B
> >isnotequal = (A%v .ne. B%v)
> >  end function isnotequal
> > 
> > Do our fortran stub generators have any notion of types? Specifically types 
> > that originate from petsc?
> > 
> > Best regards,
> > 
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > Cell: (312) 694-3391
> > 
> > > On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> > > 
> > > On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> > > 
> > >>> My [partial] change is in branch balay/reorg-f90-for-xlf
> > >> 
> > >> Satish is this branch pushed? I’d like to send it to the ibm folks to 
> > >> see if it works for them as well.
> > > 
> > > Sorry - pushed now. From what I remember - it didn't work.
> > > 
> > > Satish
> > > 
> > > 
> > >> They also added this extra follow up:
> > >> 
> > >> The change we did in the source files is to replace all the "use pet*" 
> > >> statements in all the Interface blocks with IMPORT statement.
> > >> 
> > >> The nature of this workaround is to reduce the number of symbols that 
> > >> the compiler have to create, which exceeded the limitation and caused 
> > >> ICE.
> > >> 
> > >> Every USE statement in an interface block opens up the module symbol 
> > >> file and reads all the symbols from it and creates an entity for each 
> > >> symbol in compiler. This is unnecessary when the module already has the 
> > >> same USE statement in the module scope. Instead, users can use IMPORT 
> > >> statement to make the module symbols accessible inside interface face 
> > >> blocks.
> > >> 
> > >> With the change, the compiler would only read the module symbol file 
> > >> once and create one set of symbols where the old code reads the module 
> > >> symbol files as many times as the number of the USE statements in 
> > >> Interface blocks and create that many sets of duplicate symbols. 
> > >> Replacing those USE statements with IMPORT statements also shortens the 
> > >> compile time significantly.
> > >> 
> > >> Best regards,
> > >> 
> > >> Jacob Faibussowitsch
> > >> (Jacob Fai - booss - oh - vitch)
> > >> Cell: (312) 694-3391
> > >> 
> > >>> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
> > >>>  wrote:
> > >>> 
> > >>> The only change I can get working (i.e avoid compile errors) is to 
> > >>> split petscvecmod.F90 into 2 source files - but I don't know if this 
> > >>> will help with xlf..
> > >>> 
> > >>> My [partial] change is in branch balay/reorg-f90-for-xlf
> > >>> 
> > >>> Satish
> > >>> 
> > >>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> > >>> 
> >  On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> >  
> > > Sure - once any change works locally [for gcc and xlf]
> > > 
> > > When I try - I get a bunch of errors.. [yet to digest them.]
> >  
> > > Can you please give the following source code workaround a try?
> > > Since there is already "use petscvecdefdummy" at the module 
> > > scope, one workaround might be to remove the unnecessary "use 
> > > petscv

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Tue, 16 Mar 2021, Matthew Knepley wrote:

> On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
> wrote:
> 
> > To me the best solution is to first split the files when possible (it is
> > often not or just too painful) and then adding imports if needed.
> >
> >
> > I’m not sure I agree. Bear in mind my familiarity with fortran is very
> > limited, but to me the “use” statement is similar to #include  in C
> > or using namespace xxx in cpp.
> >
> > Our fortran interface is not optimal code atm, it’s as if we put a
> >  "#include ” before every Vec function in the C source
> > (assuming include guards are not used). Sure you can fix this by splitting
> > the vec src files in two, but at the end of the day that’s a bandaid
> > solution. I think I can get the import stuff to work reasonably well (a lot
> > of the infrastructure to capture types is already present in the current
> > generatefortranstubs).
> >
> 
> If you can get this to work, we should definitely do it. Was there a
> question about getting it to work?

Likely these additional changes should be done starting with Barry's split-up 
of sources.

Satish

> 
>   Thanks,
> 
>  Matt
> 
> 
> > Best regards,
> >
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > Cell: (312) 694-3391
> >
> > On Mar 16, 2021, at 12:56, Barry Smith  wrote:
> >
> >
> >Jacob,
> >
> >  I split  the mat definitions in the MR
> > https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
> > memory requirements enough to get builds through on some VM that failed
> > previously ran out of memory (with gfortran).
> >
> >  The petsc*mod.F90 files are all handwritten so it is ok to manually
> > change them with imports if that helps the IBM compiler. To me the best
> > solution is to first split the files when possible (it is often not or just
> > too painful) and then adding imports if needed.
> >
> >  Note also the MR
> > https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
> > a great deal with the IBM compiler or not.
> >
> >  Thanks
> >
> >  Barry
> >
> >
> >
> > On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
> > wrote:
> >
> > Ok something I have gotten to work, but only doing things by hand in
> > petscvecmod.F90:
> >
> > diff --git a/src/vec/f90-mod/petscvecmod.F90
> > b/src/vec/f90-mod/petscvecmod.F90
> > index 0c447156b9..ef3e2e2e55 100644
> > --- a/src/vec/f90-mod/petscvecmod.F90
> > +++ b/src/vec/f90-mod/petscvecmod.F90
> >  module petscisdef
> >  use petscisdefdummy
> >  interface operator(.ne.)
> >  function isnotequal(A,B)
> > -  use petscisdefdummy
> > + import tIS
> >logical isnotequal
> >
> >type(tIS), intent(in) :: A,B
> >  end function isnotequal
> >
> > This works for everything wrapped in a module which contains an interface
> > block. For dangling functions the following works:
> >
> >  function isnotequal(A,B)
> > -  use petscisdefdummy
> > + use petscisdef, only: tIs
> >logical isnotequal
> >type(tIS), intent(in) :: A,B
> >isnotequal = (A%v .ne. B%v)
> >  end function isnotequal
> >
> > Do our fortran stub generators have any notion of types? Specifically
> > types that originate from petsc?
> >
> > Best regards,
> >
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > Cell: (312) 694-3391
> >
> > On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> >
> > On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> >
> > My [partial] change is in branch balay/reorg-f90-for-xlf
> >
> >
> > Satish is this branch pushed? I’d like to send it to the ibm folks to see
> > if it works for them as well.
> >
> >
> > Sorry - pushed now. From what I remember - it didn't work.
> >
> > Satish
> >
> >
> > They also added this extra follow up:
> >
> > The change we did in the source files is to replace all the "use pet*"
> > statements in all the Interface blocks with IMPORT statement.
> >
> > The nature of this workaround is to reduce the number of symbols that the
> > compiler have to create, which exceeded the limitation and caused ICE.
> >
> > Every USE statement in an interface block opens up the module symbol file
> > and reads all the symbols from it and creates an entity for each symbol in
> > compiler. This is unnecessary when the module already has the same USE
> > statement in the module scope. Instead, users can use IMPORT statement to
> > make the module symbols accessible inside interface face blocks.
> >
> > With the change, the compiler would only read the module symbol file once
> > and create one set of symbols where the old code reads the module symbol
> > files as many times as the number of the USE statements in Interface blocks
> > and create that many sets of duplicate symbols. Replacing those USE
> > statements with IMPORT statements also shortens the compile time
> > significantly.
> >
> > Best regards,
> >
> > Jacob Faibussowitsch
> > (Jacob Fai - booss - oh - vitch)
> > Cell: (312) 694-3391
> >
> > On Mar 3, 2021, at 13:43, Satish Balay via p

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Matthew Knepley]

On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch 
wrote:

> To me the best solution is to first split the files when possible (it is
> often not or just too painful) and then adding imports if needed.
>
>
> I’m not sure I agree. Bear in mind my familiarity with fortran is very
> limited, but to me the “use” statement is similar to #include  in C
> or using namespace xxx in cpp.
>
> Our fortran interface is not optimal code atm, it’s as if we put a
>  "#include ” before every Vec function in the C source
> (assuming include guards are not used). Sure you can fix this by splitting
> the vec src files in two, but at the end of the day that’s a bandaid
> solution. I think I can get the import stuff to work reasonably well (a lot
> of the infrastructure to capture types is already present in the current
> generatefortranstubs).
>

If you can get this to work, we should definitely do it. Was there a
question about getting it to work?

  Thanks,

 Matt


> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
>
> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
>
>
>Jacob,
>
>  I split  the mat definitions in the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
> memory requirements enough to get builds through on some VM that failed
> previously ran out of memory (with gfortran).
>
>  The petsc*mod.F90 files are all handwritten so it is ok to manually
> change them with imports if that helps the IBM compiler. To me the best
> solution is to first split the files when possible (it is often not or just
> too painful) and then adding imports if needed.
>
>  Note also the MR
> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
> a great deal with the IBM compiler or not.
>
>  Thanks
>
>  Barry
>
>
>
> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch 
> wrote:
>
> Ok something I have gotten to work, but only doing things by hand in
> petscvecmod.F90:
>
> diff --git a/src/vec/f90-mod/petscvecmod.F90
> b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
>  module petscisdef
>  use petscisdefdummy
>  interface operator(.ne.)
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + import tIS
>logical isnotequal
>
>type(tIS), intent(in) :: A,B
>  end function isnotequal
>
> This works for everything wrapped in a module which contains an interface
> block. For dangling functions the following works:
>
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + use petscisdef, only: tIs
>logical isnotequal
>type(tIS), intent(in) :: A,B
>isnotequal = (A%v .ne. B%v)
>  end function isnotequal
>
> Do our fortran stub generators have any notion of types? Specifically
> types that originate from petsc?
>
> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
>
> On Mar 16, 2021, at 11:11, Satish Balay  wrote:
>
> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>
> My [partial] change is in branch balay/reorg-f90-for-xlf
>
>
> Satish is this branch pushed? I’d like to send it to the ibm folks to see
> if it works for them as well.
>
>
> Sorry - pushed now. From what I remember - it didn't work.
>
> Satish
>
>
> They also added this extra follow up:
>
> The change we did in the source files is to replace all the "use pet*"
> statements in all the Interface blocks with IMPORT statement.
>
> The nature of this workaround is to reduce the number of symbols that the
> compiler have to create, which exceeded the limitation and caused ICE.
>
> Every USE statement in an interface block opens up the module symbol file
> and reads all the symbols from it and creates an entity for each symbol in
> compiler. This is unnecessary when the module already has the same USE
> statement in the module scope. Instead, users can use IMPORT statement to
> make the module symbols accessible inside interface face blocks.
>
> With the change, the compiler would only read the module symbol file once
> and create one set of symbols where the old code reads the module symbol
> files as many times as the number of the USE statements in Interface blocks
> and create that many sets of duplicate symbols. Replacing those USE
> statements with IMPORT statements also shortens the compile time
> significantly.
>
> Best regards,
>
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
>
> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev <
> petsc-dev@mcs.anl.gov> wrote:
>
> The only change I can get working (i.e avoid compile errors) is to split
> petscvecmod.F90 into 2 source files - but I don't know if this will help
> with xlf..
>
> My [partial] change is in branch balay/reorg-f90-for-xlf
>
> Satish
>
> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
>
> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
>
> Sure - once any change works locally

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

> To me the best solution is to first split the files when possible (it is 
> often not or just too painful) and then adding imports if needed. 

I’m not sure I agree. Bear in mind my familiarity with fortran is very limited, 
but to me the “use” statement is similar to #include  in C or using 
namespace xxx in cpp.

Our fortran interface is not optimal code atm, it’s as if we put a  "#include 
” before every Vec function in the C source (assuming include 
guards are not used). Sure you can fix this by splitting the vec src files in 
two, but at the end of the day that’s a bandaid solution. I think I can get the 
import stuff to work reasonably well (a lot of the infrastructure to capture 
types is already present in the current generatefortranstubs).

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 12:56, Barry Smith  wrote:
> 
> 
>Jacob,
> 
>  I split  the mat definitions in the MR 
> https://gitlab.com/petsc/petsc/-/merge_requests/3696 
>  and this reduced 
> memory requirements enough to get builds through on some VM that failed 
> previously ran out of memory (with gfortran). 
> 
>  The petsc*mod.F90 files are all handwritten so it is ok to manually 
> change them with imports if that helps the IBM compiler. To me the best 
> solution is to first split the files when possible (it is often not or just 
> too painful) and then adding imports if needed. 
> 
>  Note also the MR 
> https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs 
>  which may help a 
> great deal with the IBM compiler or not.
> 
>  Thanks
> 
>  Barry
> 
> 
> 
>> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch > > wrote:
>> 
>> Ok something I have gotten to work, but only doing things by hand in 
>> petscvecmod.F90:
>> 
>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
>> b/src/vec/f90-mod/petscvecmod.F90
>> index 0c447156b9..ef3e2e2e55 100644
>> --- a/src/vec/f90-mod/petscvecmod.F90
>> +++ b/src/vec/f90-mod/petscvecmod.F90
>>  module petscisdef
>>  use petscisdefdummy
>>  interface operator(.ne.)
>>  function isnotequal(A,B)
>> -  use petscisdefdummy
>> + import tIS
>>logical isnotequal
>>  
>>type(tIS), intent(in) :: A,B
>>  end function isnotequal
>> 
>> This works for everything wrapped in a module which contains an interface 
>> block. For dangling functions the following works:
>> 
>>  function isnotequal(A,B)
>> -  use petscisdefdummy
>> + use petscisdef, only: tIs
>>logical isnotequal
>>type(tIS), intent(in) :: A,B
>>isnotequal = (A%v .ne. B%v)
>>  end function isnotequal
>> 
>> Do our fortran stub generators have any notion of types? Specifically types 
>> that originate from petsc?
>> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>>> On Mar 16, 2021, at 11:11, Satish Balay >> > wrote:
>>> 
>>> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>>> 
> My [partial] change is in branch balay/reorg-f90-for-xlf
 
 Satish is this branch pushed? I’d like to send it to the ibm folks to see 
 if it works for them as well.
>>> 
>>> Sorry - pushed now. From what I remember - it didn't work.
>>> 
>>> Satish
>>> 
>>> 
 They also added this extra follow up:
 
 The change we did in the source files is to replace all the "use pet*" 
 statements in all the Interface blocks with IMPORT statement.
 
 The nature of this workaround is to reduce the number of symbols that the 
 compiler have to create, which exceeded the limitation and caused ICE.
 
 Every USE statement in an interface block opens up the module symbol file 
 and reads all the symbols from it and creates an entity for each symbol in 
 compiler. This is unnecessary when the module already has the same USE 
 statement in the module scope. Instead, users can use IMPORT statement to 
 make the module symbols accessible inside interface face blocks.
 
 With the change, the compiler would only read the module symbol file once 
 and create one set of symbols where the old code reads the module symbol 
 files as many times as the number of the USE statements in Interface 
 blocks and create that many sets of duplicate symbols. Replacing those USE 
 statements with IMPORT statements also shortens the compile time 
 significantly.
 
 Best regards,
 
 Jacob Faibussowitsch
 (Jacob Fai - booss - oh - vitch)
 Cell: (312) 694-3391
 
> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
> mailto:petsc-dev@mcs.anl.gov>> wrote:
> 
> The only change I can get working (i.e avoid compile errors) is to split 
> petscvecmod.F90 into 2 source files - but 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

 Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:

> Ok something I have gotten to work, but only doing things by hand in 
> petscvecmod.F90:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90 b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
>  module petscisdef
>  use petscisdefdummy
>  interface operator(.ne.)
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + import tIS
>logical isnotequal 
> 
>type(tIS), intent(in) :: A,B

generatefortranstubs.py has some hakey parsing code. I guess it can be updated 
to do this.

i.e - look for 'type(tIS)' and add 'import tIS'

>  end function isnotequal
> 
> This works for everything wrapped in a module which contains an interface 
> block. For dangling functions the following works:

Hm - maybe these are only on the custom side [and not ftn-auto side]

Satish
> 
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + use petscisdef, only: tIs


>logical isnotequal
>type(tIS), intent(in) :: A,B
>isnotequal = (A%v .ne. B%v)
>  end function isnotequal
> 
> Do our fortran stub generators have any notion of types? Specifically types 
> that originate from petsc?
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> > 
> > On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> > 
> >>> My [partial] change is in branch balay/reorg-f90-for-xlf
> >> 
> >> Satish is this branch pushed? I’d like to send it to the ibm folks to see 
> >> if it works for them as well.
> > 
> > Sorry - pushed now. From what I remember - it didn't work.
> > 
> > Satish
> > 
> > 
> >> They also added this extra follow up:
> >> 
> >> The change we did in the source files is to replace all the "use pet*" 
> >> statements in all the Interface blocks with IMPORT statement.
> >> 
> >> The nature of this workaround is to reduce the number of symbols that the 
> >> compiler have to create, which exceeded the limitation and caused ICE.
> >> 
> >> Every USE statement in an interface block opens up the module symbol file 
> >> and reads all the symbols from it and creates an entity for each symbol in 
> >> compiler. This is unnecessary when the module already has the same USE 
> >> statement in the module scope. Instead, users can use IMPORT statement to 
> >> make the module symbols accessible inside interface face blocks.
> >> 
> >> With the change, the compiler would only read the module symbol file once 
> >> and create one set of symbols where the old code reads the module symbol 
> >> files as many times as the number of the USE statements in Interface 
> >> blocks and create that many sets of duplicate symbols. Replacing those USE 
> >> statements with IMPORT statements also shortens the compile time 
> >> significantly.
> >> 
> >> Best regards,
> >> 
> >> Jacob Faibussowitsch
> >> (Jacob Fai - booss - oh - vitch)
> >> Cell: (312) 694-3391
> >> 
> >>> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
> >>>  wrote:
> >>> 
> >>> The only change I can get working (i.e avoid compile errors) is to split 
> >>> petscvecmod.F90 into 2 source files - but I don't know if this will help 
> >>> with xlf..
> >>> 
> >>> My [partial] change is in branch balay/reorg-f90-for-xlf
> >>> 
> >>> Satish
> >>> 
> >>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> >>> 
>  On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
>  
> > Sure - once any change works locally [for gcc and xlf]
> > 
> > When I try - I get a bunch of errors.. [yet to digest them.]
>  
> > Can you please give the following source code workaround a try?
> > Since there is already "use petscvecdefdummy" at the module scope, 
> > one workaround might be to remove the unnecessary "use 
> > petscvecdefdummy" in vecnotequal and vecequals 
> > and all similar procedures.
> > 
> > For example, the test case has:
> >  module petscvecdef
> >  use petscvecdefdummy
> > ...
> >  function vecnotequal(A,B)
> >use petscvecdefdummy
> >logical vecnotequal
> >type(tVec), intent(in) :: A,B
> >vecnotequal = (A%v .ne. B%v)
> >  end function
>  
>  
>  Ok - try this suggestion:
>  
>  diff --git a/src/vec/f90-mod/petscvecmod.F90 
>  b/src/vec/f90-mod/petscvecmod.F90
>  index 0c447156b9..81968c7ca1 100644
>  --- a/src/vec/f90-mod/petscvecmod.F90
>  +++ b/src/vec/f90-mod/petscvecmod.F90
>  @@ -77,7 +77,6 @@
> use petscvecdefdummy
> interface operator(.ne.)
>   function vecnotequal(A,B)
>  -use petscvecdefdummy
> logical vecnotequal
> type(tVec), intent(

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

   Jacob,

 I split  the mat definitions in the MR 
https://gitlab.com/petsc/petsc/-/merge_requests/3696 
 and this reduced memory 
requirements enough to get builds through on some VM that failed previously ran 
out of memory (with gfortran). 

 The petsc*mod.F90 files are all handwritten so it is ok to manually change 
them with imports if that helps the IBM compiler. To me the best solution is to 
first split the files when possible (it is often not or just too painful) and 
then adding imports if needed. 

 Note also the MR 
https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs 
 which may help a 
great deal with the IBM compiler or not.

 Thanks

 Barry



> On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch  
> wrote:
> 
> Ok something I have gotten to work, but only doing things by hand in 
> petscvecmod.F90:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90 b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..ef3e2e2e55 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
>  module petscisdef
>  use petscisdefdummy
>  interface operator(.ne.)
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + import tIS
>logical isnotequal 
> 
>type(tIS), intent(in) :: A,B
>  end function isnotequal
> 
> This works for everything wrapped in a module which contains an interface 
> block. For dangling functions the following works:
> 
>  function isnotequal(A,B)
> -  use petscisdefdummy
> + use petscisdef, only: tIs
>logical isnotequal
>type(tIS), intent(in) :: A,B
>isnotequal = (A%v .ne. B%v)
>  end function isnotequal
> 
> Do our fortran stub generators have any notion of types? Specifically types 
> that originate from petsc?
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
>> On Mar 16, 2021, at 11:11, Satish Balay > > wrote:
>> 
>> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
>> 
 My [partial] change is in branch balay/reorg-f90-for-xlf
>>> 
>>> Satish is this branch pushed? I’d like to send it to the ibm folks to see 
>>> if it works for them as well.
>> 
>> Sorry - pushed now. From what I remember - it didn't work.
>> 
>> Satish
>> 
>> 
>>> They also added this extra follow up:
>>> 
>>> The change we did in the source files is to replace all the "use pet*" 
>>> statements in all the Interface blocks with IMPORT statement.
>>> 
>>> The nature of this workaround is to reduce the number of symbols that the 
>>> compiler have to create, which exceeded the limitation and caused ICE.
>>> 
>>> Every USE statement in an interface block opens up the module symbol file 
>>> and reads all the symbols from it and creates an entity for each symbol in 
>>> compiler. This is unnecessary when the module already has the same USE 
>>> statement in the module scope. Instead, users can use IMPORT statement to 
>>> make the module symbols accessible inside interface face blocks.
>>> 
>>> With the change, the compiler would only read the module symbol file once 
>>> and create one set of symbols where the old code reads the module symbol 
>>> files as many times as the number of the USE statements in Interface blocks 
>>> and create that many sets of duplicate symbols. Replacing those USE 
>>> statements with IMPORT statements also shortens the compile time 
>>> significantly.
>>> 
>>> Best regards,
>>> 
>>> Jacob Faibussowitsch
>>> (Jacob Fai - booss - oh - vitch)
>>> Cell: (312) 694-3391
>>> 
 On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
 mailto:petsc-dev@mcs.anl.gov>> wrote:
 
 The only change I can get working (i.e avoid compile errors) is to split 
 petscvecmod.F90 into 2 source files - but I don't know if this will help 
 with xlf..
 
 My [partial] change is in branch balay/reorg-f90-for-xlf
 
 Satish
 
 On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
 
> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> 
>> Sure - once any change works locally [for gcc and xlf]
>> 
>> When I try - I get a bunch of errors.. [yet to digest them.]
> 
>> Can you please give the following source code workaround a try?
>> Since there is already "use petscvecdefdummy" at the module scope, 
>> one workaround might be to remove the unnecessary "use 
>> petscvecdefdummy" in vecnotequal and vecequals 
>> and all similar procedures.
>> 
>> For example, the test case has:
>>  module petscvecdef
>>  use petscvecdefdummy
>> ...
>>  function vecnotequal(A,B)
>>use petscvecdefdummy
>>logical vecnotequal
>>type(tVec), intent(in) :: A,B
>>vecnotequal 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

Ok something I have gotten to work, but only doing things by hand in 
petscvecmod.F90:

diff --git a/src/vec/f90-mod/petscvecmod.F90 b/src/vec/f90-mod/petscvecmod.F90
index 0c447156b9..ef3e2e2e55 100644
--- a/src/vec/f90-mod/petscvecmod.F90
+++ b/src/vec/f90-mod/petscvecmod.F90
 module petscisdef
 use petscisdefdummy
 interface operator(.ne.)
 function isnotequal(A,B)
-  use petscisdefdummy
+ import tIS
   logical isnotequal   
  
   type(tIS), intent(in) :: A,B
 end function isnotequal

This works for everything wrapped in a module which contains an interface 
block. For dangling functions the following works:

 function isnotequal(A,B)
-  use petscisdefdummy
+ use petscisdef, only: tIs
   logical isnotequal
   type(tIS), intent(in) :: A,B
   isnotequal = (A%v .ne. B%v)
 end function isnotequal

Do our fortran stub generators have any notion of types? Specifically types 
that originate from petsc?

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 16, 2021, at 11:11, Satish Balay  wrote:
> 
> On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
> 
>>> My [partial] change is in branch balay/reorg-f90-for-xlf
>> 
>> Satish is this branch pushed? I’d like to send it to the ibm folks to see if 
>> it works for them as well.
> 
> Sorry - pushed now. From what I remember - it didn't work.
> 
> Satish
> 
> 
>> They also added this extra follow up:
>> 
>> The change we did in the source files is to replace all the "use pet*" 
>> statements in all the Interface blocks with IMPORT statement.
>> 
>> The nature of this workaround is to reduce the number of symbols that the 
>> compiler have to create, which exceeded the limitation and caused ICE.
>> 
>> Every USE statement in an interface block opens up the module symbol file 
>> and reads all the symbols from it and creates an entity for each symbol in 
>> compiler. This is unnecessary when the module already has the same USE 
>> statement in the module scope. Instead, users can use IMPORT statement to 
>> make the module symbols accessible inside interface face blocks.
>> 
>> With the change, the compiler would only read the module symbol file once 
>> and create one set of symbols where the old code reads the module symbol 
>> files as many times as the number of the USE statements in Interface blocks 
>> and create that many sets of duplicate symbols. Replacing those USE 
>> statements with IMPORT statements also shortens the compile time 
>> significantly.
>> 
>> Best regards,
>> 
>> Jacob Faibussowitsch
>> (Jacob Fai - booss - oh - vitch)
>> Cell: (312) 694-3391
>> 
>>> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
>>>  wrote:
>>> 
>>> The only change I can get working (i.e avoid compile errors) is to split 
>>> petscvecmod.F90 into 2 source files - but I don't know if this will help 
>>> with xlf..
>>> 
>>> My [partial] change is in branch balay/reorg-f90-for-xlf
>>> 
>>> Satish
>>> 
>>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
>>> 
 On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
 
> Sure - once any change works locally [for gcc and xlf]
> 
> When I try - I get a bunch of errors.. [yet to digest them.]
 
> Can you please give the following source code workaround a try?
> Since there is already "use petscvecdefdummy" at the module scope, 
> one workaround might be to remove the unnecessary "use 
> petscvecdefdummy" in vecnotequal and vecequals 
> and all similar procedures.
> 
> For example, the test case has:
>  module petscvecdef
>  use petscvecdefdummy
> ...
>  function vecnotequal(A,B)
>use petscvecdefdummy
>logical vecnotequal
>type(tVec), intent(in) :: A,B
>vecnotequal = (A%v .ne. B%v)
>  end function
 
 
 Ok - try this suggestion:
 
 diff --git a/src/vec/f90-mod/petscvecmod.F90 
 b/src/vec/f90-mod/petscvecmod.F90
 index 0c447156b9..81968c7ca1 100644
 --- a/src/vec/f90-mod/petscvecmod.F90
 +++ b/src/vec/f90-mod/petscvecmod.F90
 @@ -77,7 +77,6 @@
use petscvecdefdummy
interface operator(.ne.)
  function vecnotequal(A,B)
 -use petscvecdefdummy
logical vecnotequal
type(tVec), intent(in) :: A,B
  end function
 
 
> 
 FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
 /home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:
 
  81 | type(tVec), intent(in) :: A,B
 |  1
 Error: Derived type ‘tvec’ at (1) is being used before it is defined
 /home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:
 
   2 |   use petscvecdef
 |  1
 Fatal Error: Cannot open

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:

> > My [partial] change is in branch balay/reorg-f90-for-xlf
> 
> Satish is this branch pushed? I’d like to send it to the ibm folks to see if 
> it works for them as well.

Sorry - pushed now. From what I remember - it didn't work.

Satish


> They also added this extra follow up:
> 
> The change we did in the source files is to replace all the "use pet*" 
> statements in all the Interface blocks with IMPORT statement.
> 
> The nature of this workaround is to reduce the number of symbols that the 
> compiler have to create, which exceeded the limitation and caused ICE.
> 
> Every USE statement in an interface block opens up the module symbol file and 
> reads all the symbols from it and creates an entity for each symbol in 
> compiler. This is unnecessary when the module already has the same USE 
> statement in the module scope. Instead, users can use IMPORT statement to 
> make the module symbols accessible inside interface face blocks.
> 
> With the change, the compiler would only read the module symbol file once and 
> create one set of symbols where the old code reads the module symbol files as 
> many times as the number of the USE statements in Interface blocks and create 
> that many sets of duplicate symbols. Replacing those USE statements with 
> IMPORT statements also shortens the compile time significantly.
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev 
> >  wrote:
> > 
> > The only change I can get working (i.e avoid compile errors) is to split 
> > petscvecmod.F90 into 2 source files - but I don't know if this will help 
> > with xlf..
> > 
> > My [partial] change is in branch balay/reorg-f90-for-xlf
> > 
> > Satish
> > 
> > On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> > 
> >> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> >> 
> >>> Sure - once any change works locally [for gcc and xlf]
> >>> 
> >>> When I try - I get a bunch of errors.. [yet to digest them.]
> >> 
> >>> Can you please give the following source code workaround a try?
> >>> Since there is already "use petscvecdefdummy" at the module scope, 
> >>> one workaround might be to remove the unnecessary "use 
> >>> petscvecdefdummy" in vecnotequal and vecequals 
> >>> and all similar procedures.
> >>> 
> >>> For example, the test case has:
> >>>   module petscvecdef
> >>>   use petscvecdefdummy
> >>> ...
> >>>   function vecnotequal(A,B)
> >>> use petscvecdefdummy
> >>> logical vecnotequal
> >>> type(tVec), intent(in) :: A,B
> >>> vecnotequal = (A%v .ne. B%v)
> >>>   end function
> >> 
> >> 
> >> Ok - try this suggestion:
> >> 
> >> diff --git a/src/vec/f90-mod/petscvecmod.F90 
> >> b/src/vec/f90-mod/petscvecmod.F90
> >> index 0c447156b9..81968c7ca1 100644
> >> --- a/src/vec/f90-mod/petscvecmod.F90
> >> +++ b/src/vec/f90-mod/petscvecmod.F90
> >> @@ -77,7 +77,6 @@
> >> use petscvecdefdummy
> >> interface operator(.ne.)
> >>   function vecnotequal(A,B)
> >> -use petscvecdefdummy
> >> logical vecnotequal
> >> type(tVec), intent(in) :: A,B
> >>   end function
> >> 
> >> 
> >>> 
> >>  FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
> >> /home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:
> >> 
> >>   81 | type(tVec), intent(in) :: A,B
> >>  |  1
> >> Error: Derived type ‘tvec’ at (1) is being used before it is defined
> >> /home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:
> >> 
> >>2 |   use petscvecdef
> >>  |  1
> >> Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1): 
> >> No such file or directory
> >> <<
> >> 
> >> Satish
> 
> 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

> My [partial] change is in branch balay/reorg-f90-for-xlf

Satish is this branch pushed? I’d like to send it to the ibm folks to see if it 
works for them as well. They also added this extra follow up:

The change we did in the source files is to replace all the "use pet*" 
statements in all the Interface blocks with IMPORT statement.

The nature of this workaround is to reduce the number of symbols that the 
compiler have to create, which exceeded the limitation and caused ICE.

Every USE statement in an interface block opens up the module symbol file and 
reads all the symbols from it and creates an entity for each symbol in 
compiler. This is unnecessary when the module already has the same USE 
statement in the module scope. Instead, users can use IMPORT statement to make 
the module symbols accessible inside interface face blocks.

With the change, the compiler would only read the module symbol file once and 
create one set of symbols where the old code reads the module symbol files as 
many times as the number of the USE statements in Interface blocks and create 
that many sets of duplicate symbols. Replacing those USE statements with IMPORT 
statements also shortens the compile time significantly.

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev  
> wrote:
> 
> The only change I can get working (i.e avoid compile errors) is to split 
> petscvecmod.F90 into 2 source files - but I don't know if this will help with 
> xlf..
> 
> My [partial] change is in branch balay/reorg-f90-for-xlf
> 
> Satish
> 
> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> 
>> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
>> 
>>> Sure - once any change works locally [for gcc and xlf]
>>> 
>>> When I try - I get a bunch of errors.. [yet to digest them.]
>> 
>>> Can you please give the following source code workaround a try?
>>> Since there is already "use petscvecdefdummy" at the module scope, one 
>>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
>>> vecnotequal and vecequals 
>>> and all similar procedures.
>>> 
>>> For example, the test case has:
>>>   module petscvecdef
>>>   use petscvecdefdummy
>>> ...
>>>   function vecnotequal(A,B)
>>> use petscvecdefdummy
>>> logical vecnotequal
>>> type(tVec), intent(in) :: A,B
>>> vecnotequal = (A%v .ne. B%v)
>>>   end function
>> 
>> 
>> Ok - try this suggestion:
>> 
>> diff --git a/src/vec/f90-mod/petscvecmod.F90 
>> b/src/vec/f90-mod/petscvecmod.F90
>> index 0c447156b9..81968c7ca1 100644
>> --- a/src/vec/f90-mod/petscvecmod.F90
>> +++ b/src/vec/f90-mod/petscvecmod.F90
>> @@ -77,7 +77,6 @@
>> use petscvecdefdummy
>> interface operator(.ne.)
>>   function vecnotequal(A,B)
>> -use petscvecdefdummy
>> logical vecnotequal
>> type(tVec), intent(in) :: A,B
>>   end function
>> 
>> 
>>> 
>>  FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
>> /home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:
>> 
>>   81 | type(tVec), intent(in) :: A,B
>>  |  1
>> Error: Derived type ‘tvec’ at (1) is being used before it is defined
>> /home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:
>> 
>>2 |   use petscvecdef
>>  |  1
>> Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1): 
>> No such file or directory
>> <<
>> 
>> Satish

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

The only change I can get working (i.e avoid compile errors) is to split 
petscvecmod.F90 into 2 source files - but I don't know if this will help with 
xlf..

My [partial] change is in branch balay/reorg-f90-for-xlf

Satish

On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:

> On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
> 
> > Sure - once any change works locally [for gcc and xlf]
> > 
> > When I try - I get a bunch of errors.. [yet to digest them.]
> 
> > > >>> Can you please give the following source code workaround a try?
> > > >>> Since there is already "use petscvecdefdummy" at the module scope, 
> > > >>> one workaround might be to remove the unnecessary "use 
> > > >>> petscvecdefdummy" in vecnotequal and vecequals 
> > > >>> and all similar procedures.
> > > >>> 
> > > >>> For example, the test case has:
> > > >>>module petscvecdef
> > > >>>use petscvecdefdummy
> > > >>> ...
> > > >>>function vecnotequal(A,B)
> > > >>>  use petscvecdefdummy
> > > >>>  logical vecnotequal
> > > >>>  type(tVec), intent(in) :: A,B
> > > >>>  vecnotequal = (A%v .ne. B%v)
> > > >>>end function
> 
> 
> Ok - try this suggestion:
> 
> diff --git a/src/vec/f90-mod/petscvecmod.F90 b/src/vec/f90-mod/petscvecmod.F90
> index 0c447156b9..81968c7ca1 100644
> --- a/src/vec/f90-mod/petscvecmod.F90
> +++ b/src/vec/f90-mod/petscvecmod.F90
> @@ -77,7 +77,6 @@
>  use petscvecdefdummy
>  interface operator(.ne.)
>function vecnotequal(A,B)
> -use petscvecdefdummy
>  logical vecnotequal
>  type(tVec), intent(in) :: A,B
>end function
> 
> 
> >
>   FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
> /home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:
> 
>81 | type(tVec), intent(in) :: A,B
>   |  1
> Error: Derived type ‘tvec’ at (1) is being used before it is defined
> /home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:
> 
> 2 |   use petscvecdef
>   |  1
> Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1): No 
> such file or directory
> <<
> 
> Satish

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:

> Sure - once any change works locally [for gcc and xlf]
> 
> When I try - I get a bunch of errors.. [yet to digest them.]

> > >>> Can you please give the following source code workaround a try?
> > >>> Since there is already "use petscvecdefdummy" at the module scope, one 
> > >>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
> > >>> vecnotequal and vecequals 
> > >>> and all similar procedures.
> > >>> 
> > >>> For example, the test case has:
> > >>>module petscvecdef
> > >>>use petscvecdefdummy
> > >>> ...
> > >>>function vecnotequal(A,B)
> > >>>  use petscvecdefdummy
> > >>>  logical vecnotequal
> > >>>  type(tVec), intent(in) :: A,B
> > >>>  vecnotequal = (A%v .ne. B%v)
> > >>>end function


Ok - try this suggestion:

diff --git a/src/vec/f90-mod/petscvecmod.F90 b/src/vec/f90-mod/petscvecmod.F90
index 0c447156b9..81968c7ca1 100644
--- a/src/vec/f90-mod/petscvecmod.F90
+++ b/src/vec/f90-mod/petscvecmod.F90
@@ -77,7 +77,6 @@
 use petscvecdefdummy
 interface operator(.ne.)
   function vecnotequal(A,B)
-use petscvecdefdummy
 logical vecnotequal
 type(tVec), intent(in) :: A,B
   end function


>
  FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
/home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:

   81 | type(tVec), intent(in) :: A,B
  |  1
Error: Derived type ‘tvec’ at (1) is being used before it is defined
/home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:

2 |   use petscvecdef
  |  1
Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1): No 
such file or directory
<<

Satish

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Satish Balay via petsc-dev]

Sure - once any change works locally [for gcc and xlf]

When I try - I get a bunch of errors.. [yet to digest them.]

Satish

On Wed, 3 Mar 2021, Jacob Faibussowitsch wrote:

> > I'm not sure what would happen if these 'use' statements are removed [whats 
> > required and what can be removed?]
> > 
> > The relevant code that adds this is in 
> > lib/petsc/bin/maint/generatefortranstubs.py
> > 
> >  fd.write('  use petsc'+mansec+'def\n')
> 
> I suppose we can run it through CI, see if it breaks? 
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
> > On Mar 3, 2021, at 12:49, Satish Balay  wrote:
> > 
> > On Wed, 3 Mar 2021, Jacob Faibussowitsch wrote:
> > 
> >> Hello All,
> >> 
> >> I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago 
> >> that would crash the compiler when compiling petsc fortran interfaces. The 
> >> TL;DR of it is that the xl compiler creates a function dictionary for 
> >> every function imported in fortran modules, and since petsc fortran 
> >> interfaces seem to import entire packages writ-large this exceeds the 
> >> number of dictionary entries (2**21):
> >> 
> >>> The reason for the Internal Compiler Error is because we can't grow an 
> >>> interal dictionary anymore (ie we hit a 2**21 limit).
> >>> The file contains many module procedures and interfaces that use the same 
> >>> helper module. As a result, we are importing the dictionary entries for 
> >>> that module repeatedly reaching 
> >>> the limit.
> >>> 
> >>> Can you please give the following source code workaround a try?
> >>> Since there is already "use petscvecdefdummy" at the module scope, one 
> >>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
> >>> vecnotequal and vecequals 
> >>> and all similar procedures.
> >>> 
> >>> For example, the test case has:
> >>>module petscvecdef
> >>>use petscvecdefdummy
> >>> ...
> >>>function vecnotequal(A,B)
> >>>  use petscvecdefdummy
> >>>  logical vecnotequal
> >>>  type(tVec), intent(in) :: A,B
> >>>  vecnotequal = (A%v .ne. B%v)
> >>>end function
> >>>function vecequals(A,B)
> >>>  use petscvecdefdummy
> >>>  logical vecequals
> >>>  type(tVec), intent(in) :: A,B
> >>>  vecequals = (A%v .eq. B%v)
> >>>end function
> >>> ...
> >>> end module
> >>> Another workaround would be to put the procedure definitions from this 
> >>> large module into several submodules.  Each submodule would be able to 
> >>> accommodate a dictionary with 2**21 entries.
> >>> 
> >>> 
> >>> Please let us know if one of the above workarounds resolve the issue.
> >> 
> >> 
> >> The proposed fix from IBM would be to pull “use moduleXXX” out of 
> >> subroutines or to have our auto-fortran interfaces detect which symbols to 
> >> include from the respective modules and only include those in the 
> >> subroutines. I’m not familiar at all with how the interfaces are generated 
> >> so I don’t even know if this is possible.
> > 
> > I'm not sure what would happen if these 'use' statements are removed [whats 
> > required and what can be removed?]
> > 
> > The relevant code that adds this is in 
> > lib/petsc/bin/maint/generatefortranstubs.py
> > 
> >  fd.write('  use petsc'+mansec+'def\n')
> > 
> > Satish
> > 
> >>> IBM provided the following additional explanation and example. Can the 
> >>> process used to generate these routines and functions determine the 
> >>> specific symbols required and then use the only keyword or import 
> >>> statement to include them?
> >>> 
> >>> When factoring out use statements out of module procedures, you can just 
> >>> delete them.  But you can't completely remove them from interface blocks. 
> >>>  Instead, you can limit them either by using use , only:  
> >>> or import  . if the hundreds of use statements in the program are 
> >>> factored out / limited in this way, that should reduce the dictionary 
> >>> size sufficiently for the program to compile.
> >>> 
> >>> For example
> >>>  Interface
> >>>Subroutine VecRestoreArrayReadF90(v,array,ierr)
> >>>  use petscvecdef
> >>>  real(kind=selected_real_kind(10)), pointer :: array(:)
> >>>  integer(kind=selected_int_kind(5)) ierr
> >>>  type(tVec) v
> >>>End Subroutine
> >>>  End Interface
> >>> 
> >>> imports all symbols from petscvecdef into the dictionary even though we 
> >>> only need tVec .  So we can either:
> >>> 
> >>>  Interface
> >>>Subroutine VecRestoreArrayReadF90(v,array,ierr)
> >>>  use petscvecdef, only: tVec
> >>>  implicit none
> >>>  real(kind=selected_real_kind(10)), pointer :: array(:)
> >>>  integer(kind=selected_int_kind(5)) ierr
> >>>  type(tVec) v
> >>>End Subroutine
> >>>  End Interface
> >>> 
> >>> or if use petscvecdef is used in 

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jacob Faibussowitsch]

> I'm not sure what would happen if these 'use' statements are removed [whats 
> required and what can be removed?]
> 
> The relevant code that adds this is in 
> lib/petsc/bin/maint/generatefortranstubs.py
> 
>  fd.write('  use petsc'+mansec+'def\n')

I suppose we can run it through CI, see if it breaks? 

Best regards,

Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391

> On Mar 3, 2021, at 12:49, Satish Balay  wrote:
> 
> On Wed, 3 Mar 2021, Jacob Faibussowitsch wrote:
> 
>> Hello All,
>> 
>> I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago 
>> that would crash the compiler when compiling petsc fortran interfaces. The 
>> TL;DR of it is that the xl compiler creates a function dictionary for every 
>> function imported in fortran modules, and since petsc fortran interfaces 
>> seem to import entire packages writ-large this exceeds the number of 
>> dictionary entries (2**21):
>> 
>>> The reason for the Internal Compiler Error is because we can't grow an 
>>> interal dictionary anymore (ie we hit a 2**21 limit).
>>> The file contains many module procedures and interfaces that use the same 
>>> helper module. As a result, we are importing the dictionary entries for 
>>> that module repeatedly reaching 
>>> the limit.
>>> 
>>> Can you please give the following source code workaround a try?
>>> Since there is already "use petscvecdefdummy" at the module scope, one 
>>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
>>> vecnotequal and vecequals 
>>> and all similar procedures.
>>> 
>>> For example, the test case has:
>>>module petscvecdef
>>>use petscvecdefdummy
>>> ...
>>>function vecnotequal(A,B)
>>>  use petscvecdefdummy
>>>  logical vecnotequal
>>>  type(tVec), intent(in) :: A,B
>>>  vecnotequal = (A%v .ne. B%v)
>>>end function
>>>function vecequals(A,B)
>>>  use petscvecdefdummy
>>>  logical vecequals
>>>  type(tVec), intent(in) :: A,B
>>>  vecequals = (A%v .eq. B%v)
>>>end function
>>> ...
>>> end module
>>> Another workaround would be to put the procedure definitions from this 
>>> large module into several submodules.  Each submodule would be able to 
>>> accommodate a dictionary with 2**21 entries.
>>> 
>>> 
>>> Please let us know if one of the above workarounds resolve the issue.
>> 
>> 
>> The proposed fix from IBM would be to pull “use moduleXXX” out of 
>> subroutines or to have our auto-fortran interfaces detect which symbols to 
>> include from the respective modules and only include those in the 
>> subroutines. I’m not familiar at all with how the interfaces are generated 
>> so I don’t even know if this is possible.
> 
> I'm not sure what would happen if these 'use' statements are removed [whats 
> required and what can be removed?]
> 
> The relevant code that adds this is in 
> lib/petsc/bin/maint/generatefortranstubs.py
> 
>  fd.write('  use petsc'+mansec+'def\n')
> 
> Satish
> 
>>> IBM provided the following additional explanation and example. Can the 
>>> process used to generate these routines and functions determine the 
>>> specific symbols required and then use the only keyword or import statement 
>>> to include them?
>>> 
>>> When factoring out use statements out of module procedures, you can just 
>>> delete them.  But you can't completely remove them from interface blocks.  
>>> Instead, you can limit them either by using use , only:  or 
>>> import  . if the hundreds of use statements in the program are 
>>> factored out / limited in this way, that should reduce the dictionary size 
>>> sufficiently for the program to compile.
>>> 
>>> For example
>>>  Interface
>>>Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>  use petscvecdef
>>>  real(kind=selected_real_kind(10)), pointer :: array(:)
>>>  integer(kind=selected_int_kind(5)) ierr
>>>  type(tVec) v
>>>End Subroutine
>>>  End Interface
>>> 
>>> imports all symbols from petscvecdef into the dictionary even though we 
>>> only need tVec .  So we can either:
>>> 
>>>  Interface
>>>Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>  use petscvecdef, only: tVec
>>>  implicit none
>>>  real(kind=selected_real_kind(10)), pointer :: array(:)
>>>  integer(kind=selected_int_kind(5)) ierr
>>>  type(tVec) v
>>>End Subroutine
>>>  End Interface
>>> 
>>> or if use petscvecdef is used in the outer scope, we can:
>>>  Interface
>>>Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>  import tVec
>>>  implicit none
>>>  real(kind=selected_real_kind(10)), pointer :: array(:)
>>>  integer(kind=selected_int_kind(5)) ierr
>>>  type(tVec) v
>>>End Subroutine
>>>  End Interface
>>> (The two methods (use, only vs import) are equivalent in

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Jed Brown]

I wonder if gfortran has a similar "bug" but larger capacity that would explain 
why it's so much more expensive to compile the Fortran interface files than to 
compile all of PETSc.

Barry Smith  writes:

>PETSc stacks the Fortran modules in the same way it stacks the C include 
> files.   So the TAO module includes all the Fortran modules below it etc.  It 
> would be nearly impossible to disentangle the bits and pieces without 
> introducing a more painful user experience. For example use PCTypes, use 
> PCFunctions, use KSPTypes,  impossible to use and impossible to maintain.
>
>This is a completely artificial bug of IBM's own making in their compiler 
> that we should not have to work around.
>
>Barry
>
>
>> On Mar 3, 2021, at 12:10 PM, Jacob Faibussowitsch  
>> wrote:
>> 
>> Hello All,
>> 
>> I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago 
>> that would crash the compiler when compiling petsc fortran interfaces. The 
>> TL;DR of it is that the xl compiler creates a function dictionary for every 
>> function imported in fortran modules, and since petsc fortran interfaces 
>> seem to import entire packages writ-large this exceeds the number of 
>> dictionary entries (2**21):
>> 
>>> The reason for the Internal Compiler Error is because we can't grow an 
>>> interal dictionary anymore (ie we hit a 2**21 limit).
>>> The file contains many module procedures and interfaces that use the same 
>>> helper module. As a result, we are importing the dictionary entries for 
>>> that module repeatedly reaching 
>>> the limit.
>>>  
>>> Can you please give the following source code workaround a try?
>>> Since there is already "use petscvecdefdummy" at the module scope, one 
>>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
>>> vecnotequal and vecequals 
>>> and all similar procedures.
>>>  
>>> For example, the test case has:
>>> module petscvecdef
>>> use petscvecdefdummy
>>> ...
>>> function vecnotequal(A,B)
>>>   use petscvecdefdummy
>>>   logical vecnotequal
>>>   type(tVec), intent(in) :: A,B
>>>   vecnotequal = (A%v .ne. B%v)
>>> end function
>>> function vecequals(A,B)
>>>   use petscvecdefdummy
>>>   logical vecequals
>>>   type(tVec), intent(in) :: A,B
>>>   vecequals = (A%v .eq. B%v)
>>> end function
>>> ...
>>> end module
>>> Another workaround would be to put the procedure definitions from this 
>>> large module into several submodules.  Each submodule would be able to 
>>> accommodate a dictionary with 2**21 entries.
>>>  
>>>  
>>> Please let us know if one of the above workarounds resolve the issue.
>> 
>> 
>> The proposed fix from IBM would be to pull “use moduleXXX” out of 
>> subroutines or to have our auto-fortran interfaces detect which symbols to 
>> include from the respective modules and only include those in the 
>> subroutines. I’m not familiar at all with how the interfaces are generated 
>> so I don’t even know if this is possible.
>>> IBM provided the following additional explanation and example. Can the 
>>> process used to generate these routines and functions determine the 
>>> specific symbols required and then use the only keyword or import statement 
>>> to include them?
>>> 
>>>  When factoring out use statements out of module procedures, you can just 
>>> delete them.  But you can't completely remove them from interface blocks.  
>>> Instead, you can limit them either by using use , only:  or 
>>> import  .  if the hundreds of use statements in the program are 
>>> factored out / limited in this way, that should reduce the dictionary size 
>>> sufficiently for the program to compile.
>>>  
>>> For example
>>>   Interface
>>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>   use petscvecdef
>>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>>   integer(kind=selected_int_kind(5)) ierr
>>>   type(tVec) v
>>> End Subroutine
>>>   End Interface
>>>  
>>> imports all symbols from petscvecdef into the dictionary even though we 
>>> only need tVec .  So we can either:
>>>  
>>>   Interface
>>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>   use petscvecdef, only: tVec
>>>   implicit none
>>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>>   integer(kind=selected_int_kind(5)) ierr
>>>   type(tVec) v
>>> End Subroutine
>>>   End Interface
>>>  
>>> or if use petscvecdef is used in the outer scope, we can:
>>>   Interface
>>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>>   import tVec
>>>   implicit none
>>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>>   integer(kind=selected_int_kind(5)) ierr
>>>   type(tVec) v
>>> End Subroutine
>>>   End Interface
>>> (The two

Re: [petsc-dev] Case TS005062693 - XLF: ICE in xlfentry compiling a module with 358 subroutines

[Barry Smith]

   PETSc stacks the Fortran modules in the same way it stacks the C include 
files.   So the TAO module includes all the Fortran modules below it etc.  It 
would be nearly impossible to disentangle the bits and pieces without 
introducing a more painful user experience. For example use PCTypes, use 
PCFunctions, use KSPTypes,  impossible to use and impossible to maintain.

   This is a completely artificial bug of IBM's own making in their compiler 
that we should not have to work around.

   Barry


> On Mar 3, 2021, at 12:10 PM, Jacob Faibussowitsch  wrote:
> 
> Hello All,
> 
> I discovered a compiler bug in the IBM xl fortran compiler a few weeks ago 
> that would crash the compiler when compiling petsc fortran interfaces. The 
> TL;DR of it is that the xl compiler creates a function dictionary for every 
> function imported in fortran modules, and since petsc fortran interfaces seem 
> to import entire packages writ-large this exceeds the number of dictionary 
> entries (2**21):
> 
>> The reason for the Internal Compiler Error is because we can't grow an 
>> interal dictionary anymore (ie we hit a 2**21 limit).
>> The file contains many module procedures and interfaces that use the same 
>> helper module. As a result, we are importing the dictionary entries for that 
>> module repeatedly reaching 
>> the limit.
>>  
>> Can you please give the following source code workaround a try?
>> Since there is already "use petscvecdefdummy" at the module scope, one 
>> workaround might be to remove the unnecessary "use petscvecdefdummy" in 
>> vecnotequal and vecequals 
>> and all similar procedures.
>>  
>> For example, the test case has:
>> module petscvecdef
>> use petscvecdefdummy
>> ...
>> function vecnotequal(A,B)
>>   use petscvecdefdummy
>>   logical vecnotequal
>>   type(tVec), intent(in) :: A,B
>>   vecnotequal = (A%v .ne. B%v)
>> end function
>> function vecequals(A,B)
>>   use petscvecdefdummy
>>   logical vecequals
>>   type(tVec), intent(in) :: A,B
>>   vecequals = (A%v .eq. B%v)
>> end function
>> ...
>> end module
>> Another workaround would be to put the procedure definitions from this large 
>> module into several submodules.  Each submodule would be able to accommodate 
>> a dictionary with 2**21 entries.
>>  
>>  
>> Please let us know if one of the above workarounds resolve the issue.
> 
> 
> The proposed fix from IBM would be to pull “use moduleXXX” out of subroutines 
> or to have our auto-fortran interfaces detect which symbols to include from 
> the respective modules and only include those in the subroutines. I’m not 
> familiar at all with how the interfaces are generated so I don’t even know if 
> this is possible.
>> IBM provided the following additional explanation and example. Can the 
>> process used to generate these routines and functions determine the specific 
>> symbols required and then use the only keyword or import statement to 
>> include them?
>> 
>>  When factoring out use statements out of module procedures, you can just 
>> delete them.  But you can't completely remove them from interface blocks.  
>> Instead, you can limit them either by using use , only:  or 
>> import  .  if the hundreds of use statements in the program are 
>> factored out / limited in this way, that should reduce the dictionary size 
>> sufficiently for the program to compile.
>>  
>> For example
>>   Interface
>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>   use petscvecdef
>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>   integer(kind=selected_int_kind(5)) ierr
>>   type(tVec) v
>> End Subroutine
>>   End Interface
>>  
>> imports all symbols from petscvecdef into the dictionary even though we only 
>> need tVec .  So we can either:
>>  
>>   Interface
>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>   use petscvecdef, only: tVec
>>   implicit none
>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>   integer(kind=selected_int_kind(5)) ierr
>>   type(tVec) v
>> End Subroutine
>>   End Interface
>>  
>> or if use petscvecdef is used in the outer scope, we can:
>>   Interface
>> Subroutine VecRestoreArrayReadF90(v,array,ierr)
>>   import tVec
>>   implicit none
>>   real(kind=selected_real_kind(10)), pointer :: array(:)
>>   integer(kind=selected_int_kind(5)) ierr
>>   type(tVec) v
>> End Subroutine
>>   End Interface
>> (The two methods (use, only vs import) are equivalent in terms of impact to 
>> the dictionary.)
>> 
> 
> Is this compiler ~feature~ something that we intend to work around? Thoughts?
> 
> Best regards,
> 
> Jacob Faibussowitsch
> (Jacob Fai - booss - oh - vitch)
> Cell: (312) 694-3391
> 
>> Begin forwarded message:
>> 
>> From: "Roy Mu