Re: Generated files in libgfortran for Fortran intrinsic procedures (was: Updated Sourceware infrastructure plans)
Am Donnerstag, dem 18.04.2024 um 14:01 +0200 schrieb Tobias Burnus:
> Hi Janne,
>
> Janne Blomqvist wrote:
> > back when I was active I did think about this
> > issue. IMHO the best of my ideas was to convert these into C++
> > templates.
I haven't looked at libgfortran but I didn't find it problematic
at all to use C in similar numerical code and this helps
with portability.
Either I use macros, which I keep short and then do not find
inferior to templates (having used C++ for years previously) or
- if there is really a lot of code that needs to be specialized
for a type - simply by using includes:
#define matmul_type double
#include "matmul_impl.c"
Martin
>
> I think this will work – but we have to be super careful:
>
> With C++, there is the problem that we definitely do not want to add
> dependency on libstdc++ nor to use some features which require special
> hardware support (like exceptions [always bad], symbol aliases, ...). —
> On some systems, a full C++ support might be not available, like
> embedded systems (including some odd embedded OS) or offloading devices.
>
> The libstdc++ dependency would be detected by linking as we currently
> do. For in-language features, we have to ensure the appropriate flags
> -fno-exceptions (and probably a few more). And it should be clear what
> language features to use.
>
> If we do, I think that would surely be an option.
>
> > What we're essentially doing with the M4 stuff and the
> > proposed in-house Python reimplementation is to make up for lack of
> > monomorphization in plain old C. Rather than doing some DIY templates,
> > switch the implementation language to something which has that feature
> > built-in, in this case C++. No need to convert the entire libgfortran
> > to C++ if you don't want to, just those objects that are generated
> > from the M4 templates. Something like
> >
> > template
> > void matmul(T* a, T* b, T* c, ...)
> > {
> > // actual matmul code here
> > }
> >
> > extern "C" {
> >// Instantiate template for every type and export the symbol
> >void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
> >{
> > matmul(a, b, c, ...);
> >}
> >// And so on for other types
> > }
>
> Cheers,
>
> Tobias
Generated files in libgfortran for Fortran intrinsic procedures (was: Updated Sourceware infrastructure plans)
Hi Janne,
Janne Blomqvist wrote:
back when I was active I did think about this
issue. IMHO the best of my ideas was to convert these into C++
templates.
I think this will work – but we have to be super careful:
With C++, there is the problem that we definitely do not want to add
dependency on libstdc++ nor to use some features which require special
hardware support (like exceptions [always bad], symbol aliases, ...). —
On some systems, a full C++ support might be not available, like
embedded systems (including some odd embedded OS) or offloading devices.
The libstdc++ dependency would be detected by linking as we currently
do. For in-language features, we have to ensure the appropriate flags
-fno-exceptions (and probably a few more). And it should be clear what
language features to use.
If we do, I think that would surely be an option.
What we're essentially doing with the M4 stuff and the
proposed in-house Python reimplementation is to make up for lack of
monomorphization in plain old C. Rather than doing some DIY templates,
switch the implementation language to something which has that feature
built-in, in this case C++. No need to convert the entire libgfortran
to C++ if you don't want to, just those objects that are generated
from the M4 templates. Something like
template
void matmul(T* a, T* b, T* c, ...)
{
// actual matmul code here
}
extern "C" {
// Instantiate template for every type and export the symbol
void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
{
matmul(a, b, c, ...);
}
// And so on for other types
}
Cheers,
Tobias
Re: Updated Sourceware infrastructure plans
On Thu, Apr 18, 2024 at 11:15 AM FX Coudert wrote:
>
> > I regenerate auto* files from time to time for libgfortran. Regenerating
> > them has always been very fragile (using --enable-maintainer-mode),
> > and difficult to get right.
>
> I have never found them difficult to regenerate, but if you have only a non
> maintainer build, it is a pain to have to make a new maintainer build for a
> minor change.
>
> Moreover, our m4 code is particularly painful to use and unreadable. I have
> been wondering for some time: should we switch to simpler Python scripts? It
> would also mean that we would have fewer files in the generated/ folder:
> right now, every time we add new combinations of types, we have a
> combinatorial explosion of files.
>
> $ ls generated/sum_*
> generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c
> generated/sum_i16.c generated/sum_i4.c generated/sum_r10.c
> generated/sum_r17.c generated/sum_r8.c
> generated/sum_c16.c generated/sum_c4.c generated/sum_i1.c
> generated/sum_i2.c generated/sum_i8.c generated/sum_r16.c generated/sum_r4.c
>
> We could imagine having a single file for all sum intrinsics.
>
> How do Fortran maintainers feel about that?
For the time being I'm not an active maintainer, so my opinion doesn't
per se have weight, but back when I was active I did think about this
issue. IMHO the best of my ideas was to convert these into C++
templates. What we're essentially doing with the M4 stuff and the
proposed in-house Python reimplementation is to make up for lack of
monomorphization in plain old C. Rather than doing some DIY templates,
switch the implementation language to something which has that feature
built-in, in this case C++. No need to convert the entire libgfortran
to C++ if you don't want to, just those objects that are generated
from the M4 templates. Something like
template
void matmul(T* a, T* b, T* c, ...)
{
// actual matmul code here
}
extern "C" {
// Instantiate template for every type and export the symbol
void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
{
matmul(a, b, c, ...);
}
// And so on for other types
}
--
Janne Blomqvist
[PATCH] libgfortran: Fix up the autoreconf warnings.
@tschwinge since he did quite a bit of work on getting autoreconf to work in the GCC-13 cycle. This does not address the issues with regenerating lib code, but it does make things somewhat smoother for cases where the updates are only in Makefile.am, configure.ac or libtool.m4 for example. It is based on a patch I've been using on the release branches for Darwin (written because I wasted a day on a warning missed among the wall of output). You should now be able to run "autoreconf -fv" in the libgfortran directory with only informational output (no warnings). So far only tested very lightly on trunk - but posting early in case it helps the way forward. thanks Iain --- 8< --- This means using sub-dirs and amending some of the recipes accordingly. libgfortran/ChangeLog: * Makefile.am: Use sub-dirs, amend recipies accordingly. * Makefile.in: Regenerate. Signed-off-by: Iain Sandoe --- libgfortran/Makefile.am | 1431 +++--- libgfortran/Makefile.in | 9848 ++- 2 files changed, 4126 insertions(+), 7153 deletions(-) diff --git a/libgfortran/Makefile.am b/libgfortran/Makefile.am index 9f8a4f69863..8bef1729219 100644 --- a/libgfortran/Makefile.am +++ b/libgfortran/Makefile.am @@ -1,5 +1,6 @@ ## Process this file with automake to produce Makefile.in +AUTOMAKE_OPTIONS = foreign subdir-objects ACLOCAL_AMFLAGS = -I .. -I ../config @@ -239,629 +240,629 @@ runtime/stop.c endif i_all_c= \ -$(srcdir)/generated/all_l1.c \ -$(srcdir)/generated/all_l2.c \ -$(srcdir)/generated/all_l4.c \ -$(srcdir)/generated/all_l8.c \ -$(srcdir)/generated/all_l16.c +generated/all_l1.c \ +generated/all_l2.c \ +generated/all_l4.c \ +generated/all_l8.c \ +generated/all_l16.c i_any_c= \ -$(srcdir)/generated/any_l1.c \ -$(srcdir)/generated/any_l2.c \ -$(srcdir)/generated/any_l4.c \ -$(srcdir)/generated/any_l8.c \ -$(srcdir)/generated/any_l16.c +generated/any_l1.c \ +generated/any_l2.c \ +generated/any_l4.c \ +generated/any_l8.c \ +generated/any_l16.c i_bessel_c= \ -$(srcdir)/generated/bessel_r4.c \ -$(srcdir)/generated/bessel_r8.c \ -$(srcdir)/generated/bessel_r10.c \ -$(srcdir)/generated/bessel_r16.c \ -$(srcdir)/generated/bessel_r17.c +generated/bessel_r4.c \ +generated/bessel_r8.c \ +generated/bessel_r10.c \ +generated/bessel_r16.c \ +generated/bessel_r17.c i_count_c= \ -$(srcdir)/generated/count_1_l.c \ -$(srcdir)/generated/count_2_l.c \ -$(srcdir)/generated/count_4_l.c \ -$(srcdir)/generated/count_8_l.c \ -$(srcdir)/generated/count_16_l.c +generated/count_1_l.c \ +generated/count_2_l.c \ +generated/count_4_l.c \ +generated/count_8_l.c \ +generated/count_16_l.c i_iall_c= \ -$(srcdir)/generated/iall_i1.c \ -$(srcdir)/generated/iall_i2.c \ -$(srcdir)/generated/iall_i4.c \ -$(srcdir)/generated/iall_i8.c \ -$(srcdir)/generated/iall_i16.c +generated/iall_i1.c \ +generated/iall_i2.c \ +generated/iall_i4.c \ +generated/iall_i8.c \ +generated/iall_i16.c i_iany_c= \ -$(srcdir)/generated/iany_i1.c \ -$(srcdir)/generated/iany_i2.c \ -$(srcdir)/generated/iany_i4.c \ -$(srcdir)/generated/iany_i8.c \ -$(srcdir)/generated/iany_i16.c +generated/iany_i1.c \ +generated/iany_i2.c \ +generated/iany_i4.c \ +generated/iany_i8.c \ +generated/iany_i16.c i_iparity_c= \ -$(srcdir)/generated/iparity_i1.c \ -$(srcdir)/generated/iparity_i2.c \ -$(srcdir)/generated/iparity_i4.c \ -$(srcdir)/generated/iparity_i8.c \ -$(srcdir)/generated/iparity_i16.c +generated/iparity_i1.c \ +generated/iparity_i2.c \ +generated/iparity_i4.c \ +generated/iparity_i8.c \ +generated/iparity_i16.c i_findloc0_c= \ -$(srcdir)/generated/findloc0_i1.c \ -$(srcdir)/generated/findloc0_i2.c \ -$(srcdir)/generated/findloc0_i4.c \ -$(srcdir)/generated/findloc0_i8.c \ -$(srcdir)/generated/findloc0_i16.c \ -$(srcdir)/generated/findloc0_r4.c \ -$(srcdir)/generated/findloc0_r8.c \ -$(srcdir)/generated/findloc0_r10.c \ -$(srcdir)/generated/findloc0_r16.c \ -$(srcdir)/generated/findloc0_r17.c \ -$(srcdir)/generated/findloc0_c4.c \ -$(srcdir)/generated/findloc0_c8.c \ -$(srcdir)/generated/findloc0_c10.c \ -$(srcdir)/generated/findloc0_c16.c \ -$(srcdir)/generated/findloc0_c17.c +generated/findloc0_i1.c \ +generated/findloc0_i2.c \ +generated/findloc0_i4.c \ +generated/findloc0_i8.c \ +generated/findloc0_i16.c \ +generated/findloc0_r4.c \ +generated/findloc0_r8.c \ +generated/findloc0_r10.c \ +generated/findloc0_r16.c \ +generated/findloc0_r17.c \ +generated/findloc0_c4.c \ +generated/findloc0_c8.c \ +generated/findloc0_c10.c \ +generated/findloc0_c16.c \ +generated/findloc0_c17.c i_findloc0s_c= \ -$(srcdir)/generated/findloc0_s1.c \ -$(srcdir)/generated/findloc0_s4.c +generated/findloc0_s1.c \ +generated/findloc0_s4.c i_findloc1_c= \ -$(srcdir)/generated/findloc1_i1.c \ -$(srcdir)/generated/findloc1_i2.c \ -$(srcdir)/generated/findloc1_i4.c \ -$(srcdir)/generated/findloc1_i8.c \ -$(srcdir)/generated/findloc1_i16.c \ -$(srcdir)/generated/findloc1_r4.c \ -$(srcdir)/generated/findloc1_r8.c \
Re: Updated Sourceware infrastructure plans
Hi, On Thu, 18 Apr 2024 at 10:15, FX Coudert wrote: > > > I regenerate auto* files from time to time for libgfortran. Regenerating > > them has always been very fragile (using --enable-maintainer-mode), > > and difficult to get right. > > I have never found them difficult to regenerate, but if you have only a non > maintainer build, it is a pain to have to make a new maintainer build for a > minor change. > FWIW, we have noticed lots of warnings from autoreconf in libgfortran. I didn't try to investigate, since the regenerated files are identical to what is currently in the repo. For instance, you can download the "stdio" output from the autoregen.py step in https://builder.sourceware.org/buildbot/#/builders/269/builds/4373 Thanks, Christophe > Moreover, our m4 code is particularly painful to use and unreadable. I have > been wondering for some time: should we switch to simpler Python scripts? It > would also mean that we would have fewer files in the generated/ folder: > right now, every time we add new combinations of types, we have a > combinatorial explosion of files. > > $ ls generated/sum_* > generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c > generated/sum_i16.c generated/sum_i4.c generated/sum_r10.c > generated/sum_r17.c generated/sum_r8.c > generated/sum_c16.c generated/sum_c4.c generated/sum_i1.c > generated/sum_i2.c generated/sum_i8.c generated/sum_r16.c generated/sum_r4.c > > We could imagine having a single file for all sum intrinsics. > > How do Fortran maintainers feel about that? > > FX
Re: Updated Sourceware infrastructure plans
> I regenerate auto* files from time to time for libgfortran. Regenerating > them has always been very fragile (using --enable-maintainer-mode), > and difficult to get right. I have never found them difficult to regenerate, but if you have only a non maintainer build, it is a pain to have to make a new maintainer build for a minor change. Moreover, our m4 code is particularly painful to use and unreadable. I have been wondering for some time: should we switch to simpler Python scripts? It would also mean that we would have fewer files in the generated/ folder: right now, every time we add new combinations of types, we have a combinatorial explosion of files. $ ls generated/sum_* generated/sum_c10.c generated/sum_c17.c generated/sum_c8.c generated/sum_i16.c generated/sum_i4.c generated/sum_r10.c generated/sum_r17.c generated/sum_r8.c generated/sum_c16.c generated/sum_c4.c generated/sum_i1.c generated/sum_i2.c generated/sum_i8.c generated/sum_r16.c generated/sum_r4.c We could imagine having a single file for all sum intrinsics. How do Fortran maintainers feel about that? FX
Re: Updated Sourceware infrastructure plans
Am 18.04.24 um 01:27 schrieb Mark Wielaard: We also should make sure that all generated files (either in git or in the release/snapshot tar balls) can be reliably and reproducibly regenerated. This also helps the (pre-commit) CI buildbots. We already have the autoregen bots for gcc and binutils-gdb. And Christoph has been working on extending the scripts to regenerate more kinds of files. I regenerate auto* files from time to time for libgfortran. Regenerating them has always been very fragile (using --enable-maintainer-mode), and difficult to get right. If there is a better process available to do it the right way is that is documented and easy to use, this will make work easier. If not, it has the potential to stop the work I am planning to contribute in a project that is about a month from starting (and maybe stop the project altogether). Can anybody point me towards the tools that will be the gold standard in the future, and the reproducible way of regenerating them? Best regards Thomas
