Re: RFC: Building a minimal libgfortran for nvptx

[Thomas Schwinge]

Hi!

On 2015-03-11T22:48:22+0100, I wrote:
> On Fri, 14 Nov 2014 18:08:32 +0100, Bernd Schmidt  
> wrote:
>> New patch below, [...]
>
> ... got committed.  I now committed the following in r221363:

> libgfortran LIBGFOR_MINIMAL enhancements.

..., and in r276690 have now committed the attached to "Revise
'libgfortran/runtime/minimal.c' to better conform to the original
sources".


Grüße
 Thomas


From c6c3841de5ec2b37d8b579bd7bce54bef811c064 Mon Sep 17 00:00:00 2001
From: tschwinge 
Date: Tue, 8 Oct 2019 10:20:31 +
Subject: [PATCH 1/3] Revise 'libgfortran/runtime/minimal.c' to better conform
 to the original sources

	libgfortran/
	* runtime/minimal.c: Revise.

git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@276690 138bc75d-0d04-0410-961f-82ee72b054a4
---
 libgfortran/ChangeLog |   4 +
 libgfortran/runtime/minimal.c | 237 +++---
 2 files changed, 169 insertions(+), 72 deletions(-)

diff --git a/libgfortran/ChangeLog b/libgfortran/ChangeLog
index 7736e5da937..9e3b1f8bad8 100644
--- a/libgfortran/ChangeLog
+++ b/libgfortran/ChangeLog
@@ -1,3 +1,7 @@
+2019-10-08  Thomas Schwinge  
+
+	* runtime/minimal.c: Revise.
+
 2019-10-05  Paul Thomas  
 
 	PR fortran/91926
diff --git a/libgfortran/runtime/minimal.c b/libgfortran/runtime/minimal.c
index c1993b99be7..a633bc1ce0f 100644
--- a/libgfortran/runtime/minimal.c
+++ b/libgfortran/runtime/minimal.c
@@ -23,13 +23,38 @@ see the files COPYING3 and COPYING.RUNTIME respectively.  If not, see
 .  */
 
 #include "libgfortran.h"
-#include 
 
+#include 
 
 #ifdef HAVE_UNISTD_H
 #include 
 #endif
 
+
+#if __nvptx__
+/* Map "exit" to "abort"; see PR85463 '[nvptx] "exit" in offloaded region
+   doesn't terminate process'.  */
+# undef exit
+# define exit(status) do { (void) (status); abort (); } while (0)
+#endif
+
+
+#if __nvptx__
+/* 'printf' is all we have.  */
+# undef estr_vprintf
+# define estr_vprintf vprintf
+#else
+# error TODO
+#endif
+
+
+/* runtime/environ.c */
+
+options_t options;
+
+
+/* runtime/main.c */
+
 /* Stupid function to be sure the constructor is always linked in, even
in the case of static linking.  See PR libfortran/22298 for details.  */
 void
@@ -38,11 +63,126 @@ stupid_function_name_for_static_linking (void)
   return;
 }
 
-options_t options;
 
 static int argc_save;
 static char **argv_save;
 
+
+/* Set the saved values of the command line arguments.  */
+
+void
+set_args (int argc, char **argv)
+{
+  argc_save = argc;
+  argv_save = argv;
+}
+iexport(set_args);
+
+
+/* Retrieve the saved values of the command line arguments.  */
+
+void
+get_args (int *argc, char ***argv)
+{
+  *argc = argc_save;
+  *argv = argv_save;
+}
+
+
+/* runtime/error.c */
+
+/* Write a null-terminated C string to standard error. This function
+   is async-signal-safe.  */
+
+ssize_t
+estr_write (const char *str)
+{
+  return write (STDERR_FILENO, str, strlen (str));
+}
+
+
+/* printf() like function for for printing to stderr.  Uses a stack
+   allocated buffer and doesn't lock stderr, so it should be safe to
+   use from within a signal handler.  */
+
+int
+st_printf (const char * format, ...)
+{
+  int written;
+  va_list ap;
+  va_start (ap, format);
+  written = estr_vprintf (format, ap);
+  va_end (ap);
+  return written;
+}
+
+
+/* sys_abort()-- Terminate the program showing backtrace and dumping
+   core.  */
+
+void
+sys_abort (void)
+{
+  /* If backtracing is enabled, print backtrace and disable signal
+ handler for ABRT.  */
+  if (options.backtrace == 1
+  || (options.backtrace == -1 && compile_options.backtrace == 1))
+{
+  estr_write ("\nProgram aborted.\n");
+}
+
+  abort();
+}
+
+
+/* Exit in case of error termination. If backtracing is enabled, print
+   backtrace, then exit.  */
+
+void
+exit_error (int status)
+{
+  if (options.backtrace == 1
+  || (options.backtrace == -1 && compile_options.backtrace == 1))
+{
+  estr_write ("\nError termination.\n");
+}
+  exit (status);
+}
+
+
+/* show_locus()-- Print a line number and filename describing where
+ * something went wrong */
+
+void
+show_locus (st_parameter_common *cmp)
+{
+  char *filename;
+
+  if (!options.locus || cmp == NULL || cmp->filename == NULL)
+return;
+  
+  if (cmp->unit > 0)
+{
+  filename = /* TODO filename_from_unit (cmp->unit) */ NULL;
+
+  if (filename != NULL)
+	{
+	  st_printf ("At line %d of file %s (unit = %d, file = '%s')\n",
+		   (int) cmp->line, cmp->filename, (int) cmp->unit, filename);
+	  free (filename);
+	}
+  else
+	{
+	  st_printf ("At line %d of file %s (unit = %d)\n",
+		   (int) cmp->line, cmp->filename, (int) cmp->unit);
+	}
+  return;
+}
+
+  st_printf ("At line %d of file %s\n", (int) cmp->line, cmp->filename);
+}
+
+
 /* recursion_check()-- It's possible for additional errors to occur
  * during fatal error processing.  We detect this condition here and
  * exit with code 4 immediately. */
@@ -70,9 +210,10 @@ void
 

Re: RFC: Building a minimal libgfortran for nvptx

[Thomas Schwinge]

Hi!

On Fri, 14 Nov 2014 18:08:32 +0100, Bernd Schmidt ber...@codesourcery.com 
wrote:
 New patch below, [...]

... got committed.  I now committed the following in r221363:

commit 83ba0e65833dd081db921f8c2b3277316590753c
Author: tschwinge tschwinge@138bc75d-0d04-0410-961f-82ee72b054a4
Date:   Wed Mar 11 21:42:56 2015 +

libgfortran LIBGFOR_MINIMAL enhancements.

Based on GCC trunk r220892, for nvptx-none:

=== gfortran Summary ===

# of expected passes[-31320-]{+32117+}
# of unexpected failures[-7222-]{+6821+}
# of expected failures  78
# of unresolved testcases   [-6441-]{+6158+}
# of untested testcases [-432-]{+391+}
# of unsupported tests  639

libgfortran/
* caf/single.c (caf_runtime_error): Revert 2014-11-28 changes.
* runtime/minimal.c (STRERR_MAXSZ): Don't define.
(runtime_error_at, sys_abort): Bring more in line with the
non-LIBGFOR_MINIMAL code.
(runtime_warning_at, internal_error): New functions.

git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@221363 
138bc75d-0d04-0410-961f-82ee72b054a4
---
 libgfortran/ChangeLog |8 ++
 libgfortran/caf/single.c  |3 +--
 libgfortran/runtime/minimal.c |   58 ++---
 3 files changed, 64 insertions(+), 5 deletions(-)

diff --git libgfortran/ChangeLog libgfortran/ChangeLog
index 97ee01b..5b201d2 100644
--- libgfortran/ChangeLog
+++ libgfortran/ChangeLog
@@ -1,3 +1,11 @@
+2015-03-11  Thomas Schwinge  tho...@codesourcery.com
+
+   * caf/single.c (caf_runtime_error): Revert 2014-11-28 changes.
+   * runtime/minimal.c (STRERR_MAXSZ): Don't define.
+   (runtime_error_at, sys_abort): Bring more in line with the
+   non-LIBGFOR_MINIMAL code.
+   (runtime_warning_at, internal_error): New functions.
+
 2015-03-11  Janne Blomqvist  j...@gcc.gnu.org
 
PR libfortran/65200
diff --git libgfortran/caf/single.c libgfortran/caf/single.c
index daef281..6c58286 100644
--- libgfortran/caf/single.c
+++ libgfortran/caf/single.c
@@ -48,14 +48,13 @@ caf_static_t *caf_static_list = NULL;
 static void
 caf_runtime_error (const char *message, ...)
 {
-#ifndef LIBGFOR_MINIMAL
   va_list ap;
   fprintf (stderr, Fortran runtime error: );
   va_start (ap, message);
   vfprintf (stderr, message, ap);
   va_end (ap);
   fprintf (stderr, \n);
-#endif
+
   /* FIXME: Shutdown the Fortran RTL to flush the buffer.  PR 43849.  */
   exit (EXIT_FAILURE);
 }
diff --git libgfortran/runtime/minimal.c libgfortran/runtime/minimal.c
index 7ef4647..72a134a 100644
--- libgfortran/runtime/minimal.c
+++ libgfortran/runtime/minimal.c
@@ -73,7 +73,10 @@ recursion_check (void)
   magic = MAGIC;
 }
 
-#define STRERR_MAXSZ 256
+
+/* os_error()-- Operating system error.  We get a message from the
+ * operating system, show it and leave.  Some operating system errors
+ * are caught and processed by the library.  If not, we come here. */
 
 void
 os_error (const char *message)
@@ -85,6 +88,10 @@ os_error (const char *message)
 }
 iexport(os_error);
 
+
+/* void runtime_error()-- These are errors associated with an
+ * invalid fortran program. */
+
 void
 runtime_error (const char *message, ...)
 {
@@ -109,7 +116,8 @@ runtime_error_at (const char *where, const char *message, 
...)
   va_list ap;
 
   recursion_check ();
-  printf (Fortran runtime error: );
+  printf (%s, where);
+  printf (\nFortran runtime error: );
   va_start (ap, message);
   vprintf (message, ap);
   va_end (ap);
@@ -118,6 +126,43 @@ runtime_error_at (const char *where, const char *message, 
...)
 }
 iexport(runtime_error_at);
 
+
+void
+runtime_warning_at (const char *where, const char *message, ...)
+{
+  va_list ap;
+
+  printf (%s, where);
+  printf (\nFortran runtime warning: );
+  va_start (ap, message);
+  vprintf (message, ap);
+  va_end (ap);
+  printf (\n);
+}
+iexport(runtime_warning_at);
+
+
+/* void internal_error()-- These are this-can't-happen errors
+ * that indicate something deeply wrong. */
+
+void
+internal_error (st_parameter_common *cmp, const char *message)
+{
+  recursion_check ();
+  printf (Internal Error: );
+  printf (%s, message);
+  printf (\n);
+
+  /* This function call is here to get the main.o object file included
+ when linking statically. This works because error.o is supposed to
+ be always linked in (and the function call is in internal_error
+ because hopefully it doesn't happen too often).  */
+  stupid_function_name_for_static_linking();
+
+  exit (3);
+}
+
+
 /* Return the full path of the executable.  */
 char *
 full_exe_path (void)
@@ -153,6 +198,13 @@ get_args (int *argc, char ***argv)
 void
 sys_abort (void)
 {
-  printf (Abort called.\n);
+  /* If backtracing is enabled, print backtrace and disable signal
+ handler for ABRT.  */
+  if (options.backtrace == 1
+  || (options.backtrace == -1  

Re: RFC: Building a minimal libgfortran for nvptx

[Bernd Schmidt]

On 11/14/2014 10:28 PM, Tobias Burnus wrote:

All in all: Okay when tesing succeeded. I still prefer some words what's
excluded (or included) in minimal as comment in configure.ac, but the
patch is also okay without.


I thought you meant something more than adding a comment. I've added 
this in the configure script, and committed the patch after testing:


# For GPU offloading, not everything in libfortran can be supported.
# Currently, the only target that has this problem is nvptx.  The
# following is a (partial) list of features that are unsupportable on
# this particular target:
# * Constructors
# * alloca
# * C library support for I/O, with printf as the one notable exception
# * C library support for other features such as signal, environment
#   variables, time functions

I wouldn't worry too much about breaking nvptx by accident, we can 
surely clean up any such problems should they arise.



Bernd

Re: RFC: Building a minimal libgfortran for nvptx

[Bernd Schmidt]

Hi Tobias,


Does printf work? I thought I/O is not supported? Or does it just accept
it for linking and drop it? I think Janne's patch has already dealt with
the issue of stack allocation.


printf (or more accurately vprintf) is supported by ptx as a magic 
builtin function. We have a printf wrapper around vprintf in our newlib. 
What was Janne's patch?



What I dislike a bit about the feature is that it is not clear what
features will be supported for LIBGFOR_MINIMAL. Maybe configure.ac would
be a good place to describe which features are included there (e.g. no
I/O but printf etc.) and which aren't. That will make it easier to see
what has to be modifed if one will add another differently bare system
later on.


My view is that it makes no sense to generalize this at the moment when 
it is unknown what another target that uses this would look like. The 
time to revisit this is when there is another target.



+#if 0
+/* Initialize the runtime library.  */

#if 0'ed code shouldn't be in the patch.


Ok, removed - I'll put it back if we ever support constructors.


+@LIBGFOR_MINIMAL_FALSE@gfor_helper_src = intrinsics/associated.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/abort.c intrinsics/access.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/args.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/bit_intrinsics.c \
...
+@LIBGFOR_MINIMAL_TRUE@gfor_helper_src = intrinsics/associated.c \
+@LIBGFOR_MINIMAL_TRUE@intrinsics/abort.c intrinsics/args.c \
...

Can't one write this differently, avoiding having most lines repeated
and only a few missing from the second set?


Modified. New patch below, how does this look? Regenerated files not 
included. Testing in progress.



Bernd

	* Makefile.am (AM_CFLAGS): Add -DLIBGFOR_MINIMAL if LIBGFOR_MINIMAL.
	(gfor_io_src, gfor_heper_src, gfor_src): Split into minimal and
	always included sources.
	* Makefile.in: Regenerate.
	* configure.ac (LIBGFOR_MINIMAL): New AM_CONDITIONAL.
	* configure: Regenerate.
	* caf/single.c (caf_runtime_error): Don't print messages if
	LIBGFOR_MINIMAL.
	* runtime/compile_options.c (fatal_error_in_progress,
	show_signal, backtrace_handler, maybe_find_addr2line): Guard with
	!defined LIBGFOR_MINIMAL.
	(set_options): Likewise for the backtrace code.
	* runtime/minimal.c: New file.

diff --git a/libgfortran/Makefile.am b/libgfortran/Makefile.am
index a058a01..31eb986 100644
--- a/libgfortran/Makefile.am
+++ b/libgfortran/Makefile.am
@@ -77,7 +77,16 @@ AM_CFLAGS += $(SECTION_FLAGS)
 AM_CFLAGS += $(IEEE_FLAGS)
 AM_FCFLAGS += $(IEEE_FLAGS)
 
+if LIBGFOR_MINIMAL
+AM_CFLAGS += -DLIBGFOR_MINIMAL
+endif
+
 gfor_io_src= \
+io/size_from_kind.c
+
+if !LIBGFOR_MINIMAL
+
+gfor_io_src+= \
 io/close.c \
 io/file_pos.c \
 io/format.c \
@@ -87,7 +96,6 @@ io/list_read.c \
 io/lock.c \
 io/open.c \
 io/read.c \
-io/size_from_kind.c \
 io/transfer.c \
 io/transfer128.c \
 io/unit.c \
@@ -95,6 +103,8 @@ io/unix.c \
 io/write.c \
 io/fbuf.c
 
+endif
+
 gfor_io_headers= \
 io/io.h \
 io/fbuf.h \
@@ -104,67 +114,73 @@ io/unix.h
 gfor_helper_src= \
 intrinsics/associated.c \
 intrinsics/abort.c \
-intrinsics/access.c \
 intrinsics/args.c \
 intrinsics/bit_intrinsics.c \
-intrinsics/c99_functions.c \
-intrinsics/chdir.c \
-intrinsics/chmod.c \
-intrinsics/clock.c \
-intrinsics/cpu_time.c \
 intrinsics/cshift0.c \
-intrinsics/ctime.c \
-intrinsics/date_and_time.c \
-intrinsics/dtime.c \
-intrinsics/env.c \
 intrinsics/eoshift0.c \
 intrinsics/eoshift2.c \
 intrinsics/erfc_scaled.c \
-intrinsics/etime.c \
-intrinsics/execute_command_line.c \
-intrinsics/exit.c \
 intrinsics/extends_type_of.c \
 intrinsics/fnum.c \
-intrinsics/gerror.c \
-intrinsics/getcwd.c \
-intrinsics/getlog.c \
-intrinsics/getXid.c \
-intrinsics/hostnm.c \
 intrinsics/ierrno.c \
 intrinsics/ishftc.c \
 intrinsics/iso_c_generated_procs.c \
 intrinsics/iso_c_binding.c \
-intrinsics/kill.c \
-intrinsics/link.c \
 intrinsics/malloc.c \
 intrinsics/mvbits.c \
 intrinsics/move_alloc.c \
 intrinsics/pack_generic.c \
-intrinsics/perror.c \
 intrinsics/selected_char_kind.c \
-intrinsics/signal.c \
 intrinsics/size.c \
-intrinsics/sleep.c \
 intrinsics/spread_generic.c \
 intrinsics/string_intrinsics.c \
-intrinsics/system.c \
 intrinsics/rand.c \
 intrinsics/random.c \
-intrinsics/rename.c \
 intrinsics/reshape_generic.c \
 intrinsics/reshape_packed.c \
 intrinsics/selected_int_kind.f90 \
 intrinsics/selected_real_kind.f90 \
+intrinsics/transpose_generic.c \
+intrinsics/unpack_generic.c \
+runtime/in_pack_generic.c \
+runtime/in_unpack_generic.c
+
+if !LIBGFOR_MINIMAL
+
+gfor_helper_src+= \
+intrinsics/access.c \
+intrinsics/c99_functions.c \
+intrinsics/chdir.c \
+intrinsics/chmod.c \
+intrinsics/clock.c \
+intrinsics/cpu_time.c \
+intrinsics/ctime.c \
+intrinsics/date_and_time.c \
+intrinsics/dtime.c \
+intrinsics/env.c \
+intrinsics/etime.c \
+intrinsics/execute_command_line.c \
+intrinsics/exit.c \
+intrinsics/gerror.c \
+intrinsics/getcwd.c \
+intrinsics/getlog.c \
+intrinsics/getXid.c \
+intrinsics/hostnm.c \

Re: RFC: Building a minimal libgfortran for nvptx

[Tobias Burnus]

Hi Tobias,

Bernd Schmidt wrote:

Does printf work? I thought I/O is not supported? Or does it just accept
it for linking and drop it? I think Janne's patch has already dealt with
the issue of stack allocation.


printf (or more accurately vprintf) is supported by ptx as a magic 
builtin function. We have a printf wrapper around vprintf in our newlib.


And that prints to the normal screen? That makes debugging easier than I 
had hoped for.



What was Janne's patch?


He changed stack allocation to static/heap allocation; the patch was 
committed 36h ago, broke Cesar's bootstrap and is here: 
https://gcc.gnu.org/ml/fortran/2014-11/msg00052.html




What I dislike a bit about the feature is that it is not clear what
features will be supported for LIBGFOR_MINIMAL. Maybe configure.ac would
be a good place to describe which features are included there (e.g. no
I/O but printf etc.) and which aren't. That will make it easier to see
what has to be modifed if one will add another differently bare system
later on.


My view is that it makes no sense to generalize this at the moment 
when it is unknown what another target that uses this would look like. 
The time to revisit this is when there is another target.


Well, one can still add some notes what is supported and what isn't – 
not only for that hypothecial case but also when doing other changes to 
the library (adding new functions).



Can't one write this differently, avoiding having most lines repeated
and only a few missing from the second set?


Modified. New patch below, how does this look? Regenerated files not 
included. Testing in progress.


I like it more that way :-)


All in all: Okay when tesing succeeded. I still prefer some words what's 
excluded (or included) in minimal as comment in configure.ac, but the 
patch is also okay without.


Thanks,

Tobias

Re: RFC: Building a minimal libgfortran for nvptx

[Janne Blomqvist]

On Fri, Nov 14, 2014 at 11:28 PM, Tobias Burnus bur...@net-b.de wrote:
 Hi Tobias,

 Bernd Schmidt wrote:

 Does printf work? I thought I/O is not supported? Or does it just accept
 it for linking and drop it? I think Janne's patch has already dealt with
 the issue of stack allocation.


 printf (or more accurately vprintf) is supported by ptx as a magic builtin
 function. We have a printf wrapper around vprintf in our newlib.

So the normal stdin/out/err file descriptors are not available?

 And that prints to the normal screen? That makes debugging easier than I had
 hoped for.

 What was Janne's patch?


 He changed stack allocation to static/heap allocation; the patch was
 committed 36h ago, broke Cesar's bootstrap and is here:
 https://gcc.gnu.org/ml/fortran/2014-11/msg00052.html

To be fair, most of the fixes were in the I/O library, or for various
I/O related syscall wrappers, which probably aren't of interest for
the nvptx backend. In any case, libgfortran doesn't use alloca() nor
VLA's anymore, if there were any issues related to that when doing
your patch you might want to review it  and see if you can make the
minimal version a bit more complete.

 What I dislike a bit about the feature is that it is not clear what
 features will be supported for LIBGFOR_MINIMAL. Maybe configure.ac would
 be a good place to describe which features are included there (e.g. no
 I/O but printf etc.) and which aren't. That will make it easier to see
 what has to be modifed if one will add another differently bare system
 later on.


 My view is that it makes no sense to generalize this at the moment when it
 is unknown what another target that uses this would look like. The time to
 revisit this is when there is another target.

Are you committed to helping whoever will be doing the support for the
next minimal target then? Otherwise it seems a bit unfair to leave it
to them to clean up the mess?

 Well, one can still add some notes what is supported and what isn't – not
 only for that hypothecial case but also when doing other changes to the
 library (adding new functions).

Yes, something like this would be nice. Otherwise it seems really easy
to inadvertently break nvptx.





-- 
Janne Blomqvist

Re: RFC: Building a minimal libgfortran for nvptx

[Tobias Burnus]

Hi all,

since everyone seems to agree that it makes in principle sense to have a 
reduce libgfortran (even though a full one is nicer, if the system 
permits), let's look at the patch itself ;-)


Bernd Schmidt wrote:
The ptx port by its nature is lacking features that are expected on 
normal machines, such as alloca and indirect jumps. We have a subset 
of the C library which contains functions that can be implemented on 
the target (excluding things like file I/O other than printf which is 
a ptx builtin).


It would be good to be able to also build as much of libgfortran as 
possible, and the following patch is what I've been using so far. It 
recognizes the target at configure time and restricts the list of 
compiled files, as well as providing a LIBGFOR_MINIMAL define for 
#ifdef tests. There's also a new file minimal.c which contains 
alternative implementations of some functionality (using printf to 
write error messages rather than fprintf and such). Constructors are 
currently unimplemented on ptx and therefore the init function is 
commented out.


Does printf work? I thought I/O is not supported? Or does it just accept 
it for linking and drop it? I think Janne's patch has already dealt with 
the issue of stack allocation.


What I dislike a bit about the feature is that it is not clear what 
features will be supported for LIBGFOR_MINIMAL. Maybe configure.ac would 
be a good place to describe which features are included there (e.g. no 
I/O but printf etc.) and which aren't. That will make it easier to see 
what has to be modifed if one will add another differently bare system 
later on.


+#if 0
+/* Initialize the runtime library.  */

#if 0'ed code shouldn't be in the patch.


+@LIBGFOR_MINIMAL_FALSE@gfor_helper_src = intrinsics/associated.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/abort.c intrinsics/access.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/args.c \
+@LIBGFOR_MINIMAL_FALSE@intrinsics/bit_intrinsics.c \
...
+@LIBGFOR_MINIMAL_TRUE@gfor_helper_src = intrinsics/associated.c \
+@LIBGFOR_MINIMAL_TRUE@ intrinsics/abort.c intrinsics/args.c \
...

Can't one write this differently, avoiding having most lines repeated 
and only a few missing from the second set? I am not an automake expert, 
but can't one simply split this into different make variables - one set 
which is in both and one which is not in minimal?


Otherwise, the RFC looks good to me.

Tobias

Tobias

RFC: Building a minimal libgfortran for nvptx

[Bernd Schmidt]

The ptx port by its nature is lacking features that are expected on 
normal machines, such as alloca and indirect jumps. We have a subset 
of the C library which contains functions that can be implemented on the 
target (excluding things like file I/O other than printf which is a ptx 
builtin).


It would be good to be able to also build as much of libgfortran as 
possible, and the following patch is what I've been using so far. It 
recognizes the target at configure time and restricts the list of 
compiled files, as well as providing a LIBGFOR_MINIMAL define for #ifdef 
tests. There's also a new file minimal.c which contains alternative 
implementations of some functionality (using printf to write error 
messages rather than fprintf and such). Constructors are currently 
unimplemented on ptx and therefore the init function is commented out.


Comments on the approach, do the Fortran maintainers have a preference 
how this should look? The whole thing is good enough to substantially 
reduce the number of failures when trying to run the Fortran testsuites 
on nvptx (although many remain).



Bernd
Index: libgfortran/Makefile.am
===
--- libgfortran/Makefile.am.orig
+++ libgfortran/Makefile.am
@@ -77,6 +77,14 @@ AM_CFLAGS += $(SECTION_FLAGS)
 AM_CFLAGS += $(IEEE_FLAGS)
 AM_FCFLAGS += $(IEEE_FLAGS)
 
+if LIBGFOR_MINIMAL
+AM_CFLAGS += -DLIBGFOR_MINIMAL
+endif
+
+if LIBGFOR_MINIMAL
+gfor_io_src= \
+io/size_from_kind.c
+else
 gfor_io_src= \
 io/close.c \
 io/file_pos.c \
@@ -94,6 +102,7 @@ io/unit.c \
 io/unix.c \
 io/write.c \
 io/fbuf.c
+endif
 
 gfor_io_headers= \
 io/io.h \
@@ -101,6 +110,41 @@ io/fbuf.h \
 io/format.h \
 io/unix.h
 
+if LIBGFOR_MINIMAL
+gfor_helper_src= \
+intrinsics/associated.c \
+intrinsics/abort.c \
+intrinsics/args.c \
+intrinsics/bit_intrinsics.c \
+intrinsics/cshift0.c \
+intrinsics/eoshift0.c \
+intrinsics/eoshift2.c \
+intrinsics/erfc_scaled.c \
+intrinsics/extends_type_of.c \
+intrinsics/fnum.c \
+intrinsics/ierrno.c \
+intrinsics/ishftc.c \
+intrinsics/iso_c_generated_procs.c \
+intrinsics/iso_c_binding.c \
+intrinsics/malloc.c \
+intrinsics/mvbits.c \
+intrinsics/move_alloc.c \
+intrinsics/pack_generic.c \
+intrinsics/selected_char_kind.c \
+intrinsics/size.c \
+intrinsics/spread_generic.c \
+intrinsics/string_intrinsics.c \
+intrinsics/rand.c \
+intrinsics/random.c \
+intrinsics/reshape_generic.c \
+intrinsics/reshape_packed.c \
+intrinsics/selected_int_kind.f90 \
+intrinsics/selected_real_kind.f90 \
+intrinsics/transpose_generic.c \
+intrinsics/unpack_generic.c \
+runtime/in_pack_generic.c \
+runtime/in_unpack_generic.c
+else
 gfor_helper_src= \
 intrinsics/associated.c \
 intrinsics/abort.c \
@@ -165,6 +209,7 @@ intrinsics/unlink.c \
 intrinsics/unpack_generic.c \
 runtime/in_pack_generic.c \
 runtime/in_unpack_generic.c
+endif
 
 if IEEE_SUPPORT
 
@@ -181,6 +226,15 @@ gfor_ieee_src=
 
 endif
 
+if LIBGFOR_MINIMAL
+gfor_src= \
+runtime/bounds.c \
+runtime/compile_options.c \
+runtime/memory.c \
+runtime/minimal.c \
+runtime/string.c \
+runtime/select.c
+else
 gfor_src= \
 runtime/backtrace.c \
 runtime/bounds.c \
@@ -195,6 +249,7 @@ runtime/pause.c \
 runtime/stop.c \
 runtime/string.c \
 runtime/select.c
+endif
 
 i_all_c= \
 $(srcdir)/generated/all_l1.c \
Index: libgfortran/Makefile.in
===
--- libgfortran/Makefile.in.orig
+++ libgfortran/Makefile.in
@@ -54,10 +54,11 @@ POST_UNINSTALL = :
 build_triplet = @build@
 host_triplet = @host@
 target_triplet = @target@
-@IEEE_SUPPORT_TRUE@am__append_1 = ieee/ieee_helper.c
+@LIBGFOR_MINIMAL_TRUE@am__append_1 = -DLIBGFOR_MINIMAL
+@IEEE_SUPPORT_TRUE@am__append_2 = ieee/ieee_helper.c
 
 # dummy sources for libtool
-@onestep_TRUE@am__append_2 = libgfortran_c.c libgfortran_f.f90
+@onestep_TRUE@am__append_3 = libgfortran_c.c libgfortran_f.f90
 subdir = .
 DIST_COMMON = ChangeLog $(srcdir)/Makefile.in $(srcdir)/Makefile.am \
 	$(top_srcdir)/configure $(am__configure_deps) \
@@ -121,9 +122,13 @@ libcaf_single_la_LIBADD =
 am_libcaf_single_la_OBJECTS = single.lo
 libcaf_single_la_OBJECTS = $(am_libcaf_single_la_OBJECTS)
 libgfortran_la_LIBADD =
-am__objects_1 = backtrace.lo bounds.lo compile_options.lo \
-	convert_char.lo environ.lo error.lo fpu.lo main.lo memory.lo \
-	pause.lo stop.lo string.lo select.lo
+@LIBGFOR_MINIMAL_FALSE@am__objects_1 = backtrace.lo bounds.lo \
+@LIBGFOR_MINIMAL_FALSE@	compile_options.lo convert_char.lo \
+@LIBGFOR_MINIMAL_FALSE@	environ.lo error.lo fpu.lo main.lo \
+@LIBGFOR_MINIMAL_FALSE@	memory.lo pause.lo stop.lo string.lo \
+@LIBGFOR_MINIMAL_FALSE@	select.lo
+@LIBGFOR_MINIMAL_TRUE@am__objects_1 = bounds.lo compile_options.lo \
+@LIBGFOR_MINIMAL_TRUE@	memory.lo minimal.lo string.lo select.lo
 am__objects_2 = all_l1.lo all_l2.lo all_l4.lo all_l8.lo all_l16.lo
 am__objects_3 = any_l1.lo any_l2.lo any_l4.lo any_l8.lo any_l16.lo
 am__objects_4 = count_1_l.lo count_2_l.lo count_4_l.lo 

Re: RFC: Building a minimal libgfortran for nvptx

[Steve Kargl]

On Tue, Nov 04, 2014 at 03:34:43PM +0100, Bernd Schmidt wrote:
 The ptx port by its nature is lacking features that are expected on 
 normal machines, such as alloca and indirect jumps. We have a subset 
 of the C library which contains functions that can be implemented on the 
 target (excluding things like file I/O other than printf which is a ptx 
 builtin).
 
 It would be good to be able to also build as much of libgfortran as 
 possible, and the following patch is what I've been using so far. It 
 recognizes the target at configure time and restricts the list of 
 compiled files, as well as providing a LIBGFOR_MINIMAL define for #ifdef 
 tests. There's also a new file minimal.c which contains alternative 
 implementations of some functionality (using printf to write error 
 messages rather than fprintf and such). Constructors are currently 
 unimplemented on ptx and therefore the init function is commented out.
 
 Comments on the approach, do the Fortran maintainers have a preference 
 how this should look? The whole thing is good enough to substantially 
 reduce the number of failures when trying to run the Fortran testsuites 
 on nvptx (although many remain).
 

It is unclear to me from reading the diff whether this patch 
cause gfortran on ptx to knowingly violate the fortran standard.
If the answer is yes, this patch causes gfortran on ptx to
violate the standard, then the patch is IMHO unacceptable.

-- 
steve

Re: RFC: Building a minimal libgfortran for nvptx

[Bernd Schmidt]

On 11/04/2014 04:41 PM, Steve Kargl wrote:

It is unclear to me from reading the diff whether this patch
cause gfortran on ptx to knowingly violate the fortran standard.
If the answer is yes, this patch causes gfortran on ptx to
violate the standard, then the patch is IMHO unacceptable.


I don't have the Fortran standard, but I assume that missing pieces in 
the library would be a violation. However, the alternative is no Fortran 
(library) support at all, which doesn't seem like an improvement. The 
target simply does not allow full language support, even for something 
like C.


Note that the intention is not to support Fortran (or any other 
language) directly targetting ptx code. The only way it's supposed to be 
used is as an accelerator for OpenACC offloading.



Bernd

Re: RFC: Building a minimal libgfortran for nvptx

[FX]

 Comments on the approach, do the Fortran maintainers have a preference how 
 this should look? The whole thing is good enough to substantially reduce the 
 number of failures when trying to run the Fortran testsuites on nvptx 
 (although many remain).

I’m afraid I don’t really see the point. Maybe I’ll say it differently that 
Steve has: what would be the state of Fortran on such platform, and would it 
have users? It looks that, at the very least, your target wouldn’t be able to 
do any I/O.

Unless we have a good reason to think there will be users, because the state of 
support will be good, I don’t see that we should add to the library 
maintainance burden by special-casing targets.

Also: if other targets come along that have this need, how does your strategy 
scale up?

Thanks,
FX

Re: RFC: Building a minimal libgfortran for nvptx

[Bernd Schmidt]

On 11/04/2014 04:59 PM, FX wrote:

Comments on the approach, do the Fortran maintainers have a
preference how this should look? The whole thing is good enough to
substantially reduce the number of failures when trying to run the
Fortran testsuites on nvptx (although many remain).


I’m afraid I don’t really see the point. Maybe I’ll say it
differently that Steve has: what would be the state of Fortran on
such platform, and would it have users? It looks that, at the very
least, your target wouldn’t be able to do any I/O.


It would be used through OpenACC - and that I do believe that does have 
Fortran users, with other compilers. We can make Fortran OpenACC work 
for simple testcases without any runtime library, but IMO it would be 
better to go as far as possible in supporting whatever can be made to 
work. I/O is the major piece that is missing, but you'd expect that to 
be done on the host rather than on the accelerator.



Bernd

Re: RFC: Building a minimal libgfortran for nvptx

[Jakub Jelinek]

On Tue, Nov 04, 2014 at 07:41:42AM -0800, Steve Kargl wrote:
 On Tue, Nov 04, 2014 at 03:34:43PM +0100, Bernd Schmidt wrote:
  The ptx port by its nature is lacking features that are expected on 
  normal machines, such as alloca and indirect jumps. We have a subset 
  of the C library which contains functions that can be implemented on the 
  target (excluding things like file I/O other than printf which is a ptx 
  builtin).
  
  It would be good to be able to also build as much of libgfortran as 
  possible, and the following patch is what I've been using so far. It 
  recognizes the target at configure time and restricts the list of 
  compiled files, as well as providing a LIBGFOR_MINIMAL define for #ifdef 
  tests. There's also a new file minimal.c which contains alternative 
  implementations of some functionality (using printf to write error 
  messages rather than fprintf and such). Constructors are currently 
  unimplemented on ptx and therefore the init function is commented out.
  
  Comments on the approach, do the Fortran maintainers have a preference 
  how this should look? The whole thing is good enough to substantially 
  reduce the number of failures when trying to run the Fortran testsuites 
  on nvptx (although many remain).
  
 
 It is unclear to me from reading the diff whether this patch 
 cause gfortran on ptx to knowingly violate the fortran standard.
 If the answer is yes, this patch causes gfortran on ptx to
 violate the standard, then the patch is IMHO unacceptable.

The point is, if the target can implement just a subset of the Fortran (or
C or C++) standards, then ideally if you use anything that is not supported
would just cause always host fallback, the code will still work, but will
not be offloaded.  So even supporting a subset of the standard is
worthwhile, usually one will just offload the most performance critical
parts of his code.

Jakub

Re: RFC: Building a minimal libgfortran for nvptx

[FX]

 The point is, if the target can implement just a subset of the Fortran (or
 C or C++) standards, then ideally if you use anything that is not supported
 would just cause always host fallback, the code will still work, but will
 not be offloaded.  So even supporting a subset of the standard is
 worthwhile, usually one will just offload the most performance critical
 parts of his code.

Do we have the architecture for that in place in GCC in general, and in the 
Fortran front-end in particular? I’d be interested to see how it works…

FX

Re: RFC: Building a minimal libgfortran for nvptx

[Jakub Jelinek]

On Tue, Nov 04, 2014 at 05:15:52PM +0100, FX wrote:
  The point is, if the target can implement just a subset of the Fortran (or
  C or C++) standards, then ideally if you use anything that is not supported
  would just cause always host fallback, the code will still work, but will
  not be offloaded.  So even supporting a subset of the standard is
  worthwhile, usually one will just offload the most performance critical
  parts of his code.
 
 Do we have the architecture for that in place in GCC in general, and in
 the Fortran front-end in particular?  I’d be interested to see how it
 works…

See https://gcc.gnu.org/wiki/Offloading and kyukhin/gomp4-offload and
branches/gomp-4_0-branch branches.  Both are in the process of being merged
into trunk these days.

Jakub

Re: RFC: Building a minimal libgfortran for nvptx

[Steve Kargl]

On Tue, Nov 04, 2014 at 04:54:54PM +0100, Bernd Schmidt wrote:
 On 11/04/2014 04:41 PM, Steve Kargl wrote:
  It is unclear to me from reading the diff whether this patch
  cause gfortran on ptx to knowingly violate the fortran standard.
  If the answer is yes, this patch causes gfortran on ptx to
  violate the standard, then the patch is IMHO unacceptable.
 
 I don't have the Fortran standard, but I assume that missing pieces in 
 the library would be a violation. However, the alternative is no Fortran 
 (library) support at all, which doesn't seem like an improvement. The 
 target simply does not allow full language support, even for something 
 like C.
 
 Note that the intention is not to support Fortran (or any other 
 language) directly targetting ptx code. The only way it's supposed to be 
 used is as an accelerator for OpenACC offloading.
 

I see.  I get nervous when a patch appears that throws away
a part of the runtime library.  There are typically unintended
consequences, which then becomes a support issue.

-- 
Steve

Re: RFC: Building a minimal libgfortran for nvptx

[FX]

 See https://gcc.gnu.org/wiki/Offloading and kyukhin/gomp4-offload and
 branches/gomp-4_0-branch branches.  Both are in the process of being merged
 into trunk these days.

Thanks for the link, I’ll look into it.
I suppose then it makes sense to provide partial libgfortran support, assuming 
someone is volunteering to make offloading work nicely for Fortran code and 
maintain it.

FX

Re: RFC: Building a minimal libgfortran for nvptx

[Jeff Law]

On 11/04/14 09:11, Jakub Jelinek wrote:

On Tue, Nov 04, 2014 at 07:41:42AM -0800, Steve Kargl wrote:

On Tue, Nov 04, 2014 at 03:34:43PM +0100, Bernd Schmidt wrote:

The ptx port by its nature is lacking features that are expected on
normal machines, such as alloca and indirect jumps. We have a subset
of the C library which contains functions that can be implemented on the
target (excluding things like file I/O other than printf which is a ptx
builtin).

It would be good to be able to also build as much of libgfortran as
possible, and the following patch is what I've been using so far. It
recognizes the target at configure time and restricts the list of
compiled files, as well as providing a LIBGFOR_MINIMAL define for #ifdef
tests. There's also a new file minimal.c which contains alternative
implementations of some functionality (using printf to write error
messages rather than fprintf and such). Constructors are currently
unimplemented on ptx and therefore the init function is commented out.

Comments on the approach, do the Fortran maintainers have a preference
how this should look? The whole thing is good enough to substantially
reduce the number of failures when trying to run the Fortran testsuites
on nvptx (although many remain).



It is unclear to me from reading the diff whether this patch
cause gfortran on ptx to knowingly violate the fortran standard.
If the answer is yes, this patch causes gfortran on ptx to
violate the standard, then the patch is IMHO unacceptable.


The point is, if the target can implement just a subset of the Fortran (or
C or C++) standards, then ideally if you use anything that is not supported
would just cause always host fallback, the code will still work, but will
not be offloaded.  So even supporting a subset of the standard is
worthwhile, usually one will just offload the most performance critical
parts of his code.
Also note there's a reasonable chance that the GPUs will continue to 
evolve and will be able to support more of the standard language 
features.  Not sure if they'll ever do the IO side of Fortran, but they 
could always surprise us.


jeff

Re: RFC: Building a minimal libgfortran for nvptx

[Steve Kargl]

On Tue, Nov 04, 2014 at 10:20:53AM -0700, Jeff Law wrote:
 On 11/04/14 09:11, Jakub Jelinek wrote:
  On Tue, Nov 04, 2014 at 07:41:42AM -0800, Steve Kargl wrote:
  On Tue, Nov 04, 2014 at 03:34:43PM +0100, Bernd Schmidt wrote:
  The ptx port by its nature is lacking features that are expected on
  normal machines, such as alloca and indirect jumps. We have a subset
  of the C library which contains functions that can be implemented on the
  target (excluding things like file I/O other than printf which is a ptx
  builtin).
 
  It would be good to be able to also build as much of libgfortran as
  possible, and the following patch is what I've been using so far. It
  recognizes the target at configure time and restricts the list of
  compiled files, as well as providing a LIBGFOR_MINIMAL define for #ifdef
  tests. There's also a new file minimal.c which contains alternative
  implementations of some functionality (using printf to write error
  messages rather than fprintf and such). Constructors are currently
  unimplemented on ptx and therefore the init function is commented out.
 
  Comments on the approach, do the Fortran maintainers have a preference
  how this should look? The whole thing is good enough to substantially
  reduce the number of failures when trying to run the Fortran testsuites
  on nvptx (although many remain).
 
 
  It is unclear to me from reading the diff whether this patch
  cause gfortran on ptx to knowingly violate the fortran standard.
  If the answer is yes, this patch causes gfortran on ptx to
  violate the standard, then the patch is IMHO unacceptable.
 
  The point is, if the target can implement just a subset of the Fortran (or
  C or C++) standards, then ideally if you use anything that is not supported
  would just cause always host fallback, the code will still work, but will
  not be offloaded.  So even supporting a subset of the standard is
  worthwhile, usually one will just offload the most performance critical
  parts of his code.
 Also note there's a reasonable chance that the GPUs will continue to 
 evolve and will be able to support more of the standard language 
 features.  Not sure if they'll ever do the IO side of Fortran, but they 
 could always surprise us.
 

Thanks for the explanation.  Certainly mapping Fortran's array syntax
and many of the intrinsic subprograms to an accelerator may provide a
nice speed improvement.  It wasn't clear to me from Bernd original message
that the patch was intended for OpenACC offloading.  I was assuming
that ptx was a new cpu architecture, which had limited capabilites.

-- 
Steve

Re: RFC: Building a minimal libgfortran for nvptx

[Jeff Law]

On 11/04/14 08:54, Bernd Schmidt wrote:

Note that the intention is not to support Fortran (or any other
language) directly targetting ptx code. The only way it's supposed to be
used is as an accelerator for OpenACC offloading.
Right.  To reiterate for everyone, offloading is the goal of the nvptx 
port.  Normal C, C++, Fortran code really isn't interesting, though the 
direction the hardware is going should allow more and more C, C++  
Fortran code to be used as-is on the GPU.


Bernd has done a fair amount of work to allow normal-ish C, C++  
Fortran code to run, that's really been to allow running GCC's testsuite 
to shake out the first level bugs in the ptx support.


Jeff

Re: RFC: Building a minimal libgfortran for nvptx

[N.M. Maclaren]

On Nov 4 2014, Jeff Law wrote:

On 11/04/14 09:11, Jakub Jelinek wrote:


The point is, if the target can implement just a subset of the Fortran 
(or C or C++) standards, then ideally if you use anything that is not 
supported would just cause always host fallback, the code will still 
work, but will not be offloaded. So even supporting a subset of the 
standard is worthwhile, usually one will just offload the most 
performance critical parts of his code.


As I see it, this isn't a free-standing compilation environment, but a
component of one for heterogeneous architectures.  There are similar
issues for some embedded systems, in several languages.  That doesn't
fit well with the current build model, unfortunately :-(

Also note there's a reasonable chance that the GPUs will continue to 
evolve and will be able to support more of the standard language 
features.  Not sure if they'll ever do the IO side of Fortran, but they 
could always surprise us.


I am almost certain that the current situation is going to change
significantly, probably by 2020, but there is as yet no indication of
how.  And it wouldn't be entirely reasonable to say nothing should be
done for this sort of use until the situation becomes clear.


Regards,
Nick Maclaren.