Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 02:15:14PM -0500, David Edelsohn wrote: > On Sat, Jan 8, 2022 at 1:59 PM Michael Meissner > wrote: > > > > On Sat, Jan 08, 2022 at 03:18:07PM +0100, Jakub Jelinek wrote: > > > On Sat, Jan 08, 2022 at 03:13:10PM +0100, Thomas Koenig wrote: > > > > > > > > On 08.01.22 15:02, Jakub Jelinek via Fortran wrote: > > > > > Note, as for byteswapping, apparently it wasn't ever working right fox > > > > > the IBM extended real(kind=16) and complex(kind=16). > > > > > > > > The lack of bug reports since the conversion feature was introduced in > > > > 2006, more than 15 years ago, tells us something, I guess... > > > > > > powerpc64le was only introduced in GCC 4.8 in 2013, so slightly less > > > than that, but still. > > > Either nobody interchanges/shares fortran unformatted data between > > > powerpc big and little endian, or if they do, they don't use real(kind=16) > > > or complex(kind=16) in there... > > > > I still wish I had had the forethought when we were setting up the LE ABI to > > change the default 128-bit format to IEEE instead of IBM. But alas, I > > didn't. > > You would still need converters between the big endian IBM format and little > > endian IEEE format, but it would have avoided a lot of the problems where > > GCC > > assumes there is only one floating point format for each size. > > Mike, > > The LE ABI initial target was Power8 and IEEE128 hardware support was > added to Power9. The ABI was a conscious decision. IEEE 128 was not a > viable requirement for the LE ABI at the time of the transition. Yes I know, but my memory is we (the GCC group within IBM) at least knew that IEEE 128-bit was coming towards the end of the ABI definition period. But perhaps not. In any case, it doesn't much matter now, as it is all ancient history. -- Michael Meissner, IBM PO Box 98, Ayer, Massachusetts, USA, 01432 email: meiss...@linux.ibm.com
Re: [power-ieee128] OPEN CONV
On Sat, Jan 8, 2022 at 1:59 PM Michael Meissner wrote: > > On Sat, Jan 08, 2022 at 03:18:07PM +0100, Jakub Jelinek wrote: > > On Sat, Jan 08, 2022 at 03:13:10PM +0100, Thomas Koenig wrote: > > > > > > On 08.01.22 15:02, Jakub Jelinek via Fortran wrote: > > > > Note, as for byteswapping, apparently it wasn't ever working right fox > > > > the IBM extended real(kind=16) and complex(kind=16). > > > > > > The lack of bug reports since the conversion feature was introduced in > > > 2006, more than 15 years ago, tells us something, I guess... > > > > powerpc64le was only introduced in GCC 4.8 in 2013, so slightly less > > than that, but still. > > Either nobody interchanges/shares fortran unformatted data between > > powerpc big and little endian, or if they do, they don't use real(kind=16) > > or complex(kind=16) in there... > > I still wish I had had the forethought when we were setting up the LE ABI to > change the default 128-bit format to IEEE instead of IBM. But alas, I didn't. > You would still need converters between the big endian IBM format and little > endian IEEE format, but it would have avoided a lot of the problems where GCC > assumes there is only one floating point format for each size. Mike, The LE ABI initial target was Power8 and IEEE128 hardware support was added to Power9. The ABI was a conscious decision. IEEE 128 was not a viable requirement for the LE ABI at the time of the transition. Thanks, David
Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 03:18:07PM +0100, Jakub Jelinek wrote: > On Sat, Jan 08, 2022 at 03:13:10PM +0100, Thomas Koenig wrote: > > > > On 08.01.22 15:02, Jakub Jelinek via Fortran wrote: > > > Note, as for byteswapping, apparently it wasn't ever working right fox > > > the IBM extended real(kind=16) and complex(kind=16). > > > > The lack of bug reports since the conversion feature was introduced in > > 2006, more than 15 years ago, tells us something, I guess... > > powerpc64le was only introduced in GCC 4.8 in 2013, so slightly less > than that, but still. > Either nobody interchanges/shares fortran unformatted data between > powerpc big and little endian, or if they do, they don't use real(kind=16) > or complex(kind=16) in there... I still wish I had had the forethought when we were setting up the LE ABI to change the default 128-bit format to IEEE instead of IBM. But alas, I didn't. You would still need converters between the big endian IBM format and little endian IEEE format, but it would have avoided a lot of the problems where GCC assumes there is only one floating point format for each size. -- Michael Meissner, IBM PO Box 98, Ayer, Massachusetts, USA, 01432 email: meiss...@linux.ibm.com
Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 03:13:10PM +0100, Thomas Koenig wrote: > > On 08.01.22 15:02, Jakub Jelinek via Fortran wrote: > > Note, as for byteswapping, apparently it wasn't ever working right fox > > the IBM extended real(kind=16) and complex(kind=16). > > The lack of bug reports since the conversion feature was introduced in > 2006, more than 15 years ago, tells us something, I guess... powerpc64le was only introduced in GCC 4.8 in 2013, so slightly less than that, but still. Either nobody interchanges/shares fortran unformatted data between powerpc big and little endian, or if they do, they don't use real(kind=16) or complex(kind=16) in there... Jakub
Re: [power-ieee128] OPEN CONV
On 08.01.22 15:02, Jakub Jelinek via Fortran wrote: Note, as for byteswapping, apparently it wasn't ever working right fox the IBM extended real(kind=16) and complex(kind=16). The lack of bug reports since the conversion feature was introduced in 2006, more than 15 years ago, tells us something, I guess...
Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 12:10:56PM +0100, Jakub Jelinek via Gcc-patches wrote:
> One reason for that is that neither conversion is lossless, neither format
> is a subset or superset of the other. Yes, IEEE quad has both much bigger
> exponent range (-16382..16383 vs. -1022..1023) and slightly bigger fixed
> precision (113 vs. 106 bits).
> But IBM extended has that weirdo numerically awful flexible precision where
> certain numbers can have much bigger precision than those 106 bits, up to
> 2048+52 or so. So there is rounding in both directions.
> So, after distros switch to -mabi=ieeelongdouble by default or when people
> use -mabi=ieeelongdouble on their programs, they'd better store that format
> into data files by default, without the need of some magic CONVERT= options,
> env vars or command line options. Only in the case where they need to
> interact with -mabi=ibmlongdouble environments, they need to take some
> action.
Note, as for byteswapping, apparently it wasn't ever working right fox
the IBM extended real(kind=16) and complex(kind=16).
Because unlike IEEE extended or integral types, it seems powerpc*-*-*
doesn't actually fully byteswap those between little and big endian.
Proof:
long double a =
0.L;
compiled little endian IBM long double:
.size a, 16
a:
.long 1431655765
.long 1070945621
.long 1431655766
.long 1014322517
compiled big endian IBM long double:
.size a, 16
a:
.long 1070945621
.long 1431655765
.long 1014322517
.long 1431655766
compiled little endian IEEE long double:
.size a, 16
a:
.long 1431655765
.long 1431655765
.long 1431655765
.long 1073567061
compiled big endian IEEE long double:
.size a, 16
a:
.long 1073567061
.long 1431655765
.long 1431655765
.long 1431655765
where the numbers in .long arguments are 32-bit numbers stored in the
selected endianity. Compiled with -mlong-double-64 little endian:
.size a, 8
a:
.long 1431655765
.long 1070945621
and big endian:
.size a, 8
a:
.long 1070945621
.long 1431655765
Unless I'm misreading this, for IEEE long double, or double (and I bet float
too) byteswapping the whole numbers is what is needed for interoperability
between powerpc64{,le}-linux, for IBM long double we'd actually want to
byteswap it as 2 real(kind=8) numbers and not one real(kind=16) one, i.e.
the numbers are always stored as the more significant double followed by
less significant double in memory.
Jakub
Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 12:00:38PM +0100, Jakub Jelinek via Gcc-patches wrote: > And IMHO the default like for byte-swapping should be the native > format, i.e. the one the program actually used. One reason for that is that neither conversion is lossless, neither format is a subset or superset of the other. Yes, IEEE quad has both much bigger exponent range (-16382..16383 vs. -1022..1023) and slightly bigger fixed precision (113 vs. 106 bits). But IBM extended has that weirdo numerically awful flexible precision where certain numbers can have much bigger precision than those 106 bits, up to 2048+52 or so. So there is rounding in both directions. So, after distros switch to -mabi=ieeelongdouble by default or when people use -mabi=ieeelongdouble on their programs, they'd better store that format into data files by default, without the need of some magic CONVERT= options, env vars or command line options. Only in the case where they need to interact with -mabi=ibmlongdouble environments, they need to take some action. Jakub
Re: [power-ieee128] OPEN CONV
On Sat, Jan 08, 2022 at 11:07:24AM +0100, Thomas Koenig wrote: > I have tried to unravel the different cases here, I count six > (lumping together the environment variables, the CONVERT specifier > and -fconvert, and leaving out the byte swapping) > > CompilerConvert Read action Write action > > IEEENone NoneNone > IEEEIEEE NoneNone > IEEEIBM IBM->IEEE IEEE->IBM > > IBM None NoneNone > IBM IEEE IEEE->IBM IBM->IEEE > IBM IBM NoneNone > > From this table, it is clear that the compiler has to inform > the library about the option it is using, I think it is best > encoded in the number passed to _gfortran_set_convert. Whether the compiler is using IEEE or IBM real(kind=16) or complex(kind=16) for a particular spot (which doesn't have to be the same in the whole program) is known to the library by the kind argument it provides to the I/O routines, if it is kind=16, it is IBM, if it is kind=17, it is IEEE. See the patch I've posted, which does one thing when the runtime kind (i.e. abi_kind on the compiler side) is 17 and convert says r16_ibm, and another thing when runtime kind is 16 and convert says r16_ieee. Other cases shouldn't need conversion. And IMHO the default like for byte-swapping should be the native format, i.e. the one the program actually used. The only thing that should be encoded in _gfortran_set_convert is -fconvertWHATEVER command line option IMO. Jakub
Re: [power-ieee128] OPEN CONV
On 07.01.22 22:48, Jakub Jelinek wrote: On Fri, Jan 07, 2022 at 10:40:50PM +0100, Thomas Koenig wrote: One thing that one has to watch out for is a big-endian IBM long double file, so the byte swapping will have to be done before assigning the value. I've tried to handle that right, i.e. on unformatted read with byte-swapping and r16 <-> r17 conversions first do byte-swapping and then r16 <-> r17 conversions, while for unformatted writes first r16 <-> r17 conversions and then byte-swapping. I have tried to unravel the different cases here, I count six (lumping together the environment variables, the CONVERT specifier and -fconvert, and leaving out the byte swapping) CompilerConvert Read action Write action IEEENone NoneNone IEEEIEEE NoneNone IEEEIBM IBM->IEEE IEEE->IBM IBM None NoneNone IBM IEEE IEEE->IBM IBM->IEEE IBM IBM NoneNone From this table, it is clear that the compiler has to inform the library about the option it is using, I think it is best encoded in the number passed to _gfortran_set_convert. Old programs should continue to run with the new library, so the absence of a call to _gfortran_set_convert, or a call which sets byte swapping, should have the old meaning, i.e IBM long double. A program which uses IEEE long double should then call _gfortran_set_convert with a suitable argument to let the library know what to do, just in case. I think this is what I will start working on. Best regards Thomas
Re: [power-ieee128] OPEN CONV
On Fri, Jan 07, 2022 at 10:40:50PM +0100, Thomas Koenig wrote: > One thing that one has to watch out for is a big-endian IBM long double > file, so the byte swapping will have to be done before assigning > the value. I've tried to handle that right, i.e. on unformatted read with byte-swapping and r16 <-> r17 conversions first do byte-swapping and then r16 <-> r17 conversions, while for unformatted writes first r16 <-> r17 conversions and then byte-swapping. Jakub
Re: [power-ieee128] OPEN CONV
On 07.01.22 20:52, Jakub Jelinek wrote: Here is completely untested patch that implements something, but doesn't implement the gcc option stuff, nor the CONVERT= syntax to supply multiple conversion options nor done anything about env var nor any testcases. But it tries to have the native/swap/big/little choice orthogonal from native/r16_ieee/r16_ibm with r16_ieee and r16_ibm only supported on ppc64le-linux. For INQUIRE it has the so far perhaps manageable set of possibilities handled so that it uses string literals and doesn't have to construct those strings at runtime (haven't studied how it would need to be done). I'm afraid I don't know that stuff enough to move forward from this. That looks like the direction to go. I will take it from there and see if I can find some hours over the weekend to work on something. Unfortunately, my daytime job will kick in again on Monday :-) One thing that one has to watch out for is a big-endian IBM long double file, so the byte swapping will have to be done before assigning the value. Best regards Thomas
Re: [power-ieee128] OPEN CONV
On Fri, Jan 07, 2022 at 11:26:15AM +0100, Thomas Koenig wrote:
> In
>
> https://gcc.gnu.org/pipermail/fortran/2021-October/056895.html
>
> I made a suggestion how how the format could look like. I used
> a plus sign instead of a comma because I thought the environment
> variable should follow the same syntax as the CONVERT specifier,
> and I did not want to think about having commas in there :-)
>
> Thinking about this again after some time, I think the syntax of
> the environment variable would be clearer if the keywords for
> the two conversions were separate, so somethig like
>
> big_endian;r16_ieee;r16_ibm:10-20;
>
> for the environment variable and
>
> CONVERT="big_endian,r16_ibm"
>
> would probably be better.
Here is completely untested patch that implements something,
but doesn't implement the gcc option stuff, nor the CONVERT=
syntax to supply multiple conversion options nor done anything
about env var nor any testcases.
But it tries to have the native/swap/big/little choice orthogonal from
native/r16_ieee/r16_ibm with r16_ieee and r16_ibm only supported on
ppc64le-linux.
For INQUIRE it has the so far perhaps manageable set of possibilities
handled so that it uses string literals and doesn't have to construct
those strings at runtime (haven't studied how it would need to be done).
I'm afraid I don't know that stuff enough to move forward from this.
--- gcc/fortran/libgfortran.h.jj2022-01-07 18:41:55.473722388 +0100
+++ gcc/fortran/libgfortran.h 2022-01-07 19:14:23.881784305 +0100
@@ -86,14 +86,16 @@ along with GCC; see the file COPYING3.
#define GFC_INVALID_UNIT -3
/* Possible values for the CONVERT I/O specifier. */
-/* Keep in sync with GFC_FLAG_CONVERT_* in gcc/flags.h. */
+/* Keep in sync with GFC_FLAG_CONVERT_* in gcc/flag-types.h. */
typedef enum
{
GFC_CONVERT_NONE = -1,
GFC_CONVERT_NATIVE = 0,
GFC_CONVERT_SWAP,
GFC_CONVERT_BIG,
- GFC_CONVERT_LITTLE
+ GFC_CONVERT_LITTLE,
+ GFC_CONVERT_R16_IEEE = 4,
+ GFC_CONVERT_R16_IBM = 8
}
unit_convert;
--- gcc/flag-types.h.jj 2022-01-07 18:41:55.452722678 +0100
+++ gcc/flag-types.h2022-01-07 19:13:55.953170776 +0100
@@ -424,7 +424,9 @@ enum gfc_convert
GFC_FLAG_CONVERT_NATIVE = 0,
GFC_FLAG_CONVERT_SWAP,
GFC_FLAG_CONVERT_BIG,
- GFC_FLAG_CONVERT_LITTLE
+ GFC_FLAG_CONVERT_LITTLE,
+ GFC_FLAG_CONVERT_R16_IEEE = 4,
+ GFC_FLAG_CONVERT_R16_IBM = 8
};
--- libgfortran/io/open.c.jj2022-01-07 18:41:56.078714031 +0100
+++ libgfortran/io/open.c 2022-01-07 19:19:11.582780100 +0100
@@ -153,6 +153,10 @@ static const st_option convert_opt[] =
{ "swap", GFC_CONVERT_SWAP},
{ "big_endian", GFC_CONVERT_BIG},
{ "little_endian", GFC_CONVERT_LITTLE},
+#ifdef HAVE_GFC_REAL_17
+ { "r16_ieee", GFC_CONVERT_R16_IEEE},
+ { "r16_ibm", GFC_CONVERT_R16_IBM},
+#endif
{ NULL, 0}
};
@@ -820,7 +824,14 @@ st_open (st_parameter_open *opp)
else
conv = compile_options.convert;
}
-
+
+ flags.convert = 0;
+
+#ifdef HAVE_GFC_REAL_17
+ flags.convert = conv & (GFC_CONVERT_R16_IEEE | GFC_CONVERT_R16_IBM);
+ conv &= ~(GFC_CONVERT_R16_IEEE | GFC_CONVERT_R16_IBM);
+#endif
+
switch (conv)
{
case GFC_CONVERT_NATIVE:
@@ -840,7 +851,7 @@ st_open (st_parameter_open *opp)
break;
}
- flags.convert = conv;
+ flags.convert |= conv;
if (flags.position != POSITION_UNSPECIFIED
&& flags.access == ACCESS_DIRECT)
--- libgfortran/io/transfer.c.jj2022-01-07 18:41:56.080714003 +0100
+++ libgfortran/io/transfer.c 2022-01-07 20:43:36.146920392 +0100
@@ -1126,7 +1126,11 @@ unformatted_read (st_parameter_dt *dtp,
size *= GFC_SIZE_OF_CHAR_KIND(kind);
read_block_direct (dtp, dest, size * nelems);
- if (unlikely (dtp->u.p.current_unit->flags.convert == GFC_CONVERT_SWAP)
+ int convert = dtp->u.p.current_unit->flags.convert;
+#ifdef HAVE_GFC_REAL_17
+ convert &= ~(GFC_CONVERT_R16_IEEE | GFC_CONVERT_R16_IBM);
+#endif
+ if (unlikely (convert == GFC_CONVERT_SWAP)
&& kind != 1)
{
/* Handle wide chracters. */
@@ -1144,6 +1148,48 @@ unformatted_read (st_parameter_dt *dtp,
}
bswap_array (dest, dest, size, nelems);
}
+#ifdef HAVE_GFC_REAL_17
+ if ((dtp->u.p.current_unit->flags.convert & GFC_CONVERT_R16_IEEE)
+ && kind == 16
+ && (type == BT_REAL || type == BT_COMPLEX))
+{
+ if (type == BT_COMPLEX && size == 32)
+ {
+ nelems *= 2;
+ size /= 2;
+ }
+ char *pd = dest;
+ for (size_t i = 0; i < nelems; i++)
+ {
+ GFC_REAL_16 r16;
+ GFC_REAL_17 r17;
+ memcpy (, pd, 16);
+ r16 = r17;
+ memcpy (pd, , 16);
+ pd += size;
+ }
+}
+ else if ((dtp->u.p.current_unit->flags.convert & GFC_CONVERT_R16_IBM)
+ && kind == 17
+ && (type == BT_REAL || type == BT_COMPLEX))
+{
+ if (type == BT_COMPLEX && size == 32)
+ {
+ nelems *= 2;
+ size /= 2;
+ }
+
Re: [power-ieee128] OPEN CONV
On 07.01.22 10:22, Jakub Jelinek wrote: On Thu, Jan 06, 2022 at 09:01:54PM +0100, Thomas Koenig wrote: On 06.01.22 06:00, Michael Meissner via Fortran wrote: What is still missing is the conversion for unformatted I/O, both ways. I'll start doing some stuff on it. Just one question: What are functions that I can use to convert from IBM long double to IEEE and long double and vice versa? It was in an e-mail somewhere, but I cannot find it at the moment. So, what's the plan for that? Can't find CONVERT= in Fortran 2018, so I assume it is a non-standard extension, Correct. https://www.intel.com/content/www/us/en/develop/documentation/fortran-compiler-oneapi-dev-guide-and-reference/top/language-reference/file-operation-i-o-statements/open-statement-specifiers/open-convert-specifier.html#open-convert-specifier documents the Intel one We followed Intel on that one. NAG also has it (although the details differ). and we accept CONVERT='native' CONVERT='swap' CONVERT='big_endian' CONVERT='little_endian' Now, I suppose for powerpc64le we want to add some more, but the question is how they play together with the byteswapping and how to name them, so that it is clear they talk about REAL/COMPLEX KIND=16 format and nothing else. Can we (or do we) want to allow multiple comma separated strings from the orthogonal choices, like CONVERT='big_endian,ibm_extended' CONVERT='swap,ieee_extended' or add ibm_extended, ieee_extended and strings that combine those with swap, big_endian and little_endian ibm_extended_swap, ieee_extended_little etc.? In https://gcc.gnu.org/pipermail/fortran/2021-October/056895.html I made a suggestion how how the format could look like. I used a plus sign instead of a comma because I thought the environment variable should follow the same syntax as the CONVERT specifier, and I did not want to think about having commas in there :-) Thinking about this again after some time, I think the syntax of the environment variable would be clearer if the keywords for the two conversions were separate, so somethig like big_endian;r16_ieee;r16_ibm:10-20; for the environment variable and CONVERT="big_endian,r16_ibm" would probably be better. Best regards Thomas
