Re: VLA representation in GCC internals

[Martin Uecker via Gcc]

Am Samstag, dem 09.11.2024 um 11:29 +0100 schrieb Alejandro Colomar via Gcc:
> On Sat, Nov 09, 2024 at 09:38:45AM GMT, Martin Uecker wrote:
> > Am Samstag, dem 09.11.2024 um 00:54 +0100 schrieb Alejandro Colomar via Gcc:
> > > Hi Martin,
> > > 
> > > I'm in the process of rebasing my __countof__ changes after your patch
> > > that fixes support for [*] and [0].
> > > 
> > > I should update the implementation of the following function:
> > > 
> > >   static bool
> > >   is_top_array_vla (tree type)
> > >   {
> > > bool zero, star, var;
> > > tree d;
> > > 
> > > if (TREE_CODE (type) != ARRAY_TYPE)
> > >   return false;
> > > if (!COMPLETE_TYPE_P (type))
> > >   return false;
> > > 
> > > d = TYPE_DOMAIN (type);
> > > zero = !TYPE_MAX_VALUE (d);
> > > star = (zero && C_TYPE_VARIABLE_SIZE (type));
> > > if (star)
> > >   return true;
> > > if (zero)
> > >   return false;
> > > 
> > > var = (TREE_CODE (TYPE_MIN_VALUE (d)) != INTEGER_CST
> > >   || TREE_CODE (TYPE_MAX_VALUE (d)) != INTEGER_CST);
> > > return var;
> > >   }
> > > 
> > > The 'star' calculation should be updated.  Would you mind proposing an
> > > implementation of this function that works with your changes?  Thanks!
> > > 
> > You can just eliminate the code for the star as it would now
> > automatically end up as variable.
> 
> Thanks!
> 
> Have a lovely day!
> Alex

Just committed, so you could rebase on trunk now.

I also wanted to change zero representation to be the same as
in the C++ FE, but I need to rethink this.  Maybe this change
already solves most of our problems.

BTW: My main practical issue with zero-sized arrays is that the
UB sanitizers triggers for zero-sized variable arrays.

Martin

Re: VLA representation in GCC internals

[Martin Uecker via Gcc]

Am Samstag, dem 09.11.2024 um 00:54 +0100 schrieb Alejandro Colomar via Gcc:
> Hi Martin,
> 
> I'm in the process of rebasing my __countof__ changes after your patch
> that fixes support for [*] and [0].
> 
> I should update the implementation of the following function:
> 
>   static bool
>   is_top_array_vla (tree type)
>   {
> bool zero, star, var;
> tree d;
> 
> if (TREE_CODE (type) != ARRAY_TYPE)
>   return false;
> if (!COMPLETE_TYPE_P (type))
>   return false;
> 
> d = TYPE_DOMAIN (type);
> zero = !TYPE_MAX_VALUE (d);
> star = (zero && C_TYPE_VARIABLE_SIZE (type));
> if (star)
>   return true;
> if (zero)
>   return false;
> 
> var = (TREE_CODE (TYPE_MIN_VALUE (d)) != INTEGER_CST
>   || TREE_CODE (TYPE_MAX_VALUE (d)) != INTEGER_CST);
> return var;
>   }
> 
> The 'star' calculation should be updated.  Would you mind proposing an
> implementation of this function that works with your changes?  Thanks!
> 
You can just eliminate the code for the star as it would now
automatically end up as variable.

Martin

Re: 'defer' (n3199) concerns

[Martin Uecker via Gcc]

Am Freitag, dem 08.11.2024 um 16:40 +0100 schrieb Alejandro Colomar via Gcc:
> Hi JeanHeyd,
> 
> I was involved this week in a fix for a bug I wrote some months ago
> about a call to free(3) with a bad pointer.
> 
> The simplest reproducer is:
> 
>   $ cat strsep_bad.c
>   #include 
>   #include 
>   #include 
> 
>   int
>   main(void)
>   {
>   char  *p;
> 
>   p = strdup("123;");
>   strsep(&p, ";");
>   free(p);
> 
>   puts("done");
>   }
> 
>   $ cc -Wall -Wextra strsep.c 
>   $ ./a.out 
>   free(): invalid pointer
>   Aborted
> 
> A fix for that program is to store a copy of the original pointer:
> 
>   $ cat strsep_good.c
>   #include 
>   #include 
>   #include 
> 
>   int
>   main(void)
>   {
>   char  *p, *dup;
> 
>   p = dup = strdup("123;");
>   strsep(&p, ";");
>   free(dup);
> 
>   puts("done");
>   }
> 
> While I could sympathize with 'defer', I'm wondering if it may simplify
> some code avoiding goto's, at the expense of being obscurely dangerous
> in other cases like this one.  I suspect one could blindly attempt to do
> the following:
> 
>   $ cat strsep_bad_defer.c
> #include 
> #include 
> #include 
> 
> int
> main(void)
> {
> char  *p;
> 
> p = strdup("123;");
>   defer free(p);
> 
> strsep(&p, ";");
> 
> printf("done");
> }
> 
> Since 'p' in free(p) is evaluated after strsep(3), it is equivalent to
> the first program, which is bogus.  I think goto better codifies the
> evaluation order, and allows the programmer to think about what it's
> doing.
> 
> So defer would only be good for the common case, which is usually
> simple-enough that goto should be enough.  And it's bad in those corner
> cases where you really to be careful.  I think I've changed my mind
> about defer to not want it.
> 
> I wanted to have this thought written in a mailing list to be able to
> reference it.

When we wrote the original proposal for C23 Robert had a collection
of code examples before and after rewriting to use defer.  At that
time I somewhat lost a bit of interest in the feature ;-)   But many 
people seem to like it.   I think it would be useful to revisit
these (and more) realistic code examples to get some clarity.

Martin

Re: examples of (so called) time-travel optimisations in GCC?

[Martin Uecker via Gcc]

Am Samstag, dem 26.10.2024 um 18:55 +0200 schrieb Richard Biener via Gcc:
> 
> > Am 26.10.2024 um 17:30 schrieb Iain Sandoe :
> > 
> > Hi,
> > 
> > The background here is that I made a trial implementation of P1494r4 - 
> > std::observable() - and want to produce testcases.
> > 
> > —— so …..
> > 
> > I am looking for either examples where GCC produces time-travel 
> > optimisation (or alternately some evidence that these are not expected).
> > 

A conforming compiler can not do time-travel optimizations
across function calls. For a function call the compiler must
assume  that it exits the program or uses a non-local return,
so it can not assume that UB that comes later will happen.

This then typically applies to I/O which is hidden behind
function calls.

Where compilers do not observe this, this causes bugs, e.g.
this old bug which broke Postgresql:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41239

Or this recent in MSVC which I found when investigating
this question (now fixed):
https://developercommunity.visualstudio.com/t/Invalid-optimization-in-CC/10337428?q=muecker

C23 clarifies that time-travel is not allowed (it was never
really supported by the wording in the C standard).

One area where GCC is not conforming are volatile accesses:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104800

Martin

> > (In case someone is not familiar with the time-travel analogy) a simple 
> > example from P1494:
> > 
> > void b(int &r, int *p) {
> >  if (!p) std::fprintf(stderr, "count: %d\n", ++r);
> >  if (!p) std::fprintf(stderr, "p is null\n");
> >  *p += r; // p may be assumed non-null
> > }
> > 
> > Since dereferencing p is UB, the compiler may assume that p is non-null - 
> > and therefore elide the check(s) before (thus time-travel).
> > 
> > In practice, (at least on the platform I am testing on)  GCC trunk does not 
> > do this - it does reorder code to eliminate the second null test - but it 
> > still prints both sets of output.
> 
> I think we at least do not optimistically propagate backwards - that is, the 
> conditional code may not exit or infinitely loop.  Printf due to callbacks 
> might, so I suggest to rework to tell GCC the conditional code will still 
> make sure the dereference is reached.

Re: C23 status on cppreference

[Martin Uecker via Gcc]

Am Mittwoch, dem 16.10.2024 um 13:27 +0200 schrieb Jakub Jelinek via Gcc:
> Hi!
> 
> https://en.cppreference.com/w/c/compiler_support
> has a table with compiler support for C23.
> I've added #embed and [[unsequenced]]/[[reproducible]] in there
> yesterday, but am wondering about the accuracy of the rest.
> 
> Given the switch to -std=gnu23 preparation, I wonder what is still
> unimplemented in GCC 15.
> The cppreference page mentions as unimplemented on the GCC side
> N2653 - Type change of u8 string literals
> and as only partially implemented
> N2341 - IEEE 754 decimal floating-point types
> N2601 - IEEE 754 interchange and extended types
> N2900 - Empty initializers
> N2934 - New spelling of keywords
> N3033 - __VA_OPT__
> with following comments in the wiki source:
> N2341: Only TR 24732 mentioned; some requirements in core language missing
> N2601: Only TS 18661-3 mentioned; the status of conformance is unknown
> N2900: missing support for scalars and VLAs

N2900 should be supported since 13.1

Martin

> 
> Are some of the papers/features known to be fully implemented (since which
> version)?  E.g. for __VA_OPT__ I remember doing (and Jason too) various fixes
> in the past few years, like PR89971, PR103415, PR101488.  Not really sure
> what exactly C23 requires.
> 
>   Jakub
> 

Re: #pragma once behavior

[Martin Uecker via Gcc]

There was a recent related proposal for C23.

https://www9.open-std.org/JTC1/SC22/WG14/www/docs/n2896.htm

See also the email by Linus Torvalds referenced in
this paper.

Note that this proposal was not adopted for ISO C23.
I can't find when it was discussed,  but IIRC the general
criticism was that the regular form is not reliable and
difficult to standardize any specific rules and that the
form with ID does not add much value over traditional
include guards.

Martin



Am Freitag, dem 06.09.2024 um 00:03 -0500 schrieb Jeremy Rifkin:
> Hello,
> 
> I'm looking at #pragma once behavior among the major C/C++ compilers as
> part of a proposal paper for standardizing #pragma once. (This is
> apparently a very controversial topic)
> 
> To put my question up-front: Would GCC ever be open to altering its
> #pragma once behavior to bring it more in-line with behavior from other
> compilers and possibly more in-line with what users expect?
> 
> To elaborate more:
> 
> Design decisions for #pragma once essentially boil down to a file-based
> definitions vs a content-based definition of "same file".
> 
> A file-based definition is easier to reason about and more in-line with
> what users expect, however, distinct copies of headers can't be handled
> and multiple mount points are problematic.
> 
> A content-based definition works for distinct copies, multiple mount
> points, and is completely sufficient 99% of the time, however, it could
> potentially break in hard-to-debug ways in a few notable cases (more
> information later).
> 
> Currently the three major C/C++ compilers treat #pragma once very differently:
> - GCC uses file mtime + file contents
> - Clang uses inodes
> - MSVC uses file path
> 
> None of the major compilers have documented their #pragma once semantics.
> 
> In practice all three of these approaches work pretty well most of the
> time (which is why people feel comfortable using #pragma once). However,
> they can each break in their own ways.
> 
> As mentioned earlier, clang and MSVC's file-based definitions of "same
> file" break for multiple mount points and multiple copies of the same
> header. MSVC's approach breaks for symbolic links and hard links.
> 
> GCC's hybrid approach can break in surprising ways. I have three
> examples to share:
> 
> Example 1:
> 
> Consider a scenario such as:
> 
> usr/
>   include/
> library_a/
>   library_main.hpp
>   foo.hpp
> library_b/
>   library_main.hpp
>   foo.hpp
> src/
>   main.cpp
> 
> main.cpp:
> #include "library_a/library_main.hpp"
> #include "library_b/library_main.hpp"
> 
> And both library_main.hpp's have:
> #pragma once
> #include "foo.hpp"
> 
> Example 2:
> 
> namespace v1 {
> #include "library_v1.hpp"
> }
> namespace v2 {
> #include "library_v2.hpp"
> }
> 
> Where both library headers include their own copy of a shared header
> using #pragma once.
> 
> Example 3:
> 
> usr/
>   include/
> library/
>   library.hpp
>   vendored-dependency.hpp
> src/
>   main.cpp
>   vendored-dependency.hpp
> 
> main.cpp:
> #include "vendored-dependency.hpp"
> #include 
> 
> library.hpp:
> #pragma once
> #include "vendored-dependency.hpp"
> 
> Assuming the same contents byte-for-byte of vendored-dependency.hpp, and
> it uses #pragma once.
> 
> Each of these examples are plausible scenarios where two files with the
> same contents could be #included. In each example, on GCC, the code can
> work or break based on mtime:
> - Example 1: Breaks if mtimes for library_main.hpp happen to be the same
> - Example 2: Breaks if mtimes for the shared dependency copies happen to
> be the same
> - Example 3: Only works if mtimes are the same
> 
> File mtimes can happen to match sometimes, e.g. in a fresh git clone.
> However, this is a rather fickle criteria to rely on and could easily
> diverge in the middle of development. Notably, Example 2 was shared with
> me as an example where #pragma once worked great in development and
> broke in CI.
> 
> Additionally, while GCC's approach might be able to handle multiple
> mounts better than other approaches, it can still break under multiple
> mounts if mtime resolution differs.
> 
> Obviously there is no silver bullet for making #pragma once work
> perfectly all the time, however, I think it's easier to provide clear
> guarantees for #pragma once behavior when the definition of "same file"
> is based on file identity on device, i.e. device id + inode.
> 
> Would GCC ever consider using device id + inode instead of mtime +
> contents for #pragma once? 
> 
> I presume the primary reason against changing the mtime + file contents
> approach in GCC would be caution over breaking any existing use. While
> the three examples above are cases where fickle mtime can be
> problematic, and I can't imagine any situations where mtime could
> reliably be relied upon, I do understand the degree of caution required
> for changes like this.
> 
> 
> Cheers,
> Jeremy

Re: stack arenas using alloca

[Martin Uecker via Gcc]

Am Freitag, dem 23.08.2024 um 15:46 +1200 schrieb Michael Clark via Gcc:
> On 8/23/24 15:24, Michael Clark wrote:
> > On 8/15/24 06:24, Michael Clark wrote:
> > > Hi Folks,
> > > 
> > like I said this is crazy talk as alloca isn't even in the C standard. 
> > but VLAs are, and the current implementation of VLAs depends on alloca.
> 
> one more thing. it doesn't require PT_GNU_STACK or writable stacks like 
> GCC nested functions. :-) so I think it is safer but it does have safety 
> issues, mostly related to stack overflows but its going to need some 
> careful analysis with respect to ROP. returning deduced VLA types with 
> bounds references in frame metadata and a chain of frame pointers for 
> dynamic alloca might help. also my experiment in LLVM needs clobbers in 
> the parent frame because while I was able to modify the epilogue based 
> on an attribute, I so far haven't figured out how to make it look to the 
> translator like an alloca in the parent frame so that subsequent 
> references to earlier locals save their stack pointer in a register.
> 
> this was my first test function:
> 
> char* f(int n) __attribute__((allocareturn))
> {
>  static const char* fmt = "num-%d";
>  int len = snprintf(0, 0, fmt, n);
>  char *str = __builtin_alloca(len+1);
>  snprintf(str, len+1, fmt, n);
>  return str;
> }
> 
> but this also works:
> 
> char* f(int n) __attribute__((allocareturn))
> {
>  static const char* fmt = "num-%d";
>  int len = snprintf(0, 0, fmt, n);
>  char str[len+1];
>  snprintf(str, len+1, fmt, n);
>  return str;
> }
> 
> and this is even better:
> 
> f(int n) -> auto
> {
>  static const char* fmt = "num-%d";
>  int len = snprintf(0, 0, fmt, n);
>  char str[len+1];
>  snprintf(str, len+1, fmt, n);
>  return str;
> }
> 
> like a reference in an implicit copy constructor for a structure.
> 
> struct anon123456 { int &n; int arr[n]; };
> 
> that would be positioned at the end of the fixed frame so that the 
> parent function knows where the bound is. like I said. crazy talk.

In your example, the length would already be known to the caller.

In this case, this should be easier.  Or do you also want this to
work for when the length is computed inside the callee?


If you return a structure, this would typically (always) returned
in a memory region allocated by the caller. So for a known constant
size you can allocate on the caller stack already:

struct foo { char buf[128]; } f();


In GCC this also works for dynamic size, when you put a VLA inside
the struct

struct foo { char buf[n]; } f();

but this is only allowed as a nested function.

https://godbolt.org/z/YTY6MMf15


But in principle, this works already.I we syntactically allowed

auto f(int n) -> struct foo { char buf[n]; };

then this should work also at file scope for a caller supplied size.


If the size needs to be computed in the callee then this is much
harder.

Martin





> 
> Michael.

Re: Spurious -Wsequence-point with auto

[Martin Uecker via Gcc]

Am Freitag, dem 09.08.2024 um 01:10 +0200 schrieb Alejandro Colomar via Gcc:
> Hi,
> 
> I've seen some bogus warning in GCC that suggests that some use of auto
> may cause undefined behavior (due to double evaluation).
> 
> $ cat auto.c 
> #include 
> 
> int
> main(void)
> {
>   int  i = 3;
>   int  a[2 * i];
>   int  (*p)[2 * i];
> 
>   i = 1;
>   p = &a;
>   auto q = p + i++; 
> }
> $ gcc -Wsequence-point -std=c23 auto.c 
> auto.c: In function ‘main’:
> auto.c:12:23: warning: operation on ‘i’ may be undefined [-Wsequence-point]
>12 | auto q = p + i++;
>   |  ~^~
> 
> 
> I originally found this problem here:
> 
> 
> And I suspect the warning is confusing auto with typeof().  If typeof()
> was used, this would indeed cause double evaluation, once before the =
> and once after the =, causing UB.
> 
> Maybe it's due to how auto is implemented?  I suspect it's doing
> internally:
> 
>   typeof(p + i++) q = p + i++;
> 
> and that would be problematic.  I can reproduce it with both gcc-13 and
> gcc-14.
> 

Can you file a bug if there isn't one already?  

BTW: Double evaluation is not only a problem for semantics, but also because
it can cause long compile times, cf. the 45 MB macro expansion example
in Linux...

Martin

Re: Promote -Wno-sizeof-array-argument to an error

[Martin Uecker via Gcc]

Am Donnerstag, dem 08.08.2024 um 02:36 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> Can we promote -Wno-sizeof-array-argument to a hard error?  I don't
> think there's any legitimate use sizeof() on such a parameter.

I am a bit worried that it might prevent people from adding size information
to arguments, by transforming later use of sizeof on such a pointer argument
into a hard error.

So I would not do this at this time, until we have fully evolved the
size checking and the benefits are clear.

> 
> It would be an incompatible extension to ISO C, which would make sure
> that there are no remaining uses of sizeof(array_param), which would
> itself allow in the future --if n2906 or something similar is accepted--
> repurposing sizeof() to return the size in bytes of the array (instead
> of the size of a pointer) without worrying about breaking existing code.

I agree with this goal, but this is a long term goal.  


Martin

> 

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Martin Uecker via Gcc]

Am Samstag, dem 27.07.2024 um 00:26 +0200 schrieb Alejandro Colomar:
> On Sat, Jul 27, 2024 at 12:03:20AM GMT, Martin Uecker wrote:
> > > Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> > > adding diagnostics, it would help.
> > 
> > Both GCC and Clang already have such diagnostics and/or run-time checks:
> > 
> > https://godbolt.org/z/MPnxqb9h7
> 
> Hi Martin,
> 
> I guess that's prior art enough to make this UB in ISO C.  Is there any
> paper for this already?  Does any of your paper cover that?  Should I
> prepare one?
> 

What do you mean by "this"?  Adding UB would likely see a lot
of opposition, even where this could enable run-time checks.  

N2906 would make 

int foo(char f[4]);
int foo(char f[5]);

a constraint violation (although having those types be incompatible
could also cause UB indirectly, this would not be its main effect).

So I think brining a new version of this paper forward would be
a possible next step, addressing the issues raised in the past.

Martin

Re: n3294 - The restrict function attribute as a replacement of the restrict qualifier

[Martin Uecker via Gcc]

Am Freitag, dem 26.07.2024 um 23:49 +0200 schrieb Alejandro Colomar via Gcc:
> On Fri, Jul 26, 2024 at 09:22:42PM GMT, Joseph Myers wrote:
> > On Fri, 26 Jul 2024, Alejandro Colomar via Gcc wrote:
> > 
> > > > See reflector message SC22WG14.18575, 17 Nov 2020 (the former convenor 
> > > > replying when I asked about just that paper).
> > > 
> > > Where can I find reflector messages?
> > 
> > https://www.open-std.org/jtc1/sc22/wg14/18575
> 
> Thanks!
> 
> > 
> > > And another one to propose that [n] means the same as [static n] except
> > > for the nonnull property of static.
> > 
> > I'm not convinced that introducing extra undefined behavior for things 
> > that have been valid since C89 (which would be the effect of such a change 
> > for any code that passes a smaller array) is a good idea - the general 
> > mood is to *reduce* undefined behavior.
> 
> While [n] has always _officially_ meant the same as [], it has never
> made any sense to write code like that.  Unofficially, it has always
> meant the obvious thing.
> 
> Maybe if GNU C compilers (GCC and Clang) add it first as an extension,
> adding diagnostics, it would help.

Both GCC and Clang already have such diagnostics and/or run-time checks:

https://godbolt.org/z/MPnxqb9h7

Martin


> 
> Does anyone know of any existing code that uses [n] for meaning anything
> other than "n elements are available to the function"?
> 
> Functions that specify [n] most likely (definitely?) already mean that
> n elements are accessed, and thus passing something different than n
> elements results in UB one way or another.  Having the compiler enforce
> that via diagnostics and UB is probably an improvement.
> 
> Cheers,
> Alex
> 
> > 
> > -- 
> > Joseph S. Myers
> > josmy...@redhat.com
> > 
> 

Re: IFNDR on UB? [was: Straw poll on shifts with out of range operands]

[Martin Uecker via Gcc]

Am Montag, dem 01.07.2024 um 15:19 +0200 schrieb Matthias Kretz:
> On Sunday, 30 June 2024 08:33:35 GMT+2 Martin Uecker wrote:
> > Am Sonntag, dem 30.06.2024 um 05:03 +0200 schrieb Matthias Kretz:
> > > On Saturday, 29 June 2024 16:20:55 GMT+2 Martin Uecker wrote:
> > > > Am Samstag, dem 29.06.2024 um 08:50 -0500 schrieb Matthias Kretz via 
> Gcc:
> > > > > 
,
..
> > 
> > > > But I am not sure how this is relevant here as this affects only
> > > > observable behavior and the only case where GCC does not seem to
> > > > already conform to this is volatile.
> > > 
> > > Now you lost me.
> > 
> > Consider the following example:
> > 
> > int f(int x)
> > {
> >  int r = 0;
> >  if (x < 10)
> >r = 1;
> >  if (x < 10)
> >__builtin_unreachable();
> >  return r;
> > }
> > 
> > But removing the store to 'r' here as GCC does:
> > 
> > https://godbolt.org/z/h7qqrGsbz
> > 
> > can simply be justified by the "as if" principle as
> > any other optimization, it does not need to rely on a weird
> > intepretation that the UB from __builin_unreachable() travels
> > back in time.
> 
> I don't know of anybody saying that "time-travel optimization" refers to 
> anything different than what you're showing here. 

The C++ standard allows also removing or changing previous
observable behavior.

"However, if any such execution contains an undefined operation,
this International Standard places no requirement on the implementation
executing that program with that input (not even with regard to
operations preceding the first undefined operation)."

This was discussed extensively in the past, e.g. when discussing
the specification for memset_explicit.


> The part that people find 
> scary is when this "as if" happens at a distance, like in
> https://godbolt.org/z/jP4x1c3E6

True, although without "true time travel" (i.e. excluding
time-travel affecting previous observable behavior), this is
limited in the damage it can do (e.g. can not undo changes
committed to a log or control inputs to an external machine)
and has a much simpler interpretation that does not need to
refer to metaphysical concepts: Instead of the UB  affecting 
previous behavior by travelling back in time, it just has
arbitrary behavior that might undo the store.

> 
> > > [...]
> > 
> > I think it is a good idea. The compiler can optionally treat UB as
> > a translation time error. We discussed similar ideas in the past
> > in WG14. But this will only work for very specific instances of UB
> > under certain conditions.
> 
> Yes. But it's an exact match of the "time-travel" cases. I.e. whenever const-
> prop determines "unreachable" the code could be ill-formed.

I am not sure it is so simple.  What about if all this is in
dead code?  I assume one needs a rule similar to "for a functions,
when it is not possible to invoke the function without reaching
an operation which has UB, then the program ill-formed." or
"if a function call expression always invokes UB when executed,
then...". I guess the later is would apply to the precondition
viplations you mentioned WG21 is considering.

> 
> 
> > > > Also IMHO this should be split up from
> > > > UBsan which has specific semantics and upstream dependencies
> > > > which are are not always ideal.  (But UBSan could share the
> > > > same infrastructure)
> > > 
> > > I'm not sure what you're thinking of here. UBsan detects UB at runtime
> > > whereas my '-fharden=1' proposal is about flagging UB as ill-formed on
> > > compile-time. So UBsan is a more verbose '-fharden=2' then?
> > 
> > Yes, I was talking about the -fharden=2 case. In principle UBSan
> > with traps instead of diagnostics would do this. In practice,
> > I think we need something which is not tied to UBSan.
> 
> Yes, basically a deployable variant of UBsan?
> 

Yes, there is something such as -fcf-protection
or -fvtable-verify

Martin

> 
> On Sunday, 30 June 2024 08:56:41 GMT+2 Martin Uecker wrote:
> > 0) nothing
> > 1) expands to __builtin_unreachable()
> > 2) expands to __builtin_trap()
> > 3) expands to a __builtin_warning (as suggested before
> > by Martin Sebor) that causes the backend to emit an error
> > in a very late pass when the __builtin_warning has not
> > been removed during optimization.
> 
> This __builtin_warning seems to be equivalent to my __error() function, using 
> a [[gnu::warning]] attribute instead of [[gnu::error]]. Which is certainly 
> another viable build/-fharden/whateverwecallit mode.
> 

Re: Apply function attributes (e.g., [[gnu::access()]]) to pointees too

[Martin Uecker via Gcc]

Am Donnerstag, dem 11.07.2024 um 11:35 +0200 schrieb Alejandro Colomar via Gcc:
> Hi,
> 
> I was wondering how we could extend attributes such as gnu::access() to
> apply it to pointees too.  Currently, there's no way to specify the
> access mode of a pointee.
> 
> Let's take for example strsep(3):
> 
> With current syntax, this is what we can specify:
> 
>   [[gnu::access(read_write, 1)]]
>   [[gnu::access(read_only, 2)]]
>   [[gnu::nonnull(1, 2)]]
>   [[gnu::null_terminated_string_arg(2)]]
>   char *
>   strsep(char **restrict sp, const char *delim);

The main problem from a user perspective is that
these are attributes on the function declaration
and not on the argument (type).

> 
> I was thinking that with floating numbers, one could specify the number
> of dereferences with a number after the decimal point.  It's a bit
> weird, since the floating point is interpreted as two separate integer
> numbers separated by a '.', but could work.  In this case:
> 
>   [[gnu::access(read_write, 1)]]
>   [[gnu::access(read_write, 1.1)]]
>   [[gnu::access(read_only, 2)]]
>   [[gnu::nonnull(1, 2)]]
>   [[gnu::null_terminated_string_arg(1.1)]]
>   [[gnu::null_terminated_string_arg(2)]]
>   char *
>   strsep(char **restrict sp, const char *delim);
> 
> Which would mark the pointer *sp as read_write and a string.  What do
> you think about it?

If the attributes could be applied to the type, then
one could attach them directly at an intermediate
pointer level, which would be more intuitive and
less fragile.


Martin

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker via Gcc]

Am Montag, dem 08.07.2024 um 22:17 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Mon, Jul 08, 2024 at 06:05:08PM GMT, Martin Uecker wrote:
> > Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> > > On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:
> > 
> > ...
> > > And then have it mean something strict, such as: The object pointed to
> > > by the pointer is not pointed to by any other pointer; period.
> > > 
> > > This definition is already what -Wrestrict seems to understand.
> > 
> > One of the main uses of restrict is scientific computing. In this
> > context such a definition of "restrict" would not work for many 
> > important use cases. But I agree that for warning purposes the
> > definition of "restrict" in ISO C is not helpful.
> 
> Do you have some examples of functions where this matters and is
> important?  I'm curious to see them.  Maybe we find some alternative.

In many numerical algorithms you want to operate on
different parts of the same array object.  E.g. for matrix
decompositions you want to take a row / column and add it 
to another. Other examples are algorithms that decompose
some input (.e.g. high and low band in a wavelet transform)
and store it into the same output array, etc.

Without new notation for strided array slicing, one
fundamentally needs the flexibility of restrict that
only guarantuees that actual accesses do not conflict.

But this then implies that one can not use restrict as a
contract specification on function prototypes, but has
to analyze the implementation of a function to see if
it is used correctly.  But I would not see it as a design 
problem of restrict. It was simply not the intended use 
case when originally designed. 


> 
> > > > Has the C standard clarified the meaning of 'restrict' since that
> > > > discussion?  Without that, I wasn't planning to touch 'restrict' in
> > > > GCC's -fanalyzer.
> > > 
> > > Meh; no they didn't.  
> > 
> > There were examples added in C23 and there are now several papers
> > being under discussion.
> 
> Hmm, yeah, the examples help with the formal definition.  I was thinking
> of the definition itself, which I still find quite confusing.  :-)

Indeed.

Martin

> 
> Have a lovely night!
> Alex
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker via Gcc]

Am Montag, dem 08.07.2024 um 17:01 +0200 schrieb Alejandro Colomar:
> On Mon, Jul 08, 2024 at 10:30:48AM GMT, David Malcolm wrote:

...
> And then have it mean something strict, such as: The object pointed to
> by the pointer is not pointed to by any other pointer; period.
> 
> This definition is already what -Wrestrict seems to understand.

One of the main uses of restrict is scientific computing. In this
context such a definition of "restrict" would not work for many 
important use cases. But I agree that for warning purposes the
definition of "restrict" in ISO C is not helpful.

> 
> > Later, I added a new -Wanalyzer-overlapping-buffers warning in GCC 14,
> > which simply has a hardcoded set of standard library functions that it
> > "knows" to warn about.
> 
> Hmmm, so it doesn't help at all for anything other than libc.  Ok.
> 
> > Has the C standard clarified the meaning of 'restrict' since that
> > discussion?  Without that, I wasn't planning to touch 'restrict' in
> > GCC's -fanalyzer.
> 
> Meh; no they didn't.  

There were examples added in C23 and there are now several papers
being under discussion.


> I understand.  That's why I don't like innovations
> in ISO C, and prefer that implementations innovate with real stuff.

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker via Gcc]

Am Sonntag, dem 07.07.2024 um 13:07 +0200 schrieb Alejandro Colomar via Gcc:
> Hi Martin,
> 
> On Sun, Jul 07, 2024 at 09:15:23AM GMT, Martin Uecker wrote:
> > 
> > Hi Alejandro,
> > 
> > if in caller it is known that endptr has access mode "write_only"
> > then it can conclude that the content of *endptr has access mode
> > "none", couldn't it?
> 
> H.  I think you're correct.  I'll incorporate that and see how it
> affects the caller.
> 
> At first glance, I think it would result in
> 
>   nptraccess(read_only)   alias *endptr
>   endptr  access(write_only)  unique
>   errno   access(read_write)  unique
>   *endptr access(none)alias nptr
> 
> Which is actually having perfect information, regardless of 'restrict'
> on nptr.  :-)

Yes, but my point is that even with "restrict" a smarter
compiler could then also be smart enough not to warn even
when *endptr aliases nptr.

> 
> > You also need to discuss backwards compatibility.  Changing
> > the type of those functions can break valid programs.
> 
> I might be forgetting about other possibilities, but the only one I had
> in mind that could break API would be function pointers.  However, a
> small experiment seems to say it doesn't:

Right, the outermost qualifiers are ignored, so this is not a
compatibility problem.  So I think this is not an issue, but
it is worth pointing it out.

Martin

> 
>   $ cat strtolp.c 
>   #include 
> 
>   long
>   alx_strtol(const char *nptr, char **restrict endptr, int base)
>   {
>   return strtol(nptr, endptr, base);
>   }
> 
>   typedef long (*strtolp_t)(const char *restrict nptr,
> char **restrict endptr, int base);
>   typedef long (*strtolpnr_t)(const char *nptr,
>  char **restrict endptr, int base);
> 
>   int
>   main(void)
>   {
>   [[maybe_unused]] strtolp_ta = &strtol;
>   [[maybe_unused]] strtolpnr_t  b = &strtol;
>   [[maybe_unused]] strtolp_tc = &alx_strtol;
>   [[maybe_unused]] strtolpnr_t  d = &alx_strtol;
>   }
> 
>   $ cc -Wall -Wextra strtolp.c 
>   $
> 
> Anyway, I'll say that it doesn't seem to break API.
> 
> >  You would
> > need to make a case that this is unlikely to affect any real
> > world program.
> 
> If you have something else in mind that could break API, please let me
> know, and I'll add it to the experiments.
> 
> Thanks!
> 
> Have a lovely day!
> Alex
> 

Re: WG14 paper for removing restrict from nptr in strtol(3)

[Martin Uecker via Gcc]

Hi Alejandro,

if in caller it is known that endptr has access mode "write_only"
then it can conclude that the content of *endptr has access mode
"none", couldn't it?

You also need to discuss backwards compatibility.  Changing
the type of those functions can break valid programs.  You would
need to make a case that this is unlikely to affect any real
world program.

Martin

Am Sonntag, dem 07.07.2024 um 03:58 +0200 schrieb Alejandro Colomar:
> Hi,
> 
> I've incorporated feedback, and here's a new revision, let's call it
> v0.2, of the draft for a WG14 paper.  I've attached the man(7) source,
> and the generated PDF.
> 
> Cheers,
> Alex
> 
> 

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker via Gcc]

Am Freitag, dem 05.07.2024 um 21:28 +0200 schrieb Alejandro Colomar:

...
> 
> > > Showing that you can contrive a case where a const char* restrict and
> > > char** restrict can alias doesn't mean there's a problem with strtol.
> > 
> > I think his point is that a const char* restrict and something which
> > is stored in a char* whose address is then passed can alias and there
> > a warning would make sense in other situations.   
> 
> Indeed.
> 
> > But I am also not convinced removing restrict would be an improvement.
> > It would make more sense to have an annotation that indicates that
> > endptr is only used as output.
> 
> What is the benefit of keeping restrict there?  It doesn't provide any
> benefits, AFAICS.

Not really I think.  I am generally not a fan of restrict.
IMHO it is misdesigned and I would like to see it replaced
with something better.  But I also not convinced it really
helps to remove it here.

If we marked endptr as "write_only" (which it might already
be) then a future warning mechanism for -Wrestrict could
ignore the content of *endptr. 

Martin

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker via Gcc]

Am Freitag, dem 05.07.2024 um 17:24 +0100 schrieb Jonathan Wakely:
> On Fri, 5 Jul 2024 at 17:02, Xi Ruoyao via Gcc  wrote:
> > 
> > On Fri, 2024-07-05 at 17:53 +0200, Alejandro Colomar wrote:
> > > At least, I hope there's consensus that while current GCC doesn't warn
> > > about this, ideally it should, which means it should warn for valid uses
> > > of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> > > POSIX, and glibc.
> > 
> > It **shouldn't**.  strtol will only violate restrict if it's wrongly
> > implemented, or something dumb is done like "strtol((const char*) &p,
> > &p, 0)".
> > 
> > See my previous reply.
> 
> Right, is there a valid use of strtol where a warning would be justified?
> 
> Showing that you can contrive a case where a const char* restrict and
> char** restrict can alias doesn't mean there's a problem with strtol.

I think his point is that a const char* restrict and something which
is stored in a char* whose address is then passed can alias and there
a warning would make sense in other situations.   

But I am also not convinced removing restrict would be an improvement.
It would make more sense to have an annotation that indicates that
endptr is only used as output.

Martin  

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker via Gcc]

Am Freitag, dem 05.07.2024 um 17:53 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Fri, Jul 05, 2024 at 05:34:55PM GMT, Martin Uecker wrote:
> > > I've written functions that more closely resemble strtol(3), to show
> > > that in the end they all share the same issue regarding const-ness:
> 
> (Above I meant s/const/restrict/)
> 
> > > 
> > >   $ cat d.c 
> > >   int d(const char *restrict ca, char *restrict a)
> > >   {
> > >   return ca > a;
> > >   }
> > > 
> > >   int main(void)
> > >   {
> > >   char x = 3;
> > >   char *xp = &x;
> > >   d(xp, xp);
> > >   }
> > >   $ cc -Wall -Wextra d.c 
> > >   d.c: In function ‘main’:
> > >   d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter 
> > > aliases with argument 1 [-Wrestrict]
> > >  10 | d(xp, xp);
> > > | ^
> > > 
> > > This trivial program causes a diagnostic.  (Although I think the '>'
> > > should also cause a diagnostic!!)
> > > 
> > > Let's add a reference, to resemble strtol(3):
> > > 
> > >   $ cat d2.c 
> > >   int d2(const char *restrict ca, char *restrict *restrict ap)
> > >   {
> > >   return ca > *ap;
> > >   }
> > > 
> > >   int main(void)
> > >   {
> > >   char x = 3;
> > >   char *xp = &x;
> > >   d2(xp, &xp);
> > >   }
> > >   $ cc -Wall -Wextra d2.c 
> > >   $ 
> > > 
> > > Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
> > > are these programs any different regarding pointer restrict-ness?
> > 
> > It would require data flow anaylsis to produce the diagnostic while
> > the first can simply be diagnosed by comparing arguments.
> 
> Agree.  It seems like a task for -fanalyzer.
> 
>   $ cc -Wall -Wextra -fanalyzer -fuse-linker-plugin -flto d2.c
>   $
> 
> I'm unable to trigger that at all.  It's probably not implemented, I
> guess.  I've updated the bug report
>  to change the
> component to 'analyzer'.
> 
> At least, I hope there's consensus that while current GCC doesn't warn
> about this, ideally it should, which means it should warn for valid uses
> of strtol(3), which means strtol(3) should be fixed, in all of ISO,
> POSIX, and glibc.

I am not sure. 

> 
> > > > > Well, I don't know how to report that defect to WG14.  If you help me,
> > > > > I'll be pleased to do so.  Do they have a public mailing list or
> > > > > anything like that?
> > > > 
> > > > One can submit clarification or change requests:
> > > > 
> > > > https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html
> 
> P.S.:
> 
> I've sent a mail to UNE (the Spanish National Body for ISO), and
> asked them about joining WG14.  Let's see what they say.
> 
> P.S. 2:
> 
> I'm also preparing a paper; would you mind championing it if I'm not yet
> able to do it when it's ready?

Guests can present too.

> 
> P.S. 3:
> 
> Do you know of any Spanish member of WG14?  Maybe I can talk with them
> to have more information about how they work.

You could ask Miguel Ojeda.

Martin

> 

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker via Gcc]

Am Freitag, dem 05.07.2024 um 17:23 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On Fri, Jul 05, 2024 at 05:02:15PM GMT, Martin Uecker wrote:
> > > But when the thing gets non-trivial, as in strtol(3), GCC misses the
> > > -Wrestrict diagnostic, as reported in
> > > .
> > > 
> > > Let's write a reproducer by altering the dumb.c program from above, with
> > > just another reference:
> > > 
> > >   int
> > >   dumb2(int *restrict a, int *restrict *restrict ap)
> > >   {
> > >   // We don't access the objects
> > >   return a == *ap;
> > >   }
> > > 
> > >   int
> > >   main(void)
> > >   {
> > >   int x = 3;
> > >   int *xp = &x;
> > > 
> > >   return dumb2(&x, &xp);
> > >   }
> > > 
> > > GCC doesn't report anything bad here, even though it's basically the
> > > same as the program from above:
> > > 
> > >   $ cc -Wall -Wextra dumb2.c
> > >   $
> > 
> > strtol does have  a "char * restrict * restrict" though, so the
> > situation is different.   A "char **" and a "const char *"
> > shouldn't alias anyway. 
> 
> Pedantically, it is actually declared as 'char **restrict' (the inner
> one is not declared as restrict, even though it will be restricted,
> since there are no other unrestricted pointers).
> 
> I've written functions that more closely resemble strtol(3), to show
> that in the end they all share the same issue regarding const-ness:
> 
>   $ cat d.c 
>   int d(const char *restrict ca, char *restrict a)
>   {
>   return ca > a;
>   }
> 
>   int main(void)
>   {
>   char x = 3;
>   char *xp = &x;
>   d(xp, xp);
>   }
>   $ cc -Wall -Wextra d.c 
>   d.c: In function ‘main’:
>   d.c:10:9: warning: passing argument 2 to ‘restrict’-qualified parameter 
> aliases with argument 1 [-Wrestrict]
>  10 | d(xp, xp);
> | ^
> 
> This trivial program causes a diagnostic.  (Although I think the '>'
> should also cause a diagnostic!!)
> 
> Let's add a reference, to resemble strtol(3):
> 
>   $ cat d2.c 
>   int d2(const char *restrict ca, char *restrict *restrict ap)
>   {
>   return ca > *ap;
>   }
> 
>   int main(void)
>   {
>   char x = 3;
>   char *xp = &x;
>   d2(xp, &xp);
>   }
>   $ cc -Wall -Wextra d2.c 
>   $ 
> 
> Why does this not cause a -Wrestrict diagnostic, while d.c does?  How
> are these programs any different regarding pointer restrict-ness?

It would require data flow anaylsis to produce the diagnostic while
the first can simply be diagnosed by comparing arguments.

Martin

> 
> > > Well, I don't know how to report that defect to WG14.  If you help me,
> > > I'll be pleased to do so.  Do they have a public mailing list or
> > > anything like that?
> > 
> > One can submit clarification or change requests:
> > 
> > https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html
> 
> Thanks!  Will do.  Anyway, I think this should be discussed in glibc/gcc
> in parallel, since it's clearly a missed diagnostic, and possibly a
> dangerous use of restrict if the compiler does any assumptions that
> shouldn't be done.
> 
> Have a lovely day!
> Alex
> 

Re: [PATCH v1] Remove 'restrict' from 'nptr' in strtol(3)-like functions

[Martin Uecker via Gcc]

Am Freitag, dem 05.07.2024 um 16:37 +0200 schrieb Alejandro Colomar via Gcc:
> [CC += linux-man@, since we're discussing an API documented there, and
>  the manual page would also need to be updated]
> 
> Hi Xi,  Jakub,
> 
> On Fri, Jul 05, 2024 at 09:38:21PM GMT, Xi Ruoyao wrote:
> > On Fri, 2024-07-05 at 15:03 +0200, Alejandro Colomar wrote:
> > > ISO C specifies these APIs as accepting a restricted pointer in their
> > > first parameter:
> > > 
> > > $ stdc c99 strtol
> > > long int strtol(const char *restrict nptr, char **restrict endptr, int 
> > > base);
> > > $ stdc c11 strtol
> > > long int strtol(const char *restrict nptr, char **restrict endptr, int 
> > > base);
> > > 
> > > However, it should be considered a defect in ISO C.  It's common to see
> > > code that aliases it:
> > > 
> > >   char str[] = "10 20";
> > > 
> > >   p = str;
> > >   a = strtol(p, &p, 0);  // Let's ignore error handling for
> > >   b = strtol(p, &p, 0);  // simplicity.
> > 
> > Why this is wrong?
> > 
> > During the execution of strtol() the only expression accessing the
> > object "p" is *endptr.  When the body of strtol() refers "nptr" it
> > accesses a different object, not "p".
> 
> 
> 
> Theoretically, 'restrict' is defined in terms of accesses, not just
> references, so it's fine for strtol(3) to hold two references of p in
> restrict pointers.  That is, the following code is valid:
> 
>   int
>   dumb(int *restrict a, int *restrict also_a)
>   {
>   // We don't access the objects
>   return a == also_a;
>   }
> 
>   int
>   main(void)
>   {
>   int x = 3;
> 
>   return dumb(&x, &x);
>   }
> 
> However, in practice that's dumb.  The caller cannot know that the
> function doesn't access the object, so it must be cautious and enable
> -Wrestrict, which should be paranoid and do not allow passing references
> to the same object in different arguments, just in case the function
> decides to access to objects.  Of course, GCC reports a diagnostic for
> the previous code:
> 
>   $ cc -Wall -Wextra dumb.c 
>   dumb.c: In function ‘main’:
>   dumb.c:13:21: warning: passing argument 1 to ‘restrict’-qualified 
> parameter aliases with argument 2 [-Wrestrict]
>  13 | return dumb(&x, &x);
> | ^~  ~~
> 
> ... even when there's no UB, since the object is not being accessed.
> 
> But when the thing gets non-trivial, as in strtol(3), GCC misses the
> -Wrestrict diagnostic, as reported in
> .
> 
> Let's write a reproducer by altering the dumb.c program from above, with
> just another reference:
> 
>   int
>   dumb2(int *restrict a, int *restrict *restrict ap)
>   {
>   // We don't access the objects
>   return a == *ap;
>   }
> 
>   int
>   main(void)
>   {
>   int x = 3;
>   int *xp = &x;
> 
>   return dumb2(&x, &xp);
>   }
> 
> GCC doesn't report anything bad here, even though it's basically the
> same as the program from above:
> 
>   $ cc -Wall -Wextra dumb2.c
>   $

strtol does have  a "char * restrict * restrict" though, so the
situation is different.   A "char **" and a "const char *"
shouldn't alias anyway. 


> 
> Again, there's no UB, but we really want to be cautious and get a
> diagnostic as callers, just in case the callee decides to access the
> object; we never know.
> 
> So, GCC should be patched to report a warning in the program above.
> That will also cause strtol(3) to start issuing warnings in use cases
> like the one I showed.
> 
> Even further, let's try something really weird: inequality comparison,
> which is only defined for pointers to the same array object:
> 
>   int
>   dumb3(int *restrict a, int *restrict *restrict ap)
>   {
>   // We don't access the objects
>   return a > *ap;
>   }
> 
>   int
>   main(void)
>   {
>   int x = 3;
>   int *xp = &x;
> 
>   return dumb3(&x, &xp);
>   }
> 
> The behavior is still defined, since the obnjects are not accessed, but
> the compiler should really warn, on both sides:
> 
> -  The caller is passing references to the same object in restricted
>parameters, which is a red flag.
> 
> -  The callee is comparing for inequality pointers that should, under
>normal circumstances, cause Undefined Behavior.
> 
> 
> > And if this is really wrong you should report it to WG14 before changing
> > glibc.
> 
> Well, I don't know how to report that defect to WG14.  If you help me,
> I'll be pleased to do so.  Do they have a public mailing list or
> anything like that?

One can submit clarification or change requests:

https://www.open-std.org/jtc1/sc22/wg14/www/contributing.html

Martin

Re: IFNDR on UB? [was: Straw poll on shifts with out of range operands]

[Martin Uecker via Gcc]

Actually, it is very much aligned with what I want in C.
In general I want to have pragma-based compilation modes
for memory safety:

https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3211.pdf

(Bjarne Stroustrup has a proposal for profiles in C++ which
goes in similar direction I think)

>From an implementation point of view, if we annotated all
operations with UB in the front ends with a new

__builtin_undefined()

that - depending on configuration and/or mode - does:

0) nothing
1) expands to __builtin_unreachable()
2) expands to __builtin_trap()  
3) expands to a __builtin_warning (as suggested before
by Martin Sebor) that causes the backend to emit an error
in a very late pass when the __builtin_warning has not
been removed during optimization.

Then this would solve all my problems related to UB.

Martin

Am Sonntag, dem 30.06.2024 um 08:33 +0200 schrieb Martin Uecker via Gcc:
> Am Sonntag, dem 30.06.2024 um 05:03 +0200 schrieb Matthias Kretz:
> > On Saturday, 29 June 2024 16:20:55 GMT+2 Martin Uecker wrote:
> > > Am Samstag, dem 29.06.2024 um 08:50 -0500 schrieb Matthias Kretz via Gcc:
> > > > I.e. once UB becomes IFNDR, the dreaded time-travel optimizations can't
> > > > happen anymore. Instead precondition checks bubble up because otherwise
> > > > the program is ill-formed.
> > > 
> > > It is not clear to mean what you mean by this?
> > 
> > It would help if you could point out what is unclear to you. I assume you 
> > know 
> > IFNDR? And I gave an example for the "bubbling up" of precondition checks 
> > directly above your quoted paragraph.
> 
> I think I understood it now:  You want to make UB be IFNDR so
> that the compiler is allowed to diagnose it at translation
> time in certain cases (although this would not generally be
> required for IFNDR).
> 
> > 
> > > Note that in C time-travel optimizations are already not allowed.
> > 
> > Then, calling __builtin_unreachable is non-conforming for C? ... at least 
> > in 
> > the English sense of "this code is impossible to reach", which implies that 
> > the condition leading up to it must be 'false', allowing time-travel 
> > optimization. Or how would C define 'unreachable'?
> 
> __builtin_uneachable is an extension, it can do whatever it wants.
> 
> But note that compilers do not seem to eliminate the control flow path
> leading to it:
> 
> 
> https://godbolt.org/z/coq9Yra1j
> 
> So even if it is defined in terms of C's UB, these implementations
> would still be conforming to C.
> 
> > 
> > > But I am not sure how this is relevant here as this affects only
> > > observable behavior and the only case where GCC does not seem to
> > > already conform to this is volatile.
> > 
> > Now you lost me.
> 
> Consider the following example:
> 
> int f(int x)
> {
>  int r = 0;
>  if (x < 10)
>r = 1;
>  if (x < 10)
>__builtin_unreachable();
>  return r;
> }
> 
> But removing the store to 'r' here as GCC does:
> 
> https://godbolt.org/z/h7qqrGsbz
> 
> can simply be justified by the "as if" principle as
> any other optimization, it does not need to rely on a weird
> intepretation that the UB from __builin_unreachable() travels
> back in time.
> 
> > 
> > > Of course, C++ may be different but I suspect that some of the
> > > discussion is confusing compiler bugs with time-travel:
> > 
> > "some of the discussion" is referring to what?
> 
> To discussions inside WG21 that seems to believe that it
> is important that compilers can do  time-travel optimizations,
> when this is actually not the case.
> 
> > 
> > > > Again, I don't believe this would be conforming to the C++ standard. 
> > > > But I
> > > > believe it's a very interesting mode to add as a compiler flag.
> > > > 
> > > > -fharden=0 (default)
> > > > -fharden=1 (make UB ill-formed or unreachable)
> > > > -fharden=2 (make UB ill-formed or trap)
> > > > 
> > > > If there's interest I'd be willing to look into a patch to libstdc++,
> > > > building upon the above sketch as a starting point. Ultimately, if this
> > > > becomes a viable build mode, I'd like to have a replacement for the
> > > > [[gnu::error("")]] hack with a dedicated builtin.
> > > 
> > > -fharden should never turn this into unreachable.
> > 
> > Well, if the default is 'unreachable' and the next step is 'ill-formed or 
> > unreachable' it

Re: IFNDR on UB? [was: Straw poll on shifts with out of range operands]

[Martin Uecker via Gcc]

Am Sonntag, dem 30.06.2024 um 05:03 +0200 schrieb Matthias Kretz:
> On Saturday, 29 June 2024 16:20:55 GMT+2 Martin Uecker wrote:
> > Am Samstag, dem 29.06.2024 um 08:50 -0500 schrieb Matthias Kretz via Gcc:
> > > I.e. once UB becomes IFNDR, the dreaded time-travel optimizations can't
> > > happen anymore. Instead precondition checks bubble up because otherwise
> > > the program is ill-formed.
> > 
> > It is not clear to mean what you mean by this?
> 
> It would help if you could point out what is unclear to you. I assume you 
> know 
> IFNDR? And I gave an example for the "bubbling up" of precondition checks 
> directly above your quoted paragraph.

I think I understood it now:  You want to make UB be IFNDR so
that the compiler is allowed to diagnose it at translation
time in certain cases (although this would not generally be
required for IFNDR).

> 
> > Note that in C time-travel optimizations are already not allowed.
> 
> Then, calling __builtin_unreachable is non-conforming for C? ... at least in 
> the English sense of "this code is impossible to reach", which implies that 
> the condition leading up to it must be 'false', allowing time-travel 
> optimization. Or how would C define 'unreachable'?

__builtin_uneachable is an extension, it can do whatever it wants.

But note that compilers do not seem to eliminate the control flow path
leading to it:


https://godbolt.org/z/coq9Yra1j

So even if it is defined in terms of C's UB, these implementations
would still be conforming to C.

> 
> > But I am not sure how this is relevant here as this affects only
> > observable behavior and the only case where GCC does not seem to
> > already conform to this is volatile.
> 
> Now you lost me.

Consider the following example:

int f(int x)
{
 int r = 0;
 if (x < 10)
   r = 1;
 if (x < 10)
   __builtin_unreachable();
 return r;
}

But removing the store to 'r' here as GCC does:

https://godbolt.org/z/h7qqrGsbz

can simply be justified by the "as if" principle as
any other optimization, it does not need to rely on a weird
intepretation that the UB from __builin_unreachable() travels
back in time.

> 
> > Of course, C++ may be different but I suspect that some of the
> > discussion is confusing compiler bugs with time-travel:
> 
> "some of the discussion" is referring to what?

To discussions inside WG21 that seems to believe that it
is important that compilers can do  time-travel optimizations,
when this is actually not the case.

> 
> > > Again, I don't believe this would be conforming to the C++ standard. But I
> > > believe it's a very interesting mode to add as a compiler flag.
> > > 
> > > -fharden=0 (default)
> > > -fharden=1 (make UB ill-formed or unreachable)
> > > -fharden=2 (make UB ill-formed or trap)
> > > 
> > > If there's interest I'd be willing to look into a patch to libstdc++,
> > > building upon the above sketch as a starting point. Ultimately, if this
> > > becomes a viable build mode, I'd like to have a replacement for the
> > > [[gnu::error("")]] hack with a dedicated builtin.
> > 
> > -fharden should never turn this into unreachable.
> 
> Well, if the default is 'unreachable' and the next step is 'ill-formed or 
> unreachable' it's a step up. But I'm all for a better name.

I think it is a good idea. The compiler can optionally treat UB as
a translation time error. We discussed similar ideas in the past
in WG14. But this will only work for very specific instances of UB
under certain conditions.

> 
> > IMHO the FEs should insert the conditional traps when requested to
> > and the middle end could then treat it as UB and more freely
> > decide what to do.
> 
> Right I was thinking of turning my library-solution hack into a builtin (if 
> it 
> shows potential). The behavior of which then depends on a compiler flag. Then 
> both library and language UB could invoke that builtin. E.g. 'operator+(int, 
> int)' would add '__check_precondition(not __builtin_add_overflow_p(a, b, a));'
> With my proposed '-fharden=1 -O2' you'd then get a compilation error on 
> '0x7fff' + 1', but no code size increase for all other additions. With 
> '-fharden=2 -O2' the 'lea' would turn into an actual 'add' instruction with 
> subsequent 'jo' to 'ud2' (on x86).

Yes, I fully agree with this.  
> 
> > Also IMHO this should be split up from
> > UBsan which has specific semantics and upstream dependencies
> > which are are not always ideal.  (But UBSan could share the
> > same infrastructure)
> 
> I'm not sure what you're thinking of here. UBsan detects UB at runtime 
> whereas 
> my '-fharden=1' proposal is about flagging UB as ill-formed on compile-time. 
> So UBsan is a more verbose '-fharden=2' then?

Yes, I was talking about the -fharden=2 case. In principle UBSan
with traps instead of diagnostics would do this. In practice,
I think we need something which is not tied to UBSan.

Martin


> 
> - Matthias
> 

Re: IFNDR on UB? [was: Straw poll on shifts with out of range operands]

[Martin Uecker via Gcc]

Am Samstag, dem 29.06.2024 um 08:50 -0500 schrieb Matthias Kretz via Gcc:


...
> I.e. once UB becomes IFNDR, the dreaded time-travel optimizations can't 
> happen 
> anymore. Instead precondition checks bubble up because otherwise the program 
> is ill-formed.

It is not clear to mean what you mean by this?

Note that in C time-travel optimizations are already not allowed.
But I am not sure how this is relevant here as this affects only
observable behavior and the only case where GCC does not seem to
already conform to this is volatile.

Of course, C++ may be different but I suspect that some of the
discussion is confusing compiler bugs with time-travel:

https://developercommunity.visualstudio.com/t/Invalid-optimization-in-CC/10337428?q=muecker



> 
> Again, I don't believe this would be conforming to the C++ standard. But I 
> believe it's a very interesting mode to add as a compiler flag.
> 
> -fharden=0 (default)
> -fharden=1 (make UB ill-formed or unreachable)
> -fharden=2 (make UB ill-formed or trap)
> 
> If there's interest I'd be willing to look into a patch to libstdc++, 
> building 
> upon the above sketch as a starting point. Ultimately, if this becomes a 
> viable build mode, I'd like to have a replacement for the [[gnu::error("")]] 
> hack with a dedicated builtin.

-fharden should never turn this into unreachable.

But I agree that we should have options for different choices. 

IMHO the FEs should insert the conditional traps when requested to
and the middle end could then treat it as UB and more freely
decide what to do.  Also IMHO this should be split up from
UBsan which has specific semantics and upstream dependencies
which are are not always ideal.  (But UBSan could share the
same infrastructure)

Martin

Re: consistent unspecified pointer comparison

[Martin Uecker via Gcc]

Am Donnerstag, dem 27.06.2024 um 12:05 -0700 schrieb Andrew Pinski via Gcc:
> On Thu, Jun 27, 2024 at 11:57 AM Jason Merrill via Gcc  
> wrote:
> > 
> > On Thu, Jun 27, 2024 at 2:38 PM Richard Biener
> >  wrote:
> > > > Am 27.06.2024 um 19:04 schrieb Jason Merrill via Gcc :
> > > > 
> > > > https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2434r1.html
> > > > proposes to require that repeated unspecified comparisons be
> > > > self-consistent, which does not match current behavior in either GCC
> > > > or Clang.  The argument is that the current allowance to be
> > > > inconsistent is user-unfriendly and does not enable significant
> > > > optimizations.  Any feedback about this?
> > > 
> > > Can you give an example of an unspecified comparison?  I think the only 
> > > way to do what the paper wants is for the implementation to make the 
> > > comparison specified (without the need to document it).  Is the 
> > > self-consistency required only within some specified scope (a single 
> > > expression?) or even across TUs (which might be compiled by different 
> > > compilers or compiler versions)?
> > > 
> > > So my feedback would be to make the comparison well-defined.
> > > 
> > > I’m still curious about which ones are unspecified now.
> > 
> > https://eel.is/c++draft/expr#eq-3.1
> > "If one pointer represents the address of a complete object, and
> > another pointer represents the address one past the last element of a
> > different complete object, the result of the comparison is
> > unspecified."
> > 
> > This is historically unspecified primarily because we don't want to
> > force a particular layout of multiple variables.
> > 
> > See the example under "consequences for implementations" in the paper.
> 
> There is instability due to floating point too;
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93681
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93806
> 
> and uninitialized variables:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93301
> (but that might be fixed via https://wg21.link/P2795R5).

For pointer comparison:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61502


Martin

> 
> > 
> > Jason
> > 

Re: consistent unspecified pointer comparison

[Martin Uecker via Gcc]

Am Donnerstag, dem 27.06.2024 um 13:02 -0400 schrieb Jason Merrill via Gcc:
> https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2434r1.html
> proposes to require that repeated unspecified comparisons be
> self-consistent, which does not match current behavior in either GCC
> or Clang.  The argument is that the current allowance to be
> inconsistent is user-unfriendly and does not enable significant
> optimizations.  Any feedback about this?

Making pointer comparison self-consistent in cases where there is
no UB makes a lot of sense, because everything else is dangerous.

This is not the only thing this paper proposes. I do not think
angelic / demonic nondeterminism is good language design, and
especially think it is not a good fit for systems languages
such as C or C++. Also most programmers will not understand it.

Martin

Re: Union initialization semantics

[Martin Uecker via Gcc]

Am Mittwoch, dem 19.06.2024 um 13:59 +0100 schrieb Jonathan Wakely via Gcc:
> On Wed, 19 Jun 2024 at 11:57, Alexander Monakov  wrote:
> > 
> > Hello,
> > 
> > I vaguely remember there was a recent, maybe within last two months, 
> > discussion
> > about semantics of union initialization where sizeof(first member) is less 
> > than
> > sizeof(union). The question was whether it's okay to initialize just that 
> > first
> > member and leave garbage bits in the other, larger, members of the union, 
> > like
> > in this example:
> > 
> > union A {
> > char a;
> > long : 0;
> > };
> > 
> > void fn(void *);
> > 
> > void my(void)
> > {
> > union A a = { 0 };
> > fn(&a);
> > }
> > 
> > (except in my example there's no other named member, but I think the example
> > in that discussion was less contrived)
> > 
> > Perhaps somebody remembers where it was (I'm thinking Bugzilla) and could 
> > point
> > me to it? My attempts to search for it aren't turning anything up so far.
> 
> Somebody asked about this internally at Red Hat recently, and I
> responded with this quote from C17 6.2.6.1 p7:
> "When a value is stored in a member of an object of union type, the
> bytes of the object representation that do not correspond to that
> member but do correspond to other members take unspecified values. "
> 
> This looks related too:
> https://discourse.llvm.org/t/union-initialization-and-aliasing-clang-18-seems-to-miscompile-musl/77724/3
> They don't seem to have found the quote above though.
> 
> I think it got reported to GCC's bugzilla too, I'll see if I can find it 
> again.
> 
> > If someone knows what semantics GCC implements, that also would be welcome.
> 
> GCC seems to initialize the trailing bits, unnecessarily.

Note that C23 will require the padding bits to be initialized with zero
for default initialization {}.

Martin

Re: check_qualified_type

[Martin Uecker via Gcc]

Am Montag, dem 17.06.2024 um 15:40 +0200 schrieb Jakub Jelinek:
> On Mon, Jun 17, 2024 at 03:33:05PM +0200, Martin Uecker wrote:
> > > I've done that and that was because build_qualified_type uses that
> > > predicate, where qualified types created by build_qualified_type have
> > > as TYPE_CANONICAL the qualified type of the main variant of the canonical
> > > type, while in all other cases TYPE_CANONICAL is just the main variant of
> > > the type.
> > > Guess we could also just do
> > >   if (TYPE_QUALS (x) == TYPE_QUALS (t))
> > > TYPE_CANONICAL (x) = TYPE_CANONICAL (t);
> > >   else if (TYPE_CANONICAL (t) != t
> > >  || TYPE_QUALS (x) != TYPE_QUALS (TYPE_CANONICAL (t)))
> > > TYPE_CANONICAL (x)
> > >   = build_qualified_type (TYPE_CANONICAL (t), TYPE_QUALS (x));
> > >   else
> > > TYPE_CANONICAL (x) = x;
> > > 
> > 
> > Ok, that works. I think the final "else" is then then impossible to reach
> > and can be eliminated as well, because if TYPE_CANONICAL (t) == t then 
> > TYPE_QUALS (x) == TYPE_QUALS (TYPE_CANONICAL (t)) is identical to
> > TYPE_QUALS (x) == TYPE_QUALS (t) which is the first case.
> 
> If c_update_type_canonical is only ever called for the main variants of the
> type and they always have !TYPE_QUALS (t), then yes.
> But if we rely on that, perhaps we should gcc_checking_assert that.
> So
>   gcc_checking_assert (t == TYPE_MAIN_VARIANT (t) && !TYPE_QUALS (t));
> or something similar at the start of the function.

It calls itself recursively on pointers to the type.  But to
me the third branch looks dead in any case, because the first
two cover all possibilities.

Martin

> Then we could also change the
>   for (tree x = TYPE_MAIN_VARIANT (t); x; x = TYPE_NEXT_VARIANT (x))
> to
>   for (tree x = t; x; x = TYPE_NEXT_VARIANT (x))
> and
> if (TYPE_QUALS (x) == TYPE_QUALS (t))
> ...
> to
> if (!TYPE_QUALS (x))
>   TYPE_CANONICAL (x) = TYPE_CANONICAL (t);
> else
>   build_qualified_type (TYPE_CANONICAL (t), TYPE_QUALS (x));
> 





> 

Re: check_qualified_type

[Martin Uecker via Gcc]

Am Montag, dem 17.06.2024 um 14:57 +0200 schrieb Jakub Jelinek:
> On Mon, Jun 17, 2024 at 02:42:05PM +0200, Richard Biener wrote:
> > > > > I am trying to understand what check_qualified_type
> > > > > does exactly. The direct comparison of TYPE_NAMES seems incorrect
> > > > > for C and its use is c_update_type_canonical then causes
> > > > > PR114930 and PR115502.  In the later function I think
> > > > > it is not really needed and I guess one could simply remove
> > > > > it, but I wonder if it works incorrectly in other cases 
> > > > > too?
> > > > 
> > > > TYPE_NAMES is compared because IIRC typedefs are recorded as variants
> > > > and 'const T' isn't the same as 'const int' with typedef int T.
> > > 
> > > so if it is intentional that it differentiates between 
> > > 
> > > struct foo
> > > 
> > > and
> > > 
> > > typedef struct foo bar
> > > 
> > > then I will change c_update_type_canonical to not use it,
> > > because both types should have the same TYPE_CANONICAL
> > 
> > The check is supposed to differentiate between variants and all variants
> > have the same TYPE_CANONICAL so I'm not sure why you considered using
> > this function for canonical type compute?
> 
> I've done that and that was because build_qualified_type uses that
> predicate, where qualified types created by build_qualified_type have
> as TYPE_CANONICAL the qualified type of the main variant of the canonical
> type, while in all other cases TYPE_CANONICAL is just the main variant of
> the type.
> Guess we could also just do
>   if (TYPE_QUALS (x) == TYPE_QUALS (t))
> TYPE_CANONICAL (x) = TYPE_CANONICAL (t);
>   else if (TYPE_CANONICAL (t) != t
>  || TYPE_QUALS (x) != TYPE_QUALS (TYPE_CANONICAL (t)))
> TYPE_CANONICAL (x)
>   = build_qualified_type (TYPE_CANONICAL (t), TYPE_QUALS (x));
>   else
> TYPE_CANONICAL (x) = x;
> 

Ok, that works. I think the final "else" is then then impossible to reach
and can be eliminated as well, because if TYPE_CANONICAL (t) == t then 
TYPE_QUALS (x) == TYPE_QUALS (TYPE_CANONICAL (t)) is identical to
TYPE_QUALS (x) == TYPE_QUALS (t) which is the first case.

Martin

Re: -Wcast-qual consistency with initialization conversion and double pointer types

[Martin Uecker via Gcc]

Am Montag, dem 17.06.2024 um 12:06 + schrieb Joseph Myers:
> On Sun, 16 Jun 2024, Martin Uecker via Gcc wrote:
> 
> > I think it should not warn about:
> > 
> > char *x;
> > *(char * volatile *)&x;
> > 
> > as this is regular qualifier adding and this is
> > a bug in GCC.
> > 
> > I would guess it looks at all qualifiers added at
> > all level but should ignore the one on the first level.
> 
> This is meant to be implementing, as an extension to C, the C++ rules 
> (where converting from char** to const char** is unsafe, but converting 
> from char** to const char*const* is safe).  So the first question is what 
> C++ thinks of this conversion.
> 
Note that this is about the case where no third-level qualifier
is added. We should still warn about converting from char** to
const char **, and to volatile char ** but probably not (I think)
when converting to char*const*, const char*const*, 
volatile char*const*, and also not when converting
to char*volatile*.  So not when all intermediate casts
are const but also not when only a qualifier is added
to the second level but not to deeper levels. 

Martin

Re: check_qualified_type

[Martin Uecker via Gcc]

Am Montag, dem 17.06.2024 um 08:01 +0200 schrieb Richard Biener via Gcc:
> On Sun, 16 Jun 2024, Martin Uecker wrote:
> 
> > 
> > 
> > I am trying to understand what check_qualified_type
> > does exactly. The direct comparison of TYPE_NAMES seems incorrect
> > for C and its use is c_update_type_canonical then causes
> > PR114930 and PR115502.  In the later function I think
> > it is not really needed and I guess one could simply remove
> > it, but I wonder if it works incorrectly in other cases 
> > too?
> 
> TYPE_NAMES is compared because IIRC typedefs are recorded as variants
> and 'const T' isn't the same as 'const int' with typedef int T.

so if it is intentional that it differentiates between 

struct foo

and

typedef struct foo bar

then I will change c_update_type_canonical to not use it,
because both types should have the same TYPE_CANONICAL

Martin

check_qualified_type

[Martin Uecker via Gcc]

I am trying to understand what check_qualified_type
does exactly. The direct comparison of TYPE_NAMES seems incorrect
for C and its use is c_update_type_canonical then causes
PR114930 and PR115502.  In the later function I think
it is not really needed and I guess one could simply remove
it, but I wonder if it works incorrectly in other cases 
too?


Martin

Re: -Wcast-qual consistency with initialization conversion and double pointer types

[Martin Uecker via Gcc]

I think it should not warn about:

char *x;
*(char * volatile *)&x;

as this is regular qualifier adding and this is
a bug in GCC.

I would guess it looks at all qualifiers added at
all level but should ignore the one on the first level.

Martin


Am Samstag, dem 15.06.2024 um 10:17 -0700 schrieb Ryan Libby via Gcc:
> I'm not a C language expert and I'm looking for advice on whether a
> -Wcast-qual diagnostic in one situation and not another is intentional
> behavior.
> 
> Here's a set of examples (same as attachment).
> 
> % cat cast-qual-example.c
> #define F(name, type, qual) \
> typedef type t_##name;  \
> void name(void) {   \
> t_##name x = 0, y, z;   \
> y = *(t_##name qual *)&x;   \
> z = *(t_##name qual *){&x}; \
> }
> 
> F(fcc, char, const)
> F(fpc, char *, const)
> F(fcv, char, volatile)
> F(fpv, char *, volatile)
> 
> void fpv2(void) {
> char *x = 0, *y, *z;
> y = *(char * volatile *)&x;
> z = *(char * volatile *){&x};
> }
> 
> void eg1(void) {
> /* Adapted from -Wcast-qual doc */
> char v0 = 'v';
> char *v1 = &v0;
> char **p = &v1;
> /* p is char ** value.  */
> char * volatile *q = (char * volatile *) p;
> /* Assignment of volatile pointer to char is OK. */
> char u0 = 'u';
> char * volatile u1 = &u0;
> *q = u1;
> /* Now *q is accessed through a non-volatile-qualified pointer. */
> *p = 0;
> }
> 
> void eg2(void) {
> char v = 'v';
> char *p = &v;
> /* p is char * value.  */
> char volatile *q = (char volatile *) p;
> /* Assignment of volatile char is OK (and also plain char). */
> char volatile u = 'u';
> *q = u;
> /* Now *q is accessed through a non-volatile-qualified pointer. */
> *p = 0;
> }
> 
> % gcc13 -std=c17 -Wall -Wextra -Wcast-qual -Wno-unused -c
> cast-qual-example.c -o /dev/null
> cast-qual-example.c: In function 'fpv':
> cast-qual-example.c:5:14: warning: to be safe all intermediate
> pointers in cast from 'char **' to 'char * volatile*' must be 'const'
> qualified [-Wcast-qual]
> 5 | y = *(t_##name qual *)&x;   \
>   |  ^
> cast-qual-example.c:12:1: note: in expansion of macro 'F'
>12 | F(fpv, char *, volatile)
>   | ^
> cast-qual-example.c: In function 'fpv2':
> cast-qual-example.c:16:14: warning: to be safe all intermediate
> pointers in cast from 'char **' to 'char * volatile*' must be 'const'
> qualified [-Wcast-qual]
>16 | y = *(char * volatile *)&x;
>   |  ^
> cast-qual-example.c: In function 'eg1':
> cast-qual-example.c:26:30: warning: to be safe all intermediate
> pointers in cast from 'char **' to 'char * volatile*' must be 'const'
> qualified [-Wcast-qual]
>26 | char * volatile *q = (char * volatile *) p;
>   |  ^
> % clang -std=c17 -Wall -Wextra -Wcast-qual -Wno-unused -c
> cast-qual-example.c -o /dev/null
> %
> 
> The macro and typedef are to illustrate the point, they aren't otherwise
> needed, and fpv2 shows the same thing without them.
> 
> So, in the conversion of char ** to char * volatile *, the cast before
> the assignment of y is diagnosed, but the conversion in the
> initialization of the compound literal for the assignment of z is not.
> 
> First, is the cast construct actually different from the initialization
> construct in terms of safety?  I would think not, but maybe I am
> missing something.
> 
> I think that both assignment expressions in fpv as a whole are
> ultimately safe, considering also the immediate dereference of the
> temporary outer pointer value.
> 
> In eg1 and eg2 I modified examples from the -Wcast-qual documentation.
> eg1 is diagnosed, eg2 is not.
> 
> I think that the *p assignment in eg1 might be undefined behavior
> (6.7.3, referring to an object with volatile-qualified type (*q) through
> an lvalue without volatile-qualified type (*p)).
> 
> But then I don't get why the same wouldn't be true if we take away the
> inner pointer and repeat the exercise with plain char (eg1 vs eg2).
> 
> So, what's going on here?  Is the gcc behavior intentional?  Is it
> consistent?  And is there a recommended way to construct a temporary
> volatile pointer to an object (which may itself be a pointer) without
> tripping -Wcast-qual, without just casting away type information (as in,
> without intermediate casts through void *, uintptr_t, etc), and
> preferably also without undefined behavior?
> 
> I have checked that the behavior is the same with current sources and
> -std=c23 (gcc (GCC) 15.0.0 20240614 (experimental)).
> 
> P.s. I have seen gcc bug 84166 that advises that the -Wcast-qual warning
> from the cast is intentional in that case.  I think this case is
> different because in that case the qualifiers are on the innermost type.
> 
> Thank you,
> 
> Ryan L

Re: strtol(3) with QChar arguments

[Martin Uecker via Gcc]

Am Sonntag, dem 05.05.2024 um 15:13 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> I was wondering why C23 didn't use QChar for strtol(3).  It has the same
> problems that string functions have: a const input string and a
> non-const output string (the endptr).

I am not sure whether strtol was discussed.

> 
> I think endptr should have the same constness of the string passed to
> strtol(3), no?
> 
> Should this be addressed for C3x?  For liba2i.git, I'm working on
> const-generic versions of strtol(3) wrappers, which have helped simplify
> the const/non-const mix of pointers in shadow.git.

One potential issue is that for strtol such a change would break
all callers that pass a const-qualified pointer as first argument
and provide an argument for enptr second, which now has to be
a pointer to a non-const pointer.

For the functions we changed this breaks only cases where
a const qualified pointer is passed and then the result
is assigned to a non-const pointer, which could already be
considered questionable in existing code.

Martin

> 
> Have a lovely day!
> Alex
> 

Re: Generated files in libgfortran for Fortran intrinsic procedures (was: Updated Sourceware infrastructure plans)

[Martin Uecker via Gcc]

Am Donnerstag, dem 18.04.2024 um 14:01 +0200 schrieb Tobias Burnus:
> Hi Janne,
> 
> Janne Blomqvist wrote:
> > back when I was active I did think about this
> > issue. IMHO the best of my ideas was to convert these into C++
> > templates.

I haven't looked at libgfortran but I didn't find it problematic
at all to use C in similar numerical code and this helps
with portability. 

Either I use macros, which I keep short and then do not find
inferior to templates (having used C++ for years previously) or 
- if there is really a lot of code that needs to be specialized 
for a type - simply by using includes:

#define matmul_type double
#include "matmul_impl.c"

Martin


> 
> I think this will work – but we have to be super careful:
> 
> With C++, there is the problem that we definitely do not want to add 
> dependency on libstdc++ nor to use some features which require special 
> hardware support (like exceptions [always bad], symbol aliases, ...). — 
> On some systems, a full C++ support might be not available, like 
> embedded systems (including some odd embedded OS) or offloading devices.
> 
> The libstdc++ dependency would be detected by linking as we currently 
> do. For in-language features, we have to ensure the appropriate flags 
> -fno-exceptions (and probably a few more). And it should be clear what 
> language features to use.
> 
> If we do, I think that would surely be an option.
> 
> > What we're essentially doing with the M4 stuff and the
> > proposed in-house Python reimplementation is to make up for lack of
> > monomorphization in plain old C. Rather than doing some DIY templates,
> > switch the implementation language to something which has that feature
> > built-in, in this case C++.  No need to convert the entire libgfortran
> > to C++ if you don't want to, just those objects that are generated
> > from the M4 templates. Something like
> > 
> > template
> > void matmul(T* a, T* b, T* c, ...)
> > {
> > // actual matmul code here
> > }
> > 
> > extern "C" {
> >// Instantiate template for every type and export the symbol
> >void matmul_r4(gfc_array_r4* a, gfc_array_r4* b, gfc_array_r4* c, ...)
> >{
> >  matmul(a, b, c, ...);
> >}
> >// And so on for other types
> > }
> 
> Cheers,
> 
> Tobias

Re: Sourceware mitigating and preventing the next xz-backdoor

[Martin Uecker via Gcc]

Am Samstag, dem 06.04.2024 um 15:00 +0200 schrieb Richard Biener:
> On Fri, Apr 5, 2024 at 11:18 PM Andrew Sutton via Gcc  wrote:
> > 
> > > 
> > > 
> > > 
> > > > I think the key difference here is that Autotools allows arbitrarily
> > > generated code to be executed at any time. More modern build systems
> > > require the use of specific commands/files to run arbitrary code, e.g.
> > > CMake (IIRC [`execute_process()`][2] and [`ExternalProject`][3]), Meson
> > > ([`run_command()`][1]), Cargo ([`build.rs`][4]).\
> > > 
> > > To me it seems that Cargo is the absolute worst case with respect to
> > > supply chain attacks.
> > > 
> > > It pulls in dependencies recursively from a relatively uncurated
> > > list of projects, puts the source of all those dependencies into a
> > > hidden directory in home, and runs Build.rs automatically with
> > > user permissions.
> > > 
> > 
> > 100% this. Wait until you learn how proc macros work.
> 
> proc macro execution should be heavily sandboxed, otherwise it seems
> compiling something is enough to get arbitrary code executed with the
> permission of the compiling user.  I mean it's not rocket science - browsers
> do this for javascript.  Hmm, we need a webassembly target ;)

This would be useful anyhow. 

And locking down the compiler using landlock to only access specified
files / directories would also be nice in general.

Martin

Re: Sourceware mitigating and preventing the next xz-backdoor

[Martin Uecker via Gcc]

Am Mittwoch, dem 03.04.2024 um 13:46 -0500 schrieb Jonathon Anderson via Gcc:
> Hello all,
> 
> On Wed, 2024-04-03 at 16:00 +0200, Michael Matz wrote:
> > > My take a way is that software needs to become less complex. Do 
> > > we really still need complex build systems such as autoconf?
> > 
> > (And, FWIW, testing for features isn't "complex".  And have you looked at 
> > other build systems?  I have, and none of them are less complex, just 
> > opaque in different ways from make+autotools).
> 
> Some brief opinions from a humble end-user:
> 
> I think the key difference here is that Autotools allows arbitrarily 
> generated code to be executed at any time. More modern build systems require 
> the use of specific commands/files to run arbitrary code, e.g. CMake (IIRC 
> [`execute_process()`][2] and [`ExternalProject`][3]), Meson 
> ([`run_command()`][1]), Cargo ([`build.rs`][4]).\

To me it seems that Cargo is the absolute worst case with respect to
supply chain attacks.

It pulls in dependencies recursively from a relatively uncurated
list of projects, puts the source of all those dependencies into a 
hidden directory in home, and runs Build.rs automatically with
user permissions.

Martin





> IMHO there are similarities here to the memory "safety" of Rust: Rust code 
> can have memory errors, but it can only come from Rust code declared as 
> `unsafe` (or bugs in the compiler itself). The scope is limited and those 
> scopes can be audited with more powerful microscopes... and removed if/when 
> the build system gains first-class support upstream.
> 
> There are other features in the newest build systems listed here (Meson and 
> Cargo) that make this particular attack vector harder. These build systems 
> don't have release tarballs with auxiliary files (e.g. [Meson's is very close 
> to `git archive`][5]), nor do their DSLs allow writing files to the source 
> tree. One could imagine a build/CI worker where the source tree is a 
> read-only bind-mount of a `git archive` extract, that might help defend 
> against attacks of this specific design.
> 
> It's also worth noting that Meson and Cargo use non-Turing-complete 
> declarative DSLs for their build configuration. This implies there is an 
> upper bound on the [cyclomatic complexity][6]-per-line of the build script 
> DSL itself. That doesn't mean you can't write complex Meson code (I have), 
> but it ends up being suspiciously long and thus clear something complex and 
> out of the ordinary is being done.
> 
> Of course, this doesn't make the build system any less complex, but projects 
> using newer build systems seem easier to secure and audit than those using 
> overly flexible build systems like Autotools and maybe even CMake. IMHO using 
> a late-model build system is a relatively low technical hurdle to overcome 
> for the benefits noted above, switching should be considered and in a 
> positive light.
> 
> (For context: my team recently switched our main C/C++ project from Autotools 
> to Meson. The one-time refactor itself was an effort, but after that we got 
> back up to speed quickly and we haven't looked back. Other projects may have 
> an easier time using an unofficial port in the [Meson WrapDB][7] as a 
> starting point.)
> 
> -Jonathon
> 
> [1]: https://mesonbuild.com/External-commands.html
> [2]: 
> https://cmake.org/cmake/help/latest/command/execute_process.html#execute-process
> [3]: https://cmake.org/cmake/help/latest/module/ExternalProject.html
> [4]: https://doc.rust-lang.org/cargo/reference/build-scripts.html
> [5]: https://mesonbuild.com/Creating-releases.html
> [6]: https://en.wikipedia.org/wiki/Cyclomatic_complexity
> [7]: https://mesonbuild.com/Wrapdb-projects.html

Re: Sourceware mitigating and preventing the next xz-backdoor

[Martin Uecker via Gcc]

Am Mittwoch, dem 03.04.2024 um 18:02 +0200 schrieb Michael Matz:
> Hello,
> 
> On Wed, 3 Apr 2024, Martin Uecker wrote:
> 
> > The backdoor was hidden in a complicated autoconf script...
> 
> Which itself had multiple layers and could just as well have been a 
> complicated cmake function.

Don't me wrong, cmake is no way better. Another problem was 
actually hidden in a cmake test in upstream git in plain
sight:

https://git.tukaani.org/?p=xz.git;a=commitdiff;h=f9cf4c05edd14dedfe63833f8ccbe41b55823b00;hp=af071ef7702debef4f1d324616a0137a5001c14c

> 
> > > (And, FWIW, testing for features isn't "complex".  And have you looked at 
> > > other build systems?  I have, and none of them are less complex, just 
> > > opaque in different ways from make+autotools).
> > 
> > I ask a very specific question: To what extend is testing 
> > for features instead of semantic versions and/or supported
> > standards still necessary?
> 
> I can't answer this with absolute certainty, but points to consider: the 
> semantic versions need to be maintained just as well, in some magic way.  

It would certainly need to be maintained just like now autoconf
configuration needs to be maintained.

> Because ultimately software depend on features of dependencies not on 
> arbitrary numbers given to them.  The numbers encode these features, in 
> the best case, when there are no errors.  So, no, version numbers are not 
> a replacement for feature tests, they are a proxy.  One that is manually 
> maintained, and hence prone to errors.

Tests are also prone to errors and - as the example above shows -
also very fragile and susceptible to manipulation.

> 
> Now, supported standards: which one? ;-)  Or more in earnest: while on 
> this mailing list here we could chose a certain set, POSIX, some 
> languages, Windows, MacOS (versions so-and-so).  What about other 
> software relying on other 3rdparty feature providers (libraries or system 
> services)?  Without standards?
> 
> So, without absolute certainty, but with a little bit of it: yes, feature 
> tests are required in general.  That doesn't mean that we could not 
> do away with quite some of them for (e.g.) GCC, those that hold true on 
> any platform we support.  But we can't get rid of the infrastructure for 
> that, and can't get rid of certain classes of tests.
> 
> > This seems like a problematic approach that may have been necessary 
> > decades ago, but it seems it may be time to move on.
> 
> I don't see that.  Many aspects of systems remain non-standardized.

This is just part of the problem.

Martin

> 
> 
> Ciao,
> Michael.

Re: Sourceware mitigating and preventing the next xz-backdoor

[Martin Uecker via Gcc]

Am Mittwoch, dem 03.04.2024 um 16:00 +0200 schrieb Michael Matz:
> Hello,
> 
> On Wed, 3 Apr 2024, Martin Uecker via Gcc wrote:
> 
> > > > Seems reasonable, but note that it wouldn't make any difference to
> > > > this attack.  The liblzma library was modified to corrupt the sshd
> > > > binary, when sshd was linked against liblzma.  The actual attack
> > > > occurred via a connection to a corrupt sshd.  If sshd was running as
> > > > root, as is normal, the attacker had root access to the machine.  None
> > > > of the attacking steps had anything to do with having root access
> > > > while building or installing the program.
> > 
> > There does not seem a single good solution against something like this.
> > 
> > My take a way is that software needs to become less complex. Do 
> > we really still need complex build systems such as autoconf?
> 
> Do we really need complex languages like C++ to write our software in?  
> SCNR :)  

Probably not.

> Complexity lies in the eye of the beholder, but to be honest in 
> the software that we're dealing with here, the build system or autoconf 
> does _not_ come to mind first when thinking about complexity.

The backdoor was hidden in a complicated autoconf script...

> 
> (And, FWIW, testing for features isn't "complex".  And have you looked at 
> other build systems?  I have, and none of them are less complex, just 
> opaque in different ways from make+autotools).

I ask a very specific question: To what extend is testing 
for features instead of semantic versions and/or supported
standards still necessary?  This seems like a problematic approach
that  may have been necessary decades ago, but it seems it may be
time to move on.


Martin

Re: Sourceware mitigating and preventing the next xz-backdoor

[Martin Uecker via Gcc]

Am Dienstag, dem 02.04.2024 um 13:28 -0700 schrieb Ian Lance Taylor via Gcc:
> > On Tue, Apr 2, 2024 at 1:21 PM Paul Koning via Gcc  wrote:
> > > > 
> > > > Would it help to require (rather than just recommend) "don't use root 
> > > > except for the actual 'install' step" ?
> > 
> > Seems reasonable, but note that it wouldn't make any difference to
> > this attack.  The liblzma library was modified to corrupt the sshd
> > binary, when sshd was linked against liblzma.  The actual attack
> > occurred via a connection to a corrupt sshd.  If sshd was running as
> > root, as is normal, the attacker had root access to the machine.  None
> > of the attacking steps had anything to do with having root access
> > while building or installing the program.

There does not seem a single good solution against something like this.

My take a way is that software needs to become less complex. Do 
we really still need complex build systems such as autoconf?
Are there still so many different configurations with subtle differences 
that every single feature needs to be tested individually by
running code at build time?

Martin

Re: aliasing

[Martin Uecker via Gcc]

Am Montag, dem 18.03.2024 um 14:21 +0100 schrieb Richard Biener:
> On Mon, Mar 18, 2024 at 12:56 PM Martin Uecker  wrote:
> > 
> > Am Montag, dem 18.03.2024 um 11:55 +0100 schrieb Martin Uecker:
> > > Am Montag, dem 18.03.2024 um 09:26 +0100 schrieb Richard Biener:
> > > > On Mon, Mar 18, 2024 at 8:03 AM Martin Uecker  wrote:
> > > > 
> > > 
> > > > 
> > > > Let me give you an complication example made valid in C++:
> > > > 
> > > > struct B { float x; float y; };
> > > > struct X { int n; char buf[8]; } x, y;
> > > > 
> > > > void foo(struct B *b)
> > > > {
> > > >   memcpy (x.buf, b, sizeof (struct B)); // in C++:  new (x.buf) B (*b);
> > > 
> > > Let's make it an explicit store for the moment
> > > (should not make a difference though):
> > > 
> > > *(struct B*)x.buf = *b;
> > > 
> > > >   y = x; // (*)
> > > > }
> > > > 
> > > > What's the effective type of 'x' in the 'y = x' copy?
> > > 
> > > Good point. The existing wording would take the declared
> > > type of x as the effective type, but this may not be
> > > what you are interested in. Let's assume that x has no declared
> > > type but that it had effective type struct X before the
> > > store to x.buf (because of an even earlier store to
> > > x with type struct X).
> > > 
> > > There is a general question how stores to subobjects
> > > affect effective types and I do not think this is clear
> > > even before this proposed change.
> > 
> > Actually, I think this is not allowed because:
> > 
> > "An object shall have its stored value accessed only by an
> > lvalue expression that has one of the following types:
> > 
> > — a type compatible with the effective type of the object,
> > ...
> > — an aggregate or union type that includes one of the
> > aforementioned types among its members (including,
> > recursively, a member of a subaggregate or contained union), or
> > 
> > — a character type."
> > 
> > ... and we would need to move "a character type" above
> > in the list to make it defined.
> 
> So after
> 
> *(struct B*)x.buf = *b;
> 
> 'x' cannot be used to access itself?  In particular also
> an access to 'x.n' is affected by this?

According to the current wording and assuming x has no
a declared type,  x.buf would acquire an effective 
type of struct B. Then if  x.buf is read as part of 
a x it is accessed with an lvalue of struct X (which
does not include a struct B but a character buffer).

So yes, currently it would  be undefined behavior 
and the proposed wording would not change this. Clearly,
we should include an additional change to fix this.

> 
> You are right that the current wording of the standard doesn't
> clarify any of this but this kind of storage abstraction is used
> commonly in the embedded world when there's no runtime
> library providing allocation.  And you said you want to make
> the standard closer to implementation practice ...

Well, we are working on it... Any help is much appreciated.

> 
> Elsewhere when doing 'y = x' people refer to the wording that
> aggregates are copied elementwise but it's not specified how
> those elementwise accesses work - the lvalues are still of type
> X here or are new lvalues implicitly formed and fall under the
> other wordings? 

I think there is no wording for elementwise copy.

My understanding is that the 

"...an aggregate or union type that includes..."

wording above is supposed to define this via an lvalue
access with aggregate or union type.  It blesses the
implied access to the elements via the access with 
an lvalue which has the type of the aggregate.  


>  Can I thus form an effective type of X by
> storing it's subobjects at respective offsets (ignoring padding,
> for example) and can I then use an lvalue of type 'X' to access
> the whole aggregate?

I think this is defined behavior.  The subjects get
their effective types via the individual stores and then 
the access using lvalue of type 'X' is ok according to
the "..an aggregate or union type that includes.."
rule.


Martin

Re: aliasing

[Martin Uecker via Gcc]

awkward or inefficient 
> to write code that is completely "safe" (in terms of having fully 
> defined behaviour from the C standards or from implementation-dependent 
> behaviour).  Making your own dynamic memory allocation functions is one 
> such case.  So I have a tendency to jump on any suggestion of changes to 
> the C (or C++) standards that could let people write such essential code 
> in a safer or more efficient manner.

That something is undefined does not automatically mean it is 
forbidden or unsafe.  It simply means it is not portable.  I think
in the embedded space it will be difficult to make everything well
defined.  But I fully agree that widely used techniques should
ideally be based on defined behavior and we should  change the
standard accordingly.

> 
> > > (It is also not uncommon to have the backing space allocated by the
> > > linker, but then it falls under the existing "no declared type" case.)
> > 
> > Yes, although with the change we would make the "no declared type" also
> > be byte arrays, so there is then simply no difference anymore.
> > 
> 
> Fair enough.  (Linker-defined storage does not just have no declared 
> type, it has no directly declared size or other properties either.  The 
> start and the stop of the storage area is typically declared as "extern 
> uint8_t __space_start[], __space_stop[];", or perhaps as single 
> characters or uint32_t types.  The space in between is just calculated 
> as the difference between pointers to these.)
> 
> > > 
> > > 
> > > I would not want uint32_t to be considered an "alias anything" type, but
> > > I have occasionally seen such types used for memory store backings.  It
> > > is perhaps worth considering defining "byte type" as "non-atomic
> > > character type, [u]int8_t (if they exist), or other
> > > implementation-defined types".
> > 
> > This could make sense, the question is whether we want to encourage
> > the use of other types for this use case, as this would then not
> > be portable.
> 
> I think uint8_t should be highly portable, except to targets where it 
> does not exist (and in this day and age, that basically means some DSP 
> devices that have 16-bit, 24-bit or 32-bit char).
> 
> There is precedence for this wording, however, in 6.7.2.1p5 for 
> bit-fields - "A bit-field shall have a type that is a qualified or 
> unqualified version of _Bool, signed int, unsigned int, or some other 
> implementation-defined type".
> 
> I think it should be clear enough that using an implementation-defined 
> type rather than a character type would potentially limit portability. 
> For the kinds of systems I am thinking off, extreme portability is 
> normally not of prime concern - efficiency on a particular target with a 
> particular compiler is often more important.

Thanks, I will bring back this information to WG14.
> 
> > 
> > Are there important reason for not using "unsigned char" ?
> > 
> 
> What is "important" is often a subjective matter.  One reason many 
> people use "uint8_t" is that they prefer to be explicit about sizes, and 
> would rather have a hard error if the code is used on a target that 
> doesn't support the size.  Some coding standards, such as the very 
> common (though IMHO somewhat flawed) MISRA standard, strongly encourage 
> size-specific types and consider the use of "int" or "unsigned char" as 
> a violation of their rules and directives.  Many libraries and code 
> bases with a history older than C99 have their own typedef names for 
> size-specific types or low-level storage types, such as "sys_uint8", 
> "BYTE", "u8", and so on, and users may prefer these for consistency. 
> And for people with a background in hardware or assembly (not uncommon 
> for small systems embedded programming), or other languages such as 
> Rust, "unsigned char" sounds vague, poorly defined, and somewhat 
> meaningless as a type name for a raw byte of memory or a minimal sized 
> unsigned integer.
> 
> Of course most alternative names for bytes would be typedefs of 
> "unsigned char" and therefore work just the same way.  But as noted 
> before, uint8_t could be defined in another manner on some systems (and 
> on GCC for the AVR, it /is/ defined in a different way - though I have 
> no idea why).
> 
> And bigger types, such as uint32_t, have been used to force alignment 
> for backing store (either because the compiler did not support _Alignas, 
> or the programmer did not know about it).  (But I am not suggesting that 
>

Re: aliasing

[Martin Uecker via Gcc]

Am Montag, dem 18.03.2024 um 11:55 +0100 schrieb Martin Uecker:
> Am Montag, dem 18.03.2024 um 09:26 +0100 schrieb Richard Biener:
> > On Mon, Mar 18, 2024 at 8:03 AM Martin Uecker  wrote:
> > 
> 
> > 
> > Let me give you an complication example made valid in C++:
> > 
> > struct B { float x; float y; };
> > struct X { int n; char buf[8]; } x, y;
> > 
> > void foo(struct B *b)
> > {
> >   memcpy (x.buf, b, sizeof (struct B)); // in C++:  new (x.buf) B (*b);
> 
> Let's make it an explicit store for the moment
> (should not make a difference though):
> 
> *(struct B*)x.buf = *b;
> 
> >   y = x; // (*)
> > }
> > 
> > What's the effective type of 'x' in the 'y = x' copy? 
> 
> Good point. The existing wording would take the declared
> type of x as the effective type, but this may not be
> what you are interested in. Let's assume that x has no declared
> type but that it had effective type struct X before the
> store to x.buf (because of an even earlier store to 
> x with type struct X).
> 
> There is a general question how stores to subobjects
> affect effective types and I do not think this is clear
> even before this proposed change.

Actually, I think this is not allowed because:

"An object shall have its stored value accessed only by an
lvalue expression that has one of the following types:

— a type compatible with the effective type of the object,
...
— an aggregate or union type that includes one of the
aforementioned types among its members (including,
recursively, a member of a subaggregate or contained union), or

— a character type."

... and we would need to move "a character type" above
in the list to make it defined.

Martin

Re: aliasing

[Martin Uecker via Gcc]

Hi David,

Am Montag, dem 18.03.2024 um 10:00 +0100 schrieb David Brown:
> Hi,
> 
> I would very glad to see this change in the standards.
> 
> 
> Should "byte type" include all character types (signed, unsigned and 
> plain), or should it be restricted to "unsigned char" since that is the 
> "byte" type ?  (I think allowing all character types makes sense, but 
> only unsigned char is guaranteed to be suitable for general object 
> backing store.)

At the moment, the special type that can access all others are
all non-atomic character types.  So for symmetry reasons, it
seems that this is also what we want for backing store.

I am not sure what you mean by "only unsigned char". Are you talking
about C++?  "unsigned char" has no special role in C.

> 
> Should it also include "uint8_t" (if it exists) ?  "uint8_t" is often an 
> alias for "unsigned char", but it could be something different, like an 
> alias for __UINT8_TYPE__, or "unsigned int 
> __attribute__((mode(QImode)))", which is used in the AVR gcc port.

I think this might be a reason to not include it, as it could
affect aliasing analysis. At least, this would be a different
independent change to consider.

> 
> In my line of work - small-systems embedded development - it is common 
> to have "home-made" or specialised memory allocation systems rather than 
> relying on a generic heap.  This is, I think, some of the "existing 
> practice" that you are considering here - there is a "backing store" of 
> some sort that can be allocated and used as objects of a type other than 
> the declared type of the backing store.  While a simple unsigned char 
> array is a very common kind of backing store, there are others that are 
> used, and it would be good to be sure of the correctness guarantees for 
> these.  Possibilities that I have seen include:
> 
> unsigned char heap1[N];
> 
> uint8_t heap2[N];
> 
> union {
>   double dummy_for_alignment;
>   char heap[N];
> } heap3;
> 
> struct {
>   uint32_t capacity;
>   uint8_t * p_next_free;
>   uint8_t heap[N];
> } heap4;
> 
> uint32_t heap5[N];
> 
> Apart from this last one, if "uint8_t" is guaranteed to be a "byte 
> type", then I believe your wording means that these unions and structs 
> would also work as "byte arrays".  But it might be useful to add a 
> footnote clarifying that.
> 

I need to think about this. 

> (It is also not uncommon to have the backing space allocated by the 
> linker, but then it falls under the existing "no declared type" case.)

Yes, although with the change we would make the "no declared type" also 
be byte arrays, so there is then simply no difference anymore.

> 
> 
> I would not want uint32_t to be considered an "alias anything" type, but 
> I have occasionally seen such types used for memory store backings.  It 
> is perhaps worth considering defining "byte type" as "non-atomic 
> character type, [u]int8_t (if they exist), or other 
> implementation-defined types".

This could make sense, the question is whether we want to encourage
the use of other types for this use case, as this would then not
be portable.

Are there important reason for not using "unsigned char" ?

> 
> Some other compilers might guarantee not to do type-based alias analysis 
> and thus view all types as "byte types" in this way.  For gcc, there 
> could be a kind of reverse "may_alias" type attribute to create such types.
> 
> 
> 
> There are a number of other features that could make allocation 
> functions more efficient and safer in use, and which could be ideally be 
> standardised in the C standards or at least added as gcc extensions, but 
> I think that's more than you are looking for here!

It is possible to submit proposal to WG14.

Martin


> 
> David
> 
> 
> 
> On 18/03/2024 08:03, Martin Uecker via Gcc wrote:
> > 
> > Hi,
> > 
> > can you please take a quick look at this? This is intended to align
> > the C standard with existing practice with respect to aliasing by
> > removing the special rules for "objects with no declared type" and
> > making it fully symmetric and only based on types with non-atomic
> > character types being able to alias everything.
> > 
> > 
> > Unrelated to this change, I have another question:  I wonder if GCC
> > (or any other compiler) actually exploits the " or is copied as an
> > array of  byte type, " rule to  make assumptions about the effective
> > types of the target array? I know

Re: aliasing

[Martin Uecker via Gcc]

Am Montag, dem 18.03.2024 um 09:26 +0100 schrieb Richard Biener:
> On Mon, Mar 18, 2024 at 8:03 AM Martin Uecker  wrote:
> > 
> > 
> > Hi,
> > 
> > can you please take a quick look at this? This is intended to align
> > the C standard with existing practice with respect to aliasing by
> > removing the special rules for "objects with no declared type" and
> > making it fully symmetric and only based on types with non-atomic
> > character types being able to alias everything.
> > 
> > 
> > Unrelated to this change, I have another question:  I wonder if GCC
> > (or any other compiler) actually exploits the " or is copied as an
> > array of  byte type, " rule to  make assumptions about the effective
> > types of the target array?
> 
> We do not make assumptions about this anymore.  We did in the
> past (might be a distant past) transform say
> 
> struct X { int i; float f; } a, b;
> 
> void foo ()
> {
>   __builtin_memcpy (&a, &b, sizeof (struct X));
> }
> 
> into
> 
>   a = b;
> 
> which has an lvalue of type struct X.  But this assumed b's effective
> type was X.  Nowadays we treat the copy as using alias set zero.
> That effectively means the destination gets its effective type "cleared"
> (all subsequent accesses are valid to access storage with the effective
> type of a byte array).

Ok, thanks!  I wonder whether we should remove this special rule
from the standard.  I mostly worried about the "copied as an
array of  byte type" wording which seems difficult to precisely
define.

> 
> > I know compilers do this work memcpy...
> > Maybe also if a loop is transformed to memcpy?
> 
> We currently do not preserve the original effective type of the destination
> (or the effective type used to access the source) when doing this.  With
> some tricks we could (we also lose aligment guarantees of the original
> accesses).
> 
> > Martin
> > 
> > 
> > Add the following definition after 3.5, paragraph 2:
> > 
> > byte array
> > object having either no declared type or an array of objects declared with 
> > a byte type
> > 
> > byte type
> > non-atomic character type

This essentially becomes the "alias anything" type.

> > 
> > Modify 6.5,paragraph 6:
> > The effective type of an object that is not a byte array, for an access to 
> > its
> > stored value, is the declared type of the object.97) If a value is
> > stored into a byte array through an lvalue having a byte type, then
> > the type of the lvalue becomes the effective type of the object for that
> > access and for subsequent accesses that do not modify the stored value.
> > If a value is copied into a byte array using memcpy or memmove, or is
> > copied as an array of byte type, then the effective type of the
> > modified object for that access and for subsequent accesses that do not
> > modify the value is the effective type of the object from which the
> > value is copied, if it has one. For all other accesses to a byte array,
> > the effective type of the object is simply the type of the lvalue used
> > for the access.
> 
> What's the purpose of this change?  To me this reads more confusing and
> complicated than what I find in the c23 draft from April last year.

Note that C23 has been finalized. This change is proposed for the
revision after c23. 

> 
> I'll note that GCC does not take advantage of "The effective type of an
> object for an access to its stored value is the declard type of the object",
> instead it always relies on the type of the lvalue (treating non-atomic
> character types specially, as well as treating all string ops like memcpy
> or strcpy as using a character type for the access) and the effective type
> of the object for that access and for subsequent accesses that do not
> modify the stored value always becomes that of the lvalue type used for
> the access.

Understood.

> 
> Let me give you an complication example made valid in C++:
> 
> struct B { float x; float y; };
> struct X { int n; char buf[8]; } x, y;
> 
> void foo(struct B *b)
> {
>   memcpy (x.buf, b, sizeof (struct B)); // in C++:  new (x.buf) B (*b);

Let's make it an explicit store for the moment
(should not make a difference though):

*(struct B*)x.buf = *b;

>   y = x; // (*)
> }
> 
> What's the effective type of 'x' in the 'y = x' copy? 

Good point. The existing wording would take the declared
type of x as the effective type, but this may not be
what you are interested in. Let's assume that x has no declared
type but that it had effective type struct X before the
store to x.buf (because of an even earlier store to 
x with type struct X).

There is a general question how stores to subobjects
affect effective types and I do not think this is clear
even before this proposed change.


>  With your new
> wording, does 'B' transfer to x.buf with memcpy?  

Yes, it would. At least this is the intention.

Note that this would currently be undefined behavior
because x.buf has a declared type. So this is main
thing we want to change, i.e. making this defined.

> What's 

aliasing

[Martin Uecker via Gcc]

Hi,

can you please take a quick look at this? This is intended to align
the C standard with existing practice with respect to aliasing by
removing the special rules for "objects with no declared type" and
making it fully symmetric and only based on types with non-atomic
character types being able to alias everything.


Unrelated to this change, I have another question:  I wonder if GCC
(or any other compiler) actually exploits the " or is copied as an
array of  byte type, " rule to  make assumptions about the effective
types of the target array? I know compilers do this work memcpy...  
Maybe also if a loop is transformed to memcpy?

Martin


Add the following definition after 3.5, paragraph 2:

byte array
object having either no declared type or an array of objects declared with a 
byte type

byte type
non-atomic character type

Modify 6.5,paragraph 6:
The effective type of an object that is not a byte array, for an access to its
stored value, is the declared type of the object.97) If a value is
stored into a byte array through an lvalue having a byte type, then
the type of the lvalue becomes the effective type of the object for that
access and for subsequent accesses that do not modify the stored value.
If a value is copied into a byte array using memcpy or memmove, or is 
copied as an array of byte type, then the effective type of the
modified object for that access and for subsequent accesses that do not
modify the value is the effective type of the object from which the
value is copied, if it has one. For all other accesses to a byte array,
the effective type of the object is simply the type of the lvalue used
for the access.

https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3230.pdf

Re: Question on -fwrapv and -fwrapv-pointer

[Martin Uecker via Gcc]

Am Mittwoch, dem 20.09.2023 um 13:40 -0700 schrieb Kees Cook via Gcc:
> On Sat, Sep 16, 2023 at 10:36:52AM +0200, Martin Uecker wrote:
> > > On Fri, Sep 15, 2023 at 08:18:28AM -0700, Andrew Pinski wrote:
> > > > On Fri, Sep 15, 2023 at 8:12 AM Qing Zhao  wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > > On Sep 15, 2023, at 3:43 AM, Xi Ruoyao  wrote:
> > > > > > 
> > > > > > On Thu, 2023-09-14 at 21:41 +, Qing Zhao wrote:
> > > > > > > > > CLANG already provided -fsanitize=unsigned-integer-overflow. 
> > > > > > > > > GCC
> > > > > > > > > might need to do the same.
> > > > > > > > 
> > > > > > > > NO. There is no such thing as unsigned integer overflow. That 
> > > > > > > > option
> > > > > > > > is badly designed and the GCC community has rejected a few 
> > > > > > > > times now
> > > > > > > > having that sanitizer before. It is bad form to have a 
> > > > > > > > sanitizer for
> > > > > > > > well defined code.
> > > > > > > 
> > > > > > > Even though unsigned integer overflow is well defined, it might be
> > > > > > > unintentional, shall we warn user about this?
> > > > > > 
> > > > > > *Everything* could be unintentional and should be warned then.  GCC 
> > > > > > is a
> > > > > > compiler, not an advanced AI educating the programmers.
> > > > > 
> > > > > Well, you are right in some sense. -:)
> > > > > 
> > > > > However, overflow is one important source for security flaws, it’s 
> > > > > important  for compilers to detect
> > > > > overflows in the programs in general.
> > > > 
> > > > Except it is NOT an overflow. Rather it is wrapping. That is a big
> > > > point here. unsigned wraps and does NOT overflow. Yes there is a major
> > > > difference.
> > > 
> > > Right, yes. I will try to pick my language very carefully. :)
> > > 
> > > The practical problem I am trying to solve in the 30 million lines of
> > > Linux kernel code is that of catching arithmetic wrap-around. The
> > > problem is one of evolving the code -- I can't just drop -fwrapv and
> > > -fwrapv-pointer because it's not possible to fix all the cases at once.
> > > (And we really don't want to reintroduce undefined behavior.)
> > > 
> > > So, for signed, pointer, and unsigned types, we need:
> > > 
> > > a) No arithmetic UB -- everything needs to have deterministic behavior.
> > >The current solution here is "-fno-strict-overflow", which eliminates
> > >the UB and makes sure everything wraps.
> > > 
> > > b) A way to run-time warn/trap on overflow/underflow/wrap-around. This
> > >would work with -fsanitize=[signed-integer|pointer]-overflow except
> > >due to "a)" we always wrap. And there isn't currently coverage like
> > >this for unsigned (in GCC).
> > > 
> > > Our problem is that the kernel is filled with a mix of places where there
> > > is intended wrap-around and unintended wrap-around. We can chip away at
> > > fixing the intended wrap-around that we can find with static analyzers,
> > > etc, but at the end of the day there is a long tail of finding the places
> > > where intended wrap-around is hiding. But when the refactoring is
> > > sufficiently completely, we can move the wrap-around warning to a trap,
> > > and the kernel will not longer have this class of security flaw.
> > > 
> > > As a real-world example, here is a bug where a u8 wraps around causing
> > > an under-allocation that allowed for a heap overwrite:
> > > 
> > > https://git.kernel.org/linus/6311071a0562
> > > https://elixir.bootlin.com/linux/v6.5/source/net/wireless/nl80211.c#L5422
> > > 
> > > If there were more than 255 elements in a linked list, the allocation
> > > would be too small, and the second loop would write past the end of the
> > > allocation. This is a pretty classic allocation underflow and linear
> > > heap write overflow security flaw. (And it would be trivially stopped by
> > > trapping on the u8 wrap around.)
> > > 
> > > So, I want to be able to catch that at run-time. But we also have code
> > > doing things like "if (ulong + offset < ulong) { ... }":
> > > 
> > > https://elixir.bootlin.com/linux/v6.5/source/drivers/crypto/axis/artpec6_crypto.c#L1187
> > > 
> > > This is easy for a static analyzer to find and we can replace it with a
> > > non-wrapping test (e.g. __builtin_add_overflow()), but we'll not find
> > > them all immediately, especially for the signed and pointer cases.
> > > 
> > > So, I need to retain the "everything wraps" behavior while still being
> > > able to detect when it happens.
> > 
> > 
> > Hi Kees,
> > 
> > I have a couple of questions:
> > 
> > Currently, my thinking was that you would use signed integers
> > if you want the usual integer arithmetic rules we know from
> > elementary school and if you overflow this is clearly a bug 
> > you can diagnose with UBsan.
> > 
> > There are people who think that signed overflow should be
> > defined to wrap, but I think this would be a severe
> > mistake because then code would start to rely on it, which
> > makes it then difficult to dif

Re: Question on -fwrapv and -fwrapv-pointer

[Martin Uecker via Gcc]

Am Montag, dem 18.09.2023 um 10:47 +0200 schrieb Richard Biener via Gcc:
> On Mon, Sep 18, 2023 at 10:17 AM Martin Uecker  wrote:
> > 
> > Am Montag, dem 18.09.2023 um 09:31 +0200 schrieb Richard Biener via Gcc:
> > > On Sat, Sep 16, 2023 at 10:38 AM Martin Uecker via Gcc  
> > > wrote:
> > > > 
> > > > 
> > > > 
> > > > (moved to gcc@)
> > > > 
> > > > > On Fri, Sep 15, 2023 at 08:18:28AM -0700, Andrew Pinski wrote:
> > > > > > On Fri, Sep 15, 2023 at 8:12 AM Qing Zhao  
> > > > > > wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > On Sep 15, 2023, at 3:43 AM, Xi Ruoyao  
> > > > > > > > wrote:
> > > > > > > > 
> > > > > > > > On Thu, 2023-09-14 at 21:41 +, Qing Zhao wrote:
> > > > > > > > > > > CLANG already provided 
> > > > > > > > > > > -fsanitize=unsigned-integer-overflow. GCC
> > > > > > > > > > > might need to do the same.
> > > > > > > > > > 
> > > > > > > > > > NO. There is no such thing as unsigned integer overflow. 
> > > > > > > > > > That option
> > > > > > > > > > is badly designed and the GCC community has rejected a few 
> > > > > > > > > > times now
> > > > > > > > > > having that sanitizer before. It is bad form to have a 
> > > > > > > > > > sanitizer for
> > > > > > > > > > well defined code.
> > > > > > > > > 
> > > > > > > > > Even though unsigned integer overflow is well defined, it 
> > > > > > > > > might be
> > > > > > > > > unintentional, shall we warn user about this?
> > > > > > > > 
> > > > > > > > *Everything* could be unintentional and should be warned then.  
> > > > > > > > GCC is a
> > > > > > > > compiler, not an advanced AI educating the programmers.
> > > > > > > 
> > > > > > > Well, you are right in some sense. -:)
> > > > > > > 
> > > > > > > However, overflow is one important source for security flaws, 
> > > > > > > it’s important  for compilers to detect
> > > > > > > overflows in the programs in general.
> > > > > > 
> > > > > > Except it is NOT an overflow. Rather it is wrapping. That is a big
> > > > > > point here. unsigned wraps and does NOT overflow. Yes there is a 
> > > > > > major
> > > > > > difference.
> > > > > 
> > > > > Right, yes. I will try to pick my language very carefully. :)
> > > > > 
> > > > > The practical problem I am trying to solve in the 30 million lines of
> > > > > Linux kernel code is that of catching arithmetic wrap-around. The
> > > > > problem is one of evolving the code -- I can't just drop -fwrapv and
> > > > > -fwrapv-pointer because it's not possible to fix all the cases at 
> > > > > once.
> > > > > (And we really don't want to reintroduce undefined behavior.)
> > > > > 
> > > > > So, for signed, pointer, and unsigned types, we need:
> > > > > 
> > > > > a) No arithmetic UB -- everything needs to have deterministic 
> > > > > behavior.
> > > > >The current solution here is "-fno-strict-overflow", which 
> > > > > eliminates
> > > > >the UB and makes sure everything wraps.
> > > > > 
> > > > > b) A way to run-time warn/trap on overflow/underflow/wrap-around. This
> > > > >would work with -fsanitize=[signed-integer|pointer]-overflow except
> > > > >due to "a)" we always wrap. And there isn't currently coverage like
> > > > >this for unsigned (in GCC).
> > > > > 
> > > > > Our problem is that the kernel is filled with a mix of places where 
> > > > > there
> > > > > is intended wrap-around and unintended wrap-around. We can chip away 
> > > > > at
> > > > > fixing the intended wrap-around 

Re: Question on -fwrapv and -fwrapv-pointer

[Martin Uecker via Gcc]

Am Montag, dem 18.09.2023 um 09:31 +0200 schrieb Richard Biener via Gcc:
> On Sat, Sep 16, 2023 at 10:38 AM Martin Uecker via Gcc  
> wrote:
> > 
> > 
> > 
> > (moved to gcc@)
> > 
> > > On Fri, Sep 15, 2023 at 08:18:28AM -0700, Andrew Pinski wrote:
> > > > On Fri, Sep 15, 2023 at 8:12 AM Qing Zhao  wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > > On Sep 15, 2023, at 3:43 AM, Xi Ruoyao  wrote:
> > > > > > 
> > > > > > On Thu, 2023-09-14 at 21:41 +, Qing Zhao wrote:
> > > > > > > > > CLANG already provided -fsanitize=unsigned-integer-overflow. 
> > > > > > > > > GCC
> > > > > > > > > might need to do the same.
> > > > > > > > 
> > > > > > > > NO. There is no such thing as unsigned integer overflow. That 
> > > > > > > > option
> > > > > > > > is badly designed and the GCC community has rejected a few 
> > > > > > > > times now
> > > > > > > > having that sanitizer before. It is bad form to have a 
> > > > > > > > sanitizer for
> > > > > > > > well defined code.
> > > > > > > 
> > > > > > > Even though unsigned integer overflow is well defined, it might be
> > > > > > > unintentional, shall we warn user about this?
> > > > > > 
> > > > > > *Everything* could be unintentional and should be warned then.  GCC 
> > > > > > is a
> > > > > > compiler, not an advanced AI educating the programmers.
> > > > > 
> > > > > Well, you are right in some sense. -:)
> > > > > 
> > > > > However, overflow is one important source for security flaws, it’s 
> > > > > important  for compilers to detect
> > > > > overflows in the programs in general.
> > > > 
> > > > Except it is NOT an overflow. Rather it is wrapping. That is a big
> > > > point here. unsigned wraps and does NOT overflow. Yes there is a major
> > > > difference.
> > > 
> > > Right, yes. I will try to pick my language very carefully. :)
> > > 
> > > The practical problem I am trying to solve in the 30 million lines of
> > > Linux kernel code is that of catching arithmetic wrap-around. The
> > > problem is one of evolving the code -- I can't just drop -fwrapv and
> > > -fwrapv-pointer because it's not possible to fix all the cases at once.
> > > (And we really don't want to reintroduce undefined behavior.)
> > > 
> > > So, for signed, pointer, and unsigned types, we need:
> > > 
> > > a) No arithmetic UB -- everything needs to have deterministic behavior.
> > >The current solution here is "-fno-strict-overflow", which eliminates
> > >the UB and makes sure everything wraps.
> > > 
> > > b) A way to run-time warn/trap on overflow/underflow/wrap-around. This
> > >would work with -fsanitize=[signed-integer|pointer]-overflow except
> > >due to "a)" we always wrap. And there isn't currently coverage like
> > >this for unsigned (in GCC).
> > > 
> > > Our problem is that the kernel is filled with a mix of places where there
> > > is intended wrap-around and unintended wrap-around. We can chip away at
> > > fixing the intended wrap-around that we can find with static analyzers,
> > > etc, but at the end of the day there is a long tail of finding the places
> > > where intended wrap-around is hiding. But when the refactoring is
> > > sufficiently completely, we can move the wrap-around warning to a trap,
> > > and the kernel will not longer have this class of security flaw.
> > > 
> > > As a real-world example, here is a bug where a u8 wraps around causing
> > > an under-allocation that allowed for a heap overwrite:
> > > 
> > > https://git.kernel.org/linus/6311071a0562
> > > https://elixir.bootlin.com/linux/v6.5/source/net/wireless/nl80211.c#L5422
> > > 
> > > If there were more than 255 elements in a linked list, the allocation
> > > would be too small, and the second loop would write past the end of the
> > > allocation. This is a pretty classic allocation underflow and linear
> > > heap write overflow security flaw. (And it would be trivially stopped by
> > > trapping o

Re: Question on -fwrapv and -fwrapv-pointer

[Martin Uecker via Gcc]

(moved to gcc@)

> On Fri, Sep 15, 2023 at 08:18:28AM -0700, Andrew Pinski wrote:
> > On Fri, Sep 15, 2023 at 8:12 AM Qing Zhao  wrote:
> > >
> > >
> > >
> > > > On Sep 15, 2023, at 3:43 AM, Xi Ruoyao  wrote:
> > > >
> > > > On Thu, 2023-09-14 at 21:41 +, Qing Zhao wrote:
> > >  CLANG already provided -fsanitize=unsigned-integer-overflow. GCC
> > >  might need to do the same.
> > > >>>
> > > >>> NO. There is no such thing as unsigned integer overflow. That option
> > > >>> is badly designed and the GCC community has rejected a few times now
> > > >>> having that sanitizer before. It is bad form to have a sanitizer for
> > > >>> well defined code.
> > > >>
> > > >> Even though unsigned integer overflow is well defined, it might be
> > > >> unintentional, shall we warn user about this?
> > > >
> > > > *Everything* could be unintentional and should be warned then.  GCC is a
> > > > compiler, not an advanced AI educating the programmers.
> > >
> > > Well, you are right in some sense. -:)
> > >
> > > However, overflow is one important source for security flaws, it’s 
> > > important  for compilers to detect
> > > overflows in the programs in general.
> > 
> > Except it is NOT an overflow. Rather it is wrapping. That is a big
> > point here. unsigned wraps and does NOT overflow. Yes there is a major
> > difference.
> 
> Right, yes. I will try to pick my language very carefully. :)
> 
> The practical problem I am trying to solve in the 30 million lines of
> Linux kernel code is that of catching arithmetic wrap-around. The
> problem is one of evolving the code -- I can't just drop -fwrapv and
> -fwrapv-pointer because it's not possible to fix all the cases at once.
> (And we really don't want to reintroduce undefined behavior.)
> 
> So, for signed, pointer, and unsigned types, we need:
> 
> a) No arithmetic UB -- everything needs to have deterministic behavior.
>The current solution here is "-fno-strict-overflow", which eliminates
>the UB and makes sure everything wraps.
> 
> b) A way to run-time warn/trap on overflow/underflow/wrap-around. This
>would work with -fsanitize=[signed-integer|pointer]-overflow except
>due to "a)" we always wrap. And there isn't currently coverage like
>this for unsigned (in GCC).
> 
> Our problem is that the kernel is filled with a mix of places where there
> is intended wrap-around and unintended wrap-around. We can chip away at
> fixing the intended wrap-around that we can find with static analyzers,
> etc, but at the end of the day there is a long tail of finding the places
> where intended wrap-around is hiding. But when the refactoring is
> sufficiently completely, we can move the wrap-around warning to a trap,
> and the kernel will not longer have this class of security flaw.
> 
> As a real-world example, here is a bug where a u8 wraps around causing
> an under-allocation that allowed for a heap overwrite:
> 
> https://git.kernel.org/linus/6311071a0562
> https://elixir.bootlin.com/linux/v6.5/source/net/wireless/nl80211.c#L5422
> 
> If there were more than 255 elements in a linked list, the allocation
> would be too small, and the second loop would write past the end of the
> allocation. This is a pretty classic allocation underflow and linear
> heap write overflow security flaw. (And it would be trivially stopped by
> trapping on the u8 wrap around.)
> 
> So, I want to be able to catch that at run-time. But we also have code
> doing things like "if (ulong + offset < ulong) { ... }":
> 
> https://elixir.bootlin.com/linux/v6.5/source/drivers/crypto/axis/artpec6_crypto.c#L1187
> 
> This is easy for a static analyzer to find and we can replace it with a
> non-wrapping test (e.g. __builtin_add_overflow()), but we'll not find
> them all immediately, especially for the signed and pointer cases.
> 
> So, I need to retain the "everything wraps" behavior while still being
> able to detect when it happens.


Hi Kees,

I have a couple of questions:

Currently, my thinking was that you would use signed integers
if you want the usual integer arithmetic rules we know from
elementary school and if you overflow this is clearly a bug 
you can diagnose with UBsan.

There are people who think that signed overflow should be
defined to wrap, but I think this would be a severe
mistake because then code would start to rely on it, which
makes it then difficult to differentiate between bugs and
intended uses (e.g. the unfortunate situation you have 
with the kernel).

I assume you want to combine UBSan plus wrapping for
production use?  Or only for testing?   Or in other words:
why would testing UBSan and production with wrapping
not be sufficient to find and fix all bugs? 

Wrapping would not be correct because it may lead to
logic errors or use-after-free etc.  I assume it is still
preferred because it more deterministic than whatever comes
out of the optimizer assuming that overflow has UB. Is this
the reasoning applied here?


For unsigned the intended use case is m

Re: Complex numbers in compilers - upcoming GNU Tools Cauldron.

[Martin Uecker via Gcc]

Am Dienstag, dem 12.09.2023 um 11:25 +0200 schrieb Richard Biener via Gcc:
> On Tue, Sep 5, 2023 at 10:44 PM Toon Moene  wrote:
> > 
> > This is going to be an interesting discussion.
> > 
> > In the upcoming GNU Tools Cauldron meeting the representation of complex
> > numbers in GCC will be discussed from the following "starting point":
> > 
> > "Complex numbers are used to describe many physical phenomenons and are
> > of prime importance in data signal processing. Nevertheless, despite
> > being part of the C and C++ standards since C99, they are still not
> > completely first class citizens in mainstream compilers."
> > 
> > *This* is from the Fortran 66 Standard (http://moene.org/~toon/f66.pdf -
> > a photocopy of the 1966 Standard):
> > 
> > - - - - -
> > 
> > Chapter 4. Data Types:
> > ...
> > 4.2.4 Complex Type.
> > 
> > A complex datum is processor approximation to the value of a complex number.
> > ...
> > 
> > - - - - -
> > 
> > I can recall people complaining about the way complex arithmetic was
> > handled by compilers since the late 70s.
> > 
> > This is even obvious in weather forecasting software I have to deal with
> > *today* (all written in Fortran). Some models use complex variables to
> > encode the "spectral" (wave-decomposed) computations in parts where that
> > is useful - others just "degrade" those algorithms to explicitly use reals.
> 
> Lack of applications / benchmarks using complex numbers is also a
> problem for any work on this.
> 

I could probably provide some examples such as a FFT, 
complex Gaussian random number generation, mandelbrot
set computation, etc.

Martin

user branches

[Martin Uecker via Gcc]

Hi all,

is it still possible to have user branches in the repository?

If so, how do I create one?  Simply pushing to users/uecker/vla
or something is rejected.

Martin

Re: [wish] Flexible array members in unions

[Martin Uecker via Gcc]

Am Donnerstag, dem 18.05.2023 um 20:59 + schrieb Qing Zhao:
> 
> > On May 18, 2023, at 12:25 PM, Martin Uecker via Gcc  wrote:
> > 
> > 
> > 
> > > On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc  
> > > wrote:
> > > > 
> > > > On Thu, May 11, 2023 at 08:53:52PM +, Joseph Myers wrote:
> > > > > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> > > > > 
> > > > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > > > > [...]
> > > > > > > > Would you allow flexible array members in unions?  Is there any
> > > > > > > > strong reason to disallow them?
> > > > > > 
> > > > > > Yes please!! And alone in a struct, too.
> > > > > > 
> > > > > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > > > > (especially since they _can_ be constructed with some fancy tricks,
> > > > > > and they behave as expected.) My understanding is that it's 
> > > > > > disallowed
> > > > > > due to an overly strict reading of the very terse language that 
> > > > > > created
> > > > > > flexible arrays in C99.
> > > > > 
> > > > > Standard C has no such thing as a zero-size object or type, which 
> > > > > would
> > > > > lead to problems with a struct or union that only contains a flexible
> > > > > array member there.
> > 
> > (I think it is fundamentally not too problematic to have zero-size
> > objects, although it would take some work to specify the semantics
> > exactly.)
> > 
> > But my preference would be to make structs / unions with FAM an
> > incomplete type which would then restrict their use (for the cases
> > now supported we would need backwards compatible exceptions).
> > We could then allow such a struct / union as the last member
> > of another struct / union which would make this an incomplete
> > type too.
> 
> Yes, I like this approach. 
> And we can make them GCC extensions first,  promote to C standard later.
> 
> My proposed patch sets (originally targeted on GCC13, now might need to 
> target on GCC14) will
> make one part of the above a GCC extension:
> Allowing the struct with FAM as the last member of another struct. (See 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101832)
> 
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614794.html
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614793.html
> https://gcc.gnu.org/pipermail/gcc-patches/2023-March/614790.html
> 
> I’d like these changes going into GCC first to improve this area.

Seems reasonable to me. Thanks!

Martin


> 
> > 
> > We then would need a special macro (based on a builtin) instead
> > of sizeof to get the size, but this would be safer anyway.
> > 
> > In principle, an even better solution would be to allow dynamic
> > arrays because then it has a dynamic bound where the type with
> > the bound could propagate to some user. Bounds checking would
> > work as expected and more cases.
> > 
> > struct foo {
> >  int len;
> >  char buf[.len];
> > };
> > 
> > But this takes a bit more work to get right.
> > 
> > > > 
> > > > Ah-ha, okay. That root cause makes sense now.
> > > 
> > > Hmm. but then the workaround
> > > 
> > > struct X {
> > >  int n;
> > >  union u {
> > >  char at_least_size_one;
> > >  int iarr[];
> > >  short sarr[];
> > >  };
> > > };
> > > 
> > > doesn't work either.  We could make that a GNU extension without
> > > adverse effects?
> > 
> > I think we could allow this even without the "at_least_size_one"
> > without a problem when allowing the use of such unions only as
> > a last member of some structure. Allowing it elsewhere seems
> > questionable anyway.
> 
> Yes,  Such an union can be treated as an flexible array member 
> (just multiple flexible arrays sharing the same storage).  Therefore it’s 
> reasonable
> To only allow it as the last field of a structure. 
> 
> thanks.
> 
> Qing.
> 
> > 
> > > Richard.
> > > 
> > > > Why are zero-sized objects missing in Standard C? Or, perhaps, the 
> > > > better
> > > > question is: what's needed to support the idea of a zero-sized object?
> > 
> > Probably a lot of convincing that it actually does not cause problems,
> > and is useful. Also a lot of work in making sure the standard is revised
> > everywhere where it is necessary. I think zero sized objects and
> > especially arrays are very useful also to avoid special code for corner
> > cases in numerical algorithms. But I think here some restrictions on
> > the use of the FAM will do.
> > 
> > 
> > Martin
> 

Re: [wish] Flexible array members in unions

[Martin Uecker via Gcc]

> On Thu, May 11, 2023 at 11:14 PM Kees Cook via Gcc  wrote:
> >
> > On Thu, May 11, 2023 at 08:53:52PM +, Joseph Myers wrote:
> > > On Thu, 11 May 2023, Kees Cook via Gcc wrote:
> > >
> > > > On Thu, May 11, 2023 at 06:29:10PM +0200, Alejandro Colomar wrote:
> > > > > On 5/11/23 18:07, Alejandro Colomar wrote:
> > > > > [...]
> > > > > > Would you allow flexible array members in unions?  Is there any
> > > > > > strong reason to disallow them?
> > > >
> > > > Yes please!! And alone in a struct, too.
> > > >
> > > > AFAICT, there is no mechanical/architectural reason to disallow them
> > > > (especially since they _can_ be constructed with some fancy tricks,
> > > > and they behave as expected.) My understanding is that it's disallowed
> > > > due to an overly strict reading of the very terse language that created
> > > > flexible arrays in C99.
> > >
> > > Standard C has no such thing as a zero-size object or type, which would
> > > lead to problems with a struct or union that only contains a flexible
> > > array member there.

(I think it is fundamentally not too problematic to have zero-size
objects, although it would take some work to specify the semantics
exactly.)

But my preference would be to make structs / unions with FAM an
incomplete type which would then restrict their use (for the cases
now supported we would need backwards compatible exceptions).
We could then allow such a struct / union as the last member
of another struct / union which would make this an incomplete
type too.

We then would need a special macro (based on a builtin) instead
of sizeof to get the size, but this would be safer anyway.

In principle, an even better solution would be to allow dynamic
arrays because then it has a dynamic bound where the type with
the bound could propagate to some user. Bounds checking would
work as expected and more cases.

struct foo {
  int len;
  char buf[.len];
};

But this takes a bit more work to get right.

> >
> > Ah-ha, okay. That root cause makes sense now.
> 
> Hmm. but then the workaround
> 
> struct X {
>   int n;
>   union u {
>   char at_least_size_one;
>   int iarr[];
>   short sarr[];
>   };
> };
> 
> doesn't work either.  We could make that a GNU extension without
> adverse effects?

I think we could allow this even without the "at_least_size_one"
without a problem when allowing the use of such unions only as
a last member of some structure. Allowing it elsewhere seems
questionable anyway.

> Richard.
> 
> > Why are zero-sized objects missing in Standard C? Or, perhaps, the better
> > question is: what's needed to support the idea of a zero-sized object?

Probably a lot of convincing that it actually does not cause problems,
and is useful. Also a lot of work in making sure the standard is revised
everywhere where it is necessary. I think zero sized objects and
especially arrays are very useful also to avoid special code for corner
cases in numerical algorithms. But I think here some restrictions on
the use of the FAM will do.


Martin


 

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Freitag, dem 24.02.2023 um 10:01 -0600 schrieb Serge E. Hallyn:
> On Fri, Feb 24, 2023 at 09:36:45AM +0100, Martin Uecker wrote:
> > Am Donnerstag, dem 23.02.2023 um 19:21 -0600 schrieb Serge E. Hallyn:

...
> > 
> > Yes, but one comment about terminology:. The C standard
> > differentiates between the representation, i.e. the bytes on
> > the stack, and the value.  The representation is converted to
> > a value during lvalue conversion.  For an invalid pointer
> > the representation is indeterminate because it now does not
> > point to a valid object anymore.  So it is not possible to
> > convert the representation to a value during lvalue conversion.
> > In other words, it does not make sense to speak of the value
> > of the pointer anymore.
> 
> I'm sure there are, especially from an implementer's point of view,
> great reasons for this.
> 
> However, as just a user, the "value" of 'void *p' should absolutely
> not be tied to whatever is at that address.

Think about it in this way: The set of possible values for a pointer
is the set of objects that exist at a point in time. If one object
disappears, a pointer can not point to it anymore. So it is not that
the pointer changes, but the set of valid values.

>   I'm given a simple
> linear memory space, under which sits an entirely different view
> obfuscated by page tables, but that doesn't concern me.  if I say
> void *p = -1, then if I print p, then I expect to see that value.

If you store an integer into a pointer (you need a cast), then
this is implementation-defined and may also produce an invalid
pointer.

> 
> Since I'm complaining about standards I'm picking and choosing here,
> but I'll still point at the printf(3) manpage :)  :
> 
>    p  The  void * pointer argument is printed in hexadecimal (as if 
> by %#x
>   or %#lx).

This is valid if the pointer is valid, but if the pointer
is invalid, this is undefined behavior.

In C one not think about pointers as addresses. They
are abstract handles that point to objects, and compilers
do exploit this for optimization.

If you need an address, you can cast it to uintptr_t
(but see below).

> 
> > > I realize C11 appears to have changed that.  I fear that in doing so it
> > > actually risks increasing the confusion about pointers.  IMO it's much
> > > easier to reason about
> > > 
> > >   o = realloc(p, X);
> > > 
> > > (and more baroque constructions) when keeping in mind that o, p, and the
> > > object pointed to by either one are all different things.
> > > 
> > 
> > What did change in C11? As far as I know, the pointer model
> > did not change in C11.
> 
> I haven't looked in more detail, and don't really plan to, but my
> understanding is that the text of:
> 
>   The lifetime of an object is the portion of program execution during which 
> storage is
>   guaranteed to be reserved for it. An object exists, has a constant address, 
> and retains
>   its last-stored value throughout its lifetime. If an object is referred to 
> outside of its
>   lifetime, the behavior is undefined. The value of a pointer becomes 
> indeterminate when
>   the object it points to (or just past) reaches the end of its lifetime.
> 
> (especially the last sentence) was new.

This is not new.

C99 "The value of a pointer becomes indeterminate when
the object it points to reaches the end of its lifetime."

C90: "The value of a pointer that referred to an object
with automatic storage duration that is no longer
guaranteed to be reserved is indeterminate."

and

"The value of a pointer that refers to freed space is
indeterminate."

> Maybe the words "value of a pointer" don't mean what I think they
> mean.  But that's the phrase to which I object.  The n bytes on
> the stack, p, are not changed just because something happened with
> the accounting for the memory at the address represented by that
> value.  If they do, then that's not 'C' any more.

It is not about the bytes of the pointer changing. But if
the object is freed they do not represent a valid pointer
anymore.  There were CPUs that trapped when an invalid
address is loaded, e.g. because the data segment for the
object was removed from the segment tables. So this is a 
rule in portable 'C'  for more than 30 years.

Nowadays compilers exploit the knowledge that the
object is freed. So you can not reliably use such
a pointer. If you do this, your code will be broken on
most modern compilers.


> 
> > > > > Reading an uninitialized value of automatic storage whose
> > > > > address was not taken is undefined behavior, so everything
> > > > > is possible afterwards.
> > > > > 
> > > > > An uninitialized variable whose address was taken has a
> > > > > representation which can represent an unspecified value
> > > > > or a no-value (trap) representation. Reading the
> > > > > representation itself is always ok and gives consistent
> > > > > results. Reading the variable can be undefined behavior
> > > > > iff it is a trap representation, otherwise

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Freitag, dem 24.02.2023 um 03:01 + schrieb Peter Lafreniere:

...
> 
> > Maybe it could do an exception for printing, that is, reading a pointer
> > is not a problem in itself, a long as you don't compare it, but I'm not
> > such an expert about this.
> 
> One last thought: with the above strict interpretation of the c standard,
> it would become nigh on impossible to implement the malloc(3) family of
> functions in c themselves. I vote for the "shared storage" interpretation
> of the c11 standard that is actually implemented rather than this abstract
> liveness oriented interpretation.

This is a bit of a misunderstanding about what "undefined behavior" means
in ISO C. It simply means that ISO C does not specify the behavior.  This
does not mean it is illegal to do something which has undefined behavior.

Instead, it means you can not rely on the ISO C standard for portable
behavior.  So if you implement "malloc" in C itself you will probably
rely on "undefined behavior", but this is perfectly fine.  The C standard
specifies behavior of "malloc", but does not care how it is implemented.


Martin

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Freitag, dem 24.02.2023 um 02:42 +0100 schrieb Alex Colomar:
> Hi Serge, Martin,
> 
> On 2/24/23 02:21, Serge E. Hallyn wrote:
> > > Does all this imply that the following is well defined behavior (and shall
> > > print what one would expect)?
> > > 
> > >    free(p);
> > > 
> > >    (void) &p;  // take the address
> > >    // or maybe we should (void) memcmp(&p, &p, sizeof(p)); ?
> > > 
> > >    printf("%p\n", p);  // we took previously its address,
> > >    // so now it has to hold consistently
> > >    // the previous value
> > > 
> > > 
> > > This feels weird.  And a bit of a Schroedinger's pointer.  I'm not 
> > > entirely
> > > convinced, but might be.
> > 
> > Again, p is just an n byte variable which happens to have (one hopes)
> > pointed at a previously malloc'd address.
> > 
> > And I'd argue that pre-C11, this was not confusing, and would not have
> > felt weird to you.
> > 
> > But I am most grateful to you for having brought this to my attention.
> > I may not agree with it and not like it, but it's right there in the
> > spec, so time for me to adjust :)
> 
> I'll try to show why this feels weird to me (even in C89):
> 
> 
> alx@dell7760:~/tmp$ cat pointers.c
> #include 
> #include 
> 
> 
> int
> main(void)
> {
>   char  *p, *q;
> 
>   p = malloc(42);
>   if (p == NULL)
>   exit(1);
> 
>   q = realloc(p, 42);
>   if (q == NULL)
>   exit(1);
> 
>   (void) &p;  // If we remove this, we get -Wuse-after-free
> 
>   printf("(%p == %p) = %i\n", p, q, (p == q));
> }
> alx@dell7760:~/tmp$ cc -Wall -Wextra pointers.c  -Wuse-after-free=3
> alx@dell7760:~/tmp$ ./a.out
> (0x5642cd9022a0 == 0x5642cd9022a0) = 1
> 

No, you can't do the comparison or use the value of 'p'
because 'p' is not a valid pointer. (The address taken
makes no difference here, but it may confuse the
compiler so that it does not warn.)

> 
> This pointers point to different objects (actually, one of them doesn't 
> even point to an object anymore), so they can't compare equal, according 
> to both:
> 
> 
> 
> 
> 
> (I believe C89 already had the concept of lifetime well defined as it is 
> now, so the object had finished it's lifetime after realloc(3)).
> 
> How can we justify that true, if the pointer don't point to the same 
> object?  And how can we justify a hypothetical false (which compilers 
> don't implement), if compilers will really just read the value?  To 
> implement this as well defined behavior, it could result in no other 
> than false, and it would require heavy overhead for the compilers to 
> detect that the seemingly-equal values are indeed different, don't you 
> think?  The easiest solution is for the standard to just declare this 
> outlaw, IMO.

This is undefined behavior, so the comparison can return false
or true or crash or whatever.  

Martin

> 
> Maybe it could do an exception for printing, that is, reading a pointer 
> is not a problem in itself, a long as you don't compare it, but I'm not 
> such an expert about this.
> 
> Cheers,
> 
> Alex
> 
> > 
> > -serge
> 
> -- 
> 
> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
> 

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Donnerstag, dem 23.02.2023 um 19:21 -0600 schrieb Serge E. Hallyn:
> On Fri, Feb 24, 2023 at 01:02:54AM +0100, Alex Colomar wrote:
> > Hi Martin,
> > 
> > On 2/23/23 20:57, Martin Uecker wrote:
> > > Am Donnerstag, dem 23.02.2023 um 20:23 +0100 schrieb Alex Colomar:
> > > > Hi Martin,
> > > > 
> > > > On 2/17/23 14:48, Martin Uecker wrote:
> > > > > > This new wording doesn't even allow one to use memcmp(3);
> > > > > > just reading the pointer value, however you do it, is UB.
> > > > > 
> > > > > memcmp would not use the pointer value but work
> > > > > on the representation bytes and is still allowed.
> > > > 
> > > > Hmm, interesting.  It's rather unspecified behavior. Still
> > > > unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or
> > > > false randomly; the compiler may compile out the call to memcmp(3),
> > > > since it knows it won't produce any observable behavior.
> > > > 
> > > > 
> > > 
> > > No, I think several things get mixed up here.
> > > 
> > > The representation of a pointer that becomes invalid
> > > does not change.
> > > 
> > > So (0 === memcmp(&p, &p, sizeof(p)) always
> > > evaluates to true.
> > > 
> > > Also in general, an unspecified value is simply unspecified
> > > but does not change anymore.
> 
> Right.  p is its own thing - n bytes on the stack containing some value.
> Once it comes into scope, it doesn't change on its own.  And if I do
> free(p) or o = realloc(p), then the value of p itself - the n bytes on
> the stack - does not change.

Yes, but one comment about terminology:. The C standard
differentiates between the representation, i.e. the bytes on
the stack, and the value.  The representation is converted to
a value during lvalue conversion.  For an invalid pointer
the representation is indeterminate because it now does not
point to a valid object anymore.  So it is not possible to
convert the representation to a value during lvalue conversion.
In other words, it does not make sense to speak of the value
of the pointer anymore.

> I realize C11 appears to have changed that.  I fear that in doing so it
> actually risks increasing the confusion about pointers.  IMO it's much
> easier to reason about
> 
>   o = realloc(p, X);
> 
> (and more baroque constructions) when keeping in mind that o, p, and the
> object pointed to by either one are all different things.
> 

What did change in C11? As far as I know, the pointer model
did not change in C11.

> > > Reading an uninitialized value of automatic storage whose
> > > address was not taken is undefined behavior, so everything
> > > is possible afterwards.
> > > 
> > > An uninitialized variable whose address was taken has a
> > > representation which can represent an unspecified value
> > > or a no-value (trap) representation. Reading the
> > > representation itself is always ok and gives consistent
> > > results. Reading the variable can be undefined behavior
> > > iff it is a trap representation, otherwise you get
> > > the unspecified value which is stored there.
> > > 
> > > At least this is my reading of the C standard. Compilers
> > > are not full conformant.
> > 
> > Does all this imply that the following is well defined behavior (and shall
> > print what one would expect)?
> > 
> >   free(p);
> > 
> >   (void) &p;  // take the address
> >   // or maybe we should (void) memcmp(&p, &p, sizeof(p)); ?
> > 
> >   printf("%p\n", p);  // we took previously its address,
> >   // so now it has to hold consistently
> >   // the previous value
> > 
> > 

No, the printf is not well defined, because the lvalue conversion
of the pointer with indeterminate representation may lead to
undefined behavior.


Martin


> > This feels weird.  And a bit of a Schroedinger's pointer.  I'm not entirely
> > convinced, but might be.
> 
> Again, p is just an n byte variable which happens to have (one hopes)
> pointed at a previously malloc'd address.
> 
> And I'd argue that pre-C11, this was not confusing, and would not have
> felt weird to you.
> 
> But I am most grateful to you for having brought this to my attention.
> I may not agree with it and not like it, but it's right there in the
> spec, so time for me to adjust :)
> 

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Donnerstag, dem 23.02.2023 um 20:23 +0100 schrieb Alex Colomar:
> Hi Martin,
> 
> On 2/17/23 14:48, Martin Uecker wrote:
> > > This new wording doesn't even allow one to use memcmp(3);
> > > just reading the pointer value, however you do it, is UB.
> > 
> > memcmp would not use the pointer value but work
> > on the representation bytes and is still allowed.
> 
> Hmm, interesting.  It's rather unspecified behavior. Still 
> unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or 
> false randomly; the compiler may compile out the call to memcmp(3), 
> since it knows it won't produce any observable behavior.
> 
> 

No, I think several things get mixed up here.

The representation of a pointer that becomes invalid
does not change.  

So (0 === memcmp(&p, &p, sizeof(p)) always
evaluates to true.

Also in general, an unspecified value is simply unspecified
but does not change anymore.

Reading an uninitialized value of automatic storage whose
address was not taken is undefined behavior, so everything
is possible afterwards.

An uninitialized variable whose address was taken has a
representation which can represent an unspecified value
or a no-value (trap) representation. Reading the 
representation itself is always ok and gives consistent
results. Reading the variable can be undefined behavior
iff it is a trap representation, otherwise you get
the unspecified value which is stored there.

At least this is my reading of the C standard. Compilers
are not full conformant.


Martin

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Freitag, dem 17.02.2023 um 12:35 +0100 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On 2/17/23 09:12, Martin Uecker wrote:
> > Am Freitag, dem 17.02.2023 um 02:04 +0100 schrieb Alejandro Colomar
> > 
> > 
> > > 
> > > [...]
> > > 
> > > > 
> > > > I'm not convinced that it's useful to the end-user to warn about
> > > > the
> > > > "use of q itself" case.
> > > 
> > > I didn't quote the standard because I couldn't find it.  I was
> > > searching in C11,
> > > and it seems that it was only implicitly Undefined Behavior, without
> > > explicit
> > > spelling (the value of the pointer was indeterminate, according to
> > > C11).
> > 
> > The value becomes indeterminate according to 6.2.4p2 in C11.
> > An indeterminate value may be a trap representation 3.19.2
> > If such a trap representation is read (the pointer, not
> > just the pointed-to object), the behavior is undefined
> > according to 6.2.6.1p5. So it is explitely UB and was
> > already in C99 or even before.
> 
> What if the comparison is performed as uintptr_t?

According to the standard, it is allowed to convert a valid
pointer to uintptr_t and then compare it after the object
is freed.  Unfortunately, many compilers will miscompile
such comparisons.  This does not only affect C/C++ but
also happens in "safe" Rust:
https://godbolt.org/z/e53b5a53q

But conversion to uintptr_t after the object is freed
is undefined behavior.


> You wouldn't have trap representations, would you?
> Or we could even go to memcmp(3) to compare as char,
> if paranoic enough :)

We renamed "trap representations" to 
"non-value representation" in C2X because people tend
to misunderstand this term.  It does not necessarily
imply a trap. It simply means that the object
representation does not correspond to a valid value
of the type.

A pointer that is not NULL but also does not
point to an object would have a trap representation
(now: "non-value representation").

A boolean that stores a representation that does
not correspond to true or false is also a non-value 
representation.  In such cases already a load 
(lvalue conversion) of such a value causes
undefined behavior.

Existing (correct) compiler optimizations can break
code which does comparisons of pointers to dead
objects.  So it is not really safe to use such
pointers even if the architecture does not trap
on load of such pointers.

For pointers, there is also the subtle issue that the
representation might be the same in memory as a new
valid pointer to a new object, but it is still
considered a trap representation due to provenance.

> > 
> > 
> > > Now C23 will better clarify that reading such a pointer value (not
> > > ever dereferencing) is Undefined Behavior.
> > 
> > We did not change this for C23.
> 
> C11:
> 
> The value of a pointer becomes indeterminate when
> the object it points to (or just past)
> reaches the end of its lifetime.
> 
> 
> C2x (N3054 is the latest I know):
> 
> If a pointer value is used in an evaluation after
> the object the pointer points to (or just past)
> reaches the end of its lifetime,
> the behavior is undefined.
> 
> 
> 
> This new wording doesn't even allow one to use memcmp(3);
> just reading the pointer value, however you do it, is UB.

memcmp would not use the pointer value but work
on the representation bytes and is still allowed.


Martin


> > 
> > Only the terminology regarding trap representation
> > (now: non-value representation) and indeterminate
> > values (now: indeterminate representation) was revised.
> > 
> > 
> > There are proposal to define bevahior for such
> > pointers, but I think this would be a mistake.
> > (although somehow I ended up being a co-author 
> > of this paper),
> > 
> > The reason is that every use of such a pointer 
> > risks running into sublte issues related to pointer 
> > provenance.
> > 
> > So in my opinion it would be very useful to warn about
> > all uses of invalid pointers, because fixing this is
> > much easier than understanding and fixing provenance
> > issues which might arise from incorrect use of such
> > pointers.
> 
> Agree; making this defined behavior doesn't seem a good idea.
> 
> Cheers,
> 
> Alex
> 
> > 
> > 
> > Martin
> 

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Freitag, dem 17.02.2023 um 02:04 +0100 schrieb Alejandro Colomar


> 
> [...]
> 
> > 
> > I'm not convinced that it's useful to the end-user to warn about
> > the
> > "use of q itself" case.
> 
> I didn't quote the standard because I couldn't find it.  I was
> searching in C11,
> and it seems that it was only implicitly Undefined Behavior, without
> explicit
> spelling (the value of the pointer was indeterminate, according to
> C11).

The value becomes indeterminate according to 6.2.4p2 in C11.
An indeterminate value may be a trap representation 3.19.2
If such a trap representation is read (the pointer, not
just the pointed-to object), the behavior is undefined
according to 6.2.6.1p5. So it is explitely UB and was
already in C99 or even before.


> Now C23 will better clarify that reading such a pointer value (not
> ever dereferencing) is Undefined Behavior.

We did not change this for C23.

Only the terminology regarding trap representation
(now: non-value representation) and indeterminate
values (now: indeterminate representation) was revised.


There are proposal to define bevahior for such
pointers, but I think this would be a mistake.
(although somehow I ended up being a co-author 
of this paper),

The reason is that every use of such a pointer 
risks running into sublte issues related to pointer 
provenance.

So in my opinion it would be very useful to warn about
all uses of invalid pointers, because fixing this is
much easier than understanding and fixing provenance
issues which might arise from incorrect use of such
pointers.


Martin



> There was a discussion in linux-man@ some years ago, which now I
> realize it
> didn't end up being applied (I thought we had applied a patch, but it
> seems we
> didn't).  I'll check if we still need such a patch (and I guess we
> do, since
> we're having this conversation).
> 
> Using the pointer is _wrong_.  And by wrong, I mean that it's
> Undefined Behavior.
> I think that alone is enough to issue a warning.  Especially, since
> the compiler
> already has that information; otherwise, it couldn't have warned
> about line 67
> of my example program.  I could understand if due to optimizations
> the compiler
> lost that information, so it couldn't warn, but in this case, there's
> no excuse.
> 
> The benefit for users?  They'll realize that the code they wrote is
> bad.  Not even
> suspicious, as some warnings warn about suspicious code.  This case
> is
> uncontroversially wrong.  That code has no valid reason to be written
> that way,
> under ISO C.
> 
> Cheers,
> 
> Alex
> 
> > 
> > Dave
> 
> -- 
> 
> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Dienstag, dem 29.11.2022 um 16:53 + schrieb Jonathan Wakely:
> On Tue, 29 Nov 2022 at 16:49, Joseph Myers wrote:
> > 
> > On Tue, 29 Nov 2022, Michael Matz via Gcc wrote:
> > 
> > > like.  But I'm generally doubtful of this whole feature within C
> > > itself.
> > > It serves a purpose in documentation, so in man-pages it seems
> > > fine enough
> > > (but then still could use a different puncuator to not be
> > > confusable with
> > > C syntax).
> > 
> > In man-pages you don't need to invent syntax at all.  You can write
> > 
> > int f(char buf[n], int n);
> > 
> > and in the context of a man page it will be clear to readers what
> > is
> > meant,
> 
> Considerably more clear than new invented syntax IMHO.

True, but I think it would be a mistake to use code in
man pages which then does not work as expected (or even
is subtle wrong) in actual code.

Martin

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Sonntag, den 13.11.2022, 16:15 +0100 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On 11/13/22 15:58, Martin Uecker wrote:
> > Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:
> > > On 11/13/22 14:33, Alejandro Colomar wrote:
> > > > On 11/13/22 14:19, Alejandro Colomar wrote:
> > > > > > But there are not only syntactical problems, because
> > > > > > also the type of the parameter might become relevant
> > > > > > and then you can get circular dependencies:
> > > > > > 
> > > > > > void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);
> > > > > 
> > > > > This seems to be a difficult stone in the road.
> > 
> > But note that GNU forward declarations solve this nicely.
> 
> How would that above be solved with GNU fwd decl?  I'm guessing that it 
> can't. 
> How do you forward declare incomplete VMTs?.

You can't express it. This was my point: it is impossible
to create circular dependencies.

...

> > > > > {
> > > > >   for (/* void */; dst <= end; dst++) {
> > > > >   *dst = *src++;
> > > > >   if (*dst == '\0')
> > > > >   return dst;
> > > > >   }
> > > > >   /* Truncation detected */
> > > > >   *end = '\0';
> > > > > 
> > > > > #if !defined(NDEBUG)
> > > > >   /* Consume the rest of the input string. */
> > > > >   while (*src++) {};
> > > > > #endif
> > > > > 
> > > > >   return end + 1;
> > > > > }
> > > > And I forgot to say it:  Default promotions rank high (probably the 
> > > > highest) in
> > > > my list of most hated features^Wbugs in C.
> > 
> > If you replaced them with explicit conversion you then have
> > to add by hand all the time, I am pretty sure most people
> > would hate this more. (and it could also hide bugs)
> 
> Yeah, casts are also in my top 3 list of things to avoid (although in this 
> case 
> there's no bug); maybe a bit over default promotions :)
> 
> I didn't mean that all promotions are bad.  Just the gratuitous ones, like 
> promoting everything to int before even needing it.  That makes uint16_t a 
> theoretical type, because whenever you try to use it, you end up with a 
> signed 
> 32-bit type; fun heh? :P  _BitInt() solves that for me.

uint16_t is for storing data.  My expectation is that people
will find _BitInt() difficult and error-prone to use for
small sizes.  But maybe I am wrong...

> But sure, in (1u + 1l), promotions are fine to get a common type.
> 
> > > > I wouldn't convert it to size_t, but
> > > > rather follow normal promotion rules.
> > 
> > The point of making it size_t is that you then
> > do need to know the type of the parameter to make
> > sense of the expression. If the type matters, then you get
> > mutual dependencies as in the example above.
> 
> Except if you treat incomplete types as... incomplete types (for which 
> sizeof() 
> is disallowed by the standard).  And the issue we're having is that the types 
> are not yet complete at the time we're using them, aren't they?

It is not an incomplete type. When doing parsing and do not have
a declaration we know nothing about it (not just not the size).
If we assume we know the type (by looking ahead) we get mutual
dependencies.

Also the capability to parse, fold, and do type checking
in one go is something worth preserving in my opinion. 

Martin


> Kind of like the initialization order fiasco, but since we're in a limited 
> scope, we can detect it.

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Sonntag, den 13.11.2022, 15:02 +0100 schrieb Alejandro Colomar:
> 
> On 11/13/22 14:33, Alejandro Colomar wrote:
> > Hi Martin,
> > 
> > On 11/13/22 14:19, Alejandro Colomar wrote:
> > > > But there are not only syntactical problems, because
> > > > also the type of the parameter might become relevant
> > > > and then you can get circular dependencies:
> > > > 
> > > > void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);
> > > 
> > > This seems to be a difficult stone in the road.

But note that GNU forward declarations solve this nicely.

> > > 
> > > > I am not sure what would the best way to fix it. One
> > > > could specifiy that parameters referred to by
> > > > the .identifer syntax must of some integer type and
> > > > that the sub-expression .identifer is always
> > > > converted to a 'size_t'.
> > > 
> > > That makes sense, but then overnight some quite useful thing came to my 
> > > mind 
> > > that would not be possible with this limitation:
> > > 
> > > 
> > > 
> > > 
> > > char *
> > > stpecpy(char dst[.end - .dst], char *src, char end[1])
> 
> Heh, I got an off-by-one error.  It should be dst[.end - .dst + 1], of 
> course, 
> and then the result of the whole expression would be 0, which is fine as 
> size_t.
> 
> So, never mind.

.end and .dst would have pointer size though.

> > > {
> > >  for (/* void */; dst <= end; dst++) {
> > >  *dst = *src++;
> > >  if (*dst == '\0')
> > >  return dst;
> > >  }
> > >  /* Truncation detected */
> > >  *end = '\0';
> > > 
> > > #if !defined(NDEBUG)
> > >  /* Consume the rest of the input string. */
> > >  while (*src++) {};
> > > #endif
> > > 
> > >  return end + 1;
> > > }
> > And I forgot to say it:  Default promotions rank high (probably the 
> > highest) in 
> > my list of most hated features^Wbugs in C. 

If you replaced them with explicit conversion you then have
to add by hand all the time, I am pretty sure most people
would hate this more. (and it could also hide bugs)

> > I wouldn't convert it to size_t, but 
> > rather follow normal promotion rules.

The point of making it size_t is that you then
do need to know the type of the parameter to make
sense of the expression. If the type matters, then you get
mutual dependencies as in the example above. 

> > Since you can use anything between INTMAX_MIN and UINTMAX_MAX for accessing 
> > an 
> > array (which took me some time to understand), I'd also allow the same 
> > here. So, 
> > the type of the expression between [] could perfectly be signed or unsigned.
> > 
> > So, you could use size_t for very high indices, or e.g. ptrdiff_t if you 
> > want to 
> > allow negative numbers.  In the function above, since dst can be a pointer 
> > to 
> > one-past-the-end (it represents a previous truncation; that's why the test 
> > dst<=end), forcing a size_t conversion would disallow that syntax.

Yes, this then does not work.

Martin


> > Cheers,
> > 
> > Alex
> > 
> 
> -- 
> 

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Samstag, den 12.11.2022, 14:54 + schrieb Joseph Myers:
> On Sat, 12 Nov 2022, Alejandro Colomar via Gcc wrote:
> 
> > Since it's to be used as an rvalue, not as a lvalue, I guess a
> > postfix-expression wouldn't be the right one.
> 
> Several forms of postfix-expression are only rvalues.
> 
> > > (with a special rule about how the identifier is interpreted, different
> > > from the normal scope rules)?  If so, then ".a = 1" could either match
> > > assignment-expression directly (assigning to the postfix-expression ".a").
> > 
> > No, assigning to a function parameter from within another parameter
> > declaration wouldn't make sense.  They should be readonly.  Side effects
> > should be forbidden, I think.
> 
> Such assignments are already allowed.  In a function definition, the side 
> effects (including in size expressions for array parameters adjusted to 
> pointers) take place before entry to the function body.
> 
> And, in any case, if you did have a constraint disallowing such 
> assignments, it wouldn't suffice for syntactic disambiguation (see the 
> previous point I made about that; I have some rough notes towards a WG14 
> paper on syntactic disambiguation, but haven't converted them into a 
> coherent paper).

My idea was to only allow

array-declarator : direct-declarator [ . identifier ]

and only for parameter (not nested inside structs declared
in parameter list) as a first step because it seems this 
would exclude all difficult cases.

But if we need to allow more complicated expressions, then
it starts getting more complicated.

One could could allow more generic expressions, and
specify that the .identifier refers to a
parameter in
the nearest lexically enclosing parameter list or
struct/union.

Then

void foo(struct bar { int x; char c[.x] } a, int x);

would not be allowed (which is good because then we
could later use the syntax also inside structs). If
we apply scoping rules, the following would work:

struct bar { int y; };
void foo(char p[((struct bar){ .y = .x }).y], int x);

But not:

struct bar { int y; };
void foo(char p[((struct bar){ .y = .y }).y], int y);


But there are not only syntactical problems, because
also the type of the parameter might become relevant
and then you can get circular dependencies:

void foo(char (*a)[sizeof *.b], char (*b)[sizeof *.a]);

I am not sure what would the best way to fix it. One
could specifiy that parameters referred to by 
the .identifer syntax must of some integer type and
that the sub-expression .identifer is always
converted to a 'size_t'. 

Maybe one should also add a constraint that all new
type length expressions, i.e. using the syntax,
can not have side effects. Or even that they follow
all the rules of integer constant expressions with
the fictitious assumption that all . identifer 
sub-expressions are integer constant expressions.
The rationale being that this would facilitate
compile time reasoning about length expressions.
 

Martin

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Samstag, den 12.11.2022, 01:09 + schrieb Joseph Myers:
> On Fri, 11 Nov 2022, Martin Uecker via Gcc wrote:
> 
> > > Even a compiler extension requires the level of detail of specification 
> > > that you get with a WG14 paper (and the level of work on finding bugs in 
> > > that specification), to avoid the problem we've had before with too many 
> > > features added in GCC 2.x days where a poorly defined feature is 
> > > "whatever 
> > > the compiler accepts".
> > 
> > I think the effort needed to specify the feature correctly
> > can be minimized by making the first version of the feature
> > as simple as possible.  
> 
> The version of constexpr in the current C2x working draft is more or less 
> as simple as possible.  It also went through lots of revisions to get 
> there.  I'm currently testing an implementation of C2x constexpr for GCC 
> 13, and there are still several issues with the specification I found in 
> the implementation process, beyond those raised in WG14 discussions, for 
> which I'll need to raise NB comments to clarify things.

constexpr had no implementation experience in C at all and
always suspected that C++ experience should somehow count is
not really justified.  

> I think that illustrates that you need the several iterations on the 
> specification process, *and* making it as simple as possible, *and* 
> getting implementation experience, *and* the implementation experience 
> being with a close eye to what it implies for all the details in the 
> specification rather than just getting something vaguely functional but 
> not clearly specified.

I agree. We should work on specification and on prototyping
new features in parallel.

Martin

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Donnerstag, den 10.11.2022, 23:19 + schrieb Joseph Myers:
> On Thu, 10 Nov 2022, Martin Uecker via Gcc wrote:
> 
> > One problem with WG14 papers is that people put in too much,
> > because the overhead is so high and the standard is not updated
> > very often.  It would be better to build such feature more
> > incrementally, which could be done more easily with a compiler
> > extension.  One could start supporting just [.x] but not more
> > complicated expressions.
> 
> Even a compiler extension requires the level of detail of specification 
> that you get with a WG14 paper (and the level of work on finding bugs in 
> that specification), to avoid the problem we've had before with too many 
> features added in GCC 2.x days where a poorly defined feature is "whatever 
> the compiler accepts".

I think the effort needed to specify the feature correctly
can be minimized by making the first version of the feature
as simple as possible.  

> If you use .x as the notation but don't limit it to [.x], you have a 
> completely new ambiguity between ordinary identifiers and member names
> 
> struct s { int a; };
> void f(int a, int b[((struct s) { .a = 1 }).a]);
> 
> where it's newly ambiguous whether ".a = 1" is an assignment to the 
> expression ".a" or a use of a designated initializer.

If we only allowed [ . a ] then this example would not be allowed.

If need more flexibility, we could incrementally extend it.

> (I think that if you add any syntax for this, GNU VLA forward declarations 
> are clearly to be preferred to inventing something new like [.x] which 
> introduces its own problems.)

I also prefer this.

I proposed forward declarations but WG14 and also people in this
discussion did not like them.  If we would actually start using
them, we could propose them again for the next revision.

Martin

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Donnerstag, den 10.11.2022, 01:39 + schrieb Joseph Myers:
> On Thu, 10 Nov 2022, Joseph Myers wrote:
> 
> > On Thu, 10 Nov 2022, Alejandro Colomar via Gcc wrote:
> > 
> > > I've shown the three kinds of prototypes that have been changed:
> > > 
> > > -  Normal VLA; nothing fancy except for the '.'.
> > > -  Complex size expressions.
> > > -  'void *' VLAs (assuming GNU conventions: sizeof(void *)==1).
> > 
> > That doesn't cover any of the tricky issues with such proposals, such as 
> > the choice of which entity is referred to by the parameter name when there 
> > are multiple nested parameter lists that use the same parameter name, or 
> > when the identifier is visible from an outer scope (including in 
> > particular the case where it's declared as a typedef name in an outer 
> > scope).
> 
> In fact I can't tell from these examples whether you mean for a '.' token 
> after '[' to have special semantics, or whether you mean to have a special 
> '. identifier' form of expression valid in certain context (each of which 
> introduces its own complications; for the former, typedef names from outer 
> scopes are problematic; for the latter, it's designated initializers where 
> you get complications, for example).  Designing new syntax that doesn't 
> cause ambiguity is generally tricky, and this sort of language extension 
> is the kind of thing where you'd expect to so through at least five 
> iterations of a WG14 paper before you have something like a sound 
> specification.

I am not sure what Alejandro has in mind exactly, but my idea of using
a new notation [.identifier] would be to limit it to accessing other
parameter names in the same parameter list only, so that there is 

1) no ambiguity what is referred to  and  
2) one can access parameters which come later 

If we want to specify something like this, I think we should also
restrict what kind of expressions one allows, e.g. it has to
be side-effect free.  But maybe we want to make this even more
restrictive (at least initially).

One problem with WG14 papers is that people put in too much,
because the overhead is so high and the standard is not updated
very often.  It would be better to build such feature more
incrementally, which could be done more easily with a compiler
extension.  One could start supporting just [.x] but not more
complicated expressions.

Later WG14 can still accept or reject or modify this proposal
based on the experience we get.

(I would also be happy with using GNU forward declarations, and
I am not sure why people dislike them so much.) 


Martin

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Hi Alejandro,

Am Samstag, den 03.09.2022, 16:59 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On 9/3/22 16:35, Martin Uecker wrote:
> > Am Samstag, den 03.09.2022, 15:41 +0200 schrieb Alejandro Colomar:
> > > Hi Martin,
> > > 
> > > On 9/3/22 14:47, Martin Uecker wrote:
> > > [...]
> > > 
> > > > GCC will warn if the bound is specified inconsistently between
> > > > declarations and also emit warnings if it can see that a buffer
> > > > which is passed is too small:
> > > > 
> > > > https://godbolt.org/z/PsjPG1nv7
> > > 
> > > That's very good news!
> > > 
> > > BTW, it's nice to see that GCC doesn't need 'static' for array
> > > parameters.  I never understood what the static keyword adds there.
> > > There's no way one can specify an array size an mean anything other than
> > > requiring that, for a non-null pointer, the array should have at least
> > > that size.
> > 
> >  From the C standard's point of view,
> > 
> > void foo(int n, char buf[n]);
> > 
> > is semantically equivalent to
> > 
> > void foo(int, char *buf);
> > 
> > and without 'static' the 'n' has no further meaning
> > (this is different for pointers to arrays).
> 
> I know.  I just don't understand the rationale for that decission. :/

I guess it made sense in the past, but is simply not
what we need today.

> > The static keyword implies that the pointer is be valid and
> > non-zero and that there must be at least 'n' elements
> > accessible, so in some sense it is stronger (it implies
> > alid non-zero pointers), but at the same time it does not
> > imply a bound.
> 
> That stronger meaning, I think is a mistake by the standard.
> Basically, [static n] means the same as [n] combined with [[gnu::nonnull]].
> What the standard should have done would be to keep those two things 
> separate, since one may want to declare non-null non-array pointers, or 
> possibly-null array ones.  So the standard should have standardized some 
> form of nonnull for that.  

I agree the situation is not good.  

> But the recent discussion about presenting 
> nonnull pointers as [static 1] is horrible.  But let's wait till the 
> future hopefully fixes this.

yes, [static 1] is problematic because then the number
can not be used as a bound anymore. 

My experience is that if one wants to see something fixed,
one has to push for it.  Standardization is meant
to standardize existing practice, so if we want to see
this improved, we can not wait for this.

> > But I agree that 'n' without 'static' should simply imply
> > a bound and I think we should use it this way even when
> > the standard currently does not attach a meaning to it.
> 
> Yep.
> 
> [...]
> 
> > > What about taking something from K&R functions for this?:
> > > 
> > > int foo(q; w; int a[q], int q, int s[w], int w);
> > > 
> > > By not specifying the types, the syntax is again short.
> > > This is left-to-right, so no problems with global variables, and no need
> > > for complex parsers.
> > > Also, by not specifying types, now it's more obvious to the naked eye
> > > that there's a difference:
> > 
> > I am ok with the syntax, but I am not sure how this would
> > work. If the type is determined only later you would still
> > have to change parsers (some C compilers do type
> > checking  and folding during parsing, so need the types
> > to be known during parsing) and you also still have the
> > problem with the mutual dependencies.
> 
> This syntax resembles a lot K&R syntax.  Any C compiler that supports 
> them (and I guess most compilers out there do) should be easily 
> convertible to support this syntax (at least more easily than other 
> alternatives).  But this is just a guess.

In K&R syntax this worked for definition:

void foo(y, n)
 int n;
 int y[n];
{ ...

But this worked because you could reorder the
declarations so that later declarations could
refer to previous ones.

So one could do

int foo(int n, char buf[n];  buf, n);

where the second part defines the order of
the parameter or

int foo(buf, n; int n, char buf[n]);

where the first part defins the order,
but the declarations need to have the size
first. But then you need to specify each
parameter twice...


> > We thought about using this syntax
> > 
> > int foo(char buf[.n], int n);
> > 
> > because it is new syntax which means we can restrict the
> > size to be the name of a parameter instead of allowing
> > arbitrary expressions, which then makes forward references
> > less problematic.  It is also consistent with designators in
> > initializers and could also be extend to annotate
> > flexible array members or for storing pointers to arrays
> > in structures:
> 
> It's not crazy.  I don't have much to argue against it.
> 
> > struct {
> >int n;
> >char buf[.n];
> > };
> > 
> > struct {
> >int n;
> >char (*buf)[.n];
> > };
> 
> Perhaps some doubts about how this would work for nested structures, but 
> not unreasonable.

It is not implemented though...

Martin


> Cheers,
> 
> Alex
> 
> -- 
> Alejand

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

Am Samstag, den 03.09.2022, 15:41 +0200 schrieb Alejandro Colomar:
> Hi Martin,
> 
> On 9/3/22 14:47, Martin Uecker wrote:
> [...]
> 
> > GCC will warn if the bound is specified inconsistently between
> > declarations and also emit warnings if it can see that a buffer
> > which is passed is too small:
> > 
> > https://godbolt.org/z/PsjPG1nv7
> 
> That's very good news!
> 
> BTW, it's nice to see that GCC doesn't need 'static' for array 
> parameters.  I never understood what the static keyword adds there. 
> There's no way one can specify an array size an mean anything other than 
> requiring that, for a non-null pointer, the array should have at least 
> that size.

>From the C standard's point of view,

void foo(int n, char buf[n]);

is semantically equivalent to

void foo(int, char *buf);

and without 'static' the 'n' has no further meaning
(this is different for pointers to arrays).

The static keyword implies that the pointer is be valid and
non-zero and that there must be at least 'n' elements
accessible, so in some sense it is stronger (it implies 
alid non-zero pointers), but at the same time it does not
imply a bound.

But I agree that 'n' without 'static' should simply imply
a bound and I think we should use it this way even when
the standard currently does not attach a meaning to it.

> > 
> > BTW: If you declare pointers to arrays (not first elements) you
> > can get run-time bounds checking with UBSan:
> > 
> > https://godbolt.org/z/TvMo89WfP
> 
> Couldn't that be caught at compile time?  n is certainly out of bounds 
> always for such an array, since the last element is n-1.

Yes, in this example it could (and ideally should) be
detected at compile time.

But this notation already today allows passing of a bound
across API  boundaries and thus enables run-time detection of
out-of-bound accesses even in scenarious where it could
not be found at compile time.

> > 
> > > Also, new code can be designed from the beginning so that sizes go
> > > before their corresponding arrays, so that new code won't typically be
> > > affected by the lack of this feature in the language.
> > > 
> > > This leaves us with legacy code, especially libc, which just works, and
> > > doesn't have any urgent needs to change their prototypes in this regard
> > > (they could, to improve static analysis, but not what we'd call urgent).
> > 
> > It would be useful step to find out-of-bounds problem in
> > applications using libc.
> 
> Yep, it would be very useful for that.  Not urgent, but yes, very useful.
> 
> 
> > > Let's take an example:
> > > 
> > > 
> > >  int getnameinfo(const struct sockaddr *restrict addr,
> > >  socklen_t addrlen,
> > >  char *restrict host, socklen_t hostlen,
> > >  char *restrict serv, socklen_t servlen,
> > >  int flags);
> > > 
> > > and some transformations:
> > > 
> > > 
> > >  int getnameinfo(const struct sockaddr *restrict addr,
> > >  socklen_t addrlen,
> > >  char host[restrict hostlen], socklen_t hostlen,
> > >  char serv[restrict servlen], socklen_t servlen,
> > >  int flags);
> > > 
> > > 
> > >  int getnameinfo(socklen_t hostlen;
> > >  socklen_t servlen;
> > >  const struct sockaddr *restrict addr,
> > >  socklen_t addrlen,
> > >  char host[restrict hostlen], socklen_t hostlen,
> > >  char serv[restrict servlen], socklen_t servlen,
> > >  int flags);
> > > 
> > > (I'm not sure if I used correct GNU syntax, since I never used that
> > > extension myself.)
> > > 
> > > The first transformation above is non-ambiguous, as concise as possible,
> > > and its only issue is that it might complicate the implementation a bit
> > > too much.  I don't think forward-using a parameter's size would be too
> > > much of a parsing problem for human readers.
> > 
> > I personally find the second form not terrible.  Being
> > able to read code left-to-right, top-down is helpful in more
> > complicated examples.
> > 
> > 
> > 
> > > The second one is unnecessarily long and verbose, and semicolons are not
> > > very distinguishable from commas, for human readers, which may be very
> > > confusing.
> > > 
> > >  int foo(int a; int b[a], int a);
> > >  int foo(int a, int b[a], int o);
> > > 
> > > Those two are very different to the compiler, and yet very similar to
> > > the human eye.  I don't like it.  The fact that it allows for simpler
> > > compilers isn't enough to overcome the readability issues.
> > 
> > This is true, I would probably use it with a comma and/or
> > syntax highlighting.
> > 
> > 
> > > I think I'd prefer having the forward-using syntax as a non-standard
> > > extension --or a standard but optional language feat

Re: [PATCH] Various pages: SYNOPSIS: Use VLA syntax in function parameters

[Martin Uecker via Gcc]

...
> > 
> >Whether or not you feel like the manpages are the best place to 
> > start that, I'll leave up to you!
> 
> I'll try to defend the reasons to start this in the man-pages.
> 
> This feature is mostly for documentation purposes, not being meaningful 
> for code at all (for some meaning of meaningful), since it won't change 
> the function definition in any way, nor the calls to it.  At least not 
> by itself; static analysis may get some benefits, though.


GCC will warn if the bound is specified inconsistently between
declarations and also emit warnings if it can see that a buffer
which is passed is too small:

https://godbolt.org/z/PsjPG1nv7


BTW: If you declare pointers to arrays (not first elements) you
can get run-time bounds checking with UBSan:

https://godbolt.org/z/TvMo89WfP


> 
> Also, new code can be designed from the beginning so that sizes go 
> before their corresponding arrays, so that new code won't typically be 
> affected by the lack of this feature in the language.
> 
> This leaves us with legacy code, especially libc, which just works, and 
> doesn't have any urgent needs to change their prototypes in this regard 
> (they could, to improve static analysis, but not what we'd call urgent).

It would be useful step to find out-of-bounds problem in
applications using libc.


> And since most people don't go around reading libc headers searching for 
> function declarations (especially since there are manual pages that show 
> them nicely), it's not like the documentation of the code depends on how 
> the function is _actually_ declared in code (that's why I also defended 
> documenting restrict even if glibc wouldn't have cared to declare it), 
> but it depends basically on what the manual pages say about the 
> function.  If the manual pages say a function gets 'restrict' params, it 
> means it gets 'restrict' params, no matter what the code says, and if it 
> doesn't, the function accepts overlapping pointers, at least for most of 
> the public (modulo manual page bugs, that is).
> 
> So this extension could very well be added by the manual pages, as a 
> form of documentation, and then maybe picked up by compilers that have 
> enough resources to implement it.
> 
> 
> Considering that this feature is mostly about documentation (and a bit 
> of static analysis too), the documentation should be something appealing 
> to the reader.
> 
> 
> Let's take an example:
> 
> 
> int getnameinfo(const struct sockaddr *restrict addr,
> socklen_t addrlen,
> char *restrict host, socklen_t hostlen,
> char *restrict serv, socklen_t servlen,
> int flags);
> 
> and some transformations:
> 
> 
> int getnameinfo(const struct sockaddr *restrict addr,
> socklen_t addrlen,
> char host[restrict hostlen], socklen_t hostlen,
> char serv[restrict servlen], socklen_t servlen,
> int flags);
> 
> 
> int getnameinfo(socklen_t hostlen;
> socklen_t servlen;
> const struct sockaddr *restrict addr,
> socklen_t addrlen,
> char host[restrict hostlen], socklen_t hostlen,
> char serv[restrict servlen], socklen_t servlen,
> int flags);
> 
> (I'm not sure if I used correct GNU syntax, since I never used that 
> extension myself.)
> 
> The first transformation above is non-ambiguous, as concise as possible, 
> and its only issue is that it might complicate the implementation a bit 
> too much.  I don't think forward-using a parameter's size would be too 
> much of a parsing problem for human readers.


I personally find the second form not terrible.  Being
able to read code left-to-right, top-down is helpful in more
complicated examples.



> The second one is unnecessarily long and verbose, and semicolons are not 
> very distinguishable from commas, for human readers, which may be very 
> confusing.
> 
> int foo(int a; int b[a], int a);
> int foo(int a, int b[a], int o);
> 
> Those two are very different to the compiler, and yet very similar to 
> the human eye.  I don't like it.  The fact that it allows for simpler 
> compilers isn't enough to overcome the readability issues.

This is true, I would probably use it with a comma and/or
syntax highlighting.


> I think I'd prefer having the forward-using syntax as a non-standard 
> extension --or a standard but optional language feature-- to avoid 
> forcing small compilers to implement it, rather than having the GNU 
> extension standardized in all compilers.

The problems with the second form are:

- it is not 100% backwards compatible (which maybe ok though) as
the semantics of the following code changes:

int n;
int foo(int a[n], int n); // refers to different n!

Code written for new com

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Dienstag, den 18.01.2022, 09:31 +0100 schrieb Richard Biener:
> On Mon, Jan 17, 2022 at 3:11 PM Michael Matz via Gcc  wrote:
> > Hello,
> > 
> > On Sat, 15 Jan 2022, Martin Uecker wrote:
> > 
> > > > Because it interferes with existing optimisations. An explicit
> > > > checkpoint has a clear meaning. Using every volatile access that way
> > > > will hurt performance of code that doesn't require that behaviour for
> > > > correctness.
> > > 
> > > This is why I would like to understand better what real use cases of
> > > performance sensitive code actually make use of volatile and are
> > > negatively affected. Then one could discuss the tradeoffs.
> > 
> > But you seem to ignore whatever we say in this thread.  There are now
> > multiple examples that demonstrate problems with your proposal as imagined
> > (for lack of a _concrete_ proposal with wording from you), problems that
> > don't involve volatile at all.  They all stem from the fact that you order
> > UB with respect to all side effects (because you haven't said how you want
> > to avoid such total ordering with all side effects).

Again, this is simply not what I am proposing. I don't
want to order UB with all side effects.

You are right, there is not yet a specific proposal. But
at the moment I simply wanted to understand the impact of
reordering traps and volatile.

> > As I said upthread: you need to define a concept of time at whose
> > granularity you want to limit the effects of UB, and the borders of each
> > time step can't simply be (all) the existing side effects.  Then you need
> > to have wording of what it means for UB to occur within such time step, in
> > particular if multiple UB happens within one (for best results it should
> > simply be UB, not individual instances of different UBs).
> > 
> > If you look at the C++ proposal (thanks Jonathan) I think you will find
> > that if you replace 'std::observable' with 'sequence point containing a
> > volatile access' that you basically end up with what you wanted.  The
> > crucial point being that the time steps (epochs in that proposal) aren't
> > defined by all side effects but by a specific and explicit thing only (new
> > function in the proposal, volatile accesses in an alternative).
> > 
> > FWIW: I think for a new language feature reusing volatile accesses as the
> > clock ticks are the worse choice: if you intend that feature to be used
> > for writing safer programs (a reasonable thing) I think being explicit and
> > at the same time null-overhead is better (i.e. a new internal
> > function/keyword/builtin, specified to have no effects except moving the
> > clock forward).  volatile accesses obviously already exist and hence are
> > easier to integrate into the standard, but in a given new/safe program,
> > whenever you see a volatile access you would always need to ask 'is thise
> > for clock ticks, or is it a "real" volatile access for memmap IO'.
> 
> I guess Martin want's to have accesses to volatiles handled the same as
> function calls where we do not know whether the function call will return
> or terminate the program normally.  As if the volatile access could have
> a similar effect (it might actually reboot the machine or so - but of course
> that and anything else I can imagine would be far from "normal termination
> of the program").  That's technically possible to implement with a yet unknown
> amount of work.

Yes. thanks! Semantically this is equivalent to what I want.

> Btw, I'm not sure we all agree that (*) in the following program doesn't make
> it invoke UB and thus the compiler is not free to re-order the
> offending statement
> to before the exit (0) call.  Thus UB is only "realized" if a stmt
> containing it is
> executed in the abstract machine.
> 
> int main()
> {
>exit(0);
>1 / 0;  /// (*)
> }

Yes, this not clear although there seems to be some
understanding there is a difference between 
compile-time UB and run-time UB and I think the
standard should make it clear what is what.

Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 15.01.2022, 16:38 -0500 schrieb Paul Koning:
> > On Jan 15, 2022, at 4:28 PM, Martin Sebor  wrote:
> > 
> > On 1/14/22 07:58, Paul Koning via Gcc wrote:
> > > > On Jan 14, 2022, at 9:15 AM, Michael Matz via Gcc  
> > > > wrote:
> > > > 
> > > > > ...
> > > > But right now that's equivalent to making it observable,
> > > > because we don't have any other terms than observable or
> > > > undefined.  As aluded to later you would have to
> > > > introduce a new concept, something pseudo-observable,
> > > > which you then started doing.  So, see below.
> > > I find it really hard to view this notion of doing work for UB with any 
> > > favor.  The way I see
> > > it is that a program having UB is synonymous with "defective program" and 
> > > for the compiler to
> > > do extra work for these doesn't make much sense to me.
> > 
> > This is also the official position of the C committee on record,
> > but it's one that's now being challenged.

"nonportable or erroneous" is the official position.

> > > If the issue is specifically the handling of overflow traps, perhaps a 
> > > better answer would be
> > > to argue for language changes that manage such events explicitly rather 
> > > than having them be
> > > undefined behavior.  Another (similar) option might be to choose a 
> > > language in which this is
> > > done.  (Is Ada such a language?  I don't remember.)
> > 
> > A change to the language standard is only feasible if it doesn't
> > overly constrain existing implementations. 
> 
> I was thinking that if a new feature is involved, rather than a new 
> definition of behavior for
> existing code, it wouldn't be a constraint on existing implementations (in 
> the sense of "what the
> compiler does for existing code written to the current rules").  In other 
> words, suppose there was
> a concept of "trapping operations" that could be enabled by some new 
> mechanism in the program
> text.  If you use that, then you're asking the compiler to do more work and 
> your code may get
> slower or bigger.  But if you don't, the existing rules apply and nothing bad 
> happens (other than
> that the compiler is somewhat larger and more complex due to the support for 
> both cases).

There are also different proposal for doing something like this,
e.g. making certain undefined behaviour defined as trapping
operations, either as a language variant or by default.

But this is not my idea here, I want to limit the impact of UB
on defective programs - accepting the reality that in the real
world programs often have defects and any serious field of 
engineering needs to deal with this in a better way than to
say "the ISO standard says no requirements - so you loose".

Imagine an aurospace, biomedical, mechanical, or civil engineer
saying: " It makes no sense to consider for the case where one
part fails, this is then just then a defective
airplane/CT scanner/car/bridge.  Not worth spending extra
resources on it, and a non-defective airplane might potentially
be a little bit slower if we were to give you some guarantees
in this failure case. First you need to show that this has no
performance impact at all to anybody anywhere, then maybe we
consider this." (When, at the same time there is quite
substantial damage caused by defective C programs).

I thought limiting the impact of UB on previous defined I/O
would be a rather modest step towards more reliable software,
considering that this is already the case for most I/O and
it seems only some volatile accesses would need fixing (where
I still do not see how this could affect performance anywhere
where it actually matters). 


Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 15.01.2022, 16:33 + schrieb Jonathan Wakely:
> On Sat, 15 Jan 2022, 09:00 Martin Uecker,  wrote:
> 
> > Am Freitag, den 14.01.2022, 19:54 + schrieb Jonathan Wakely:
> > > On Fri, 14 Jan 2022, 14:17 Michael Matz via Gcc, 
> > wrote:
> > > > Hello,
> > > > 
> > > > On Thu, 13 Jan 2022, Martin Uecker wrote:
> > > > 
> > > > > > > >  Handling all volatile accesses in the very same way would be
> > > > > > > > possible but quite some work I don't see much value in.
> > > > > > > 
> > > > > > > I see some value.
> > > > > > > 
> > > > > > > But an alternative could be to remove volatile
> > > > > > > from the observable behavior in the standard
> > > > > > > or make it implementation-defined whether it
> > > > > > > is observable or not.
> > > > > > 
> > > > > > But you are actually arguing for making UB be observable
> > > > > 
> > > > > No, I am arguing for UB not to have the power
> > > > > to go back in time and change previous defined
> > > > > observable behavior.
> > > > 
> > > > But right now that's equivalent to making it observable,
> > > > because we don't have any other terms than observable or
> > > > undefined.  As aluded to later you would have to
> > > > introduce a new concept, something pseudo-observable,
> > > > which you then started doing.  So, see below.
> > > > 
> > > > > > That's
> > > > > > much different from making volatile not be
> > > > > > observable anymore (which  obviously would
> > > > > > be a bad idea), and is also much harder to
> > > > > 
> > > > > I tend to agree that volatile should be
> > > > > considered observable. But volatile is
> > > > > a bit implementation-defined anyway, so this
> > > > > would be a compromise so that implementations
> > > > > do not have to make all the implied changes
> > > > > if we revise the meaning of UB.
> > > > 
> > > > Using volatile accesses for memory mapped IO is a much stronger
> > use-case
> > > > than your wish of using volatile accesses to block moving of UB as a
> > > > debugging aid, and the former absolutely needs some guarantees, so I
> > don't
> > > > think it would be a compromise at all.  Mkaing volatile not be
> > observable
> > > > would break the C language.
> > > > 
> > > > > > Well, what you _actually_ want is an implied
> > > > > > dependency between some UB and volatile accesses
> > > > > > (and _only_ those, not e.g. with other UB), and the
> > > > > > difficulty now is to define "some" and to create
> > > > > > the dependency without making that specific UB
> > > > > > to be properly observable.
> > > > > 
> > > > > Yes, this is what I actually want.
> > > > > 
> > > > > >  I think to define this
> > > > > > all rigorously seems futile (you need a new
> > > > > > category between observable  and UB), so it comes
> > > > > > down to compiler QoI on a case by case basis.
> > > > > 
> > > > > We would simply change UB to mean "arbitrary
> > > > > behavior at the point of time the erraneous
> > > > > construct is encountered at run-time"  and
> > > > > not "the complete program is invalid all
> > > > > together". I see no problem in specifying this
> > > > > (even in a formally precise way)
> > > > 
> > > > First you need to define "point in time", a concept which doesn't exist
> > > > yet in C.  The obvious choice is of course observable behaviour in the
> > > > execution environment and its specified ordering from the abstract
> > > > machine, as clarified via sequence points.  With that your "at the
> > point
> > > > in time" becomes something like "after all side effects of previous
> > > > sequence point, but strictly before all side effects of next sequence
> > > > point".
> > > > 
> > > > But doing that would have very far reaching consequences, as already
> > > > stated in this thread.  The above would basically make undefined
> > behaviour
> > > > be reliably countable, and all implementations would need to produce
> > the
> > > > same counts of UB.  That in turn disables many code movement and
> > > > commonization transformations, e.g. this:
> > > > 
> > > > int a = ..., b = ...;
> > > > int x = a + b;
> > > > int y = a + b;
> > > > 
> > > > can't be transformed into "y = x = a + b" anymore, because the addition
> > > > _might_ overflow, and if it does you have two UBs originally but would
> > > > have one UB after.  I know that you don't want to inhibit this or
> > similar
> > > > transformations, but that would be the result of making UB countable,
> > > > which is the result of forcing UB to happen at specific points in time.
> > > > So, I continue to see problems in precisely specifying what you want,
> > _but
> > > > not more_.
> > > > 
> > > > I think all models in which you order the happening of UB with respect
> > to
> > > > existing side effects (per abstract machine, so it includes
> > modification
> > > > of objects!) have this same problem, it always becomes a side effect
> > > > itself (one where you don't specify what actually happens, but a side
> > > > effect nontheles

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Freitag, den 14.01.2022, 19:54 + schrieb Jonathan Wakely:
> On Fri, 14 Jan 2022, 14:17 Michael Matz via Gcc,  wrote:
> 
> > Hello,
> > 
> > On Thu, 13 Jan 2022, Martin Uecker wrote:
> > 
> > > > > >  Handling all volatile accesses in the very same way would be
> > > > > > possible but quite some work I don't see much value in.
> > > > > 
> > > > > I see some value.
> > > > > 
> > > > > But an alternative could be to remove volatile
> > > > > from the observable behavior in the standard
> > > > > or make it implementation-defined whether it
> > > > > is observable or not.
> > > > 
> > > > But you are actually arguing for making UB be observable
> > > 
> > > No, I am arguing for UB not to have the power
> > > to go back in time and change previous defined
> > > observable behavior.
> > 
> > But right now that's equivalent to making it observable,
> > because we don't have any other terms than observable or
> > undefined.  As aluded to later you would have to
> > introduce a new concept, something pseudo-observable,
> > which you then started doing.  So, see below.
> > 
> > > > That's
> > > > much different from making volatile not be
> > > > observable anymore (which  obviously would
> > > > be a bad idea), and is also much harder to
> > > 
> > > I tend to agree that volatile should be
> > > considered observable. But volatile is
> > > a bit implementation-defined anyway, so this
> > > would be a compromise so that implementations
> > > do not have to make all the implied changes
> > > if we revise the meaning of UB.
> > 
> > Using volatile accesses for memory mapped IO is a much stronger use-case
> > than your wish of using volatile accesses to block moving of UB as a
> > debugging aid, and the former absolutely needs some guarantees, so I don't
> > think it would be a compromise at all.  Mkaing volatile not be observable
> > would break the C language.
> > 
> > > > Well, what you _actually_ want is an implied
> > > > dependency between some UB and volatile accesses
> > > > (and _only_ those, not e.g. with other UB), and the
> > > > difficulty now is to define "some" and to create
> > > > the dependency without making that specific UB
> > > > to be properly observable.
> > > 
> > > Yes, this is what I actually want.
> > > 
> > > >  I think to define this
> > > > all rigorously seems futile (you need a new
> > > > category between observable  and UB), so it comes
> > > > down to compiler QoI on a case by case basis.
> > > 
> > > We would simply change UB to mean "arbitrary
> > > behavior at the point of time the erraneous
> > > construct is encountered at run-time"  and
> > > not "the complete program is invalid all
> > > together". I see no problem in specifying this
> > > (even in a formally precise way)
> > 
> > First you need to define "point in time", a concept which doesn't exist
> > yet in C.  The obvious choice is of course observable behaviour in the
> > execution environment and its specified ordering from the abstract
> > machine, as clarified via sequence points.  With that your "at the point
> > in time" becomes something like "after all side effects of previous
> > sequence point, but strictly before all side effects of next sequence
> > point".
> > 
> > But doing that would have very far reaching consequences, as already
> > stated in this thread.  The above would basically make undefined behaviour
> > be reliably countable, and all implementations would need to produce the
> > same counts of UB.  That in turn disables many code movement and
> > commonization transformations, e.g. this:
> > 
> > int a = ..., b = ...;
> > int x = a + b;
> > int y = a + b;
> > 
> > can't be transformed into "y = x = a + b" anymore, because the addition
> > _might_ overflow, and if it does you have two UBs originally but would
> > have one UB after.  I know that you don't want to inhibit this or similar
> > transformations, but that would be the result of making UB countable,
> > which is the result of forcing UB to happen at specific points in time.
> > So, I continue to see problems in precisely specifying what you want, _but
> > not more_.
> > 
> > I think all models in which you order the happening of UB with respect to
> > existing side effects (per abstract machine, so it includes modification
> > of objects!) have this same problem, it always becomes a side effect
> > itself (one where you don't specify what actually happens, but a side
> > effect nontheless) and hence becomes observable.
> > 
> 
> The C++ committee is currently considering this paper:
> 
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p1494r2.html
> 
> I think this explicit barrier-like solution is better than trying to use
> volatile accesses to achieve something similar.

Can you explain why?  To me a solution which would make
it "just work" (and also fixes existing code) seems 
better than letting programmers jump through even 
more hoops, especially if only difficult corner
cases are affected.


Martin


> 
> 
> > 
> 

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Freitag, den 14.01.2022, 14:15 + schrieb Michael Matz:
> Hello,
> 
> On Thu, 13 Jan 2022, Martin Uecker wrote:

...

> > >  I think to define this 
> > > all rigorously seems futile (you need a new
> > > category between observable  and UB), so it comes
> > > down to compiler QoI on a case by case basis.
> > 
> > We would simply change UB to mean "arbitrary
> > behavior at the point of time the erraneous
> > construct is encountered at run-time"  and 
> > not "the complete program is invalid all
> > together". I see no problem in specifying this
> > (even in a formally precise way)
> 
> First you need to define "point in time", a concept which doesn't exist 
> yet in C.  The obvious choice is of course observable behaviour in the 
> execution environment and its specified ordering from the abstract 
> machine, as clarified via sequence points.  With that your "at the point 
> in time" becomes something like "after all side effects of previous 
> sequence point, but strictly before all side effects of next sequence 
> point".

Yes, all observable side effects sequenced before
the erroneous operation should be preserved. We
also need to consider multi-threading (happens-before)
But I do not think there is any need to discuss the
precise wording now.

> But doing that would have very far reaching consequences, as already
> stated in this thread.  

We already agreed that UB already works like this
relative to every function call.  So why do you
think this could have far reaching consequences
when we also require this for volatile accesses
- considering that volatile accesses are not nearly
as common as function calls, and often already
limit optimization?

We had a lot of trouble even finding examples where
compiler currently exhibit behavior that would need
to change.

> The above would basically make undefined behaviour 
> be reliably countable, and all implementations would need to produce the 
> same counts of UB.  That in turn disables many code movement and 
> commonization transformations, e.g. this:
> 
> int a = ..., b = ...;
> int x = a + b;
> int y = a + b;
> 
> can't be transformed into "y = x = a + b" anymore, because the addition 
> _might_ overflow, and if it does you have two UBs originally but would 
> have one UB after.  I know that you don't want to inhibit this or similar 
> transformations, but that would be the result of making UB countable, 
> which is the result of forcing UB to happen at specific points in time.  
> So, I continue to see problems in precisely specifying what you want, _but 
> not more_.

Don't worry, I do not want to make UB observable or
countable. Your example does not contain observable
behavior, so would be unaffected.

> I think all models in which you order the happening of UB with respect to 
> existing side effects (per abstract machine, so it includes modification 
> of objects!) have this same problem, it always becomes a side effect 
> itself (one where you don't specify what actually happens, but a side 
> effect nontheless) and hence becomes observable.

I don't think so. The standard always only defines behavior.
Here, would only guarantee that observable behavior before
a specific time point stays defined. For this we do not make
UB observable or countable because the statements we make 
is not about the UB,  but about the defined behavior before.


Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Donnerstag, den 13.01.2022, 16:45 + schrieb Michael Matz:
> Hello,
> 
> On Tue, 11 Jan 2022, Martin Uecker via Gcc wrote:
> 
> > >  Handling all volatile accesses in the
> > > very same way would be possible but quite some work I don't
> > > see much value in.
> > 
> > I see some value. 
> > 
> > But an alternative could be to remove volatile
> > from the observable behavior in the standard
> > or make it implementation-defined whether it
> > is observable or not.
> 
> But you are actually arguing for making UB be observable

No, I am arguing for UB not to have the power
to go back in time and change previous defined
observable behavior.  The reason is that it
makes it impossible to give any guarantees
about partial executions of a program  e.g.
that a transaction completed or privileges
were dropped or a log message went out to
the log server), when there is the possibility
that later there could be some UB in the program.


Making UB observable would be even stronger.
For example I do not mind a compiler sinking
a division by zero across other observable
behavior, which would not be possible if we
consider the trap to be observable.

>  (which then  directly implies an ordering
> with respect to volatile accesses). 

Yes, it would, but "no time-travel" only implies
some weaker constraints for UB and only if
it actually traps or could affect observable
behavior in other ways.

For I/O using function calls these constraints
are already fulfilled.

>  That's 
> much different from making volatile not be
> observable anymore (which  obviously would
> be a bad idea), and is also much harder to

I tend to agree that volatile should be
considered observable. But volatile is
a bit implementation-defined anyway, so this
would be a compromise so that implementations
do not have to make all the implied changes
if we revise the meaning of UB.

> do, it's  the nature of undefined behaviour
> to be hard to define :)
>
> Well, what you _actually_ want is an implied
> dependency between some UB and volatile accesses
> (and _only_ those, not e.g. with other UB), and the 
> difficulty now is to define "some" and to create
> the dependency without making that specific UB
> to be properly observable. 

Yes, this is what I actually want.

>  I think to define this 
> all rigorously seems futile (you need a new
> category between observable  and UB), so it comes
> down to compiler QoI on a case by case basis.

We would simply change UB to mean "arbitrary
behavior at the point of time the erraneous
construct is encountered at run-time"  and 
not "the complete program is invalid all
together". I see no problem in specifying this
(even in a formally precise way)

Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Dienstag, den 11.01.2022, 10:13 +0100 schrieb Richard Biener:
> On Tue, Jan 11, 2022 at 9:17 AM Martin Uecker  wrote:
> > Am Dienstag, den 11.01.2022, 08:11 +0100 schrieb Richard Biener:
> > > On Mon, Jan 10, 2022 at 6:36 PM Martin Uecker  wrote:
> > > > Am Montag, den 10.01.2022, 10:04 +0100 schrieb Richard Biener:

...
> Consider
> 
> int a[1024];
> void foo (volatile int *p, float *q)
> {
>for (int i = 0; i < 1024; ++i)
>   {
>  *p = 1;
>  a[i] = *q;
>   }
> }
> 
> we happily apply invariant motion to the load from *q, making
> it cross the store to the volatile object at *p.  Now, q might be
> NULL upon entry to the function and thus this transform
> would violate the volatile "I/O" constraint (since I/O is observable)
> and thus we will crash (which is UB) before doing the first I/O.
> 
> That's an example I'd consider important for performance and
> also a case that shows that usually the compiler will have a
> very hard time proving UB cannot happen (as opposed to the
> usual stance where it can assume it doesn't).

I can see that the transformation in general is important,
but why is it important if the "volatile" is there?

I would assume that performance-sensitive code usually
does not volatile. 

Or in other words, what is the purpose of volatile in 
this example if not either I/O or to prevent such
optimizations?

> The case we run into sth similar is with use of uninitialized
> variables where proving some variable is initialized is nearly
> impossible (copy initialization from a variable that is not
> initialized is not initialization).

These are the C++ rules.

For C, an automatic variables which is not initialized and
(as long as the address is not taken), it is directly UB.


> We've mainly settled to the stance that only program termination
> is observable which means if we do not know that a function
> call will always return normally we have to avoid hoisting
> observable UB across such function call (and I/O routines
> usually fall into this category because they are not annotated
> as always returning). 

Yes.

>  Handling all volatile accesses in the
> very same way would be possible but quite some work I don't
> see much value in.

I see some value. 

But an alternative could be to remove volatile
from the observable behavior in the standard
or make it implementation-defined whether it
is observable or not.

Martin


> Richard.
> 
> > Martin
> > 
> > > > > > I am trying to figure out whether this is feasible.
> > > > > 
> > > > > For PRE yes, you'd just need to include the observable stmts you
> > > > > care in the set of stmts that cause PRE to set BB_MAY_NOTRETURN.
> > > > > In general this is of course harder.
> > > > 
> > > > What other passes would need to be checked?
> > > 
> > > All that do code motion by design or by accident.  The difficulty is
> > > that the resulting "wrong IL" is not wrong per se, just the difference is
> > > which is hard to write a checker for (well, in priciple you could copy the
> > > IL before passes and compare to the IL after)
> > > 
> > > > And do you think there is any negative impact on
> > > > an important optimization (considering this affects
> > > > only volatile accesses)?
> > > 
> > > Probably not.  But then semantics of 'volatile' are very weak defined
> > > so I'd like
> > > to see a reference to a part of the standard that supports declaring this
> > > (and only this - the 'volatile' case) a bug.
> > > 
> > > > > > > GCC assumes by default that divide is trappable but stores not 
> > > > > > > are not
> > > > > > > observable. This is where -fnon-call-exceptions come into play.
> > > > > > 
> > > > > > Ok, thanks! I will look at this!
> > > > > > 
> > > > > > > In the second case, GCC assumes reducing trappable instructions 
> > > > > > > are
> > > > > > > fine.
> > > > > > 
> > > > > > -fnon-call-exceptions would treat trapping instructions
> > > > > > as defined (and trapping) instead of UB? This is
> > > > > > then probably even stronger than the requirement above.
> > > > > 
> > > > > No, I don't think it turns UB into defined behavior.  Some frontends 
> > > > > might
> > > > > expect that to some extent.  So even with -fnon-call-exceptions we'd
> > > > > happily do the re-ordering unless the exception is catched in the same
> > > > > function.
> > > > 
> > > > Thanks,
> > > > Martin
> > > > 
> > > > > > > Note I thought -fno-delete-dead-exceptions would fix the sink
> > > > > > > but it didn't.
> > > > > > 
> > > > > > Martin
> > > > > > 
> > > > > > 

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Dienstag, den 11.01.2022, 08:11 +0100 schrieb Richard Biener:
> On Mon, Jan 10, 2022 at 6:36 PM Martin Uecker  wrote:
> > Am Montag, den 10.01.2022, 10:04 +0100 schrieb Richard Biener:

Hi Richard,

> > > > For volatile, it seems this would need some tweaks.
> > > 
> > > Yes, likewise when re-ordering (observable) traps like
> > > 
> > >   r = a / b;
> > >   q = c / d;
> > 
> > I think this could also be useful. But at the moment I am
> > concerned about the effect previous defined behavior
> > being affected. For this, reordering traps is OK.  Also
> > sinking traps across observable behavior is OK. Just
> > hoisting it up across observable behavior would
> > be a problem.
> 
> But in general that would apply to all UB.  Consider
> 
> int foo (int a, int b)
> {
>if (a < b)
>  return a + b;
>bar ();
>return a + b;
> }
> 
> we are happily hoisting a + b to the start of the function
> but of course a + b can invoke UB.  We consider that to
> not matter because we eventually invoke this UB anyway.
> Unless of course bar() does not return.

Yes.

> I realize that UB in a + b isn't (usually) observable but
> UB resulting in traps are.

Code motion for UB which then does not cause
a change in observable behavior would still be ok.

So my understanding is that you can not hoist a potentially
trapping operation across a function call, but if it is
UB which is implemented in way that just produces some
random result but does not trap then this is ok.

It would also be wrong if it affects the arguments for
the function call. Here MSVC seems to do this:

https://godbolt.org/z/8a8fTW8qP

This seems incorect because if the call does not
return there is no UB. I did not observe this with
GCC or another compiler.

> So I'm still wondering why you think that 'volatile' makes
> a critical difference we ought to honor?  I don't remember
> 'volatile' being special in the definition of the abstract
> machine with regarding to observability (as opposed to
> sequence points).

It is because it is used for I/O.   Sequence points only
matter for the semantics of the abstract machine, so
according to "as-if" rule optimizers can do whatever
they want as long as the observable behavior is the same
"as-if" it followed the rules of the abstract machine.

This observable behavior that needs to be preserved is
defined as I/O and volatile accesses. The relevant
part o the standard is this:

"5.1.2.3 Program execution" paragraph 6

The least requirements on a conforming implementation are:

— Accesses to volatile objects are evaluated strictly
according to the rules of the abstract machine.
— At program termination, all data written into files
shall be identical to the result that execution
of the program according to the abstract semantics would 
have produced.
— The input and output dynamics of interactive devices 
shall take place as specified in 7.21.3.

The intent of these requirements is that unbuffered or
line-buffered output appear as soon as possible, to 
ensure that prompting messages actually appear prior
to a program waiting for input.

This is the observable behavior of the program."



Martin

> > > > I am trying to figure out whether this is feasible.
> > > 
> > > For PRE yes, you'd just need to include the observable stmts you
> > > care in the set of stmts that cause PRE to set BB_MAY_NOTRETURN.
> > > In general this is of course harder.
> > 
> > What other passes would need to be checked?
> 
> All that do code motion by design or by accident.  The difficulty is
> that the resulting "wrong IL" is not wrong per se, just the difference is
> which is hard to write a checker for (well, in priciple you could copy the
> IL before passes and compare to the IL after)
> 
> > And do you think there is any negative impact on
> > an important optimization (considering this affects
> > only volatile accesses)?
> 
> Probably not.  But then semantics of 'volatile' are very weak defined
> so I'd like
> to see a reference to a part of the standard that supports declaring this
> (and only this - the 'volatile' case) a bug.
> 
> > > > > GCC assumes by default that divide is trappable but stores not are not
> > > > > observable. This is where -fnon-call-exceptions come into play.
> > > > 
> > > > Ok, thanks! I will look at this!
> > > > 
> > > > > In the second case, GCC assumes reducing trappable instructions are
> > > > > fine.
> > > > 
> > > > -fnon-call-exceptions would treat trapping instructions
> > > > as defined (and trapping) instead of UB? This is
> > > > then probably even stronger than the requirement above.
> > > 
> > > No, I don't think it turns UB into defined behavior.  Some frontends might
> > > expect that to some extent.  So even with -fnon-call-exceptions we'd
> > > happily do the re-ordering unless the exception is catched in the same
> > > function.
> > 
> > Thanks,
> > Martin
> > 
> > > > > Note I thought -fno-delete-dead-exceptions would fix the sink
> > > > > but it didn't.
> > > > 
> > > > Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Montag, den 10.01.2022, 10:04 +0100 schrieb Richard Biener:
> On Sat, Jan 8, 2022 at 10:09 PM Martin Uecker via Gcc  wrote:
> > Am Samstag, den 08.01.2022, 10:35 -0800 schrieb Andrew Pinski:
> > > On Sat, Jan 8, 2022 at 12:33 AM Martin Uecker via Gcc  
> > > wrote:
> > > > Hi Richard,
> > > > 
> > > > I have a question regarding reodering of volatile
> > > > accesses and trapping operations. My initial
> > > > assumption (and  hope) was that compilers take
> > > > care to avoid creating traps that are incorrectly
> > > > ordered relative to observable behavior.
> > > > 
> > > > I had trouble finding examples, and my cursory
> > > > glace at the code seemed to confirm that GCC
> > > > carefully avoids this.  But then someone showed
> > > > me this example, where this can happen in GCC:
> > > > 
> > > > 
> > > > volatile int x;
> > > > 
> > > > int foo(int a, int b, _Bool store_to_x)
> > > > {
> > > >   if (!store_to_x)
> > > > return a / b;
> > > >   x = b;
> > > >   return a / b;
> > > > }
> > > > 
> > > > 
> > > > https://godbolt.org/z/vq3r8vjxr
> > > 
> > > The question becomes what is a trapping instruction vs an undefined
> > > instruction?
> > > For floating point types, it is well defined what is a trapping
> > > instruction while for integer types it is not well defined.
> > > On some (many?) targets dividing by 0 is just undefined and does not
> > > trap (powerpc, aarch64, arm and many others; MIPS it depends on the
> > > options passed to GCC if the conditional trap should be inserted or
> > > not).
> > > The other side is if there is undefined code on the path, should
> > > observable results happen first (stores to volatile/atomics, etc.)?
> > 
> > I think for volatile stores and I/O, I think it would be
> > nice of we could guarantee that those happen before the UB
> > ruins the day. (I am not sure about atomics, those are
> > not directly obsevable)
> > 
> > For I/O this is probably already the case (?).
> 
> I/O usually happens through function calls where this is usually
> already guaranteed as GCC doesn't know whether the function
> will always return normally so the UB of a divide by zero might
> be properly guarded.

Yes.

> > For volatile, it seems this would need some tweaks.
> 
> Yes, likewise when re-ordering (observable) traps like
> 
>   r = a / b;
>   q = c / d;

I think this could also be useful. But at the moment I am
concerned about the effect previous defined behavior
being affected. For this, reordering traps is OK.  Also
sinking traps across observable behavior is OK. Just
hoisting it up across observable behavior would 
be a problem.

> > I am trying to figure out whether this is feasible.
> 
> For PRE yes, you'd just need to include the observable stmts you
> care in the set of stmts that cause PRE to set BB_MAY_NOTRETURN.
> In general this is of course harder.

What other passes would need to be checked?

And do you think there is any negative impact on
an important optimization (considering this affects
only volatile accesses)?

> > > GCC assumes by default that divide is trappable but stores not are not
> > > observable. This is where -fnon-call-exceptions come into play.
> > 
> > Ok, thanks! I will look at this!
> > 
> > > In the second case, GCC assumes reducing trappable instructions are
> > > fine.
> > 
> > -fnon-call-exceptions would treat trapping instructions
> > as defined (and trapping) instead of UB? This is
> > then probably even stronger than the requirement above.
> 
> No, I don't think it turns UB into defined behavior.  Some frontends might
> expect that to some extent.  So even with -fnon-call-exceptions we'd
> happily do the re-ordering unless the exception is catched in the same
> function.

Thanks,
Martin

> > > Note I thought -fno-delete-dead-exceptions would fix the sink
> > > but it didn't.
> > 
> > Martin
> > 
> > 

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 08.01.2022, 10:35 -0800 schrieb Andrew Pinski:
> On Sat, Jan 8, 2022 at 12:33 AM Martin Uecker via Gcc  wrote:
> > 
> > Hi Richard,
> > 
> > I have a question regarding reodering of volatile
> > accesses and trapping operations. My initial
> > assumption (and  hope) was that compilers take
> > care to avoid creating traps that are incorrectly
> > ordered relative to observable behavior.
> > 
> > I had trouble finding examples, and my cursory
> > glace at the code seemed to confirm that GCC
> > carefully avoids this.  But then someone showed
> > me this example, where this can happen in GCC:
> > 
> > 
> > volatile int x;
> > 
> > int foo(int a, int b, _Bool store_to_x)
> > {
> >   if (!store_to_x)
> > return a / b;
> >   x = b;
> >   return a / b;
> > }
> > 
> > 
> > https://godbolt.org/z/vq3r8vjxr
> 
> The question becomes what is a trapping instruction vs an undefined
> instruction?
> For floating point types, it is well defined what is a trapping
> instruction while for integer types it is not well defined.
> On some (many?) targets dividing by 0 is just undefined and does not
> trap (powerpc, aarch64, arm and many others; MIPS it depends on the
> options passed to GCC if the conditional trap should be inserted or
> not).

> The other side is if there is undefined code on the path, should
> observable results happen first (stores to volatile/atomics, etc.)?

I think for volatile stores and I/O, I think it would be
nice of we could guarantee that those happen before the UB
ruins the day. (I am not sure about atomics, those are 
not directly obsevable)

For I/O this is probably already the case (?).
For volatile, it seems this would need some tweaks.

I am trying to figure out whether this is feasible.

> GCC assumes by default that divide is trappable but stores not are not
> observable. This is where -fnon-call-exceptions come into play.

Ok, thanks! I will look at this!

> In the second case, GCC assumes reducing trappable instructions are
> fine. 

-fnon-call-exceptions would treat trapping instructions
as defined (and trapping) instead of UB? This is
then probably even stronger than the requirement above.

> Note I thought -fno-delete-dead-exceptions would fix the sink
> but it didn't.


Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 08.01.2022, 16:03 +0100 schrieb David Brown:
> On 08/01/2022 09:32, Martin Uecker via Gcc wrote:
> > Hi Richard,
> > 
> > I have a question regarding reodering of volatile
> > accesses and trapping operations. My initial
> > assumption (and  hope) was that compilers take
> > care to avoid creating traps that are incorrectly
> > ordered relative to observable behavior.
> > 
> > I had trouble finding examples, and my cursory
> > glace at the code seemed to confirm that GCC
> > carefully avoids this.  But then someone showed
> > me this example, where this can happen in GCC:
> > 
> > 
> > volatile int x;
> > 
> > int foo(int a, int b, _Bool store_to_x)
> > {
> >   if (!store_to_x)
> > return a / b;
> >   x = b;
> >   return a / b;
> > }
> > 
> > 
> > https://godbolt.org/z/vq3r8vjxr
> > 
> > In this example a division is hoisted 
> > before the volatile store. (the division
> > by zero which could trap is UB, of course).
> > 
> 
> Doesn't this depend on whether the trap is considered "observable
> behaviour", or "undefined behaviour" ?
> 
> If (on the given target cpu and OS, and with any relevant compiler
> flags) dividing by zero is guaranteed to give a trap with specific known
> behaviour, then it is observable behaviour and thus should be ordered
> carefully with respect to the volatile accesses.
> 
> On the other hand, if division by 0 is considered undefined behaviour
> (the C and C++ standards explicitly mark it as undefined, but a compiler
> can of course define its behaviour) then the compiler can assume it does
> not happen, or you don't care about the result of the program if it
> happens.  Undefined behaviour can be freely re-ordered around volatile
> accesses, as far as I understand it - though that can come as a surprise
> to some people.

In C++ has wording that makes it clear that this reordering
is allowed. In C, some people also see it this way.  In my
opinion, this is not clear and I always read the standard
in a different way (i.e. run-time UB happens at a point
in time but can not go backwards at change previous
defined behavior).

But in any case, I would find it much more useful if it is
guaranteed to not affect previous observable behavior.
This would make volatile more useful, which in my opinion
is preferable to introducing another language feature to
work around this issue.

This of course assumes that this reodering around volatile
accesses and I/O is not essential for optimization.

Martin


> 
> I don't know which of these views gcc takes - I think both are valid.
> But it might be worth noting in the reference manual.
> 
> David
> 
> 
> 
> > As Martin Sebor pointed out this is done
> > as part of redundancy elimination 
> > in tree-ssa-pre.c and that this might
> > simply be an oversight (and could then be
> > fixed with a small change).
> > 
> > Could you clarify whether such reordering
> > is intentional and could be exploited in
> > general also in other optimizations or
> > confirm that this is an oversight that
> > affects only this specific case?
> > 
> > If this is intentional, are there examples
> > where this is important for optimization?
> > 
> > 
> > Martin
> > 
> > 
> > 
> > 
> > 
> > 
> > 

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 08.01.2022, 15:41 +0100 schrieb Eric Botcazou:
> > Yes, although I think potentially trapping ops
> > are not moved before calls (as this would be
> > incorrect).  So do you think it would be feasable
> > to prevent this for volatile too?
> 
> Feasible probably, but why would this be desirable in C?  It's not Java!

It would allow us to still give at least some guarantees about the
observable behavior of programs that later in their execution
encounter UB (e.g. that an transaction with an external
device is correctly completed).  Considering the fact that it
is virtually impossible to prove that any realistic C program
is completely free of UB, this is would be very useful.

As another example, there was recently the a proposal about
adding a safe memory erasure function to the standard lib.
It was pointed out that volatile stores would not be enough
to be sure that the compiler safely erased some sensitive
information, because an optimization based on later UB in 
the program could undo this.  

There is now also a proposal for C++ to introduce std::observable,
which would require similar ordering constraints.  But this would
require the programmer to annotate the program correctly.
Most C programmers would assume that volatile accesses already
provides this guarantee, so actually doing so would be good.

Or a more practical example: While debugging some embedded
device, it would also be very annoying if the compilers reorders
some trap before some debugging output. I could easily imagine
loosing hours figuring out what happens.


Martin

Re: reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Am Samstag, den 08.01.2022, 13:41 +0100 schrieb Richard Biener:
> On January 8, 2022 9:32:24 AM GMT+01:00, Martin Uecker  
> wrote:
> > Hi Richard,

thank you for your quick response!

> > I have a question regarding reodering of volatile
> > accesses and trapping operations. My initial
> > assumption (and  hope) was that compilers take
> > care to avoid creating traps that are incorrectly
> > ordered relative to observable behavior.
> > 
> > I had trouble finding examples, and my cursory
> > glace at the code seemed to confirm that GCC
> > carefully avoids this.  But then someone showed
> > me this example, where this can happen in GCC:
> > 
> > 
> > volatile int x;
> > 
> > int foo(int a, int b, _Bool store_to_x)
> > {
> >  if (!store_to_x)
> >return a / b;
> >  x = b;
> >  return a / b;
> > }
> > 
> > 
> > https://godbolt.org/z/vq3r8vjxr
> > 
> > In this example a division is hoisted 
> > before the volatile store. (the division
> > by zero which could trap is UB, of course).
> > 
> > As Martin Sebor pointed out this is done
> > as part of redundancy elimination 
> > in tree-ssa-pre.c and that this might
> > simply be an oversight (and could then be
> > fixed with a small change).
> > 
> > Could you clarify whether such reordering
> > is intentional and could be exploited in
> > general also in other optimizations or
> > confirm that this is an oversight that
> > affects only this specific case?
> > 
> > If this is intentional, are there examples
> > where this is important for optimization?
> 
> In general there is no data flow information that
> prevents traps from being reordered with respect
> to volatile accesses. 

Yes, although I think potentially trapping ops
are not moved before calls (as this would be
incorrect).  So do you think it would be feasable
to prevent this for volatile too?

> The specific case could be
> easily mitigated in PRE. Another case would be
> 
> A = c / d;
> X = 1;
> If (use_a)
>   Bar (a);
> 
> Where we'd sink a across x into the guarded Bb I suspect. 

Yes. Related example:

https://godbolt.org/z/5WGhadre3

volatile int x;
void bar(int a);

void foo(int c, int d)
{
  int a = c / d;
  x = 1;
  if (d)
bar(a);
}

foo:
mov DWORD PTR x[rip], 1
testesi, esi
jne .L4
ret
.L4:
mov eax, edi
cdq
idivesi
mov edi, eax
jmp bar


It would be nice to prevent this too, although
I am less concerned about this direction, as
the UB has already happened so there is not
much we could guarantee about this anyway.

In the other case, it could affect correct
code before the trap. 

Martin


> (sorry for the odd formatting, writing this on a mobile device). 
> 
> Richard. 
> > Martin
> > 
> > 
> > 
> > 
> > 
> > 

reordering of trapping operations and volatile

[Martin Uecker via Gcc]

Hi Richard,

I have a question regarding reodering of volatile
accesses and trapping operations. My initial
assumption (and  hope) was that compilers take
care to avoid creating traps that are incorrectly
ordered relative to observable behavior.

I had trouble finding examples, and my cursory
glace at the code seemed to confirm that GCC
carefully avoids this.  But then someone showed
me this example, where this can happen in GCC:


volatile int x;

int foo(int a, int b, _Bool store_to_x)
{
  if (!store_to_x)
return a / b;
  x = b;
  return a / b;
}


https://godbolt.org/z/vq3r8vjxr

In this example a division is hoisted 
before the volatile store. (the division
by zero which could trap is UB, of course).

As Martin Sebor pointed out this is done
as part of redundancy elimination 
in tree-ssa-pre.c and that this might
simply be an oversight (and could then be
fixed with a small change).

Could you clarify whether such reordering
is intentional and could be exploited in
general also in other optimizations or
confirm that this is an oversight that
affects only this specific case?

If this is intentional, are there examples
where this is important for optimization?


Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Freitag, den 26.11.2021, 15:48 + schrieb Jonathan Wakely:
> On Fri, 26 Nov 2021 at 15:41, Martin Uecker  wrote:
> > Am Freitag, den 26.11.2021, 09:24 + schrieb Jonathan Wakely:
> > > On Fri, 26 Nov 2021 at 09:00, Martin Uecker via Gcc  
> > > wrote:
> > > > Am Freitag, den 26.11.2021, 09:29 +0100 schrieb Eric Botcazou:
> > > > > > This is a silent and dangerous incorrect code generation issue.
> > > > > 
> > > > > Let's avoid this kind of FUD, please, builtins are low-level devices 
> > > > > and
> > > > > people must know what they are doing and be prepared for caveats.
> > > > 
> > > > Sorry, I do not think this FUD. One needs to look at the assembly
> > > > and know *very specific* details about the platform and atomics
> > > > to understand that it does not work and silently (!) breaks on
> > > > GCC (and not clang) in very rare cases. This with no indication
> > > > about this in the documentation which clearly implies that
> > > > this works for "all types".
> > > 
> > > It does. Two objects of type T are still the same type whether or not
> > > they are both aligned to sizeof(T).
> > 
> > Yes, but only under special circumstances which are
> > not automatically fulfilled for all types.
> 
> Erm, two objects of type T are always the same type.

It works only under special circumstances...

> My point is that the docs say "works for all types", not "works for
> all objects of all types".
> 
> You are claiming the docs are misleading, I'm saying you're misreading
> them. They do not say "all objects of all types".

It also does not expicitly say it works on all moon
phases. 

Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Freitag, den 26.11.2021, 09:24 + schrieb Jonathan Wakely:
> On Fri, 26 Nov 2021 at 09:00, Martin Uecker via Gcc  wrote:
> > Am Freitag, den 26.11.2021, 09:29 +0100 schrieb Eric Botcazou:
> > > > This is a silent and dangerous incorrect code generation issue.
> > > 
> > > Let's avoid this kind of FUD, please, builtins are low-level devices and
> > > people must know what they are doing and be prepared for caveats.
> > 
> > Sorry, I do not think this FUD. One needs to look at the assembly
> > and know *very specific* details about the platform and atomics
> > to understand that it does not work and silently (!) breaks on
> > GCC (and not clang) in very rare cases. This with no indication
> > about this in the documentation which clearly implies that
> > this works for "all types".
> 
> It does. Two objects of type T are still the same type whether or not
> they are both aligned to sizeof(T).

Yes, but only under special circumstances which are
not automatically fulfilled for all types.

> > > > If these functions are not meant to be used to exising
> > > > data,  then at least the documentation needs to be changed
> > > > and include a big warning that this only happens to work
> > > > corectly if the data has  sufficient alignment for the
> > > > specific architecture (which of course makes it impossible
> > > > to use this in a portable way).
> > > 
> > > The last part of the sentence is again a blatant overstatement
> > 
> > Then please explain how one is supposed to use in a portable way if
> > it sometimes work or not based on platform-specific alignment
> > properties.
> 
> Either align your variable to its size, or don't use non-portable
> compiler built-ins, use _Atomic.
>
> If you want to be able to perform atomic operations on non-_Atomic
> objects portably, that seems like a useful addition to the C standard
> library (like std::atomic_ref in C++20).

The problem is that atomic_ref also only works if the
alignment fulfills special requirements.  So it can not
always be used on existing data structures.  

I think it would be helpful if a compiler supports this by
providing suitable builtins, which it could easily do.  

Since the existing builtins are already documented in a
way that implies that they allow exactly this, it would
be better and safer to just make this work.


Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Freitag, den 26.11.2021, 09:29 +0100 schrieb Eric Botcazou:
> > This is a silent and dangerous incorrect code generation issue.
> 
> Let's avoid this kind of FUD, please, builtins are low-level devices and 
> people must know what they are doing and be prepared for caveats.

Sorry, I do not think this FUD. One needs to look at the assembly
and know *very specific* details about the platform and atomics
to understand that it does not work and silently (!) breaks on
GCC (and not clang) in very rare cases. This with no indication
about this in the documentation which clearly implies that
this works for "all types".

> > If these functions are not meant to be used to exising
> > data,  then at least the documentation needs to be changed
> > and include a big warning that this only happens to work
> > corectly if the data has  sufficient alignment for the
> > specific architecture (which of course makes it impossible
> > to use this in a portable way).
> 
> The last part of the sentence is again a blatant overstatement

Then please explain how one is supposed to use in a portable way if
it sometimes work or not based on platform-specific alignment
properties. 

>  but I agree 
> that the alignment caveat ought to be documented.  Please suggest a wording 
> to 
> that effect and post a patch onto the gcc-patches@ ML.  Thanks in advance.


Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Sonntag, den 07.11.2021, 10:08 +0100 schrieb Martin Uecker:
> It would be great if somebody could take a look at
> PR96159. 
> 
> It seems we do not do atomic accesses correctly
> when the alignment is insufficient for a lockfree
> access, but I think we should fall back to a
> library call in this case (as clang does).
> 
> This is very unfortunate as it is an important
> functionality to be able to do atomic accesses 
> on non-atomic types and it seems there is no way
> to achieve this.
> 
> Also documentation and various descriptions of
> the atomic functions imply that this is expected
> to work.
> 
> But maybe I am missing something and the generated
> code is indeed safe.
> 
> Martin
> 

Could this bug be confirmed please? 

This is a silent and dangerous incorrect code generation issue.  

If these functions are not meant to be used to exising
data,  then at least the documentation needs to be changed
and include a big warning that this only happens to work
corectly if the data has  sufficient alignment for the
specific architecture (which of course makes it impossible
to use this in a portable way).

I would then propose to add atomic_load_safe,
so that it is possible to use such functionality safely
on existing data structures which is an important use
case.

Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Montag, den 08.11.2021, 11:59 + schrieb Jonathan Wakely:
> On Sun, 7 Nov 2021, 09:08 Martin Uecker wrote:
> 
> > It would be great if somebody could take a look at
> > PR96159.
> > 
> > It seems we do not do atomic accesses correctly
> > when the alignment is insufficient for a lockfree
> > access, but I think we should fall back to a
> > library call in this case (as clang does).
> > 
> > This is very unfortunate as it is an important
> > functionality to be able to do atomic accesses
> > on non-atomic types and it seems there is no way
> > to achieve this.
> > 
> 
> C++ solves this with std::atomic_ref which has a precondition that the
> non-atomic object being operated on is suitably aligned.

>  Working with
> arbitrary objects is not supported, the user must ensure the alignment is
> correct.
> 
> That seems like a reasonable restriction to me.

It is in so far not reasonable as you can not use atomic
operations safely on existing data structures, but
this is an important requirement in some applications.

And it would trivial to support just by calling into
libatomic as clang does.

Martin

Re: atomic_load

[Martin Uecker via Gcc]

Am Sonntag, den 07.11.2021, 10:24 +0100 schrieb Florian Weimer:
> * Martin Uecker via Gcc:
> 
> > It would be great if somebody could take a look at
> > PR96159. 
> > 
> > It seems we do not do atomic accesses correctly
> > when the alignment is insufficient for a lockfree
> > access, but I think we should fall back to a
> > library call in this case (as clang does).
> > 
> > This is very unfortunate as it is an important
> > functionality to be able to do atomic accesses 
> > on non-atomic types and it seems there is no way
> > to achieve this.
> 
> GCC should probably warn about this, and ensure that objects declared
> _Atomic have increased alignment.  But adding run-time checks to all
> atomic accesses does not make sense.  We want to move user code away
> from bespoke atomics implementations, and this would be contrary to that
> goal.

You do not build trust in your API if you silently
create incorrect code.  

If the alignment is sufficient as guaranteed by the
type, clang also does not use a run-time check.


Martin

atomic_load

[Martin Uecker via Gcc]

It would be great if somebody could take a look at
PR96159. 

It seems we do not do atomic accesses correctly
when the alignment is insufficient for a lockfree
access, but I think we should fall back to a
library call in this case (as clang does).

This is very unfortunate as it is an important
functionality to be able to do atomic accesses 
on non-atomic types and it seems there is no way
to achieve this.

Also documentation and various descriptions of
the atomic functions imply that this is expected
to work.

But maybe I am missing something and the generated
code is indeed safe.

Martin

Re: [RFC] Adding a new attribute to function param to mark it as constant

[Martin Uecker via Gcc]

> On Wed, 4 Aug 2021 at 03:27, Segher Boessenkool
>  wrote:
> >
> > Hi!
> >
> > On Fri, Jul 23, 2021 at 04:23:42PM +0530, Prathamesh Kulkarni via Gcc wrote:
> > > The constraint here is that, vshl_n intrinsics require that the
> > > second arg (__b),
> > > should be an immediate value.
> >
> > Something that matches the "n" constraint, not necessarily a literal,
> > but stricter than just "immediate".  It probably is a good idea to allow
> > only "integer constant expression"s, so that the validity of the source
> > code does not depend on what the optimisers do with the code.
> >
> > > As Richard suggested, sth like:
> > > void foo(int x __attribute__((literal_constant (min_val, max_val)));
> >
> > The Linux kernel has a macro __is_constexpr to test if something is an
> > integer constant expression, see  .  That is a much
> > better idea imo.  There could be a builtin for that of course, but an
> > attribute is less powerful, less usable, less useful.
> Hi Segher,
> Thanks for the suggestions. I am not sure tho if we could use a macro
> similar to __is_constexpr
> to check if parameter is constant inside an inline function (which is
> the case for intrinsics) ?
> 
> For eg:
> #define __is_constexpr(x) \
> (sizeof(int) == sizeof(*(8 ? ((void *)((long)(x) * 0l)) : (int *)8)))
> 
> inline int foo(const int x)
> {
>   _Static_assert (__is_constexpr (x));
>   return x;
> }
> 
> int main()
> {
>   return foo (1);
> }
> 
> results in:
> foo.c: In function ‘foo’:
> foo.c:8:3: error: static assertion failed
> 8 |   _Static_assert (__is_constexpr (x));
> 
> Initially we tried to use __Static_assert (__builtin_constant_p (arg))
> for the same purpose but that did not work
> because while parsing the intrinsic function, the FE cannot determine
> if the arg is indeed a constant.
> I guess the static assertion or __is_constexpr would work only if the
> intrinsic were defined as a macro instead of an inline function ?
> Or am I misunderstanding ?

You have to use it at the call site:

#define __is_constexpr(x) \
 (sizeof(int) == sizeof(*(8 ? ((void *)((long)(x) * 0l)) : (int *)8)))

#define foo(x) foo(({ _Static_assert(__is_constexpr(x), "no ICE"); (x); }))

inline int (foo)(const int x)
{
  return x;
}

int main()
{
  foo(1);
  int n = 1;
  foo(n);   // error
}


--Martin