Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 11/17/22 13:35, Bruno Haible wrote:


Clang will surely not acquire knowledge about "every library", right,
only about the C library according to relevant standards (ISO C, POSIX)?


I don't know the Clang developers' plans. But if I wanted Clang to be 
picky then yes, I'd have it know about every library the program links 
to. It's some work but doable via techniques like -flto. Let's hope this 
isn't enabled by default.




There is a
documented way to ask Clang and GCC "assume a free-standing implementation,
not an environment that has the ISO C / POSIX / LSB / ... functions":
   1) -ffreestanding [1][2]
   2) -fno-builtin


Thanks, good suggestion. If this suffices to pacify Clang, we can use it 
in Autoconf: that is, we cause ./configure to link a simple program with 
-fno-builtin and if that works, then ./configure uses -fno-builtin when 
doing "does this function exist?" link-time tests.


This hack wouldn't suffice if Clang starts doing link-time type checking 
by default. However, link-time type checking doesn't seem to be on the 
Clang developers' to-do list, so we should be OK at least for now.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Bruno Haible]

Paul Eggert wrote:
> > AC_CHECK_FUNC *should not*  just probe for linkability of a symbol
> 
> ... Autoconf cannot 
> be expected to know every signature of every function in every library.

Clang will surely not acquire knowledge about "every library", right,
only about the C library according to relevant standards (ISO C, POSIX)?

> ... In the meantime if 
> Clang becomes pickier by default it would be helpful if there were a 
> well-defined way to shut off Clang's pickiness.

To me it seems the problem is not Clang's pickiness, but rather the
assumptions that it makes about the target environment. There is a
documented way to ask Clang and GCC "assume a free-standing implementation,
not an environment that has the ISO C / POSIX / LSB / ... functions":
  1) -ffreestanding [1][2]
  2) -fno-builtin

Can we assume that these options will continue to work?

If so, all Autoconf needs to do is to pass these options to the compiler
in AC_CHECK_FUNC checks. Then we don't need to add knowledge about each
of the hundreds of standard libc functions into Autoconf.

Bruno

[1] https://clang.llvm.org/docs/ClangCommandLineReference.html
[2] https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/C-Dialect-Options.html

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 10:59, Zack Weinberg wrote:

I'm generally in agreement with Rich Felker's argument 
(inhttps://ewontfix.com/13/) that AC_CHECK_FUNC*should not*  just probe for 
linkability of a symbol


So am I. I'm not saying Autoconf should never change here, only that the 
change would not be trivial. It would require changing many configure.ac 
scripts scattered over many software projects, because Autoconf cannot 
be expected to know every signature of every function in every library.


Any such transition could not be done in a week, or a month, or even a 
year. I would guess it would take a decade at least. In the meantime if 
Clang becomes pickier by default it would be helpful if there were a 
well-defined way to shut off Clang's pickiness. If the Clang developers 
provide such a way we can use it; if not, Autoconf will just have to do 
what it always does, and figure a way out anyway (hey! it could drop 
into assembler...).


Things would be simpler if Clang became pickier by default only for 
declarations that are not "char foo();" or "char foo(void);". Then, 
existing 'configure' scripts would still work.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 10:40, Jeffrey Walton wrote:

This line of arguments is not persuasive. It is full of logical fallacies.


... none of which you stated.

No matter how we solve the problem, it will be a hack that exploits 
"logical fallacies" (whatever that means). However, a reaction "You 
violated the C standard! You deserve to be punished!" is not the best 
one for the overall software ecosystem. Lots of programs violate the C 
standard every day, and Clang supports them anyway.


Yesterday I dealt with this Autoconf bug report:

https://lists.gnu.org/r/autoconf/2022-11/msg00092.html

which basically said, "Here's some longstanding buggy code that uses 
Autoconf. This buggy code happened to work in the previous stable 
Autoconf version, but it stopped working in the bleeding-edge version."


Did I respond, "That's buggy code and it deserves to be punished?" No, I 
responded that it's buggy code that needs to be fixed (and gave a fix), 
but fixing this sort of thing is a hassle for distributors and so I also 
installed a minor hack to bleeding-edge Autoconf that lets the buggy 
code work again, at least for now. 



It would help if Clang developers could cooperate to address this 
potential problem with stricter compilation defaults. It's a real 
problem. And it shouldn't require much work on the Clang side to address it.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jason Merrill via Gcc]

On Sat, Nov 12, 2022 at 7:44 PM Paul Eggert  wrote:

> On 2022-11-11 07:11, Aaron Ballman wrote:
> > Clang doesn't require such a linker (we work with various system
> linkers).
>
> As long as the system linkers continue to work as they have
> traditionally worked, we're fine.
>
> > the frontend perspective, we can't tell the difference between
> > "trust me this is safe because it never gets executed" and "this is a
> > CVE"
>
> If some system linker ever attempts to reject links with mismatched
> signatures, Autoconf-generated code will need to have a way to shut that
> off. I hope Clang maintainers can be cajoled into supporting that, if
> the time comes. Perhaps there can be a #pragma, or a compile-time
> option, to do that.
>

There has been discussion of the problems with compile-time options
elsewhere in the thread, but the #pragma idea sounds promising, as older
compilers can just ignore it.

Jason

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz via Gcc]

Hello,

On Wed, 16 Nov 2022, Paul Eggert wrote:

> On 2022-11-16 06:26, Michael Matz wrote:
> > char foobar(void);
> > int main(void) {
> >return  != 0;
> > }
> 
> That still has undefined behavior according to draft C23,

This is correct (and also holds for the actually working variant later, 
with a volatile variable).  If your argument is then that as both 
solutions for the link-test problem are relying on undefined behaviour 
they are equivalent and hence no change is needed, you have a point, but I 
disagree.  In practice one (with the call) will cause more problems than 
the other (with address taking).

> If Clang's threatened pickiness were of some real use elsewhere, it 
> might be justifiable for default Clang to break Autoconf. But so far we 
> haven't seen real-world uses that would justify this pickiness for 
> Autoconf's use of 'char memset_explicit(void);'.

Note that both, GCC and clang, already warn (not error out!) about the 
mismatching decl, even without any headers.  So we are in the pickiness 
era already.

I.e. a C file containing just a single line "char printf(void);" will be 
warned about, by default.  There is about nothing that autoconf could do 
to rectify this, except containing a long list of prototypes for 
well-known functions, with the associated maintenance hassle.  But 
autoconf _can_ do something about how the decls are used in the 
link-tests.


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

On Wed, Nov 16, 2022, at 1:17 PM, Paul Eggert wrote:
...
> If Clang's threatened pickiness were of some real use elsewhere, it 
> might be justifiable for default Clang to break Autoconf. But so far we 
> haven't seen real-world uses that would justify this pickiness for 
> Autoconf's use of 'char memset_explicit(void);'.

I don't have time this week to really get into this argument but I want to 
point out that I'm generally in agreement with Rich Felker's argument (in 
https://ewontfix.com/13/) that AC_CHECK_FUNC *should not* just probe for 
linkability of a symbol, because:

 - Not including the appropriate headers means that the probe bypasses 
compile-time symbol renaming and therefore probes for *the wrong symbol*, 
potentially causing both false detection and false non-detection (to tie it to 
another subthread, notice that one of the things -D_TIME_BITS=64 triggers (from 
glibc's headers) is enable dozens of __REDIRECT annotations in time.h)

 - Only probing the symbol, and not its type signature, means for instance that 
if the program expects GNU strerror_r but the system provides POSIX strerror_r, 
or vice versa, Autoconf's check will succeed but the program will fail to 
compile (in more subtle cases it might be miscompiled instead)

(N.B. I *don't* agree with the assertion at the bottom of that page that 
"taking explicit action to prevent linking against [symbols intended to be 
exposed for binary compatibility only], involves hacks that are even worse and 
more fragile than what configure is doing" -- yes, it sucks that the toolchain 
support for ELF symbol versioning is still mostly absent, 20 years after the 
concept was introduced, but `asm(".symver __strtod_l_compat, 
strtod_l@SOME_CONCRETE_VERSION_TAG")` is straightforward (if cryptic) and 
robust on all the platforms where it works at all.)

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jeffrey Walton via Gcc]

On Wed, Nov 16, 2022 at 1:18 PM Paul Eggert  wrote:
> ...
> If Clang's threatened pickiness were of some real use elsewhere, it
> might be justifiable for default Clang to break Autoconf. But so far we
> haven't seen real-world uses that would justify this pickiness for
> Autoconf's use of 'char memset_explicit(void);'.

This line of arguments is not persuasive. It is full of logical fallacies.

Maybe the Autoconf folks should argue something along the lines of,
"this is the _only_ way to do it because <...>."

Jeff

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-16 06:26, Michael Matz wrote:

char foobar(void);
int main(void) {
   return  != 0;
}


That still has undefined behavior according to draft C23, which says 
behavior is undefined when foobar is (say) 'memset_explicit' because the 
declaration 'char memset_explicit(void);' disagrees with draft C23's 
declaration of 'memset_explicit'. It doesn't matter whether there's a 
call to 'memset_explicit'. See draft C23 §6.2.7, which says "All 
declarations that refer to the same object or function shall have 
compatible type; otherwise, the behavior is undefined."


If Clang's threatened pickiness were of some real use elsewhere, it 
might be justifiable for default Clang to break Autoconf. But so far we 
haven't seen real-world uses that would justify this pickiness for 
Autoconf's use of 'char memset_explicit(void);'.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Wed, 16 Nov 2022 at 16:34, Michael Matz  wrote:
>
> Hello,
>
> On Wed, 16 Nov 2022, Jonathan Wakely wrote:
>
> > > > Unrelated but I was a bit tempted to ask for throwing in
> > > > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > > > it, but I suppose we don't want the world to burn too much,
> > >
> > > :-)  It's IMHO a bug in the standard that it misses "if any of its
> > > associated headers are included" in the item for reservation of external
> > > linkage identifiers; it has that for all other items about reserved
> > > identifiers in the Library clause.  If that restriction were added you
> > > couldn't justify erroring on the example at hand (because it doesn't
> > > include e.g.  and then printf wouldn't be reserved).  A warning
> > > is of course always okay and reasonable.  As is, you could justify
> > > erroring out, but I too think that would be overzealous.
> >
> >
> > I think that's very intentional and not a defect in the standard.
> >
> > If one TU was allowed to define:
> >
> > void printf() { }
> >
> > and have that compiled into the program, then that would cause
> > unexpected behaviour for every other TU which includes  and
> > calls printf. They would get the non-standard rogue printf.
>
> True.  But suppose the restriction would be added.  I could argue that
> then your problem program (in some other TU) _does_ include the header,
> hence the identifier would have been reserved and so the above definition
> would have been wrong.  I.e. I think adding the restriction wouldn't allow
> the problematic situation either.
>
> I'm aware that the argument would then invoke all the usual problems of
> what constitutes a full program, and if that includes the library even
> when not including headers and so on.  And in any case currently the
> standard does say they're reserved so it's idle speculation anyway :)

Since we're idly speculating anyway ...

I'd always assumed the "if any of its associated headers is included"
meant in the current TU, but it doesn't actually say that. Which does
suggest that I can't use the identifier "assert" anywhere in a program
if any TU in the program includes . Huh.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz via Gcc]

Hello,

On Wed, 16 Nov 2022, Jonathan Wakely wrote:

> > > Unrelated but I was a bit tempted to ask for throwing in
> > > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > > it, but I suppose we don't want the world to burn too much,
> >
> > :-)  It's IMHO a bug in the standard that it misses "if any of its
> > associated headers are included" in the item for reservation of external
> > linkage identifiers; it has that for all other items about reserved
> > identifiers in the Library clause.  If that restriction were added you
> > couldn't justify erroring on the example at hand (because it doesn't
> > include e.g.  and then printf wouldn't be reserved).  A warning
> > is of course always okay and reasonable.  As is, you could justify
> > erroring out, but I too think that would be overzealous.
> 
> 
> I think that's very intentional and not a defect in the standard.
> 
> If one TU was allowed to define:
> 
> void printf() { }
> 
> and have that compiled into the program, then that would cause
> unexpected behaviour for every other TU which includes  and
> calls printf. They would get the non-standard rogue printf.

True.  But suppose the restriction would be added.  I could argue that 
then your problem program (in some other TU) _does_ include the header, 
hence the identifier would have been reserved and so the above definition 
would have been wrong.  I.e. I think adding the restriction wouldn't allow 
the problematic situation either.

I'm aware that the argument would then invoke all the usual problems of 
what constitutes a full program, and if that includes the library even 
when not including headers and so on.  And in any case currently the 
standard does say they're reserved so it's idle speculation anyway :)


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Wed, 16 Nov 2022 at 15:59, Michael Matz  wrote:
>
> Hello,
>
> On Wed, 16 Nov 2022, Sam James wrote:
>
> > Unrelated but I was a bit tempted to ask for throwing in
> > -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at
> > it, but I suppose we don't want the world to burn too much,
>
> :-)  It's IMHO a bug in the standard that it misses "if any of its
> associated headers are included" in the item for reservation of external
> linkage identifiers; it has that for all other items about reserved
> identifiers in the Library clause.  If that restriction were added you
> couldn't justify erroring on the example at hand (because it doesn't
> include e.g.  and then printf wouldn't be reserved).  A warning
> is of course always okay and reasonable.  As is, you could justify
> erroring out, but I too think that would be overzealous.


I think that's very intentional and not a defect in the standard.

If one TU was allowed to define:

void printf() { }

and have that compiled into the program, then that would cause
unexpected behaviour for every other TU which includes  and
calls printf. They would get the non-standard rogue printf.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz via Gcc]

Hello,

On Wed, 16 Nov 2022, Sam James wrote:

> Unrelated but I was a bit tempted to ask for throwing in 
> -Wbuiltin-declaration-mismatch to default -Werror while Clang 16 was at 
> it, but I suppose we don't want the world to burn too much,

:-)  It's IMHO a bug in the standard that it misses "if any of its 
associated headers are included" in the item for reservation of external 
linkage identifiers; it has that for all other items about reserved 
identifiers in the Library clause.  If that restriction were added you 
couldn't justify erroring on the example at hand (because it doesn't 
include e.g.  and then printf wouldn't be reserved).  A warning 
is of course always okay and reasonable.  As is, you could justify 
erroring out, but I too think that would be overzealous.


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 16 Nov 2022, at 15:27, Richard Biener  wrote:
> 
> On Wed, Nov 16, 2022 at 4:02 PM Michael Matz via Gcc  wrote:
>> 
>> Hey,
>> 
>> On Wed, 16 Nov 2022, Alexander Monakov wrote:
>> 
 The idea is so obvious that I'm probably missing something, why autoconf
 can't use that idiom instead.  But perhaps the (historic?) reasons why it
 couldn't be used are gone now?
>>> 
>>> Ironically, modern GCC and LLVM optimize ' != 0' to '1' even at -O0,
>>> and thus no symbol reference remains in the resulting assembly.
>> 
>> Err, right, *head-->table*.
>> Playing with volatile should help:
>> 
>> char foobar(void);
>> char (* volatile ptr)(void);
>> int main(void) {
>>ptr = foobar;
>>return ptr != 0;
>> }
> 
> using printf for foobar this works even with GCC 2.95.2 and with trunk
> and -Wall diagnoses
> 
> t.c:1:6: warning: conflicting types for built-in function 'printf';
> expected 'int(const char *, ...)' [-Wbuiltin-declaration-mismatch]
>1 | char printf(void);
>  |  ^~
> t.c:1:1: note: 'printf' is declared in header ''
>  +++ |+#include 
>1 | char printf(void);
> 
> so without -Werror this should be fine.
> 
Unrelated but I was a bit tempted to ask for throwing in 
-Wbuiltin-declaration-mismatch
to default -Werror while Clang 16 was at it, but I suppose we don't want the 
world to
burn too much, and it's got a very obvious usecase (this one) whereas implicit
func decls are too hard to justify.

> Richard.


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Richard Biener via Gcc]

On Wed, Nov 16, 2022 at 4:02 PM Michael Matz via Gcc  wrote:
>
> Hey,
>
> On Wed, 16 Nov 2022, Alexander Monakov wrote:
>
> > > The idea is so obvious that I'm probably missing something, why autoconf
> > > can't use that idiom instead.  But perhaps the (historic?) reasons why it
> > > couldn't be used are gone now?
> >
> > Ironically, modern GCC and LLVM optimize ' != 0' to '1' even at -O0,
> > and thus no symbol reference remains in the resulting assembly.
>
> Err, right, *head-->table*.
> Playing with volatile should help:
>
> char foobar(void);
> char (* volatile ptr)(void);
> int main(void) {
> ptr = foobar;
> return ptr != 0;
> }

using printf for foobar this works even with GCC 2.95.2 and with trunk
and -Wall diagnoses

t.c:1:6: warning: conflicting types for built-in function 'printf';
expected 'int(const char *, ...)' [-Wbuiltin-declaration-mismatch]
1 | char printf(void);
  |  ^~
t.c:1:1: note: 'printf' is declared in header ''
  +++ |+#include 
1 | char printf(void);

so without -Werror this should be fine.

Richard.

>
>
> Ciao,
> Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz via Gcc]

Hey,

On Wed, 16 Nov 2022, Alexander Monakov wrote:

> > The idea is so obvious that I'm probably missing something, why autoconf 
> > can't use that idiom instead.  But perhaps the (historic?) reasons why it 
> > couldn't be used are gone now?
> 
> Ironically, modern GCC and LLVM optimize ' != 0' to '1' even at -O0,
> and thus no symbol reference remains in the resulting assembly.

Err, right, *head-->table*.
Playing with volatile should help:

char foobar(void);
char (* volatile ptr)(void);
int main(void) {
ptr = foobar;
return ptr != 0;
}


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Alexander Monakov via Gcc]

On Wed, 16 Nov 2022, Michael Matz via Gcc wrote:

> I sympathize, and I would think a compiler emitting an error (not a 
> warning) in the situation at hand (in absence of -Werror) is overly 
> pedantic.  But, could autoconf perhaps avoid the problem?  AFAICS the 
> ac_fn_c_check_func really does only a link test to check for symbol 
> existence, and the perceived problem is that the call statement in main() 
> invokes UB.  So, let's avoid the call then while retaining the access to 
> the symbol?  Like:
> 
> -
> char foobar(void);
> int main(void) {
>   return  != 0;
> }
> -
> 
> No call involved: no reason for compiler to complain.  The prototype decl 
> itself will still be "wrong", but compilers complaining about that (in 
> absence of a pre-existing different prototype, which is avoided by 
> autoconf) seem unlikely.
> 
> Obviously this program will also say "foobar exists" if it's a data 
> symbol, but that's the same with the variant using the call on most 
> platforms (after all it's not run).
> 
> The idea is so obvious that I'm probably missing something, why autoconf 
> can't use that idiom instead.  But perhaps the (historic?) reasons why it 
> couldn't be used are gone now?

Ironically, modern GCC and LLVM optimize ' != 0' to '1' even at -O0,
and thus no symbol reference remains in the resulting assembly.

Alexander

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Michael Matz via Gcc]

Hi,

On Tue, 15 Nov 2022, Paul Eggert wrote:

> On 2022-11-15 11:27, Jonathan Wakely wrote:
> > Another perspective is that autoconf shouldn't get in the way of
> > making the C and C++ toolchain more secure by default.
> 
> Can you cite any examples of a real-world security flaw what would be 
> found by Clang erroring out because 'char foo(void);' is the wrong 
> prototype? Is it plausible that any such security flaw exists?
> 
> On the contrary, it's more likely that Clang's erroring out here would 
> *introduce* a security flaw, because it would cause 'configure' to 
> incorrectly infer that an important security-relevant function is 
> missing and that a flawed substitute needs to be used.
> 
> Let's focus on real problems rather than worrying about imaginary ones.

I sympathize, and I would think a compiler emitting an error (not a 
warning) in the situation at hand (in absence of -Werror) is overly 
pedantic.  But, could autoconf perhaps avoid the problem?  AFAICS the 
ac_fn_c_check_func really does only a link test to check for symbol 
existence, and the perceived problem is that the call statement in main() 
invokes UB.  So, let's avoid the call then while retaining the access to 
the symbol?  Like:

-
char foobar(void);
int main(void) {
  return  != 0;
}
-

No call involved: no reason for compiler to complain.  The prototype decl 
itself will still be "wrong", but compilers complaining about that (in 
absence of a pre-existing different prototype, which is avoided by 
autoconf) seem unlikely.

Obviously this program will also say "foobar exists" if it's a data 
symbol, but that's the same with the variant using the call on most 
platforms (after all it's not run).

The idea is so obvious that I'm probably missing something, why autoconf 
can't use that idiom instead.  But perhaps the (historic?) reasons why it 
couldn't be used are gone now?


Ciao,
Michael.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Bob Friesenhahn]

On Tue, 15 Nov 2022, Sam James wrote:





On 13 Nov 2022, at 00:43, Paul Eggert  wrote:

On 2022-11-11 07:11, Aaron Ballman wrote:

We believe the runtime behavior is sufficiently dangerous to
warrant a conservative view that any call to a function will be a call
that gets executed at runtime, hence a definitive signature mismatch
is something we feel comfortable diagnosing (in some form) by default.


As long as these diagnostics by default do not cause the compiler to exit with nonzero status, we 
should be OK with Autoconf-generated 'configure' scripts. Although there will be problems with 
people who run "./configure CFLAGS='-Werror'", that sort of usage has always been 
problematic and unsupported by Autoconf, so we can simply continue to tell people "don't do 
that".



Is there somewhere in the autoconf docs we actually say this?

I've seen a few instances of folks adding it themselves very
early in their configure scripts (which is a pain for distros
anyway) which then ends up affecting the rest.


Autoconf can help with this issue due to GCC and some other compilers 
providing extensions (usually a pragma) to control warnings while 
compiling the C code.  So configure can run without -Werror, but 
Autoconf could help by providing an easy way for enabling -Werror 
while compiling the application.


Of course the above does not require Autoconf since application 
developers can figure it out by themselves using preprocessor logic 
and knowledge of compiler-specific behavior.


If Autoconf is able to help, then the convoluted code can be in just 
one place (in Autoconf).


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Ben Boeckel via Gcc]

On Tue, Nov 15, 2022 at 15:09:19 -0800, Paul Eggert wrote:
> This may be a hack, but it's a *good* hack. It's likely to fix 
> real-world bugs that would be caused if Clang becomes overly pedantic by 
> default here. And the probability of introducing real-world bugs is 
> essentially zero.

FWIW, CMake uses the same signature for detecting whether a function
exists or not:


https://gitlab.kitware.com/cmake/cmake/-/blob/master/Modules/CheckFunctionExists.c

It's also been like this (without the `void`) for 20 years; the `void`
argument was added 6 years ago:


https://gitlab.kitware.com/cmake/cmake/-/commits/master/Modules/CheckFunctionExists.c

--Ben

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

Can you cite any examples of a real-world security flaw what would be
found by Clang erroring out because 'char foo(void);' is the wrong
prototype? Is it plausible that any such security flaw exists?



CVE-2006-1174 is a possibly reasonable example.


CVE-2006-1174 is not an example, because no prototype 'char foo(void);' 
was involved.[1]


Here's a more concrete proposal that should allay any realistic concerns 
about CVEs. If Clang decides to change its behavior in this area, so 
that Clang starts to exit with nonzero status if the user declares a 
function with a prototype incompatible with the C library, it would be a 
service to real users if Clang merely issues a warning and does *not* 
error out if the declaration is 'char foo();' (current Autoconf) or 
'char foo(void);' (future Autoconf).


This may be a hack, but it's a *good* hack. It's likely to fix 
real-world bugs that would be caused if Clang becomes overly pedantic by 
default here. And the probability of introducing real-world bugs is 
essentially zero.




You can run into that same bug
with use of `-Werror`


That's OK, as the Autoconf documentation already says "don't do that". 
And if Clang exits with nonzero status in the above situation when 
-Werror is specified, that's fine too.



[1] CVE-2006-1174 is not even an example of prototype mismatch, as the 
bad C code did not misdeclare the function in question: it used a POSIX 
standard include file for the function prototype, which is the standard 
way to do things.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Tue, Nov 15, 2022 at 3:27 PM Paul Eggert  wrote:
>
> On 2022-11-15 11:27, Jonathan Wakely wrote:
> > Another perspective is that autoconf shouldn't get in the way of
> > making the C and C++ toolchain more secure by default.
>
> Can you cite any examples of a real-world security flaw what would be
> found by Clang erroring out because 'char foo(void);' is the wrong
> prototype? Is it plausible that any such security flaw exists?

CVE-2006-1174 is a possibly reasonable example. That one is
specifically about the K C open() interface, but the reason the CVE
happened was because a required argument was not passed which is
exactly the kind of problem you'd get from a prototype mismatch.

I think autoconf's usage pattern is well outside of common C coding
practices. Most folks who call a function expect the call to plausibly
happen at runtime (rather than do so just to see if the linker will
complain or not), and I don't know of another context in which anyone
expects calling a function with an incorrect signature will lead to
good outcomes.

> On the contrary, it's more likely that Clang's erroring out here would
> *introduce* a security flaw, because it would cause 'configure' to
> incorrectly infer that an important security-relevant function is
> missing and that a flawed substitute needs to be used.
>
> Let's focus on real problems rather than worrying about imaginary ones.

If the symbol exists and `configure` says it does not, that's the bug
and it's not with the host compiler. You can run into that same bug
with use of `-Werror`, as others have pointed out. So strengthening
warnings doesn't introduce any NEW problems into autoconf, it
exacerbates existing ones.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Tue, Nov 15, 2022 at 2:08 PM Paul Eggert  wrote:
>
> On 2022-11-15 06:50, Jonathan Wakely wrote:
> > Could you clarify what you mean, with a concrete example? Surely as
> > long as errors are reported on stderr and the compiler exits with
> > non-zero status, that's an acceptable way to report errors?
>
> Not if the "error" is harmless as far as Autoconf is concerned, which is
> what led to this thread. The concrete example here is that Autoconf
> needs to check whether a function can be linked to (as opposed to
> checking the function's signature). Clang shouldn't get in the way.

What is harmless to autoconf is a critical flaw in another context.

> In lots of places the C standard says behavior is undefined, even though
> the behavior is fine on the current platform for the intended use. It's
> not just the example we're talking about; adding zero to a null pointer
> is another such example.
>
> In such cases it's OK for Clang to warn, but having Clang exit with
> nonzero status is overkill and counterproductive.

I don't know that this is particularly persuasive -- it effectively
boils down to another variant of "I want to rely on a specific
behavior for something that is UB". I don't think Clang can promise
that we're not going to turn more statically-known UB into errors.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-15 11:27, Jonathan Wakely wrote:

Another perspective is that autoconf shouldn't get in the way of
making the C and C++ toolchain more secure by default.


Can you cite any examples of a real-world security flaw what would be 
found by Clang erroring out because 'char foo(void);' is the wrong 
prototype? Is it plausible that any such security flaw exists?


On the contrary, it's more likely that Clang's erroring out here would 
*introduce* a security flaw, because it would cause 'configure' to 
incorrectly infer that an important security-relevant function is 
missing and that a flawed substitute needs to be used.


Let's focus on real problems rather than worrying about imaginary ones.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Tue, 15 Nov 2022 at 19:08, Paul Eggert  wrote:
>
> On 2022-11-15 06:50, Jonathan Wakely wrote:
> > Could you clarify what you mean, with a concrete example? Surely as
> > long as errors are reported on stderr and the compiler exits with
> > non-zero status, that's an acceptable way to report errors?
>
> Not if the "error" is harmless as far as Autoconf is concerned, which is
> what led to this thread. The concrete example here is that Autoconf
> needs to check whether a function can be linked to (as opposed to
> checking the function's signature). Clang shouldn't get in the way.

Another perspective is that autoconf shouldn't get in the way of
making the C and C++ toolchain more secure by default.

>
> In lots of places the C standard says behavior is undefined, even though
> the behavior is fine on the current platform for the intended use. It's
> not just the example we're talking about; adding zero to a null pointer
> is another such example.
>
> In such cases it's OK for Clang to warn, but having Clang exit with
> nonzero status is overkill and counterproductive.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-15 06:50, Jonathan Wakely wrote:

Could you clarify what you mean, with a concrete example? Surely as
long as errors are reported on stderr and the compiler exits with
non-zero status, that's an acceptable way to report errors?


Not if the "error" is harmless as far as Autoconf is concerned, which is 
what led to this thread. The concrete example here is that Autoconf 
needs to check whether a function can be linked to (as opposed to 
checking the function's signature). Clang shouldn't get in the way.


In lots of places the C standard says behavior is undefined, even though 
the behavior is fine on the current platform for the intended use. It's 
not just the example we're talking about; adding zero to a null pointer 
is another such example.


In such cases it's OK for Clang to warn, but having Clang exit with 
nonzero status is overkill and counterproductive.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Mon, 14 Nov 2022 at 18:15, Paul Eggert  wrote:
>
> On 2022-11-14 04:41, Aaron Ballman wrote:
> > it's generally a problem when autoconf relies on invalid
> > language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can
> break Autoconf.

There's a difference between checking whether an invalid construct
works, where that construct is the subject of the test, and
incidentally relying on invalid language constructs as part of testing
for other things.



> > issues of security
> > like statically known instances of UB
>
> It's fine to report those; I'm not saying don't report them. All I'm
> saying is that Clang should be careful about *how* it reports them.

Could you clarify what you mean, with a concrete example? Surely as
long as errors are reported on stderr and the compiler exits with
non-zero status, that's an acceptable way to report errors? What kind
of changes to error reporting are you saying to be careful with?

If Clang starts to diagnose a given provable-UB case (or any other
construct) as an error instead of a warning, then it seems entirely
correct for autoconf to report that the case does not work. That's the
desired behaviour, isn't it?

What we don't want is for autoconf to start reporting that *other*
things don't work, as a result of autoconf relying on UB or ill-formed
code when trying to check other things like the presence of function
'foo'. And that's why autoconf should avoid using invalid/undefined
code when possible.

>
> At the very least if there are going to be changes in this area, the
> Clang developers should notify Autoconf (and presumably other)
> downstream users of the changes, and provide a supported way to get the
> old behavior for reporting, and give downstream time to adapt.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 15 Nov 2022, at 13:30, Zack Weinberg  wrote:
> 
> On Tue, Nov 15, 2022, at 12:03 AM, Sam James wrote:
>>> On 13 Nov 2022, at 00:43, Paul Eggert  wrote:
>>> 
>>> Although there will be problems with people who run "./configure 
>>> CFLAGS='-Werror'", that sort of usage has always been problematic and 
>>> unsupported by Autoconf, so we can simply continue to tell people "don't do 
>>> that".
>>> 
>> 
>> Is there somewhere in the autoconf docs we actually say this?
>> 
>> I've seen a few instances of folks adding it themselves very
>> early in their configure scripts (which is a pain for distros
>> anyway) which then ends up affecting the rest.
> 
> It's mentioned in the NEWS entry for 2.70: 
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/NEWS#n170

Thanks, sorry, I didn't think to check NEWS. This is good enough for me to link 
folks to, anyway, for the time being.

> Note that there have been bug reports for cases where running builds with 
> more warnings than configure uses (and I think also -Werror), means that 
> configure checks spuriously succeed (i.e. configure reports that something is 
> available, but then you get compiler errors on the code that tries to use 
> it).  I can't remember any concrete examples right now, though.

I can totally imagine that, don't stress about the examples.



signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

On Tue, Nov 15, 2022, at 12:03 AM, Sam James wrote:
>> On 13 Nov 2022, at 00:43, Paul Eggert  wrote:
>> 
>> Although there will be problems with people who run "./configure 
>> CFLAGS='-Werror'", that sort of usage has always been problematic and 
>> unsupported by Autoconf, so we can simply continue to tell people "don't do 
>> that".
>> 
>
> Is there somewhere in the autoconf docs we actually say this?
>
> I've seen a few instances of folks adding it themselves very
> early in their configure scripts (which is a pain for distros
> anyway) which then ends up affecting the rest.

It's mentioned in the NEWS entry for 2.70: 
https://git.savannah.gnu.org/cgit/autoconf.git/tree/NEWS#n170

It should be discussed in the actual manual as well, but I've been reluctant to 
add anything about warnings to the manual as long as Autoconf proper doesn't 
have any support for controlling warnings.

Note that there have been bug reports for cases where running builds with more 
warnings than configure uses (and I think also -Werror), means that configure 
checks spuriously succeed (i.e. configure reports that something is available, 
but then you get compiler errors on the code that tries to use it).  I can't 
remember any concrete examples right now, though.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 13 Nov 2022, at 00:43, Paul Eggert  wrote:
> 
> On 2022-11-11 07:11, Aaron Ballman wrote:
>> We believe the runtime behavior is sufficiently dangerous to
>> warrant a conservative view that any call to a function will be a call
>> that gets executed at runtime, hence a definitive signature mismatch
>> is something we feel comfortable diagnosing (in some form) by default.
> 
> As long as these diagnostics by default do not cause the compiler to exit 
> with nonzero status, we should be OK with Autoconf-generated 'configure' 
> scripts. Although there will be problems with people who run "./configure 
> CFLAGS='-Werror'", that sort of usage has always been problematic and 
> unsupported by Autoconf, so we can simply continue to tell people "don't do 
> that".
> 

Is there somewhere in the autoconf docs we actually say this?

I've seen a few instances of folks adding it themselves very
early in their configure scripts (which is a pain for distros
anyway) which then ends up affecting the rest.


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Mon, Nov 14, 2022 at 1:14 PM Paul Eggert  wrote:
>
> On 2022-11-14 04:41, Aaron Ballman wrote:
> > it's generally a problem when autoconf relies on invalid
> > language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can
> break Autoconf.
>
> > issues of security
> > like statically known instances of UB
>
> It's fine to report those; I'm not saying don't report them. All I'm
> saying is that Clang should be careful about *how* it reports them.
>
> At the very least if there are going to be changes in this area, the
> Clang developers should notify Autoconf (and presumably other)
> downstream users of the changes, and provide a supported way to get the
> old behavior for reporting, and give downstream time to adapt.

Definitely agreed about the communication aspects! I mentioned this upthread:

FWIW, we're working on improving communication
about potentially disruptive changes to Clang, so you might want to
consider either subscribing to the clang-vendors code review group at
https://reviews.llvm.org/project/members/113/ (if you want to be
involved in code review before things land) or the Announcements
discourse channel at https://discourse.llvm.org/c/announce/ (if you
want to be notified after something lands but before Clang ships).

One other thing we've done recently is starting to call out
potentially disruptive changes in the release notes as well:
https://clang.llvm.org/docs/ReleaseNotes.html#potentially-breaking-changes
-- but this is more for notifying folks after a release goes out, so
one of the other approaches is more proactive if the goal is to alert
Clang developers to serious deployment problems before we ship.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Florian Weimer via Gcc]

* Paul Eggert:

> On 2022-11-14 04:41, Aaron Ballman wrote:
>> it's generally a problem when autoconf relies on invalid
>> language constructs
>
> Autoconf *must* rely on invalid language constructs, if only to test
> whether the language constructs work. And Clang therefore must be 
> careful about how it diagnoses invalid constructs. Clang shouldn't
> willy-nilly change the way it reports invalid constructs, as that can 
> break Autoconf.

This is only true for the status quo.  We could finally band together
and define an interface between autoconf and the toolchain that avoids
feature probing through source code fragments for common cases.  It
might make configure scripts to run quite a bit faster, too.

That being said, system compilers need to be careful when turning
warnings into errors by default, but that doesn't mean we should never
do such changes, particularly when we know based on interactions with
programmers that the warnings are not sufficient for avoiding wasted
time.

Thanks,
Florian

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-14 04:41, Aaron Ballman wrote:

it's generally a problem when autoconf relies on invalid
language constructs


Autoconf *must* rely on invalid language constructs, if only to test 
whether the language constructs work. And Clang therefore must be 
careful about how it diagnoses invalid constructs. Clang shouldn't 
willy-nilly change the way it reports invalid constructs, as that can 
break Autoconf.



issues of security
like statically known instances of UB


It's fine to report those; I'm not saying don't report them. All I'm 
saying is that Clang should be careful about *how* it reports them.


At the very least if there are going to be changes in this area, the 
Clang developers should notify Autoconf (and presumably other) 
downstream users of the changes, and provide a supported way to get the 
old behavior for reporting, and give downstream time to adapt.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Sat, Nov 12, 2022 at 7:43 PM Paul Eggert  wrote:
>
> On 2022-11-11 07:11, Aaron Ballman wrote:
> > We believe the runtime behavior is sufficiently dangerous to
> > warrant a conservative view that any call to a function will be a call
> > that gets executed at runtime, hence a definitive signature mismatch
> > is something we feel comfortable diagnosing (in some form) by default.
>
> As long as these diagnostics by default do not cause the compiler to
> exit with nonzero status, we should be OK with Autoconf-generated
> 'configure' scripts. Although there will be problems with people who run
> "./configure CFLAGS='-Werror'", that sort of usage has always been
> problematic and unsupported by Autoconf, so we can simply continue to
> tell people "don't do that".

That's good to know, but is a problem more generally -- we are
strengthening more diagnostics to be warnings that are treated as an
error by default. This gives our users the best experience in terms of
diagnostic behavior -- they're clearly alerted to serious issues in
their code (either issues of conformance, like with use of implicit
int or implicit function decls in C99 or later, or issues of security
like statically known instances of UB), but they still have the chance
to downgrade the diagnostic back into a warning (good as a temporary
solution to start migrating code) or disable the diagnostic entirely
(good if you plan to never update your compiler version but otherwise
not recommended). Some of these diagnostics are expected to change to
be error-only diagnostics in the future, so this strengthening helps
to set user expectations as well.

That's why it's generally a problem when autoconf relies on invalid
language constructs -- it creates a tension between the autoconf uses
and improving the C ecosystem.  The autoconf uses aren't always
unreasonable, but are very much a special case scenario compared to
general C development. I suspect that as the security posture of the C
language and its implementations improves in response to recent
concerns around suitability of the language
(https://media.defense.gov/2022/Nov/10/2003112742/-1/-1/0/CSI_SOFTWARE_MEMORY_SAFETY.PDF),
this tension will come up more frequently.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-11 07:11, Aaron Ballman wrote:

Clang doesn't require such a linker (we work with various system linkers).


As long as the system linkers continue to work as they have 
traditionally worked, we're fine.



the frontend perspective, we can't tell the difference between
"trust me this is safe because it never gets executed" and "this is a
CVE"


If some system linker ever attempts to reject links with mismatched 
signatures, Autoconf-generated code will need to have a way to shut that 
off. I hope Clang maintainers can be cajoled into supporting that, if 
the time comes. Perhaps there can be a #pragma, or a compile-time 
option, to do that.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-11 07:11, Aaron Ballman wrote:

We believe the runtime behavior is sufficiently dangerous to
warrant a conservative view that any call to a function will be a call
that gets executed at runtime, hence a definitive signature mismatch
is something we feel comfortable diagnosing (in some form) by default.


As long as these diagnostics by default do not cause the compiler to 
exit with nonzero status, we should be OK with Autoconf-generated 
'configure' scripts. Although there will be problems with people who run 
"./configure CFLAGS='-Werror'", that sort of usage has always been 
problematic and unsupported by Autoconf, so we can simply continue to 
tell people "don't do that".

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Wookey  writes:
> On 2022-11-10 19:08 +0100, Florian Weimer wrote:
>> based on a limited attempt to get this fixed about three years
>> ago, I expect that many of the problematic packages have not had their
>> configure scripts regenerated using autoconf for a decade or more.  This
>> means that as an autoconf maintainer, you unfortunately won't be able to
>> help us much.
>
> We changed the default in debian to re-autoconf on build a few years
> ago precisely so that changes in the tools (particularly new arch
> support) were picked up even by code that was not being re-released or
> released without autofoo updates.  This has worked remarkably well.
>
> So changes in the tools will get used, at least in that context, which
> includes a fairly hefty pile of crufty old code. I have no feeling for
> how many packages are actually affected by this. Is there a quick way to test?

In the run-up to Autoconf 2.70 I got Lucas to do a Debian archive
rebuild and mass bug filing, so, look through Debian packages for
patches being carried for configure.ac and/or bundled .m4 files?

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Wookey]

On 2022-11-10 19:08 +0100, Florian Weimer wrote:
> * Zack Weinberg via Gcc:
> 
> > It’s come to my attention (via https://lwn.net/Articles/913505/ and
> > https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> > Clang both plan to disable several “legacy” C language features by
> > default in a near-future release (GCC 14, Clang 16) (see the Fedora
> > wiki link for a list).

> based on a limited attempt to get this fixed about three years
> ago, I expect that many of the problematic packages have not had their
> configure scripts regenerated using autoconf for a decade or more.  This
> means that as an autoconf maintainer, you unfortunately won't be able to
> help us much.

We changed the default in debian to re-autoconf on build a few years
ago precisely so that changes in the tools (particularly new arch
support) were picked up even by code that was not being re-released or
released without autofoo updates.  This has worked remarkably well.

So changes in the tools will get used, at least in that context, which
includes a fairly hefty pile of crufty old code. I have no feeling for
how many packages are actually affected by this. Is there a quick way to test?

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Sam James  writes:

>> On 12 Nov 2022, at 03:40, Zack Weinberg  wrote:
>> This is definitely more work than I can see myself doing on a volunteer
>> basis, but a 2.69.1 patch release — nothing that’s not already on trunk,
>> cherry pick the changes needed to support the newer compilers (and
>> also newer Perl and Bash and M4) is a thing that could happen.
>
> I didn't want to ask you to do this because I felt fortunate enough
> you were volunteering to handle 2.72, but this would indeed be a help,
> because then I won't have to try persuade people they should totally upgrade,
> and it should happen naturally enough with distro upgrades.

To be clear, I am *not* volunteering to do this.  It would be
significantly more work than I can carve out the time for.

Whoever does this will need to go through the *entire* list of changes
since the original 2.69 release, to find all of the changes that improve
compatibility with newer versions of tools, disentangle each from any
other changes applied in the same commit, and then do a whole lot of
testing.  It’s tedious, and whoever does it should get paid to do it.
I’m guessing at least one full 40-hour week of work, and you should
budget for three times that, going by how much more work the 2.70
release was than we anticipated.

I can *advise* anyone who takes on the job, and review their changes,
but that’s it.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Paul Eggert  writes:

> On 2022-11-10 19:33, Zack Weinberg wrote:
>
>> It would be relatively easy for me to take a couple hours this
>> weekend and put out a 2.72 release with everything that's already in
>> trunk and nothing else.  Anyone have reasons I _shouldn't_ do that?
>
> I don't have anything other than some doc updates, which I just now
> installed (see attached).

couple of notes below

>>> Note that in autoconf git, we've also got
>>> https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=f6657256a37da44c987c04bf9cd75575dfca3b60
>>> which is going to affect time_t efforts too
>> I have not been following the y2038 work closely.  Is it going to
>> affect things in a good way or a bad way??
>
> Both. […]

(let’s leave this discussion to the other thread)

> -  and ‘false’ work anyway.  This is for compatibility with C 2023 and
> +  and ‘false’ work anyway.  This is for compatibility with C23 and

Why are you changing four-digit years to two-digit years?  I know you’re
old enough to remember Y2K :-)

>  All other headers, including all headers
>  from later revisions of the C standard, need to be tested for
> +if your program is intended to be portable to C89
>  (@pxref{Header Files}).

I don’t understand what this addition is supposed to mean.  Not all
“modern” systems supply all of the C99 headers, even now.

> -The C standard says a call @code{malloc (0)} is implementation
> +The C standard says a successful call @code{malloc (0)} is implementation
>  dependent.  It can return either @code{NULL} or a new non-null pointer.

Style nit: how can a _call_ be implementation dependent?  Suggest
something like “The C standard says that _the result of_ a successful
call _to_ @code{malloc (0)} is implementation dependent.” (underscores
just to mark additions, not part of suggested text)

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 12 Nov 2022, at 00:53, Paul Eggert  wrote:
> 
> On 2022-11-11 15:25, Sam James wrote:
>> That's not a judgement on whether the changes will ultimately remain in 
>> autoconf, I'm just
>> hesitant to allow a discussion I've kicked off to derail something that we 
>> were planning
>> on doing anyway.
>> What do you think?
> 
> I'm hesitant to do that partly because the changes to _TIME_BITS are already 
> released in multiple packages and need to be dealt with, regardless of 
> whether they're backed out of Autoconf. This is because they've been in 
> Gnulib since July and several packages based on these Gnulib changes have 
> been released since then. Current Gnulib assumes these changes will appear in 
> the next Autoconf release; if that's not true, we'll need to upgrade Gnulib 
> and in the meantime the other packages released since July would still have 
> the changes whatever we do with Gnulib and/or Autoconf.
> 
> Since distros need to deal with the issue anyway, regardless of what Autoconf 
> and/or Gnulib does, I don't see why backing the changes out of Autoconf will 
> help all that much.
> 
> It should pretty easy for a distro to say "hold on, I don't want 64-bit 
> time_t yet" without changing either Autoconf or Gnulib so if you want to go 
> that route please feel free to do so.

The fact it's already shipped in gnulib & that the "real problem" is in glibc 
IMO means that I don't feel
strongly about reverting it.

You're right that distros have the toggle so as long as we mention this 
prominently enough in NEWS,
I don't have a strong objection to master being released as-is.


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Joseph Myers]

On Fri, 11 Nov 2022, Zack Weinberg via Gcc wrote:

> These are also a trip hazard for novices, and the only way to turn them
> off is with -std=cXX, which also turns another trip hazard (trigraphs)
> *on*… so yeah, anything you can do to help speed up their removal, I
> think it’d be worthwhile.

As of GCC 13, -std=c2x will disable trigraphs, since they've been removed 
from C2x.

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 12 Nov 2022, at 03:40, Zack Weinberg  wrote:
> 
> Florian Weimer  writes:
>> based on a limited attempt to get this fixed about three years
>> ago, I expect that many of the problematic packages have not had their
>> configure scripts regenerated using autoconf for a decade or more.  This
>> means that as an autoconf maintainer, you unfortunately won't be able to
>> help us much.
> 
> I’m sadly not surprised.
> 
> This is definitely more work than I can see myself doing on a volunteer
> basis, but a 2.69.1 patch release — nothing that’s not already on trunk,
> cherry pick the changes needed to support the newer compilers (and
> also newer Perl and Bash and M4) is a thing that could happen.

I didn't want to ask you to do this because I felt fortunate enough
you were volunteering to handle 2.72, but this would indeed be a help,
because then I won't have to try persuade people they should totally upgrade,
and it should happen naturally enough with distro upgrades.

If you are willing, that would be welcome.

Of course, we'll have to go lobby them, but that is what it is :)



signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Florian Weimer  writes:
> based on a limited attempt to get this fixed about three years
> ago, I expect that many of the problematic packages have not had their
> configure scripts regenerated using autoconf for a decade or more.  This
> means that as an autoconf maintainer, you unfortunately won't be able to
> help us much.

I’m sadly not surprised.

This is definitely more work than I can see myself doing on a volunteer
basis, but a 2.69.1 patch release — nothing that’s not already on trunk,
cherry pick the changes needed to support the newer compilers (and
also newer Perl and Bash and M4) is a thing that could happen.

> Thanks, these changes are going to be helpful to get a clean run from
> our Fedora tester.

Autoconf’s own test suite is sadly not very thorough.  If you find more
problems I will prioritize them.

> Once you include the header, you also need to know function parameters,
> otherwise you won't be able to form a valid call.

You can assign to a function pointer variable if you know the complete
type signature, which is desirable for other reasons (see reply to Rich).
Needing to know how to form argument *values* could be much more trouble,
but I don’t think it should be necessary.

>> p.s. GCC and Clang folks: As long as you’re changing the defaults out
>> from under people,
>
> Hmph, I wouldn't frame it this way.  We are aware of GCC's special role
> as the system compiler.  We're trying to upstream the changes to sources
> before flipping the compiler default.  (The burden of being a system
> compiler and all that.)  A 25-year transition period evidently wasn't
> enough, so some effort is still needed.  We may conclude that removing
> these extensions is too costly even in 2024.

I didn’t mean to imply that I disliked any of the changes.  In fact,
with my day job (CS professor) hat on, I am quite looking forward to not
having to warn the kids about these legacy features anymore (we don’t
_teach_ them, but they inevitably use them by accident, particularly
implicit function declarations, and then get confused because ‘cc’ with
no -W options doesn’t catch the mistake).

>> can you please also remove the last few predefined
>> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
>> -std=gnuXX modes?
>
> That's a good point, I'll think about how we can instrument GCC to
> support tracking that.  We won't be able help with -Darm on the Fedora
> side (the AArch64 port doesn't have that, and there's no longer a Fedora
> 32-bit Arm port), but -Dlinux and -Dunix we can help with.

These are also a trip hazard for novices, and the only way to turn them
off is with -std=cXX, which also turns another trip hazard (trigraphs)
*on*… so yeah, anything you can do to help speed up their removal, I
think it’d be worthwhile.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Rich Felker  writes:
> On Thu, Nov 10, 2022 at 12:16:20PM -0500, Zack Weinberg wrote:
>> The biggest remaining (potential) problem, that I’m aware of, is that
>> AC_CHECK_FUNC unconditionally declares the function we’re probing for
>> as ‘char NAME (void)’, and asks the compiler to call it with no
>> arguments, regardless of what its prototype actually is.
…
> Thanks for bringing this up. It is very important and I am very much
> in favor of making these changes and doing it in a way that existing
> broken and unmaintained software can be made to work just by
> re-generating configure scripts with up-to-date autoconf, even if that
> means hard-coding a list of headers needed to get the right
> declarations and automatically pulling them in.

Right.  In principle, I think AC_CHECK_FUNCS([symbol]), where ‘symbol’ is
in either ISO C or in XSI, could be made to do a check of the form you
suggest at the end of https://ewontfix.com/13/

#include 
#include 
int main()
{
double (*p)(const char *, char **, locale_t) = strtod_l;
}

It’s “just” a matter of compiling that big list of headers and expected
function signatures.  I’d also want to do something to ensure that this
assignment is not optimized out, so the linker has to process an
undefined reference to the symbol.

For symbols that are not covered by the built-in list, we could require
people to indicate the necessary headers and signature somehow.
Hypothetical notation

AC_CHECK_FUNCS([
[argp_parse, [argp.h],
   [error_t], [const struct argp *, int, char **,
   unsigned int, int *, void *]]
])

Note that this still isn’t perfect; if some system provides a function
with an identical type signature, but different *semantics*, to the one
the application wants, no compilation test can detect that.  Autoconf’s
not going to step away from its “avoid compile-and-run tests, that
breaks cross compilation” stance.

> I've been writing/complaining about autoconf doing this wrong for
> decades, with the best writeup around 9 years ago at
> https://ewontfix.com/13/. Part of the reason is that this has bitten
> musl libc users over and over again due to configure finding symbols
> that were intended only as ABI-compat and trying to use them (without
> declarations) at the source level

I vaguely recall some cases where this bit glibc and Apple’s libc as
well.

In principle, you’re supposed to be able to declare some ISO C functions
yourself, e.g.

extern int printf(const char *, ...);
int main(void) {
printf("hello world\n");
}

is strictly conforming per C99, but this bypasses any symbol renaming
applied by stdio.h.

> What I'd like to see happen is complete deprecation of the autoconf
> link-only tests

Do you have a concrete list of documented Autoconf macros that you would
like to see deprecated for this reason?

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

Nick Bowler  writes:
> My gut feeling is that Autoconf should just determine the necessary
> options to get compatible behaviour out of these modern compilers, at
> least for the purpose of running configure tests.  For example, Autoconf
> should probably build the AC_CHECK_FUNC programs using gcc's
> -fno-builtin option

I fear this will cause more problems than it solves.  Messing with
compiler options inside a configure script has a track record of
clashing with “outer” build tools that expect to be able to dictate the
options.

> It saddens me to see so much breakage happening in "modern C", a
> language that has (until now) a long history of new language features
> being carefully introduced to avoid these sort of problems.

I don’t exactly _disagree_ with this.  Quite a few of the compatibility-
breaking changes going into C2x (promoting ‘bool’ to a true keyword, for
instance, and changing the meaning of an empty argument list in a
function declaration) strike me as unnecessary churn.  However, the
specific set of changes that are under discussion right now—removal of
implicit function declarations, implicit int, and old-style function
definitions from the _default_ language accepted by C compilers—I’m very
much in favor of, because they make life significantly easier for people
writing _new_ code.  It’s not healthy for a language to always
prioritize old code over new code.

(Yes, you _can_ opt in to all three of those changes now, but you have
to type a bunch of -W options.  With my day job hat on, I am very much
looking forward to a day where ‘cc test.c’ errors out on implicit
function declarations, because then I won’t have to _explain_ implicit
function declarations, and why they are dangerous, to my students
anymore.)

>> p.s. GCC and Clang folks: As long as you’re changing the defaults out
>> from under people, can you please also remove the last few predefined
>> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
>> -std=gnuXX modes?
>
> Meh, even though these macros are a small thing I don't accept the
> "things are breaking anyway so let's break even more things" attitude.

Getting rid of these is another change that will make life easier for
people writing new code.

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-11 15:25, Sam James wrote:

That's not a judgement on whether the changes will ultimately remain in 
autoconf, I'm just
hesitant to allow a discussion I've kicked off to derail something that we were 
planning
on doing anyway.

What do you think?


I'm hesitant to do that partly because the changes to _TIME_BITS are 
already released in multiple packages and need to be dealt with, 
regardless of whether they're backed out of Autoconf. This is because 
they've been in Gnulib since July and several packages based on these 
Gnulib changes have been released since then. Current Gnulib assumes 
these changes will appear in the next Autoconf release; if that's not 
true, we'll need to upgrade Gnulib and in the meantime the other 
packages released since July would still have the changes whatever we do 
with Gnulib and/or Autoconf.


Since distros need to deal with the issue anyway, regardless of what 
Autoconf and/or Gnulib does, I don't see why backing the changes out of 
Autoconf will help all that much.


It should pretty easy for a distro to say "hold on, I don't want 64-bit 
time_t yet" without changing either Autoconf or Gnulib so if you want to 
go that route please feel free to do so.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 11 Nov 2022, at 03:33, Zack Weinberg  wrote:
> 
> On Thu, Nov 10, 2022, at 10:08 PM, Sam James wrote:
>>> On 10 Nov 2022, at 21:10, Michael Orlitzky  wrote:
>>> While everyone else is discussing big ideas, it would be helpful for me
>>> personally if autoconf just made a release with the latest bugfixes.
>> 
>> Before I dive into the rest of this thread: yes, this is one of
>> my main thoughts on the matter. Autoconf has a huge network
>> effect problem and letting the existing fixes start to propagate
>> would be most helpful.
> 
> It would be relatively easy for me to take a couple hours this weekend and 
> put out a 2.72 release with everything that's already in trunk and nothing 
> else.  Anyone have reasons I _shouldn't_ do that?
> 
>> Note that in autoconf git, we've also got
>> https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=f6657256a37da44c987c04bf9cd75575dfca3b60
>> which is going to affect time_t efforts too
> 
> I have not been following the y2038 work closely.  Is it going to affect 
> things in a good way or a bad way??
> 

Back to the original thread: I suspect it might be a better idea to 
(temporarily) revert the two changes
and omit it from 2.72 to allow the other changes to get out.

That's not a judgement on whether the changes will ultimately remain in 
autoconf, I'm just
hesitant to allow a discussion I've kicked off to derail something that we were 
planning
on doing anyway.

What do you think?


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Thu, Nov 10, 2022 at 4:05 PM Paul Eggert  wrote:
>
> On 2022-11-10 10:19, Aaron Ballman wrote:
> > In terms of the Clang side of things, I don't think we've formed any
> > sort of official stance on how to handle that yet. It's UB (you can
> > declare the C standard library interface without UB but calling any
> > function with a mismatched signature is UB)
>
> The Autoconf-generated code is never executed, so this is not a runtime
> issue; it's merely an issue of undefined behavior during translation.

FWIW, the only thing the (Clang) compiler is aware of is translation.
So from the frontend perspective, we can't tell the difference between
"trust me this is safe because it never gets executed" and "this is a
CVE". We believe the runtime behavior is sufficiently dangerous to
warrant a conservative view that any call to a function will be a call
that gets executed at runtime, hence a definitive signature mismatch
is something we feel comfortable diagnosing (in some form) by default.

> A
> problem could occur with a picky compiler or linker that rejects modules
> with mismatched function type declarations. Does Clang do that, or
> require or use such a linker? If not, there's no practical problem here.
> If so, it'd be helpful if Clang continued to support its traditional
> behavior that doesn't reject Autoconf's test cases, and for this to be
> the default.

Clang doesn't require such a linker (we work with various system linkers).

> Autoconf arose because one cannot ask something like "Can I call the
> renameat2 function with its usual signature?" in standard C code and one
> must escape into something like the shell to handle such questions. C23
> and GCC and Clang have added a few features to answer such questions,
> such as __has_include. But these features don't go nearly far enough.
> For example, __has_include tells me only whether the include file
>  exists; it won't tell me whether I can successfully include
> , or whether  will declare the function 'bar', or whether
> 'bar' will have a signature compatible with my code's calls to 'bar'.
>
> If I could request a single thing from the C23/GCC/Clang side, I'd ask
> for better facilities to be able to ask such questions within C code,
> without using the shell. Then a good chunk of Autoconf could dry up and
> blow away.
>
> I realize that I'm asking for a lot. For example, a traditional
> implementation cannot answer the renameat2 question without consulting
> the linker. That being said, this ability is essential for modular
> programming at the low level, and if compilers don't provide this
> ability Autoconf will simply have to do the best it can, regardless of
> whether it generates source code that relies on undefined behavior.

This would be challenging for an implementation like Clang where we
work with an arbitrary C runtime library (which may be dynamically
loaded on the target machine) and an arbitrary linker.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 10 Nov 2022, at 17:16, Zack Weinberg  wrote:
> 
> I’m the closest thing Autoconf has to a lead maintainer at present.
> 
> It’s come to my attention (via https://lwn.net/Articles/913505/ and
> https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> Clang both plan to disable several “legacy” C language features by
> default in a near-future release (GCC 14, Clang 16) (see the Fedora
> wiki link for a list).  I understand that this change potentially
> breaks a lot of old dusty code, and in particular that
> Autoconf-generated configure scripts use constructs that may *silently
> give the wrong answer to a probe* when a stricter compiler is in use.

Thank you for asking. The fixes in git get us there, I think, as far
as you can, with the exception of the stuff you (and I) mention below.

> 
> [...]
> 
> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.
> 
> How important do you think it is for this to be fixed?
> 
> Are there any other changes you would like to see in a near-future
> Autoconf 2.72 in order to make this transition easier?

This might be a WONTFIX but let me mention it just for
the record:
1. AC_CHECK_FUNCS is, as you've noticed, very noisy.

I would support having a hardcoded list for certain CHOSTs
as Rich suggests to reduce noise, but I don't think we can
do this accurately very quickly.

I think for Gentoo's efforts, I might just build up a set
of cache variables for glibc/musl on each arch to
reduce the noise in logs. But that's time consuming
and brittle still, so I'm not sure.

(Note that Gentoo and Fedora are taking two complementary
but different approaches here:
we're diffing config.logs and other compiler
output, in addition to build logs, while Florian for Fedora
Is building a list of functions which *we know* are available
In a specific environment and patching gcc to spit out
logs when something in said list is missing. This mitigates
noise for things like functions in libbsd, which I'm finding
a bit of a pain.)

I'll see what others say.

2. I often have to set the following cache variables to
reduce noise in logs:
* ac_cv_c_undeclared_builtin_options="none needed"
* ac_cv_header_sys_types_h_makedev=no
* gl_cv_compiler_check_decl_option="-Werror=implicit-function-declaration" 
(obviously this is gnulib)
* gl_cv_minmax_in_limits_h=no

I don't know if we can do anything to make these tests smarter
or just leave it as-is. It's fine if we can't, as exporting the cache
vars is not a bad workaround for us when doing testing.

> 
> zw
> 
> p.s. GCC and Clang folks: As long as you’re changing the defaults out
> from under people, can you please also remove the last few predefined
> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
> -std=gnuXX modes?

I support this as well. This kind of thing has led to endless
bugs in userland, see https://reviews.llvm.org/D137511.



signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 11 Nov 2022, at 03:33, Zack Weinberg  wrote:
> 
> On Thu, Nov 10, 2022, at 10:08 PM, Sam James wrote:
>>> On 10 Nov 2022, at 21:10, Michael Orlitzky  wrote:
>>> While everyone else is discussing big ideas, it would be helpful for me
>>> personally if autoconf just made a release with the latest bugfixes.
>> 
>> Before I dive into the rest of this thread: yes, this is one of
>> my main thoughts on the matter. Autoconf has a huge network
>> effect problem and letting the existing fixes start to propagate
>> would be most helpful.
> 
> It would be relatively easy for me to take a couple hours this weekend and 
> put out a 2.72 release with everything that's already in trunk and nothing 
> else.  Anyone have reasons I _shouldn't_ do that?
> [...]
> 
> I have not been following the y2038 work closely.  Is it going to affect 
> things in a good way or a bad way??

I've started a discussion on libc-alpha about this, but I think it depends on 
how you view
the migration. I've come to the conclusion it's probably good but only after 
thinking
about it a lot. I wish it'd been discussed on the mailing lists first, as it's 
not obvious
that it's okay, and I'm not sure if others will even share my view.

Let's have the conversation there as it'll be easier to track.

Thanks for prompting me to write this up finally.


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

On Thu, Nov 10, 2022, at 10:08 PM, Sam James wrote:
>> On 10 Nov 2022, at 21:10, Michael Orlitzky  wrote:
>> While everyone else is discussing big ideas, it would be helpful for me
>> personally if autoconf just made a release with the latest bugfixes.
>
> Before I dive into the rest of this thread: yes, this is one of
> my main thoughts on the matter. Autoconf has a huge network
> effect problem and letting the existing fixes start to propagate
> would be most helpful.

It would be relatively easy for me to take a couple hours this weekend and put 
out a 2.72 release with everything that's already in trunk and nothing else.  
Anyone have reasons I _shouldn't_ do that?

> Note that in autoconf git, we've also got 
> https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=f6657256a37da44c987c04bf9cd75575dfca3b60
> which is going to affect time_t efforts too

I have not been following the y2038 work closely.  Is it going to affect things 
in a good way or a bad way??

zw

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Sam James via Gcc]

> On 10 Nov 2022, at 21:10, Michael Orlitzky  wrote:
> 
> On Thu, 2022-11-10 at 12:16 -0500, Zack Weinberg wrote:
>> 
>> Nobody has a whole lot of time to work on Autoconf at present, but I
>> would like to ask, anyway, what Autoconf could potentially do to make
>> this transition easier.
> 
> While everyone else is discussing big ideas, it would be helpful for me
> personally if autoconf just made a release with the latest bugfixes.
> 
> I report these compatibility issues upstream, and it's awkward to have
> to tell the maintainers that they must cherry-pick a git commit,
> rebuild autoconf, and then autoreconf their project before they can
> even even think about exporting CFLAGS="-Werror=..." to build-test
> their project.
> 
> 

Before I dive into the rest of this thread: yes, this is one of
my main thoughts on the matter. Autoconf has a huge network
effect problem and letting the existing fixes start to propagate
would be most helpful.

Like it or not (and I don't like it), various software tries
to jam in -Werror (and sometimes -pedantic-errors, see rsync!) to their
configure scripts which means even prototype issues end up
causing problems.

Note that in autoconf git, we've also got 
https://git.savannah.gnu.org/cgit/autoconf.git/commit/?id=f6657256a37da44c987c04bf9cd75575dfca3b60
which is going to affect time_t efforts too, but that's
for another thread on libc-alpha at some point
when I want to bring up some issues.


signature.asc
Description: Message signed with OpenPGP

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Florian Weimer via Gcc]

* Rich Felker:

> I've been writing/complaining about autoconf doing this wrong for
> decades, with the best writeup around 9 years ago at
> https://ewontfix.com/13/. Part of the reason is that this has bitten
> musl libc users over and over again due to configure finding symbols
> that were intended only as ABI-compat and trying to use them (without
> declarations) at the source level, leading to various messes, some of
> which we're only just extricating ourselves from now:
>
> https://git.musl-libc.org/cgit/musl/commit/?id=246f1c811448f37a44b41cd8df8d0ef9736d95f4
> https://git.musl-libc.org/cgit/musl/commit/?id=25e6fee27f4a293728dd15b659170e7b9c7db9bc

glibc and autoconf use a __stub_ macro handshake to achieve this.  It
allows you declare and define functions and still have the corresponding
autoconf test fail.  This facility does not depend on GNU-style symbol
versioning.

> But aside from issues like this, just the fact that autoconf was
> precluding making -Werror=implicit-function-declaration default must
> have wasted tens if not hundreds of thousands of human hours debugging
> broken builds.
>
> What I'd like to see happen is complete deprecation of the autoconf
> link-only tests -- only keeping them for use by legacy unmaintained
> projects in the form where they actually implicitly include the right
> header and test compile and link using that.

It may be technically the right thing to do, but for tests that check
for more than just symbol presence, we might need something that has a
third failure mode—“cannot tell”—beyond “feature is there” and “feature
is missing”.  With a binary approach, it is just too easy to produce a
broken test that fails for the wrong reasons.  With compile-time
testing, such unexpected failures become more likely.  Maybe something
like “if strerror_r exists, it must be usable like this, or usable like
that, but not neither or both at the same time”.  Doing this properly
probably needs toolchain support (which is more likely to become
generally available in C++ than in C).

Thanks,
Florian

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-10 10:19, Aaron Ballman wrote:

In terms of the Clang side of things, I don't think we've formed any
sort of official stance on how to handle that yet. It's UB (you can
declare the C standard library interface without UB but calling any
function with a mismatched signature is UB)


The Autoconf-generated code is never executed, so this is not a runtime 
issue; it's merely an issue of undefined behavior during translation. A 
problem could occur with a picky compiler or linker that rejects modules 
with mismatched function type declarations. Does Clang do that, or 
require or use such a linker? If not, there's no practical problem here. 
If so, it'd be helpful if Clang continued to support its traditional 
behavior that doesn't reject Autoconf's test cases, and for this to be 
the default.


Autoconf arose because one cannot ask something like "Can I call the 
renameat2 function with its usual signature?" in standard C code and one 
must escape into something like the shell to handle such questions. C23 
and GCC and Clang have added a few features to answer such questions, 
such as __has_include. But these features don't go nearly far enough. 
For example, __has_include tells me only whether the include file 
 exists; it won't tell me whether I can successfully include 
, or whether  will declare the function 'bar', or whether 
'bar' will have a signature compatible with my code's calls to 'bar'.


If I could request a single thing from the C23/GCC/Clang side, I'd ask 
for better facilities to be able to ask such questions within C code, 
without using the shell. Then a good chunk of Autoconf could dry up and 
blow away.


I realize that I'm asking for a lot. For example, a traditional 
implementation cannot answer the renameat2 question without consulting 
the linker. That being said, this ability is essential for modular 
programming at the low level, and if compilers don't provide this 
ability Autoconf will simply have to do the best it can, regardless of 
whether it generates source code that relies on undefined behavior.

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Paul Eggert]

On 2022-11-10 09:16, Zack Weinberg wrote:

Changes to handle C23 built-in ‘bool’ better are under development but
the design has not yet been finalized.


[I'm cc'ing this to bug-gnulib too.]

To my mind this is the biggest outstanding issue in Autoconf as far as 
C23 goes, as the upgrade path for Autoconf's existing bool support is 
not entirely clear. As Florian mentioned, distros can't assume Autoconf 
upgrades when building other packages; that being said, we should get 
Autoconf's bool support fixed sooner rather than later as bool hassles 
will remain until Autoconf is fixed and these fixes are propagated to 
Autoconf's users.


Here's the main Autoconf issue issue with bool. Traditionally, Autoconf 
supported K C, C89, C99, etc. At some point (I'm not sure when), 
Autoconf started requiring C89 or later. Is it now OK for Autoconf to 
require C99 or later, as far as bool is concerned? If so, that'll 
considerably simplify the ongoing maintenance hassle for bool.


Requiring C99-or-later bool is the option that Gnulib has taken. Its 
'stdbool' module and its gl_C_BOOL macro assumes C99 or later, and as 
far as I know no Gnulib-using package is using Gnulib's 'stdbool-c99' 
module which we kept around in case somebody still needed bool to work 
atop a C89 system. (We considered supporting C23 bool atop C89 but it 
was too painful.)


If we follow Gnulib's lead, Autoconf will generate a config.h that does 
"#include " on pre-C23 systems, and this config.h will not 
not work on pre-C99 systems. This of course could in theory break some 
programs, just as compiling them with C23 can break them. But I don't 
see any better option at this point. And besides, what package outside 
of a museum still requires C89 and don't work with C99?

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Thu, Nov 10, 2022 at 1:12 PM Jonathan Wakely via cfe-commits
 wrote:
>
> On Thu, 10 Nov 2022 at 17:58, Jonathan Wakely wrote:
> >
> > On Thu, 10 Nov 2022 at 17:52, Nick Bowler wrote:
> > > It saddens me to see so much breakage happening in "modern C", a
> > > language that has (until now) a long history of new language features
> > > being carefully introduced to avoid these sort of problems.
> >
> > The features were introduced in 1999.
>
> Well, some of them were. Some more are coming now in C2x but the
> problem has existed since C99 it's just that compilers "protected"
> most users from having to fix their code at the time. But it's been
> more than two decades and it's time to accept that missing prototypes,
> implicit conversions between pointers and ints etc are a hazard and
> should be diagnosed not ignored for the benefit of people who never
> want to fix their questionable code.

Functions without prototypes were deprecated in ANSI C and came into
ISO C as deprecated. They were obsoleted with the release of the 2nd
edition of K C (circa 1978) when prototypes were introduced, IIRC.
They've literally never been a recommended practice in standard C.
Implicit function declarations and implicit int were outright removed
from C99 without a deprecation period due to the security concerns
they caused.

~Aaron

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Aaron Ballman via Gcc]

On Thu, Nov 10, 2022 at 12:16 PM Zack Weinberg via cfe-commits
 wrote:
>
> I’m the closest thing Autoconf has to a lead maintainer at present.
>
> It’s come to my attention (via https://lwn.net/Articles/913505/ and
> https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> Clang both plan to disable several “legacy” C language features by
> default in a near-future release (GCC 14, Clang 16) (see the Fedora
> wiki link for a list).  I understand that this change potentially
> breaks a lot of old dusty code, and in particular that
> Autoconf-generated configure scripts use constructs that may *silently
> give the wrong answer to a probe* when a stricter compiler is in use.
>
> Nobody has a whole lot of time to work on Autoconf at present, but I
> would like to ask, anyway, what Autoconf could potentially do to make
> this transition easier.  I’m already aware that the test code Autoconf
> 2.71 uses to probe for C89/C99/C11 support is broken; this has been
> fixed in development trunk to the extent it is possible for me to test
> it with GCC 12 (commit:
> ).
> Several other places using K function definitions and/or
> unprototyped function declarations (including the ubiquitously used
> AC_CHECK_FUNC) have also been fixed on trunk,
> .
> Changes to handle C23 built-in ‘bool’ better are under development but
> the design has not yet been finalized.

Thank you for all of your efforts in modernizing autoconf in response
to all these changes, it's greatly appreciated!

> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.

In terms of the Clang side of things, I don't think we've formed any
sort of official stance on how to handle that yet. It's UB (you can
declare the C standard library interface without UB but calling any
function with a mismatched signature is UB) and that UB has some
amount of security implications associated with it, so I would say
there's a potential we might want to upgrade the diagnostic severity,
but it's not assured. FWIW, we're working on improving communication
about potentially disruptive changes to Clang, so you might want to
consider either subscribing to the clang-vendors code review group at
https://reviews.llvm.org/project/members/113/ (if you want to be
involved in code review before things land) or the Announcements
discourse channel at https://discourse.llvm.org/c/announce/ (if you
want to be notified after something lands but before Clang ships).

> How important do you think it is for this to be fixed?
>
> Are there any other changes you would like to see in a near-future
> Autoconf 2.72 in order to make this transition easier?

I don't have a specific list, but as a general request: moving away
from deprecated facilities of C or reliance on UB is a very pragmatic
idea given that the C committee is recapturing some of that design
space (like what happened with K C signatures) and implementers are
trying to improve the security posture for C.

> zw
>
> p.s. GCC and Clang folks: As long as you’re changing the defaults out
> from under people, can you please also remove the last few predefined
> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
> -std=gnuXX modes?

If we can do so without breaking the world, I personally think it
would be nice to remove them.

~Aaron

> ___
> cfe-commits mailing list
> cfe-comm...@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Thu, 10 Nov 2022 at 17:58, Jonathan Wakely wrote:
>
> On Thu, 10 Nov 2022 at 17:52, Nick Bowler wrote:
> > It saddens me to see so much breakage happening in "modern C", a
> > language that has (until now) a long history of new language features
> > being carefully introduced to avoid these sort of problems.
>
> The features were introduced in 1999.

Well, some of them were. Some more are coming now in C2x but the
problem has existed since C99 it's just that compilers "protected"
most users from having to fix their code at the time. But it's been
more than two decades and it's time to accept that missing prototypes,
implicit conversions between pointers and ints etc are a hazard and
should be diagnosed not ignored for the benefit of people who never
want to fix their questionable code.

> Compilers just continued to
> accept broken code, because people seem to think accepting garbage
> forever is "compatibility".

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Florian Weimer via Gcc]

* Zack Weinberg via Gcc:

> I’m the closest thing Autoconf has to a lead maintainer at present.
>
> It’s come to my attention (via https://lwn.net/Articles/913505/ and
> https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> Clang both plan to disable several “legacy” C language features by
> default in a near-future release (GCC 14, Clang 16) (see the Fedora
> wiki link for a list).  I understand that this change potentially
> breaks a lot of old dusty code, and in particular that
> Autoconf-generated configure scripts use constructs that may *silently
> give the wrong answer to a probe* when a stricter compiler is in use.

Thank you for reaching out.  The c-std-porting list isn't even
bootstrapped yet, the Fedora documentation is still in flux, and the
Fedora oversight body just approved working on this in the Fedora
context.  That LWN article caught me a bit off guard.

Anyway, based on a limited attempt to get this fixed about three years
ago, I expect that many of the problematic packages have not had their
configure scripts regenerated using autoconf for a decade or more.  This
means that as an autoconf maintainer, you unfortunately won't be able to
help us much.

> Nobody has a whole lot of time to work on Autoconf at present, but I
> would like to ask, anyway, what Autoconf could potentially do to make
> this transition easier.  I’m already aware that the test code Autoconf
> 2.71 uses to probe for C89/C99/C11 support is broken; this has been
> fixed in development trunk to the extent it is possible for me to test
> it with GCC 12 (commit:
> ).

Thanks, these changes are going to be helpful to get a clean run from
our Fedora tester.

I also noticed some issues in the automake test suite, and I hope
Frederic can have a look at them:

  automake: Port to modern C
  

> Several other places using K function definitions and/or
> unprototyped function declarations (including the ubiquitously used
> AC_CHECK_FUNC) have also been fixed on trunk,
> .

One reason I hesitate to recommend wide testing with -Werror=… is that
it is actually impossible to model future GCC behavior with -Werror=…
options today.  I very much doubt that the AC_CHECK_FUNC change [that
is, () → (void)] will ever be required by real GCC.  I'm worried that
some of this -Werror=… testing merely introduces unnecessary churn.
These changes help maintainers who wnat to build their packages with
CC="gcc -Werror=…" or something like that, so I guess they still make
sense for autoconf.  But perhaps not as a general model for everyone
else.

> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.

I think t his approach is actually the most portable approach, at least
as far as GCC is concerned.  GCC treats a function as a compiler
built-in only if the declared prototype matches its expectations.  I
doubt there are any such functions returning char, which makes this
declaration a good choice.  I do not expect any medium-term changes to
the situation here.  The warning will stay, it will not turn into an
error.

> Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.

Once you include the header, you also need to know function parameters,
otherwise you won't be able to form a valid call.  Python
setuptools/distutils tried to do something along this lines, but I can't
see how it can work, as explained here:

  Unclear how CCompiler.has_function works for functions with parameters
  

The char-returning prototype does not seem so bad in the end.  The
function is actually called and the result returned from main, so it's
even compatible with LTO.

> How important do you think it is for this to be fixed?

In isolation?  Not important at all.  Maybe it 

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Rich Felker]

On Thu, Nov 10, 2022 at 12:16:20PM -0500, Zack Weinberg wrote:
> I’m the closest thing Autoconf has to a lead maintainer at present.
> 
> It’s come to my attention (via https://lwn.net/Articles/913505/ and
> https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
> Clang both plan to disable several “legacy” C language features by
> default in a near-future release (GCC 14, Clang 16) (see the Fedora
> wiki link for a list).  I understand that this change potentially
> breaks a lot of old dusty code, and in particular that
> Autoconf-generated configure scripts use constructs that may *silently
> give the wrong answer to a probe* when a stricter compiler is in use.
> 
> Nobody has a whole lot of time to work on Autoconf at present, but I
> would like to ask, anyway, what Autoconf could potentially do to make
> this transition easier.  I’m already aware that the test code Autoconf
> 2.71 uses to probe for C89/C99/C11 support is broken; this has been
> fixed in development trunk to the extent it is possible for me to test
> it with GCC 12 (commit:
> ).
> Several other places using K function definitions and/or
> unprototyped function declarations (including the ubiquitously used
> AC_CHECK_FUNC) have also been fixed on trunk,
> .
> Changes to handle C23 built-in ‘bool’ better are under development but
> the design has not yet been finalized.
> 
> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.
> 
> How important do you think it is for this to be fixed?
> 
> Are there any other changes you would like to see in a near-future
> Autoconf 2.72 in order to make this transition easier?

Thanks for bringing this up. It is very important and I am very much
in favor of making these changes and doing it in a way that existing
broken and unmaintained software can be made to work just by
re-generating configure scripts with up-to-date autoconf, even if that
means hard-coding a list of headers needed to get the right
declarations and automatically pulling them in. Otherwise this is
going to be a gigantic burden on distro maintainers/systems
integrators.

I've been writing/complaining about autoconf doing this wrong for
decades, with the best writeup around 9 years ago at
https://ewontfix.com/13/. Part of the reason is that this has bitten
musl libc users over and over again due to configure finding symbols
that were intended only as ABI-compat and trying to use them (without
declarations) at the source level, leading to various messes, some of
which we're only just extricating ourselves from now:

https://git.musl-libc.org/cgit/musl/commit/?id=246f1c811448f37a44b41cd8df8d0ef9736d95f4
https://git.musl-libc.org/cgit/musl/commit/?id=25e6fee27f4a293728dd15b659170e7b9c7db9bc

But aside from issues like this, just the fact that autoconf was
precluding making -Werror=implicit-function-declaration default must
have wasted tens if not hundreds of thousands of human hours debugging
broken builds.

What I'd like to see happen is complete deprecation of the autoconf
link-only tests -- only keeping them for use by legacy unmaintained
projects in the form where they actually implicitly include the right
header and test compile and link using that.

Rich

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Jonathan Wakely via Gcc]

On Thu, 10 Nov 2022 at 17:52, Nick Bowler wrote:
> It saddens me to see so much breakage happening in "modern C", a
> language that has (until now) a long history of new language features
> being carefully introduced to avoid these sort of problems.

The features were introduced in 1999. Compilers just continued to
accept broken code, because people seem to think accepting garbage
forever is "compatibility".

Re: How can Autoconf help with the transition to stricter compilation defaults?

[Nick Bowler]

On 2022-11-10, Zack Weinberg  wrote:
> The biggest remaining (potential) problem, that I’m aware of, is that
> AC_CHECK_FUNC unconditionally declares the function we’re probing for
> as ‘char NAME (void)’, and asks the compiler to call it with no
> arguments, regardless of what its prototype actually is.  It is not
> clear to me whether this will still work with the planned changes to
> the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
> triggered by ‘extern char memcpy(void);’ (or any other standard
> library function whose prototype is coded into the compiler) and this
> already causes problems for people who run configure scripts with
> CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
> have to build a comprehensive list of library functions into Autoconf,
> mapping each to either its documented prototype or to a header where
> it ought to be declared; in the latter case we would also have to make
> e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
> sys/socket.h netdb.h]) which might mess up configure scripts that
> aren’t expecting headers to be probed at that point.
>
> How important do you think it is for this to be fixed?

My gut feeling is that Autoconf should just determine the necessary
options to get compatible behaviour out of these modern compilers, at
least for the purpose of running configure tests.  For example, Autoconf
should probably build the AC_CHECK_FUNC programs using gcc's
-fno-builtin option, which should avoid problems with gcc complaining
about memcpy (and may also improve test accuracy, since gcc won't use
its knowledge of C library behaviour to possibly elide the call to
memcpy).

It saddens me to see so much breakage happening in "modern C", a
language that has (until now) a long history of new language features
being carefully introduced to avoid these sort of problems.

The fact that even the C standard authors don't even seem to care about
existing codebases is a concerning change in direction.  Nobody has
learned anything from the Python 3 debacle, I guess.

> p.s. GCC and Clang folks: As long as you’re changing the defaults out
> from under people, can you please also remove the last few predefined
> user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
> -std=gnuXX modes?

Meh, even though these macros are a small thing I don't accept the
"things are breaking anyway so let's break even more things" attitude.
This was something that many library authors did during the python 3
transition and that just made the problems orders of magnitude more
horrible.

Cheers,
  Nick

How can Autoconf help with the transition to stricter compilation defaults?

[Zack Weinberg via Gcc]

I’m the closest thing Autoconf has to a lead maintainer at present.

It’s come to my attention (via https://lwn.net/Articles/913505/ and
https://fedoraproject.org/wiki/Changes/PortingToModernC) that GCC and
Clang both plan to disable several “legacy” C language features by
default in a near-future release (GCC 14, Clang 16) (see the Fedora
wiki link for a list).  I understand that this change potentially
breaks a lot of old dusty code, and in particular that
Autoconf-generated configure scripts use constructs that may *silently
give the wrong answer to a probe* when a stricter compiler is in use.

Nobody has a whole lot of time to work on Autoconf at present, but I
would like to ask, anyway, what Autoconf could potentially do to make
this transition easier.  I’m already aware that the test code Autoconf
2.71 uses to probe for C89/C99/C11 support is broken; this has been
fixed in development trunk to the extent it is possible for me to test
it with GCC 12 (commit:
).
Several other places using K function definitions and/or
unprototyped function declarations (including the ubiquitously used
AC_CHECK_FUNC) have also been fixed on trunk,
.
Changes to handle C23 built-in ‘bool’ better are under development but
the design has not yet been finalized.

The biggest remaining (potential) problem, that I’m aware of, is that
AC_CHECK_FUNC unconditionally declares the function we’re probing for
as ‘char NAME (void)’, and asks the compiler to call it with no
arguments, regardless of what its prototype actually is.  It is not
clear to me whether this will still work with the planned changes to
the compilers.  Both GCC 12 and Clang 14 have on-by-default warnings
triggered by ‘extern char memcpy(void);’ (or any other standard
library function whose prototype is coded into the compiler) and this
already causes problems for people who run configure scripts with
CC='cc -Werror'.  Unfortunately this is very hard to fix — we would
have to build a comprehensive list of library functions into Autoconf,
mapping each to either its documented prototype or to a header where
it ought to be declared; in the latter case we would also have to make
e.g. AC_CHECK_FUNCS([getaddrinfo]) imply AC_CHECK_HEADERS([sys/types.h
sys/socket.h netdb.h]) which might mess up configure scripts that
aren’t expecting headers to be probed at that point.

How important do you think it is for this to be fixed?

Are there any other changes you would like to see in a near-future
Autoconf 2.72 in order to make this transition easier?

zw

p.s. GCC and Clang folks: As long as you’re changing the defaults out
from under people, can you please also remove the last few predefined
user-namespace macros (-Dlinux, -Dunix, -Darm, etc) from all the
-std=gnuXX modes?