Re: Using __gnu_lto_slim to detect -fno-fat-lto-objects

[Martin Liška]

On 2/22/23 09:28, Florian Weimer via Gcc wrote:
> * Richard Biener:
> 
>> On Wed, Feb 22, 2023 at 9:19 AM Florian Weimer via Gcc  
>> wrote:
>>>
>>> Can we use the COMMON symbol __gnu_lto_slim to detect
>>> -fno-fat-lto-objects on contemporary GNU/Linux (with the LTO linker
>>> plugin)?
>>
>> Yes.
> 
> Great, thanks.
> 
>>> We currently build the distribution with -ffat-lto-objects, and I want
>>> to switch away from that.  Packages will need to opt in to
>>> -ffat-lto-objects if static objects they build escape the buildroot.

Hello.

We use the same approach where only a selected packages enable FAT lto objects.
You can take a look at our META bug where such packages are linked (in most 
cases):
https://bugzilla.opensuse.org/show_bug.cgi?id=1133084

Or one can easily grep it from here:
https://github.com/bmwiedemann/openSUSE

$ git grep 'fat-lto-objects'
(see attachment)

>>> And to make sure that this opt-in happens, I want to fail the build if
>>> there would be any -fno-fat-lto-objects objects leaking.
>>
>> For SUSE we're checking that no LTO bytecode leaks instead, thus we check
>> for __gnu_lto_v? (I think).  The reason is that even for static libraries
>> we do not want to ship LTO bytecode.
> 
> We build with -ffat-lto-objects, and this means we can create perfectly
> fine object files by stripping the LTO data:
> 
> 

We strip this way:
https://github.com/openSUSE/brp-check-suse/blob/2f7add6f0b7f5c2e9698d180a4761a10fb808482/brp-15-strip-debug#L38

> 
> This means that so far, we only had to fix LTO compilation problems in
> the packages, but not teach individual packages about LTO and non-LTO
> object files.  Of course it's wasteful because few packages actually
> install the object files (without a final link into a program or shared
> object), and that's what I want to fix.

And we integrated a rpmlint check that checks we don't ship a LTO bytecode:
https://github.com/rpm-software-management/rpmlint/blob/aa1f710c1b3ff409e42068c2b067ab2eee200b1d/rpmlint/checks/BinariesCheck.py#L277-L283

and there's one related check where we look if a static library contains 
something:
https://github.com/rpm-software-management/rpmlint/blob/aa1f710c1b3ff409e42068c2b067ab2eee200b1d/rpmlint/checks/BinariesCheck.py#L230-L255

it can happen that it only contained bytecode that was eventually striped.

Cheers,
Martin

> 
> Florian
> 
packages/a/ade/ade.changes:- Use -ffat-lto-objects to provide a proper static 
library.
packages/a/ade/ade.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/a/adios/.rev:- Fix aarch64 LTO build with -ffat-lto-objects flag
packages/a/adios/adios.changes:- Fix aarch64 LTO build with -ffat-lto-objects 
flag
packages/a/adios/adios.spec:%global _lto_cflags %{?_lto_cflags} 
-ffat-lto-objects
packages/a/aircrack-ng/aircrack-ng.spec:export CFLAGS="-ffat-lto-objects 
-fcommon"
packages/a/antlr/antlr-bootstrap.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/a/antlr/antlr.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/a/attr/attr.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/b/barcode/barcode.changes:- Build using -ffat-lto-objects
packages/b/barcode/barcode.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/b/binutils/.rev:- Use -ffat-lto-objects in order to 
provide assembly for static libs
packages/b/binutils/binutils.changes:- Use -ffat-lto-objects in order to 
provide assembly for static libs
packages/b/bison/bison.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/b/boinc-client/.rev:- Build boinc-client using 
-ffat-lto-objects
packages/b/boinc-client/boinc-client.changes:- Build boinc-client using 
-ffat-lto-objects
packages/b/boinc-client/boinc-client.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/b/bridge-utils/bridge-utils.changes:- Use -ffat-lto-objects when 
building static libraries.
packages/b/bridge-utils/bridge-utils.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/c/Catch2/Catch2.spec:%global _lto_cflags %{?_lto_cflags} 
-ffat-lto-objects
packages/c/carla/carla.changes:  * Do not use -ffat-lto-objects
packages/c/cgns/cgns.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/c/coturn/coturn.spec:%global _lto_cflags %{?_lto_cflags} 
-ffat-lto-objects
packages/c/criu/criu.changes:- Fix build on Tumbleweed: Add -ffat-lto-objects 
to lto cflags.
packages/d/dhcp/dhcp.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/d/dpkg/dpkg.spec:%global _lto_cflags %{_lto_cflags} -ffat-lto-objects
packages/d/dpkg/update-alternatives.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/e/e2fsprogs/e2fsprogs.spec:%global _lto_cflags %{_lto_cflags} 
-ffat-lto-objects
packages/e/elfutils/.rev:[  123s] created with -ffat-lto-objects option.
packages/e/elfutils/.rev:  And adding -ffat-lto-objects breaks 
run-show-di

RISC-V V C Intrinsic API v1.0 release meeting reminder (February 23rd, 2022)

[Eop Chen via Gcc]

Hi all,

A reminder that the next open meeting to discuss on the RISC-V V C Intrinsic 
API v1.0 is going to
be held later on 2022/02/23 6AM (GMT -7) / 11PM (GMT +8).

The agenda can be found in the second page of the meeting slides (link 
).
Please join the calendar to be constantly notified - Google calender link 
,
 ICal 

We also have a mailing list now hosted by RISC-V International (link 
).

Regards,

eop Chen

Willing to contribute to the project Idea "Fortran – DO CONCURRENT"

[varma datla via Gcc]

Hi, My name is Satyanarayana Varma. I am willing to contribute to the
project idea "Fortran – DO CONCURRENT". Please suggest some resources to me
so that I can learn your technologies. Could you please assign me some good
first issues for the starting.

Thank you,

Regards.
Satyanarayana Datla

Re: [GSoC][Static Analyzer] Some questions and request for a small patch to work on

[James K. Lowden]

On Wed, 22 Feb 2023 14:03:36 +
Jonathan Wakely via Gcc  wrote:

> I think GCC trunk won't even build on M2, you need Iain Sandoe's
> out-of-tree patches.

https://gitlab.cobolworx.com/COBOLworx/gcc-cobol/-/jobs/2822

We've been building on aarch64 based on GCC trunk since December. 

--jkl

Re: [GSoC][Static Analyzer] Some questions and request for a small patch to work on

[Jonathan Wakely via Gcc]

On Thu, 23 Feb 2023 at 15:57, James K. Lowden wrote:
>
> On Wed, 22 Feb 2023 14:03:36 +
> Jonathan Wakely via Gcc  wrote:
>
> > I think GCC trunk won't even build on M2, you need Iain Sandoe's
> > out-of-tree patches.
>
> https://gitlab.cobolworx.com/COBOLworx/gcc-cobol/-/jobs/2822
>
> We've been building on aarch64 based on GCC trunk since December.

It looks like you're using aarch64-linux-gnu, but we're talking about
macOS on M2. I should have been more precise, but for the vast
majority of end users with M2 hardware, they're using macOS.

GCC has been accepted as a GSoC mentoring org

[Martin Jambor]

Hello,

I'm happy to announce that GCC has been accepted as a mentoring
organization in Google Summer of Code 2023!

The program is now in a phase that is called

  "Potential GSoC contributors discuss application ideas with mentoring
   organizations"

and so let's do just that.  Contributors will then be applying in
between March 20 and April 4.

So if any of you is still thinking of mentoring a project, now is the
right time to propose it (see
https://gcc.gnu.org/pipermail/gcc/2023-January/240495.html).

Thanks!

Martin

Re: Missed warning (-Wuse-after-free)

[Alex Colomar via Gcc]

Hi Martin,

On 2/17/23 14:48, Martin Uecker wrote:

This new wording doesn't even allow one to use memcmp(3);
just reading the pointer value, however you do it, is UB.


memcmp would not use the pointer value but work
on the representation bytes and is still allowed.


Hmm, interesting.  It's rather unspecified behavior.  Still 
unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or 
false randomly; the compiler may compile out the call to memcmp(3), 
since it knows it won't produce any observable behavior.




Cheers!

Alex

--

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5



OpenPGP_signature
Description: OpenPGP digital signature

Re: Missed warning (-Wuse-after-free)

[Martin Uecker via Gcc]

Am Donnerstag, dem 23.02.2023 um 20:23 +0100 schrieb Alex Colomar:
> Hi Martin,
> 
> On 2/17/23 14:48, Martin Uecker wrote:
> > > This new wording doesn't even allow one to use memcmp(3);
> > > just reading the pointer value, however you do it, is UB.
> > 
> > memcmp would not use the pointer value but work
> > on the representation bytes and is still allowed.
> 
> Hmm, interesting.  It's rather unspecified behavior. Still 
> unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or 
> false randomly; the compiler may compile out the call to memcmp(3), 
> since it knows it won't produce any observable behavior.
> 
> 

No, I think several things get mixed up here.

The representation of a pointer that becomes invalid
does not change.  

So (0 === memcmp(&p, &p, sizeof(p)) always
evaluates to true.

Also in general, an unspecified value is simply unspecified
but does not change anymore.

Reading an uninitialized value of automatic storage whose
address was not taken is undefined behavior, so everything
is possible afterwards.

An uninitialized variable whose address was taken has a
representation which can represent an unspecified value
or a no-value (trap) representation. Reading the 
representation itself is always ok and gives consistent
results. Reading the variable can be undefined behavior
iff it is a trap representation, otherwise you get
the unspecified value which is stored there.

At least this is my reading of the C standard. Compilers
are not full conformant.


Martin

gcc-10-20230223 is now available

[GCC Administrator via Gcc]

Snapshot gcc-10-20230223 is now available on
  https://gcc.gnu.org/pub/gcc/snapshots/10-20230223/
and on various mirrors, see http://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 10 git branch
with the following options: git://gcc.gnu.org/git/gcc.git branch 
releases/gcc-10 revision 4c08446bd1091bb3c0b706cfb5a4fa015b79031f

You'll find:

 gcc-10-20230223.tar.xz   Complete GCC

  SHA256=cf10fa93a485dd5404baa6c15e9f569b4ef4cd05432d0239dceb52decd35267c
  SHA1=7711870df5e5c07ed8b0ba5b7515646e5bb9acd1

Diffs from 10-20230216 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-10
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.

Re: Missed warning (-Wuse-after-free)

[Alex Colomar via Gcc]

Hi Martin,

On 2/23/23 20:57, Martin Uecker wrote:

Am Donnerstag, dem 23.02.2023 um 20:23 +0100 schrieb Alex Colomar:

Hi Martin,

On 2/17/23 14:48, Martin Uecker wrote:

This new wording doesn't even allow one to use memcmp(3);
just reading the pointer value, however you do it, is UB.


memcmp would not use the pointer value but work
on the representation bytes and is still allowed.


Hmm, interesting.  It's rather unspecified behavior. Still
unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or
false randomly; the compiler may compile out the call to memcmp(3),
since it knows it won't produce any observable behavior.




No, I think several things get mixed up here.

The representation of a pointer that becomes invalid
does not change.

So (0 === memcmp(&p, &p, sizeof(p)) always
evaluates to true.

Also in general, an unspecified value is simply unspecified
but does not change anymore.

Reading an uninitialized value of automatic storage whose
address was not taken is undefined behavior, so everything
is possible afterwards.

An uninitialized variable whose address was taken has a
representation which can represent an unspecified value
or a no-value (trap) representation. Reading the
representation itself is always ok and gives consistent
results. Reading the variable can be undefined behavior
iff it is a trap representation, otherwise you get
the unspecified value which is stored there.

At least this is my reading of the C standard. Compilers
are not full conformant.


Does all this imply that the following is well defined behavior (and 
shall print what one would expect)?


  free(p);

  (void) &p;  // take the address
  // or maybe we should (void) memcmp(&p, &p, sizeof(p)); ?

  printf("%p\n", p);  // we took previously its address,
  // so now it has to hold consistently
  // the previous value


This feels weird.  And a bit of a Schroedinger's pointer.  I'm not 
entirely convinced, but might be.


Cheers,

Alex




Martin









--

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5



OpenPGP_signature
Description: OpenPGP digital signature

Re: Missed warning (-Wuse-after-free)

[Serge E. Hallyn]

On Fri, Feb 24, 2023 at 01:02:54AM +0100, Alex Colomar wrote:
> Hi Martin,
> 
> On 2/23/23 20:57, Martin Uecker wrote:
> > Am Donnerstag, dem 23.02.2023 um 20:23 +0100 schrieb Alex Colomar:
> > > Hi Martin,
> > > 
> > > On 2/17/23 14:48, Martin Uecker wrote:
> > > > > This new wording doesn't even allow one to use memcmp(3);
> > > > > just reading the pointer value, however you do it, is UB.
> > > > 
> > > > memcmp would not use the pointer value but work
> > > > on the representation bytes and is still allowed.
> > > 
> > > Hmm, interesting.  It's rather unspecified behavior. Still
> > > unpredictable: (memcmp(&p, &p, sizeof(p) == 0) might evaluate to true or
> > > false randomly; the compiler may compile out the call to memcmp(3),
> > > since it knows it won't produce any observable behavior.
> > > 
> > > 
> > 
> > No, I think several things get mixed up here.
> > 
> > The representation of a pointer that becomes invalid
> > does not change.
> > 
> > So (0 === memcmp(&p, &p, sizeof(p)) always
> > evaluates to true.
> > 
> > Also in general, an unspecified value is simply unspecified
> > but does not change anymore.

Right.  p is its own thing - n bytes on the stack containing some value.
Once it comes into scope, it doesn't change on its own.  And if I do
free(p) or o = realloc(p), then the value of p itself - the n bytes on
the stack - does not change.

I realize C11 appears to have changed that.  I fear that in doing so it
actually risks increasing the confusion about pointers.  IMO it's much
easier to reason about

o = realloc(p, X);

(and more baroque constructions) when keeping in mind that o, p, and the
object pointed to by either one are all different things.

> > Reading an uninitialized value of automatic storage whose
> > address was not taken is undefined behavior, so everything
> > is possible afterwards.
> > 
> > An uninitialized variable whose address was taken has a
> > representation which can represent an unspecified value
> > or a no-value (trap) representation. Reading the
> > representation itself is always ok and gives consistent
> > results. Reading the variable can be undefined behavior
> > iff it is a trap representation, otherwise you get
> > the unspecified value which is stored there.
> > 
> > At least this is my reading of the C standard. Compilers
> > are not full conformant.
> 
> Does all this imply that the following is well defined behavior (and shall
> print what one would expect)?
> 
>   free(p);
> 
>   (void) &p;  // take the address
>   // or maybe we should (void) memcmp(&p, &p, sizeof(p)); ?
> 
>   printf("%p\n", p);  // we took previously its address,
>   // so now it has to hold consistently
>   // the previous value
> 
> 
> This feels weird.  And a bit of a Schroedinger's pointer.  I'm not entirely
> convinced, but might be.

Again, p is just an n byte variable which happens to have (one hopes)
pointed at a previously malloc'd address.

And I'd argue that pre-C11, this was not confusing, and would not have
felt weird to you.

But I am most grateful to you for having brought this to my attention.
I may not agree with it and not like it, but it's right there in the
spec, so time for me to adjust :)

-serge

Re: Missed warning (-Wuse-after-free)

[Alex Colomar via Gcc]

Hi Serge, Martin,

On 2/24/23 02:21, Serge E. Hallyn wrote:

Does all this imply that the following is well defined behavior (and shall
print what one would expect)?

   free(p);

   (void) &p;  // take the address
   // or maybe we should (void) memcmp(&p, &p, sizeof(p)); ?

   printf("%p\n", p);  // we took previously its address,
   // so now it has to hold consistently
   // the previous value


This feels weird.  And a bit of a Schroedinger's pointer.  I'm not entirely
convinced, but might be.


Again, p is just an n byte variable which happens to have (one hopes)
pointed at a previously malloc'd address.

And I'd argue that pre-C11, this was not confusing, and would not have
felt weird to you.

But I am most grateful to you for having brought this to my attention.
I may not agree with it and not like it, but it's right there in the
spec, so time for me to adjust :)


I'll try to show why this feels weird to me (even in C89):


alx@dell7760:~/tmp$ cat pointers.c
#include 
#include 


int
main(void)
{
char  *p, *q;

p = malloc(42);
if (p == NULL)
exit(1);

q = realloc(p, 42);
if (q == NULL)
exit(1);

(void) &p;  // If we remove this, we get -Wuse-after-free

printf("(%p == %p) = %i\n", p, q, (p == q));
}
alx@dell7760:~/tmp$ cc -Wall -Wextra pointers.c  -Wuse-after-free=3
alx@dell7760:~/tmp$ ./a.out
(0x5642cd9022a0 == 0x5642cd9022a0) = 1


This pointers point to different objects (actually, one of them doesn't 
even point to an object anymore), so they can't compare equal, according 
to both:






(I believe C89 already had the concept of lifetime well defined as it is 
now, so the object had finished it's lifetime after realloc(3)).


How can we justify that true, if the pointer don't point to the same 
object?  And how can we justify a hypothetical false (which compilers 
don't implement), if compilers will really just read the value?  To 
implement this as well defined behavior, it could result in no other 
than false, and it would require heavy overhead for the compilers to 
detect that the seemingly-equal values are indeed different, don't you 
think?  The easiest solution is for the standard to just declare this 
outlaw, IMO.


Maybe it could do an exception for printing, that is, reading a pointer 
is not a problem in itself, a long as you don't compare it, but I'm not 
such an expert about this.


Cheers,

Alex



-serge


--

GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5



OpenPGP_signature
Description: OpenPGP digital signature

Re: Missed warning (-Wuse-after-free)

[Peter Lafreniere via Gcc]

If I may add my thoughts here,

On Thursday, February 23rd, 2023 at 20:42, Alex Colomar wrote:
> I'll try to show why this feels weird to me (even in C89):
>
>
> alx@dell7760:~/tmp$ cat pointers.c
> #include 
>
> #include 
>
>
>
> int
> main(void)
> {
> char *p, *q;
>
> p = malloc(42);
> if (p == NULL)
> exit(1);
>
> q = realloc(p, 42);
> if (q == NULL)
> exit(1);
>
> (void) &p; // If we remove this, we get -Wuse-after-free
>
> printf("(%p == %p) = %i\n", p, q, (p == q));
> }
> alx@dell7760:~/tmp$ cc -Wall -Wextra pointers.c -Wuse-after-free=3
> alx@dell7760:~/tmp$ ./a.out
> (0x5642cd9022a0 == 0x5642cd9022a0) = 1
>
>
> This pointers point to different objects (actually, one of them doesn't
> even point to an object anymore), so they can't compare equal, according
> to both:

In this case p and q both _do_ point to the same object, which is valid.
As per [https://port70.net/~nsz/c/c11/n1570.html#3.15], if the pointers both
refer to the same storage region *[1] then they, by definition, refer to the
same object and so should compare as true.

This equality is entirely up to the implementation of your libc's realloc and
cannot be depended upon, so the warning is absolutely warranted *[2]. I don't
believe that it's correct for gcc to do more based off of external
implementation-defined behavior.

*[1]: Which may or may not need to be sized. I'm not an expert at reading specs,
and besides, in this case the pointer is of the same type and value. This point
seems more related to aliasing rules than lifetimes.

*[2]: There are, however, genuine cases where the reuse could be valid. Namely
that if q is null, then p is guaranteed to still point to a valid allocation.
Fortunately, I can't get gcc to trigger a false warning in this case.

>
> http://port70.net/~nsz/c/c11/n1570.html#6.5.9p6
>
>
> http://port70.net/~nsz/c/c89/c89-draft.html#3.3.9

Just to be transparent, I'm working exclusively off of the first link.
Also, this is just a passing interpretation of the c11 spec.

>
>
> (I believe C89 already had the concept of lifetime well defined as it is
> now, so the object had finished it's lifetime after realloc(3)).
>
> How can we justify that true, if the pointer don't point to the same
> object? And how can we justify a hypothetical false (which compilers
> don't implement), if compilers will really just read the value? To
> implement this as well defined behavior, it could result in no other
> than false, and it would require heavy overhead for the compilers to
> detect that the seemingly-equal values are indeed different, don't you
> think? The easiest solution is for the standard to just declare this
> outlaw, IMO.

Again, this is an issue with defining an object.

> Maybe it could do an exception for printing, that is, reading a pointer
> is not a problem in itself, a long as you don't compare it, but I'm not
> such an expert about this.

One last thought: with the above strict interpretation of the c standard,
it would become nigh on impossible to implement the malloc(3) family of
functions in c themselves. I vote for the "shared storage" interpretation
of the c11 standard that is actually implemented rather than this abstract
liveness oriented interpretation.

--

Cheers,
Peter Lafreniere 

"Who malloc()'s for malloc()?" - Myself, 2023-02-24