Re: AW: AW: AW: Problem using gcov

[Martin Liška]

On 02/26/2018 03:41 PM, Alexander Fichtinger wrote:
> Ok, thanks for the info.
> 
> I just saw the description in the bug-report.
> When we document this for our customers, it's ok, I think. 

Thanks for understanding. I would just note that there's couple of similar 
issues
you probably want to be aware of:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83434
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83505
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83587
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83616
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83678
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813

Martin

> 
> Thanks and kind regards,
> Alexander
> 
> -Ursprüngliche Nachricht-
> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag von 
> Martin Liška
> Gesendet: Montag, 26. Februar 2018 15:36
> An: Alexander Fichtinger ; gcc@gcc.gnu.org
> Betreff: Re: AW: AW: Problem using gcov
> 
> On 02/26/2018 03:02 PM, Alexander Fichtinger wrote:
>> Thanks Martin!
>>
>> I've seen that the target milestone for this bug is 8.0.
> 
> I did, but as described in 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572#c1
> I'm not planning to fix it in the next releases. It's just a minor issue in 
> GCOV infrastructure.
> 
> Thank you for understanding,
> Martin
> 
>>
>> When is 8.0 going to be released?
>>
>> Thanks and kind regards,
>> Alexander
>>
>> -Ursprüngliche Nachricht-
>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>> von Martin Liška
>> Gesendet: Montag, 26. Februar 2018 14:57
>> An: Alexander Fichtinger ; 
>> gcc@gcc.gnu.org
>> Betreff: Re: AW: Problem using gcov
>>
>> On 02/26/2018 11:30 AM, Alexander Fichtinger wrote:
>>> Hi Martin,
>>>
>>> sorry that it took me so long to create the standalone sample, but I was 
>>> very busy the last few days.
>>>
>>>
>>> We can reproduce the strange behavior in a standalone scenario.
>>>
>>> I've provided it in the attachments.
>>> In the ZIP-file there is also a README.txt which contains the commands 
>>> which I called.
>>>
>>> And the line with the "strange" execution count is also mentioned.
>>>
>>> Thank you and kind regards,
>>>
>>> Alexander
>>>
>>>
>>> -Ursprüngliche Nachricht-
>>> Von: Martin Liška [mailto:mli...@suse.cz]
>>> Gesendet: Donnerstag, 22. Februar 2018 10:43
>>> An: Alexander Fichtinger ; 
>>> gcc@gcc.gnu.org
>>> Betreff: Re: Problem using gcov
>>>
>>> On 02/21/2018 02:33 PM, Alexander Fichtinger wrote:
 When you look at line 23 it is called 1 time. Line 24 is called 2 times 
 (the bold one).
 We already looked in the disassembly but did not see a reason why the line 
 should be executed 2 times.

 Can anyone help here, please?

 Kind regards,
 Alex
>>>
>>> Hi.
>>>
>>> Sure, can you please send a self-contained test-case that I can run?
>>>
>>> Thank you,
>>> Martin
>>>
>>
>> No problem, I can confirm the problem:
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572
>>
>> Martin
>>
> 
> 

AW: AW: AW: AW: Problem using gcov

[Alexander Fichtinger]

Ok, thanks for the information.

Do we get all coverage-related problems when we monitor the Bugs with the 
Component "gcov-profile"?
Or are there other components concerning coverage?

We are using gcov in our Testframework (used for Automotive software) in order 
to determine the Code-Coverage, 
so it is important for us to know if there are any bugs which could compromise 
the reported test coverage.


Thanks and kind regards,
Alexander

-Ursprüngliche Nachricht-
Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag von Martin 
Liška
Gesendet: Dienstag, 27. Februar 2018 09:01
An: Alexander Fichtinger ; gcc@gcc.gnu.org
Betreff: Re: AW: AW: AW: Problem using gcov

On 02/26/2018 03:41 PM, Alexander Fichtinger wrote:
> Ok, thanks for the info.
> 
> I just saw the description in the bug-report.
> When we document this for our customers, it's ok, I think. 

Thanks for understanding. I would just note that there's couple of similar 
issues you probably want to be aware of:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83434
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83505
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83587
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83616
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83678
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813

Martin

> 
> Thanks and kind regards,
> Alexander
> 
> -Ursprüngliche Nachricht-
> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
> von Martin Liška
> Gesendet: Montag, 26. Februar 2018 15:36
> An: Alexander Fichtinger ; 
> gcc@gcc.gnu.org
> Betreff: Re: AW: AW: Problem using gcov
> 
> On 02/26/2018 03:02 PM, Alexander Fichtinger wrote:
>> Thanks Martin!
>>
>> I've seen that the target milestone for this bug is 8.0.
> 
> I did, but as described in 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572#c1
> I'm not planning to fix it in the next releases. It's just a minor issue in 
> GCOV infrastructure.
> 
> Thank you for understanding,
> Martin
> 
>>
>> When is 8.0 going to be released?
>>
>> Thanks and kind regards,
>> Alexander
>>
>> -Ursprüngliche Nachricht-
>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>> von Martin Liška
>> Gesendet: Montag, 26. Februar 2018 14:57
>> An: Alexander Fichtinger ;
>> gcc@gcc.gnu.org
>> Betreff: Re: AW: Problem using gcov
>>
>> On 02/26/2018 11:30 AM, Alexander Fichtinger wrote:
>>> Hi Martin,
>>>
>>> sorry that it took me so long to create the standalone sample, but I was 
>>> very busy the last few days.
>>>
>>>
>>> We can reproduce the strange behavior in a standalone scenario.
>>>
>>> I've provided it in the attachments.
>>> In the ZIP-file there is also a README.txt which contains the commands 
>>> which I called.
>>>
>>> And the line with the "strange" execution count is also mentioned.
>>>
>>> Thank you and kind regards,
>>>
>>> Alexander
>>>
>>>
>>> -Ursprüngliche Nachricht-
>>> Von: Martin Liška [mailto:mli...@suse.cz]
>>> Gesendet: Donnerstag, 22. Februar 2018 10:43
>>> An: Alexander Fichtinger ;
>>> gcc@gcc.gnu.org
>>> Betreff: Re: Problem using gcov
>>>
>>> On 02/21/2018 02:33 PM, Alexander Fichtinger wrote:
 When you look at line 23 it is called 1 time. Line 24 is called 2 times 
 (the bold one).
 We already looked in the disassembly but did not see a reason why the line 
 should be executed 2 times.

 Can anyone help here, please?

 Kind regards,
 Alex
>>>
>>> Hi.
>>>
>>> Sure, can you please send a self-contained test-case that I can run?
>>>
>>> Thank you,
>>> Martin
>>>
>>
>> No problem, I can confirm the problem:
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572
>>
>> Martin
>>
> 
> 

Run (some?) ELF constructors after applying RELRO protection

[Florian Weimer]

I think it would be a nice addition to the toolchain if it were possible 
to programatically initialize data in the RELRO section.  We do this in 
glibc, but I don't think this is currently supported for general use.


One important application is to allocate a memory region with mmap, on 
which protection flags can be changed as needed.  This way, the 
application can have a read-only path to its own configuration data, for 
example.


Do you think this would be worthwhile to implement?  Any suggestions how 
we should do it, without needing binutils/GCC/glibc updates?


Thanks,
Florian

Re: GCC interpretation of C11 atomics (DR 459)

[Florian Weimer]

On 02/26/2018 07:36 PM, Janne Blomqvist wrote:

There is no such architectural guarantee. At least on some
micro-architecture (AMD Opteron "Istanbul") it's possible to construct
a test which fails, proving that at least on that micro-arch SSE2
load/store isn't guaranteed to be atomic.


Looks like I was wrong.  Ugh.

Thanks,
Florian

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Ramana Radhakrishnan]

On Mon, Feb 26, 2018 at 10:45 PM, Ruslan Nikolaev via gcc
 wrote:
> Thanks, everyone, for the output, it is very useful. I am just proposing to 
> consider the change unless there are clear roadblocks. (Either design choice 
> is probably OK with respect to the standard formally speaking, but there are 
> some clear advantages also.) I wrote a summary of pros & cons (which, of 
> course, is slightly biased towards the change :) )
> I also opened Bug 84563 with the rationale.
>
>
> Pros of the proposed approach:
> 1. Ability to use guaranteed lock-free double-width atomics (when mcx16 is 
> specified for x86-64, and always for arm64) in more or less portable manner 
> across different supported architectures (without resorting to non-standard 
> extensions or writing separate assembly code for each architecture). 
> Hopefully, the behavior may also be made more or less consistent across 
> different compilers over time. It is already the case for clang/llvm. As 
> mentioned, double-width lock-free atomics have real practical use (ABA tags 
> for pointers).
>
> 2. More likely to find a bug immediately if a programmer tries to do 
> something that is not guaranteed by the standard (i.e., getting segfault on 
> read-only memory when using double-width atomic_load). This is true even if 
> mcx16 is not used, as most CPUs have cmpxchg16b, and libatomic will use it.On 
> the other hand, atomic_load implemented through locks may have hard-to-find 
> and debug issues in signal handlers, interrupt contexts, etc when a 
> programmer erroneously assumes that atomic_load is non-blocking
>
> 3. For arm64 the corresponding instructions are always available, no need for 
> mcx16 flag or redirection to libatomic at all (libatomic may still keep old 
> implementation for backward compatibility).

That is going to create an ABI break on AArch64. Think about binaries
produced by old releases GCC that use locks in libatomic and those
used by new GCC. The way to fix this in AArch64 if there is a
guarantee from the standard that there are no  problems with read-only
locations is to implement the change in libatomic. You cannot have the
same region of memory protected by locks in older binaries and the
appropriate load / store instructions in new binaries.

Ramana


> 4. Faster & easy to analyze code when mcx16 is specified.
>
> 5. Ability to tell for sure if the implementation is lock-free by checking 
> corresponding C11 flag when mcx16 is specified. When unspecified, the flag 
> will be false to accommodate the worse-case scenario.
>
> 6. Consistent behavior everywhere on all platforms regardless of IFFUNC, 
> mcx16 flag, etc. If cmpxchg16b is available, it is always used (platforms 
> that do not support IFFUNC will use function pointers for redirection). The 
> only thing the mcx16 flag changes is removing indirection to libatomic and 
> giving guaranteed lock_free flag for corresponding types. (BTW, in practice, 
> if you use the flag, you should know what you are doing already)
>
> 7. Ability to finally deprecate old __sync builtins, and use new and more 
> advanced __atomic everywhere.
>
>
> Cons of the proposed approach:
>
> 1. Compiler may place const atomic objects to .rodata. (Avoided by making 
> sure _Atomic objects with the size > 8 are not placed in .rodata + clarifying 
> that casting random .rodata objects for double-width atomics is undefined and 
> is not allowed.)
>
> 2. Backward compatibility concerns if used outside glibc/IFFUNC. Most likely, 
> even in this case, not an issue since all calls there are already redirected 
> to libatomic anyway, and statically-linked binaries will not interact with 
> new binaries directly.
> 3. Read-only memory for atomic_load will not be supported for double-width 
> types. But it is actually better than hiding the problem under the carpet 
> (current behavior is actually even worse because it is inconsistent across 
> different platforms, i.e. different for x86-64 in Linux and arm64). Anyway, 
> it is better to use a lock-based approach explicitly if for whatever reason 
> it is more preferable (read-only memory, performance (?), etc).
> -- Ruslan

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Mon, 2018-02-26 at 19:39 +, Ruslan Nikolaev via gcc wrote:
> Torvald, I definitely do not want to insist on this design choice, but it 
> makes sense to at least seriuously consider it given the concerns I 
> described. And especially because IFFUNC in libatomic already redirects to 
> cmpxchg16b,

That's because we want to keep the old (but wrong) behavior unchanged,
at least for existing code, until there's a time when we switch it.
Given that we also don't declare those types lock-free anymore, new code
will know that it's not safe to use them for cases such as inter-process
communication (because if not lock-free, it's also not address-free
anymore).

>  > Not getting the performance usually associated with atomic loads can be
> > a big problem for code that tries to be portable.
> 
> I do not think it is a common use case anyway. How often atomic_load is used 
> on double-width operations?

A portable program doesn't have to think about things like double-width,
or whether the platform is something like x32 vs. x86-64.  What a
portable program cares about is whether atomic ops are lock-free on a
particular 64b integer type or not.  If they are, you want to use them
to synchronize (e.g., counters), and then it can matter a lot whether a
load is actually a load or just creates lots of contention.  If they
aren't available, the program knows that it has to find a different way
to synchronize (e.g., build the 64b counter out of 32b operations). 

> If a programmer needs some guarantees and does not care about lock-freedom, 
> why not use a regular lock here?

They do care about whether atomic operations are natively supported on
that particular type -- and that should include a load.

> This way nothing magical happens. Otherwise, he will may hit unexpected 
> issues in places like signal handlers (which is hard to debug since it will 
> hang only once in a while). With cmpxchg16b, it is at least more or less 
> reproducible: if you tried to use it on read-only memory, you will 
> immediately get a segfault.

Nobody is proposing to mark things as lock-free if they aren't.  Thus, I
don't see any change to what's usable in signal handlers.

> > I think I now remember why we "didn't fix" libatomic: There might be
> > compiled code out there that does use the wide CAS, so changing
> > libatomic from the status quo to using its intenral locks could break
> > programs.
> Well, it already happens for Linux and glibc. There nothing will break. For 
> other architectures, it would be good to implement the same, so that 
> consistent behavior is observed everywhere.

It's not about consistency across archs, but consistency for existing
code.  New code or new implementations should just do the right thing,
which is requiring a natively supported atomic load of the particular
size/alignment.

> 
> > No, they only said that it doesn't need to be a concern for the
> > standard.  Implementations have to pay attention to more things, so it
> > is a concern for implementation.
> Yes, but the only problem I see is that it is currently placed to .rodata 
> when const is used.

I and others are of different opinion:  Load performance matters,
inter-process communication on read-only memory matters, and it's useful
to have the builtins work on not just _Atomic types but general integer
types with proper alignment (e.g., look at how glibc uses the builtins
in a code base that is not C11 or more recent).

> It is easy to resolve: just do not place it there for _Atomic objects > 8 
> bytes. Then also clarify that a programmer cannot safely cast some arbitrary 
> object that can be placed in .rodata to use with atomic_load.

That doesn't help with the use cases I listed previously.

> It needs to be addressed anyway, as there is already a segfault for provided 
> example in x86-64 and Linux even with redirection to libatomic.
> 
> > It's not "visible" in the abstract machine under some setting of the
> > as-if rule.  But it is definitely visible in an implementation in which
> > the effects of read-only memory are visible (see my example of mapping
> > memory from another process read-only so as to read data from that
> > process).
> True but it is not defined for read-only memory anyway,

The standard doesn't specify read-only memory, so it also doesn't forbid
the concept.  The implementation takes it into account though, and thus
it's defined in that context.

> and no assumptions can be made in portable code. 

No you can make assumptions, given what we want the implementation to
do.  We might need to explain that better (or at all) in the docs, but
the idea is that *new* code can expect lock-free atomics to both have a
true atomic load (ie, performance-wise) and have loads work on
read-only-mapped memory.

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 10:22 +, Ramana Radhakrishnan wrote:
> The way to fix this in AArch64 if there is a
> guarantee from the standard that there are no  problems with read-only
> locations is to implement the change in libatomic.

Even though the standard doesn't specify read-only memory, I think that
consensus in ISO C++ SG1 (ie, the concurrency study group) exists that
it makes sense for implementations to not declare something lock-free if
the hardware doesn't provide a true atomic load for the particular
size/alignment.  It is an implementation-level decision though (given
that the details of the as-if rule depend on what's doable on the
particular implementation), and I do not see a reason to change GCC's
stance on this.

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Mon, 2018-02-26 at 22:45 +, Ruslan Nikolaev via gcc wrote:
> Thanks, everyone, for the output, it is very useful. I am just proposing to 
> consider the change unless there are clear roadblocks. (Either design choice 
> is probably OK with respect to the standard formally speaking, but there are 
> some clear advantages also.) I wrote a summary of pros & cons (which, of 
> course, is slightly biased towards the change :) )
> I also opened Bug 84563 with the rationale.

This bug summarizes your perspective on the matter.  I'd call that not
just slightly biased :)

I do not see a reason to change GCC's position regarding this topic.  We
should update the docs though to clarify the intent and guarantees GCC's
implementation gives, I suppose.

The reasons have been discussed elsewhere in this thread already, so I'm
not going to repeat them here.

> 1. Ability to use guaranteed lock-free double-width atomics (when mcx16 is 
> specified for x86-64, and always for arm64) in more or less portable manner 
> across different supported architectures (without resorting to non-standard 
> extensions or writing separate assembly code for each architecture).

That's a valid goal, but it does not imply that we should mess with how
atomics are implemented by default, nor should we mess with the default
use cases.  This goal wants something special, and that is exposing the
fact that *only* a CAS is available to synchronize atomically on a
particular type.  That is an extension of the existing atomics design.

There are different ways to expose such an extension, with one being to
simply provide a __atomic_special_cas builtin or something like that.
It would have the same synchronization semantics as the normal CAS, but
concurrent access between the special CAS and the normal atomics would
be considered a data race (so making sure that there's no guaranteed
atomicity between them).  It could have a fallback to libatomic for ease
of use, and it could be defined for smaller types too.  This would be
portable, and would allow us to separate the different use cases.

> 7. Ability to finally deprecate old __sync builtins, and use new and more 
> advanced __atomic everywhere.

The topic we're currently discussing does not significantly affect when
we can remove __sync builtins, IMO.

Re: AW: AW: AW: AW: Problem using gcov

[Martin Liška]

On 02/27/2018 09:13 AM, Alexander Fichtinger wrote:
> Ok, thanks for the information.
> 
> Do we get all coverage-related problems when we monitor the Bugs with the 
> Component "gcov-profile"?
> Or are there other components concerning coverage?

Yes, I mark all gcov bugs with this component.

> 
> We are using gcov in our Testframework (used for Automotive software) in 
> order to determine the Code-Coverage, 
> so it is important for us to know if there are any bugs which could 
> compromise the reported test coverage.

Understand that!
Martin

> 
> 
> Thanks and kind regards,
> Alexander
> 
> -Ursprüngliche Nachricht-
> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag von 
> Martin Liška
> Gesendet: Dienstag, 27. Februar 2018 09:01
> An: Alexander Fichtinger ; gcc@gcc.gnu.org
> Betreff: Re: AW: AW: AW: Problem using gcov
> 
> On 02/26/2018 03:41 PM, Alexander Fichtinger wrote:
>> Ok, thanks for the info.
>>
>> I just saw the description in the bug-report.
>> When we document this for our customers, it's ok, I think. 
> 
> Thanks for understanding. I would just note that there's couple of similar 
> issues you probably want to be aware of:
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83434
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83505
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83587
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83616
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83678
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
> 
> Martin
> 
>>
>> Thanks and kind regards,
>> Alexander
>>
>> -Ursprüngliche Nachricht-
>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>> von Martin Liška
>> Gesendet: Montag, 26. Februar 2018 15:36
>> An: Alexander Fichtinger ; 
>> gcc@gcc.gnu.org
>> Betreff: Re: AW: AW: Problem using gcov
>>
>> On 02/26/2018 03:02 PM, Alexander Fichtinger wrote:
>>> Thanks Martin!
>>>
>>> I've seen that the target milestone for this bug is 8.0.
>>
>> I did, but as described in 
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572#c1
>> I'm not planning to fix it in the next releases. It's just a minor issue in 
>> GCOV infrastructure.
>>
>> Thank you for understanding,
>> Martin
>>
>>>
>>> When is 8.0 going to be released?
>>>
>>> Thanks and kind regards,
>>> Alexander
>>>
>>> -Ursprüngliche Nachricht-
>>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>>> von Martin Liška
>>> Gesendet: Montag, 26. Februar 2018 14:57
>>> An: Alexander Fichtinger ;
>>> gcc@gcc.gnu.org
>>> Betreff: Re: AW: Problem using gcov
>>>
>>> On 02/26/2018 11:30 AM, Alexander Fichtinger wrote:
 Hi Martin,

 sorry that it took me so long to create the standalone sample, but I was 
 very busy the last few days.


 We can reproduce the strange behavior in a standalone scenario.

 I've provided it in the attachments.
 In the ZIP-file there is also a README.txt which contains the commands 
 which I called.

 And the line with the "strange" execution count is also mentioned.

 Thank you and kind regards,

 Alexander


 -Ursprüngliche Nachricht-
 Von: Martin Liška [mailto:mli...@suse.cz]
 Gesendet: Donnerstag, 22. Februar 2018 10:43
 An: Alexander Fichtinger ;
 gcc@gcc.gnu.org
 Betreff: Re: Problem using gcov

 On 02/21/2018 02:33 PM, Alexander Fichtinger wrote:
> When you look at line 23 it is called 1 time. Line 24 is called 2 times 
> (the bold one).
> We already looked in the disassembly but did not see a reason why the 
> line should be executed 2 times.
>
> Can anyone help here, please?
>
> Kind regards,
> Alex

 Hi.

 Sure, can you please send a self-contained test-case that I can run?

 Thank you,
 Martin

>>>
>>> No problem, I can confirm the problem:
>>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572
>>>
>>> Martin
>>>
>>
>>
> 
> 

AW: AW: AW: AW: AW: Problem using gcov

[Alexander Fichtinger]

Ok, thank you.

Alexander

-Ursprüngliche Nachricht-
Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag von Martin 
Liška
Gesendet: Dienstag, 27. Februar 2018 13:39
An: Alexander Fichtinger ; gcc@gcc.gnu.org
Betreff: Re: AW: AW: AW: AW: Problem using gcov

On 02/27/2018 09:13 AM, Alexander Fichtinger wrote:
> Ok, thanks for the information.
> 
> Do we get all coverage-related problems when we monitor the Bugs with the 
> Component "gcov-profile"?
> Or are there other components concerning coverage?

Yes, I mark all gcov bugs with this component.

> 
> We are using gcov in our Testframework (used for Automotive software) 
> in order to determine the Code-Coverage, so it is important for us to know if 
> there are any bugs which could compromise the reported test coverage.

Understand that!
Martin

> 
> 
> Thanks and kind regards,
> Alexander
> 
> -Ursprüngliche Nachricht-
> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
> von Martin Liška
> Gesendet: Dienstag, 27. Februar 2018 09:01
> An: Alexander Fichtinger ; 
> gcc@gcc.gnu.org
> Betreff: Re: AW: AW: AW: Problem using gcov
> 
> On 02/26/2018 03:41 PM, Alexander Fichtinger wrote:
>> Ok, thanks for the info.
>>
>> I just saw the description in the bug-report.
>> When we document this for our customers, it's ok, I think. 
> 
> Thanks for understanding. I would just note that there's couple of similar 
> issues you probably want to be aware of:
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83434
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83505
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83587
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83616
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83678
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83813
> 
> Martin
> 
>>
>> Thanks and kind regards,
>> Alexander
>>
>> -Ursprüngliche Nachricht-
>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>> von Martin Liška
>> Gesendet: Montag, 26. Februar 2018 15:36
>> An: Alexander Fichtinger ;
>> gcc@gcc.gnu.org
>> Betreff: Re: AW: AW: Problem using gcov
>>
>> On 02/26/2018 03:02 PM, Alexander Fichtinger wrote:
>>> Thanks Martin!
>>>
>>> I've seen that the target milestone for this bug is 8.0.
>>
>> I did, but as described in
>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572#c1
>> I'm not planning to fix it in the next releases. It's just a minor issue in 
>> GCOV infrastructure.
>>
>> Thank you for understanding,
>> Martin
>>
>>>
>>> When is 8.0 going to be released?
>>>
>>> Thanks and kind regards,
>>> Alexander
>>>
>>> -Ursprüngliche Nachricht-
>>> Von: gcc-ow...@gcc.gnu.org [mailto:gcc-ow...@gcc.gnu.org] Im Auftrag 
>>> von Martin Liška
>>> Gesendet: Montag, 26. Februar 2018 14:57
>>> An: Alexander Fichtinger ;
>>> gcc@gcc.gnu.org
>>> Betreff: Re: AW: Problem using gcov
>>>
>>> On 02/26/2018 11:30 AM, Alexander Fichtinger wrote:
 Hi Martin,

 sorry that it took me so long to create the standalone sample, but I was 
 very busy the last few days.


 We can reproduce the strange behavior in a standalone scenario.

 I've provided it in the attachments.
 In the ZIP-file there is also a README.txt which contains the commands 
 which I called.

 And the line with the "strange" execution count is also mentioned.

 Thank you and kind regards,

 Alexander


 -Ursprüngliche Nachricht-
 Von: Martin Liška [mailto:mli...@suse.cz]
 Gesendet: Donnerstag, 22. Februar 2018 10:43
 An: Alexander Fichtinger ;
 gcc@gcc.gnu.org
 Betreff: Re: Problem using gcov

 On 02/21/2018 02:33 PM, Alexander Fichtinger wrote:
> When you look at line 23 it is called 1 time. Line 24 is called 2 times 
> (the bold one).
> We already looked in the disassembly but did not see a reason why the 
> line should be executed 2 times.
>
> Can anyone help here, please?
>
> Kind regards,
> Alex

 Hi.

 Sure, can you please send a self-contained test-case that I can run?

 Thank you,
 Martin

>>>
>>> No problem, I can confirm the problem:
>>> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84572
>>>
>>> Martin
>>>
>>
>>
> 
> 

Both GCC and GDB buildbot use gcc114

[Yao Qi]

Hi Paulo,
I noticed that GDB buildbot pending build queue on aarch64-linux
becomes longer and longer,
https://gdb-build.sergiodj.net/builders/Ubuntu-AArch64-m64
as it takes longer to finish each build and test.

Looks that you deployed aarch64-linux buildslave for GCC buildbot
on gcc114 as well, that is reason that GDB build and test is slowed
down (GCC build and test is slowed down too).

We'd better avoid using the same machine for two buildbots.  Are there
any easy way to move one buildslave to a different machine like gcc115
or gcc116.  As far as I know, they are identical.

-- 
Yao (齐尧)

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

Formally speaking, either implementation satisfies C11 because the standard 
allows much leeway in the interpretation here. But, of course, it is kind of 
annoying that double-width types (and that also includes potentially 64-bit on 
some 32-bit processors, e.g. i586 also has cmpxchg8b and no official way to 
read atomically otherwise) need special handling and compiler extensions which 
basically means that in a number of cases I cannot write portable code, I need 
to put a bunch of architecture-dependent ifdefs, for say, 64 bit atomics even. 
(And all this mess to accommodate almost non-existent case when someone wants 
to use atomic_load on read-only memory for wide types, in which no good 
solution exists anyway.)

Particularly, imagine when someones writes some lock-free code for different 
types (in templates, macros, etc). It basically uses same C11 atomic primitives 
but for various integer sizes. Now I need special handling for larger types 
because whatever libatomic provides does not guarantee lock-freedom (i.e., 
useless) which otherwise I do not need. True that wider types may not be 
available across all architectures, but I would prefer to have generic and 
standard-conformant code at least for those that have them.



> That's a valid goal, but it does not imply that we should mess with how
> atomics are implemented by default, nor should we mess with the default
> use cases.  This goal wants something special, and that is exposing the
> fact that *only* a CAS is available to synchronize atomically on a
>particular type.  That is an extension of the existing atomics design.

See above

> The standard doesn't specify read-only memory, so it also doesn't forbid
> the concept.  The implementation takes it into account though, and thus
> it's defined in that context.
But my point is that a programmer cannot rely on this feature anyway unless 
she/he wants to write code which compiles only with gcc. It is unspecified by 
the standard and implementations that use read-modify-write for atomic_load are 
perfectly valid. The whole point to have this standard in the first place is to 
allow code be compiled by different compilers, otherwise people can just rely 
on gcc-specific extensions.


> The topic we're currently discussing does not significantly affect when
> we can remove __sync builtins, IMO.

They are the only builtins that directly expose double-width operations. Short 
of using assembly fall-backs, they are the only option right now.

> They do care about whether atomic operations are natively supported on
> that particular type -- and that should include a load.
I think, the whole point to have atomic operations is ability to provide 
lock-free operations whenever possible. Even though standard does not guarantee 
it, that is almost the only sane use case. Otherwise, there is no point -- you 
can always use locks. If they do not care about lock-freedom, they should just 
use locks.


> Nobody is proposing to mark things as lock-free if they aren't.  Thus, I
> don't see any change to what's usable in signal handlers.
It is not obvious to anyone that atomic_load will block. It will *not* for 
single-width types. So, again we see differences for single- and double-width 
types. Even though you do not have problems with read-only memory, you have 
another problem for double-width types which may be even more subtle and much 
harder to debug in a number of cases. Of course, no one can make an assumption 
that it will not block, but the same can be said about read-only memory.
Anyway, I do not have a horse in the race... I just proposed to consider this 
change for a number of legitimate use cases, but it is eventually up to the gcc 
developers to decide.
-- Ruslan
   

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Szabolcs Nagy]

On 27/02/18 12:56, Ruslan Nikolaev wrote:

Formally speaking, either implementation satisfies C11 because the standard 
allows much leeway in the interpretation here.


no,

1) your proposal would make gcc non-conforming to iso c unless it changes how 
static const objects are emitted.
2) the two implementations are not abi compatible, the choice is already made, 
changing it is an abi break.
3) Torvald pointed out further considerations such as users expecting lock-free 
atomic loads to be faster than stores.

the solutions is to add a language extension, but that requires careful design.

Re: GSoC

[Richard Biener]

On Mon, Feb 26, 2018 at 5:35 PM, Dushyant Pratap Singh
 wrote:
> Hello GCC community,
> My name is Dushyant and I am interested in Gsoc at GCC .I am pursuing
> Integrated B.Tech (Computer Science) and M.Tech (software engineering) from
> Gautam Buddha University, India. Currently I am in 4th year of the
> programme.
>
> I have experience in Competitive Programming (primary language C++) and
> have won and participated in many of the Algorithmic & Data Structures
> competitions.
>
> Here is my Linkedin:
> https://www.linkedin.com/in/dushyant-pratap-singh-044658128/
>
> I have taken a course on Compiler Design in my last semester in which i
> scored A+ grade.
>
> Project that i am most interested in is Textual Representation of LTO
> Object Files.
>
> In my compiler lab I learned about the Compilation process how the
> preproccessed files (.i file) ,assembly code (.s file) ,object code (.o
> file) and finally executable file(a.out) were generated (as they were
> temporary files they needed an extra GCC flag -save-temps to store
> "temporary files" permanently) using gcc –Wall –save-temps prog1.c –o prog1
> .
>
> I have started to investigate on converting binary files to human readable
> forms.It would be great if someone could explain how to get started with
> the process.

The suggested way was to handle things similar to the gcov tools -- piggy-back
on the existing {lto,tree}-streamer-in.c files, #ifdef'ing relevant parts so you
can link them into a new lto-dump "driver" binary.

To get started you might want to look at lto/lto-object.c and/or follow the
lto1 binary when it processes its input files.

Eventually integrating the dumping of a textual representation into
the lto1 binary itself would also be feasible.  Still the actual dumping
should probably be intermangled with the binary reading code for
ease of maintainance.

Note the project isn't about providing a way of editing the textual
representation and reading that back in but only for for inspection / debugging.

There are several interesting pieces of information in the LTO IL so
having a way to dump only parts with a specifyable level of detail
is desirable (thus the suggestion to do this in a new binary rather
than lto1).

Thanks for the interest in this project!

Richard.


> Regards
>
> Dushyant Pratap Singh

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

> 1) your proposal would make gcc non-conforming to iso c unless it changes how 
> static const objects are emitted.
I do not think, ISO C requires to put const objects to .rodata. And it is 
easily solved by not placing it there for _Atomic objects that cannot be safely 
loaded from read-only memory.

> 2) the two implementations are not abi compatible, the choice is already 
> made, changing it is an abi break.
Since current implementations redirects to libatomic anyway, almost nothing 
should break. The only case it will break -- if somebody erroneously used 
atomic_load for 128-bit type on read-only memory (which is, again, not 
guaranteed by the standard). In practice, this case almost non-existent. The 
worst that may happen -- you will a segfault right away.

> 3) Torvald pointed out further considerations such as users expecting 
> lock-free atomic loads to be faster than stores.

Is it even true? Is it faster to use some global lock (implemented through RMW) 
than a single RMW operation? If you use this global lock, you will not get 
loads faster than stores.

   

Re: determining aggregate member from MEM_REF

[Richard Biener]

On Mon, Feb 26, 2018 at 9:04 PM, Jeff Law  wrote:
> On 02/26/2018 08:44 AM, Martin Sebor wrote:
>>
>> Folding things to MEM_REF (or rather, folding them too early)
>> makes all kinds of analysis harder: not just warnings but even
>> optimization.  I've raised a whole slew of bugs for the strlen
>> pass alone where folding string functions to MEM_REF defeats
>> useful downstream optimizations.  Making strlen (and all other
>> passes that might benefit from the original detail) work hard
>> to deal with MEM_REF isn't a good design solution.  It forces
>> the complexity that MEM_REF is meant to remove back into its
>> clients.  Worse, because of the loss of detail, the results
>> are unavoidably suboptimal (at least for certain kinds of
>> analyses).
> I haven't looked specifically at the MEM_REF folding, but I wouldn't be
> surprised to find cases where deferral ultimately results in regressions.

MEM_REF was introduced to fix representational shortcomings, not
mainly to make the representation more compact.  For example there
wasn't a way to recod that an access is volatile or un-aligned without
taking the address of sth, casting that thing and then dereferencing it.
That causes havoc in alias-analysis because many more things are
now addressable.  Also it was introduced to fix wrong-code bugs but
at the same time not cause missed optimizations.  That's when
propagating "structural addresses" into dereferences which is
generally invalid if you preserve the structure of the address.

With losing some of the canonicalization we could allow
MEM[&a.b.c.d, + 5] instead of forcing the .b.c.d into the constant
offset.  Or we could add a third operand so we have MEM[&a, +10, &a.b.c.d]
or so.  But all that comes at a cost and cannot solve all the issues
so I'm hesitant to do sth (costly) like that.

> When to fold & lower is a hard problem.  There is a constant tension
> between trying to fold early as it often leads to generally better code
> vs folding later which may help other sets of code, particularly when
> folding results in an inability to recover data.
>
> There generally is not an optimal solution to these problems; we have to
> take a pragmatic approach.  So if you can defer and not regress, then by
> all means propose patches.  But I think you'll find that to defer means
> you have to beef up stuff later in the pipeline.

Note we already do defer some stuff - not that I like that - but ultimately
you get to the point where defering hinders exactly the optimization you
want to perform to get good diagnostics.  So it remains a chicken-and-egg
issue.

Defering to a reasonable point also means that by practical means you
could have done a proper IPA static analysis pass in the first place.

Richard.

> jeff

Re: GSOC 2018 - Textual LTO dump tool project

[Richard Biener]

On Sun, Feb 25, 2018 at 10:46 AM, Martin Jambor  wrote:
> Hello Hrishikesh,
>
> I apologize for replying to you this late, this has been a busy week
> and now I am traveling.
>
> On Mon, Feb 19 2018, Hrishikesh Kulkarni wrote:
>> Hi,
>>
>> I am Hrishikesh Kulkarni currently studying as an undergrad student in
>> Computer Engineering at Pune University, India. I find compilers quite
>> interesting as a subject,  and would like to apply to GSoC to gain some
>> understanding of how real-world compilers work. So far, I have managed to
>> build gcc and perform some simple tweaks to the codebase. In particular, I
>> would like to apply to the Textual LTO dump tool project.
>>
>
> I must say I am impressed by the research you have already done.
> Nevertheless, please note that Ray Kim has also expressed interest in
> the project.  Martin Liska will be the mentor, so I will let him drive
> the selection process.  On the other hand, Ray also liked another
> project, so maybe he will pick that and everyone will be happy.
>
>> As far as I understand, the motivation for LTO framework was to enable
>> cross file interprocedural optimizations, and for this purpose an ipa pass
>> is divided into following three stages:
>>
>>1.
>>
>>LGEN: The pass does a local analysis of the function and generates a
>>“summary”, ie, the information relevant to the pass and writes it to LTO
>>object file.
>
> A pass might do that, but the output of the whole stage is not just the
> pass summaries, it also writes the function IL (the function gimple
> statements, above all) to the object file.
>
>>2.
>>
>>WPA: The LTO object files are given as input to the linker, which then
>>invokes the lto1 frontend to perform global ipa analysis over the
>>call-graph and write optimized summaries to LTO object files
>>(partitioning). The global ipa analysis is done over summary and not the
>>actual function bodies.
>
> Well... note that partitioning actually means dividing the whole
> compiled program/library into chunks that are then compiled
> independently in the LTRANS stage.  But you are basically right that WPA
> does also do whole-program analysis based on summaries and then writes
> its decisions to optimization summaries, yes.
>
>>3.
>
>>
>>LTRANS: The partitions are read back, and the function bodies are
>>reconstructed from summary and are then compiled to produce real object
>>files.
>
> Function bodies and the summaries are distinct things.  The body
> consists of gimple statements and all the associated stuff (such as
> types, so a lot of stuff), whereas when we refer to summaries, we mean
> small chunks of data that interprocedural optimizations such as inlining
> or IPA-CP scurry away because they cannot feasibly work on bodies of the
> entire program.
>
> But apart from this terminology issue, you are basically correct, at the
> LTRANS stage, IPA passes apply transformations to the bodies according
> to the optimization summary generated by the WPA phase.  And then, all
> normal, intra-procedural passes and code generation runs.
>
>>
>>
>> If I understand correctly, the motivation for textual LTO dump tool is to
>> easily analyze contents of LTO object file, similar to readelf or objdump ?
>
> That is how I understand it too, but Martin may have some further uses
> in mind.
>
>>
>> Assume that LTO object file contains in pureconst section: 0b0110 (0b for
>> binary prefix) corresponding to values of fs->pure_const_state and
>> fs->state_previously_known.
>>
>> If I understand correctly, the output of dump tool should then be:
>>
>> pure_const pass:
>>
>> pure_const_state = IPA_PURE (enum value of pure_const_state_e corresponding
>> to 0b01)
>>
>> state_previously_known = IPA_NEITHER (enum value of pure_const_state_e
>> corresponding to 0b10)
>>
>> Is this the expected output of the dump tool ?
>
> I think the tool would have to a bit more than just dumping summaries of
> IPA passes.  I tend to think that the task should also include dumping
> gimple bodies (but we already do that in GCC and so it should be mostly
> easy) and also of types (that are merged as one of the first steps of
> WPA and interesting things happen when mergingit does something
> "interesting").  And perhaps quite a bit more.  Martin?
>
>>
>> I am reasonably familiar working with C, C++ and python. My prior
>> experience includes opportunities to work in areas of NLP. Some of my
>> accomplishments in the area include presenting project VicharDhara- A
>> thought Mapper that was selected among top five ideas in Accenture
>> Innovation Challenge among 7000 nationwide entries. My paper on this topic
>> won the best paper award in IEEE Conference ICCUBEA-2017. My previous work
>> was focused on simple parsers, student psychology, thought process
>> detection for team selection.
>
> Interesting, congratulations.
>
>>
>> In the interim, I have been through a few docs on GCC and LTO [1][2][3] and
>> am trying to write 

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 12:56 +, Ruslan Nikolaev via gcc wrote:
> But, of course, it is kind of annoying that double-width types (and that also 
> includes potentially 64-bit on some 32-bit processors, e.g. i586 also has 
> cmpxchg8b and no official way to read atomically otherwise) need special 
> handling and compiler extensions which basically means that in a number of 
> cases I cannot write portable code, I need to put a bunch of 
> architecture-dependent ifdefs, for say, 64 bit atomics even.

The extension I outlined gives you a portable to use wide CAS, provided
that the particular implementation supports this extension.

Whether the standard covers this extension is a different matter.  You
can certainly propose it to the C and/or C++ committees, but that gets
easier if you can show existing practice.

> (And all this mess to accommodate almost non-existent case when someone wants 
> to use atomic_load on read-only memory for wide types, in which no good 
> solution exists anyway.)

You keep repeating that claim.  You also keep ignoring the point about
what kind of performance programs can expect atomic loads to have, if
they are declared lock-free.

Also note that the use case is *not* about wider-than-machine-word
accesses on read-only memory, but whether a portable use of atomics
(which doesn't have to consider machine word size) can use atomic loads
on read-only memory. 

> Particularly, imagine when someones writes some lock-free code for different 
> types (in templates, macros, etc). It basically uses same C11 atomic 
> primitives but for various integer sizes. Now I need special handling for 
> larger types because whatever libatomic provides does not guarantee 
> lock-freedom (i.e., useless) which otherwise I do not need.

If you use C11 atomics, you're bound to the usage intended by the
standard.  Which means that if you need lock freedom, you must check or
ensure that the implementation provides it to you.  If you make an
implicit assumption there beyond what the standard promises you, you are
not writing portable C11 concurrent code -- you are writing
architecture/platform-specific code.

Now, imagine someone writes atomic code using C11, and expects atomic
loads to actually perform like loads and lot like RMWs -- which is what
most concurrent code does, in particular lots of nonblocking  code ...

> True that wider types may not be available across all architectures, but I 
> would prefer to have generic and standard-conformant code at least for those 
> that have them.

It is conforming to the standard.

> > The standard doesn't specify read-only memory, so it also doesn't forbid
> > the concept.  The implementation takes it into account though, and thus
> > it's defined in that context.
> But my point is that a programmer cannot rely on this feature anyway unless 
> she/he wants to write code which compiles only with gcc.

She/he wants to rely on implementation-specific behavior, so that's not
a problem.

> It is unspecified by the standard and implementations that use 
> read-modify-write for atomic_load are perfectly valid. The whole point to 
> have this standard in the first place is to allow code be compiled by 
> different compilers, otherwise people can just rely on gcc-specific 
> extensions.

And they can, because GCC's behavior conforms to the standard.  It
doesn't have the implementation-specific properties you prefer, but
that's not about the standard but about your personal preferences.

> > The topic we're currently discussing does not significantly affect when
> > we can remove __sync builtins, IMO.
> 
> They are the only builtins that directly expose double-width operations. 
> Short of using assembly fall-backs, they are the only option right now.

We can still have an extension such as the one I outlined.

> > They do care about whether atomic operations are natively supported on
> > that particular type -- and that should include a load.
> I think, the whole point to have atomic operations is ability to provide 
> lock-free operations whenever possible. Even though standard does not 
> guarantee it, that is almost the only sane use case. Otherwise, there is no 
> point -- you can always use locks. If they do not care about lock-freedom, 
> they should just use locks.

The standards actually just promise you obstruction freedom.  Forward
progress guarantees are a part of the intention behind the lock-free
class, but not all of it.  There's address-freedom too, and an implicit
assumption about what rough class of performance a particular operation
is in.

The majority of synchronization code will care much more about
performance than about the operation being actually lock-free or not
(things like signal handlers or C++ unsequenced execution policy the are
exceptions).  Case in point: Lots of concurrent code built out of
lock-free atomics is actually not lock-free but has blocking parts -- so
it couldn't be used in things like signal handlers anyway.

> 
> > Nobody

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 13:04 +, Szabolcs Nagy wrote:
> the solutions is to add a language extension

I think this only needs a library interface, at least when we're just
considering the __atomic builtins.  On the C/C++ level, it might amount
to just another atomic type, which only has a CAS however; this could be
probably modelled entirely through default atomics (not implemented
though), and so wouldn't need a language or memory model extension. 

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 13:16 +, Ruslan Nikolaev via gcc wrote:
> > 3) Torvald pointed out further considerations such as users expecting 
> > lock-free atomic loads to be faster than stores.
> 
> Is it even true? Is it faster to use some global lock (implemented through 
> RMW) than a single RMW operation? If you use this global lock, you will not 
> get loads faster than stores.

If GCC declares a type as lock-free, atomic loads on this type will be
natively supported through some sort of load instruction.  That means
they are faster than stores under concurrent accesses, in particular
when there are concurrent atomic loads (for all major HW we care about).

If there is no natively supported atomic load, GCC will not declare the
type to be lock-free.

Nobody made statement about performance of locks vs. RMWs.

Re: GCC interpretation of C11 atomics (DR 459)

[Simon Wright]

On 27 Feb 2018, at 12:56, Ruslan Nikolaev via gcc  wrote:
> 
> And all this mess to accommodate almost non-existent case when someone wants 
> to use atomic_load on read-only memory for wide types, in which no good 
> solution exists anyway

Sorry to butt in, but - if it's ROM why would you need atomic load anyway? (of 
course, if it's just a constant view of the object, reason is obvious)

Re: GCC interpretation of C11 atomics (DR 459)

[Florian Weimer]

On 02/27/2018 05:40 PM, Simon Wright wrote:

Sorry to butt in, but - if it's ROM why would you need atomic load anyway? (of 
course, if it's just a constant view of the object, reason is obvious)


On many systems, the read-only nature of a memory region is a 
thread-local or process-local attribute.  Other parts of the system 
might have a different view on the same memory region.


Some CPUs support memory protection keys, which provide a cheap way to 
switch memory from read-only to read-write and back, and those switching 
operations deliberately do not involve a memory barrier.


Thanks,
Florian

Re: Fw: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

Torvald, thank you for your output, but I think, this discussion gets a little 
pointless. There is nothing else I can add since gcc folks are reluctant to 
this change anyway. In my opinion, there is no compelling reason against such 
an implementation (it is perfectly fine with the standard, read-only memory is 
not guaranteed for atomic_load anyway). Even binary compatibility that was 
mentioned is unlikely to be an issue if implemented as I described. And finally 
this is something that can actually be useful in practice (at least as far as I 
can judge from my experience). By the way, this issue was already raised 
multiple times during last couple of years by different people who actually use 
it for various real projects (bugs were eventually closed as 'INVALID').
All described challenges are purely technical and can easily be resolved. 
Moreover, clang/llvm chose this implementation, and it seems very logical and 
non-confusing to me. It certainly makes sense to expose hardware capabilities 
through standard interfaces whenever possible.

For my projects, I will simply fall back to my own implementation using inline 
assembly (at least for now) because, unfortunately, it is the only thing that 
is guaranteed to work outside of clang/llvm in the foreseeable future (__sync 
functions have some limitations and do not look like an attractive option 
either, by the way).



On Tuesday, February 27, 2018 11:21 AM, Torvald Riegel  
wrote:
 

 On Tue, 2018-02-27 at 13:16 +, Ruslan Nikolaev via gcc wrote:
> > 3) Torvald pointed out further considerations such as users expecting 
> > lock-free atomic loads to be faster than stores.
> 
> Is it even true? Is it faster to use some global lock (implemented through 
> RMW) than a single RMW operation? If you use this global lock, you will not 
> get loads faster than stores.

If GCC declares a type as lock-free, atomic loads on this type will be
natively supported through some sort of load instruction.  That means
they are faster than stores under concurrent accesses, in particular
when there are concurrent atomic loads (for all major HW we care about).

If there is no natively supported atomic load, GCC will not declare the
type to be lock-free.

Nobody made statement about performance of locks vs. RMWs.





   

Re: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 16:40 +, Simon Wright wrote:
> On 27 Feb 2018, at 12:56, Ruslan Nikolaev via gcc  wrote:
> > 
> > And all this mess to accommodate almost non-existent case when someone 
> > wants to use atomic_load on read-only memory for wide types, in which no 
> > good solution exists anyway
> 
> Sorry to butt in, but - if it's ROM why would you need atomic load anyway? 
> (of course, if it's just a constant view of the object, reason is obvious)

Consider a producer-consumer relationship between two processes where
the producer doesn't want to wait for the consumer.  For example, the
producer could be an application that's being traced, and the consumer
is a trace aggregation tool.  The producer can provide a read-only
mapping to the consumer, and put a nonblocking ring buffer or something
similar in there.  That allows the consumer to read, but it still needs
atomic access because the consumer is modifying the ring buffer
concurrently.

Re: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

> Consider a producer-consumer relationship between two processes where
> the producer doesn't want to wait for the consumer.  For example, the
> producer could be an application that's being traced, and the consumer
> is a trace aggregation tool.  The producer can provide a read-only
> mapping to the consumer, and put a nonblocking ring buffer or something
> similar in there.  That allows the consumer to read, but it still needs
> atomic access because the consumer is modifying the ring buffer
> concurrently.
Sorry for getting into someone's else conversation... And what good solution 
gcc offers right now? It forces producer and consumer to use lock-based (BTW: 
global lock!) approach for *both* producer and consumer if we are talking about 
128-bit types.  Therefore, sometimes producers *will* wait (by, effectively, 
blocking). Basically, it becomes useless. In this case, I would rather use a 
lock-based approach which at least does not use a global lock. On the contrary, 
the alternative implementation would have been at least useful when both 
producers and consumers have full (RW) access.

Anyway, I already said that I personally will go with assembly inlines for 
right now. I just wanted to raise this concern since other people may find it 
useful in their projects.


   

Re: GCC interpretation of C11 atomics (DR 459)

[Simon Wright]

On 27 Feb 2018, at 17:07, Torvald Riegel  wrote:
> 
> On Tue, 2018-02-27 at 16:40 +, Simon Wright wrote:
>> On 27 Feb 2018, at 12:56, Ruslan Nikolaev via gcc  wrote:
>>> 
>>> And all this mess to accommodate almost non-existent case when someone 
>>> wants to use atomic_load on read-only memory for wide types, in which no 
>>> good solution exists anyway
>> 
>> Sorry to butt in, but - if it's ROM why would you need atomic load anyway? 
>> (of course, if it's just a constant view of the object, reason is obvious)
> 
> Consider a producer-consumer relationship between two processes where
> the producer doesn't want to wait for the consumer.  For example, the
> producer could be an application that's being traced, and the consumer
> is a trace aggregation tool.  The producer can provide a read-only
> mapping to the consumer, and put a nonblocking ring buffer or something
> similar in there.  That allows the consumer to read, but it still needs
> atomic access because the consumer is modifying the ring buffer
> concurrently.

OK, got that, thanks (this is what I meant by "just a constant view of the 
object", btw).

Misled by "read-only memory" since in the embedded world ROM (usually actually 
in Flash) is effectively read-only to all.

Re: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 17:29 +, Ruslan Nikolaev wrote:
> 
> 
> > Consider a producer-consumer relationship between two processes where
> > the producer doesn't want to wait for the consumer.  For example, the
> > producer could be an application that's being traced, and the consumer
> > is a trace aggregation tool.  The producer can provide a read-only
> > mapping to the consumer, and put a nonblocking ring buffer or something
> > similar in there.  That allows the consumer to read, but it still needs
> > atomic access because the consumer is modifying the ring buffer
> > concurrently.
> Sorry for getting into someone's else conversation... And what good solution 
> gcc offers right now? It forces producer and consumer to use lock-based (BTW: 
> global lock!)

It's not one global lock, but a lock from an array of locks (global per
process, though).

> approach for *both* producer and consumer if we are talking about 128-bit 
> types.

But we're not talking about that special case of 128b types here.  The
majority of synchronization doesn't need more than machine word size.

> Therefore, sometimes producers *will* wait (by, effectively, blocking). 
> Basically, it becomes useless.

No, such a program would have a bug anyway.  It wouldn't even
synchronize properly.  Please make yourself familiar with what the
standard means by "address-free".  This use case needs address-free, so
that's what the program has to ensure (and it can test that portably).
Only lock-free gives you address-free.

> In this case, I would rather use a lock-based approach which at least does 
> not use a global lock.

The lock would need to be shared between processes in the example I
gave.  You have to build your own lock for that currently, because C/C++
don't give you any process-shared locks.

Re: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

> But we're not talking about that special case of 128b types here.  The
> majority of synchronization doesn't need more than machine word size.
Then why do you worry about read-only access for 128b types? (it is a special 
case anyway).

> No, such a program would have a bug anyway.  It wouldn't even
> synchronize properly. 

Therefore, it was not a valid example / use case (for 128-bit) in the first 
place. It was a *valid* example for smaller atomics, though. But that is 
exactly my point -- your current solution for 128 bit does not add any 
practical value except when you want to use lock-based solution (but see my 
explanation below).


> The lock would need to be shared between processes in the example I
> gave.  You have to build your own lock for that currently, because C/C++
> don't give you any process-shared locks.
At least in Linux, you can simply use eventfd(2) to reliably do it (without 
relying on "array of locks"). Given that it is not a very common use case, does 
not seem to need to have special C standard for this. And whatever C11 provides 
can not be relied upon anyway since you do not have strict guarantees that 
read-only memory is supported for larger types. For example, clang (and 
possibly other compilers) will break this assumption. At least, I would prefer 
to use eventfd in my application (if ever needed at all) since it has reliable 
and well-defined behavior in Linux.


   

Autodesk Users List.

[Amy Hodge]

Hi,





I would like to know if you are interested in acquiring Network Software Users 
List.



Information fields we provide for each contact in the list: MYVRMS, Volanté 
Systems, TravelNet Solutions, Givex, InnQuest Software, TurfCare Supply Corp, 
Software Developers Inc, and Software Development Technologies, TurfCare Supply 
Corp.



Let me know if you are interested in any other technology or industry so that I 
can get back to you with the counts and pricing.



Regards,

Amy Hodge

Marketing Executive

To opt out, please reply with OPT OUT in the Subject Line.

Re: Both GCC and GDB buildbot use gcc114

[Paulo Matos]

On 27/02/18 13:53, Yao Qi wrote:
> Hi Paulo,
> I noticed that GDB buildbot pending build queue on aarch64-linux
> becomes longer and longer,
> https://gdb-build.sergiodj.net/builders/Ubuntu-AArch64-m64
> as it takes longer to finish each build and test.
> 
> Looks that you deployed aarch64-linux buildslave for GCC buildbot
> on gcc114 as well, that is reason that GDB build and test is slowed
> down (GCC build and test is slowed down too).
> 
> We'd better avoid using the same machine for two buildbots.  Are there
> any easy way to move one buildslave to a different machine like gcc115
> or gcc116.  As far as I know, they are identical.
>

Apologies for the clash on resources. Using gcc115 and gcc116 only now.

Kind regards,

-- 
Paulo Matos

Re: GCC interpretation of C11 atomics (DR 459)

[Torvald Riegel]

On Tue, 2018-02-27 at 19:32 +, Ruslan Nikolaev wrote:
> 
> > But we're not talking about that special case of 128b types here.  The
> > majority of synchronization doesn't need more than machine word size.
> Then why do you worry about read-only access for 128b types? (it is a special 
> case anyway).

The types that are machine-word-sized and less determine what the
default is.  The 128b types are the outlier, and they should conform to
what we want for all other types.  The context of my statement was which
the producer-consumer example I brought up.

> > No, such a program would have a bug anyway.  It wouldn't even
> > synchronize properly. 
> 
> Therefore, it was not a valid example / use case (for 128-bit) in the first 
> place.

You brought up the 128b case, and claimed that there was no proper
solution for it.  Now you are saying it's an invalid example.  If it is,
then I don't see what you are trying to argue for.

> It was a *valid* example for smaller atomics, though. But that is exactly my 
> point -- your current solution for 128 bit does not add any practical value 
> except when you want to use lock-based solution (but see my explanation 
> below).
> 
> 
> > The lock would need to be shared between processes in the example I
> > gave.  You have to build your own lock for that currently, because C/C++
> > don't give you any process-shared locks.
> At least in Linux, you can simply use eventfd(2) to reliably do it (without 
> relying on "array of locks").

You wanted to have something that's portable, and focused on the C
standard.  Now you are saying this doesn't matter anymore.  This seems
inconsistent.
My statement was in the context of portable C/C++ programs.

> Given that it is not a very common use case, does not seem to need to have 
> special C standard for this.

Did you mean to say that the fact that there's no support in ISO C for
it means that it's not a use case it matters?  If so, that's not the
case.  The standards simply don't support all use cases that have
relevance to their users.

> And whatever C11 provides can not be relied upon anyway since you do not have 
> strict guarantees that read-only memory is supported for larger types.

We discussed this topic already.  I'm not going to repeat myself.

> For example, clang (and possibly other compilers) will break this assumption. 
> At least, I would prefer to use eventfd in my application (if ever needed at 
> all) since it has reliable and well-defined behavior in Linux.

I don't have any opinion on what you personally use or don't use.

I care about GCC users as a whole, of course.  But it seems to me we
discussed all aspects of this, so I'd prefer this discussion to
eventually conclude.

Re: GCC interpretation of C11 atomics (DR 459)

[Ruslan Nikolaev via gcc]

Torvald, I think this discussion, indeed, gets pointless. Some of your 
responses clearly take my comments out of larger picture and context of the 
discussion.
One thing is clear that either implementation is fine with the standard 
(formally speaking) simply because the standard allows too much leeway on how 
you implement atomics. In fact, as I mentioned clang/llvm implements it 
differently. I find it as a weakness of the standard, actually, because for 
portable (across different compilers), the only thing you can more or less 
safely assume are single-width types.
Thank you for your output and discussion.

Re: Create a new mirror

[KoDDoS Mirror]

Hello Gerald,

We added missing cronjob to update it. It should be updated in less than 
6 hours.


Regards
Martin


Hi Martin,

On Mon, 6 Nov 2017, KoDDoS Mirror wrote:

The mirror is setup. Here is a patch as I do not have write access to
push it.

I was just going to commit the following patch (sorry for the
delay), alas noticed your mirror does not seem to, umm, mirror?

The expectation is that you'll mirror additions/deletions on a
regular base (ideally no less than daily).  Are you planning to
put that in place?

Gerald

Index: mirrors.html
===
RCS file: /cvs/gcc/wwwdocs/htdocs/mirrors.html,v
retrieving revision 1.246
diff -u -r1.246 mirrors.html
--- mirrors.html22 Oct 2017 11:46:12 -  1.246
+++ mirrors.html26 Feb 2018 21:16:57 -
@@ -40,6 +40,10 @@
ftp://nl.mirror.babylon.network/gcc/";>ftp://nl.mirror.babylon.network/gcc/
 |
rsync://nl.mirror.babylon.network/gcc/,
thanks to Tim Semeijn (noc@babylon.network) at Babylon Network.
+The Netherlands, Dronten:
+  http://mirror.koddos.net/gcc/";>http://mirror.koddos.net/gcc/ |
+  rsync://mirror.koddos.net/gcc/,
+  thanks to Martin (mir...@koddos.net) at KoDDoS.
  The Netherlands, Nijmegen: ftp://ftp.nluug.nl/mirror/languages/gcc";>ftp.nluug.nl, thanks to Jan Cristiaan van 
Winkel (jc at ATComputing.nl)
  Russia, Novosibirsk: http://mirror.linux-ia64.org/gnu/gcc/";>http://mirror.linux-ia64.org/gnu/gcc/, thanks to 
Daniel Volchixin 
  Slovakia, Bratislava: http://gcc.fyxm.net/";>gcc.fyxm.net, thanks 
to Jan Teluch (admin at 2600.sk)

Re: finding bugs deferred from previous releases

[Alexandre Oliva]

On Feb  8, 2018, Richard Biener  wrote:

> Add a 'defered' keyword?

Done:

  deferred:

This bug was deemed too risky to attempt to fix during stabilization
stages.  Deferred to a development stage of a subsequent release.

-- 
Alexandre Oliva, freedom fighterhttp://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist|Red Hat Brasil GNU Toolchain Engineer