Re: ASAN status and glibc 2.27

[Yuri Gribov]

On Mon, Mar 19, 2018 at 10:17 AM, Vincent Lefevre
 wrote:
> Hi,
>
> Any news about the ASAN compatibility with glibc 2.27 on x86?
> Will this be fixed soon? This is important as this is a blocker.
>
> FYI, I had reported:
>
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84761

Asan runtime library is developed in LLVM and only occasionally merged
to GCC so I don't think you'll get much support in this list. Maybe
you could comment in their tracker
(https://github.com/google/sanitizers/issues/914)? AFAIK they've been
discussing with Glibc community recently, they ideally want to fix it
once and for all with new Glibc ThreadProperties API
(https://sourceware.org/ml/libc-alpha/2018-02/msg00567.html). It would
be great it someone volunteered to implement this...

-Y

Re: gcc behavior on memory exhaustion

[Yuri Gribov]

On Wed, Aug 9, 2017 at 3:14 PM, Andreas Schwab  wrote:
> On Aug 09 2017, Yuri Gribov  wrote:
>
>> On Wed, Aug 9, 2017 at 2:49 PM, Andrew Haley  wrote:
>>> On 09/08/17 14:05, Andrew Roberts wrote:
>>>> 2) It would be nice to see some sort of out of memory error, rather than
>>>> just an ICE.
>>>
>>> There's nothing we can do: the kernel killed us.  We can't emit any
>>> message before we die.  (killed) tells you that we were killed, but
>>> we don't know who done it.
>>
>> Well, driver could check syslog...
>
> The syslog is very system dependent and may not even be readable by
> unprivileged users.

It's best-effort of course.

> Andreas.
>
> --
> Andreas Schwab, SUSE Labs, sch...@suse.de
> GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
> "And now for something completely different."

Re: gcc behavior on memory exhaustion

[Yuri Gribov]

On Wed, Aug 9, 2017 at 2:49 PM, Andrew Haley  wrote:
> On 09/08/17 14:05, Andrew Roberts wrote:
>> 2) It would be nice to see some sort of out of memory error, rather than
>> just an ICE.
>
> There's nothing we can do: the kernel killed us.  We can't emit any
> message before we die.  (killed) tells you that we were killed, but
> we don't know who done it.

Well, driver could check syslog...

> --
> Andrew Haley
> Java Platform Lead Engineer
> Red Hat UK Ltd. 
> EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: Killing old dead bugs

[Yuri Gribov]

On Wed, Jul 19, 2017 at 7:15 PM, Eric Gallager  wrote:
> On 7/18/17, Yuri Gribov  wrote:
>> On Tue, Jul 18, 2017 at 3:54 PM, Martin Sebor  wrote:
>>> On 07/17/2017 02:25 PM, Yuri Gribov wrote:
>>>>
>>>> On Mon, Jul 17, 2017 at 4:23 PM, Martin Sebor  wrote:
>>>>>
>>>>> On 07/17/2017 02:14 AM, Yuri Gribov wrote:
>>>>>>
>>>>>>
>>>>>> Hi Mikhail,
>>>>>>
>>>>>> On Sun, Jul 2, 2017 at 6:39 PM, Mikhail Maltsev 
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> Hi. Yes, bug maintenance is appreciated. See this message and replies
>>>>>>> to it: https://gcc.gnu.org/ml/gcc/2016-04/msg00258.html .
>>>>>>
>>>>>>
>>>>>>
>>>>>> Replies in your link suggest to leave a final comment in bugs with
>>>>>> explanatory suggestion to close them so that maintainers who read
>>>>>> gcc-bugs list hopefully notice them and act appropriately.
>>>>>> Unfortunately I found this to _not_ work in practice. Below you can
>>>>>> see a list of bugs I've investigated (and often bisected) in the past
>>>>>> weeks - none of them were closed by maintainers (or at least
>>>>>> commented).
>>>>>>
>>>>>> So I'm afraid we have to conclude that there's no working process to
>>>>>> close stale bugs in place (which may be one of the reasons of bugcount
>>>>>> growth).
>>>>>
>>>>>
>>>>>
>>>>> The informal process that some (most?) of us have been following
>>>>> is to close them with a comment explaining our rationale.
>>>>> It's good to fill in the Known to fail/Known to work versions if they
>>>>> can be determined.  Mentioning the commit that fixed the bug as
>>>>> you did for the bugs below is ideal.  Adding a test case if one
>>>>> doesn't exist in the test suite is also very useful, though quite
>>>>> a bit more work.  In my experience, if a bug is closed that should
>>>>> stay open, someone usually notices pretty quickly and reopens it,
>>>>> so I wouldn't be too worried about doing something wrong.
>>>>
>>>>
>>>> Martin,
>>>>
>>>> Firstly, thanks for detailed explanation.
>>>>
>>>> What to do about bugs originating in upstream packages?  I noticed
>>>> they sometimes get closed with "RESOLVED MOVED" resolution
>>>> (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58841) but often this
>>>> does not happen and they just hang in tracker forever for no good
>>>> reason.
>>>>
>>>> Actually what I tried to emphasize is that it's impossible for a
>>>> casual commiter (who does not have maintainer access to Bugzilla i.e.
>>>> rights to close bugs) to contribute to project by cleaning stale bugs,
>>>> because requests to close them are mostly ignored (because
>>>> maintainers, obviously, have much more interesting work to do).
>>>
>>>
>>> I take your point.  I didn't realize closing bugs was restricted.
>>> Given the work you've done on the bugs below (and elsewhere) you
>>> should be able to close them.  If you aren't and would like to be
>>> able to, please request it by emailing overse...@gcc.gnu.org ((at
>>> least I think that's the right way to go about it), or follow up
>>> here and I'm sure someone with the right karma will make it happen.
>>
>> Jonathan also mentioned something not immediately obvious in IRC:
>> logging into BZ with gcc.gnu.org account provides elevated privileges.
>> So if you have write access, you should get extra BZ rights for free.
>>
>
> Is there a way to do this with a password thru a graphical web
> browser? The instructions I found on the SVN write page
> <https://gcc.gnu.org/svnwrite.html> only mentioned ssh-ing into the
> server. It says "Your gcc.gnu.org account... is what you use for
> Bugzilla" but doesn't actually say how to do that.

AFAIR the only way to login to BZ with GNU account is to reset
password (so that BZ sends reset email to the linked account which you
specified via `ssh usern...@gcc.gnu.org email ...').

> I tried using the
> same password as I do for the Bugzilla account linked to the email
> that my gcc

Re: Killing old dead bugs

[Yuri Gribov]

On Tue, Jul 18, 2017 at 3:54 PM, Martin Sebor  wrote:
> On 07/17/2017 02:25 PM, Yuri Gribov wrote:
>>
>> On Mon, Jul 17, 2017 at 4:23 PM, Martin Sebor  wrote:
>>>
>>> On 07/17/2017 02:14 AM, Yuri Gribov wrote:
>>>>
>>>>
>>>> Hi Mikhail,
>>>>
>>>> On Sun, Jul 2, 2017 at 6:39 PM, Mikhail Maltsev 
>>>> wrote:
>>>>>
>>>>>
>>>>> Hi. Yes, bug maintenance is appreciated. See this message and replies
>>>>> to it: https://gcc.gnu.org/ml/gcc/2016-04/msg00258.html .
>>>>
>>>>
>>>>
>>>> Replies in your link suggest to leave a final comment in bugs with
>>>> explanatory suggestion to close them so that maintainers who read
>>>> gcc-bugs list hopefully notice them and act appropriately.
>>>> Unfortunately I found this to _not_ work in practice. Below you can
>>>> see a list of bugs I've investigated (and often bisected) in the past
>>>> weeks - none of them were closed by maintainers (or at least
>>>> commented).
>>>>
>>>> So I'm afraid we have to conclude that there's no working process to
>>>> close stale bugs in place (which may be one of the reasons of bugcount
>>>> growth).
>>>
>>>
>>>
>>> The informal process that some (most?) of us have been following
>>> is to close them with a comment explaining our rationale.
>>> It's good to fill in the Known to fail/Known to work versions if they
>>> can be determined.  Mentioning the commit that fixed the bug as
>>> you did for the bugs below is ideal.  Adding a test case if one
>>> doesn't exist in the test suite is also very useful, though quite
>>> a bit more work.  In my experience, if a bug is closed that should
>>> stay open, someone usually notices pretty quickly and reopens it,
>>> so I wouldn't be too worried about doing something wrong.
>>
>>
>> Martin,
>>
>> Firstly, thanks for detailed explanation.
>>
>> What to do about bugs originating in upstream packages?  I noticed
>> they sometimes get closed with "RESOLVED MOVED" resolution
>> (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58841) but often this
>> does not happen and they just hang in tracker forever for no good
>> reason.
>>
>> Actually what I tried to emphasize is that it's impossible for a
>> casual commiter (who does not have maintainer access to Bugzilla i.e.
>> rights to close bugs) to contribute to project by cleaning stale bugs,
>> because requests to close them are mostly ignored (because
>> maintainers, obviously, have much more interesting work to do).
>
>
> I take your point.  I didn't realize closing bugs was restricted.
> Given the work you've done on the bugs below (and elsewhere) you
> should be able to close them.  If you aren't and would like to be
> able to, please request it by emailing overse...@gcc.gnu.org ((at
> least I think that's the right way to go about it), or follow up
> here and I'm sure someone with the right karma will make it happen.

Jonathan also mentioned something not immediately obvious in IRC:
logging into BZ with gcc.gnu.org account provides elevated privileges.
So if you have write access, you should get extra BZ rights for free.

>>> The process for managing bugs is in more detail described here:
>>>
>>>   https://gcc.gnu.org/bugs/management.html
>>>
>>> If you think it should be clarified in some way please feel free
>>> to send in a patch.
>>>
>>> Martin
>>>
>>>
>>>>
>>>> * Bug 41992 - ICE on invalid dereferencing of void *
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00860.html)
>>>> * Bug 63245 - renderMemorySnippet shouldn't show more bytes than the
>>>> underlying type
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00645.html)
>>>> * Bug 61693 - [asan] is not intercepting aligned_alloc
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00643.html)
>>>> * Bug 61771 - Test failures in ASan testsuite on ARM Linux due to FP
>>>> format mismatch between libasan and GCC
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00646.html)
>>>> * Bug 78028 - ASAN doesn't find memory leak
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00653.html)
>>>> * Bug 55316 - gcc/libsanitizer/asan/asan_linux.cc:70:3: error: #error
>>>> "Unsupported arch"
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00636.html)
>>>> * Bug 78654 - ubsan can lead to excessive stack usage
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00640.html)
>>>> * Bug 60892 - GCC (libsanitizer) fails to build with Linux 2.6.21
>>>> headers (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00649.html)
>>>> * Bug 61995 - gcc 4.9.1 fails to compile with error in libsanitizer
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00648.html)
>>>> * Bug 80027 - ASAN breaks DT_RPATH $ORIGIN in dlopen()
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00787.html)
>>>> * Bug 54123 - inline functions not optimized as well as static inline
>>>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg01321.html)
>>>>
>>>> -Y
>>>>
>>>
>

Re: Killing old dead bugs

[Yuri Gribov]

On Mon, Jul 17, 2017 at 4:23 PM, Martin Sebor  wrote:
> On 07/17/2017 02:14 AM, Yuri Gribov wrote:
>>
>> Hi Mikhail,
>>
>> On Sun, Jul 2, 2017 at 6:39 PM, Mikhail Maltsev 
>> wrote:
>>>
>>> Hi. Yes, bug maintenance is appreciated. See this message and replies
>>> to it: https://gcc.gnu.org/ml/gcc/2016-04/msg00258.html .
>>
>>
>> Replies in your link suggest to leave a final comment in bugs with
>> explanatory suggestion to close them so that maintainers who read
>> gcc-bugs list hopefully notice them and act appropriately.
>> Unfortunately I found this to _not_ work in practice. Below you can
>> see a list of bugs I've investigated (and often bisected) in the past
>> weeks - none of them were closed by maintainers (or at least
>> commented).
>>
>> So I'm afraid we have to conclude that there's no working process to
>> close stale bugs in place (which may be one of the reasons of bugcount
>> growth).
>
>
> The informal process that some (most?) of us have been following
> is to close them with a comment explaining our rationale.
> It's good to fill in the Known to fail/Known to work versions if they
> can be determined.  Mentioning the commit that fixed the bug as
> you did for the bugs below is ideal.  Adding a test case if one
> doesn't exist in the test suite is also very useful, though quite
> a bit more work.  In my experience, if a bug is closed that should
> stay open, someone usually notices pretty quickly and reopens it,
> so I wouldn't be too worried about doing something wrong.

Martin,

Firstly, thanks for detailed explanation.

What to do about bugs originating in upstream packages?  I noticed
they sometimes get closed with "RESOLVED MOVED" resolution
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58841) but often this
does not happen and they just hang in tracker forever for no good
reason.

Actually what I tried to emphasize is that it's impossible for a
casual commiter (who does not have maintainer access to Bugzilla i.e.
rights to close bugs) to contribute to project by cleaning stale bugs,
because requests to close them are mostly ignored (because
maintainers, obviously, have much more interesting work to do).

> The process for managing bugs is in more detail described here:
>
>   https://gcc.gnu.org/bugs/management.html
>
> If you think it should be clarified in some way please feel free
> to send in a patch.
>
> Martin
>
>
>>
>> * Bug 41992 - ICE on invalid dereferencing of void *
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00860.html)
>> * Bug 63245 - renderMemorySnippet shouldn't show more bytes than the
>> underlying type
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00645.html)
>> * Bug 61693 - [asan] is not intercepting aligned_alloc
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00643.html)
>> * Bug 61771 - Test failures in ASan testsuite on ARM Linux due to FP
>> format mismatch between libasan and GCC
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00646.html)
>> * Bug 78028 - ASAN doesn't find memory leak
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00653.html)
>> * Bug 55316 - gcc/libsanitizer/asan/asan_linux.cc:70:3: error: #error
>> "Unsupported arch"
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00636.html)
>> * Bug 78654 - ubsan can lead to excessive stack usage
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00640.html)
>> * Bug 60892 - GCC (libsanitizer) fails to build with Linux 2.6.21
>> headers (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00649.html)
>> * Bug 61995 - gcc 4.9.1 fails to compile with error in libsanitizer
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00648.html)
>> * Bug 80027 - ASAN breaks DT_RPATH $ORIGIN in dlopen()
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00787.html)
>> * Bug 54123 - inline functions not optimized as well as static inline
>> (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg01321.html)
>>
>> -Y
>>
>

Re: Killing old dead bugs

[Yuri Gribov]

Hi Mikhail,

On Sun, Jul 2, 2017 at 6:39 PM, Mikhail Maltsev  wrote:
> Hi. Yes, bug maintenance is appreciated. See this message and replies
> to it: https://gcc.gnu.org/ml/gcc/2016-04/msg00258.html .

Replies in your link suggest to leave a final comment in bugs with
explanatory suggestion to close them so that maintainers who read
gcc-bugs list hopefully notice them and act appropriately.
Unfortunately I found this to _not_ work in practice. Below you can
see a list of bugs I've investigated (and often bisected) in the past
weeks - none of them were closed by maintainers (or at least
commented).

So I'm afraid we have to conclude that there's no working process to
close stale bugs in place (which may be one of the reasons of bugcount
growth).

* Bug 41992 - ICE on invalid dereferencing of void *
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00860.html)
* Bug 63245 - renderMemorySnippet shouldn't show more bytes than the
underlying type
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00645.html)
* Bug 61693 - [asan] is not intercepting aligned_alloc
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00643.html)
* Bug 61771 - Test failures in ASan testsuite on ARM Linux due to FP
format mismatch between libasan and GCC
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00646.html)
* Bug 78028 - ASAN doesn't find memory leak
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00653.html)
* Bug 55316 - gcc/libsanitizer/asan/asan_linux.cc:70:3: error: #error
"Unsupported arch"
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00636.html)
* Bug 78654 - ubsan can lead to excessive stack usage
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00640.html)
* Bug 60892 - GCC (libsanitizer) fails to build with Linux 2.6.21
headers (https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00649.html)
* Bug 61995 - gcc 4.9.1 fails to compile with error in libsanitizer
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00648.html)
* Bug 80027 - ASAN breaks DT_RPATH $ORIGIN in dlopen()
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg00787.html)
* Bug 54123 - inline functions not optimized as well as static inline
(https://gcc.gnu.org/ml/gcc-bugs/2017-07/msg01321.html)

-Y

Re: Linux and Windows generate different binaries

[Yuri Gribov]

On Sat, Jul 15, 2017 at 10:01 PM, Alexander Monakov  wrote:
> On Fri, 14 Jul 2017, Yuri Gribov wrote:
>> I've also detect transitiveness violation compare_assert_loc
>> (tree-vrp.c), will send fix once tests are done.
>
> There are more issues still, see the thread starting at
> https://gcc.gnu.org/ml/gcc-patches/2017-07/msg00899.html

Nice!

I've also reproduced some of these bugs (tree-vrp.c and
gimple-ssa-store-merging.c). But these are transitiveness errors
whereas this thread is mainly about unstable sorts so I didn't report
them here.

-Y

Re: Linux and Windows generate different binaries

[Yuri Gribov]

On Fri, Jul 14, 2017 at 8:45 AM, Yuri Gribov  wrote:
> FWIW I've done a quick analysis of recent gcc source code using
> https://github.com/yugr/sortcheck and found lots of comparison
> functions which can return 0 for different objects.
>
> All these may cause arrays to be sorted differently on different
> platforms but it's not immediately clear whether this may cause actual
> difference in generated code.
>
> Here's a list for cc1 (I can analyze other executables e.g. generators
> and cc1plus if needed).

Looked at generators, we have three comparison routines which return 0
for different objects but all seem to be safe i.e. can't influence
code generated by GCC.

alt_state_cmp (genautomata.c) - intentional, duplicates are removed afterwards.

optab_rcode_cmp (genopinit.c) - compares objects with rcode == UNKNOWN
as equal but that's fine as later codes treats them in the same way
(sets their code_to_optab_ table to unknown_optab).

subroutine_candidate_cmp (genrecog.c) - this compares function
candidates with same number of statements in them as equal. This may
cause different split of state recognition code to recog_%d
subroutines but should not change semantics of code as a whole.

-Y

Re: Linux and Windows generate different binaries

[Yuri Gribov]

On Thu, Jul 13, 2017 at 4:56 AM, Klaus Kruse Pedersen (Klaus)
 wrote:
> On Wed, 2017-07-12 at 08:57 -0600, Sandra Loosemore wrote:
>> On 07/12/2017 05:07 AM, Klaus Kruse Pedersen (Klaus) wrote:
>> > I have seen reproducible builds being discussed here, but what is
>> > the
>> > position on inter c-lib and OS reproducible builds?
>>
>> I think we consider unstable sort problems bugs and have fixed them
>> in
>> the past.  Bugzilla search turned up #28964 and I remember at least
>> one
>> more recent instance of this as well (although not the details any
>> more).
>
>
> Yes, 28964 is similar to the issue I was hit by.
>
> By extension, does that mean that all qsort compare/rank functions that
> can return 0, should be considered buggy?
>
> I went through a some of the 140'ish rank functions - and it does
> indeed look like considerable effort went into returning only +1 and
> -1.
>
> A general pattern seem to be:
>
> return da ? 1 : -1;
>
> And comments like:
>
>   /* If regs are equally good, sort by their numbers, so that the
>  results of qsort leave nothing to chance.  */
>
>
> But there are exceptions, all rank functions in
>
> tree-ssa-loop-ivopts.c,
> tree-ssa-loop-niter.c
> tree-ssa-loop-im.c
>
> can return 0.

FWIW I've done a quick analysis of recent gcc source code using
https://github.com/yugr/sortcheck and found lots of comparison
functions which can return 0 for different objects.

All these may cause arrays to be sorted differently on different
platforms but it's not immediately clear whether this may cause actual
difference in generated code.

Here's a list for cc1 (I can analyze other executables e.g. generators
and cc1plus if needed).

allocno_hard_regs_compare (ira-color.c) - sorts registers according to
their spill cost.

compare_access_positions (tree-sra.c) - accesses to same var in
different statements may be compared equal e.g.
  (gdb) p debug_gimple_stmt((**(access_p *)prev).stmt)
  # VUSE <.MEM_6(D)>
  _1 = s_7(D)->b;
  $6 = void
  (gdb) p debug_gimple_stmt((**(access_p *)val).stmt)
  # .MEM_11 = VDEF <.MEM_10>
  s_7(D)->b = 0B;
  $7 = void

sort_bbs_in_loop_postorder_cmp (tree-ssa-loop-im.c) - basic blocks
sorted according to loop in which they occur.

sort_locs_in_loop_postorder_cmp (tree-ssa-loop-im.c) - ditto for memrefs.

group_compare_offset (tree-ssa-loop-ivopts.c) - accesses in different
statements may be compared equal e.g.
  (gdb) cal debug_gimple_stmt((**(struct iv_use **)prev).stmt)
  # .MEM_626 = VDEF <.MEM_346>
  adpm[i_313] = adpm[_24];
  (gdb) cal debug_gimple_stmt((**(struct iv_use **)val).stmt)
  # .MEM_627 = VDEF <.MEM_626>
  adpm[i_313].next = _25;

common_cand_cmp (tree-ssa-loop-ivopts.c) - sorts IV candidates based
on number of uses.

object_range_compare_func (ira-build.c) - compares equal for different
subregs of the same registers e.g.
  (gdb) p **(ira_object_t *)val
  $4 = {allocno = 0x80, conflicts_array = 0x20, live_ranges =
0x7614e180, subword = -169917888, conflicts_array_size = 32767,
  id = -160201072, min = 32767, max = -169985296, conflict_hard_regs =
{0, 0}, total_conflict_hard_regs = {0, 0},
  num_accumulated_conflicts = 0, conflict_vec_p = 0}
  (gdb) p **(ira_object_t *)prev
  $5 = {allocno = 0x80, conflicts_array = 0x20, live_ranges =
0x7614e180, subword = -166434272, conflicts_array_size = 32767,
  id = -160201072, min = 32767, max = -169985776, conflict_hard_regs =
{0, 0}, total_conflict_hard_regs = {0, 0},
  num_accumulated_conflicts = 0, conflict_vec_p = 0}

compare_address_parts (loop-invariant.c) - sorts rtx based on operator
precedence.

I've also detect transitiveness violation compare_assert_loc
(tree-vrp.c), will send fix once tests are done.

-Y

Re: Linux and Windows generate different binaries

[Yuri Gribov]

On Thu, Jul 13, 2017 at 4:56 AM, Klaus Kruse Pedersen (Klaus)
 wrote:
> On Wed, 2017-07-12 at 08:57 -0600, Sandra Loosemore wrote:
>> On 07/12/2017 05:07 AM, Klaus Kruse Pedersen (Klaus) wrote:
>> > I have seen reproducible builds being discussed here, but what is
>> > the
>> > position on inter c-lib and OS reproducible builds?
>>
>> I think we consider unstable sort problems bugs and have fixed them
>> in
>> the past.  Bugzilla search turned up #28964 and I remember at least
>> one
>> more recent instance of this as well (although not the details any
>> more).
>
>
> Yes, 28964 is similar to the issue I was hit by.
>
> By extension, does that mean that all qsort compare/rank functions that
> can return 0, should be considered buggy?
>
> I went through a some of the 140'ish rank functions - and it does
> indeed look like considerable effort went into returning only +1 and
> -1.
>
> A general pattern seem to be:
>
> return da ? 1 : -1;
>
> And comments like:
>
>   /* If regs are equally good, sort by their numbers, so that the
>  results of qsort leave nothing to chance.  */
>
>
> But there are exceptions, all rank functions in
>
> tree-ssa-loop-ivopts.c,
> tree-ssa-loop-niter.c
> tree-ssa-loop-im.c
>
> can return 0.

One more issue with some of qsort callbacks is that they do not always
satisfy ordering axioms which in practice may result in random
variations in output. I once reported this in
https://gcc.gnu.org/ml/gcc-patches/2015-12/msg02141.html but didn't
follow up.

-Y

Re: x86 branches vs conditional moves

[Yuri Gribov]

On Sat, Jul 8, 2017 at 12:30 AM, Michael Clark  wrote:
> Hi,
>
> Curious about this codegen:
>
> - https://godbolt.org/g/5XxP5S
>
> Why does gcc branch on _Bool, but emits a conditional move for an integer? 
> can it emit cmovne instead of branching? also curious where one would change 
> this to learn about GCC internals.

Seems to be caused by a limitation in ifconvert pass which can not
handle slightly more complex code in select_bool.

Return there contains implicit != operations on a and b which clobbers
 CC register:
  (insn 8 7 9 3 (set (reg:CCZ 17 flags)
(compare:CCZ (mem/c:SI (symbol_ref:DI ("a"))
(const_int 0 [0]
  (insn 9 8 13 3 (set (reg:QI 90 [  ])
(ne:QI (reg:CCZ 17 flags)
(const_int 0 [0]
(RTL filtered for brevity). This aborts if-conversion in
noce_process_if_block here
  if (!bb_valid_for_noce_process_p (then_bb, cond, &then_cost,
&if_info->then_simple))
return false;
because insn_valid_noce_process_p returns false for the first insn above.

> It’s not a bug, but it is a performance issue (*1).

Well, it a performance bug then.

> I was just curious under which conditions the ternary operator is lowered to 
> cmov on x86 and found this difference in lowering.
>
> Michael
>
> [1] https://github.com/xiadz/cmov
>
>
> #include 
>
> extern int a;
> extern int b;
> extern int c;
> extern _Bool C;
>
> int select_int()
> {
> return c ? a : b;
> }
>
> _Bool select_bool()
> {
> return C ? a : b;
> }
>
> _Bool a_bool()
> {
> return 2;
> }
>
> select_int():
> mov eax, DWORD PTR c[rip]
> testeax, eax
> mov eax, DWORD PTR a[rip]
> cmove   eax, DWORD PTR b[rip]
> ret
> select_bool():
> cmp BYTE PTR C[rip], 0
> jne .L8
> mov eax, DWORD PTR b[rip]
> testeax, eax
> setne   al
> ret
> .L8:
> mov edx, DWORD PTR a[rip]
> testedx, edx
> setne   al
> ret
> a_bool():
> mov eax, 1
> ret

Re: Missed optimization with const member

[Yuri Gribov]

On Wed, Jul 5, 2017 at 12:14 PM, Jonathan Wakely  wrote:
> On 5 July 2017 at 10:13, Oleg Endo wrote:
>> Hi,
>>
>> On Wed, 2017-07-05 at 02:02 +0200, Geza Herman wrote:
>>>
>>> Here's what happens: in callInitA(), an Object put onto the stack (which
>>> has a const member variable, initialized to 0). Then somefunction called
>>> (which is intentionally not defined). Then ~Object() is called, which
>>> has an "if", which has a not-immediately-obvious, but always false
>>> condition. Compiling with -03, everything gets inlined.
>>>
>>> My question is about the inlined ~Object(). As m_initType is always 0,
>>> why does not optimize the destructor GCC away? GCC inserts code that
>>> checks the value of m_initType.
>>>
>>> Is it because such construct is rare in practice? Or is it hard to do an
>>> optimization like that?
>>
>> It's not safe to optimize it away because the compiler does not know
>> what "somefunction" does.  Theoretically, it could cast the "Object&"
>> to some subclass and overwrite the const variable.  "const" does not
>> mean that the memory location is read-only in some way.
>
> No, that would be undefined behaviour. The data member is defined as
> const, so it's not possible to write to that member without undefined
> behaviour. A variable defined with a const type is not the same as a
> variable accessed through a pointer/reference to a const type.
>
> Furthermore, casting the Object to a derived class would also be
> undefined, because the dynamic type is Object, not some derived type.

There's a related PR about this
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67886). basic.lifetime
allows to devirtualize second call to foo in
  Base *p = new Derived;
  p->foo();
  p->foo();
to
  Base *p = new Derived;
  Derived::foo(p);
  Derived::foo(p);
because first call is not allowed to changed dynamic type of p. Clang
optimizes this but GCC does not.

> I think the reason it's not optimized away is for this case:
>
> void somefunction(const Object& object);
> {
>   void* p = &object;
>   object.~Object();
>   new(p) Object();
> }
>
> This means that after calling someFunction there could be a different
> object at the same location (with a possibly different value for that
> member).
>
> However, for this specific case that's also not possible, because
> there are no constructors that could initialize the member to anything
> except zero. But that requires more than just control-flow analysis,
> it also requires analysing all the available constructors to check
> there isn't one that does:
>
> Object::Object(int) : m_initType(1) { }

Killing old dead bugs

[Yuri Gribov]

Hi all,

What do I need to do to close an old bug which does not repro with
modern GCC and reporter does not care anymore (e.g.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40528)? Also, is there
some general policy about closing old bugs?

-Y

Stale wiki info about CompileFarm registration

[Yuri Gribov]

Hi all,

It seems that info at
https://gcc.gnu.org/wiki/CompileFarm#How_to_Get_Involved.3F is
out-dated: Laurent's mail is not responsive and one's supposed to use
application form at https://cfarm.tetaneutral.net/users/new/ (which
provides all the necessary guidance).

Could someone update the wiki? (I could do it myself but don't have
write access)

-Yury

Re: Throwing exceptions from a .so linked with -static-lib* ?

[Yuri Gribov]

On Thu, Jan 12, 2017 at 5:06 AM, Paul Smith  wrote:
> TL;DR:
> I have an issue where if I have a .so linked with -static-lib* making
> all STL symbols private, and if I throw an exception out of that .so to
> be caught by the caller, then I get a SIGABRT from a gcc_assert() down
> in the guts of the signal handling:
>
> #0  0x7773a428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x7773c02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x0040e938 in _Unwind_SetGR (context=, 
> index=, val=) at 
> /usr/src/cc/gcc-6.2.0/libgcc/unwind-dw2.c:271
> 271   gcc_assert (index < (int) sizeof(dwarf_reg_size_table));
>
> Should it be possible to do this successfully or am I doomed to failure?
> More details and a test case below.
>
>
> More detail:
>
> I'm trying to distribute a shared library built with the latest version
> of C++ (well, GCC 6.2 with C++14) on GNU/Linux.  I compile it with an
> older sysroot, taken from RHEL 6.3 (glibc 2.12) so it will run on older
> systems.
>
> My .so is written in C++ and programs that link it will also be written
> in C++ although they may be compiled and linked with potentially much
> older versions of GCC (like, 4.9 etc.)  I'm not worried about programs
> compiled with clang or whatever at this point.
>
> Because I want to use new C++ but want users to be able to use my .so on
> older systems, I link with -static-libgcc -static-libstdc++.  Because I
> don't want to worry about security issues etc. in system libraries, I
> don't link anything else statically.
>
> I also use a linker script to force all symbols (even libstdc++ symbols)
> to be private to my shared library except the ones I want to publish.
>
> Using "nm | grep ' [A-TV-Z] '" I can see that no other symbols besides
> mine are public.
>
> However, if my library throws an exception which I expect to be handled
> by the program linking my library, then I get a SIGABRT, as above; the
> full backtrace is:
>
> #0  0x7773a428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x7773c02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x0040e938 in _Unwind_SetGR (context=, 
> index=, val=) at 
> /usr/src/cc/gcc-6.2.0/libgcc/unwind-dw2.c:271

That seems to be
  gcc_assert (index < (int) sizeof(dwarf_reg_size_table));
which _probably_ means that unwinder is using broken unwind table.

Note that documentation for -static-libgcc explicitly mentions that
   There are several situations in which an application should
use the shared libgcc instead of the static version.  The most
   common of these is when the application wishes to throw and
catch exceptions across different shared libraries.  In that case,
   each of the libraries as well as the application itself
should use the shared libgcc.
Removing -static-libgcc fixes problem with your reprocase.

My (most probably wrong) guess is that unwind tables for main
executable and library get registered in two different instances of
libgcc (one in libgcc_s.so loaded by myprog and one statically linked
into mylib.so) which prevents normal exception propagation.

Also this thead may be relevant:
https://gcc.gnu.org/ml/gcc-help/2009-12/msg00186.html

> #3  0x004012a2 in __gxx_personality_v0 ()
> #4  0x77feb903 in _Unwind_RaiseException_Phase2 
> (exc=exc@entry=0x43b890, context=context@entry=0x7fffe330) at 
> /usr/src/cc/gcc-6.2.0/libgcc/unwind.inc:62
> #5  0x77febf8a in _Unwind_RaiseException (exc=0x43b890) at 
> /usr/src/cc/gcc-6.2.0/libgcc/unwind.inc:131
> #6  0x77fde84b in __cxa_throw () from 
> /home/psmith/src/static-eh/libmylib.so
> #7  0x77fddecb in MyLib::create () at mylib.cpp:2
> #8  0x00400da4 in main () at myprog.cpp:2
>
> I should note that if I use the GCC 5.4 that comes standard on my OS
> rather than my locally-built version I get identical behavior and
> backtrace (except not as much debuggability of course).  So I don't
> think it's an incorrect build.
>
> If I don't use -static-libstdc++ with my .so then it doesn't fail.  Also
> if I don't use a linker script to hide all the C++ symbols it doesn't
> fail (but of course anyone who links with my shared library will use my
> copy of the STL).
>
>
> Here's a repro case (this shows the problem on my Ubuntu GNOME 16.04
> GNU/Linux system with GCC 5.4 and binutils 2.26.1):
>
> ~$ cat mylib.h
> class MyLib { public: static void create(); };
>
> ~$ cat mylib.cpp
> #include "mylib.h"
> void MyLib::create() { throw 42; }
>
> ~$ cat myprog.cpp
> #include "mylib.h"
> int main() { try { MyLib::create(); } catch (...) { return 0; } return 1; }
>
> ~$ cat ver.map
> { global: _ZN5MyLib6createEv; local: *; };
>
> ~$ g++ -I. -g -fPIC -static-libgcc -static-libstdc++ \
> -Wl,--version-script=ver.map -Wl,-soname=libmylib.so \
> -shared -o libmylib.so mylib.cpp
>
> ~$ g++ -I. -g -fPIC  -L. -Wl,-rpath="\$ORIGIN" -o myprog myprog.cpp \
> -lmylib
>
> ~$ ./myprog
> Aborted (

Re: [PING][RFC] Assertions as optimization hints

[Yuri Gribov]

On Mon, Nov 28, 2016 at 4:03 PM, Vincent Lefevre  wrote:
> On 2016-11-23 16:03:44 +0000, Yuri Gribov wrote:
>> Definitely, aggressively abusing assertions like this should be a
>> per-project decision. E.g. I've seen code which parallels assertions
>> with error checking i.e. something like
>>   FILE *p = fopen(...);
>>   assert(p);  // Quickly abort in debug mode
>>   if (!p)
>> return ERROR;
>> Enabling __ASSUME_ASSERTS__ for such code would effectively disable
>> all safety checks.
>
> Yes, the problem comes from the fact that ISO C provides only one
> form of assertions. In GNU MPFR, we have two kinds of assertions
> (different from the one used above), but they are not based on
> assert():
>
>MPFR_ASSERTN(expr): assertions that should normally be checked,
>  otherwise give a hint to the compiler.
>
>MPFR_ASSERTD(expr): assertions that should be checked when testing,
>  otherwise give a hint to the compiler.
>
> Basically, MPFR_ASSERTN() will be used to do some checks on data
> provided by the caller (e.g. to detect some API misuse), and
> MPFR_ASSERTD() is there to detect internal inconsistencies (i.e.
> bugs in MPFR).
>
> IMHO, this is the correct way to do, as one can then enables hints
> without breaking code like the above one.

Yes, many projects adopted similar approach. And it's indeed a pity
that standard does not distinguish between assume and assert so we
don't have universal mechanism to inform compiler about exploitable
code invariants.

> Concerning the "per-project decision", I'm not sure that's a good
> idea: as soon as one includes a header file, __ASSUME_ASSERTS__
> could potentially break code.

Sorry, not sure I understood. __ASSUME_ASSERTS__ is supplied via
cmdline when compiling project's shared libraries and executables to
treat assertions as hints (effectively similar to MS __assume
directives). It does not propagate to project's public header files.

> Said otherwise, if the user wants
> optimization hints, it should not be based on assert().

If project can invest time in refactoring their error checking code
and instructing compiler about internal invariants - that's certainly
the way to go. The only advantage of suggested approach is ease of use
(it can be deployed in couple of minutes).

> Vincent Lefèvre  - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: [PING][RFC] Assertions as optimization hints

[Yuri Gribov]

On Wed, Nov 23, 2016 at 11:31 AM, Richard Biener
 wrote:
> On Tue, Nov 22, 2016 at 6:52 PM, Yuri Gribov  wrote:
>> Hi all,
>>
>> I've recently revisited an ancient patch from Paolo
>> (https://gcc.gnu.org/ml/gcc-patches/2004-04/msg00551.html) which uses
>> asserts as optimization hints. I've rewritten the patch to be more
>> stable under expressions with side-effects and did some basic
>> investigation of it's efficacy.
>>
>> Optimization is hidden under !defined NDEBUG && defined
>> __ASSUME_ASSERTS__. !NDEBUG-part is necessary because assertions often
>> rely on special !NDEBUG-protected support code outside of assert
>> (dedicated fields in structures and similar stuff, collectively called
>> "ghost variables"). __ASSUME_ASSERTS__ gives user a choice whether to
>> enable optimization or not (should probably be hidden under a friendly
>> compiler switch e.g. -fassume-asserts).
>>
>> I do not have access to a good machine for speed benchmarks so I only
>> looked at size improvements in few popular projects. There are no
>> revolutionary changes (0.1%-1%) but some functions see good reductions
>> which may result in noticeable runtime improvements in practice. One
>> good  example is MariaDB where you frequently find the following
>> pattern:
>>   struct A {
>> virtual void foo() { assert(0); }
>>   };
>>   ...
>>   A *a;
>>   a->foo();
>> Here the patch will prevent GCC from inlining A::foo (as it'll figure
>> out that it's impossible to occur at runtime) thus saving code size.
>>
>> Does this approach make sense in general? If it does I can probably
>> come up with more measurements.
>>
>> As a side note, at least some users may consider this a useful feature:
>> http://www.nntp.perl.org/group/perl.perl5.porters/2013/11/msg209482.html
>
> You should CC relevant maintainers or annotate the subject -- this is
> a C/C++ frontend patch introducing __builtin_has_side_effects_p
> plus a patch adding a GCC supplied assert.h header.

Thanks, Richard, I've cc-ed Joseph (who is both frontend maintainer
and also commented on old version of this patch).

> Note that from a distribution point of view I wouldn't enable assume-asserts
> for a distro-build given the random behavior of __builtin_unreachable in case
> of assert failure.

Definitely, aggressively abusing assertions like this should be a
per-project decision. E.g. I've seen code which parallels assertions
with error checking i.e. something like
  FILE *p = fopen(...);
  assert(p);  // Quickly abort in debug mode
  if (!p)
return ERROR;
Enabling __ASSUME_ASSERTS__ for such code would effectively disable
all safety checks.

On the other hand many projects use assert(0) to mark unreachable
regions of code (e.g. in default branches of switch statements) and
optimizing these out sounds like a viable decision to me.

-Iurii

[PING][RFC] Assertions as optimization hints

[Yuri Gribov]

Hi all,

I've recently revisited an ancient patch from Paolo
(https://gcc.gnu.org/ml/gcc-patches/2004-04/msg00551.html) which uses
asserts as optimization hints. I've rewritten the patch to be more
stable under expressions with side-effects and did some basic
investigation of it's efficacy.

Optimization is hidden under !defined NDEBUG && defined
__ASSUME_ASSERTS__. !NDEBUG-part is necessary because assertions often
rely on special !NDEBUG-protected support code outside of assert
(dedicated fields in structures and similar stuff, collectively called
"ghost variables"). __ASSUME_ASSERTS__ gives user a choice whether to
enable optimization or not (should probably be hidden under a friendly
compiler switch e.g. -fassume-asserts).

I do not have access to a good machine for speed benchmarks so I only
looked at size improvements in few popular projects. There are no
revolutionary changes (0.1%-1%) but some functions see good reductions
which may result in noticeable runtime improvements in practice. One
good  example is MariaDB where you frequently find the following
pattern:
  struct A {
virtual void foo() { assert(0); }
  };
  ...
  A *a;
  a->foo();
Here the patch will prevent GCC from inlining A::foo (as it'll figure
out that it's impossible to occur at runtime) thus saving code size.

Does this approach make sense in general? If it does I can probably
come up with more measurements.

As a side note, at least some users may consider this a useful feature:
http://www.nntp.perl.org/group/perl.perl5.porters/2013/11/msg209482.html

-I


0001-Optionally-treat-assertions-as-optimization-hints-dr.patch
Description: Binary data

[RFC] Assertions as optimization hints

[Yuri Gribov]

Hi all,

I've recently revisited an ancient patch from Paolo
(https://gcc.gnu.org/ml/gcc-patches/2004-04/msg00551.html) which uses
asserts as optimization hints. I've rewritten the patch to be more
stable under expressions with side-effects and did some basic
investigation of it's efficacy.

Optimization is hidden under !defined NDEBUG && defined
__ASSUME_ASSERTS__. !NDEBUG-part is necessary because assertions often
rely on special !NDEBUG-protected support code outside of assert
(dedicated fields in structures and similar stuff, collectively called
"ghost variables"). __ASSUME_ASSERTS__ gives user a choice whether to
enable optimization or not (should probably be hidden under a friendly
compiler switch e.g. -fassume-asserts).

I do not have access to a good machine for speed benchmarks so I only
looked at size improvements in few popular projects. There are no
revolutionary changes (0.1%-1%) but some functions see good reductions
which may result in noticeable runtime improvements in practice. One
good  example is MariaDB where you frequently find the following
pattern:
  struct A {
virtual void foo() { assert(0); }
  };
  ...
  A *a;
  a->foo();
Here the patch will prevent GCC from inlining A::foo (as it'll figure
out that it's impossible to occur at runtime) thus saving code size.

Does this approach make sense in general? If it does I can probably
come up with more measurements.

As a side note, at least some users may consider this a useful feature:
http://www.nntp.perl.org/group/perl.perl5.porters/2013/11/msg209482.html

-I


0001-Optionally-treat-assertions-as-optimization-hints-dr.patch
Description: Binary data

Re: [RFC][Draft patch] Introduce IntegerSanitizer in GCC.

[Yuri Gribov]

Cc John.

On Tue, Jul 12, 2016 at 10:49 AM, Maxim Ostapenko
 wrote:
> On 12/07/16 12:20, Yuri Gribov wrote:
>>
>> On Tue, Jul 12, 2016 at 9:48 AM, Maxim Ostapenko
>>  wrote:
>>>
>>> On 11/07/16 19:28, Jeff Law wrote:
>>>>
>>>> On 07/11/2016 10:08 AM, Maxim Ostapenko wrote:
>>>>>
>>>>> On 11/07/16 18:05, Jakub Jelinek wrote:
>>>>>>
>>>>>> On Tue, Jul 05, 2016 at 10:31:31AM +0300, Maxim Ostapenko wrote:
>>>>>>>
>>>>>>> CC'ing Jakub, Marek and Kostya, sanitizer maintainers in GCC.
>>>>>
>>>>>
>>>>> Jakub, thanks for your summary.
>>>>>
>>>>>> I'm not convinced it is a good idea, that is why we've intentionally
>>>>>> left it
>>>>>> out when adding UBSan support, IMHO such an option defines
>>>>>> substantially
>>>>>> different languages.
>>>>>
>>>>>
>>>>> The reason why I thought about -fsanitize=unsigned-integer-overflow
>>>>> would be useful is that people still hit on undesired integer overflows
>>>>> in their code (that may even lead to security vulnerabilities), despite
>>>>> the fact some people intentionally rely on them.
>>>>
>>>> An integer overflow where the result feeds a malloc/alloca is definitely
>>>> a
>>>> security issue.There may be others.
>>>>
>>>> So one of the questions one might reasonably try to answer is can we
>>>> limit
>>>> sanitization to those cases that are most likely going to be of interest
>>>> to
>>>> developers.
>>>
>>>
>>> Thank you for your point. I think the easiest case here is pointer
>>> overflow
>>> (since we have a dedicated place where we construct POINTER_PLUS
>>> expression).
>>> As for other cases, generally it's not easy to understand whether given
>>> binary expression is interesting or not. Of course, in some more or less
>>> trivial cases, such as
>>> void foo (unsigned a, unsigned b)
>>> {
>>>  unsigned len = a + b;
>>>  void *p = malloc (len);
>>> }
>>>
>>> we can make such decision, but in others, such as
>>>
>>> void foo (unsigned len)
>>> {
>>>  ...
>>>  void *p = malloc (len);
>>> }
>>>
>>> void bar ()
>>> {
>>>  ...
>>>  unsigned len = a + b;
>>>  foo (len);
>>> }
>>>
>>> we can't (and this is quite common code I believe). So, we can end up
>>> with
>>> missing some important cases.
>>>
>>>> I suspect that in general integer overflow happens far often than
>>>> developers realize and that there'll be so many false positives that the
>>>> results will be ignored.
>>>
>>>
>>> Yeah, even in small projects I observe a bunch of places where integer
>>> overflow happens. And yes, FPs are the real problem here...
>>
>> There are people who would tolerate FPs if the tool indeed helps to
>> find vulnerabilities. Especially if there is easy way to suppress
>> checks in set of functions/files who intentionally rely on unsigned
>> overflow (hash functions, etc.).
>>
>> What are the FP rates you see with current version of the patch?
>
>
> A lot... this depends of particular project. Say, GCC has so much code that
> essentially relies on unsigned integer overflows, that I can't even tell you
> a ratio.
> For, Firefox I saw ~90% of FP ratio (given the fact I use several hacks in
> GCC to reduce this ratio), they are mostly come from hashing code and MULT
> expressions.
>
>>
>> -Y
>>
>>
>

Re: [RFC][Draft patch] Introduce IntegerSanitizer in GCC.

[Yuri Gribov]

On Tue, Jul 12, 2016 at 10:34 AM, Jakub Jelinek  wrote:
> On Tue, Jul 12, 2016 at 10:20:55AM +0100, Yuri Gribov wrote:
>> There are people who would tolerate FPs if the tool indeed helps to
>> find vulnerabilities. Especially if there is easy way to suppress
>> checks in set of functions/files who intentionally rely on unsigned
>
> But what is the easy way to suppress it?
> For say unsigned int x, y, z;
> ...
> x = y + z;
> one can surely suppress it with
> (void) __builtin_add_overflow (y, z, &x);
> or as Segher mentioned on IRC, portably with:
> x = ((y&((~0U/2))+(z&((~0U/2)))^((y^z)&~((~0U/2));
> If there is a wider type, one can also compute in the wider type and
> then mask.
> Still, none of this look like easy way.

I was actually talking about brute-force
__attribute__((no_sanitize_unsigned)). Clang goes even further by
storing these annotations to dedicated config files (presumably to
allow easier integration to large codebases).

-Y

Re: [RFC][Draft patch] Introduce IntegerSanitizer in GCC.

[Yuri Gribov]

On Tue, Jul 12, 2016 at 9:48 AM, Maxim Ostapenko
 wrote:
> On 11/07/16 19:28, Jeff Law wrote:
>>
>> On 07/11/2016 10:08 AM, Maxim Ostapenko wrote:
>>>
>>> On 11/07/16 18:05, Jakub Jelinek wrote:

 On Tue, Jul 05, 2016 at 10:31:31AM +0300, Maxim Ostapenko wrote:
>
> CC'ing Jakub, Marek and Kostya, sanitizer maintainers in GCC.
>>>
>>>
>>> Jakub, thanks for your summary.
>>>
 I'm not convinced it is a good idea, that is why we've intentionally
 left it
 out when adding UBSan support, IMHO such an option defines substantially
 different languages.
>>>
>>>
>>> The reason why I thought about -fsanitize=unsigned-integer-overflow
>>> would be useful is that people still hit on undesired integer overflows
>>> in their code (that may even lead to security vulnerabilities), despite
>>> the fact some people intentionally rely on them.
>>
>> An integer overflow where the result feeds a malloc/alloca is definitely a
>> security issue.There may be others.
>>
>> So one of the questions one might reasonably try to answer is can we limit
>> sanitization to those cases that are most likely going to be of interest to
>> developers.
>
>
> Thank you for your point. I think the easiest case here is pointer overflow
> (since we have a dedicated place where we construct POINTER_PLUS
> expression).
> As for other cases, generally it's not easy to understand whether given
> binary expression is interesting or not. Of course, in some more or less
> trivial cases, such as
> void foo (unsigned a, unsigned b)
> {
> unsigned len = a + b;
> void *p = malloc (len);
> }
>
> we can make such decision, but in others, such as
>
> void foo (unsigned len)
> {
> ...
> void *p = malloc (len);
> }
>
> void bar ()
> {
> ...
> unsigned len = a + b;
> foo (len);
> }
>
> we can't (and this is quite common code I believe). So, we can end up with
> missing some important cases.
>
>> I suspect that in general integer overflow happens far often than
>> developers realize and that there'll be so many false positives that the
>> results will be ignored.
>
>
> Yeah, even in small projects I observe a bunch of places where integer
> overflow happens. And yes, FPs are the real problem here...

There are people who would tolerate FPs if the tool indeed helps to
find vulnerabilities. Especially if there is easy way to suppress
checks in set of functions/files who intentionally rely on unsigned
overflow (hash functions, etc.).

What are the FP rates you see with current version of the patch?

-Y

Re: How to identify the version of the LLVM AddressSanitizer integrated to GCC 4.9.3 and after

[Yuri Gribov]

On Fri, Apr 1, 2016 at 1:14 PM, Martin Liška  wrote:
> On 03/31/2016 05:48 PM, Maxim Ostapenko wrote:
>>
>> Yes, but please note, that this page describes differences between two 
>> particular revisions. For current trunk (and release) GCC and LLVM versions 
>> the situation might be different.
>>
>>>
>>> Finally any plans to integrate other sanitizer tools by LLVM in to
>>> GCC, like Memory Sanitizer, Data Flow Sanitizer ?
>>
>> AFAIK, there aren't any plans on porting MSan and DFSan to GCC (see 
>> https://gcc.gnu.org/ml/gcc/2014-10/msg0.html for MSan). TSan and UBSan 
>> are already present in GCC.
>>
>> -Maxim
>
> Hi.
>
> I was thinking about integration of MSAN to GCC (as I was hunting for an 
> issue in Firefox),
> but as the sanitizer really needs to have instrumented all shared libraries 
> that a program
> uses, I gave up. After a brief discussion with Jakub, he had the same opinion.

FYI in our experience instrumenting a complete distribution is
relatively easy (albeit boring). It took us ~2-3 months to get fully
AddressSanitized Tizen last year
(http://injoit.org/index.php/j1/article/viewFile/231/184) and Hanno
Boeck did the same to Gentoo few months ago
(https://blog.hboeck.de/archives/879-Safer-use-of-C-code-running-Gentoo-with-Address-Sanitizer.html).

So if you have a working implementation for MSan, why not throw it out
so that other people could play with it? I guess it's a big and
non-trivial piece of code.

> However, I've been working on use-after-scope sanitizer ([1]), which would 
> hopefully
> land in GCC 7, where I hope we can get even better results as the GCC has good
> scope information of local variables.

That's a cool feature indeed (something that never worked in Clang btw).

> Martin
>
> [1] https://github.com/marxin/gcc/tree/asan-use-after-scope-2
>

Re: Asm volatile causing performance regressions on ARM

[Yuri Gribov]

> asm volatile + memory clobber should be the last resort barrier, if you skip
> this out of the compiler or change its semantics (pinned by the current
> documentation) at will, it's not unlikely you break existing code in favour
> or saving some poor instructions.

Problem is that there is no current semantics. As Richard pointed out,
RTL CSE just happens (probably due to historical reasons) to stop at
volatile asm but this is not documented anywhere and people are
certainly not (and probably never were) recommended to rely on this
behavior. And other GCC optimizations may not behave similar way.

> For example, I had the case that a costly computation (division an a
> hardware that cannot divide) was moved into a section enclosed in asms which
> disabled / re-enabled interrupts.  This totally wrecked interrupt respond
> times on the machine.

Looks like a good example of volatile asm not being an optimization barrier.

> Notice that such a division had no side effects from the C side or from the
> compiler's point of view, but execution time and interrupt respond times
> cannot be ignored by any software that does system programming.

I agree that there's probably a need for an construct that would
prevent code motion. But I'm not sure whether current volatile asm is
intended (not even capable) for this.

-Y

Re: Non-portable test?

[Yuri Gribov]

> Done.  But if you have more cases, please report them.
Not yet. Thx!

-- 
Best regards,
Yuri

Re: Non-portable test?

[Yuri Gribov]

> Yes, it's possible that 64-bit ints are not supported by the testsuite.
>  Changes to fix that are welcome.

I am not a gcc developer. Could someone verify and commit this patch
for testsuite/gcc.c-torture/execute/980526-2.c?

Best regards,
Yuri


980526-2.patch
Description: Binary data

Non-portable test?

[Yuri Gribov]

Hi all,

This is my first post to the list so do not be too harsh)

I have expected all c-torture tests to be highly portable but I have
recently ran into test which relies on int being 32-bit
(execute/980526-2.c).

The test runs to_kdev_t(0x12345678) (see below) and verifies that
result equals 0x15800078. But this is true only with 32-bit ints. With
64-bits we have 0x48d15800078.

static inline kdev_t to_kdev_t(int dev)
{
   int major, minor;

   if (sizeof(kdev_t) == 16)
   return (kdev_t)dev;
   major = (dev >> 8);
   minor = (dev & 0xff);
   return ((( major ) << 22 ) | (  minor )) ;
}

Shouldn't we modify a precondition in main:
   if (sizeof (int) < 4)
 exit (0);
to be
   if (sizeof (int) != 4)
 exit (0);
or better
   if( sizeof(int)*CHAR_BIT != 32 )
 exit(0)
?

Best regards,
Yuri