[Bug sanitizer/88289] [9 regression] r266664 causes asan to fail on many/most tests on BE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88289

--- Comment #1 from Jakub Jelinek  ---
E.g. on x86_64 in user-after-scope-5.c I see:
.string "1 48 40 9 values:10"
Partition 0: size 40 align 16
values
Flushing rzbuffer at offset -128 with: f1 f1 f1 f1
Flushing rzbuffer at offset -96 with: f1 f1 00 00
Flushing rzbuffer at offset -64 with: 00 00 00 f3
Flushing rzbuffer at offset -32 with: f3 f3 f3 f3
(wonder why the var is at offset 48 bytes rather than just 32 bytes).\

But on powerpc64 I see:
.string "1 48 40 9 values:10"
Partition 0: size 40 align 8
values
Flushing rzbuffer at offset -128 with: 00 f1 f1 f1
Flushing rzbuffer at offset -96 with: 00 00 00 f1
Flushing rzbuffer at offset -64 with: 00 f3 00 00
Flushing rzbuffer at offset -32 with: 00 f3 f3 f3

There is a clear off-by-one for BIG_ENDIAN, with following patch I get:

--- gcc/asan.c.jj   2018-11-30 19:59:59.675789930 +0100
+++ gcc/asan.c  2018-11-30 23:19:55.336780260 +0100
@@ -1326,7 +1326,7 @@ asan_redzone_buffer::flush_redzone_paylo
   for (unsigned i = 0; i < RZ_BUFFER_SIZE; i++)
 {
   unsigned char v
-   = m_shadow_bytes[BYTES_BIG_ENDIAN ? RZ_BUFFER_SIZE - i : i];
+   = m_shadow_bytes[BYTES_BIG_ENDIAN ? RZ_BUFFER_SIZE - i - 1 : i];
   val |= (unsigned HOST_WIDE_INT)v << (BITS_PER_UNIT * i);
   if (dump_file && (dump_flags & TDF_DETAILS))
fprintf (dump_file, "%02x ", v);

Partition 0: size 40 align 8
values
Flushing rzbuffer at offset -128 with: f1 f1 f1 f1
Flushing rzbuffer at offset -96 with: 00 00 f1 f1
Flushing rzbuffer at offset -64 with: f3 00 00 00
Flushing rzbuffer at offset -32 with: f3 f3 f3 f3
Unpoisoning shadow stack for variable: values (40 B)

instead, which looks correct.

[Bug sanitizer/88289] [9 regression] r266664 causes asan to fail on many/most tests on BE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88289

Jakub Jelinek  changed:

   What|Removed |Added

 Status|UNCONFIRMED |ASSIGNED
   Last reconfirmed||2018-11-30
   Assignee|unassigned at gcc dot gnu.org  |jakub at gcc dot gnu.org
   Target Milestone|--- |9.0
 Ever confirmed|0   |1

[Bug sanitizer/88289] [9 regression] r266664 causes asan to fail on many/most tests on BE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88289

--- Comment #2 from Jakub Jelinek  ---
Author: jakub
Date: Sat Dec  1 07:31:56 2018
New Revision: 266708

URL: https://gcc.gnu.org/viewcvs?rev=266708&root=gcc&view=rev
Log:
PR sanitizer/88289
* asan.c (asan_redzone_buffer::flush_redzone_payload): Fix up
an off-by-one for BYTES_BIG_ENDIAN.

Modified:
trunk/gcc/ChangeLog
trunk/gcc/asan.c

[Bug sanitizer/88289] [9 regression] r266664 causes asan to fail on many/most tests on BE

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88289

Jakub Jelinek  changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution|--- |FIXED

--- Comment #3 from Jakub Jelinek  ---
I've filed PR88291 for the asan_clear_shadow ICE.