Re: libsanitizer merge from upstream r175042

2013-02-28 Thread Konstantin Serebryany 
On Fri, Feb 22, 2013 at 8:32 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 12:47:30PM +0400, Konstantin Serebryany wrote:
 Sure. ASAN_FIXED_MAPPING should be used for performance measurements
 only -- this is not a release option.
 (Added a more precise comment).

 BTW, today I think I've discovered what looks like a prelink bug,
 but perhaps we need to bump kMidMemEnd a little bit.
 With prelink -R, the libraries are randomized in the selected range of 
 addresses,
 by picking a random number in between the boundaries of the address range and 
 first
 assigning addresses to libraries above that random offset and when there is 
 no longer any room
 above it, starting from the beginning of the range.  Unfortunately the code 
 seems to have
 some issues if the random address is chosen close to the end of the range, 
 then in some
 cases some libraries could start before the range, but end after the range.

 On one of my boxes there is (only 4 libraries out of 822 have this problem,
 only discovered it because gdb is linked against one of them and I've tried to
 LD_PRELOAD=libasan.so.0 to gdb so that the debugged program would inherit 
 that):

 prelink -pv 21 | grep 0x003.*-.*0x004
 /usr/lib64/libgmlib.so.1.0.7 [0x1ea5b3cf] 
 0x003fffe0-0x00408460:
 /usr/lib64/libiec61883.so.0.1.1 [0x56363ffc] 
 0x003fffe0-0x0040c3e0:
 /usr/lib64/libncurses.so.5.9 [0x120e1b8a] 
 0x003fffe0-0x00423860:
 /usr/lib64/libsoup-2.4.so.1.5.0 [0x99ff4d51] 
 0x003fffe0-0x0046a258:

 while on others none.

 So, can kMidMemEnd be slightly incremented above this?  Either 
 0x4fULL,
 or 0x40ULL (or replace that 0f with 1f, 2f, etc.).
 Small model shared libraries can be only up to 2GB in size, so even
 0x40ULL should be big enough for most cases, but 0x4fULL
 could be even safer.  I've justed tested and 0x4fULL results in
 || `[0x10007fff8000, 0x7fff]` || HighMem||
 || `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
 || `[0x0050, 0x02008fff6fff]` || ShadowGap3 ||
 || `[0x0030, 0x004f]` || MidMem ||
 || `[0x000a7fff8000, 0x002f]` || ShadowGap2 ||
 || `[0x00067fff8000, 0x000a7fff7fff]` || MidShadow  ||
 || `[0x8fff7000, 0x00067fff7fff]` || ShadowGap  ||
 || `[0x7fff8000, 0x8fff6fff]` || LowShadow  ||
 || `[0x, 0x7fff7fff]` || LowMem ||
 MemToShadow(shadow): 0x8fff7000 0x91ff6dff 0x004091ff6e00 
 0x02008fff6fff 0x00014fff7000 0x0001cfff6fff
 and seems to work just fine.

I am sorry, I missed this message.
Indeed, the change looks safe,
http://llvm.org/viewvc/llvm-project?rev=176250view=rev

Thanks!

--kcc


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-28 Thread Jakub Jelinek 
On Thu, Feb 28, 2013 at 04:30:13PM +0400, Konstantin Serebryany wrote:
 I am sorry, I missed this message.
 Indeed, the change looks safe,
 http://llvm.org/viewvc/llvm-project?rev=176250view=rev

Thanks, here is what I've committed to gcc:

2013-02-28  Jakub Jelinek  ja...@redhat.com

* asan/asan_mapping.h (kMidMemEnd): Increase to 0x4fULL.
* asan/asan_rtl.cc (__asan_init): Increase kMidMemEnd to
0x4fULL.

--- libsanitizer/asan/asan_mapping.h(revision 176249)
+++ libsanitizer/asan/asan_mapping.h(revision 176250)
@@ -32,13 +32,13 @@
 // || `[0x0004, 0x01ff]` || ShadowGap  ||
 //
 // Special case when something is already mapped between
-// 0x0030 and 0x0040 (e.g. when prelink is installed):
+// 0x0030 and 0x0050 (e.g. when prelink is installed):
 // || `[0x10007fff8000, 0x7fff]` || HighMem||
 // || `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
-// || `[0x0040, 0x02008fff6fff]` || ShadowGap3 ||
-// || `[0x0030, 0x003f]` || MidMem ||
-// || `[0x00087fff8000, 0x002f]` || ShadowGap2 ||
-// || `[0x00067fff8000, 0x00087fff7fff]` || MidShadow  ||
+// || `[0x0050, 0x02008fff6fff]` || ShadowGap3 ||
+// || `[0x0030, 0x004f]` || MidMem ||
+// || `[0x000a7fff8000, 0x002f]` || ShadowGap2 ||
+// || `[0x00067fff8000, 0x000a7fff7fff]` || MidShadow  ||
 // || `[0x8fff7000, 0x00067fff7fff]` || ShadowGap  ||
 // || `[0x7fff8000, 0x8fff6fff]` || LowShadow  ||
 // || `[0x, 0x7fff7fff]` || LowMem ||
@@ -131,7 +131,7 @@ extern uptr AsanMappingProfile[];
 // difference between fixed and non-fixed mapping is below the noise level.
 static uptr kHighMemEnd = 0x7fffULL;
 static uptr kMidMemBeg =0x30ULL;
-static uptr kMidMemEnd =0x3fULL;
+static uptr kMidMemEnd =0x4fULL;
 #else
 SANITIZER_INTERFACE_ATTRIBUTE
 extern uptr kHighMemEnd, kMidMemBeg, kMidMemEnd;  // Initialized in 
__asan_init.
--- libsanitizer/asan/asan_rtl.cc   (revision 176249)
+++ libsanitizer/asan/asan_rtl.cc   (revision 176250)
@@ -459,7 +459,7 @@ void __asan_init() {
 #if ASAN_LINUX  defined(__x86_64__)  !ASAN_FIXED_MAPPING
   if (!full_shadow_is_available) {
 kMidMemBeg = kLowMemEnd  0x30ULL ? 0x30ULL : 0;
-kMidMemEnd = kLowMemEnd  0x30ULL ? 0x3fULL : 0;
+kMidMemEnd = kLowMemEnd  0x30ULL ? 0x4fULL : 0;
   }
 #endif
 


Jakub

Re: libsanitizer merge from upstream r175042

2013-02-22 Thread Jakub Jelinek 
On Fri, Feb 15, 2013 at 12:47:30PM +0400, Konstantin Serebryany wrote:
 Sure. ASAN_FIXED_MAPPING should be used for performance measurements
 only -- this is not a release option.
 (Added a more precise comment).

BTW, today I think I've discovered what looks like a prelink bug,
but perhaps we need to bump kMidMemEnd a little bit.
With prelink -R, the libraries are randomized in the selected range of 
addresses,
by picking a random number in between the boundaries of the address range and 
first
assigning addresses to libraries above that random offset and when there is no 
longer any room
above it, starting from the beginning of the range.  Unfortunately the code 
seems to have
some issues if the random address is chosen close to the end of the range, then 
in some
cases some libraries could start before the range, but end after the range.

On one of my boxes there is (only 4 libraries out of 822 have this problem,
only discovered it because gdb is linked against one of them and I've tried to
LD_PRELOAD=libasan.so.0 to gdb so that the debugged program would inherit that):

prelink -pv 21 | grep 0x003.*-.*0x004
/usr/lib64/libgmlib.so.1.0.7 [0x1ea5b3cf] 0x003fffe0-0x00408460:
/usr/lib64/libiec61883.so.0.1.1 [0x56363ffc] 
0x003fffe0-0x0040c3e0:
/usr/lib64/libncurses.so.5.9 [0x120e1b8a] 0x003fffe0-0x00423860:
/usr/lib64/libsoup-2.4.so.1.5.0 [0x99ff4d51] 
0x003fffe0-0x0046a258:

while on others none.

So, can kMidMemEnd be slightly incremented above this?  Either 0x4fULL,
or 0x40ULL (or replace that 0f with 1f, 2f, etc.).
Small model shared libraries can be only up to 2GB in size, so even
0x40ULL should be big enough for most cases, but 0x4fULL
could be even safer.  I've justed tested and 0x4fULL results in
|| `[0x10007fff8000, 0x7fff]` || HighMem||
|| `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
|| `[0x0050, 0x02008fff6fff]` || ShadowGap3 ||
|| `[0x0030, 0x004f]` || MidMem ||
|| `[0x000a7fff8000, 0x002f]` || ShadowGap2 ||
|| `[0x00067fff8000, 0x000a7fff7fff]` || MidShadow  ||
|| `[0x8fff7000, 0x00067fff7fff]` || ShadowGap  ||
|| `[0x7fff8000, 0x8fff6fff]` || LowShadow  ||
|| `[0x, 0x7fff7fff]` || LowMem ||
MemToShadow(shadow): 0x8fff7000 0x91ff6dff 0x004091ff6e00 
0x02008fff6fff 0x00014fff7000 0x0001cfff6fff
and seems to work just fine.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-18 Thread Jakub Jelinek 
On Fri, Feb 15, 2013 at 07:39:28AM -0800, Ian Lance Taylor wrote:
 On Thu, Feb 14, 2013 at 11:45 PM, Konstantin Serebryany
 konstantin.s.serebry...@gmail.com wrote:
 
  Unfortunately, the test does not work if gold is the system linker.
  Any suggestion on how to make the test work with either linker?
 
 I don't know of a way to set the address of the text segment for both
 GNU ld and gold.  As you have observed, GNU ld's -Ttext-segment option
 is the same as gold's -Ttext option.  GNU ld's -Ttext option is
 useless on ELF systems.
 
 I will add -Ttext-segment to gold as an alias for -Ttext, but that
 won't help you today.

Can't you use then something like:
// RUN: %clangxx_asan -m64 -DBUILD_SO=1 -fPIC -shared %s -o %t.so 
-Wl,-Ttext-segment=0x36 || %clangxx_asan -m64 -DBUILD_SO=1 -fPIC 
-shared %s -o %t.so -Wl,-Ttext=0x36 || exit 0

?

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-18 Thread Konstantin Serebryany 
On Mon, Feb 18, 2013 at 12:20 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 07:39:28AM -0800, Ian Lance Taylor wrote:
 On Thu, Feb 14, 2013 at 11:45 PM, Konstantin Serebryany
 konstantin.s.serebry...@gmail.com wrote:
 
  Unfortunately, the test does not work if gold is the system linker.
  Any suggestion on how to make the test work with either linker?

 I don't know of a way to set the address of the text segment for both
 GNU ld and gold.  As you have observed, GNU ld's -Ttext-segment option
 is the same as gold's -Ttext option.  GNU ld's -Ttext option is
 useless on ELF systems.

 I will add -Ttext-segment to gold as an alias for -Ttext, but that
 won't help you today.

 Can't you use then something like:
 // RUN: %clangxx_asan -m64 -DBUILD_SO=1 -fPIC -shared %s -o %t.so 
 -Wl,-Ttext-segment=0x36 || %clangxx_asan -m64 -DBUILD_SO=1 -fPIC 
 -shared %s -o %t.so -Wl,-Ttext=0x36 || exit 0

Yep, that's what I was going to do once Ian confirmed that -Ttext in
gold is equivalent to Ttext-segment bfd ld.
http://llvm.org/viewvc/llvm-project?rev=175431view=rev

Thanks!

--kcc



 ?

 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Jakub Jelinek 
On Fri, Feb 15, 2013 at 11:45:15AM +0400, Konstantin Serebryany wrote:
 On Thu, Feb 14, 2013 at 4:19 PM, Jakub Jelinek ja...@redhat.com wrote:
  On Thu, Feb 14, 2013 at 03:55:47PM +0400, Konstantin Serebryany wrote:
  The patch seems to work on a simple test. Let me digest it.
  I am trying to understand if there are problems with it other than the
  added complexity (which is what I don't like the most).
 
  Yes, it is some added complexity, but not too much, and something that can
  be tested regularly that it works.
 
 The complexity I am afraid of is not only in the code, but also at the
 time of execution.
 We and our users sometimes have to stare at the /proc/self/maps.
 A mapping with 1 (ZeroBase) or 3 (default) asan sections is ok, but
 6 extra asan sections becomes nearly incomprehensible, at least for me.
 
 So, how about having kMidMemBeg as a variable, set as __asan_init.
 Only if something is mapped around 0x003X we set it to non-zero.

That is fine for me.  Note that ASAN_FIXED_MAPPING 1 might not work well,
e.g. for shadow offset of 1ULL  44, there the prelink library range falls
into the low memory and thus kMidMemBeg should be 0.

 http://llvm-reviews.chandlerc.com/D411 (still needs some cleanup)

One cleanup might be avoid calling MemoryRangeIsAvailable unnecessarily
too many times.  Guess if you move:
  uptr shadow_start = kLowShadowBeg;
  if (kLowShadowBeg) shadow_start -= GetMmapGranularity();
  uptr shadow_end = kHighShadowEnd;
  bool shadow_avail = MemoryRangeIsAvailable(shadow_start, shadow_end);
before the
#if ASAN_LINUX  defined(__x86_64__)  !ASAN_FIXED_MAPPING
  if (!MemoryRangeIsAvailable(kLowShadowBeg, kHighShadowEnd)) {
kMidMemBeg = kLowMemEnd  0x30ULL ? 0x30ULL : 0;
kMidMemEnd = kLowMemEnd  0x30ULL ? 0x3fULL : 0;
  }
#endif
code, you can just use shadow_avail there and later.  Every
MemoryRangeIsAvailable reads /proc/self/maps again, right?

Also, on ppc64 the prelink library area is:
0x800100LL to 0x81LL if we want to e.g. handle flexible mapping
there (perhaps better use 0x80L as kMidMemBeg then), guess for 32-bit
architectures it is irrelevant, there is not much
point in using shadow offsets other than the default high one (which is high
enough) or 0 (then you need PIE, and thus prelink info is ignored both by
the kernel and dynamic linker).

 Unfortunately, the test does not work if gold is the system linker.
 Any suggestion on how to make the test work with either linker?

That is the question to Ian, if gold can do that at all.

As I said before, you can try something like:

#include sys/mman.h

struct A
{
  A ();
  void *ptr;
};

void *ptr;

__attribute__((no_address_safety_analysis))
A::A ()
{
  ptr = mmap ((void *) 0x36UL, 65536, PROT_READ|PROT_WRITE,
  MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
}

A __attribute__((init_priority (1))) a;

int
main ()
{
  if (a.ptr != MAP_FAILED)
{
  char *ptr = (char *) a.ptr;
  __asan_poison_memory_region (ptr, 4096);
  __asan_poison_memory_region (ptr + 61440, 4096);
  ptr[4096] = 23;
}
}

and similar (and/or test that accessing the poisoned memory fails etc.).
For gcc you want to compile with -w, so that it doesn't warn about the
reserved init_priority, not sure what clang would do.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Konstantin Serebryany 
Ian, there is a question for you below.

On Fri, Feb 15, 2013 at 12:26 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 11:45:15AM +0400, Konstantin Serebryany wrote:
 On Thu, Feb 14, 2013 at 4:19 PM, Jakub Jelinek ja...@redhat.com wrote:
  On Thu, Feb 14, 2013 at 03:55:47PM +0400, Konstantin Serebryany wrote:
  The patch seems to work on a simple test. Let me digest it.
  I am trying to understand if there are problems with it other than the
  added complexity (which is what I don't like the most).
 
  Yes, it is some added complexity, but not too much, and something that can
  be tested regularly that it works.

 The complexity I am afraid of is not only in the code, but also at the
 time of execution.
 We and our users sometimes have to stare at the /proc/self/maps.
 A mapping with 1 (ZeroBase) or 3 (default) asan sections is ok, but
 6 extra asan sections becomes nearly incomprehensible, at least for me.

 So, how about having kMidMemBeg as a variable, set as __asan_init.
 Only if something is mapped around 0x003X we set it to non-zero.

 That is fine for me.  Note that ASAN_FIXED_MAPPING 1 might not work well,
 e.g. for shadow offset of 1ULL  44, there the prelink library range falls
 into the low memory and thus kMidMemBeg should be 0.

Sure. ASAN_FIXED_MAPPING should be used for performance measurements
only -- this is not a release option.
(Added a more precise comment).


 http://llvm-reviews.chandlerc.com/D411 (still needs some cleanup)

 One cleanup might be avoid calling MemoryRangeIsAvailable unnecessarily
 too many times.  Guess if you move:
   uptr shadow_start = kLowShadowBeg;
   if (kLowShadowBeg) shadow_start -= GetMmapGranularity();
   uptr shadow_end = kHighShadowEnd;
   bool shadow_avail = MemoryRangeIsAvailable(shadow_start, shadow_end);
 before the
 #if ASAN_LINUX  defined(__x86_64__)  !ASAN_FIXED_MAPPING
   if (!MemoryRangeIsAvailable(kLowShadowBeg, kHighShadowEnd)) {
 kMidMemBeg = kLowMemEnd  0x30ULL ? 0x30ULL : 0;
 kMidMemEnd = kLowMemEnd  0x30ULL ? 0x3fULL : 0;
   }
 #endif
 code, you can just use shadow_avail there and later.  Every
 MemoryRangeIsAvailable reads /proc/self/maps again, right?

agree, done.


 Also, on ppc64 the prelink library area is:
 0x800100LL to 0x81LL if we want to e.g. handle flexible mapping
 there (perhaps better use 0x80L as kMidMemBeg then), guess for 32-bit
 architectures it is irrelevant, there is not much
 point in using shadow offsets other than the default high one (which is high
 enough) or 0 (then you need PIE, and thus prelink info is ignored both by
 the kernel and dynamic linker).

Let's worry about ppc prelink separately.
We are not changing the shadow offset on ppc (because 7fff8000 does
not seem to help it anyway)
and no one complained so far.


 Unfortunately, the test does not work if gold is the system linker.
 Any suggestion on how to make the test work with either linker?

 That is the question to Ian, if gold can do that at all.

Yep.


 As I said before, you can try something like:

 #include sys/mman.h

 struct A
 {
   A ();
   void *ptr;
 };

 void *ptr;

 __attribute__((no_address_safety_analysis))
 A::A ()
 {
   ptr = mmap ((void *) 0x36UL, 65536, PROT_READ|PROT_WRITE,
   MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
 }

 A __attribute__((init_priority (1))) a;

 int
 main ()
 {
   if (a.ptr != MAP_FAILED)
 {
   char *ptr = (char *) a.ptr;
   __asan_poison_memory_region (ptr, 4096);
   __asan_poison_memory_region (ptr + 61440, 4096);
   ptr[4096] = 23;
 }
 }

 and similar (and/or test that accessing the poisoned memory fails etc.).
 For gcc you want to compile with -w, so that it doesn't warn about the
 reserved init_priority, not sure what clang would do.

This is ungood.
First, clang doesn't like it at all:
prelink1.cc:18:18: error: init_priority attribute requires integer
constant between 101 and 65535 inclusive
A __attribute__((init_priority (1))) a;

Second, in some settings we are using ASAN_USE_PREINIT_ARRAY=1
and this may become the default on linux at some point.
so, __asan_init will get called before A::A()

Waiting for Ian's reply about gold.

--kcc


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Jakub Jelinek 
On Fri, Feb 15, 2013 at 12:47:30PM +0400, Konstantin Serebryany wrote:
 This is ungood.
 First, clang doesn't like it at all:
 prelink1.cc:18:18: error: init_priority attribute requires integer
 constant between 101 and 65535 inclusive
 A __attribute__((init_priority (1))) a;

For gcc it is just a warning, not error, so you can actually use it if you
know what you are doing.

Anyway, if gold doesn't have any way, you can always do the equivalent
of shell
which prelink 2/dev/null  prelink -r 0x36 libfoo.so
somewhere in the CMakeLists.txt.  That command doesn't affect system
libraries, can be run as normal user, and just transforms a library
from the default link state to -Wl,-Ttext-segment=0x36
state (including debug info etc.).  You'd need to apt-get install prelink
or whatever command is for that on Ubuntu on the test boxes.

OT, unrelated thing, in include/asan_interface.h there is one
#if __has_feature(address_sanitizer)
which for GCC should better be:
#if (defined __has_feature  __has_feature(address_sanitizer)) \
|| defined(__SANITIZE_ADDRESS__)
(and similarly in asan_internal.h).

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Konstantin Serebryany 
On Fri, Feb 15, 2013 at 1:05 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 12:47:30PM +0400, Konstantin Serebryany wrote:
 This is ungood.
 First, clang doesn't like it at all:
 prelink1.cc:18:18: error: init_priority attribute requires integer
 constant between 101 and 65535 inclusive
 A __attribute__((init_priority (1))) a;

 For gcc it is just a warning, not error, so you can actually use it if you
 know what you are doing.

 Anyway, if gold doesn't have any way, you can always do the equivalent
 of shell
 which prelink 2/dev/null  prelink -r 0x36 libfoo.so
 somewhere in the CMakeLists.txt.  That command doesn't affect system
 libraries, can be run as normal user, and just transforms a library
 from the default link state to -Wl,-Ttext-segment=0x36
 state (including debug info etc.).  You'd need to apt-get install prelink
 or whatever command is for that on Ubuntu on the test boxes.

that's another option, not perfect though (someone not having prelink
on his/her box may break the tests w/o noticing).


 OT, unrelated thing, in include/asan_interface.h there is one
 #if __has_feature(address_sanitizer)
 which for GCC should better be:
 #if (defined __has_feature  __has_feature(address_sanitizer)) \
 || defined(__SANITIZE_ADDRESS__)
 (and similarly in asan_internal.h).

z.c:1:44: error: missing binary operator before token (
 #if (defined __has_feature  __has_feature(address_sanitizer)) \

This should be more like the code below

#if !defined(__has_feature)
#define __has_feature(x) 0
#endif
#if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)

[hopefully not starting a holly war] Any chance to teach gcc/cpp about
__has_feature?

--kcc


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Jakub Jelinek 
On Fri, Feb 15, 2013 at 01:30:18PM +0400, Konstantin Serebryany wrote:
  OT, unrelated thing, in include/asan_interface.h there is one
  #if __has_feature(address_sanitizer)
  which for GCC should better be:
  #if (defined __has_feature  __has_feature(address_sanitizer)) \
  || defined(__SANITIZE_ADDRESS__)
  (and similarly in asan_internal.h).
 
 z.c:1:44: error: missing binary operator before token (
  #if (defined __has_feature  __has_feature(address_sanitizer)) \
 
 This should be more like the code below
 
 #if !defined(__has_feature)
 #define __has_feature(x) 0
 #endif
 #if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)

I don't care much.  Would
#if (defined(__has_feature)  __has_feature(address_sanitizer)) \
|| defined(__SANITIZE_ADDRESS__)
work?  In any case, that looks like clang bug, if you have something that
behaves like a function-like macro, #if defined macro or #ifdef macro
or #if defined macro || 1 and similar should work just fine, only if
the macro is followed by ( it should behave differently.

 [hopefully not starting a holly war] Any chance to teach gcc/cpp about
 __has_feature?

Not for GCC 4.8, that is a new feature, so certainly too late for that.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Konstantin Serebryany 
On Fri, Feb 15, 2013 at 1:37 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 01:30:18PM +0400, Konstantin Serebryany wrote:
  OT, unrelated thing, in include/asan_interface.h there is one
  #if __has_feature(address_sanitizer)
  which for GCC should better be:
  #if (defined __has_feature  __has_feature(address_sanitizer)) \
  || defined(__SANITIZE_ADDRESS__)
  (and similarly in asan_internal.h).

 z.c:1:44: error: missing binary operator before token (
  #if (defined __has_feature  __has_feature(address_sanitizer)) \

 This should be more like the code below

 #if !defined(__has_feature)
 #define __has_feature(x) 0
 #endif
 #if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)

 I don't care much.  Would
 #if (defined(__has_feature)  __has_feature(address_sanitizer)) \
 || defined(__SANITIZE_ADDRESS__)
 work?  In any case, that looks like clang bug, if you have something that
 behaves like a function-like macro, #if defined macro or #ifdef macro
 or #if defined macro || 1 and similar should work just fine, only if
 the macro is followed by ( it should behave differently.

That's gcc:

% cat z.c
#if (defined __has_feature  __has_feature(address_sanitizer)) \
|| defined(__SANITIZE_ADDRESS__)
#error OK
#else
#error NO
#endif
% gcc -c -fsanitize=address  z.c
z.c:1:44: error: missing binary operator before token (
 #if (defined __has_feature  __has_feature(address_sanitizer)) \
^
z.c:5:2: error: #error NO
 #error NO
  ^
% clang -c -fsanitize=address  z.c
z.c:3:2: error: OK
#error OK
 ^
1 error generated.
%



 [hopefully not starting a holly war] Any chance to teach gcc/cpp about
 __has_feature?

 Not for GCC 4.8, that is a new feature, so certainly too late for that.

How about 4.9?


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Konstantin Serebryany 
I've submitted http://llvm.org/viewvc/llvm-project?view=revisionrevision=175263
If it survives a few days of testing I'll do another merge to gcc.

--kcc

On Fri, Feb 15, 2013 at 1:47 PM, Konstantin Serebryany
konstantin.s.serebry...@gmail.com wrote:
 On Fri, Feb 15, 2013 at 1:37 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Fri, Feb 15, 2013 at 01:30:18PM +0400, Konstantin Serebryany wrote:
  OT, unrelated thing, in include/asan_interface.h there is one
  #if __has_feature(address_sanitizer)
  which for GCC should better be:
  #if (defined __has_feature  __has_feature(address_sanitizer)) \
  || defined(__SANITIZE_ADDRESS__)
  (and similarly in asan_internal.h).

 z.c:1:44: error: missing binary operator before token (
  #if (defined __has_feature  __has_feature(address_sanitizer)) \

 This should be more like the code below

 #if !defined(__has_feature)
 #define __has_feature(x) 0
 #endif
 #if __has_feature(address_sanitizer) || defined(__SANITIZE_ADDRESS__)

 I don't care much.  Would
 #if (defined(__has_feature)  __has_feature(address_sanitizer)) \
 || defined(__SANITIZE_ADDRESS__)
 work?  In any case, that looks like clang bug, if you have something that
 behaves like a function-like macro, #if defined macro or #ifdef macro
 or #if defined macro || 1 and similar should work just fine, only if
 the macro is followed by ( it should behave differently.

 That's gcc:

 % cat z.c
 #if (defined __has_feature  __has_feature(address_sanitizer)) \
 || defined(__SANITIZE_ADDRESS__)
 #error OK
 #else
 #error NO
 #endif
 % gcc -c -fsanitize=address  z.c
 z.c:1:44: error: missing binary operator before token (
  #if (defined __has_feature  __has_feature(address_sanitizer)) \
 ^
 z.c:5:2: error: #error NO
  #error NO
   ^
 % clang -c -fsanitize=address  z.c
 z.c:3:2: error: OK
 #error OK
  ^
 1 error generated.
 %



 [hopefully not starting a holly war] Any chance to teach gcc/cpp about
 __has_feature?

 Not for GCC 4.8, that is a new feature, so certainly too late for that.

 How about 4.9?


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-15 Thread Ian Lance Taylor 
On Thu, Feb 14, 2013 at 11:45 PM, Konstantin Serebryany
konstantin.s.serebry...@gmail.com wrote:

 Unfortunately, the test does not work if gold is the system linker.
 Any suggestion on how to make the test work with either linker?

I don't know of a way to set the address of the text segment for both
GNU ld and gold.  As you have observed, GNU ld's -Ttext-segment option
is the same as gold's -Ttext option.  GNU ld's -Ttext option is
useless on ELF systems.

I will add -Ttext-segment to gold as an alias for -Ttext, but that
won't help you today.

Sorry.

Ian

Re: libsanitizer merge from upstream r175042

2013-02-14 Thread Konstantin Serebryany 
On Wed, Feb 13, 2013 at 10:29 PM, H.J. Lu hjl.to...@gmail.com wrote:
 On Wed, Feb 13, 2013 at 1:19 AM, Konstantin Serebryany
 konstantin.s.serebry...@gmail.com wrote:
 Hi,

 The attached patch is the libsanitizer merge from upstream r175042.

 Lots of changes. Among other things:
  - x86_64 linux: change the shadow offset to 0x7fff8000 (~5% speedup)
  - the new asan allocator is enabled on Mac (was enabled on Linux before).
  - tsan finds races between atomic and plain accesses
  - better scanf interceptor, enabled by default
  - don't include linux/futex.h (fixes PR56128)
  - simple tests seem to work (again?) on PowerPC64 with 44-bit address
 space (46 AS not tested)

 Patch for libsanitizer is automatically generated by libsanitizer/merge.sh
 Tested with
 rm -rf */{*/,}libsanitizer \
make -j 50 \
make -C gcc check-g{cc,++}
 RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp'

 Our internal LLVM bots (Linux, Mac and Android) are green.

 Ok to commit?

 --kcc

 This breaks build on Linux/x32 where off_t is 64bit:

Sorry. I've committed your patch upstream as
http://llvm.org/viewvc/llvm-project?rev=175140view=rev
Feel free to submit the same directly to gcc.

Thanks!

--kcc


 In file included from
 /export/gnu/import/git/gcc/libsanitizer/interception/interception.h:20:0,
  from
 /export/gnu/import/git/gcc/libsanitizer/interception/interception_type_test.cc:15:
 /export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:221:72:
 error: size of array ‘assertion_failed__34’ is negative
  typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
 ^
 /export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:215:30:
 note: in expansion of macro ‘IMPL_COMPILER_ASSERT’
  #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
   ^
 /export/gnu/import/git/gcc/libsanitizer/interception/interception_type_test.cc:34:1:
 note: in expansion of macro ‘COMPILER_CHECK’
  COMPILER_CHECK(sizeof(OFF_T) == sizeof(off_t));
  ^
 make[7]: *** [interception_type_test.lo] Error 1

 This patch fixes it.  OK to install?

 Thanks.

 --
 H.J.
 ---
 diff --git a/libsanitizer/interception/interception.h
 b/libsanitizer/interception/interception.h
 index b4c4137..c4c5026 100644
 --- a/libsanitizer/interception/interception.h
 +++ b/libsanitizer/interception/interception.h
 @@ -28,8 +28,8 @@ typedef __sanitizer::s64  INTMAX_T;
  // WARNING: OFF_T may be different from OS type off_t, depending on
 the value of
  // _FILE_OFFSET_BITS. This definition of OFF_T matches the ABI of system 
 calls
  // like pread and mmap, as opposed to pread64 and mmap64.
 -// Mac is special.
 -#ifdef __APPLE__
 +// Mac and Linux/x86-64 are special.
 +#if defined(__APPLE__) || (defined(__linux__)  defined(__x86_64__))
  typedef __sanitizer::u64 OFF_T;
  #else
  typedef __sanitizer::uptr OFF_T;

Re: libsanitizer merge from upstream r175042

2013-02-14 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 04:19:14PM +0100, Jakub Jelinek wrote:
 Here is the patch, works just fine for me here during asan.exp testing.
 You can very easily either install and enable prelink on one of your
 x86_64-linux testing boxes, or just install it and add test that
 will say prelink -r 0x36 some test shared library and then
 just use it in sanitized program (that will also verify that you can mmap
 libraries in that range), or even just write a test that will in a
 non-instrumented ctor with lower priority than asan's priority
 mmap a few pages at 0x30 and close to 0x3f
 and store some data into those buffers later on in sanitized code.

I forgot you don't even need prelink -r 0x36 for the testing, you
can just link it as
$(CXX) -shared -Wl,-Ttext-segment=0x36 -fPIC -o testlib.so 
-fsanitize=address testlib.C
So, put some asan tests into the executable, some tests into the shared
library and link the executable against the shared library placed in the
area where prelink allocates addresses to shared libraries.
Perhaps build 3 such libraries, one at 0x30, one somewhere middle
of that range and one close to the end of the range.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-14 Thread Konstantin Serebryany 
The patch seems to work on a simple test. Let me digest it.
I am trying to understand if there are problems with it other than the
added complexity (which is what I don't like the most).

-Wl,-Ttext-segment=0x36 does not work with binutils-gold.
gold understands -Wl,-Ttext=0x36, but bfd ld doesn't.
Do you know any flag supported by both?

--kcc


On Thu, Feb 14, 2013 at 12:48 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Wed, Feb 13, 2013 at 04:19:14PM +0100, Jakub Jelinek wrote:
 Here is the patch, works just fine for me here during asan.exp testing.
 You can very easily either install and enable prelink on one of your
 x86_64-linux testing boxes, or just install it and add test that
 will say prelink -r 0x36 some test shared library and then
 just use it in sanitized program (that will also verify that you can mmap
 libraries in that range), or even just write a test that will in a
 non-instrumented ctor with lower priority than asan's priority
 mmap a few pages at 0x30 and close to 0x3f
 and store some data into those buffers later on in sanitized code.

 I forgot you don't even need prelink -r 0x36 for the testing, you
 can just link it as
 $(CXX) -shared -Wl,-Ttext-segment=0x36 -fPIC -o testlib.so 
 -fsanitize=address testlib.C
 So, put some asan tests into the executable, some tests into the shared
 library and link the executable against the shared library placed in the
 area where prelink allocates addresses to shared libraries.
 Perhaps build 3 such libraries, one at 0x30, one somewhere middle
 of that range and one close to the end of the range.

 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-14 Thread Konstantin Serebryany 
On Thu, Feb 14, 2013 at 4:19 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Thu, Feb 14, 2013 at 03:55:47PM +0400, Konstantin Serebryany wrote:
 The patch seems to work on a simple test. Let me digest it.
 I am trying to understand if there are problems with it other than the
 added complexity (which is what I don't like the most).

 Yes, it is some added complexity, but not too much, and something that can
 be tested regularly that it works.

 -Wl,-Ttext-segment=0x36 does not work with binutils-gold.
 gold understands -Wl,-Ttext=0x36, but bfd ld doesn't.
 Do you know any flag supported by both?

 -Wl,-Ttext is unfortunately something different, at least for
 the bfd linker.  -Ttext-segment aligns the base of the whole shared library,
 if you look at start of the linker script for -shared:
   /* Read-only sections, merged into text segment: */
   . = SEGMENT_START(text-segment, 0) + SIZEOF_HEADERS;
   .note.gnu.build-id : { *(.note.gnu.build-id) }
   .hash   : { *(.hash) }
   .gnu.hash   : { *(.gnu.hash) }
   .dynsym : { *(.dynsym) }
   .dynstr : { *(.dynstr) }
   .gnu.version: { *(.gnu.version) }
   .gnu.version_d  : { *(.gnu.version_d) }
   .gnu.version_r  : { *(.gnu.version_r) }
 ...
   .rela.plt   :
 {
   *(.rela.plt)
   *(.rela.iplt)
 }
   .init   :
   {
 KEEP (*(.init))
   }
   .plt: { *(.plt) *(.iplt) }
   .text   :
   {
 *(.text.unlikely .text.*_unlikely)
 *(.text.exit .text.exit.*)
 -Ttext-segment chooses the base at which ELF headers will reside.
 -Ttext aligns the .text section's start to that, so most likely the shared
 library won't even link, because .init section will be many GBs appart from
 .text section.

 CCing Ian, if gold has any way to do something similar.
 As I said, the alternative is to link the library normally, and run
 prelink -r 0x36 libtest.so on the shared library afterwards if 
 prelink is
 installed, and make sure you install it on your linux/x86-64 test boxes.

Another way is to simply force using the bfd linker (on ubuntu, when
gold is the default linker, there is still bfd linker under
/usr/bin/ld.bfd).
Still, better to have something that works for both linkers.

--kcc


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-14 Thread Konstantin Serebryany 
On Thu, Feb 14, 2013 at 4:19 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Thu, Feb 14, 2013 at 03:55:47PM +0400, Konstantin Serebryany wrote:
 The patch seems to work on a simple test. Let me digest it.
 I am trying to understand if there are problems with it other than the
 added complexity (which is what I don't like the most).

 Yes, it is some added complexity, but not too much, and something that can
 be tested regularly that it works.

The complexity I am afraid of is not only in the code, but also at the
time of execution.
We and our users sometimes have to stare at the /proc/self/maps.
A mapping with 1 (ZeroBase) or 3 (default) asan sections is ok, but
6 extra asan sections becomes nearly incomprehensible, at least for me.

So, how about having kMidMemBeg as a variable, set as __asan_init.
Only if something is mapped around 0x003X we set it to non-zero.

http://llvm-reviews.chandlerc.com/D411 (still needs some cleanup)

Unfortunately, the test does not work if gold is the system linker.
Any suggestion on how to make the test work with either linker?

Thanks,

--kcc


 -Wl,-Ttext-segment=0x36 does not work with binutils-gold.
 gold understands -Wl,-Ttext=0x36, but bfd ld doesn't.
 Do you know any flag supported by both?

 -Wl,-Ttext is unfortunately something different, at least for
 the bfd linker.  -Ttext-segment aligns the base of the whole shared library,
 if you look at start of the linker script for -shared:
   /* Read-only sections, merged into text segment: */
   . = SEGMENT_START(text-segment, 0) + SIZEOF_HEADERS;
   .note.gnu.build-id : { *(.note.gnu.build-id) }
   .hash   : { *(.hash) }
   .gnu.hash   : { *(.gnu.hash) }
   .dynsym : { *(.dynsym) }
   .dynstr : { *(.dynstr) }
   .gnu.version: { *(.gnu.version) }
   .gnu.version_d  : { *(.gnu.version_d) }
   .gnu.version_r  : { *(.gnu.version_r) }
 ...
   .rela.plt   :
 {
   *(.rela.plt)
   *(.rela.iplt)
 }
   .init   :
   {
 KEEP (*(.init))
   }
   .plt: { *(.plt) *(.iplt) }
   .text   :
   {
 *(.text.unlikely .text.*_unlikely)
 *(.text.exit .text.exit.*)
 -Ttext-segment chooses the base at which ELF headers will reside.
 -Ttext aligns the .text section's start to that, so most likely the shared
 library won't even link, because .init section will be many GBs appart from
 .text section.

 CCing Ian, if gold has any way to do something similar.
 As I said, the alternative is to link the library normally, and run
 prelink -r 0x36 libtest.so on the shared library afterwards if 
 prelink is
 installed, and make sure you install it on your linux/x86-64 test boxes.

 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 01:19:47PM +0400, Konstantin Serebryany wrote:
 The attached patch is the libsanitizer merge from upstream r175042.
 
 Lots of changes. Among other things:
  - x86_64 linux: change the shadow offset to 0x7fff8000 (~5% speedup)
  - the new asan allocator is enabled on Mac (was enabled on Linux before).
  - tsan finds races between atomic and plain accesses
  - better scanf interceptor, enabled by default
  - don't include linux/futex.h (fixes PR56128)
  - simple tests seem to work (again?) on PowerPC64 with 44-bit address
 space (46 AS not tested)
 
 Patch for libsanitizer is automatically generated by libsanitizer/merge.sh
 Tested with
 rm -rf */{*/,}libsanitizer \
make -j 50 \
make -C gcc check-g{cc,++}
 RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp'
 
 Our internal LLVM bots (Linux, Mac and Android) are green.
 
 Ok to commit?

--- libsanitizer/asan/asan_mapping.h(revision 195997)
+++ libsanitizer/asan/asan_mapping.h(working copy)
@@ -34,27 +34,16 @@
 #   if defined(__powerpc64__)
 #define SHADOW_OFFSET (1ULL  41)
 #   else
-#define SHADOW_OFFSET (1ULL  44)
+#define SHADOW_OFFSET 0x7fff8000ULL
 #   endif
 #  endif
 # endif
 #endif  // ASAN_FLEXIBLE_MAPPING_AND_OFFSET

This is inconsistent with the i386.c change.  You said the 0x7fff8000ULL
shadow offset doesn't work on Darwin, so either the above should be

+#if ASAN_MAC
+# define SHADOW_OFFSET (1ULL  44)
+#else
+# define SHADOW_OFFSET 0x7fff8000ULL
+#endif

or i386.c should use 0x7fff8000 even for TARGET_MACHO  TARGET_LP64.

--- gcc/config/i386/i386.c  (revision 195997)
+++ gcc/config/i386/i386.c  (working copy)
@@ -5436,7 +5436,9 @@
 static unsigned HOST_WIDE_INT
 ix86_asan_shadow_offset (void)
 {
-  return (unsigned HOST_WIDE_INT) 1  (TARGET_LP64 ? 44 : 29);
+  return TARGET_LP64 ? (TARGET_MACHO ? (HOST_WIDE_INT_1  44)
+ : HOST_WIDE_INT_C (0x7fff8000))
+ : (HOST_WIDE_INT_1  29);

Please use tabs instead of 8 spaces, and indent it properly (second line
: below the second ?, third line : below first ?).

+2013-02-13  Kostya Serebryany  k...@google.com
+
+   * config/i386/i386.c: use 0x7fff8000 as asan_shadow_offset on x86_64
+   linux.

Start sentence, so Use instead of use.

+   * sanitizer.def: rename __asan_init to __asan_init_v1.

Likewise, Rename.

+   * testsuite/c-c++-common/asan/strncpy-overflow-1.c: update the test
+   to match the fresh asan run-time.

Update.  Also, these two go into gcc/testsuite/ChangeLog, without
testsuite/ prefix in the pathnames.

+   * testsuite/c-c++-common/asan/rlimit-mmap-test-1.c: Ditto.
+

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 02:28:25PM +0400, Konstantin Serebryany wrote:
 Right. In LLVM we test only with ASAN_FLEXIBLE_MAPPING_AND_OFFSET==1,
 so this came unnoticed.
 Fixed in r175049.
...

This is ok, thanks.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 11:32:00AM +0100, Jakub Jelinek wrote:
 On Wed, Feb 13, 2013 at 02:28:25PM +0400, Konstantin Serebryany wrote:
  Right. In LLVM we test only with ASAN_FLEXIBLE_MAPPING_AND_OFFSET==1,
  so this came unnoticed.
  Fixed in r175049.
 ...
 
 This is ok, thanks.

Unfortunately, it seems everything fails with that change :( on Linux.
The problem is that the default prelink library range for x86_64 is
0x30LL to 0x40LL, and that unfortunately overlaps
with the 0x7fff8000LL to 0x10007fff8000LL range that asan wants to use
for the shadow mapping.  And the reason for that prelink default range is
that earlier (see e.g.
http://lwn.net/Articles/106177/
) Linux on x86_64 used much smaller virtual address space than it does now.
Not sure if there are still systems running pre-2.6.9 kernels or whenever the 
PML4
change made it into Linux kernel on x86-64 and whether people use prelink on
them.  But in any case, even if I change the prelink range now (perhaps
conditionally on the size of address space detected by prelink), it will
still cause issues.

So, either we need to revert that i386.c and asan_mapping.h (SHADOW_OFFSET)
change, or support non-contiguous shadow memory for the Linux x86-64 case.
What could work is if we had:

0x  -   0x7fff8000  low memory
0x7fff8000  -   0x8fff7000  shadow mem for low memory
0x8fff7000  -   0x00067fff8000  protected
0x00067fff8000  -   0x00087fff8000  shadow mem for mid memory
0x00087fff8000  -   0x0030  protected
0x0030  -   0x0040  mid memory
0x0040  -   0x02008fff7000  protected
0x02008fff7000  -   0x10007fff8000  shadow mem for high memory
0x10007fff8000  -   0x7fff  high memory

asan_mapping.h then would need to introduce AddrIsInMidMem and
AddrIsInMidShadow inlines (perhaps defined to false for configurations
that don't need 3 part memory), use those in AddrIsInMem
and AddrIsInShadow, tweak AddrIsInShadowGap (as it has now more gaps)
for this configuration and tweak the mapping code.

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Konstantin Serebryany 
On Wed, Feb 13, 2013 at 3:59 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Wed, Feb 13, 2013 at 11:32:00AM +0100, Jakub Jelinek wrote:
 On Wed, Feb 13, 2013 at 02:28:25PM +0400, Konstantin Serebryany wrote:
  Right. In LLVM we test only with ASAN_FLEXIBLE_MAPPING_AND_OFFSET==1,
  so this came unnoticed.
  Fixed in r175049.
 ...

 This is ok, thanks.

 Unfortunately, it seems everything fails with that change :( on Linux.
 The problem is that the default prelink library range for x86_64 is
 0x30LL to 0x40LL, and that unfortunately overlaps

Forgive my ignorance, what is the  default prelink library range?

 with the 0x7fff8000LL to 0x10007fff8000LL range that asan wants to use
 for the shadow mapping.  And the reason for that prelink default range is
 that earlier (see e.g.
 http://lwn.net/Articles/106177/
 ) Linux on x86_64 used much smaller virtual address space than it does now.
 Not sure if there are still systems running pre-2.6.9 kernels or whenever the 
 PML4
 change made it into Linux kernel on x86-64 and whether people use prelink on
 them.  But in any case, even if I change the prelink range now (perhaps
 conditionally on the size of address space detected by prelink), it will
 still cause issues.

 So, either we need to revert that i386.c and asan_mapping.h (SHADOW_OFFSET)
 change, or support non-contiguous shadow memory for the Linux x86-64 case.

I suggest to either revert or (better) to support flexible mapping and
revert the offset only in the gcc compiler module
(leaving asan-rt unchanged).

non-contiguous shadow memory sounds too scary and costly to support,
not worth the benefit.

 What could work is if we had:

 0x  -   0x7fff8000  low memory
 0x7fff8000  -   0x8fff7000  shadow mem for low memory
 0x8fff7000  -   0x00067fff8000  protected
 0x00067fff8000  -   0x00087fff8000  shadow mem for mid memory
 0x00087fff8000  -   0x0030  protected
 0x0030  -   0x0040  mid memory
 0x0040  -   0x02008fff7000  protected
 0x02008fff7000  -   0x10007fff8000  shadow mem for high memory
 0x10007fff8000  -   0x7fff  high memory

 asan_mapping.h then would need to introduce AddrIsInMidMem and
 AddrIsInMidShadow inlines (perhaps defined to false for configurations
 that don't need 3 part memory), use those in AddrIsInMem
 and AddrIsInShadow, tweak AddrIsInShadowGap (as it has now more gaps)
 for this configuration and tweak the mapping code.

 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 04:32:33PM +0400, Konstantin Serebryany wrote:
  Unfortunately, it seems everything fails with that change :( on Linux.
  The problem is that the default prelink library range for x86_64 is
  0x30LL to 0x40LL, and that unfortunately overlaps
 
 Forgive my ignorance, what is the  default prelink library range?

Prelink is a program of mine (see e.g.
http://people.redhat.com/jakub/prelink.pdf
) that speeds up dynamic linking of programs.
It is used by default on various Linux distributions.  prelinked shared
libraries (and dynamic linker) have their base addresses chosen by the
prelink program (and, by default in the default range of shared libraries
for the architecture, which is 0x30LL to 0x40LL for x86_64).
The addresses are usually (depending on prelink options) randomized upon
prelinking, so different hosts use different addresses and the same host
even after a while uses different addresses too, but by default for a few
weeks, if say glibc isn't upgraded in between, you get the same addresses
for the libraries all the time.
So, you get something like:
cat /proc/self/maps
0040-0040b000 r-xp  08:02 1201920
/usr/bin/cat
0060a000-0060b000 r--p a000 08:02 1201920
/usr/bin/cat
0060b000-0060c000 rw-p b000 08:02 1201920
/usr/bin/cat
01431000-01452000 rw-p  00:00 0  [heap]
3fe940-3fe942 r-xp  08:02 1179965
/usr/lib64/ld-2.15.so
3fe961f000-3fe962 r--p 0001f000 08:02 1179965
/usr/lib64/ld-2.15.so
3fe962-3fe9621000 rw-p 0002 08:02 1179965
/usr/lib64/ld-2.15.so
3fe9621000-3fe9622000 rw-p  00:00 0 
3fe980-3fe99ac000 r-xp  08:02 1180697
/usr/lib64/libc-2.15.so
3fe99ac000-3fe9bac000 ---p 001ac000 08:02 1180697
/usr/lib64/libc-2.15.so
3fe9bac000-3fe9bb r--p 001ac000 08:02 1180697
/usr/lib64/libc-2.15.so
3fe9bb-3fe9bb2000 rw-p 001b 08:02 1180697
/usr/lib64/libc-2.15.so
3fe9bb2000-3fe9bb7000 rw-p  00:00 0 
7fae406f5000-7fae46b22000 r--p  08:02 1215605
/usr/lib/locale/locale-archive
7fae46b22000-7fae46b25000 rw-p  00:00 0 
7fae46b42000-7fae46b43000 rw-p  00:00 0 
7fffe04f7000-7fffe0518000 rw-p  00:00 0  [stack]
7fffe05e6000-7fffe05e7000 r-xp  00:00 0  [vdso]
ff60-ff601000 r-xp  00:00 0  
[vsyscall]

Perhaps Ubuntu doesn't enable prelink by default, but all the world isn't 
Ubuntu.

 I suggest to either revert or (better) to support flexible mapping and
 revert the offset only in the gcc compiler module
 (leaving asan-rt unchanged).

I don't see how could flexible mapping help in this case, it just can't work 
either.
The only exception are PIE binaries, which by design aren't prelinked and kernel
for them disregards the prelinking of the dynamic linker, so the dynamic linker
for PIEs isn't loaded at the prelink chosen address, but at some randomized 
address.

If you try -Wl,-Ttext* with flexible mapping and use either zero base, or say 
0x7fff8000,
you run into the exactly same issues, wether with gcc or clang.

 non-contiguous shadow memory sounds too scary and costly to support,
 not worth the benefit.

Why do you think it should be too costly?

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Konstantin Serebryany 
On Wed, Feb 13, 2013 at 4:48 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Wed, Feb 13, 2013 at 04:32:33PM +0400, Konstantin Serebryany wrote:
  Unfortunately, it seems everything fails with that change :( on Linux.
  The problem is that the default prelink library range for x86_64 is
  0x30LL to 0x40LL, and that unfortunately overlaps

 Forgive my ignorance, what is the  default prelink library range?

 Prelink is a program of mine (see e.g.
 http://people.redhat.com/jakub/prelink.pdf
 ) that speeds up dynamic linking of programs.
 It is used by default on various Linux distributions.

Can it be disabled somehow (for asan)?

  prelinked shared
 libraries (and dynamic linker) have their base addresses chosen by the
 prelink program (and, by default in the default range of shared libraries
 for the architecture, which is 0x30LL to 0x40LL for x86_64).
 The addresses are usually (depending on prelink options) randomized upon
 prelinking, so different hosts use different addresses and the same host
 even after a while uses different addresses too, but by default for a few
 weeks, if say glibc isn't upgraded in between, you get the same addresses
 for the libraries all the time.
 So, you get something like:
 cat /proc/self/maps
 0040-0040b000 r-xp  08:02 1201920
 /usr/bin/cat
 0060a000-0060b000 r--p a000 08:02 1201920
 /usr/bin/cat
 0060b000-0060c000 rw-p b000 08:02 1201920
 /usr/bin/cat
 01431000-01452000 rw-p  00:00 0  
 [heap]
 3fe940-3fe942 r-xp  08:02 1179965
 /usr/lib64/ld-2.15.so
 3fe961f000-3fe962 r--p 0001f000 08:02 1179965
 /usr/lib64/ld-2.15.so
 3fe962-3fe9621000 rw-p 0002 08:02 1179965
 /usr/lib64/ld-2.15.so
 3fe9621000-3fe9622000 rw-p  00:00 0
 3fe980-3fe99ac000 r-xp  08:02 1180697
 /usr/lib64/libc-2.15.so
 3fe99ac000-3fe9bac000 ---p 001ac000 08:02 1180697
 /usr/lib64/libc-2.15.so
 3fe9bac000-3fe9bb r--p 001ac000 08:02 1180697
 /usr/lib64/libc-2.15.so
 3fe9bb-3fe9bb2000 rw-p 001b 08:02 1180697
 /usr/lib64/libc-2.15.so
 3fe9bb2000-3fe9bb7000 rw-p  00:00 0
 7fae406f5000-7fae46b22000 r--p  08:02 1215605
 /usr/lib/locale/locale-archive
 7fae46b22000-7fae46b25000 rw-p  00:00 0
 7fae46b42000-7fae46b43000 rw-p  00:00 0
 7fffe04f7000-7fffe0518000 rw-p  00:00 0  
 [stack]
 7fffe05e6000-7fffe05e7000 r-xp  00:00 0  
 [vdso]
 ff60-ff601000 r-xp  00:00 0  
 [vsyscall]

 Perhaps Ubuntu doesn't enable prelink by default, but all the world isn't 
 Ubuntu.

 I suggest to either revert or (better) to support flexible mapping and
 revert the offset only in the gcc compiler module
 (leaving asan-rt unchanged).

 I don't see how could flexible mapping help in this case, it just can't work 
 either.

If we enabled flexible mapping in the gcc build (making it more
similar to the llvm build)
we will be able to use the old 2^44 offset w/o changing the asan-rt.

 The only exception are PIE binaries, which by design aren't prelinked and 
 kernel
 for them disregards the prelinking of the dynamic linker, so the dynamic 
 linker
 for PIEs isn't loaded at the prelink chosen address, but at some randomized 
 address.

 If you try -Wl,-Ttext* with flexible mapping and use either zero base, or say 
 0x7fff8000,
 you run into the exactly same issues, wether with gcc or clang.

 non-contiguous shadow memory sounds too scary and costly to support,
 not worth the benefit.

 Why do you think it should be too costly?

That's yet another set of spaghetti ifdefs.
I'd rather revert the whole thing back to 2^44 than do that.

--kcc

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 04:57:30PM +0400, Konstantin Serebryany wrote:
 On Wed, Feb 13, 2013 at 4:48 PM, Jakub Jelinek ja...@redhat.com wrote:
  On Wed, Feb 13, 2013 at 04:32:33PM +0400, Konstantin Serebryany wrote:
   Unfortunately, it seems everything fails with that change :( on Linux.
   The problem is that the default prelink library range for x86_64 is
   0x30LL to 0x40LL, and that unfortunately overlaps
 
  Forgive my ignorance, what is the  default prelink library range?
 
  Prelink is a program of mine (see e.g.
  http://people.redhat.com/jakub/prelink.pdf
  ) that speeds up dynamic linking of programs.
  It is used by default on various Linux distributions.
 
 Can it be disabled somehow (for asan)?

No.  You can disable it for the whole system (prelink -ua), but that is not
a sane requirement to running sanitized programs.  There is
also LD_USE_LOAD_BIAS=0 env variable, but 1) that has to be set before
running the program, setting it from within libasan ctor is too late, and
more importantly 2) it still doesn't affect the mapping of the dynamic
linker.  So, with LD_USE_LOAD_BIAS=0 cat /proc/self/maps the
dynamic linker will be still mapped somewhere in the 0x30 to
0x40 range, just other shared libraries mapped in by the dynamic
linker, rather than kernel directly, will be placed at the randomized (and
usually high) addresses, typically 0x7fXX.

 If we enabled flexible mapping in the gcc build (making it more
 similar to the llvm build)
 we will be able to use the old 2^44 offset w/o changing the asan-rt.

Sure, but it will be then slower, I thought you are looking for ASAN speed
improvements.

 That's yet another set of spaghetti ifdefs.
 I'd rather revert the whole thing back to 2^44 than do that.

Please revert the two hunks now.  I'll try to implement it eventually and
try to convince you ;)

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Richard Biener 
On Wed, Feb 13, 2013 at 2:07 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Wed, Feb 13, 2013 at 04:57:30PM +0400, Konstantin Serebryany wrote:
 On Wed, Feb 13, 2013 at 4:48 PM, Jakub Jelinek ja...@redhat.com wrote:
  On Wed, Feb 13, 2013 at 04:32:33PM +0400, Konstantin Serebryany wrote:
   Unfortunately, it seems everything fails with that change :( on Linux.
   The problem is that the default prelink library range for x86_64 is
   0x30LL to 0x40LL, and that unfortunately overlaps
 
  Forgive my ignorance, what is the  default prelink library range?
 
  Prelink is a program of mine (see e.g.
  http://people.redhat.com/jakub/prelink.pdf
  ) that speeds up dynamic linking of programs.
  It is used by default on various Linux distributions.

 Can it be disabled somehow (for asan)?

 No.  You can disable it for the whole system (prelink -ua), but that is not
 a sane requirement to running sanitized programs.  There is
 also LD_USE_LOAD_BIAS=0 env variable, but 1) that has to be set before
 running the program, setting it from within libasan ctor is too late, and
 more importantly 2) it still doesn't affect the mapping of the dynamic
 linker.  So, with LD_USE_LOAD_BIAS=0 cat /proc/self/maps the
 dynamic linker will be still mapped somewhere in the 0x30 to
 0x40 range, just other shared libraries mapped in by the dynamic
 linker, rather than kernel directly, will be placed at the randomized (and
 usually high) addresses, typically 0x7fXX.

ASAN could set an ELF flag on the executable to tell the kernel not
to use prelinked objects?  That is, similar to how we handle executable
stacks?

Richard.

 If we enabled flexible mapping in the gcc build (making it more
 similar to the llvm build)
 we will be able to use the old 2^44 offset w/o changing the asan-rt.

 Sure, but it will be then slower, I thought you are looking for ASAN speed
 improvements.

 That's yet another set of spaghetti ifdefs.
 I'd rather revert the whole thing back to 2^44 than do that.

 Please revert the two hunks now.  I'll try to implement it eventually and
 try to convince you ;)

 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 02:27:56PM +0100, Richard Biener wrote:
 ASAN could set an ELF flag on the executable to tell the kernel not
 to use prelinked objects?  That is, similar to how we handle executable
 stacks?

But we don't have such a flag right now, and what should old kernels that
don't support it do with that flag?  Should it be some kind of flag that
causes the programs not to run at all on kernel  3.9 or whenever the flag
would be handled (can be done with .note.ABI-tag section, plus some flag
somewhere), or something else?  In any case, that would mean changes all
through the toolchain (glibc, linkers, kernel), compared to changing
two files in libsanitizer/asan/ (asan_mapping.h and asan_rtl.cc I believe).

Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Konstantin Serebryany 
On Wed, Feb 13, 2013 at 5:07 PM, Jakub Jelinek ja...@redhat.com wrote:
 On Wed, Feb 13, 2013 at 04:57:30PM +0400, Konstantin Serebryany wrote:
 On Wed, Feb 13, 2013 at 4:48 PM, Jakub Jelinek ja...@redhat.com wrote:
  On Wed, Feb 13, 2013 at 04:32:33PM +0400, Konstantin Serebryany wrote:
   Unfortunately, it seems everything fails with that change :( on Linux.
   The problem is that the default prelink library range for x86_64 is
   0x30LL to 0x40LL, and that unfortunately overlaps
 
  Forgive my ignorance, what is the  default prelink library range?
 
  Prelink is a program of mine (see e.g.
  http://people.redhat.com/jakub/prelink.pdf
  ) that speeds up dynamic linking of programs.
  It is used by default on various Linux distributions.

 Can it be disabled somehow (for asan)?

 No.  You can disable it for the whole system (prelink -ua), but that is not
 a sane requirement to running sanitized programs.

Why not?
:)

 There is
 also LD_USE_LOAD_BIAS=0 env variable, but 1) that has to be set before
 running the program, setting it from within libasan ctor is too late,

This we can deal with.
We already setenv+reexec on Mac to solve similar issue with Mac's
dynamic run-time.

 and
 more importantly 2) it still doesn't affect the mapping of the dynamic
 linker.  So, with LD_USE_LOAD_BIAS=0 cat /proc/self/maps the
 dynamic linker will be still mapped somewhere in the 0x30 to
 0x40 range, just other shared libraries mapped in by the dynamic
 linker, rather than kernel directly, will be placed at the randomized (and
 usually high) addresses, typically 0x7fXX.

 If we enabled flexible mapping in the gcc build (making it more
 similar to the llvm build)
 we will be able to use the old 2^44 offset w/o changing the asan-rt.

 Sure, but it will be then slower, I thought you are looking for ASAN speed
 improvements.

Yes, and we already achieved it on ubuntu :)


 That's yet another set of spaghetti ifdefs.
 I'd rather revert the whole thing back to 2^44 than do that.

 Please revert the two hunks now.

That's a bummer. We've already deployed Clang with the new setting and
will likely not want to revert it there.
If we revert the two hunks in gcc (in gcc/config/i386/i386.c and in
asan_mapping.h) we will have larger inconsistency between gcc and
clang variants.
But if we switch to flexible mapping in gcc (will require gcc to emit
the two globals) we will only need to revert gcc/config/i386/i386.c
and the run-times in clang and gcc will remain the same.

 I'll try to implement it eventually and
 try to convince you ;)

That's surely not hard to implement, but very hard to support.

--kcc


 Jakub

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jack Howarth 
On Wed, Feb 13, 2013 at 11:32:00AM +0100, Jakub Jelinek wrote:
 On Wed, Feb 13, 2013 at 02:28:25PM +0400, Konstantin Serebryany wrote:
  Right. In LLVM we test only with ASAN_FLEXIBLE_MAPPING_AND_OFFSET==1,
  so this came unnoticed.
  Fixed in r175049.
 ...
 
 This is ok, thanks.
 
   Jakub

FYI, asan.exp shows no regression at -m32/-m64 on x86_64-apple-darwin12 using
gcc svn at r196013. Thanks for the new allocator support on darwin.
Jack

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 05:39:15PM +0400, Konstantin Serebryany wrote:
  No.  You can disable it for the whole system (prelink -ua), but that is not
  a sane requirement to running sanitized programs.
 
 Why not?
 :)

Because that is a fully system operation, requires root access, etc.
The fact that some user wants to test one of his programs with Asan
shouldn't need to affect other users.

 This we can deal with.
 We already setenv+reexec on Mac to solve similar issue with Mac's
 dynamic run-time.

The reexec is problematic, what if the program already in constructors run
before __asan_init (perhaps ctors of other libraries etc.) does something
that really shouldn't be done twice?

  Sure, but it will be then slower, I thought you are looking for ASAN speed
  improvements.
 
 Yes, and we already achieved it on ubuntu :)

AFAIK prelink is available even on ubuntu, perhaps not the default.

  I'll try to implement it eventually and
  try to convince you ;)
 
 That's surely not hard to implement, but very hard to support.

Why?

Here is the patch, works just fine for me here during asan.exp testing.
You can very easily either install and enable prelink on one of your
x86_64-linux testing boxes, or just install it and add test that
will say prelink -r 0x36 some test shared library and then
just use it in sanitized program (that will also verify that you can mmap
libraries in that range), or even just write a test that will in a
non-instrumented ctor with lower priority than asan's priority
mmap a few pages at 0x30 and close to 0x3f
and store some data into those buffers later on in sanitized code.

--- asan_mapping.h.jj   2013-02-13 11:53:43.0 +0100
+++ asan_mapping.h  2013-02-13 16:00:22.821413836 +0100
@@ -61,13 +61,31 @@ extern SANITIZER_INTERFACE_ATTRIBUTE upt
 #define kHighShadowBeg  MEM_TO_SHADOW(kHighMemBeg)
 #define kHighShadowEnd  MEM_TO_SHADOW(kHighMemEnd)
 
+#if ASAN_LINUX  defined(__x86_64__)
+# define kMidMemBeg(kLowMemEnd  0x30ULL ? 0x30ULL : 0)
+# define kMidMemEnd(kLowMemEnd  0x30ULL ? 0x3fULL : 0)
+# define kMidShadowBeg MEM_TO_SHADOW(kMidMemBeg)
+# define kMidShadowEnd MEM_TO_SHADOW(kMidMemEnd)
+#else
+# define kMidMemBeg0
+# define kMidMemEnd0
+# define kMidShadowBeg 0
+# define kMidShadowEnd 0
+#endif
+
 // With the zero shadow base we can not actually map pages starting from 0.
 // This constant is somewhat arbitrary.
 #define kZeroBaseShadowStart (1  18)
 
 #define kShadowGapBeg   (kLowShadowEnd ? kLowShadowEnd + 1 \
: kZeroBaseShadowStart)
-#define kShadowGapEnd   (kHighShadowBeg - 1)
+#define kShadowGapEnd   ((kMidMemBeg ? kMidShadowBeg : kHighShadowBeg) - 1)
+
+#define kShadowGap2Beg (kMidMemBeg ? kMidShadowEnd + 1 : 0)
+#define kShadowGap2End (kMidMemBeg ? kMidMemBeg - 1 : 0)
+
+#define kShadowGap3Beg (kMidMemBeg ? kMidMemEnd + 1 : 0)
+#define kShadowGap3End (kMidMemBeg ? kHighShadowBeg - 1 : 0)
 
 namespace __asan {
 
@@ -86,8 +104,12 @@ static inline bool AddrIsInHighMem(uptr
   return a = kHighMemBeg  a = kHighMemEnd;
 }
 
+static inline bool AddrIsInMidMem(uptr a) {
+  return kMidMemBeg  a = kMidMemBeg  a = kMidMemEnd;
+}
+
 static inline bool AddrIsInMem(uptr a) {
-  return AddrIsInLowMem(a) || AddrIsInHighMem(a);
+  return AddrIsInLowMem(a) || AddrIsInMidMem(a) || AddrIsInHighMem(a);
 }
 
 static inline uptr MemToShadow(uptr p) {
@@ -99,11 +121,22 @@ static inline bool AddrIsInHighShadow(up
   return a = kHighShadowBeg  a =  kHighMemEnd;
 }
 
+static inline bool AddrIsInMidShadow(uptr a) {
+  return kMidMemBeg  a = kMidShadowBeg  a = kMidMemEnd;
+}
+
 static inline bool AddrIsInShadow(uptr a) {
-  return AddrIsInLowShadow(a) || AddrIsInHighShadow(a);
+  return AddrIsInLowShadow(a) || AddrIsInMidShadow(a) || AddrIsInHighShadow(a);
 }
 
 static inline bool AddrIsInShadowGap(uptr a) {
+  if (kMidMemBeg)
+{
+  if (a = kShadowGapEnd)
+   return SHADOW_OFFSET == 0 || a = kShadowGapBeg;
+  return (a = kShadowGap2Beg  a = kShadowGap2End)
+|| (a = kShadowGap3Beg  a = kShadowGap3End);
+}
   // In zero-based shadow mode we treat addresses near zero as addresses
   // in shadow gap as well.
   if (SHADOW_OFFSET == 0)
--- asan_rtl.cc.jj  2013-02-13 11:53:44.0 +0100
+++ asan_rtl.cc 2013-02-13 16:00:10.815483846 +0100
@@ -35,8 +35,14 @@ static void AsanDie() {
 Report(Sleeping for %d second(s)\n, flags()-sleep_before_dying);
 SleepForSeconds(flags()-sleep_before_dying);
   }
-  if (flags()-unmap_shadow_on_exit)
-UnmapOrDie((void*)kLowShadowBeg, kHighShadowEnd - kLowShadowBeg);
+  if (flags()-unmap_shadow_on_exit) {
+if (!kMidMemBeg)
+  UnmapOrDie((void*)kLowShadowBeg, kHighShadowEnd - kLowShadowBeg);
+else {
+  UnmapOrDie((void*)kLowShadowBeg, kMidMemBeg - kLowShadowBeg);
+  UnmapOrDie((void*)kMidMemEnd, kHighShadowEnd - kMidMemEnd);
+}
+  }
   if (death_callback)
 death_callback();

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jack Howarth 
On Wed, Feb 13, 2013 at 04:19:14PM +0100, Jakub Jelinek wrote:
 
 The reexec is problematic, what if the program already in constructors run
 before __asan_init (perhaps ctors of other libraries etc.) does something
 that really shouldn't be done twice?
 

Jakub,
   Wouldn't sorting all of the constructors and destructors by priority, while
retaining the original order, in collect2 solve this issue (as was proposed for
implementing intra-modular init priority support on darwin)?
Jack

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread H.J. Lu 
On Wed, Feb 13, 2013 at 1:19 AM, Konstantin Serebryany
konstantin.s.serebry...@gmail.com wrote:
 Hi,

 The attached patch is the libsanitizer merge from upstream r175042.

 Lots of changes. Among other things:
  - x86_64 linux: change the shadow offset to 0x7fff8000 (~5% speedup)
  - the new asan allocator is enabled on Mac (was enabled on Linux before).
  - tsan finds races between atomic and plain accesses
  - better scanf interceptor, enabled by default
  - don't include linux/futex.h (fixes PR56128)
  - simple tests seem to work (again?) on PowerPC64 with 44-bit address
 space (46 AS not tested)

 Patch for libsanitizer is automatically generated by libsanitizer/merge.sh
 Tested with
 rm -rf */{*/,}libsanitizer \
make -j 50 \
make -C gcc check-g{cc,++}
 RUNTESTFLAGS='--target_board=unix\{-m32,-m64\} asan.exp'

 Our internal LLVM bots (Linux, Mac and Android) are green.

 Ok to commit?

 --kcc

This breaks build on Linux/x32 where off_t is 64bit:

In file included from
/export/gnu/import/git/gcc/libsanitizer/interception/interception.h:20:0,
 from
/export/gnu/import/git/gcc/libsanitizer/interception/interception_type_test.cc:15:
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:221:72:
error: size of array ‘assertion_failed__34’ is negative
 typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
^
/export/gnu/import/git/gcc/libsanitizer/sanitizer_common/sanitizer_internal_defs.h:215:30:
note: in expansion of macro ‘IMPL_COMPILER_ASSERT’
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
  ^
/export/gnu/import/git/gcc/libsanitizer/interception/interception_type_test.cc:34:1:
note: in expansion of macro ‘COMPILER_CHECK’
 COMPILER_CHECK(sizeof(OFF_T) == sizeof(off_t));
 ^
make[7]: *** [interception_type_test.lo] Error 1

This patch fixes it.  OK to install?

Thanks.

-- 
H.J.
---
diff --git a/libsanitizer/interception/interception.h
b/libsanitizer/interception/interception.h
index b4c4137..c4c5026 100644
--- a/libsanitizer/interception/interception.h
+++ b/libsanitizer/interception/interception.h
@@ -28,8 +28,8 @@ typedef __sanitizer::s64  INTMAX_T;
 // WARNING: OFF_T may be different from OS type off_t, depending on
the value of
 // _FILE_OFFSET_BITS. This definition of OFF_T matches the ABI of system calls
 // like pread and mmap, as opposed to pread64 and mmap64.
-// Mac is special.
-#ifdef __APPLE__
+// Mac and Linux/x86-64 are special.
+#if defined(__APPLE__) || (defined(__linux__)  defined(__x86_64__))
 typedef __sanitizer::u64 OFF_T;
 #else
 typedef __sanitizer::uptr OFF_T;

Re: libsanitizer merge from upstream r175042

2013-02-13 Thread Jakub Jelinek 
On Wed, Feb 13, 2013 at 11:48:32AM -0500, Jack Howarth wrote:
 On Wed, Feb 13, 2013 at 04:19:14PM +0100, Jakub Jelinek wrote:
  
  The reexec is problematic, what if the program already in constructors run
  before __asan_init (perhaps ctors of other libraries etc.) does something
  that really shouldn't be done twice?
  
 
Wouldn't sorting all of the constructors and destructors by priority, while
 retaining the original order, in collect2 solve this issue (as was proposed 
 for
 implementing intra-modular init priority support on darwin)?

I wasn't talking about darwin at all, and on Linux inter-CU init priority
works just fine.  I'm just saying that reexec isn't always safe, because
what the program does before __asan_init can be something undesirable to be
done multiple times.

Jakub