Re: [PATCH] Disable libsanitizer before darwin10
For the record, ASan never claimed darwin9 support and isn't going to support darwin9 in the future, so it should be fine to disable building ASan on this platform. On Mon, Feb 11, 2013 at 7:55 PM, Jack Howarth howa...@bromo.med.uc.edu wrote: Iain Sandoe discovered that on intel darwin9, the asan testsuite suffers hundreds of failures due to the absence of dispatch calls (Grand Central Dispatch) prior to darwin10. The attached patch disables building libsanitizer on darwin8 and darwin9 until upstream decides to support the earlier darwin releases. Bootstrap and regression tested on x86_64-apple-darwin12. Okay for gcc trunk? Jack -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCH] fix PR sanitizer/55617
I can't see how full init_priority support can work without proper aid from ld and/or the dynamic linker. According to the Apple people, those don't treat the cross-module priorities properly, so there's little that can be done on the compiler side. On Mon, Feb 4, 2013 at 11:39 PM, Mike Stump mikest...@comcast.net wrote: On Feb 4, 2013, at 1:38 AM, Jakub Jelinek ja...@redhat.com wrote: On Mon, Feb 04, 2013 at 10:22:48AM +0100, Richard Biener wrote: Okay for gcc trunk? But that does not work across translation units, no? ISTR collect2 has support to handle constructor priorities all by itself (at link time, considering all inputs). I wonder why the patch turned from initially at least supporting intra-CU support for ctor priorities into an ugly hack for asan. I guess asan doesn't care too much about inter-CU ctor priorities, it just needs its ctors to run before anything in the same CU is called (mainly the __asan_init call), other CUs either won't be asan instrumented, then it doesn't matter, or will be, but they will have their own __asan_init call. I wonder why darwin cannot use that mechanism to support init priorities? But sure, if collect2 can be used for full init prio support, the better. It would be nice if someone contributed full init_priority support… I'd be happy to review that. A good patch for that would add it to clang for darwin, and have gcc use that same mechanism so that we can interoperate nicely. Absent interoperability… I think it would be annoying, as then you have to have a binary incompatibility to fix the one that is wrong. -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCH] fix PR sanitizer/55617
Alexander, I never claimed full init priority support however FSF gcc on darwin currently has no init priority support at all. Since Mike wanted to sort the destructors as well as the constructors and this achieves usable intra-module init priority support for FSF gcc darwin, I don't see why we don't take advantage of it. Especially considering that the constructors and destructors will now always be sorted anyway. Jack ps We will have one advantage over clang's init priority support as we can use -flto to combine all of the code modules (outside of libraries) into a single one for the sorting of constructors/destructors. This allows the g++.dg/special/conpr-3.C execution test case to operate properly on darwin with -flto. Again, remember that clang currently at least supports init priority on a intra-module level. I am just trying to leverage the sorting of constructors/destructors that we added for asan to achive the same level of functionality in FSF gcc on darwin. Jack, I understand and fully support your desire for intra-module ctor/dtor priority. My comment was meant to reply to Mike (sorry for top-posting it, again), who, as far as I understood him, wanted to see full init_priority support on Darwin, which IIUC can't be implemented without the proper linker support. LTO may help as a bandaid, but I don't think this solution scales well enough yet. Alex
Re: [PATCH] fix PR sanitizer/55617
Constructor priorities on Darwin aren't supposed to work across
translation units, see http://llvm.org/bugs/show_bug.cgi?id=12556:
I was told (by Apple folks) that darwin does not support cross-unit constructor
priorities, sorry. This is true for both gcc and llvm-gcc / clang. Within unit
priorities are emulated on darwin (via emission order).
Also, according to Nick Kledzik, constructors are executed on darwin strictly
in link order. So, when you link foo.o and bar.o, then first ctors from foo.o
are executed, then - from bar.o. Maybe this is even documented in some MachO
docs.
Yet it should be possible to delay the constructor emission for a
single TU until all the compiler passes do their job, and then sort
those constructors according to their priorities.
On Mon, Feb 4, 2013 at 1:22 PM, Richard Biener
richard.guent...@gmail.com wrote:
On Sun, Feb 3, 2013 at 5:57 AM, Jack Howarth howa...@bromo.med.uc.edu wrote:
Currently darwin is unable to utilize libasan with constructors due to
the lack of
constructor priority support on that target. The asan_finish_file routine
inserts an
essential __asan_init into the array of constructors (via the
__mod_init_func section).
However the insertion occurs at the end, and due to the lack of priority
support for
constructors, these are executed from the start of the array of constructors
on program
startup. This causes code any instrumented code that executes before the
__asan_init
call to crash.
Since darwin sets...
#undef SUPPORTS_INIT_PRIORITY
#define SUPPORTS_INIT_PRIORITY 0
in gcc/config/darwin.h, all constructors are automatically set to
#define DEFAULT_INIT_PRIORITY 65535
in gcc/collect2.c. Any code that attempts to set the constructor/destructor
priority
on darwin results in a compile time error of constructor priorities are not
supported.
So asan alone should be unique in emitting priorities different from 65535
on darwin.
The attached patch uses a va_gc vector of constructor symbol/priority
records to queue
this data as it is generated in calls to machopic_asm_out_constructor. Any
instances of
the static constructor with priority 99 emitted by asan are inserted safely
in the front
of the vector queue which retains the original order of the remaining
constructors in
the queue. The contents of the vector queue are later processed in a new
finalize_ctors
routine called from darwin_file_end if necessary. The patch also adds a
g++.dg/asan/pr55617.C
test case which is targeted to i?86-*-darwin* and x86_64-*-darwin*.
The patch reduces the failures observed when running
make -k check-g++ RUNTESTFLAGS=--target_board=unix'{-fsanitize=address}'
from 323 to only 85 on darwin (similar to the results on linux). The cov.C
testcase also
fails on gcc trunk with -fsanitize=address when recrafted into a dynamic
shared library
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55617#c28. This patch eliminates
those
crashes. This problem doesn't extend to when the shared library or module is
dlopen'd
(which works in stock gcc trunk and with this patch as well).
The patch has been bootstrap and regression tested on
x86_64-apple-darwin12.
Okay for gcc trunk?
But that does not work across translation units, no? ISTR collect2 has
support
to handle constructor priorities all by itself (at link time,
considering all inputs).
I wonder why darwin cannot use that mechanism to support init priorities?
Richard.
Jack
ps Unfortunately the flag_sort variable is unavailable inside of
machopic_asm_out_constructor
so we have to unconditionally test for priority == 99.
--
Alexander Potapenko
Software Engineer
Google Moscow
Re: [PATCH] fix PR sanitizer/55617
But that does not work across translation units, no? ISTR collect2 has support to handle constructor priorities all by itself (at link time, considering all inputs). I wonder why darwin cannot use that mechanism to support init priorities? Richard. (resending, sorry for top-posting) Constructor priorities on Darwin aren't supposed to work across translation units, see http://llvm.org/bugs/show_bug.cgi?id=12556: I was told (by Apple folks) that darwin does not support cross-unit constructor priorities, sorry. This is true for both gcc and llvm-gcc / clang. Within unit priorities are emulated on darwin (via emission order). Also, according to Nick Kledzik, constructors are executed on darwin strictly in link order. So, when you link foo.o and bar.o, then first ctors from foo.o are executed, then - from bar.o. Maybe this is even documented in some MachO docs. Yet it should be possible to delay the constructor emission for a single TU until all the compiler passes do their job, and then sort those constructors according to their priorities.
Re: [PATCH] Fix PR55599/sanitizer by disabling static libasan on darwin
In fact nothing prevents GCC from having a linkable static ASan runtime. The reason for the static library being unusable is not in the dynamic runtime itself, but in mach_override not working well with other GCC libraries (it is unable to parse some function prologues). Because ASan is transitioning to the dynamic runtime in LLVM, it's unlikely we'll put any effort into fixing mach_override. On Thu, Dec 6, 2012 at 1:23 AM, Mike Stump mikest...@comcast.net wrote: On Dec 5, 2012, at 10:43 AM, Jack Howarth howa...@bromo.med.uc.edu wrote: The transition of libasan on darwin from using mach_override to the replacement mac function imposition code results in a non-functional static libasan (PR55599). Ok unless the asan people have a better idea... -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCh][Revisedx2] Fix PR55521 by switching libsanitizer from mach_overrideto mac interpose functions on darwin
Clang hasn't fully switched to dylib interposition yet. The -fsanitize=address still links the static version of the runtime. On Tue, Dec 4, 2012 at 7:24 PM, Jack Howarth howa...@bromo.med.uc.edu wrote: On Tue, Dec 04, 2012 at 07:10:27PM +0400, Konstantin Serebryany wrote: On Tue, Dec 4, 2012 at 7:03 PM, Jack Howarth howa...@bromo.med.uc.edu wrote: On Tue, Dec 04, 2012 at 11:02:01AM +0400, Konstantin Serebryany wrote: r194120. I've tested it on Linux, but not on Mac. kcc, Tested at r194135 on x86_64-apple-darwin12. Thanks for the commit. Jack ps Since clang in svn trunk has already switched to the mac function imposition version of asan, did it? Thats what Nick Kledzik said... The clang version of asan has already moved to using interposing and DYLD_INSERT_LIBRARIES. I'll reconfirm that with llvm/clang trunk svn tonight (as the bootstrap on darwin there is currently broken). Jack it might be time to deprecate mach_override out of llvm's compiler-rt trunk so lib/interception/mach_override/, lib/interception/interception_mac.cc and lib/interception/interception_mac.h could be removed. On Tue, Dec 4, 2012 at 6:44 AM, Alexander Potapenko gli...@google.com wrote: I've added MISSING_BLOCKS_SUPPORT to LLVM compiler-rt in r169206. The rest of your change looks good to me as well. On Mon, Dec 3, 2012 at 6:33 PM, Alexander Potapenko gli...@google.com wrote: Jack, Note that MAC_INTERPOSE_FUNCTIONS is always defined in interception.h to either 0 or 1. I'm going to keep #if MAC_INTERPOSE_FUNCTIONS (adding !defined(MISSING_BLOCKS_SUPPORT) where appropriate) in libsanitizer. On Mon, Dec 3, 2012 at 11:17 AM, Mike Stump mikest...@comcast.net wrote: On Dec 3, 2012, at 8:02 AM, Jack Howarth howa...@bromo.med.uc.edu wrote: The attached patch eliminates PR 55521/sanitizer by switching libasan on darwin from using mach_override to mac function interposition So, I'm thinking the sanitizer people will just review and approve it, even though it says darwin and is heavily darwin specific… It's ok by me. -- Alexander Potapenko Software Engineer Google Moscow -- Alexander Potapenko Software Engineer Google Moscow -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCh][Revisedx2] Fix PR55521 by switching libsanitizer from mach_overrideto mac interpose functions on darwin
Jack, Note that MAC_INTERPOSE_FUNCTIONS is always defined in interception.h to either 0 or 1. I'm going to keep #if MAC_INTERPOSE_FUNCTIONS (adding !defined(MISSING_BLOCKS_SUPPORT) where appropriate) in libsanitizer. On Mon, Dec 3, 2012 at 11:17 AM, Mike Stump mikest...@comcast.net wrote: On Dec 3, 2012, at 8:02 AM, Jack Howarth howa...@bromo.med.uc.edu wrote: The attached patch eliminates PR 55521/sanitizer by switching libasan on darwin from using mach_override to mac function interposition So, I'm thinking the sanitizer people will just review and approve it, even though it says darwin and is heavily darwin specific… It's ok by me. -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCh][Revisedx2] Fix PR55521 by switching libsanitizer from mach_overrideto mac interpose functions on darwin
I've added MISSING_BLOCKS_SUPPORT to LLVM compiler-rt in r169206. The rest of your change looks good to me as well. On Mon, Dec 3, 2012 at 6:33 PM, Alexander Potapenko gli...@google.com wrote: Jack, Note that MAC_INTERPOSE_FUNCTIONS is always defined in interception.h to either 0 or 1. I'm going to keep #if MAC_INTERPOSE_FUNCTIONS (adding !defined(MISSING_BLOCKS_SUPPORT) where appropriate) in libsanitizer. On Mon, Dec 3, 2012 at 11:17 AM, Mike Stump mikest...@comcast.net wrote: On Dec 3, 2012, at 8:02 AM, Jack Howarth howa...@bromo.med.uc.edu wrote: The attached patch eliminates PR 55521/sanitizer by switching libasan on darwin from using mach_override to mac function interposition So, I'm thinking the sanitizer people will just review and approve it, even though it says darwin and is heavily darwin specific… It's ok by me. -- Alexander Potapenko Software Engineer Google Moscow -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCH] Fix PR55521 by switching libsanitizer from mach_override to mac interpose functions on darwin
On Sun, Dec 2, 2012 at 10:15 PM, Jack Howarth howa...@bromo.med.uc.edu wrote:
On Sun, Dec 02, 2012 at 10:21:02AM +0400, Alexander Potapenko wrote:
Hi Jack,
IIUC the wrappers for dispatch_async_f, dispatch_sync_f and other
dispatch_smth_f do not need blocks support in the compiler, since
regular functions are passed into them. So you may want to add the
dynamic interceptors for those back.
Alex,
This seems to only require the readjustment of the preprocessor
statements in libsanitizer/asan/dynamic/asan_interceptors_dynamic.cc.
I'll post the revised patch as soon as regression testing is finished.
The remaining problem is that dispach_async and other functions using
blocks won't be intercepted. This may lead to assertion failures in
big projects (e.g. we needed those for Chrome).
In theory that is a problem, however my understanding was that the asan
dispatch support was added for use by objc/obj-c++ code. The objc/obj-c++
in FSF gcc is very fragile, on later darwin, since it runs against the
system Objective-C runtime but the compiler is only based on a merge
of the remaining bits of code from the old FSF gcc Apple branch circa
2006. IMHO, it is insane to try to do any production work with our
objc/obj-c++ compiler and current Mac OS X releases.
Ok, this makes sense.
Overall, the change looks good. Do you want me to backport
MISSING_BLOCKS_SUPPORT into the LLVM version of the runtime?
Yes, as it should be transparent to llvm's builds.
I'll do that tomorrow morning then.
Alex
On Sun, Dec 2, 2012 at 6:43 AM, Jack Howarth howa...@bromo.med.uc.edu
wrote:
The attached patch eliminates PR 55521/sanitizer by switching libasan
on darwin
from using mach_override to mac function interposition via the importation
of the
asan/dynamic/asan_interceptors_dynamic.cc file from llvm.org's compiler-rt
svn.
The changes involve defining USING_MAC_INTERPOSE in configure.ac rather
than
rather than USING_MACH_OVERRIDE, introduction of the use of
USING_MAC_INTERPOSE
in Makefile.am to avoid building the interception subdirectory, the
passage of
-DMAC_INTERPOSE_FUNCTIONS in asan/Makefile.am when USING_MAC_INTERPOSE as
well as
the introduction of a -DMISSING_BLOCKS_SUPPORT flag to disable code that
requires
blocks support which FSF gcc lacks. The depreciated usage of
USING_MACH_OVERRIDE
is also removed from interception/Makefile.am. Bootstrapped on
x86_64-apple-darwin10,
x86_64-apple-darwin11 and x86_64-apple-darwin12. Passes...
make -k check RUNTESTFLAGS=asan.exp --target_board=unix'{-m32,-m64}'
and fixes the previously failing cond1.C test case from PR55521 on all
three targets.
Okay for gcc trunk?
Jack
--
Alexander Potapenko
Software Engineer
Google Moscow
--
Alexander Potapenko
Software Engineer
Google Moscow
Re: [PATCH] Fix PR55521 by switching libsanitizer from mach_override to mac interpose functions on darwin
Hi Jack,
IIUC the wrappers for dispatch_async_f, dispatch_sync_f and other
dispatch_smth_f do not need blocks support in the compiler, since
regular functions are passed into them. So you may want to add the
dynamic interceptors for those back.
The remaining problem is that dispach_async and other functions using
blocks won't be intercepted. This may lead to assertion failures in
big projects (e.g. we needed those for Chrome).
Overall, the change looks good. Do you want me to backport
MISSING_BLOCKS_SUPPORT into the LLVM version of the runtime?
Alex
On Sun, Dec 2, 2012 at 6:43 AM, Jack Howarth howa...@bromo.med.uc.edu wrote:
The attached patch eliminates PR 55521/sanitizer by switching libasan on
darwin
from using mach_override to mac function interposition via the importation of
the
asan/dynamic/asan_interceptors_dynamic.cc file from llvm.org's compiler-rt
svn.
The changes involve defining USING_MAC_INTERPOSE in configure.ac rather than
rather than USING_MACH_OVERRIDE, introduction of the use of
USING_MAC_INTERPOSE
in Makefile.am to avoid building the interception subdirectory, the passage of
-DMAC_INTERPOSE_FUNCTIONS in asan/Makefile.am when USING_MAC_INTERPOSE as
well as
the introduction of a -DMISSING_BLOCKS_SUPPORT flag to disable code that
requires
blocks support which FSF gcc lacks. The depreciated usage of
USING_MACH_OVERRIDE
is also removed from interception/Makefile.am. Bootstrapped on
x86_64-apple-darwin10,
x86_64-apple-darwin11 and x86_64-apple-darwin12. Passes...
make -k check RUNTESTFLAGS=asan.exp --target_board=unix'{-m32,-m64}'
and fixes the previously failing cond1.C test case from PR55521 on all three
targets.
Okay for gcc trunk?
Jack
--
Alexander Potapenko
Software Engineer
Google Moscow
Re: [PATCH][Revisedx5] Enable libsanitizer on darwin
The mach_override path looks good to me. I don't have enough knowledge
of GCC buildsystem yet to review the rest.
On Fri, Nov 23, 2012 at 8:17 PM, Jack Howarth howa...@bromo.med.uc.edu wrote:
The attached patch imports the missing mach_override/mach_override.h and
mach_override/mach_override.c files from llvm.org's compiler-rt at
r168032 | glider | 2012-11-15 03:32:16 -0500 (Thu, 15 Nov 2012) | 3 lines
[ASan] Add the lea $imm(%rip),%rax instruction to mach_override.c
The need for this has been reported by Jack Howarth
(howa...@bromo.med.uc.edu) who's porting ASan-Darwin to GCC
The patch adds darwin to the supported target list in configure.tgt and
defines USING_MACH_OVERRIDE for darwin in configure.ac. The definition of
USING_MACH_OVERRIDE is used in Makefile.am as the test for appending
mach_override/mach_override.c to libinterception_la_SOURCES.
LINK_COMMAND_SPEC_A
in gcc/config/darwin.h is modified to add an entry to handle fsanitize=address
so that the required linkages are used for libasan. The static linkage of
libasan.a
in LINK_COMMAND_SPEC_A is handle separately for -static-libstdc++ (which
requires
libstdc++.a) and the -static, -static-gcc and -static-gfortran cases. Tested
at rr193756
on x86_64-apple-darwin12 for both -m32 and -m64 with the both
use-after-free.c testcase
and...
make -k check RUNTESTFLAGS=asan.exp --target_board=unix'{-m32,-m64}'
without regressions.
Jack
ps This version adds the requested importation of the LICENSE.txt file from
llvm.org's
compiler-rt in interception/mach_override. It also sets TSAN_SUPPORTED=no in
configure.tgt
for the 'x86_64-*-darwin* | i?86-*-darwin*' case to avoid building tsan
support.
--
Alexander Potapenko
Software Engineer
Google Moscow
Re: Sparc ASAN
While trying to add support for ARM (AArch32 GNU / Linux) implementation for GCC after-hours but still keep seeing failures on my chromebook running an ubuntu fs on a ChrOS kernel, because the shadow memory apparently overlaps with normal memory. Has anyone else hit this while porting ? I've heard someone complain about this recently. glider? Running asan on *huge* binaries in 32-bit address space is a challenge indeed. My suggestion was to link with -pie. --kcc Yes, there were complaints about that recently: http://crbug.com/159816 -pie MAY help, but not if the program occupies too much address space.
Re: [PATCH 00/13] Request to merge Address Sanitizer in
Also, Alexander Potapenko is the best person to ask about asan-darwin. here. Maybe we can add him to the list of sanitizer maintainers? Seconded. At least for libsanitier/Darwin. Cheers. I can take this, but I'll be busy within the several upcoming days (till mid-next-week), so I won't be able to merge any patches from LLVM into GCC. Also, I'm not a GCC committer yet, and I guess someone will need to nominate me.
Re: [PATCH][Revised] Enable libsanitizer on darwin
Hi Jack,
most certainly the functionality of asan is not intact.
The error messages denote that mach_override couldn't parse some of
the function prologues, which means some of ASan interceptors just
won't work.
In order to fix this you need to change the DEBUG definition in
mach_override.c, look at the bytes being parsed and fix the
instruction table in mach_override.c
Please also send a patch to LLVM containing the fix (sending the patch
to the original mach_override repo makes little sense, because we've
forked it long time ago).
HTH,
Alex
On Wed, Nov 14, 2012 at 6:43 PM, Jack Howarth howa...@bromo.med.uc.edu wrote:
The attached patch assumes that mach_override/mach_override.h
and mach_override/mach_override.c has been imported by the libsanitizer
maintainers for use by darwin. The patch adds darwin to the supported
target list in configure.tgt and defines USING_MACH_OVERRIDE for darwin
in configure.ac. The definition of USING_MACH_OVERRIDE is used in
Makefile.am as the test for appending mach_override/mach_override.c
to libinterception_la_SOURCES. Tested on x86_64-apple-darwin12 against
the mach_override/mach_override.h and mach_override/mach_override.c
from llvm compiler-rt 3.2 branch. While there is some noise on the
output of asan...
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289#c14
the functionality of asan appears to be intact. Okay for gcc trunk
after the libsanitizer maintainers import the missing
mach_override/mach_override.h
and mach_override/mach_override.c files?
Jack
ps Note that this patch assumes that both mach_override.h and mach_override.c
reside in a mach_override subdirectory in interception as is the case in the
llvm's compiler-rt.
pps Patch to configure.tgt revised to use a distinct instance for darwin in
the case statement and to limit libsanitizer to i?86 and x86_64 on darwin.
libsanitizer/
2012-11-14 Jack Howarth howa...@bromo.med.uc.edu
* configure.tgt: Add darwin to supported targets.
* configure.ac: Define USING_MACH_OVERRIDE when on darwin.
* interception/Makefile.am: Compile mach_override.c when
USING_MACH_OVERRIDE defined.
* configure: Regenerated.
* interception/Makefile.in: Likewise.
Index: libsanitizer/interception/Makefile.am
===
--- libsanitizer/interception/Makefile.am (revision 193500)
+++ libsanitizer/interception/Makefile.am (working copy)
@@ -11,7 +11,11 @@ interception_files = \
interception_mac.cc \
interception_win.cc
-libinterception_la_SOURCES = $(interception_files)
+if USING_MACH_OVERRIDE
+libinterception_la_SOURCES = $(interception_files)
mach_override/mach_override.c
+else
+libinterception_la_SOURCES = $(interception_files)
+endif
# Work around what appears to be a GNU make bug handling MAKEFLAGS
# values defined in terms of make variables, as is the case for CC and
Index: libsanitizer/configure.ac
===
--- libsanitizer/configure.ac (revision 193500)
+++ libsanitizer/configure.ac (working copy)
@@ -17,6 +17,12 @@ AM_PROG_LIBTOOL
AC_SUBST(enable_shared)
AC_SUBST(enable_static)
+case $host in
+ *-*-darwin*) MACH_OVERRIDE=true ;;
+ *) MACH_OVERRIDE=false ;;
+esac
+AM_CONDITIONAL(USING_MACH_OVERRIDE, $MACH_OVERRIDE)
+
#AM_ENABLE_MULTILIB(, ..)
target_alias=${target_alias-$host_alias}
AC_SUBST(target_alias)
Index: libsanitizer/configure.tgt
===
--- libsanitizer/configure.tgt (revision 193500)
+++ libsanitizer/configure.tgt (working copy)
@@ -22,6 +22,8 @@
case ${target} in
x86_64-*-linux* | i?86-*-linux*)
;;
+ x86_64-*-darwin* | i?86-*-darwin*)
+ ;;
*)
UNSUPPORTED=1
;;
--
Alexander Potapenko
Software Engineer
Google Moscow
Re: [PATCH][Revised] Enable libsanitizer on darwin
Hi Rainer, The quick answer is no, although the expansion into GCC world may require such a guideline. We've initially implemented ASan on Linux and then ported it to Android (which is very similar to Linux) and Mac (which is very different), so we have little experience with porting yet. I've summarized the differences between Linux and Mac here: https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForMacImplementationDetails The core things to pay attention at are function wrapping (e.g. dlsym on Linux, mach_override on Mac), hooking the allocations/deallocations in the program, stack unwinding (already done for x86 and ARM), thread creation, early initialization, debug info and symbolization. Maybe something else. In fact if any of these points work on your platform differently than they do on Linux/Mac, you'll have to re-implement those. Regarding mach_override, it's a Mac OS-specific thing that we use because dlsym doesn't reliably override functions (keywords: two-level namespace) This only works on x86/x86_64 and is just a bunch of ad-hoc hacks to hot-patch the library code. Extending the instruction table in mach_override.c is irrelevant to porting, it's just a mean of making it work with some newer libraries provided by Apple. It won't work on any other OS (well, without some refactoring; in fact it's possible to use it on Linux) or any other arch (one could rewrite mach_override.c for ARM, but iOS doesn't allow code patching). I'm working on a replacement for mach_override, which will compile libasan as a dynamic library on Mac OS and preload it before running the program. It's not ready yet, so the easiest thing for GCC will be to add some more instructions to mach_override and stick to it for now. Alex On Wed, Nov 14, 2012 at 7:08 PM, Rainer Orth r...@cebitec.uni-bielefeld.de wrote: Hi Alex, most certainly the functionality of asan is not intact. The error messages denote that mach_override couldn't parse some of the function prologues, which means some of ASan interceptors just won't work. In order to fix this you need to change the DEBUG definition in mach_override.c, look at the bytes being parsed and fix the instruction table in mach_override.c is there some guideline how to port asan to a new OS or CPU? That would certainly be easier than figuring things out on your own one by one. I guess several target and os port maintainers would want to do so in GCC. Thanks. Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCH][Revised] Enable libsanitizer on darwin
On Wed, Nov 14, 2012 at 8:09 PM, Rainer Orth r...@cebitec.uni-bielefeld.de wrote: Hi Alex, thanks, that's certainly helpful. I'm primarily interested in porting to Solaris, both SPARC and x86. Several things should be similar to Linux (both being ELF systems), while other areas are certainly different (syscalls implementation etc.). We don't wrap the syscalls in ASan, if you're speaking about that. I'll certainly be looking around. One problem I see with the current code organization is duplication between different platforms: e.g. considerable parts of sanitizer_mac.cc and a new sanitizer_solaris.cc are identical. Perhaps we need more granularity in the future, but that can be decided once an initial port (with duplication to avoid treading on other port's feet) is done. At the same time, we should add infrastructure (like libffi) to only compile target-specific code on the relevant platforms, instead of wrapping every source file in #ifdef __linux__ or similar and extending the conditionals once a new platform is added. Yes, this might be a problem. I also wonder how much Solaris is like FreeBSD, which users are also interested in ASan. -- Alexander Potapenko Software Engineer Google Moscow
Re: [PATCH][Revised] Enable libsanitizer on darwin
I've responded to the bug. Sorry for offtopics unrelated to your original patch. On Wed, Nov 14, 2012 at 8:11 PM, Jack Howarth howa...@bromo.med.uc.edu wrote: On Wed, Nov 14, 2012 at 07:56:18PM +0400, Alexander Potapenko wrote: Hi Rainer, The quick answer is no, although the expansion into GCC world may require such a guideline. We've initially implemented ASan on Linux and then ported it to Android (which is very similar to Linux) and Mac (which is very different), so we have little experience with porting yet. I've summarized the differences between Linux and Mac here: https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForMacImplementationDetails The core things to pay attention at are function wrapping (e.g. dlsym on Linux, mach_override on Mac), hooking the allocations/deallocations in the program, stack unwinding (already done for x86 and ARM), thread creation, early initialization, debug info and symbolization. Maybe something else. In fact if any of these points work on your platform differently than they do on Linux/Mac, you'll have to re-implement those. Regarding mach_override, it's a Mac OS-specific thing that we use because dlsym doesn't reliably override functions (keywords: two-level namespace) This only works on x86/x86_64 and is just a bunch of ad-hoc hacks to hot-patch the library code. Extending the instruction table in mach_override.c is irrelevant to porting, it's just a mean of making it work with some newer libraries provided by Apple. It won't work on any other OS (well, without some refactoring; in fact it's possible to use it on Linux) or any other arch (one could rewrite mach_override.c for ARM, but iOS doesn't allow code patching). I'm working on a replacement for mach_override, which will compile libasan as a dynamic library on Mac OS and preload it before running the program. It's not ready yet, so the easiest thing for GCC will be to add some more instructions to mach_override and stick to it for now. Alex, Please see... http://gcc.gnu.org/bugzilla/show_bug.cgi?id=55289#c27 So far for all the test cases that I have looked at, the problem seems to be when mach_overrride.c hits the opcodes... 48 8d 5 Can you suggest a patch for that one? It may be the only blocker to asan support on darwin (for x86_64 anyway). Jack Alex On Wed, Nov 14, 2012 at 7:08 PM, Rainer Orth r...@cebitec.uni-bielefeld.de wrote: Hi Alex, most certainly the functionality of asan is not intact. The error messages denote that mach_override couldn't parse some of the function prologues, which means some of ASan interceptors just won't work. In order to fix this you need to change the DEBUG definition in mach_override.c, look at the bytes being parsed and fix the instruction table in mach_override.c is there some guideline how to port asan to a new OS or CPU? That would certainly be easier than figuring things out on your own one by one. I guess several target and os port maintainers would want to do so in GCC. Thanks. Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University -- Alexander Potapenko Software Engineer Google Moscow -- Alexander Potapenko Software Engineer Google Moscow
