Re: build issue: _REENTRANT redefined
On 11.09.2019 18:51, Kamil Rytarowski wrote:
> I've fixes few LSan issues and pushed patches upstream.
>
> I've precompiled and uploaded the patched version of the toolchain here:
>
> http://cdn.netbsd.org/pub/NetBSD/misc/kamil/llvm-clang-compilerrt-10.0.0beta_2019-09-11.tar.bz2
>
These files are intended to be extracted into /usr/local (such as
/usr/loca/bin/clang).
> $ cat leak.c
>
>
> #include
> void *p;
> int main() {
> p = malloc(7);
> p = 0; // The memory is leaked here.
> return 0;
> }
>
> $ clang -fsanitize=address -g leak.c ; ASAN_OPTIONS=detect_leaks=1
> ./a.out
>
> =
> ==19817==ERROR: LeakSanitizer: detected memory leaks
>
> Direct leak of 7 byte(s) in 1 object(s) allocated from:
> #0 0x4c15dd in malloc
> /public/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
> #1 0x41eb68 in main /tmp/leak.c:4:7
> #2 0x41ea7c in ___start (/tmp/./a.out+0x41ea7c)
>
> SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
>
>
> I recommend to wait for llvm10 to get these patches properly reviewed
> and merged instead of backporting to llvm7 in NetBSD-8.
>
> Not all LSan tests pass (there are at least 3 breaking scenarios), but
> it is already usable. Certain bugs will be omitted, others might falsely
> report problems.
>
signature.asc
Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
I've fixes few LSan issues and pushed patches upstream.
I've precompiled and uploaded the patched version of the toolchain here:
http://cdn.netbsd.org/pub/NetBSD/misc/kamil/llvm-clang-compilerrt-10.0.0beta_2019-09-11.tar.bz2
$ cat leak.c
#include
void *p;
int main() {
p = malloc(7);
p = 0; // The memory is leaked here.
return 0;
}
$ clang -fsanitize=address -g leak.c ; ASAN_OPTIONS=detect_leaks=1
./a.out
=
==19817==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 7 byte(s) in 1 object(s) allocated from:
#0 0x4c15dd in malloc
/public/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3
#1 0x41eb68 in main /tmp/leak.c:4:7
#2 0x41ea7c in ___start (/tmp/./a.out+0x41ea7c)
SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
I recommend to wait for llvm10 to get these patches properly reviewed
and merged instead of backporting to llvm7 in NetBSD-8.
Not all LSan tests pass (there are at least 3 breaking scenarios), but
it is already usable. Certain bugs will be omitted, others might falsely
report problems.
signature.asc
Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On 07.09.2019 01:08, Kamil Rytarowski wrote: > I could backport LSan/LLVM for NetBSD-9 if there would be a request. > However before that I would prefer to address the mentioned > false-positive from the atexit(3) callback. I have originally > rescheduled it for NetBSD-10. I spent some time on LSan and I have managed to fix the mentioned false-positive and fix few other problems. There are still like 2-3 problems left. There was also an issue.. reported in libc. I will investigate whether it is a false positive or not. = ==4698==ERROR: LeakSanitizer: detected memory leaks Direct leak of 2048 byte(s) in 1 object(s) allocated from: #0 0x4b7e4d in malloc /public/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cpp:145:3 #1 0x7f7ff633d9ad in strerror_l /usr/src/lib/libc/string/strerror.c:81 SUMMARY: AddressSanitizer: 2048 byte(s) leaked in 1 allocation(s). I will try to squash it and push to upstream review. I can request to pullup these changes as fixups for llvm 9.0.1 maybe. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On 07.09.2019 01:07, Kamil Rytarowski wrote: > On 07.09.2019 00:41, Thomas Klausner wrote: >> On Sat, Sep 07, 2019 at 12:36:49AM +0200, Kamil Rytarowski wrote: >>> Sanitizing compiler is available without MKSANITIZER. >> >> I tried (on 9.99.10 from Aug 26): >> >> wiz@yt:~> clang -fsanitize=address -g memory-leak.c >> wiz@yt:~> ASAN_OPTIONS=detect_leaks=1 ./a.out >> ==16509==AddressSanitizer: detect_leaks is not supported on this platform. >> > > The version of Clang/LLVM in base as of today (clang 7.x, compiler-rt > 8.x) does not support LSan/NetBSD. > > LSan/NetBSD is supported in Clang/LLVM 9.0.0. > > Michal Gorny works on porting LLVM 9.0.0 to the base in this moment. > > As an intermediate step we have updated to LLVM 8.x and NetBSD-current > from HEAD. > > https://github.com/mgorny/netbsd-src/tree/llvm8 > > Next step is to update to Clang/LLVM 9.0.0 prerelease snapshot, the > final release of 9.0.0 is still not formally out. > > NB. LSan in LLVM 9.0.0 will have one flaw on NetBSD. It will report > false positives in strict detection of leaks on process termination. > This functionality is called from an atexit(3) callback, and the > callback is fired from libc and before freeing memory of it. > > There is need to invent a solution to this false-positive. > >>> libutil is built with a sanitizer. >>> >>> This means that the code in libutil is instrumented and contains calls >>> to sanitizer runtime. >>> >>> The runtime is linked into programs, not libraries. >>> >>> This means that there is need to build each program in this setup with a >>> sanitizer. >> >> Ok, thanks for the explanation. >> > > There are some alternative approaches, but I don't want to make things more complex here. > >>> Every application that dynamically links with an instrumented library >>> has to be prebuilt with a sanitizer. >>> Do I have to recompile all of pkgsrc, and if yes, why? >>> >>> Basically yes, there is need to recompile all of pkgsrc. >> >> Do I need to use any particular settings or will this automatically >> work in such a userland? >> > > If we want to combine pkgsrc with a MKSANITIZER host, we want to reuse > the same CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS as in basesystem. > > This is by default: > > CPPFLAGS=-D_REENTRANT > CFLAGS=-fsanitize=address > CXXFLAGS=-fsanitize=address > LDFLAGS=-fsanitize=address > >>> However ASan and UBSan can be often used on the top of non-sanitized >>> libraries and this will work decently finding problems in an >>> application, but skipping issues called in libraries. >>> >>> MSan and TSan are more sensitive here and need full sanitization. >> >> I'm currently mostly interested in the leak sanitizer. >> But perhaps that's not available yet, or at least not in 9.99.10? >> Thomas >> > > I could backport LSan/LLVM for NetBSD-9 if there would be a request. > However before that I would prefer to address the mentioned > false-positive from the atexit(3) callback. I have originally > rescheduled it for NetBSD-10. > > For LSan there is no need for MKSANITIZER as all free/malloc/similar > calls are intercepted always regardless of instrumenting the code. > > It's also possible to use -fsanitize=leak, without -fsanitize=address. > It's actually a better idea here if we want to catch only leaks as > ASan/NetBSD is still not perfect. > signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On 07.09.2019 00:41, Thomas Klausner wrote: > On Sat, Sep 07, 2019 at 12:36:49AM +0200, Kamil Rytarowski wrote: >> Sanitizing compiler is available without MKSANITIZER. > > I tried (on 9.99.10 from Aug 26): > > wiz@yt:~> clang -fsanitize=address -g memory-leak.c > wiz@yt:~> ASAN_OPTIONS=detect_leaks=1 ./a.out > ==16509==AddressSanitizer: detect_leaks is not supported on this platform. > The version of Clang/LLVM in base as of today (clang 7.x, compiler-rt 8.x) does not support LSan/NetBSD. LSan/NetBSD is supported in Clang/LLVM 9.0.0. Michal Gorny works on porting LLVM 9.0.0 to the base in this moment. As an intermediate step we have updated to LLVM 8.x and NetBSD-current from HEAD. https://github.com/mgorny/netbsd-src/tree/llvm8 Next step is to update to Clang/LLVM 9.0.0 prerelease snapshot, the final release of 9.0.0 is still not formally out. NB. LSan in LLVM 9.0.0 will have one flaw on NetBSD. It will report false positives in strict detection of leaks on process termination. This functionality is called from an atexit(3) callback, and the callback is fired from libc and before freeing memory of it. There is need to invent a solution to this false-positive. >> libutil is built with a sanitizer. >> >> This means that the code in libutil is instrumented and contains calls >> to sanitizer runtime. >> >> The runtime is linked into programs, not libraries. >> >> This means that there is need to build each program in this setup with a >> sanitizer. > > Ok, thanks for the explanation. > There are some alternative approaches, but I don't want to make >> Every application that dynamically links with an instrumented library >> has to be prebuilt with a sanitizer. >> >>> Do I have to recompile all of pkgsrc, and if yes, why? >> >> Basically yes, there is need to recompile all of pkgsrc. > > Do I need to use any particular settings or will this automatically > work in such a userland? > If we want to combine pkgsrc with a MKSANITIZER host, we want to reuse the same CPPFLAGS, CFLAGS, CXXFLAGS and LDFLAGS as in basesystem. This is by default: CPPFLAGS=-D_REENTRANT CFLAGS=-fsanitize=address CXXFLAGS=-fsanitize=address LDFLAGS=-fsanitize=address >> However ASan and UBSan can be often used on the top of non-sanitized >> libraries and this will work decently finding problems in an >> application, but skipping issues called in libraries. >> >> MSan and TSan are more sensitive here and need full sanitization. > > I'm currently mostly interested in the leak sanitizer. > But perhaps that's not available yet, or at least not in 9.99.10? > Thomas > I could backport LSan/LLVM for NetBSD-9 if there would be a request. However before that I would prefer to address the mentioned false-positive from the atexit(3) callback. I have originally rescheduled it for NetBSD-10. For LSan there is no need for MKSANITIZER as all free/malloc/similar calls are intercepted always regardless of instrumenting the code. It's also possible to use -fsanitize=leak, without -fsanitize=address. It's actually a better idea here if we want to catch only leaks as ASan/NetBSD is still not perfect. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On Sat, Sep 07, 2019 at 12:36:49AM +0200, Kamil Rytarowski wrote: > Sanitizing compiler is available without MKSANITIZER. I tried (on 9.99.10 from Aug 26): wiz@yt:~> clang -fsanitize=address -g memory-leak.c wiz@yt:~> ASAN_OPTIONS=detect_leaks=1 ./a.out ==16509==AddressSanitizer: detect_leaks is not supported on this platform. > libutil is built with a sanitizer. > > This means that the code in libutil is instrumented and contains calls > to sanitizer runtime. > > The runtime is linked into programs, not libraries. > > This means that there is need to build each program in this setup with a > sanitizer. Ok, thanks for the explanation. > Every application that dynamically links with an instrumented library > has to be prebuilt with a sanitizer. > > > Do I have to recompile all of pkgsrc, and if yes, why? > > Basically yes, there is need to recompile all of pkgsrc. Do I need to use any particular settings or will this automatically work in such a userland? > However ASan and UBSan can be often used on the top of non-sanitized > libraries and this will work decently finding problems in an > application, but skipping issues called in libraries. > > MSan and TSan are more sensitive here and need full sanitization. I'm currently mostly interested in the leak sanitizer. But perhaps that's not available yet, or at least not in 9.99.10? Thomas
Re: build issue: _REENTRANT redefined
On 07.09.2019 00:14, Thomas Klausner wrote: > On Fri, Sep 06, 2019 at 09:19:49PM +0200, Kamil Rytarowski wrote: >> On 06.09.2019 21:02, Kamil Rytarowski wrote: >>> On 06.09.2019 20:57, Thomas Klausner wrote: On Fri, Sep 06, 2019 at 08:01:13PM +0200, Kamil Rytarowski wrote: > On 06.09.2019 19:46, Thomas Klausner wrote: >> On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: >>> The only supported combination is: >>> >>> HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes >> >> So I tried the suggested combination, and it didn't work for me either. >> >> --- dependall-libLLVMAsmParser --- >> In file included from >> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: >> In file included from >> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: >> In file included from >> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: >> In file included from >> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: >> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: >> fatal error: 'sanitizer/asan_interface.h' file not found >> # include >> ^~~~ >> >> build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V >> MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m >> amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R >> /usr/obj/amd64.clang.20190906.release distribution >> >> Thomas >> > > Was this a clean build? Yes, this happens with an empty DESTDIR, and 'cvs update' was not complaining about any files lying around. Thomas >>> >>> I have reproduced it. I'm going to investigate it and fix. >>> >>> It looks like for some reason clang sanitizer files are not copied into >>> tooldir. >>> >> >> OK, I know what is wrong. MKLLVMRT / X is enabled and this sequence of >> build wasn't tested and needs correction. >> >> For the time being, I recommend to remove the "-x" option from build.sh >> as there are bigger tasks to do at this point before running sanitized X. > > Well, I don't need a sanitized X, I just wanted a sanitizing compiler. Sanitizing compiler is available without MKSANITIZER. > Anyway, I followed your suggestion and updated my system. > > I still had programs installed (from the 9.99.10 without MKSANITIZER). > Running them I see stuff like this: > > # hg up -C TNF > /usr/lib/libutil.so.7: Undefined symbol > "__asan_option_detect_stack_use_after_return" (symnum = 94) > # ls -l /usr/lib/libutil* > -r--r--r-- 1 root wheel 222060 Sep 6 22:03 /usr/lib/libutil.a > lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so -> > ../../lib/libutil.so.7.24 > lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so.7 -> > ../../lib/libutil.so.7.24 > lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so.7.24 -> > ../../lib/libutil.so.7.24 > -r--r--r-- 1 root wheel 518486 Sep 6 22:03 /usr/lib/libutil_g.a > -r--r--r-- 1 root wheel 230302 Sep 6 22:03 /usr/lib/libutil_p.a > # ls -l /lib/libutil* > lrwxr-xr-x 1 root wheel 15 Sep 6 22:03 /lib/libutil.so -> > libutil.so.7.24 > lrwxr-xr-x 1 root wheel 15 Sep 6 22:03 /lib/libutil.so.7 -> > libutil.so.7.24 > -r--r--r-- 1 root wheel 415608 Sep 6 22:03 /lib/libutil.so.7.24 > # ldd /lib/libutil.so > /lib/libutil.so: > -lc.12 => /lib/libc.so.12 > # ls -l /lib/libc.so.12* > lrwxr-xr-x 1 root wheel 14 Sep 6 21:59 /lib/libc.so.12 -> > libc.so.12.213 > -r--r--r-- 1 root wheel 2178936 Sep 6 21:59 /lib/libc.so.12.213 > > Or > /usr/lib/libz.so.1: Undefined symbol > "__asan_option_detect_stack_use_after_return" (symnum = 42) > > What is the problem here? libutil is built with a sanitizer. This means that the code in libutil is instrumented and contains calls to sanitizer runtime. The runtime is linked into programs, not libraries. This means that there is need to build each program in this setup with a sanitizer. Every application that dynamically links with an instrumented library has to be prebuilt with a sanitizer. > Do I have to recompile all of pkgsrc, and if yes, why? Basically yes, there is need to recompile all of pkgsrc. There could be probably some work arounds with LD_PRELOAD and a dummy sanitizer runtime.. however certainly better to invest into sanitized pkgsrc build configuration. My workaround for breaking applications is to build programs statically or use the version from /rescue. However ASan and UBSan can be often used on the top of non-sanitized libraries and this will work decently finding problems in an application, but skipping issues called in libraries. MS
Re: build issue: _REENTRANT redefined
On Fri, Sep 06, 2019 at 09:19:49PM +0200, Kamil Rytarowski wrote: > On 06.09.2019 21:02, Kamil Rytarowski wrote: > > On 06.09.2019 20:57, Thomas Klausner wrote: > >> On Fri, Sep 06, 2019 at 08:01:13PM +0200, Kamil Rytarowski wrote: > >>> On 06.09.2019 19:46, Thomas Klausner wrote: > On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: > > The only supported combination is: > > > > HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes > > So I tried the suggested combination, and it didn't work for me either. > > --- dependall-libLLVMAsmParser --- > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: > fatal error: 'sanitizer/asan_interface.h' file not found > # include > ^~~~ > > build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V > MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m > amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R > /usr/obj/amd64.clang.20190906.release distribution > > Thomas > > >>> > >>> Was this a clean build? > >> > >> Yes, this happens with an empty DESTDIR, and 'cvs update' was not > >> complaining about any files lying around. > >> Thomas > >> > > > > I have reproduced it. I'm going to investigate it and fix. > > > > It looks like for some reason clang sanitizer files are not copied into > > tooldir. > > > > OK, I know what is wrong. MKLLVMRT / X is enabled and this sequence of > build wasn't tested and needs correction. > > For the time being, I recommend to remove the "-x" option from build.sh > as there are bigger tasks to do at this point before running sanitized X. Well, I don't need a sanitized X, I just wanted a sanitizing compiler. Anyway, I followed your suggestion and updated my system. I still had programs installed (from the 9.99.10 without MKSANITIZER). Running them I see stuff like this: # hg up -C TNF /usr/lib/libutil.so.7: Undefined symbol "__asan_option_detect_stack_use_after_return" (symnum = 94) # ls -l /usr/lib/libutil* -r--r--r-- 1 root wheel 222060 Sep 6 22:03 /usr/lib/libutil.a lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so -> ../../lib/libutil.so.7.24 lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so.7 -> ../../lib/libutil.so.7.24 lrwxr-xr-x 1 root wheel 25 Sep 6 22:03 /usr/lib/libutil.so.7.24 -> ../../lib/libutil.so.7.24 -r--r--r-- 1 root wheel 518486 Sep 6 22:03 /usr/lib/libutil_g.a -r--r--r-- 1 root wheel 230302 Sep 6 22:03 /usr/lib/libutil_p.a # ls -l /lib/libutil* lrwxr-xr-x 1 root wheel 15 Sep 6 22:03 /lib/libutil.so -> libutil.so.7.24 lrwxr-xr-x 1 root wheel 15 Sep 6 22:03 /lib/libutil.so.7 -> libutil.so.7.24 -r--r--r-- 1 root wheel 415608 Sep 6 22:03 /lib/libutil.so.7.24 # ldd /lib/libutil.so /lib/libutil.so: -lc.12 => /lib/libc.so.12 # ls -l /lib/libc.so.12* lrwxr-xr-x 1 root wheel 14 Sep 6 21:59 /lib/libc.so.12 -> libc.so.12.213 -r--r--r-- 1 root wheel 2178936 Sep 6 21:59 /lib/libc.so.12.213 Or /usr/lib/libz.so.1: Undefined symbol "__asan_option_detect_stack_use_after_return" (symnum = 42) What is the problem here? Do I have to recompile all of pkgsrc, and if yes, why? Thomas
Re: build issue: _REENTRANT redefined
On 06.09.2019 21:02, Kamil Rytarowski wrote: > On 06.09.2019 20:57, Thomas Klausner wrote: >> On Fri, Sep 06, 2019 at 08:01:13PM +0200, Kamil Rytarowski wrote: >>> On 06.09.2019 19:46, Thomas Klausner wrote: On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: > The only supported combination is: > > HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes So I tried the suggested combination, and it didn't work for me either. --- dependall-libLLVMAsmParser --- In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: fatal error: 'sanitizer/asan_interface.h' file not found # include ^~~~ build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R /usr/obj/amd64.clang.20190906.release distribution Thomas >>> >>> Was this a clean build? >> >> Yes, this happens with an empty DESTDIR, and 'cvs update' was not >> complaining about any files lying around. >> Thomas >> > > I have reproduced it. I'm going to investigate it and fix. > > It looks like for some reason clang sanitizer files are not copied into > tooldir. > OK, I know what is wrong. MKLLVMRT / X is enabled and this sequence of build wasn't tested and needs correction. For the time being, I recommend to remove the "-x" option from build.sh as there are bigger tasks to do at this point before running sanitized X. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On 06.09.2019 20:57, Thomas Klausner wrote: > On Fri, Sep 06, 2019 at 08:01:13PM +0200, Kamil Rytarowski wrote: >> On 06.09.2019 19:46, Thomas Klausner wrote: >>> On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: The only supported combination is: HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes >>> >>> So I tried the suggested combination, and it didn't work for me either. >>> >>> --- dependall-libLLVMAsmParser --- >>> In file included from >>> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: >>> In file included from >>> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: >>> In file included from >>> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: >>> In file included from >>> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: >>> /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: >>> fatal error: 'sanitizer/asan_interface.h' file not found >>> # include >>> ^~~~ >>> >>> build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V >>> MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m >>> amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R >>> /usr/obj/amd64.clang.20190906.release distribution >>> >>> Thomas >>> >> >> Was this a clean build? > > Yes, this happens with an empty DESTDIR, and 'cvs update' was not > complaining about any files lying around. > Thomas > I have reproduced it. I'm going to investigate it and fix. It looks like for some reason clang sanitizer files are not copied into tooldir. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On Fri, Sep 06, 2019 at 08:01:13PM +0200, Kamil Rytarowski wrote: > On 06.09.2019 19:46, Thomas Klausner wrote: > > On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: > >> The only supported combination is: > >> > >> HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes > > > > So I tried the suggested combination, and it didn't work for me either. > > > > --- dependall-libLLVMAsmParser --- > > In file included from > > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: > > In file included from > > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: > > In file included from > > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: > > In file included from > > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: > > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: > > fatal error: 'sanitizer/asan_interface.h' file not found > > # include > > ^~~~ > > > > build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V > > MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m > > amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R > > /usr/obj/amd64.clang.20190906.release distribution > > > > Thomas > > > > Was this a clean build? Yes, this happens with an empty DESTDIR, and 'cvs update' was not complaining about any files lying around. Thomas
Re: build issue: _REENTRANT redefined
On 06.09.2019 19:46, Thomas Klausner wrote: > On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: >> The only supported combination is: >> >> HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes > > So I tried the suggested combination, and it didn't work for me either. > > --- dependall-libLLVMAsmParser --- > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: > In file included from > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: > /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: > fatal error: 'sanitizer/asan_interface.h' file not found > # include > ^~~~ > > build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V > MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m amd64 > -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R > /usr/obj/amd64.clang.20190906.release distribution > > Thomas > Was this a clean build? I will try to reproduce these exact commands. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On Fri, Sep 06, 2019 at 03:11:55PM +0200, Kamil Rytarowski wrote: > The only supported combination is: > > HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes So I tried the suggested combination, and it didn't work for me either. --- dependall-libLLVMAsmParser --- In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.cpp:14: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/lib/AsmParser/LLLexer.h:18: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/ADT/APInt.h:19: /usr/src/external/bsd/llvm/librt/libLLVMAsmParser/../../lib/../dist/llvm/include/llvm/Support/Compiler.h:387:11: fatal error: 'sanitizer/asan_interface.h' file not found # include ^~~~ build.sh -j 16 -x -V MKDEBUG=yes -V MKDEBUGLIB=yes -V MKLLVM=yes -V MKSANITIZER=yes -V HAVE_LLVM=yes -V MKGCC=no -T /usr/obj/tools.clang -m amd64 -O /usr/obj/src.amd64 -D /usr/obj/amd64.clang.20190906 -R /usr/obj/amd64.clang.20190906.release distribution Thomas
Re: build issue: _REENTRANT redefined
On 06.09.2019 14:57, Thomas Klausner wrote: > On Fri, Sep 06, 2019 at 01:00:07PM +0100, Roy Marples wrote: >> On 06/09/2019 11:34, Thomas Klausner wrote: >>> I guess I have to turn off the gcc build as well, but for now I'd like >>> to have both compilers... >> >> I've not been able to build both for many years now. >> As my need for building xen packages out-weighs my social want for LLVM, I >> currently only use gcc :( > > That's not my experience. Without turning on MKSANITIZER, my settings > worked a couple weeks ago (and built both gcc and clang). > Thomas > I have not been working on MKSANITIZER with HAVE_LLVM=no. If it works to some extend, it is an accident. My motivation is to minimize the scope of work to a single compiler and patiently wait for GCC-9.x to land the sources that will get almost all of the sanitizer work merged. The only supported combination is: HAVE_LLVM=yes MKSANITIZER=yes MKGCC=no MKLLVM=yes That said, getting MKSANITIZER=yes HAVE_LLVM=no MKGCC=yes MKLLVM=no to work is possible, but I have not been testing it. signature.asc Description: OpenPGP digital signature
Re: build issue: _REENTRANT redefined
On Fri, Sep 06, 2019 at 01:00:07PM +0100, Roy Marples wrote: > On 06/09/2019 11:34, Thomas Klausner wrote: > > I guess I have to turn off the gcc build as well, but for now I'd like > > to have both compilers... > > I've not been able to build both for many years now. > As my need for building xen packages out-weighs my social want for LLVM, I > currently only use gcc :( That's not my experience. Without turning on MKSANITIZER, my settings worked a couple weeks ago (and built both gcc and clang). Thomas
Re: build issue: _REENTRANT redefined
On 06/09/2019 11:34, Thomas Klausner wrote: I guess I have to turn off the gcc build as well, but for now I'd like to have both compilers... I've not been able to build both for many years now. As my need for building xen packages out-weighs my social want for LLVM, I currently only use gcc :( Roy
Re: build issue: _REENTRANT redefined
On Fri, Sep 06, 2019 at 11:51:49AM +0200, Thomas Klausner wrote:
> I tried building NetBSD as of a couple minutes ago as usual with
> MKLLVM=yes but as a new option I added MKSANITIZER=yes as well.
>
> I see lots of:
>
> In file included from
> /usr/src/external/gpl2/lvm2/dist/libdm/misc/dmlib.h:26:0,
> from
> /usr/src/external/gpl2/lvm2/dist/libdm/datastruct/bitset.c:18:
> /usr/src/external/gpl2/lvm2/dist/include/lib.h:26:0: error: "_REENTRANT"
> redefined [-Werror]
> #define _REENTRANT
>
> and also:
>
> cc1: error: unrecognized command line option '-Wno-macro-redefined' [-Werror]
>
> I have MKDEBUG=yes MKDEBUGLIB=yes MKLLVM=yes MKSANITIZER=yes.
Ok, so I tried adding HAVE_LLVM=yes but then I see:
/usr/src/external/gpl3/gcc/dist/libgomp/oacc-init.c:632:21: error: unknown
attribute '__optimize__' ignored [-Werror,-Wunknown-attributes]
int __attribute__ ((__optimize__ ("O2")))
^
/usr/src/external/gpl3/gcc/dist/libgomp/oacc-init.c:635:10: error: use of
unknown builtin '__builtin_acc_on_device' [-Wimplicit-function-declaration]
return __builtin_acc_on_device (dev);
^
/usr/src/external/gpl3/gcc/dist/libgomp/oacc-init.c:635:10: error: this
function declaration is not a prototype [-Werror,-Wstrict-prototypes]
3 errors generated.
I guess I have to turn off the gcc build as well, but for now I'd like
to have both compilers...
Thomas
