Re: [R-sig-Debian] Error building and installing with GCC ASAN in a Docker container
Hi Brent, In case you aren't aware, Winston Chang has some Docker images that build R with ASAN and UBSAN, using gcc and clang: https://github.com/wch/r-debug This doesn't directly address your question, but I hope it might be helpful. Best, Josh On Thu, Feb 15, 2024 at 12:15 PM Brent G Ewing wrote: > > Hi, > > I want to build a Docker container with R and various R packages built and > installed with the gcc address sanitizer enabled. I am working on a Debian > Bookworm system running on an x86_64 architecture. > > I start with the Docker file at > > https://github.com/rocker-org/r-devel-san/blob/master/Dockerfile > > I added -fsanitize=address to compiler and linker variables in the > Dockerfile. I also import into the image a Makevars file with the contents > > CC=gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer > CXX=g++ -fsanitize=address -fno-omit-frame-pointer > FC=gfortran -fsanitize=address > LDFLAGS=-fsanitize=address > > Finally, the Docker file installs additional Debian object libraries and R > packages that I want to test. > > When the package build gets to testing whether the package can be > installed, an error message appears saying > > ==ASan runtime does not come first in initial library list; you should > either link runtime to your application or manually preload it with > LD_PRELOAD. > > For example, > > ESC[0mESC[91m** building package indices > ESC[0mESC[91m** installing vignettes > ESC[0mESC[91m** testing if installed package can be loaded from temporary > location > ESC[0m==684==ASan runtime does not come first in initial library list; you > should either link runtime to your application or manually preload it with > LD_PRELOAD. > ESC[91mERROR: loading failed > ESC[0mESC[91m* removing ‘/usr/local/lib/R/site-library/utf8’ > ESC[0mESC[91m* installing *source* package ‘isoband’ ... > ESC[0mESC[91m** package ‘isoband’ successfully unpacked and MD5 sums checked > ESC[0mESC[91m** using staged installation > ESC[0mESC[91m** libs > > I've tried preloading various libasan.so files that I find on the system > but the loader says that it cannot load them. > > My questions are > > 1. is it possible to build and install R and packages (in a container) with > the gcc address sanitizer enabled? > 2. if it is, how do I do it? > 3. is this ld_preload message relevant? If so, to which file does it refer? > Is there an ASAN runtime library file that I need to specify, and if so, > what is the path to it? > > I appreciate your consideration and assistance. > > Thank you. > Brent > > P.S. The build-related commands in the Docker file (including my address > sanitizer additions) are > > RUN cd /tmp/R-devel \ > && R_PAPERSIZE=letter \ >R_BATCHSAVE="--no-save --no-restore" \ >R_BROWSER=xdg-open \ >PAGER=/usr/bin/pager \ >PERL=/usr/bin/perl \ >R_UNZIPCMD=/usr/bin/unzip \ >R_ZIPCMD=/usr/bin/zip \ >R_PRINTCMD=/usr/bin/lpr \ >LIBnn=lib \ >AWK=/usr/bin/awk \ >CC="gcc -std=gnu99 -fsanitize=address -fsanitize=undefined > -fno-omit-frame-pointer" \ >CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic > -mtune=native" \ >FFLAGS="-g -O2 -mtune=native" \ >FCFLAGS="-g -O2 -mtune=native" \ >CXX="g++ -fsanitize=address -fsanitize=undefined,bounds-strict > -fno-omit-frame-pointer" \ >CXXFLAGS="-fsanitize=address -g -O2 -Wall -pedantic > -mtune=native" \ >CXX11FLAGS="-fsanitize=address -g -O2 -Wall -pedantic > -mtune=native" \ >CXX14FLAGS="-fsanitize=address -g -O2 -Wall -pedantic > -mtune=native" \ >CXX17FLAGS="-fsanitize=address -g -O2 -Wall -pedantic > -mtune=native" \ >LDFLAGS="-fsanitize=address" \ >MAIN_LDFLAGS="-fsanitize=address -fsanitize=undefined -pthread" \ >F77="gfortran -fsanitize=undefined" \ >FC="gfortran -fsanitize=undefined" \ >FCFLAGS="-g -O2 -mtune=native" \ >FFLAGS="-g -O2 -mtune=native" \ >./configure --enable-R-shlib \ >--without-blas \ >--without-lapack \ >--with-readline \ >--without-recommended-packages \ >--program-suffix=dev \ >--disable-openmp \ > && make \ > && make install \ > && make clean > > [[alternative HTML version deleted]] > > ___ > R-SIG-Debian mailing list > R-SIG-Debian@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-debian -- Joshua Ulrich | about.me/joshuaulrich FOSS Trading | www.fosstrading.com ___ R-SIG-Debian mailing list R-SIG-Debian@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-debian
Re: [R-sig-Debian] Error building and installing with GCC ASAN in a Docker container
Hi Brent, On 15 February 2024 at 10:14, Brent G Ewing wrote: | Hi, | | I want to build a Docker container with R and various R packages built and | installed with the gcc address sanitizer enabled. I am working on a Debian | Bookworm system running on an x86_64 architecture. | | I start with the Docker file at | | https://github.com/rocker-org/r-devel-san/blob/master/Dockerfile That container has been building weekly quite reliably for a few years now. The variant at https://github.com/rocker-org/r-devel-san-clang was dormant for a while (first an error during R package build, later a missing package) and was just resurrected and builds again. And then there is of course Winston's 'sumo' container with several builds layerd into one. I use that too: https://github.com/wch/r-debug | I added -fsanitize=address to compiler and linker variables in the | Dockerfile. I also import into the image a Makevars file with the contents | | CC=gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer | CXX=g++ -fsanitize=address -fno-omit-frame-pointer | FC=gfortran -fsanitize=address | LDFLAGS=-fsanitize=address Sure -- but I found this finicky in the past: certain combinations lead to cases that did not build or errored. So I recommend following the set that Brian Ripley has at https://www.stats.ox.ac.uk/pub/bdr/memtests/README.txt | Finally, the Docker file installs additional Debian object libraries and R | packages that I want to test. | | When the package build gets to testing whether the package can be | installed, an error message appears saying | | ==ASan runtime does not come first in initial library list; you should | either link runtime to your application or manually preload it with | LD_PRELOAD. | | For example, | | ESC[0mESC[91m** building package indices | ESC[0mESC[91m** installing vignettes | ESC[0mESC[91m** testing if installed package can be loaded from temporary | location | ESC[0m==684==ASan runtime does not come first in initial library list; you | should either link runtime to your application or manually preload it with | LD_PRELOAD. | ESC[91mERROR: loading failed | ESC[0mESC[91m* removing ‘/usr/local/lib/R/site-library/utf8’ | ESC[0mESC[91m* installing *source* package ‘isoband’ ... | ESC[0mESC[91m** package ‘isoband’ successfully unpacked and MD5 sums checked | ESC[0mESC[91m** using staged installation | ESC[0mESC[91m** libs | | I've tried preloading various libasan.so files that I find on the system | but the loader says that it cannot load them. Maybe try removing 'address' and see what happens? | My questions are | | 1. is it possible to build and install R and packages (in a container) with | the gcc address sanitizer enabled? Yes. The two Docker contains show how to. | 2. if it is, how do I do it? Follow the container recipe? Use the container? Eg I just pulled up the rocker/r-devel-san, giving 'bash' as the command / entry point, and (having mounted my local directory too) said 'RD CMD INSTALL digest_0.6.34.tar.gz'. That install the package under the instrumented R(-devel) version. Tests should now use SAN. I have fixed bugs that way. | 3. is this ld_preload message relevant? If so, to which file does it refer? | Is there an ASAN runtime library file that I need to specify, and if so, | what is the path to it? I think it is relevant but you must have ended up with a non-working combination. I no longer recall why I removed 'address'. It may have been something like this, or something else. `git blame` tells me I did so in 2018. It's been a while. As I said, these things _are_ finicky. So I recommend - starting from a known working container (see above) - establish a baseline by testing a package which has a _known_ true positive - for which I wrote a package: https://cran.r-project.org/package=sanitizers - given the baseline make the mods you want to make and re-test you still get the true positive Hope this helps, Dirk | I appreciate your consideration and assistance. | | Thank you. | Brent | | P.S. The build-related commands in the Docker file (including my address | sanitizer additions) are | | RUN cd /tmp/R-devel \ | && R_PAPERSIZE=letter \ |R_BATCHSAVE="--no-save --no-restore" \ |R_BROWSER=xdg-open \ |PAGER=/usr/bin/pager \ |PERL=/usr/bin/perl \ |R_UNZIPCMD=/usr/bin/unzip \ |R_ZIPCMD=/usr/bin/zip \ |R_PRINTCMD=/usr/bin/lpr \ |LIBnn=lib \ |AWK=/usr/bin/awk \ |CC="gcc -std=gnu99 -fsanitize=address -fsanitize=undefined | -fno-omit-frame-pointer" \ |CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic | -mtune=native" \ |FFLAGS="-g -O2 -mtune=native" \ |FCFLAGS="-g -O2 -mtune=native" \ |CXX="g++ -fsanitize=address -fsanitize=undefined,bounds-strict | -fno-omit-frame-pointer" \ |CXXFLAGS="-fsanitize=address -g -O2 -Wall -pedantic | -mtune=native" \
[R-sig-Debian] Error building and installing with GCC ASAN in a Docker container
Hi, I want to build a Docker container with R and various R packages built and installed with the gcc address sanitizer enabled. I am working on a Debian Bookworm system running on an x86_64 architecture. I start with the Docker file at https://github.com/rocker-org/r-devel-san/blob/master/Dockerfile I added -fsanitize=address to compiler and linker variables in the Dockerfile. I also import into the image a Makevars file with the contents CC=gcc -std=gnu99 -fsanitize=address -fno-omit-frame-pointer CXX=g++ -fsanitize=address -fno-omit-frame-pointer FC=gfortran -fsanitize=address LDFLAGS=-fsanitize=address Finally, the Docker file installs additional Debian object libraries and R packages that I want to test. When the package build gets to testing whether the package can be installed, an error message appears saying ==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. For example, ESC[0mESC[91m** building package indices ESC[0mESC[91m** installing vignettes ESC[0mESC[91m** testing if installed package can be loaded from temporary location ESC[0m==684==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ESC[91mERROR: loading failed ESC[0mESC[91m* removing ‘/usr/local/lib/R/site-library/utf8’ ESC[0mESC[91m* installing *source* package ‘isoband’ ... ESC[0mESC[91m** package ‘isoband’ successfully unpacked and MD5 sums checked ESC[0mESC[91m** using staged installation ESC[0mESC[91m** libs I've tried preloading various libasan.so files that I find on the system but the loader says that it cannot load them. My questions are 1. is it possible to build and install R and packages (in a container) with the gcc address sanitizer enabled? 2. if it is, how do I do it? 3. is this ld_preload message relevant? If so, to which file does it refer? Is there an ASAN runtime library file that I need to specify, and if so, what is the path to it? I appreciate your consideration and assistance. Thank you. Brent P.S. The build-related commands in the Docker file (including my address sanitizer additions) are RUN cd /tmp/R-devel \ && R_PAPERSIZE=letter \ R_BATCHSAVE="--no-save --no-restore" \ R_BROWSER=xdg-open \ PAGER=/usr/bin/pager \ PERL=/usr/bin/perl \ R_UNZIPCMD=/usr/bin/unzip \ R_ZIPCMD=/usr/bin/zip \ R_PRINTCMD=/usr/bin/lpr \ LIBnn=lib \ AWK=/usr/bin/awk \ CC="gcc -std=gnu99 -fsanitize=address -fsanitize=undefined -fno-omit-frame-pointer" \ CFLAGS="-fno-omit-frame-pointer -g -O2 -Wall -pedantic -mtune=native" \ FFLAGS="-g -O2 -mtune=native" \ FCFLAGS="-g -O2 -mtune=native" \ CXX="g++ -fsanitize=address -fsanitize=undefined,bounds-strict -fno-omit-frame-pointer" \ CXXFLAGS="-fsanitize=address -g -O2 -Wall -pedantic -mtune=native" \ CXX11FLAGS="-fsanitize=address -g -O2 -Wall -pedantic -mtune=native" \ CXX14FLAGS="-fsanitize=address -g -O2 -Wall -pedantic -mtune=native" \ CXX17FLAGS="-fsanitize=address -g -O2 -Wall -pedantic -mtune=native" \ LDFLAGS="-fsanitize=address" \ MAIN_LDFLAGS="-fsanitize=address -fsanitize=undefined -pthread" \ F77="gfortran -fsanitize=undefined" \ FC="gfortran -fsanitize=undefined" \ FCFLAGS="-g -O2 -mtune=native" \ FFLAGS="-g -O2 -mtune=native" \ ./configure --enable-R-shlib \ --without-blas \ --without-lapack \ --with-readline \ --without-recommended-packages \ --program-suffix=dev \ --disable-openmp \ && make \ && make install \ && make clean [[alternative HTML version deleted]] ___ R-SIG-Debian mailing list R-SIG-Debian@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-debian
