Re: [R-pkg-devel] how to change from -fsanitize=undefined to -fsanitize=address
Good to know. I will pass it on to Dan. He said a few times that he wasn't an expert on this :). Thanks! Youyi On Sun, Dec 1, 2019 at 6:01 AM Dirk Eddelbuettel wrote: > > On 30 November 2019 at 23:24, Youyi Fong wrote: > | Here is a solution worked out by the wonderful staff from our scientific > | computing support team at the Fred Hutchinson Cancer Research Center. I > | really don't know how Dan did it, but the way he explained to me was that > | he came across this github repository by Dirk: > | https://github.com/rocker-org/r-devel-san-clang. There was the Docker > file > | which built the clang-san image, but he did not find the image itself in > | Docker Hub. So he built it himself and pushed it as > > It lives with all the other Rocker containers here: > > https://hub.docker.com/repository/docker/rocker/r-devel-ubsan-clang > > There is a minor 'san at GitHub, ubsan at Docker' labeling issue but you or > Dan could have asked, and the Docker tag clearly links back to > > https://github.com/rocker-org/r-devel-san-clang > > which is where you started from. Anyway, glad to know it helped! > > Dirk > > -- > http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org > [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] how to change from -fsanitize=undefined to -fsanitize=address
Here is a solution worked out by the wonderful staff from our scientific
computing support team at the Fred Hutchinson Cancer Research Center. I
really don't know how Dan did it, but the way he explained to me was that
he came across this github repository by Dirk:
https://github.com/rocker-org/r-devel-san-clang. There was the Docker file
which built the clang-san image, but he did not find the image itself in
Docker Hub. So he built it himself and pushed it as
dtenenba/r-devel-san-clang.
The issue I encountered with r-debug was related to ulimit. It turns out
that it be specific to the machine I was working on. Dan used a different
machine and that went away.
Using the docker image dtenenba/r-devel-san-clang on that new machine, I
was able to install my package, and upon running the test code, to
reproduce the memory issue reported by the CRAN team.
I then tested whether r-debug might also work on that machine. It sort of
did as it returned a memory-related error when I tried
RDsan CMD INSTALL mypkg
But it is not as helpful as the behavior by dtenenba/r-devel-san-clang.
On Fri, Nov 29, 2019 at 1:48 PM Youyi Fong wrote:
> Hi Dirk,
>
> Thanks very much. Following your suggestion, I tried r-debug, following
> the readme on https://github.com/wch/r-debug. After starting docker as
> recommended,
>
> docker run --rm -ti --security-opt seccomp=unconfined wch1/r-debug
>
> I was able to call RD. But calling RDsan or RDcsan resulted in errors:
>
> root@258bfdfcce6c:/# RDsan
> ==17==ERROR: AddressSanitizer failed to allocate 0xdfff0001000
> (15392894357504) bytes at address 2008fff7000 (errno: 12)
> ==17==ReserveShadowMemoryRange failed while trying to map 0xdfff0001000
> bytes. Perhaps you're using ulimit -v
> Aborted (core dumped)
>
>
> I have also tried rhub before, following this article:
> https://cran.r-project.org/web/packages/rhub/vignettes/local-debugging.html As
> you indicated, it may have worked for others, but got the following error:
>
> >>>>> Installing system requirements
> 3b3a65d751c026b7629d2038242c92290cf97853525978381b6e59f41c8500d3
>
> >>>>> Starting Docker container
> 04d6065323ef732bf89c26df7d547efff40774206a0ac6a24b3d9cdb021cb08e
> ls: cannot access '/opt/R-*': No such file or directory
> > source('https://bioconductor.org/biocLite.R')
> Error: With R version 3.5 or greater, install Bioconductor packages using
> BiocManager; see https://bioconductor.org/install
> Execution halted
> Error in run(bash, c(file.path(wd, "rhub-linux.sh"), args), echo = TRUE, :
> System command error
>
>
>
> This is where I am stuck at right now.
>
> Thanks,
> Youyi
>
>
> On Fri, Nov 29, 2019 at 10:46 AM Dirk Eddelbuettel wrote:
>
>>
>> Youyi,
>>
>> It looks like you are trying to debug a _current_ SAN / ASAN / UBSAN error
>> against your package by leaning on documentation I wrote five years ago
>> specifically for another package (called "sanitizers") used to validate
>> the
>> initial Docker container builds I made (and to ensure the error these
>> SAN/ASAN/UBSAN containers were supposed to trigger were triggered).
>>
>> Today, your quickest bet to launch a sanitizer run may be via the rhub
>> platforms. Clearly the easiest to launch ("no download"), but at times a
>> little inconsistent. An alternative is the suite of containers
>> maintained by
>> Winston described at their repo at https://github.com/wch/r-debug at
>> available from the Docker Hub.
>>
>> All that said, exact configurations matters and the fact remains that we
>> all
>> still haven't managed to automatically provide the configurations used by
>> CRAN. There are possible "gaps" between what CRAN uses and what may be
>> available pre-built: compiler versions, builds, sanitizer options, ... and
>> these can be enough to not replicate an issue. I think I would try
>> Winston's
>> r-debug next.
>>
>> Hope this helps, Dirk
>>
>> --
>> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>>
>
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] how to change from -fsanitize=undefined to -fsanitize=address
Hi Dirk,
Thanks very much. Following your suggestion, I tried r-debug, following the
readme on https://github.com/wch/r-debug. After starting docker as
recommended,
docker run --rm -ti --security-opt seccomp=unconfined wch1/r-debug
I was able to call RD. But calling RDsan or RDcsan resulted in errors:
root@258bfdfcce6c:/# RDsan
==17==ERROR: AddressSanitizer failed to allocate 0xdfff0001000
(15392894357504) bytes at address 2008fff7000 (errno: 12)
==17==ReserveShadowMemoryRange failed while trying to map 0xdfff0001000
bytes. Perhaps you're using ulimit -v
Aborted (core dumped)
I have also tried rhub before, following this article:
https://cran.r-project.org/web/packages/rhub/vignettes/local-debugging.html As
you indicated, it may have worked for others, but got the following error:
> Installing system requirements
3b3a65d751c026b7629d2038242c92290cf97853525978381b6e59f41c8500d3
> Starting Docker container
04d6065323ef732bf89c26df7d547efff40774206a0ac6a24b3d9cdb021cb08e
ls: cannot access '/opt/R-*': No such file or directory
> source('https://bioconductor.org/biocLite.R')
Error: With R version 3.5 or greater, install Bioconductor packages using
BiocManager; see https://bioconductor.org/install
Execution halted
Error in run(bash, c(file.path(wd, "rhub-linux.sh"), args), echo = TRUE, :
System command error
This is where I am stuck at right now.
Thanks,
Youyi
On Fri, Nov 29, 2019 at 10:46 AM Dirk Eddelbuettel wrote:
>
> Youyi,
>
> It looks like you are trying to debug a _current_ SAN / ASAN / UBSAN error
> against your package by leaning on documentation I wrote five years ago
> specifically for another package (called "sanitizers") used to validate the
> initial Docker container builds I made (and to ensure the error these
> SAN/ASAN/UBSAN containers were supposed to trigger were triggered).
>
> Today, your quickest bet to launch a sanitizer run may be via the rhub
> platforms. Clearly the easiest to launch ("no download"), but at times a
> little inconsistent. An alternative is the suite of containers maintained
> by
> Winston described at their repo at https://github.com/wch/r-debug at
> available from the Docker Hub.
>
> All that said, exact configurations matters and the fact remains that we
> all
> still haven't managed to automatically provide the configurations used by
> CRAN. There are possible "gaps" between what CRAN uses and what may be
> available pre-built: compiler versions, builds, sanitizer options, ... and
> these can be enough to not replicate an issue. I think I would try
> Winston's
> r-debug next.
>
> Hope this helps, Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
>
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] how to change from -fsanitize=undefined to -fsanitize=address
Thanks. I was wondering how to do that within the context of docker containers. In the tutorial http://dirk.eddelbuettel.com/code/sanitizers.html, one calls the prebuilt Rdevel: # Rdevel CMD INSTALL sanitizers_0.1.0.tar.gz and get * installing to library '/usr/local/lib/R/site-library' * installing *source* package 'sanitizers' ... ** package 'sanitizers' successfully unpacked and MD5 sums checked ** libs g++-4.9 -fsanitize=address -I/usr/local/lib/R/include -DNDEBUG -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c heap_address.cpp -o heap_address.o g++-4.9 -fsanitize=address -I/usr/local/lib/R/include -DNDEBUG -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c stack_address.cpp -o stack_address.o g++-4.9 -fsanitize=address -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o sanitizers.so heap_address.o stack_address.o -L/usr/local/lib/R/lib -lR installing to /usr/local/lib/R/site-library/sanitizers/libs ** R ** inst But the same thing does not happen to me. I got -fsanitize=undefined instead. I did try modifying Makevars, but it did not seem to have an effect. On Fri, Nov 29, 2019 at 9:48 AM Uwe Ligges wrote: > See section 4.3.3 "Using the Address Sanitizer" in Writing R Extensions. > > Best, > Uwe Ligges > > On 29.11.2019 17:58, Youyi Fong wrote: > > Hello, I would like to reproduce this error locally: > > > https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-ASAN/chngpt/00check.log > > > > I followed the tutorial at > http://dirk.eddelbuettel.com/code/sanitizers.html, > > but what happened to me was not > > g++-4.9 -fsanitize=address -I/usr/local/lib/R/include -DNDEBUG > > -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c > > heap_address.cpp -o heap_address.o > > but > > g++-4.9 -fsanitize=undefined -I/usr/local/lib/R/include -DNDEBUG > > -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c > > heap_address.cpp -o heap_address.o > > > > And when I test my package, chngpt, I was not able to reproduce the > > error on CRAN. I am guessing that if change the compiler option from > > -fsanitize=undefined to -fsanitize=address, I might be able to reproduce > > the error. My question is how. Thanks! > > > > Youyi Fong > > > > -- > > Fred Hutchinson Cancer Research Center > > Seattle, WA > > > > [[alternative HTML version deleted]] > > > > __ > > R-package-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-package-devel > > > [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] how to change from -fsanitize=undefined to -fsanitize=address
Hello, I would like to reproduce this error locally: https://www.stats.ox.ac.uk/pub/bdr/memtests/clang-ASAN/chngpt/00check.log I followed the tutorial at http://dirk.eddelbuettel.com/code/sanitizers.html, but what happened to me was not g++-4.9 -fsanitize=address -I/usr/local/lib/R/include -DNDEBUG -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c heap_address.cpp -o heap_address.o but g++-4.9 -fsanitize=undefined -I/usr/local/lib/R/include -DNDEBUG -I/usr/local/include-fpic -pipe -Wall -pedantic -O3 -c heap_address.cpp -o heap_address.o And when I test my package, chngpt, I was not able to reproduce the error on CRAN. I am guessing that if change the compiler option from -fsanitize=undefined to -fsanitize=address, I might be able to reproduce the error. My question is how. Thanks! Youyi Fong -- Fred Hutchinson Cancer Research Center Seattle, WA [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
