Re: [R] R and ATLAS
Peter and Dirk: Thanks for the quick response -- I'm trying to get multiple CPU responses as Peter indicated should happen, but those R commands only illicit a single CPU response. When I check the libraries: ldd /usr/lib/R/bin/exec/R linux-vdso.so.1 = (0x7fffd05ff000) libR.so = /usr/lib/R/lib/libR.so (0x7f5f3b667000) libc.so.6 = /lib/libc.so.6 (0x7f5f3b313000) libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f5f3a7fe000) libgfortran.so.3 = /usr/lib/libgfortran.so.3 (0x7f5f3a512000) libm.so.6 = /lib/libm.so.6 (0x7f5f3a29) libreadline.so.6 = /lib/libreadline.so.6 (0x7f5f3a04b000) libpcre.so.3 = /lib/libpcre.so.3 (0x7f5f39e1c000) libbz2.so.1.0 = /lib/libbz2.so.1.0 (0x7f5f39c0c000) libz.so.1 = /usr/lib/libz.so.1 (0x7f5f399f4000) libdl.so.2 = /lib/libdl.so.2 (0x7f5f397f) /lib64/ld-linux-x86-64.so.2 (0x7f5f3bbec000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0x7f5f395da000) libpthread.so.0 = /lib/libpthread.so.0 (0x7f5f393bd000) libncurses.so.5 = /lib/libncurses.so.5 (0x7f5f39177000) Following /usr/lib/libblas.so.3gf : ls -l /usr/lib/libblas.so.3gf /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf And again following this: ls -l /etc/alternatives/libblas.so.3gf /etc/alternatives/libblas.so.3gf - /usr/lib/atlas-base/atlas/libblas.so.3gf So it appears to be properly linked to Atlas. Note that I got this same response from both the binary install of r-base (apt-get install r-base) and the source build of r-base-dev (apt-get source -b r-base-dev). Also, I did a tuned install of ATLAS 3.8.3 using: apt-get source atlas -t unstable cd atlas-3.8.3 DEFAULTS=n fakeroot debian/rules custom I did notice during the VERY long compilation that it seemed to correctly identify that I have 4 CPUs. Can someone confirm that the test: a = matrix(rnorm(5000*5000), 5000, 5000) b = matrix(rnorm(5000*5000), 5000, 5000) c = a%*%b Should illicit a multi-CPU response with R/ATLAS? If so, any suggestions on how to tweak my install to get it working? Thanks! --j On Wed, May 26, 2010 at 3:17 PM, Peter Langfelder peter.langfel...@gmail.com wrote: If you didn't specify an external BLAS when you ran R configure script, you are not using ATLAS. If you're not sure and you still have the output of the configure script, at the end it'll say whether it uses an external BLAS. Alternatively, you may also want to generate two random 5000x5000 matrices and do their multiplication a = matrix(rnorm(5000*5000), 5000, 5000) b = matrix(rnorm(5000*5000), 5000, 5000) c = a%*%b While the calculation is running, in a separate terminal, run top and watch how much CPU R takes. AFAIK standard installation of R is single threaded and will only use one CPU (up to 100%). ATLAS is multithreaded and (unless you configured it otherwise) it will use all available processors, so if you have a 4-core machine, you will see CPU usage of nearly 400%. Note though that this will not discriminate ATLAS from other multi-threaded BLASes. HTH, Peter On Wed, May 26, 2010 at 2:16 PM, Jonathan Greenberg greenb...@ucdavis.edu wrote: Rhelpers: I recently installed the 64-bit version of R on my Debian system, and afterwards was asked if it was compiled using ATLAS. Is there a way to test to see if R is using ATLAS? --j __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R and ATLAS
My R with Atlas configured to use 4 cores uses 4 cores. Note however that you only get the 4 cores for the actual multiplication, not the matrix generation. I'm probably several years behind the curve, but my experience is that if you download source R and do the standard ./configure make make install sequence, R will not use an external BLAS. The manual basically states as much: The linear algebra routines in R can make use of enhanced BLAS (Basic Linear Algebra Subprograms, http://www.netlib.org/blas/faq.html) routines. However, these have to be explicitly requested at configure time: R provides an internal BLAS which is well-tested and will be adequate for most uses of R. Unless the manual is out of date, I suspect that despite the apparent linking of the dynamic ATLAS library R still uses its own internal, single-threaded, BLAS. Peter On Wed, May 26, 2010 at 8:29 PM, Jonathan Greenberg greenb...@ucdavis.edu wrote: Peter and Dirk: Thanks for the quick response -- I'm trying to get multiple CPU responses as Peter indicated should happen, but those R commands only illicit a single CPU response. When I check the libraries: ldd /usr/lib/R/bin/exec/R linux-vdso.so.1 = (0x7fffd05ff000) libR.so = /usr/lib/R/lib/libR.so (0x7f5f3b667000) libc.so.6 = /lib/libc.so.6 (0x7f5f3b313000) libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f5f3a7fe000) libg fortran.so.3 = /usr/lib/libgfortran.so.3 (0x7f5f3a512000) libm.so.6 = /lib/libm.so.6 (0x7f5f3a29) libreadline.so.6 = /lib/libreadline.so.6 (0x7f5f3a04b000) libpcre.so.3 = /lib/libpcre.so.3 (0x7f5f39e1c000) libbz2.so.1.0 = /lib/libbz2.so.1.0 (0x7f5f39c0c000) libz.so.1 = /usr/lib/libz.so.1 (0x7f5f399f4000) libdl.so.2 = /lib/libdl.so.2 (0x7f5f397f) /lib64/ld-linux-x86-64.so.2 (0x7f5f3bbec000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0x7f5f395da000) libpthread.so.0 = /lib/libpthread.so.0 (0x7f5f393bd000) libncurses.so.5 = /lib/libncurses.so.5 (0x7f5f39177000) Following /usr/lib/libblas.so.3gf : ls -l /usr/lib/libblas.so.3gf /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf And again following this: ls -l /etc/alternatives/libblas.so.3gf /etc/alternatives/libblas.so.3gf - /usr/lib/atlas-base/atlas/libblas.so.3gf So it appears to be properly linked to Atlas. Note that I got this same response from both the binary install of r-base (apt-get install r-base) and the source build of r-base-dev (apt-get source -b r-base-dev). Also, I did a tuned install of ATLAS 3.8.3 using: apt-get source atlas -t unstable cd atlas-3.8.3 DEFAULTS=n fakeroot debian/rules custom I did notice during the VERY long compilation that it seemed to correctly identify that I have 4 CPUs. Can someone confirm that the test: a = matrix(rnorm(5000*5000), 5000, 5000) b = matrix(rnorm(5000*5000), 5000, 5000) c = a%*%b Should illicit a multi-CPU response with R/ATLAS? If so, any suggestions on how to tweak my install to get it working? Thanks! --j On Wed, May 26, 2010 at 3:17 PM, Peter Langfelder peter.langfel...@gmail.com wrote: If you didn't specify an external BLAS when you ran R configure script, you are not using ATLAS. If you're not sure and you still have the output of the configure script, at the end it'll say whether it uses an external BLAS. Alternatively, you may also want to generate two random 5000x5000 matrices and do their multiplication a = matrix(rnorm(5000*5000), 5000, 5000) b = matrix(rnorm(5000*5000), 5000, 5000) c = a%*%b While the calculation is running, in a separate terminal, run top and watch how much CPU R takes. AFAIK standard installation of R is single threaded and will only use one CPU (up to 100%). ATLAS is multithreaded and (unless you configured it otherwise) it will use all available processors, so if you have a 4-core machine, you will see CPU usage of nearly 400%. Note though that this will not discriminate ATLAS from other multi-threaded BLASes. HTH, Peter On Wed, May 26, 2010 at 2:16 PM, Jonathan Greenberg greenb...@ucdavis.edu wrote: Rhelpers: I recently installed the 64-bit version of R on my Debian system, and afterwards was asked if it was compiled using ATLAS. Is there a way to test to see if R is using ATLAS? --j __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained
Re: [R] R and ATLAS
On 26 May 2010 at 20:29, Jonathan Greenberg wrote: | Peter and Dirk: | | Thanks for the quick response -- I'm trying to get multiple CPU | responses as Peter indicated should happen, but those R commands only | illicit a single CPU response. | | When I check the libraries: | | ldd /usr/lib/R/bin/exec/R | linux-vdso.so.1 = (0x7fffd05ff000) | libR.so = /usr/lib/R/lib/libR.so (0x7f5f3b667000) | libc.so.6 = /lib/libc.so.6 (0x7f5f3b313000) | libblas.so.3gf = /usr/lib/libblas.so.3gf (0x7f5f3a7fe000) | libgfortran.so.3 = /usr/lib/libgfortran.so.3 (0x7f5f3a512000) | libm.so.6 = /lib/libm.so.6 (0x7f5f3a29) | libreadline.so.6 = /lib/libreadline.so.6 (0x7f5f3a04b000) | libpcre.so.3 = /lib/libpcre.so.3 (0x7f5f39e1c000) | libbz2.so.1.0 = /lib/libbz2.so.1.0 (0x7f5f39c0c000) | libz.so.1 = /usr/lib/libz.so.1 (0x7f5f399f4000) | libdl.so.2 = /lib/libdl.so.2 (0x7f5f397f) | /lib64/ld-linux-x86-64.so.2 (0x7f5f3bbec000) | libgcc_s.so.1 = /lib/libgcc_s.so.1 (0x7f5f395da000) | libpthread.so.0 = /lib/libpthread.so.0 (0x7f5f393bd000) | libncurses.so.5 = /lib/libncurses.so.5 (0x7f5f39177000) | | Following /usr/lib/libblas.so.3gf : | ls -l /usr/lib/libblas.so.3gf | /usr/lib/libblas.so.3gf - /etc/alternatives/libblas.so.3gf | | And again following this: | ls -l /etc/alternatives/libblas.so.3gf | /etc/alternatives/libblas.so.3gf - /usr/lib/atlas-base/atlas/libblas.so.3gf | | So it appears to be properly linked to Atlas. Note that I got this | same response from both the binary install of r-base (apt-get install | r-base) and the source build of r-base-dev (apt-get source -b | r-base-dev). Also, I did a tuned install of ATLAS 3.8.3 using: | | apt-get source atlas -t unstable | cd atlas-3.8.3 | DEFAULTS=n fakeroot debian/rules custom | | I did notice during the VERY long compilation that it seemed to | correctly identify that I have 4 CPUs. | | Can someone confirm that the test: | | a = matrix(rnorm(5000*5000), 5000, 5000) | b = matrix(rnorm(5000*5000), 5000, 5000) | c = a%*%b | | Should illicit a multi-CPU response with R/ATLAS? If so, any | suggestions on how to tweak my install to get it working? Thanks! I suggest that you a) talk to Sylvestre, the Atlas maintainer (CC'ed), as the last time I checked I thought our Atlas 3.8.* package were configure to be single-threaded so while you did the right thing with the local tuning, you may have omitted another toggle to give you multi-threaded Atlas b) look into Goto Blas from tacc.utexas.edu and/or Intel MKLs as well. Lastly, my preferred test is N - someNumber a - matrix(rnorm(N*N), ncol=N) n - someOtherNumber summary(replicate(n,system.time(crossprod(A))[3])) :) Dirk | --j | | On Wed, May 26, 2010 at 3:17 PM, Peter Langfelder | peter.langfel...@gmail.com wrote: | If you didn't specify an external BLAS when you ran R configure | script, you are not using ATLAS. If you're not sure and you still have | the output of the configure script, at the end it'll say whether it | uses an external BLAS. | | Alternatively, you may also want to generate two random 5000x5000 | matrices and do their multiplication | | a = matrix(rnorm(5000*5000), 5000, 5000) | b = matrix(rnorm(5000*5000), 5000, 5000) | c = a%*%b | | While the calculation is running, in a separate terminal, run top and | watch how much CPU R takes. AFAIK standard installation of R is single | threaded and will only use one CPU (up to 100%). ATLAS is | multithreaded and (unless you configured it otherwise) it will use all | available processors, so if you have a 4-core machine, you will see | CPU usage of nearly 400%. Note though that this will not discriminate | ATLAS from other multi-threaded BLASes. | | HTH, | | Peter | | | On Wed, May 26, 2010 at 2:16 PM, Jonathan Greenberg | greenb...@ucdavis.edu wrote: | Rhelpers: | | I recently installed the 64-bit version of R on my Debian system, and | afterwards was asked if it was compiled using ATLAS. Is there a way | to test to see if R is using ATLAS? | | --j | | __ | R-help@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-help | PLEASE do read the posting guide http://www.R-project.org/posting-guide.html | and provide commented, minimal, self-contained, reproducible code. | | | __ | R-help@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-help | PLEASE do read the posting guide http://www.R-project.org/posting-guide.html | and provide commented, minimal, self-contained, reproducible code. | | | __ | R-help@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-help | PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
[R] R and ATLAS
Rhelpers: I recently installed the 64-bit version of R on my Debian system, and afterwards was asked if it was compiled using ATLAS. Is there a way to test to see if R is using ATLAS? --j __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R and ATLAS
If you didn't specify an external BLAS when you ran R configure script, you are not using ATLAS. If you're not sure and you still have the output of the configure script, at the end it'll say whether it uses an external BLAS. Alternatively, you may also want to generate two random 5000x5000 matrices and do their multiplication a = matrix(rnorm(5000*5000), 5000, 5000) b = matrix(rnorm(5000*5000), 5000, 5000) c = a%*%b While the calculation is running, in a separate terminal, run top and watch how much CPU R takes. AFAIK standard installation of R is single threaded and will only use one CPU (up to 100%). ATLAS is multithreaded and (unless you configured it otherwise) it will use all available processors, so if you have a 4-core machine, you will see CPU usage of nearly 400%. Note though that this will not discriminate ATLAS from other multi-threaded BLASes. HTH, Peter On Wed, May 26, 2010 at 2:16 PM, Jonathan Greenberg greenb...@ucdavis.edu wrote: Rhelpers: I recently installed the 64-bit version of R on my Debian system, and afterwards was asked if it was compiled using ATLAS. Is there a way to test to see if R is using ATLAS? --j __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R and ATLAS
Jonathan, That would have been a good question for the r-sig-debian list ... On 26 May 2010 at 14:16, Jonathan Greenberg wrote: | Rhelpers: | | I recently installed the 64-bit version of R on my Debian system, and | afterwards was asked if it was compiled using ATLAS. Is there a way | to test to see if R is using ATLAS? Atlas is one (accelerated, tuned) implementation of an interface called Blas. R is built with support for whichever Blas you have -- which could be Atlas or non-free alternative such as Goto or Intel MKL, or it could be the slower 'reference blas'. If you also install the r-base-dev package (recommended if you build package) you will get headers and libraries for Blas, with the Atlas package being the default. Lastly, on Linux, you can check which libraries are used via the ldd tool. On my (32bit) system: e...@ron:~ ldd /usr/lib/R/bin/exec/R linux-gate.so.1 = (0xb786c000) libR.so = /usr/lib/libR.so (0xb753) libc.so.6 = /lib/i686/cmov/libc.so.6 (0xb73e9000) libblas.so.3gf = /usr/lib/atlas/libblas.so.3gf (0xb706d000) libgfortran.so.3 = /usr/lib/libgfortran.so.3 (0xb6fa8000) libm.so.6 = /lib/i686/cmov/libm.so.6 (0xb6f82000) libreadline.so.6 = /lib/libreadline.so.6 (0xb6f4d000) libpcre.so.3 = /lib/libpcre.so.3 (0xb6f1d000) libbz2.so.1.0 = /lib/libbz2.so.1.0 (0xb6f0c000) libz.so.1 = /usr/lib/libz.so.1 (0xb6ef7000) libdl.so.2 = /lib/i686/cmov/libdl.so.2 (0xb6ef3000) /lib/ld-linux.so.2 (0xb786d000) libgcc_s.so.1 = /lib/libgcc_s.so.1 (0xb6ed5000) libncurses.so.5 = /lib/libncurses.so.5 (0xb6e9b000) e...@ron:~ and you notice in the fourth line how the linker fills in the libblas.so.3gf from Atlas for libblas, eg the Blas library needed here. Similar output can be generated for /usr/lib/R/modules/lapack.so showing that Atlas also provides tuned Lapack libraries. Hope this helps. The R Install + Admin manual has more detail. -- Regards, Dirk __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] R with atlas in ubuntu karmic 64
Hi, I'm trying to install R with atlas in ubuntu karmic 64. I do a lot of matrix operations so I need performance. Doing grep sse2 /proc/cpuinfo tells me tha my CPU supports SSE2. But I don't see libatlas3gf-sse2 in the package repositories. Also, there seem to be some gotchas on how to install atlas, even the vanilla one, as described here: http://thread.gmane.org/gmane.comp.lang.r.general/159634/focus=159867 I didn't understand what Bill Dunlap proposes as a solution. Has anyone installed R with atlas in ubuntu karmic 64? Maybe writing a howto on a blog post would be a good idea. If I manage to install it, I will... Best, -Jose -- === I'm minimizing email (can write w one hand only) so if you have something that requires a long answer, please mail me a time slot and tel. number and I'll call you. Thanks for your patience.=== Jose Quesada, PhD. Max Planck Institute, Center for Adaptive Behavior and Cognition -ABC-, Lentzeallee 94, office 224, 14195 Berlin http://www.josequesada.name/ http://twitter.com/Quesada __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
