Re: [R] linking R to goto blas
Dear all, Finally -- after one week of trial & error -- the fog lifted thanks to the hints of Kazushige Goto. Therefore, I would like to report back the workaround (for Fedora Core based systems, including FC4) to the list and a pre-mature analysis, inspired by K. Goto: A brief howto (applicable to and tested on Fedore Core 4, R-2.1.0, libgoto 0.99-3): 1) Get the appropriate GOTO BLAS library for your processor core and place it in an arbitrary directory (either /usr/lib or your home directory). 2) set the environment variable OMP_NUM_THREADS=1 (in bash: "export OMP_NUM_THREADS=1") 3) change to your R source directory and run configure according to your needs: pass the following values to the "--with-blas-flag": --with-blas="-L -lgoto_xxx-xx-r0.99-3 -lpthread" (in my case: --with-blas="-L/usr/lib/ -lgoto_northwood-32-r0.99-3 -lpthread"). 4) run make & make install 5) enjoy GOTO performance BACKGROUND: According to K. Goto setting OMP_NUM_THREADS=1 results in libgoto not using the pthread library, therefore it is running in single-thread-mode, while R might still be supporting threads in general. He concludes that there is a major incompatibility with pthread which might be due to RedHat's/ Fedora's home-brewed (since RH 9) reliance on multithreading simulation provided by their Native POSIX Thread Library (NLTP). Libgoto as such is not NLTP-compatible according to K. Goto. Other distributions (and thread implementations) don't show this misbehaviour (on my Debian based system libgoto links perfectly without the workaround described above). Thanks for all your comments and contributions. //[EMAIL PROTECTED] Prof Brian Ripley wrote: > R already contains a xerbla: you should not be adding one. > > On Mon, 13 Jun 2005, Stefan Sobernig wrote: > >> Prof. Ritley, Mr. Dalgaard, thank you very much for the immediate reply >> and your efforts! >> We investigated a littlle further, though not directly following Mr. >> Dalgaard's findings >> on dynamic linking. We got aware of another short notice published on >> the R-devel list, >> providing some hints for installing R and goto (see >> http://maths.newcastle.edu.au/~rking/R/devel/03b/1267.html): >> >> >> snip ### >> [What I did to link R against Goto's BLAS (mostly following Prof. Bates' >> instruction): >> - Install libgoto*.so and symlink to libgoto.so. >> - Download xerbla.f and compile to xerbla.o. >> - Run the R configure script with --with-blas="-lgoto >> /path/to/xerbla.o". >> - Edit Makeconf and delete the path to xerbla.o in BLAS_LIB. >> - make; make check >> ] >> snip ### >> >> * Therefore, we got the auxiliary lapack routine xerbla offered at >> http://www.cs.utexas.edu/users/kgoto/libraries/xerbla.c >> and compiled it by calling: "gcc -c xerbla.c -o xerbla.o" (please note, >> that I am not that expierenced in C programming as such, >> my knowledge is limited to deploying and fixing c-coded apps). Am I >> right to say that this xerbla extension changes the entry >> point of the goto library? >> >> * Then, we re-configured R using: ./configure --prefix=/usr >> --with-blas="-L/usr/lib/ -lgoto_prescott-32-r0.99-3 >> /home/ssoberni/xerbla.o -lpthread" >> >> * Unfortunately, this resulted in another error affecting the configure >> process so that BLAS_LIBS is set to "-lblas" (pointing to standard blas >> libs): >> >> ### config.log >> configure:32429: gcc -o conftest -g -O2 -I/usr/local/include >> -L/usr/local/lib conftest.c -L/usr/lib/ -lgoto_prescott-32-r0.99-3 >> /home/ssoberni/xerbla.o -lpthread -lg2c -lm -lgcc_s -ldl -lm >&5 >> /home/ssoberni/xerbla.o(.text+0x0): In function `xerbla_': >> : multiple definition of `xerbla_' >> /tmp/ccMxAKbu.o(.text+0x0):/home/ssoberni/R-2.1.0/conftest.c:143: first >> defined here >> /usr/bin/ld: Warning: size of symbol `xerbla_' changed from 5 in >> /tmp/ccMxAKbu.o to 43 in /home/ssoberni/xerbla.o >> collect2: ld returned 1 exit status >> ### config.log >> >> We keep on trying ... >> Thanks for your thoughts ... >> >> //stefan >> >> >> Peter Dalgaard wrote: >> >>> Peter Dalgaard <[EMAIL PROTECTED]> writes: >>> >>> >>> >>>> Stefan Sobernig <[EMAIL PROTECTED]> writes: >>>> >>>> >>>> >>>>> Dear all, >>>>> >
Re: [R] linking R to goto blas
Prof. Ritley, Mr. Dalgaard, thank you very much for the immediate reply and your efforts! We investigated a littlle further, though not directly following Mr. Dalgaard's findings on dynamic linking. We got aware of another short notice published on the R-devel list, providing some hints for installing R and goto (see http://maths.newcastle.edu.au/~rking/R/devel/03b/1267.html): snip ### [What I did to link R against Goto's BLAS (mostly following Prof. Bates' instruction): - Install libgoto*.so and symlink to libgoto.so. - Download xerbla.f and compile to xerbla.o. - Run the R configure script with --with-blas="-lgoto /path/to/xerbla.o". - Edit Makeconf and delete the path to xerbla.o in BLAS_LIB. - make; make check ] snip ### * Therefore, we got the auxiliary lapack routine xerbla offered at http://www.cs.utexas.edu/users/kgoto/libraries/xerbla.c and compiled it by calling: "gcc -c xerbla.c -o xerbla.o" (please note, that I am not that expierenced in C programming as such, my knowledge is limited to deploying and fixing c-coded apps). Am I right to say that this xerbla extension changes the entry point of the goto library? * Then, we re-configured R using: ./configure --prefix=/usr --with-blas="-L/usr/lib/ -lgoto_prescott-32-r0.99-3 /home/ssoberni/xerbla.o -lpthread" * Unfortunately, this resulted in another error affecting the configure process so that BLAS_LIBS is set to "-lblas" (pointing to standard blas libs): ### config.log configure:32429: gcc -o conftest -g -O2 -I/usr/local/include -L/usr/local/lib conftest.c -L/usr/lib/ -lgoto_prescott-32-r0.99-3 /home/ssoberni/xerbla.o -lpthread -lg2c -lm -lgcc_s -ldl -lm >&5 /home/ssoberni/xerbla.o(.text+0x0): In function `xerbla_': : multiple definition of `xerbla_' /tmp/ccMxAKbu.o(.text+0x0):/home/ssoberni/R-2.1.0/conftest.c:143: first defined here /usr/bin/ld: Warning: size of symbol `xerbla_' changed from 5 in /tmp/ccMxAKbu.o to 43 in /home/ssoberni/xerbla.o collect2: ld returned 1 exit status ### config.log We keep on trying ... Thanks for your thoughts ... //stefan Peter Dalgaard wrote: >Peter Dalgaard <[EMAIL PROTECTED]> writes: > > > >>Stefan Sobernig <[EMAIL PROTECTED]> writes: >> >> >> >>>Dear all, >>> >>>I am currently trying to link R 2.1.0 to the GOTO BLAS 0.99.3 library on >>>a box running Fedora Core 3 , basically following the steps indicated in >>>the R-Admin document: >>> >>>1: I downloaded the current libgoto.xxx.so from >>>http://www.cs.utexas.edu/users/kgoto/libraries/libgoto_prescott-32-r0.99-3.so.gz, >>>a version suitable for our XEON machine (Nocona core), unpacked it to >>>/usr/lib and created a symlink libgoto.so pointing to the library. >>> >>>2: Then, I got ready to re-configure and re-compile R (2.1.0) using the >>>following configure flags: ./configure --prefix=/usr --enable-R-shlib >>>--enable-shared --with-tcltk --with-blas="-lgoto -lpthread -lm" >>> >>> >>... >> >> >>>Please, I highly appreciate any thoughts or hints as my colleagues and I >>>are eager to get into GOTO's universe. >>> >>> >>Hmm. Looks over-complicated to me. What works for me on AMD64 is to >>have a config.site file in my BUILD-GOTO directory, containing >> >> >> >>>cat config.site >>> >>> >>BLAS_LIBS="-L/home/pd/GOTO -lgoto_opt64p-r0.96 -lpthread" >>CFLAGS="-O3 -g" >>#CFLAGS="-g" >>FFLAGS=$CFLAGS >>CXXFLAGS=$CFLAGS >> >>(the .*FLAGS business is optional, of course). With this in place, a >>simple ../R/configure followed by make seems to do the trick. >> >>I'll give it a try on my FC3 system, but it's a 500 MHz PIII, so it >>takes a while... >> >> > >Hmm... That gives me the grDevices issue, which boils down to an R >that segfaults immediately upon startup, in > >#0 0x05c0aea7 in tilde_expand () from /usr/lib/libreadline.so.4 >#1 0x08170254 in R_ExpandFileName_readline ( >s=0x8bb4020 >#"/home/pd/r-patched/BUILD-GOTO/library/grDevices/R/sysdata.rdb", >#buff=0x8295300 >#"/home/pd/r-patched/BUILD-GOTO/library/grDevices/R/grDevices") >at ../../../R/src/unix/sys-std.c:406 >#2 0x0816f5da in R_ExpandFileName ( >s=0x8bb4020 >#"/home/pd/r-patched/BUILD-GOTO/library/grDevices/R/sysdata.rdb") at >#../../../R/src/unix/sys-unix.c:129 >#3 0x08167352 in R_FileE
[R] linking R to goto blas
Dear all, I am currently trying to link R 2.1.0 to the GOTO BLAS 0.99.3 library on a box running Fedora Core 3 , basically following the steps indicated in the R-Admin document: 1: I downloaded the current libgoto.xxx.so from http://www.cs.utexas.edu/users/kgoto/libraries/libgoto_prescott-32-r0.99-3.so.gz, a version suitable for our XEON machine (Nocona core), unpacked it to /usr/lib and created a symlink libgoto.so pointing to the library. 2: Then, I got ready to re-configure and re-compile R (2.1.0) using the following configure flags: ./configure --prefix=/usr --enable-R-shlib --enable-shared --with-tcltk --with-blas="-lgoto -lpthread -lm" I did read the R-Admin doc and therefore I am aware of the fact that passing "-lgoto" is supposed to be sufficient, but as a matter of fact configuring with --with-blas="-lgoto" only ends up in a libR.so being linked to the standard libblas.so. config.log reports in this settings that libgoto.xxx.so is missing links to libpthread etc. Therefore, I added the two flags "-lpthread -lm" as indicated at GOTO's website and I got a clean configure run. (Am I concluding correctly that I am using a threaded version of goto blas?) 3: Running make, however, freezed when trying to build "grDevices", without throwing any warning or error messages: [...] ../../../../library/grDevices/libs/grDevices.so is unchanged make[5]: Leaving directory `/home/ssoberni/R-2.1.0/src/library/grDevices/src' make[4]: Leaving directory `/home/ssoberni/R-2.1.0/src/library/grDevices/src' [freeze] 4: I then rummaged the R mailing list archives and stumbled over a thread dating from May this year pointing to a similar issue, concerning gcc-3.4 and broken lapack libraries provided by FC3 (see https://stat.ethz.ch/pipermail/r-devel/2005-May/033117.html). Following these opinions/ findings, I did the following (though I knew that -- in principle -- R is supposed to handle this issue by passing a --ffloat-store flag to the fortran compiler, doesn't it?): * I wanted to remove the FC3 native lapack libraries, and to my surprise, they were not installed at all (no liblapack.so.xxx in /usr/lib). * I set up an older gcc environment, i.e. the last release from the 3.3.x family (3.3.6) and tried to recompile R ending up with the same hang-up. As a last step, I tried to exclude R's internal package explicitly by setting --wihtout-lapack, which did not hava a visible effect on the building process and did not provide a workaround for the hang-up. Please, I highly appreciate any thoughts or hints as my colleagues and I are eager to get into GOTO's universe. //stefan -- Stefan Sobernig Department of Information Systems and New Media Vienna University of Economics Augasse 2-6 A - 1090 Vienna Phone: +43 - 1 - 31336 - 4878 Fax: +43 - 1 - 31336 - 746 Email: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> PubKey: http://julia.wu-wien.ac.at/~ssoberni/0x5FC2D3FA.asc <http://julia.wu-wien.ac.at/%7Essoberni/0x5FC2D3FA.asc> [[alternative HTML version deleted]] __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
[R] PL/R calls fail
Hi all,
I am currently trying to create a development environment
including PostgreSQL 8.0.0rc1, R 2.0.1 and PL/R on a system running Fedora
Cora 1.
So far, I have suceeded in setting up PostgreSQL and R as
a shared library - unfortunately I have not been able to link these
two spheres by adding the PostgreSQL add-on PL/R due to
some mysterious probs.
I thoroughly followed the setup instructions for PL/R (see
http://www.joeconway.com/plr/doc/plr-install.html)
and arrived at being able to register PL/R with PostgreSQL properly and
even to create PL/R-specific functions.
>But< whenever trying to invoke these functions, I end up with
a single exception, even for simple ones which are just supposed to return
the previously passed arguments:
##snip###
ERROR: R interpreter parse error
##snip###
A closer look at the postmaster's/ postgres's log-file reveals that a
hardly more
specific exception is thrown at this level, i.e.:
##snip###
ERROR: R interpreter parse error
DETAIL: R parse error caught in "PLR17706 <- function(arg1,arg2) {
if (arg1 > arg2)
return(arg1)
else
return(arg2)
}".
CONTEXT: In PL/R function r_max
##snip###
Even gdb monitoring did not provide any further hint, using older
PostgreSQL, R and PL/R versions
didn't make a difference.
I would kindly ask for your help and thoughts!
best regards,
Stefan
[[alternative HTML version deleted]]
__
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
