On 04/22/2017 02:57 PM, cstrato wrote:
Dear Herve,Thank you for your efforts to get xps running. I will make the changes in Makefile.arch, too, and update README and rename it to install. I am also surprised that the libs get installed under <prefix>/lib/root, and I will check if there is a reason for this. As far as I remember, long time ago someone had a similar problem, see e.g.: https://urldefense.proofpoint.com/v2/url?u=https-3A__support.bioconductor.org_p_51838_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=lvjHdGnauUA25Q4WQU1Y7Sn74tcZrxSh529Bwgg5tG4&s=P0qNlFiUPP9UB0MiGeyiKvTPduWT0c3yp24Q7hNkWDY&e= Do you use the environment variable ROOTSYS?
I gave you the full transcript of what I do for configure/build/install. I'm following the "fix location" installation procedure as described here: https://root.cern.ch/building-root so no ROOTSYS (the above document doesn't even mention that variable). H.
Best regards, Christian On 04/22/17 22:36, Hervé Pagès wrote:Hi Christian, Thanks for the update. Glad it works for you. One small thing is that, if CMAKE_INSTALL_PREFIX is specified when configuring ROOT, the ROOT libs get installed under <prefix>/lib/root, not under <prefix>/lib. I was surprised by this, but that's what I got when I installed ROOT on veracruz2. I configured with: export CC=/usr/local/clang4/bin/clang export CXX=/usr/local/clang4/bin/clang++ cmake -DCMAKE_INSTALL_PREFIX=/usr/local/root -Dgnuinstall=ON -Dfortran=OFF -Dmysql=OFF -Dsqlite=OFF ../root Then built with: cmake --build . -- -j4 Then installed with: sudo cmake --build . --target install And the libraries got installed under /usr/local/root/lib/root So when trying to install the latest xps, loading xps.so failed for me because the ROOT libraries were not found. The following change to Makefile.arch fixed the problem: $ svn diff Makefile.arch Index: Makefile.arch =================================================================== --- Makefile.arch (revision 129046) +++ Makefile.arch (working copy) @@ -261,7 +261,7 @@ CXXFLAGS = $(OPT2) -pipe -Wall -W -Woverloaded-virtual LD = $(MACOSXTARGET) g++ #LDFLAGS = $(OPT2) -LDFLAGS = $(OPT2) -rpath $(shell $(ROOTCONFIG) --prefix)/lib +LDFLAGS = $(OPT2) -rpath $(shell $(ROOTCONFIG) --prefix)/lib -rpath $(shell $(ROOTCONFIG) --prefix)/lib/root UNDEFOPT = dynamic_lookup # The SOFLAGS will be used to create the .dylib, # the .so will be created separately Also note that the rpaths specified at linking time get hardcoded in xps.so: veracruz2:src biocbuild$ otool -l xps.so | tail -n 18 Load command 31 cmd LC_RPATH cmdsize 32 path /usr/local/root/lib (offset 12) Load command 32 cmd LC_RPATH cmdsize 40 path /usr/local/root/lib/root (offset 12) Load command 33 cmd LC_FUNCTION_STARTS cmdsize 16 dataoff 4141784 datasize 14336 Load command 34 cmd LC_DATA_IN_CODE cmdsize 16 dataoff 4156120 datasize 904 So the end user will need to have the ROOT libraries at these locations too (unless s/he installs from source of course). This would need to be explained in xps README file (which BTW should preferably be named INSTALL). Thanks, H. On 04/22/2017 05:15 AM, cstrato wrote:Dear Herve, I am glad to inform you that I have just uploaded version xps_1.35.3 to BioC-dev branch. I have followed your suggestion to use -rpath and have eliminated the environment variables DYLD_LIBRARY_PATH and LD_LIBRARY_PATH. I have tested the new version on both Yosemite and on Sierra with csrutil enabled! Thus I assume that it will also run on El Capitan. Best regards, Christian P.S.: Please allow me to comment on your note on [DY]LD_LIBRARY_PATH. As you know xps was uploaded to Bioc 10 years ago (with your kind help) and is available on BioC since 9 years. At that time the environment variables [DY]LD_LIBRARY_PATH were necessary, and for many years required by ROOT. Since xps did run on the Mac on all systems from Leopard till Yosemite w/o problems I had no need to change it. Furthermore, I had not heard that the use of these variables have been discouraged, just like many other developers who only now realize that they have to use rpath or simply disable csrutil (I have realized this when googling around). On 04/21/17 00:29, Hervé Pagès wrote:Also relying on [DY]LD_LIBRARY_PATH is considered bad practice and has been discouraged for years. xps is the only Bioconductor package that relies on these variables for its configure/build process. H. On 04/20/2017 03:24 PM, Dan Tenenbaum wrote:Disabling SIP should not be done anywhere. Every page I've read on this topic strongly discourages doing this. ----- Original Message -----From: "Hervé Pagès" <hpa...@fredhutch.org> To: "cstrato" <cstr...@aon.at>, "Dan Tenenbaum" <dtene...@fredhutch.org> Cc: "bioc-devel" <bioc-devel@r-project.org> Sent: Thursday, April 20, 2017 3:17:23 PM Subject: Re: [Bioc-devel] xps build problem on veracruz2On 04/20/2017 03:01 PM, cstrato wrote:Dear Herve, Doing 'csrutil disable' does indeed solve the problem, since: 1, in this way I was able to build xps on Mac OS Sierra 2, in this way I could already help one user of xps, see: https://urldefense.proofpoint.com/v2/url?u=https-3A__support.bioconductor.org_p_90056_-2390247&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q0hKDI_veZEYmjrYbsK1Xqu--fGdfl_JmfSKLygl_dg&s=GfzgU1Ibm_scFWO58Mv_ZfxKtn-FSJgkxMW1ZBYK1Vs&e= This means, that users of xps seem to be willing to disable csrutil.I'm not a statistician but I wouldn't draw conclusions from a sample of size 1. Mac users who cannot install xps on their machine will use something else, or, if they desperately need xps, they will grab a Linux box. Sorry but disabling SIP on our Mac builders is not an option.However, I just realized that there may be an even greater problem, namely, which version of ROOT should a user install? People can download ROOT binaries built with XCode 7.x from: https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_content_release-2D53436&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q0hKDI_veZEYmjrYbsK1Xqu--fGdfl_JmfSKLygl_dg&s=LByiYPyOfsHnRc9oOqXOs9xMcfltktXgkTQ3sh5hKFc&e= for OS X 10.10 and 10.11. Thus for El Capitan they can download the following binaries: - root_v5.34.36.macosx64-10.11-clang70.dmg - root_v5.34.36.macosx64-10.11-clang70.tar.gz Did you install one of those binaries or did you compile ROOT from source?As I said earlier, I compiled ROOT 5 from source on veracruz2.With the help of my README file people could compile ROOT from source for XCode 8.x. However, you have mentioned that the CRAN people are using clang 4.0.0 for producing the Mac binaries of R and CRAN packages and thus you are using the same on veracruz2.Yes.Did you compile ROOT with XCode 7 or 8 or did you use clang 4.0.0, which is not officially supported by Apple?With clang 4.0.0.The question is whether xps built with this version of ROOT will work with the ROOT binaries which people can download from ROOT?I guess someone will need to figure this out. Note that if people need to compile their own ROOT anyway in order to be able to use the xps binary we distribute, then they should also be able to install xps from source. So that defeats the purpose of providing a binary in the first place. Cheers, H.Best regards, Christian On 04/20/17 20:00, Hervé Pagès wrote:On 04/20/2017 10:59 AM, Hervé Pagès wrote:Hi Christian, Disabling 'csrutil disable' might help xps on veracruz2 but that^^^^^^^^^^^^^^^^^^^^^^^^^^ oops, no double negative intended here. I meant, doing 'csrutil disable' might help... etc H.won't help your end users. I'm no expert in developing a package on Mac but other people on this list are. Also R-SIG-Mac might be a good place to seek help for this. Cheers, H. On 04/20/2017 10:53 AM, cstrato wrote:Dear Herve, Thank you for your efforts to try to install xps. I am glad to hear that you could build ROOT 5 from source. It's a pity that Apple does no longer allow the use of DYLD_LIBRARY_PATH. This seems to break the code of a lot of people (when googling around). I will try to change the build process and will see if I succeed. However, at the moment I have a couple of questions: 1, Is there any reason that you do not want to 'csrutil disable'? 2, It is easy to delete the test for DYLD_LIBRARY_PATH in my 'configure.in' file, however, I have also to change the 'Makefile'. Since you say that the problem can be solved by adding the -rpath flag when linking, I assume that I have to change the following line in my 'Makefile': $(LD) -bundle $(LDFLAGS) $^ $(GLIBS) $(MYLIBS) \ $(OutPutOpt) $(subst .$(DllSuf),.so,$@) Since I am not familiar with -rpath can you give me a hint how to change it? 3, There is a Wikipedia explanation for '-rpath', see: https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org_wiki_Rpath&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=ge_d4eBoKDAggNWVQUzMVPAJy250VlZuPTXcPyr20HM&e= Interestingly, there is the following line: 'Instead of specifying the -rpath to the linker, the environment variable LD_RUN_PATH can be set to the same effect.' Do you think that using LD_RUN_PATH would solve the problem? If yes, then how? 4, Googling for the DYLD_LIBRARY_PATH problem I have also found the following discussion: https://urldefense.proofpoint.com/v2/url?u=https-3A__forums.macrumors.com_threads_is-2Dit-2Dok-2Dto-2Duse-2Ddyld-5Flibrary-5Fpath-2Don-2Dmac-2Dos-2Dx-2Dand-2Dwhat-25C2-2592s-2Dthe-2Ddynamic-2Dlibrary-2Dsearch.956258_&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=yhLXARIG_RS1LNegc7tbjO1bayLOp7zy5-rzbtjCHIk&e= There, one answer mentioned: 'Also, rpath is a good idea. Also see install_name_tool, which can change the load paths of libraries.' Doing a search for 'install_name_tool' I found: https://urldefense.proofpoint.com/v2/url?u=http-3A__stackoverflow.com_questions_2985315_using-2Dinstall-2Dname-2Dtool-2Dwhats-2Dgoing-2Dwrong&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=r5LiHlxkGAeJAcciwteD2XD6ffNtWLiknvcIj9EJL8E&e= There, one answer mentioned (see also $man install_name_tool): 'Having experimented more: install_name_tool -id newname file will do the trick.' See also: https://urldefense.proofpoint.com/v2/url?u=http-3A__qin.laya.com_tech-5Fcoding-5Fhelp_dylib-5Flinking.html&d=DwIDaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=kxgD4oa4CZeT4T7uLq2m3iVTvgvFB_RKN1Rf1KcxPk0&s=FL17xv1uuzR6UF59mgJstMQ-1seE1UeKDQVVXQgXnUo&e= Do you think that either using the environment variable LD_RUN_PATH or using 'install_name_tool' could solve the problem without having to use '-rpath'? Thank you in advance. Best regards, Christian On 04/19/17 22:51, Hervé Pagès wrote:Hi Christian, So I installed ROOT 5 from source on veracruz2. It's in /usr/local/root. However, Apple's SIP (System Integrity Protection, new and enabled by default on El Capitan) is getting in the way when trying to install xps. That's because xps configure and build process relies on DYLD_LIBRARY_PATH. Problem is that this environment variable (and any other variables that control dynamic loading) is not inherited by child processes when SIP is on: veracruz2:~ biocbuild$ if test "${DYLD_LIBRARY_PATH}"; then echo 'yep!'; else echo 'nope!'; fi yep! veracruz2:~ biocbuild$ sh sh-3.2$ if test "${DYLD_LIBRARY_PATH}"; then echo 'yep!'; else echo 'nope!'; fi nope! That breaks xps configure script: veracruz2:~ biocbuild$ export LD_LIBRARY_PATH=$DYLD_LIBRARY_PATH veracruz2:~ biocbuild$ echo $LD_LIBRARY_PATH /usr/local/mysql/lib:/usr/local/root/lib/root:/ImageMagick-7.0.5/lib:/usr/local/ensembl-vep/htslib veracruz2:~ biocbuild$ R CMD INSTALL xps * installing to library ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library’ * installing *source* package ‘xps’ ... checking for gcc... clang checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether clang accepts -g... yes checking for clang option to accept ANSI C... none needed checking how to run the C preprocessor... clang -E checking for gcc... (cached) clang checking whether we are using the GNU C compiler... (cached) yes checking whether clang accepts -g... (cached) yes checking for clang option to accept ANSI C... (cached) none needed found ROOT version 5.34/36 in directory /usr/local/root xps configuration error: You must set the shell variable LD_LIBRARY_PATH to the directory where ROOT resides and re-run R CMD INSTALL e.g., (using Bourne shell syntax): export "LD_LIBRARY_PATH=$ROOTSYS/lib:$LD_LIBRARY_PATH" R CMD INSTALL xps Please consult the README file for more information ERROR: configuration failed for package ‘xps’ * removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps’ * restoring previous ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/xps’ That also breaks the dynlib mechanism because, after I managed to produce xps.so, it turns out that this shared object is linked to the ROOT libraries via the @rpath mechanism: veracruz2:src biocbuild$ otool -L xps.so xps.so: xps.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libGui.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libCore.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libCint.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libRIO.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libNet.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libHist.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libGraf.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libGraf3d.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libGpad.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libTree.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libRint.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libPostscript.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libMatrix.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libPhysics.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libMathCore.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libThread.so (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1) @rpath/libGed.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libTreePlayer.so (compatibility version 0.0.0, current version 0.0.0) @rpath/libTreeViewer.so (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0) so it can't be loaded in R:dyn.load("xps.so")Error in dyn.load("xps.so") : unable to load shared object '/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so': dlopen(/Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so, 6): Library not loaded: @rpath/libGui.so Referenced from: /Users/biocbuild/bbs-3.5-bioc/xps/src/xps.so Reason: image not found because R has no access to DYLD_LIBRARY_PATH:Sys.getenv("DYLD_LIBRARY_PATH")[1] "" This can be addressed by adding the following flag when linking: -rpath $(shell $(ROOTCONFIG) --prefix)/lib Do you think you can revisit xps configure and build process? Make sure you test it on a machine where SIP is enabled. Thanks, H.snip >>>On 03/23/17 17:47, Hervé Pagès wrote:Hi Christian, The CRAN folks are currently experimenting with clang 4.0.0 for producing the Mac binaries of R and CRAN packages so we are using the same on veracruz2. This is a version of clang that is ahead of what's in XCode 8.x or XCode 7.x. So I guess that means we'll have to compile ROOT from source on veracruz2. BTW any reason not to make xps work with ROOT 6? Cheers, H. On 03/23/2017 07:28 AM, cstrato wrote:Dear Valerie, I have seen that you have set up a new Mac server, veracruz2, running El Capitan. Although the development version of xps does even run on Mac OS Sierra, one issue still remains the same: You need to install the latest ROOT version 5, since xps does not run with ROOT 6! So you need to install on veracruz2 the same root version that you have installed on toluca2 running Maverics, i.e. root_v5.34.36.macosx64-10.11-clang70.dmg However, if you have installed on El Capitan XCode 8.x instead of XCode 7.x, then you need to compile ROOT from source, i.e.: https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_download_root-5Fv5.34.36.source.tar.gz&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q9mk6yIytaNZlSdiLX_dFwchX8Tb7ra6x3WBBNIcs2o&s=Lz7YkqZ3XwjRsYIXVTbSvbDvTM-jTyoWvoVSa1PdBDw&e= The README file of xps does explain how to compile ROOT for Sierra. This should also be valid for El Capitan running XCode 8.x. Thank you in advance. Best regards, Christian _._._._._._._._._._._._._._._._._._ C.h.r.i.s.t.i.a.n S.t.r.a.t.o.w.a V.i.e.n.n.a A.u.s.t.r.i.a e.m.a.i.l: cstrato at aon.at _._._._._._._._._._._._._._._._._._ _______________________________________________ Bioc-devel@r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwICAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=q9mk6yIytaNZlSdiLX_dFwchX8Tb7ra6x3WBBNIcs2o&s=0bNMm-aoHuwWs9yBRjyGHTxT0y3UceNADHgMjtosTWU&e=-- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319 _______________________________________________ Bioc-devel@r-project.org mailing list https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel&d=DwIF-g&c=eRAMFD45gAfqt84VtBcfhQ&r=TF6f93hjWmgMzjqP9F3thRifibmFvfjc5Ae-bzNwDGo&m=WB1ofcLb-W4SN6VNAgoSRdgRXQRPaelptAH2g0Ur7q8&s=IDfsJGqV_D7hzqLryd27eoZNIuiAIfSNATUnxMy61oo&e=-- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319
-- Hervé Pagès Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319 _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
