Re: [COOT] use of RPATH in coot libraries
Hi Folks, As a word from a dumb user / poor soul who has to maintain the installations, the fact that you can just unpack coot and it will work ANYWHERE is absolutely excellent. Despite the size and complexity of the package it was one of the simplest for me to do a central installation of here at Diamond! Just in case anyone was seriously thinking about breaking this... Cheers, Graeme -Original Message- From: Mailing list for users of COOT Crystallographic Software [mailto:c...@jiscmail.ac.uk] On Behalf Of Kevin Cowtan Sent: 02 June 2009 09:17 To: COOT@JISCMAIL.AC.UK Subject: Re: use of RPATH in coot libraries Peter Keller wrote: On Mon, 1 Jun 2009, Kevin Cowtan wrote: OK, I understand, RPATH bad. But as far as I can tell we don't use RPATH. Coot uses LD_LIBRARY_PATH, which is only set by a wrapper script which launches the actual binary, and so the use of LD_LIBRARY_PATH is never exposed to the rest of the system (which I understand is the only problem with LD_LIBRARY_PATH). Not quite: if your application starts a subprocess and runs something else, that subprocess will inherit LD_LIBRARY_PATH as set in the wrapper. That might be fatal for whatever you are running in the subprocess (unless it too is invoked from a wrapper that overrides/unsets LD_LIBRARY_PATH). On the whole, I prefer to use rpath, because it makes it easier to harden an installation against the vagaries of what users have set in their environment - using an rpath like '$ORIGIN/../lib' to set a path relative to the location of the ELF can help with installation issues. OS X and Solaris both allow editing of the runpath inside an executable, so are streets ahead of Linux here. Which breaks the (currently rather desirable) behaviour that you can unpack our Coot packages anywhere you want and they will just work. I can see that might work well for people packaging Coot for a Linux distro, where it will always be installed in the same place, but it doesn't work for us. So we have two incompatible requirements on the build system for two different situations. Given the complexity of the build system as it stands, my suggestion is that you fork or replace it (although Paul may have different views). I'm afraid I don't have the time or expertise to get involved in that though. K This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Re: [COOT] use of RPATH in coot libraries
Hi Kevin, On Tue, Jun 02, 2009 at 09:16:54AM +0100, Kevin Cowtan wrote: On the whole, I prefer to use rpath, because it makes it easier to harden an installation against the vagaries of what users have set in their environment - using an rpath like '$ORIGIN/../lib' to set a path relative to the location of the ELF can help with installation issues. OS X and Solaris both allow editing of the runpath inside an executable, so are streets ahead of Linux here. Which breaks the (currently rather desirable) behaviour that you can unpack our Coot packages anywhere you want and they will just work. I think the point Peter is making that one can use rpath _and_ $ORIGIN to exactly get this behaviour _without_ the need to add LD_LIBRARY_PATH settings! So you can unpack Coot packages anywhere you want with all its additional libraries refered with a relative rpath '$ORIGIN/../lib'. So this usage of rpath and $ORIGIN does not break the current behaviour at all. On the contrary: it avoids problems with other programs that might get called from within Coot and that would currently inherit the LD_LIBRARY_PATH that is set in the coot wrapper ... since you don't need to set LD_LIBRAYR_PATH at all. I can see that might work well for people packaging Coot for a Linux distro, where it will always be installed in the same place, but it doesn't work for us. I guess this $ORIGIN feature isn't very widely known ... see e.g. http://www.scons.org/wiki/UsingOrigin http://linuxreviews.org/man/ld.so/ http://blogs.sun.com/dipol/entry/dynamic_libraries_rpath_and_mac http://www.codeshorts.ca/tag/osx/ It works great for us. Cheers Clemens -- *** * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com * * Global Phasing Ltd. * Sheraton House, Castle Park * Cambridge CB3 0AX, UK *-- * BUSTER Development Group (http://www.globalphasing.com) ***
[COOT] Coot with phenix.refine and buster output
Dear Bleeding-edge binary [1] Coot users, I would appreciate feedback on the representation of LINKs using the output of Buster and Phenix.refine. (The last time I saw this in action (I don't have such files myself) (and prior to the most recent fixes) it crashed.) Thanks, Paul. [1] rev 2037 or later
Re: [COOT] use of RPATH in coot libraries
Hi Kevin, On Tue, Jun 02, 2009 at 02:18:25PM +0100, Kevin Cowtan wrote: OK, that is pretty cool, looks like the correct technical solution for our relocatable binaries. Yes - Peter did some great analysis of all this on Linux and OsX. I can see implementation problems. It looks to me as though this will only work if you build in place. All the coot dependencies get built in their own trees and the installed later. I suspect therefore that the link step might not work, because the rpath will be wrong until the binaries are shifted into place. Maybe that can be dealt with using LD_LIBRARY_PATH while building (but I'm out of my depth here). Yes probably ... sounds sensible. I also suspect the packagers will then be unhappy: http://wiki.debian.org/RpathIssue Yes: for packaging something into a proper distro system I wouldn't go that way. But for a tar.gz file that just gets unpacked it is a solution. Cheers Clemens -- *** * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com * * Global Phasing Ltd. * Sheraton House, Castle Park * Cambridge CB3 0AX, UK *-- * BUSTER Development Group (http://www.globalphasing.com) ***
Re: [COOT] Coot fails to read numeric chain IDs
Edward Miller wrote: Hey Folks, I'm working on refining a full capsid containing 60 chains. Using the chain ID column, I've numbered my chains A-Z, a-z, and 0-7. I was successfully able to refine my capsid in refmac using this chain ID naming scheme. However, coot fails to read in any of the chains labeled 0-7. I don't know why this is failing - I will try to make a synthetic PDB file with 60 chains to see if I can reproduce this. Relatively recently Eugene has extended mmdb to work with the hybrid_36 format. So for now you could go that route. Paul.
Re: [COOT] use of RPATH in coot libraries
Interesting discussion, and I agree that being able to post edit the paths in OS X is very useful (@executable being the equivalent to $origin). The major problem though is libtool, which does not support (ASAIK) $ORIGIN. Further, libtool uses the information in the dependencies' .la files to set rpath for libraries up the chain. This is where coot is getting rpath set. Charles -Original Message- From: Mailing list for users of COOT Crystallographic Software [mailto:c...@jiscmail.ac.uk] On Behalf Of Clemens Vonrhein Sent: 02 June 2009 14:33 To: COOT@JISCMAIL.AC.UK Subject: Re: [COOT] use of RPATH in coot libraries Hi Kevin, On Tue, Jun 02, 2009 at 02:18:25PM +0100, Kevin Cowtan wrote: OK, that is pretty cool, looks like the correct technical solution for our relocatable binaries. Yes - Peter did some great analysis of all this on Linux and OsX. I can see implementation problems. It looks to me as though this will only work if you build in place. All the coot dependencies get built in their own trees and the installed later. I suspect therefore that the link step might not work, because the rpath will be wrong until the binaries are shifted into place. Maybe that can be dealt with using LD_LIBRARY_PATH while building (but I'm out of my depth here). Yes probably ... sounds sensible. I also suspect the packagers will then be unhappy: http://wiki.debian.org/RpathIssue Yes: for packaging something into a proper distro system I wouldn't go that way. But for a tar.gz file that just gets unpacked it is a solution. Cheers Clemens -- *** * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com * * Global Phasing Ltd. * Sheraton House, Castle Park * Cambridge CB3 0AX, UK *-- * BUSTER Development Group (http://www.globalphasing.com) *** -- Scanned by iCritical.
Re: [COOT] Coot fails to read numeric chain IDs
Hey Paul, I believe I have figured out the problem - but not the solution. On closer inspection, it wasn't just the numerically labelled chain IDs that weren't read, a few of the alphabetical chains were also not read. What happens is REFMAC restarts atom numbering once 10 atoms are reached when outputting the refined coordinates. COOT was reading in the first 9 atoms, numbered 1-9 by REFMAC, then the second 9 atoms, again numbered 1-9 by REFMAC. At this point, however, REFMAC for some reason made the next atom 10 and then restarted numbering at 1. This atom, numbered 10, was not read by COOT and nor where any subsequent atoms. The input coordinates that I used for REFMAC, which again consisted of 60 chains, was created by combining sets of 10mers, thus the atom numbering restarted after each 10 chains. REFMAC had no problem refining these coordinates and COOT had no problem reading these coordinates. To further confirm that COOT is failing once it reads atom 10, I wrote a little perl script to properly renumber the atoms in the REFMAC refined coordinates. In this file, there are correctly 246960 atoms. Within my perl script, I formatted the output such that the atom numbers eat into the spaces after ATOM like so: ATOM 246960 O LEU 7 736 64.294 96.393-111.576 1.00 34.97 I see now that the last atom COOT reads in is atom number 9. Perhaps, I fear, this is a problem with the PDB format - that the column width for atom numbers can not accommodate more than 9 atoms. Ed On Tue, Jun 2, 2009 at 9:39 AM, Paul Emsley paul.ems...@bioch.ox.ac.ukwrote: Edward Miller wrote: Hey Folks, I'm working on refining a full capsid containing 60 chains. Using the chain ID column, I've numbered my chains A-Z, a-z, and 0-7. I was successfully able to refine my capsid in refmac using this chain ID naming scheme. However, coot fails to read in any of the chains labeled 0-7. I don't know why this is failing - I will try to make a synthetic PDB file with 60 chains to see if I can reproduce this. Relatively recently Eugene has extended mmdb to work with the hybrid_36 format. So for now you could go that route. Paul.
Re: [COOT] Coot fails to read numeric chain IDs
Additionally, since this was asked before, I am running COOT 0.5.2 Ed On Tue, Jun 2, 2009 at 6:00 PM, Edward Miller emiller...@gmail.com wrote: Hey Paul, I believe I have figured out the problem - but not the solution. On closer inspection, it wasn't just the numerically labelled chain IDs that weren't read, a few of the alphabetical chains were also not read. What happens is REFMAC restarts atom numbering once 10 atoms are reached when outputting the refined coordinates. COOT was reading in the first 9 atoms, numbered 1-9 by REFMAC, then the second 9 atoms, again numbered 1-9 by REFMAC. At this point, however, REFMAC for some reason made the next atom 10 and then restarted numbering at 1. This atom, numbered 10, was not read by COOT and nor where any subsequent atoms. The input coordinates that I used for REFMAC, which again consisted of 60 chains, was created by combining sets of 10mers, thus the atom numbering restarted after each 10 chains. REFMAC had no problem refining these coordinates and COOT had no problem reading these coordinates. To further confirm that COOT is failing once it reads atom 10, I wrote a little perl script to properly renumber the atoms in the REFMAC refined coordinates. In this file, there are correctly 246960 atoms. Within my perl script, I formatted the output such that the atom numbers eat into the spaces after ATOM like so: ATOM 246960 O LEU 7 736 64.294 96.393-111.576 1.00 34.97 I see now that the last atom COOT reads in is atom number 9. Perhaps, I fear, this is a problem with the PDB format - that the column width for atom numbers can not accommodate more than 9 atoms. Ed On Tue, Jun 2, 2009 at 9:39 AM, Paul Emsley paul.ems...@bioch.ox.ac.ukwrote: Edward Miller wrote: Hey Folks, I'm working on refining a full capsid containing 60 chains. Using the chain ID column, I've numbered my chains A-Z, a-z, and 0-7. I was successfully able to refine my capsid in refmac using this chain ID naming scheme. However, coot fails to read in any of the chains labeled 0-7. I don't know why this is failing - I will try to make a synthetic PDB file with 60 chains to see if I can reproduce this. Relatively recently Eugene has extended mmdb to work with the hybrid_36 format. So for now you could go that route. Paul.
Re: [COOT] Coot fails to read numeric chain IDs
On Tuesday 02 June 2009 15:00:23 Edward Miller wrote: I see now that the last atom COOT reads in is atom number 9. Perhaps, I fear, this is a problem with the PDB format - that the column width for atom numbers can not accommodate more than 9 atoms. Ed Yes. This is one of many limitations of the PDB format. We've had this discussion before, and the rational option (abandon it in favor of something better) has been nixed by the Powers That Be. Note that the current PDB standard http://www.wwpdb.org/documentation/format32/sect9.html states If a collection contains more than 99,999 total atoms, then more than one entry must be made And, in fact, the PDB splits such large models into more than one PDB file upon deposition if you haven't already done so before hand. Nevertheless, many [most?] programs deal happily with this nonsense by ignoring the sequence field altogether. What earthly good does it do you to know what was the seqeunce number for atom NZ of Lys Z34 in a particular version of your model, given that it was almost certainly different after being processed by the next program used in the course of refinement or model-fitting. The situation is somewhat different for non-protein atoms, since the sequence number is used by the CONECT records. So I'll make a modest proposal to give all protein atoms sequence number 1, at leave the other 99,998 available integers for ligands :-) -- Ethan A Merritt Biomolecular Structure Center University of Washington, Seattle 98195-7742
