I cant think that this is the job for pointless.. it is responsible for intensities, and surely only needs to use a merged file to decide on the appropriate choice of axes - eg getting your new PG3 data set on the same indexing convention? Why would you need to modify the reference data?
Eleanor
On 11/01/2010 03:41 PM, Ian Tickle wrote:
Dealing with the phases (and therefore also the Hendrickson-Lattman coefficients) on re-indexing is trivial: the phases are not changed by re-indexing because the inverse transformation must simultaneously be applied to the co-ordinates. This is because in general the re-indexing transformation is not necessarily a symmetry operator (think of P1), so you can't rely on being able to compensate for the effect on the co-ordinates by using a symmetry operator. So the effect on the phases cancels out ... unless of course your re-indexing operator inverts the hand, in which case you almost certainly don't want also to invert the hand of your co-ordinates, so in that case you must compensate by transforming the structure factors to their complex conjugates (i.e. multiply phases by -1). I guess you're thinking of the subsequent necessary transformation of the indices to the asymmetric unit, where the phases& H-L coeffs do in general change (because then you are only changing the indices, not the co-ordinates); however CAD will do that transformation for you. Incidentally this is a neat illustration of the difference between a vector and a complex number. The re-indexing transformation is a transformation of the reference frame, which as long as it doesn't invert the hand, leaves the complex structure factors invariant, so they must be complex scalars (except in centric zones where they can sometimes be represented by real scalars). The indices (whether reflection or Miller!) obviously form a 3-D vector with integer elements (unless of course you're interested in diffuse scattering when they have to be reals). Either way, this is a vector because in the general case (there will be exceptions for reflections on symmetry axes) its elements change on re-indexing (that's what re-indexing means!). If the structure were in 1-D or 2-D exactly the same would apply: the 1- or 2-D elements would still in general change on transforming the reference frame so would be represented by 1- or 2-D vectors; the structure factors would still be invariant, thus illustrating the important difference between a real scalar and a 1-D vector, and between a complex scalar and a 2-D vector. Cheers --Ian On Mon, Nov 1, 2010 at 1:28 PM, Phil Evans<p...@mrc-lmb.cam.ac.uk> wrote:I can see we need to make sure that data can come in at any point, as Is of Fs Pointless can do automatic reindexing to a reference, and will preserve all columns from a merged file, but can't cope with phases, as I've not got round to working out appropriate phase shifts on reindexing Phil On 1 Nov 2010, at 13:17, Ian Tickle wrote:Phil Yes our processing pipeline absolutely has to be able to take in data from any internal (in-house X-ray or synchrotron) or external (PDB or collaborator's) source, including those where I, sigI and freeR flag are present. One of the first things I did was to modify truncate so it would pass through the freeR flag column. If the I/sigI are present I always strip out the F/sigF columns. So it seemed logical to run truncate as the very last step, e.g.: 1. sortmtz 2. scala Steps 1& 2 only for internally collected or unmerged data. 3. refindex External merged data enters pipeline here: auto-re-index to reference. 4. cad Sort; put into standard a.u.; add freeR column from reference if not already present. 5. rescut My own prog for auto-determination of resolution cutoff based on shell<I/sigI> & completeness. 6. truncate Apply resolution cutoff; if Is available convert to Fs. I always run steps 3-6 in that order. I always check that the resolution cutoff is sensible& if Is are available I always run truncate to ensure it's done properly (i.e. correct cell contents are specified). I'm still using truncate because AFAICS ctruncate couldn't handle freeR flags (maybe that's fixed now, maybe not). Also truncate produces a more informative N(Z) plot which shows the expected distribution for a twinned crystal (I believe this feature has now been added to ctruncate). Cheers -- Ian On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans<p...@mrc-lmb.cam.ac.uk> wrote:The normal use of [c]truncate is to take intensities from Scala, so it wouldn't expect FreeR flags in the file. I suppose this should be added for other uses of the program Is this something that is often used? Do people import intensities into CCP4 to convert them to Fs? Phil On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote:Dear Peter, Since I did not hear that your problem is solved here my two cents. I did some tests using the ccp4i option "Convert Intensities to SFs" and found that here ctruncate completely ignored the FreeRflags. So my conclusion is that ctruncate does not need FreeRflags and you can use the following procedure: 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz without any special SHELX options. --> mtz 1 Careful: a FreeRflag of 1 means an unfree reflection and the free reflections have a FreeRflag of zero. 2) run ctruncate with the "Convert Intensities to SFs". You will loose your FreeRflags in this stage. --> mtz 2 3) add the FreeRflags from mtz 1 to mtz 2 using cad. If you wish, I can give you a command file which will do this. It is a somewhat roundabout procedure and I hope that this bug (or feature) will be fixed by the next release of ccp4. Best, Herman -----Original Message----- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of George M. Sheldrick Sent: Friday, October 29, 2010 12:30 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote:Hello Peter, the easiest way to overcome the problem might be to use xprep to export to sca-format and use scalepack2mtz for the conversion. That seems to be the least hasslesome way, although I am not totally sure that this procedure preserves the R-free flags set by xprep. Tim On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote:Hello Tim, Thank you for the suggestion. I have now tagged the working set as"1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist.Best, PeterDate: Thu, 28 Oct 2010 16:29:31 +0200 From: t...@shelx.uni-ac.gwdg.de Subject: Re: [ccp4bb] Bug in c_truncate? To: CCP4BB@JISCMAIL.AC.UK Hello Peter, I faintly rememeber a similar kind of problem, and think that if you replace "-1" with "0", the problem should go away. It seemed that "-1" is not an allowed flag for (some) ccp4 programs. Please let us know if this resolves the issue. Tim On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote:Dear Crystallographers, Thank you all for the emails. Below are some details of theprocedures I performed leading up to the problem.The reflection file is my own data, processed in XDS and thenflagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify.I used the GUI version of F2MTZ, with the settings below: - import file in SHELX format - "keep existing FreeR flags" - fortran format (3F4.0,2F8.3,F4.0) - added data label "I other integer" // FreeRflag The hkl file, in SHELX format, output by XPREP look somethinglike this:-26 -3 1 777.48 39.19 26 -3 -1 800.83 36.31 -26 3 -1 782.67 37.97 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 Notice the test set is flagged "-1" and the working set is notflagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi:-26 -3 1 777.48 39.19 1 26 -3 -1 800.83 36.31 1 -26 3 -1 782.67 37.97 1 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 This is the file which gives me the error message: "Problem withFREE column in input file. All flags apparently identical. Check input file.". Apparently, import to mtz works ok when I use old-truncate instead of c-truncate.Best, Peter-- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A-- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A
