[ccp4bb] Bug in c_truncate?
Hello, I've been struggling with F2MTZ and importing my hkl file into mtz by 'keeping existing freeR data'. I keep getting the error "Problem with FREE column in input file. All flags apparently identical. Check input file." At the end of the day, it appears that this only happens in ctruncate and not in the old_truncate instead of ctruncate. Has anyone experienced a similar problem? Peter
Re: [ccp4bb] Bug in c_truncate?
Dear Peter, it seems to me that you are having trouble with f2mtz and not with ctruncate, so I am confused by the subject. Can you please post - the error message, - the first couple of lines of the hkl-file you are trying to import (including one or two reflections which are flagged for Rfree), - the version of ccp4 you are using - whether you are doing the conversion from the GUI or the command line - if the latter, please also post the script you are using. Cheers, Tim On Wed, Oct 27, 2010 at 09:14:36PM -0400, Peter Chan wrote: > > Hello, > > I've been struggling with F2MTZ and importing my hkl file into mtz by > 'keeping existing freeR data'. I keep getting the error "Problem with FREE > column in input file. All flags apparently identical. Check input file." > > At the end of the day, it appears that this only happens in ctruncate and not > in the old_truncate instead of ctruncate. Has anyone experienced a similar > problem? > > Peter > -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
Re: [ccp4bb] Bug in c_truncate?
Dear Crystallographers, Thank you all for the emails. Below are some details of the procedures I performed leading up to the problem. The reflection file is my own data, processed in XDS and then flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. I used the GUI version of F2MTZ, with the settings below: - import file in SHELX format - "keep existing FreeR flags" - fortran format (3F4.0,2F8.3,F4.0) - added data label "I other integer" // FreeRflag The hkl file, in SHELX format, output by XPREP look something like this: -26 -3 1 777.48 39.19 26 -3 -1 800.83 36.31 -26 3 -1 782.67 37.97 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 Notice the test set is flagged "-1" and the working set is not flagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi: -26 -3 1 777.48 39.19 1 26 -3 -1 800.83 36.31 1 -26 3 -1 782.67 37.97 1 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 This is the file which gives me the error message: "Problem with FREE column in input file. All flags apparently identical. Check input file.". Apparently, import to mtz works ok when I use old-truncate instead of c-truncate. Best, Peter
Re: [ccp4bb] Bug in c_truncate?
Hello Peter, I faintly rememeber a similar kind of problem, and think that if you replace "-1" with "0", the problem should go away. It seemed that "-1" is not an allowed flag for (some) ccp4 programs. Please let us know if this resolves the issue. Tim On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > Dear Crystallographers, > > Thank you all for the emails. Below are some details of the procedures I > performed leading up to the problem. > > The reflection file is my own data, processed in XDS and then flagging > FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I > tried looking for known/resolved issues/updates in version 6.1.3 but could > not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. > > > I used the GUI version of F2MTZ, with the settings below: > > - import file in SHELX format > > - "keep existing FreeR flags" > > - fortran format (3F4.0,2F8.3,F4.0) > > - added data label "I other integer" // FreeRflag > > The hkl file, in SHELX format, output by XPREP look something like this: > > -26 -3 1 777.48 39.19 > 26 -3 -1 800.83 36.31 > -26 3 -1 782.67 37.97 > 27 -3 1 45.722 25.711 -1 > -27 3 1 -14.20 31.69 -1 > > Notice the test set is flagged "-1" and the working set is not flagged at > all. This actually lead to another error message in f2mtz about missing FreeR > flags. From my understanding, the SHELX flagging convention is "1" for > working and "-1" for test. So I manually tagged the working set with "1" > using vi: > > -26 -3 1 777.48 39.19 1 > 26 -3 -1 800.83 36.31 1 > -26 3 -1 782.67 37.97 1 > 27 -3 1 45.722 25.711 -1 > -27 3 1 -14.20 31.69 -1 > > This is the file which gives me the error message: "Problem with FREE column > in input file. All flags apparently identical. Check input file.". > Apparently, import to mtz works ok when I use old-truncate instead of > c-truncate. > > Best, > Peter > -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
Re: [ccp4bb] Bug in c_truncate?
Hello Tim, Thank you for the suggestion. I have now tagged the working set as "1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist. Best, Peter > Date: Thu, 28 Oct 2010 16:29:31 +0200 > From: t...@shelx.uni-ac.gwdg.de > Subject: Re: [ccp4bb] Bug in c_truncate? > To: CCP4BB@JISCMAIL.AC.UK > > Hello Peter, > > I faintly rememeber a similar kind of problem, and think that if you replace > "-1" with "0", the problem should go away. It seemed that "-1" is not an > allowed > flag for (some) ccp4 programs. > > Please let us know if this resolves the issue. > > Tim > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > Dear Crystallographers, > > > > Thank you all for the emails. Below are some details of the procedures I > > performed leading up to the problem. > > > > The reflection file is my own data, processed in XDS and then flagging > > FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. > > I tried looking for known/resolved issues/updates in version 6.1.3 but > > could not find any so I assumed it is the same version of > > f2mtz/ctruncate/uniqueify. > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > - import file in SHELX format > > > > - "keep existing FreeR flags" > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > - added data label "I other integer" // FreeRflag > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > -26 -3 1 777.48 39.19 > > 26 -3 -1 800.83 36.31 > > -26 3 -1 782.67 37.97 > > 27 -3 1 45.722 25.711 -1 > > -27 3 1 -14.20 31.69 -1 > > > > Notice the test set is flagged "-1" and the working set is not flagged at > > all. This actually lead to another error message in f2mtz about missing > > FreeR flags. From my understanding, the SHELX flagging convention is "1" > > for working and "-1" for test. So I manually tagged the working set with > > "1" using vi: > > > > -26 -3 1 777.48 39.19 1 > > 26 -3 -1 800.83 36.31 1 > > -26 3 -1 782.67 37.97 1 > > 27 -3 1 45.722 25.711 -1 > > -27 3 1 -14.20 31.69 -1 > > > > This is the file which gives me the error message: "Problem with FREE > > column in input file. All flags apparently identical. Check input file.". > > Apparently, import to mtz works ok when I use old-truncate instead of > > c-truncate. > > > > Best, > > Peter > > > -- > -- > Tim Gruene > Institut fuer anorganische Chemie > Tammannstr. 4 > D-37077 Goettingen > > phone: +49 (0)551 39 22149 > > GPG Key ID = A46BEE1A >
Re: [ccp4bb] Bug in c_truncate?
Why are you running [c]truncate? this is used to convert I -> F and I would be surprised if it recognised or preserved a FreeR column Phil On 28 Oct 2010, at 17:48, Peter Chan wrote: > Hello Tim, > > Thank you for the suggestion. I have now tagged the working set as "1" and > test set as "0". Unfortunately, it still gives the same error about all Rfree > being the same, and only in c-truncate but not old-truncate. Perhaps I should > install 6.1.3 and see if the problem still persist. > > Best, > Peter > > > Date: Thu, 28 Oct 2010 16:29:31 +0200 > > From: t...@shelx.uni-ac.gwdg.de > > Subject: Re: [ccp4bb] Bug in c_truncate? > > To: CCP4BB@JISCMAIL.AC.UK > > > > Hello Peter, > > > > I faintly rememeber a similar kind of problem, and think that if you replace > > "-1" with "0", the problem should go away. It seemed that "-1" is not an > > allowed > > flag for (some) ccp4 programs. > > > > Please let us know if this resolves the issue. > > > > Tim > > > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > > > > > > Dear Crystallographers, > > > > > > Thank you all for the emails. Below are some details of the procedures I > > > performed leading up to the problem. > > > > > > The reflection file is my own data, processed in XDS and then flagging > > > FreeR's in XPREP in thin resolution shells. I am using CCP4i version > > > 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 > > > but could not find any so I assumed it is the same version of > > > f2mtz/ctruncate/uniqueify. > > > > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > > > - import file in SHELX format > > > > > > - "keep existing FreeR flags" > > > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > > > - added data label "I other integer" // FreeRflag > > > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > > > -26 -3 1 777.48 39.19 > > > 26 -3 -1 800.83 36.31 > > > -26 3 -1 782.67 37.97 > > > 27 -3 1 45.722 25.711 -1 > > > -27 3 1 -14.20 31.69 -1 > > > > > > Notice the test set is flagged "-1" and the working set is not flagged at > > > all. This actually lead to another error message in f2mtz about missing > > > FreeR flags. From my understanding, the SHELX flagging convention is "1" > > > for working and "-1" for test. So I manually tagged the working set with > > > "1" using vi: > > > > > > -26 -3 1 777.48 39.19 1 > > > 26 -3 -1 800.83 36.31 1 > > > -26 3 -1 782.67 37.97 1 > > > 27 -3 1 45.722 25.711 -1 > > > -27 3 1 -14.20 31.69 -1 > > > > > > This is the file which gives me the error message: "Problem with FREE > > > column in input file. All flags apparently identical. Check input file.". > > > Apparently, import to mtz works ok when I use old-truncate instead of > > > c-truncate. > > > > > > Best, > > > Peter > > > > > -- > > -- > > Tim Gruene > > Institut fuer anorganische Chemie > > Tammannstr. 4 > > D-37077 Goettingen > > > > phone: +49 (0)551 39 22149 > > > > GPG Key ID = A46BEE1A > >
Re: [ccp4bb] Bug in c_truncate?
The GUI task has the option to run (c)truncate after f2mtz (if you have intensities in the input hkl file), and then uniqueify after that. I can reproduce this problem. ctruncate is losing the freeR column. At the moment, I don't know if this is a bug or a feature. As a work around, you can run ctruncate for the analyses, and re-run with truncate for the MTZ file. Tim is right, you need to use 0 instead of -1 in the CCP4 convention. HTH Martyn PS Refmac is moving towards using intensities, so that you can avoid this step. But I believe 5.5 only uses intensities for twin refinement. On Thu, 2010-10-28 at 17:55 +0100, Phil Evans wrote: > Why are you running [c]truncate? this is used to convert I -> F and I would > be surprised if it recognised or preserved a FreeR column > > Phil > > On 28 Oct 2010, at 17:48, Peter Chan wrote: > > > Hello Tim, > > > > Thank you for the suggestion. I have now tagged the working set as "1" and > > test set as "0". Unfortunately, it still gives the same error about all > > Rfree being the same, and only in c-truncate but not old-truncate. Perhaps > > I should install 6.1.3 and see if the problem still persist. > > > > Best, > > Peter > > > > > Date: Thu, 28 Oct 2010 16:29:31 +0200 > > > From: t...@shelx.uni-ac.gwdg.de > > > Subject: Re: [ccp4bb] Bug in c_truncate? > > > To: CCP4BB@JISCMAIL.AC.UK > > > > > > Hello Peter, > > > > > > I faintly rememeber a similar kind of problem, and think that if you > > > replace > > > "-1" with "0", the problem should go away. It seemed that "-1" is not an > > > allowed > > > flag for (some) ccp4 programs. > > > > > > Please let us know if this resolves the issue. > > > > > > Tim > > > > > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > > > > > > > > > > > Dear Crystallographers, > > > > > > > > Thank you all for the emails. Below are some details of the procedures > > > > I performed leading up to the problem. > > > > > > > > The reflection file is my own data, processed in XDS and then flagging > > > > FreeR's in XPREP in thin resolution shells. I am using CCP4i version > > > > 6.1.2. I tried looking for known/resolved issues/updates in version > > > > 6.1.3 but could not find any so I assumed it is the same version of > > > > f2mtz/ctruncate/uniqueify. > > > > > > > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > > > > > - import file in SHELX format > > > > > > > > - "keep existing FreeR flags" > > > > > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > > > > > - added data label "I other integer" // FreeRflag > > > > > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > > > > > -26 -3 1 777.48 39.19 > > > > 26 -3 -1 800.83 36.31 > > > > -26 3 -1 782.67 37.97 > > > > 27 -3 1 45.722 25.711 -1 > > > > -27 3 1 -14.20 31.69 -1 > > > > > > > > Notice the test set is flagged "-1" and the working set is not flagged > > > > at all. This actually lead to another error message in f2mtz about > > > > missing FreeR flags. From my understanding, the SHELX flagging > > > > convention is "1" for working and "-1" for test. So I manually tagged > > > > the working set with "1" using vi: > > > > > > > > -26 -3 1 777.48 39.19 1 > > > > 26 -3 -1 800.83 36.31 1 > > > > -26 3 -1 782.67 37.97 1 > > > > 27 -3 1 45.722 25.711 -1 > > > > -27 3 1 -14.20 31.69 -1 > > > > > > > > This is the file which gives me the error message: "Problem with FREE > > > > column in input file. All flags apparently identical. Check input > > > > file.". Apparently, import to mtz works ok when I use old-truncate > > > > instead of c-truncate. > > > > > > > > Best, > > > > Peter > > > > > > > -- > > > -- > > > Tim Gruene > > > Institut fuer anorganische Chemie > > > Tammannstr. 4 > > > D-37077 Goettingen > > > > > > phone: +49 (0)551 39 22149 > > > > > > GPG Key ID = A46BEE1A > > > -- *** * * * Dr. Martyn Winn * * * * STFC Daresbury Laboratory, Daresbury, Warrington, WA4 4AD, U.K. * * Tel: +44 1925 603455E-mail: martyn.w...@stfc.ac.uk* * Fax: +44 1925 603634Skype name: martyn.winn * * URL: http://www.ccp4.ac.uk/martyn/ * ***
Re: [ccp4bb] Bug in c_truncate?
Hello Peter, the easiest way to overcome the problem might be to use xprep to export to sca-format and use scalepack2mtz for the conversion. That seems to be the least hasslesome way, although I am not totally sure that this procedure preserves the R-free flags set by xprep. Tim On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: > > Hello Tim, > > Thank you for the suggestion. I have now tagged the working set as "1" and > test set as "0". Unfortunately, it still gives the same error about all Rfree > being the same, and only in c-truncate but not old-truncate. Perhaps I should > install 6.1.3 and see if the problem still persist. > > Best, > Peter > > > Date: Thu, 28 Oct 2010 16:29:31 +0200 > > From: t...@shelx.uni-ac.gwdg.de > > Subject: Re: [ccp4bb] Bug in c_truncate? > > To: CCP4BB@JISCMAIL.AC.UK > > > > Hello Peter, > > > > I faintly rememeber a similar kind of problem, and think that if you replace > > "-1" with "0", the problem should go away. It seemed that "-1" is not an > > allowed > > flag for (some) ccp4 programs. > > > > Please let us know if this resolves the issue. > > > > Tim > > > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > > > > > > Dear Crystallographers, > > > > > > Thank you all for the emails. Below are some details of the procedures I > > > performed leading up to the problem. > > > > > > The reflection file is my own data, processed in XDS and then flagging > > > FreeR's in XPREP in thin resolution shells. I am using CCP4i version > > > 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 > > > but could not find any so I assumed it is the same version of > > > f2mtz/ctruncate/uniqueify. > > > > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > > > - import file in SHELX format > > > > > > - "keep existing FreeR flags" > > > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > > > - added data label "I other integer" // FreeRflag > > > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > > > -26 -3 1 777.48 39.19 > > > 26 -3 -1 800.83 36.31 > > > -26 3 -1 782.67 37.97 > > > 27 -3 1 45.722 25.711 -1 > > > -27 3 1 -14.20 31.69 -1 > > > > > > Notice the test set is flagged "-1" and the working set is not flagged at > > > all. This actually lead to another error message in f2mtz about missing > > > FreeR flags. From my understanding, the SHELX flagging convention is "1" > > > for working and "-1" for test. So I manually tagged the working set with > > > "1" using vi: > > > > > > -26 -3 1 777.48 39.19 1 > > > 26 -3 -1 800.83 36.31 1 > > > -26 3 -1 782.67 37.97 1 > > > 27 -3 1 45.722 25.711 -1 > > > -27 3 1 -14.20 31.69 -1 > > > > > > This is the file which gives me the error message: "Problem with FREE > > > column in input file. All flags apparently identical. Check input file.". > > > Apparently, import to mtz works ok when I use old-truncate instead of > > > c-truncate. > > > > > > Best, > > > Peter > > > > > -- > > -- > > Tim Gruene > > Institut fuer anorganische Chemie > > Tammannstr. 4 > > D-37077 Goettingen > > > > phone: +49 (0)551 39 22149 > > > > GPG Key ID = A46BEE1A > > > -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
Re: [ccp4bb] Bug in c_truncate?
Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote: > Hello Peter, > > the easiest way to overcome the problem might be to use xprep to export to > sca-format and use scalepack2mtz for the conversion. That seems to be the > least > hasslesome way, although I am not totally sure that this procedure preserves > the > R-free flags set by xprep. > > Tim > > On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: > > > > Hello Tim, > > > > Thank you for the suggestion. I have now tagged the working set as "1" and > > test set as "0". Unfortunately, it still gives the same error about all > > Rfree being the same, and only in c-truncate but not old-truncate. Perhaps > > I should install 6.1.3 and see if the problem still persist. > > > > Best, > > Peter > > > > > Date: Thu, 28 Oct 2010 16:29:31 +0200 > > > From: t...@shelx.uni-ac.gwdg.de > > > Subject: Re: [ccp4bb] Bug in c_truncate? > > > To: CCP4BB@JISCMAIL.AC.UK > > > > > > Hello Peter, > > > > > > I faintly rememeber a similar kind of problem, and think that if you > > > replace > > > "-1" with "0", the problem should go away. It seemed that "-1" is not an > > > allowed > > > flag for (some) ccp4 programs. > > > > > > Please let us know if this resolves the issue. > > > > > > Tim > > > > > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > > > > > > > > > > > Dear Crystallographers, > > > > > > > > Thank you all for the emails. Below are some details of the procedures > > > > I performed leading up to the problem. > > > > > > > > The reflection file is my own data, processed in XDS and then flagging > > > > FreeR's in XPREP in thin resolution shells. I am using CCP4i version > > > > 6.1.2. I tried looking for known/resolved issues/updates in version > > > > 6.1.3 but could not find any so I assumed it is the same version of > > > > f2mtz/ctruncate/uniqueify. > > > > > > > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > > > > > - import file in SHELX format > > > > > > > > - "keep existing FreeR flags" > > > > > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > > > > > - added data label "I other integer" // FreeRflag > > > > > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > > > > > -26 -3 1 777.48 39.19 > > > > 26 -3 -1 800.83 36.31 > > > > -26 3 -1 782.67 37.97 > > > > 27 -3 1 45.722 25.711 -1 > > > > -27 3 1 -14.20 31.69 -1 > > > > > > > > Notice the test set is flagged "-1" and the working set is not flagged > > > > at all. This actually lead to another error message in f2mtz about > > > > missing FreeR flags. From my understanding, the SHELX flagging > > > > convention is "1" for working and "-1" for test. So I manually tagged > > > > the working set with "1" using vi: > > > > > > > > -26 -3 1 777.48 39.19 1 > > > > 26 -3 -1 800.83 36.31 1 > > > > -26 3 -1 782.67 37.97 1 > > > > 27 -3 1 45.722 25.711 -1 > > > > -27 3 1 -14.20 31.69 -1 > > > > > > > > This is the file which gives me the error message: "Problem with FREE > > > > column in input file. All flags apparently identical. Check input > > > > file.". Apparently, import to mtz works ok when I use old-truncate > > > > instead of c-truncate. > > > > > > > > Best, > > > > Peter > > > > > > > -- > > > -- > > > Tim Gruene > > > Institut fuer anorganische Chemie > > > Tammannstr. 4 > > > D-37077 Goettingen > > > > > > phone: +49 (0)551 39 22149 > > > > > > GPG Key ID = A46BEE1A > > > > > > -- > -- > Tim Gruene > Institut fuer anorganische Chemie > Tammannstr. 4 > D-37077 Goettingen > > phone: +49 (0)551 39 22149 > > GPG Key ID = A46BEE1A > >
Re: [ccp4bb] Bug in c_truncate?
Dear Peter, Since I did not hear that your problem is solved here my two cents. I did some tests using the ccp4i option "Convert Intensities to SFs" and found that here ctruncate completely ignored the FreeRflags. So my conclusion is that ctruncate does not need FreeRflags and you can use the following procedure: 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz without any special SHELX options. --> mtz 1 Careful: a FreeRflag of 1 means an unfree reflection and the free reflections have a FreeRflag of zero. 2) run ctruncate with the "Convert Intensities to SFs". You will loose your FreeRflags in this stage. --> mtz 2 3) add the FreeRflags from mtz 1 to mtz 2 using cad. If you wish, I can give you a command file which will do this. It is a somewhat roundabout procedure and I hope that this bug (or feature) will be fixed by the next release of ccp4. Best, Herman -Original Message- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of George M. Sheldrick Sent: Friday, October 29, 2010 12:30 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote: > Hello Peter, > > the easiest way to overcome the problem might be to use xprep to > export to sca-format and use scalepack2mtz for the conversion. That > seems to be the least hasslesome way, although I am not totally sure > that this procedure preserves the R-free flags set by xprep. > > Tim > > On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: > > > > Hello Tim, > > > > Thank you for the suggestion. I have now tagged the working set as "1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist. > > > > Best, > > Peter > > > > > Date: Thu, 28 Oct 2010 16:29:31 +0200 > > > From: t...@shelx.uni-ac.gwdg.de > > > Subject: Re: [ccp4bb] Bug in c_truncate? > > > To: CCP4BB@JISCMAIL.AC.UK > > > > > > Hello Peter, > > > > > > I faintly rememeber a similar kind of problem, and think that if > > > you replace "-1" with "0", the problem should go away. It seemed > > > that "-1" is not an allowed flag for (some) ccp4 programs. > > > > > > Please let us know if this resolves the issue. > > > > > > Tim > > > > > > On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > > > > > > > > > > > > > > > > > > > > Dear Crystallographers, > > > > > > > > Thank you all for the emails. Below are some details of the procedures I performed leading up to the problem. > > > > > > > > The reflection file is my own data, processed in XDS and then flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. > > > > > > > > > > > > I used the GUI version of F2MTZ, with the settings below: > > > > > > > > - import file in SHELX format > > > > > > > > - "keep existing FreeR flags" > > > > > > > > - fortran format (3F4.0,2F8.3,F4.0) > > > > > > > > - added data label "I other integer" // FreeRflag > > > > > > > > The hkl file, in SHELX format, output by XPREP look something like this: > > > > > > > > -26 -3 1 777.48 39.19 > > > > 26 -3 -1 800.83 36.31 > > > > -26 3 -1 782.67 37.97 > > > > 27 -3 1 45.722 25.711 -1 > > > > -27 3 1 -14.20 31.69 -1 > > > > > > > > Notice the test set is flagged "-1" and the working set is not flagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi: > > > &
Re: [ccp4bb] Bug in c_truncate?
The normal use of [c]truncate is to take intensities from Scala, so it wouldn't expect FreeR flags in the file. I suppose this should be added for other uses of the program Is this something that is often used? Do people import intensities into CCP4 to convert them to Fs? Phil On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: > Dear Peter, > > Since I did not hear that your problem is solved here my two cents. I > did some tests using the ccp4i option "Convert Intensities to SFs" and > found that here ctruncate completely ignored the FreeRflags. So my > conclusion is that ctruncate does not need FreeRflags and you can use > the following procedure: > > 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz > without any special SHELX options. --> mtz 1 > Careful: a FreeRflag of 1 means an unfree reflection and the free > reflections have a FreeRflag of zero. > 2) run ctruncate with the "Convert Intensities to SFs". You will loose > your FreeRflags in this stage. --> mtz 2 > 3) add the FreeRflags from mtz 1 to mtz 2 using cad. > > If you wish, I can give you a command file which will do this. It is a > somewhat roundabout procedure and I hope that this bug (or feature) will > be fixed by the next release of ccp4. > > Best, > Herman > > -Original Message- > From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of > George M. Sheldrick > Sent: Friday, October 29, 2010 12:30 PM > To: CCP4BB@JISCMAIL.AC.UK > Subject: Re: [ccp4bb] Bug in c_truncate? > > Tim, > > Although I always like to advocate XPREP, that would not work because > the .sca format - most unfortunately - does not know about free R flags. > > George > > Prof. George M. Sheldrick FRS > Dept. Structural Chemistry, > University of Goettingen, > Tammannstr. 4, > D37077 Goettingen, Germany > Tel. +49-551-39-3021 or -3068 > Fax. +49-551-39-22582 > > > On Fri, 29 Oct 2010, Tim Gruene wrote: > >> Hello Peter, >> >> the easiest way to overcome the problem might be to use xprep to >> export to sca-format and use scalepack2mtz for the conversion. That >> seems to be the least hasslesome way, although I am not totally sure >> that this procedure preserves the R-free flags set by xprep. >> >> Tim >> >> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>> >>> Hello Tim, >>> >>> Thank you for the suggestion. I have now tagged the working set as > "1" and test set as "0". Unfortunately, it still gives the same error > about all Rfree being the same, and only in c-truncate but not > old-truncate. Perhaps I should install 6.1.3 and see if the problem > still persist. >>> >>> Best, >>> Peter >>> >>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>> From: t...@shelx.uni-ac.gwdg.de >>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>> To: CCP4BB@JISCMAIL.AC.UK >>>> >>>> Hello Peter, >>>> >>>> I faintly rememeber a similar kind of problem, and think that if >>>> you replace "-1" with "0", the problem should go away. It seemed >>>> that "-1" is not an allowed flag for (some) ccp4 programs. >>>> >>>> Please let us know if this resolves the issue. >>>> >>>> Tim >>>> >>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: >>>>> >>>>> >>>>> >>>>> >>>>> Dear Crystallographers, >>>>> >>>>> Thank you all for the emails. Below are some details of the > procedures I performed leading up to the problem. >>>>> >>>>> The reflection file is my own data, processed in XDS and then > flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i > version 6.1.2. I tried looking for known/resolved issues/updates in > version 6.1.3 but could not find any so I assumed it is the same version > of f2mtz/ctruncate/uniqueify. >>>>> >>>>> >>>>> I used the GUI version of F2MTZ, with the settings below: >>>>> >>>>> - import file in SHELX format >>>>> >>>>> - "keep existing FreeR flags" >>>>> >>>>> - fortran format (3F4.0,2F8.3,F4.0) >>>>> >>>>> - added data label "I other integer" // FreeRflag >>>>> >>>>> The hkl file, in SHELX format, output by XPREP l
Re: [ccp4bb] Bug in c_truncate?
Hi Phil, I do, but the freeR flag problem is easily circumvented. One can just use cad with the output mtz from ctruncate and the input mtz: cad \ HKLIN2 $WORKDIR/raw.mtz \ HKLIN1 $WORKDIR/ctruncate.mtz \ HKLOUT $WORKDIR/ctruncate_withRfree.mtz \ <>$WORKDIR/mtz_creation.log LABIN FILE 2 E1=FREE LABIN FILE 1 ALLIN END eof Cheers, Robbie > Date: Fri, 29 Oct 2010 13:05:40 +0100 > From: p...@mrc-lmb.cam.ac.uk > Subject: Re: [ccp4bb] Bug in c_truncate? > To: CCP4BB@JISCMAIL.AC.UK > > The normal use of [c]truncate is to take intensities from Scala, so it > wouldn't expect FreeR flags in the file. > > I suppose this should be added for other uses of the program > > Is this something that is often used? Do people import intensities into CCP4 > to convert them to Fs? > > Phil > > > On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: > > > Dear Peter, > > > > Since I did not hear that your problem is solved here my two cents. I > > did some tests using the ccp4i option "Convert Intensities to SFs" and > > found that here ctruncate completely ignored the FreeRflags. So my > > conclusion is that ctruncate does not need FreeRflags and you can use > > the following procedure: > > > > 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz > > without any special SHELX options. --> mtz 1 > > Careful: a FreeRflag of 1 means an unfree reflection and the free > > reflections have a FreeRflag of zero. > > 2) run ctruncate with the "Convert Intensities to SFs". You will loose > > your FreeRflags in this stage. --> mtz 2 > > 3) add the FreeRflags from mtz 1 to mtz 2 using cad. > > > > If you wish, I can give you a command file which will do this. It is a > > somewhat roundabout procedure and I hope that this bug (or feature) will > > be fixed by the next release of ccp4. > > > > Best, > > Herman > > > > -Original Message- > > From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of > > George M. Sheldrick > > Sent: Friday, October 29, 2010 12:30 PM > > To: CCP4BB@JISCMAIL.AC.UK > > Subject: Re: [ccp4bb] Bug in c_truncate? > > > > Tim, > > > > Although I always like to advocate XPREP, that would not work because > > the .sca format - most unfortunately - does not know about free R flags. > > > > George > > > > Prof. George M. Sheldrick FRS > > Dept. Structural Chemistry, > > University of Goettingen, > > Tammannstr. 4, > > D37077 Goettingen, Germany > > Tel. +49-551-39-3021 or -3068 > > Fax. +49-551-39-22582 > > > > > > On Fri, 29 Oct 2010, Tim Gruene wrote: > > > >> Hello Peter, > >> > >> the easiest way to overcome the problem might be to use xprep to > >> export to sca-format and use scalepack2mtz for the conversion. That > >> seems to be the least hasslesome way, although I am not totally sure > >> that this procedure preserves the R-free flags set by xprep. > >> > >> Tim > >> > >> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: > >>> > >>> Hello Tim, > >>> > >>> Thank you for the suggestion. I have now tagged the working set as > > "1" and test set as "0". Unfortunately, it still gives the same error > > about all Rfree being the same, and only in c-truncate but not > > old-truncate. Perhaps I should install 6.1.3 and see if the problem > > still persist. > >>> > >>> Best, > >>> Peter > >>> > >>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 > >>>> From: t...@shelx.uni-ac.gwdg.de > >>>> Subject: Re: [ccp4bb] Bug in c_truncate? > >>>> To: CCP4BB@JISCMAIL.AC.UK > >>>> > >>>> Hello Peter, > >>>> > >>>> I faintly rememeber a similar kind of problem, and think that if > >>>> you replace "-1" with "0", the problem should go away. It seemed > >>>> that "-1" is not an allowed flag for (some) ccp4 programs. > >>>> > >>>> Please let us know if this resolves the issue. > >>>> > >>>> Tim > >>>> > >>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> Dear Crystallographers, > >>>>> > >
Re: [ccp4bb] Bug in c_truncate?
Dear Phil, I think ctruncate is used as a general tool to convert I's to F's, so it would be good to have an option to propagate FreeRflags. The workaround, of course, is to use cad for this. I have the feeling (did not check) that the problem of Peter arose because to ccp4i F2MTZ-SHELX gui was not adapted to the new ctruncate, so that is something which has to be repaired. Best Herman -Original Message- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of Phil Evans Sent: Friday, October 29, 2010 2:06 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? The normal use of [c]truncate is to take intensities from Scala, so it wouldn't expect FreeR flags in the file. I suppose this should be added for other uses of the program Is this something that is often used? Do people import intensities into CCP4 to convert them to Fs? Phil On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: > Dear Peter, > > Since I did not hear that your problem is solved here my two cents. I > did some tests using the ccp4i option "Convert Intensities to SFs" and > found that here ctruncate completely ignored the FreeRflags. So my > conclusion is that ctruncate does not need FreeRflags and you can use > the following procedure: > > 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz > without any special SHELX options. --> mtz 1 > Careful: a FreeRflag of 1 means an unfree reflection and the free > reflections have a FreeRflag of zero. > 2) run ctruncate with the "Convert Intensities to SFs". You will loose > your FreeRflags in this stage. --> mtz 2 > 3) add the FreeRflags from mtz 1 to mtz 2 using cad. > > If you wish, I can give you a command file which will do this. It is a > somewhat roundabout procedure and I hope that this bug (or feature) > will be fixed by the next release of ccp4. > > Best, > Herman > > -Original Message- > From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of > George M. Sheldrick > Sent: Friday, October 29, 2010 12:30 PM > To: CCP4BB@JISCMAIL.AC.UK > Subject: Re: [ccp4bb] Bug in c_truncate? > > Tim, > > Although I always like to advocate XPREP, that would not work because > the .sca format - most unfortunately - does not know about free R flags. > > George > > Prof. George M. Sheldrick FRS > Dept. Structural Chemistry, > University of Goettingen, > Tammannstr. 4, > D37077 Goettingen, Germany > Tel. +49-551-39-3021 or -3068 > Fax. +49-551-39-22582 > > > On Fri, 29 Oct 2010, Tim Gruene wrote: > >> Hello Peter, >> >> the easiest way to overcome the problem might be to use xprep to >> export to sca-format and use scalepack2mtz for the conversion. That >> seems to be the least hasslesome way, although I am not totally sure >> that this procedure preserves the R-free flags set by xprep. >> >> Tim >> >> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>> >>> Hello Tim, >>> >>> Thank you for the suggestion. I have now tagged the working set as > "1" and test set as "0". Unfortunately, it still gives the same error > about all Rfree being the same, and only in c-truncate but not > old-truncate. Perhaps I should install 6.1.3 and see if the problem > still persist. >>> >>> Best, >>> Peter >>> >>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>> From: t...@shelx.uni-ac.gwdg.de >>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>> To: CCP4BB@JISCMAIL.AC.UK >>>> >>>> Hello Peter, >>>> >>>> I faintly rememeber a similar kind of problem, and think that if >>>> you replace "-1" with "0", the problem should go away. It seemed >>>> that "-1" is not an allowed flag for (some) ccp4 programs. >>>> >>>> Please let us know if this resolves the issue. >>>> >>>> Tim >>>> >>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: >>>>> >>>>> >>>>> >>>>> >>>>> Dear Crystallographers, >>>>> >>>>> Thank you all for the emails. Below are some details of the > procedures I performed leading up to the problem. >>>>> >>>>> The reflection file is my own data, processed in XDS and then > flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i > version 6.1.2. I tried looking for known/resolved issues/updates in > version 6.1.3 but could not fi
Re: [ccp4bb] Bug in c_truncate?
Dear All, Thank you kindly for the replies and concerns over the problem I've experienced. I really appreciate them. I do not know how often the I -> F conversion is done in CCP4i's F2MTZ GUI, especially with the FreeR flags preserved. Nevertheless, as mentioned, the problem seems to arise because [c]truncate does not recognize the FreeR column, unlike old-truncate. Whether this is a bug or new feature, it would be great if [c]truncate is (re-)programmed to recognize the FreeR column for a more streamlined import/I -> F conversion/uniqueify process executed in F2MTZ. Also thanks for pointing out a relatively straight forward workaround using cad. Best, Peter
Re: [ccp4bb] Bug in c_truncate?
The way I do this is to use mtz2various which reads the SHELX output and (I hope) copes with its various idiosymcrasies, producing an mtz file with H k l I SigI FreeR This can then be fed to the ctruncate GUI You need - to request Ensure unique data... Copy FreeR from another mtz file Then the MTZin is the SHELX mtz conversion and the MTZ with FreeR is the same file.. I think it works although I have no data to test it. Eleanor On 10/29/2010 01:05 PM, Phil Evans wrote: The normal use of [c]truncate is to take intensities from Scala, so it wouldn't expect FreeR flags in the file. I suppose this should be added for other uses of the program Is this something that is often used? Do people import intensities into CCP4 to convert them to Fs? Phil On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: Dear Peter, Since I did not hear that your problem is solved here my two cents. I did some tests using the ccp4i option "Convert Intensities to SFs" and found that here ctruncate completely ignored the FreeRflags. So my conclusion is that ctruncate does not need FreeRflags and you can use the following procedure: 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz without any special SHELX options. --> mtz 1 Careful: a FreeRflag of 1 means an unfree reflection and the free reflections have a FreeRflag of zero. 2) run ctruncate with the "Convert Intensities to SFs". You will loose your FreeRflags in this stage. --> mtz 2 3) add the FreeRflags from mtz 1 to mtz 2 using cad. If you wish, I can give you a command file which will do this. It is a somewhat roundabout procedure and I hope that this bug (or feature) will be fixed by the next release of ccp4. Best, Herman -Original Message- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of George M. Sheldrick Sent: Friday, October 29, 2010 12:30 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote: Hello Peter, the easiest way to overcome the problem might be to use xprep to export to sca-format and use scalepack2mtz for the conversion. That seems to be the least hasslesome way, although I am not totally sure that this procedure preserves the R-free flags set by xprep. Tim On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: Hello Tim, Thank you for the suggestion. I have now tagged the working set as "1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist. Best, Peter Date: Thu, 28 Oct 2010 16:29:31 +0200 From: t...@shelx.uni-ac.gwdg.de Subject: Re: [ccp4bb] Bug in c_truncate? To: CCP4BB@JISCMAIL.AC.UK Hello Peter, I faintly rememeber a similar kind of problem, and think that if you replace "-1" with "0", the problem should go away. It seemed that "-1" is not an allowed flag for (some) ccp4 programs. Please let us know if this resolves the issue. Tim On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: Dear Crystallographers, Thank you all for the emails. Below are some details of the procedures I performed leading up to the problem. The reflection file is my own data, processed in XDS and then flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. I used the GUI version of F2MTZ, with the settings below: - import file in SHELX format - "keep existing FreeR flags" - fortran format (3F4.0,2F8.3,F4.0) - added data label "I other integer" // FreeRflag The hkl file, in SHELX format, output by XPREP look something like this: -26 -3 1 777.48 39.19 26 -3 -1 800.83 36.31 -26 3 -1 782.67 37.97 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 Notice the test set is flagged "-1" and the working set is not flagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi: -26 -3 1 777.48 39.19 1 26 -3 -1 800.83 36.31 1 -26 3 -1 782.67 37.97 1 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 This is the file which gives m
Re: [ccp4bb] Bug in c_truncate?
Phil Yes our processing pipeline absolutely has to be able to take in data from any internal (in-house X-ray or synchrotron) or external (PDB or collaborator's) source, including those where I, sigI and freeR flag are present. One of the first things I did was to modify truncate so it would pass through the freeR flag column. If the I/sigI are present I always strip out the F/sigF columns. So it seemed logical to run truncate as the very last step, e.g.: 1. sortmtz 2. scala Steps 1 & 2 only for internally collected or unmerged data. 3. refindexExternal merged data enters pipeline here: auto-re-index to reference. 4. cad Sort; put into standard a.u.; add freeR column from reference if not already present. 5. rescut My own prog for auto-determination of resolution cutoff based on shell & completeness. 6. truncate Apply resolution cutoff; if Is available convert to Fs. I always run steps 3-6 in that order. I always check that the resolution cutoff is sensible & if Is are available I always run truncate to ensure it's done properly (i.e. correct cell contents are specified). I'm still using truncate because AFAICS ctruncate couldn't handle freeR flags (maybe that's fixed now, maybe not). Also truncate produces a more informative N(Z) plot which shows the expected distribution for a twinned crystal (I believe this feature has now been added to ctruncate). Cheers -- Ian On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans wrote: > The normal use of [c]truncate is to take intensities from Scala, so it > wouldn't expect FreeR flags in the file. > > I suppose this should be added for other uses of the program > > Is this something that is often used? Do people import intensities into CCP4 > to convert them to Fs? > > Phil > > > On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: > >> Dear Peter, >> >> Since I did not hear that your problem is solved here my two cents. I >> did some tests using the ccp4i option "Convert Intensities to SFs" and >> found that here ctruncate completely ignored the FreeRflags. So my >> conclusion is that ctruncate does not need FreeRflags and you can use >> the following procedure: >> >> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >> without any special SHELX options. --> mtz 1 >> Careful: a FreeRflag of 1 means an unfree reflection and the free >> reflections have a FreeRflag of zero. >> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >> your FreeRflags in this stage. --> mtz 2 >> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >> >> If you wish, I can give you a command file which will do this. It is a >> somewhat roundabout procedure and I hope that this bug (or feature) will >> be fixed by the next release of ccp4. >> >> Best, >> Herman >> >> -Original Message- >> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of >> George M. Sheldrick >> Sent: Friday, October 29, 2010 12:30 PM >> To: CCP4BB@JISCMAIL.AC.UK >> Subject: Re: [ccp4bb] Bug in c_truncate? >> >> Tim, >> >> Although I always like to advocate XPREP, that would not work because >> the .sca format - most unfortunately - does not know about free R flags. >> >> George >> >> Prof. George M. Sheldrick FRS >> Dept. Structural Chemistry, >> University of Goettingen, >> Tammannstr. 4, >> D37077 Goettingen, Germany >> Tel. +49-551-39-3021 or -3068 >> Fax. +49-551-39-22582 >> >> >> On Fri, 29 Oct 2010, Tim Gruene wrote: >> >>> Hello Peter, >>> >>> the easiest way to overcome the problem might be to use xprep to >>> export to sca-format and use scalepack2mtz for the conversion. That >>> seems to be the least hasslesome way, although I am not totally sure >>> that this procedure preserves the R-free flags set by xprep. >>> >>> Tim >>> >>> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>>> >>>> Hello Tim, >>>> >>>> Thank you for the suggestion. I have now tagged the working set as >> "1" and test set as "0". Unfortunately, it still gives the same error >> about all Rfree being the same, and only in c-truncate but not >> old-truncate. Perhaps I should install 6.1.3 and see if the problem >> still persist. >>>> >>>> Best, >>>> Peter >>>> >>>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>>> From: t...@shelx.uni-ac.gwdg.de >>>>> Subject: R
Re: [ccp4bb] Bug in c_truncate?
herman.schreu...@sanofi-aventis.com wrote: >>> >>>> Dear Peter, >>>> >>>> Since I did not hear that your problem is solved here my two cents. I >>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>> found that here ctruncate completely ignored the FreeRflags. So my >>>> conclusion is that ctruncate does not need FreeRflags and you can use >>>> the following procedure: >>>> >>>> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >>>> without any special SHELX options. --> mtz 1 >>>> Careful: a FreeRflag of 1 means an unfree reflection and the free >>>> reflections have a FreeRflag of zero. >>>> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >>>> your FreeRflags in this stage. --> mtz 2 >>>> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >>>> >>>> If you wish, I can give you a command file which will do this. It is a >>>> somewhat roundabout procedure and I hope that this bug (or feature) will >>>> be fixed by the next release of ccp4. >>>> >>>> Best, >>>> Herman >>>> >>>> -Original Message- >>>> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of >>>> George M. Sheldrick >>>> Sent: Friday, October 29, 2010 12:30 PM >>>> To: CCP4BB@JISCMAIL.AC.UK >>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>> >>>> Tim, >>>> >>>> Although I always like to advocate XPREP, that would not work because >>>> the .sca format - most unfortunately - does not know about free R flags. >>>> >>>> George >>>> >>>> Prof. George M. Sheldrick FRS >>>> Dept. Structural Chemistry, >>>> University of Goettingen, >>>> Tammannstr. 4, >>>> D37077 Goettingen, Germany >>>> Tel. +49-551-39-3021 or -3068 >>>> Fax. +49-551-39-22582 >>>> >>>> >>>> On Fri, 29 Oct 2010, Tim Gruene wrote: >>>> >>>>> Hello Peter, >>>>> >>>>> the easiest way to overcome the problem might be to use xprep to >>>>> export to sca-format and use scalepack2mtz for the conversion. That >>>>> seems to be the least hasslesome way, although I am not totally sure >>>>> that this procedure preserves the R-free flags set by xprep. >>>>> >>>>> Tim >>>>> >>>>> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>>>>> >>>>>> Hello Tim, >>>>>> >>>>>> Thank you for the suggestion. I have now tagged the working set as >>>> "1" and test set as "0". Unfortunately, it still gives the same error >>>> about all Rfree being the same, and only in c-truncate but not >>>> old-truncate. Perhaps I should install 6.1.3 and see if the problem >>>> still persist. >>>>>> >>>>>> Best, >>>>>> Peter >>>>>> >>>>>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>>>>> From: t...@shelx.uni-ac.gwdg.de >>>>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>>>> To: CCP4BB@JISCMAIL.AC.UK >>>>>>> >>>>>>> Hello Peter, >>>>>>> >>>>>>> I faintly rememeber a similar kind of problem, and think that if >>>>>>> you replace "-1" with "0", the problem should go away. It seemed >>>>>>> that "-1" is not an allowed flag for (some) ccp4 programs. >>>>>>> >>>>>>> Please let us know if this resolves the issue. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Dear Crystallographers, >>>>>>>> >>>>>>>> Thank you all for the emails. Below are some details of the >>>> procedures I performed leading up to the problem. >>>>>>>> >>>>>>>> The reflection file is my own data, processed in XDS and then >>>
Re: [ccp4bb] Bug in c_truncate?
x27;t handle freeR flags (maybe that's fixed now, maybe not). Also >>> truncate produces a more informative N(Z) plot which shows the >>> expected distribution for a twinned crystal (I believe this feature >>> has now been added to ctruncate). >>> >>> Cheers >>> >>> -- Ian >>> >>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans wrote: >>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>> wouldn't expect FreeR flags in the file. >>>> >>>> I suppose this should be added for other uses of the program >>>> >>>> Is this something that is often used? Do people import intensities into >>>> CCP4 to convert them to Fs? >>>> >>>> Phil >>>> >>>> >>>> On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: >>>> >>>>> Dear Peter, >>>>> >>>>> Since I did not hear that your problem is solved here my two cents. I >>>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>>> found that here ctruncate completely ignored the FreeRflags. So my >>>>> conclusion is that ctruncate does not need FreeRflags and you can use >>>>> the following procedure: >>>>> >>>>> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >>>>> without any special SHELX options. --> mtz 1 >>>>> Careful: a FreeRflag of 1 means an unfree reflection and the free >>>>> reflections have a FreeRflag of zero. >>>>> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >>>>> your FreeRflags in this stage. --> mtz 2 >>>>> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >>>>> >>>>> If you wish, I can give you a command file which will do this. It is a >>>>> somewhat roundabout procedure and I hope that this bug (or feature) will >>>>> be fixed by the next release of ccp4. >>>>> >>>>> Best, >>>>> Herman >>>>> >>>>> -Original Message- >>>>> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of >>>>> George M. Sheldrick >>>>> Sent: Friday, October 29, 2010 12:30 PM >>>>> To: CCP4BB@JISCMAIL.AC.UK >>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>> >>>>> Tim, >>>>> >>>>> Although I always like to advocate XPREP, that would not work because >>>>> the .sca format - most unfortunately - does not know about free R flags. >>>>> >>>>> George >>>>> >>>>> Prof. George M. Sheldrick FRS >>>>> Dept. Structural Chemistry, >>>>> University of Goettingen, >>>>> Tammannstr. 4, >>>>> D37077 Goettingen, Germany >>>>> Tel. +49-551-39-3021 or -3068 >>>>> Fax. +49-551-39-22582 >>>>> >>>>> >>>>> On Fri, 29 Oct 2010, Tim Gruene wrote: >>>>> >>>>>> Hello Peter, >>>>>> >>>>>> the easiest way to overcome the problem might be to use xprep to >>>>>> export to sca-format and use scalepack2mtz for the conversion. That >>>>>> seems to be the least hasslesome way, although I am not totally sure >>>>>> that this procedure preserves the R-free flags set by xprep. >>>>>> >>>>>> Tim >>>>>> >>>>>> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>>>>>> >>>>>>> Hello Tim, >>>>>>> >>>>>>> Thank you for the suggestion. I have now tagged the working set as >>>>> "1" and test set as "0". Unfortunately, it still gives the same error >>>>> about all Rfree being the same, and only in c-truncate but not >>>>> old-truncate. Perhaps I should install 6.1.3 and see if the problem >>>>> still persist. >>>>>>> >>>>>>> Best, >>>>>>> Peter >>>>>>> >>>>>>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>>>>>> From: t...@shelx.uni-ac.gwdg.de >>>>>>>> Subject: Re: [ccp4bb] Bug in c_truncat
Re: [ccp4bb] Bug in c_truncate? apologies..
I suggested mtz2various for taking SHELX input which is nonsense - it actually goes the way.. Eleanor On 10/29/2010 01:05 PM, Phil Evans wrote: The normal use of [c]truncate is to take intensities from Scala, so it wouldn't expect FreeR flags in the file. I suppose this should be added for other uses of the program Is this something that is often used? Do people import intensities into CCP4 to convert them to Fs? Phil On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: Dear Peter, Since I did not hear that your problem is solved here my two cents. I did some tests using the ccp4i option "Convert Intensities to SFs" and found that here ctruncate completely ignored the FreeRflags. So my conclusion is that ctruncate does not need FreeRflags and you can use the following procedure: 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz without any special SHELX options. --> mtz 1 Careful: a FreeRflag of 1 means an unfree reflection and the free reflections have a FreeRflag of zero. 2) run ctruncate with the "Convert Intensities to SFs". You will loose your FreeRflags in this stage. --> mtz 2 3) add the FreeRflags from mtz 1 to mtz 2 using cad. If you wish, I can give you a command file which will do this. It is a somewhat roundabout procedure and I hope that this bug (or feature) will be fixed by the next release of ccp4. Best, Herman -Original Message- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of George M. Sheldrick Sent: Friday, October 29, 2010 12:30 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote: Hello Peter, the easiest way to overcome the problem might be to use xprep to export to sca-format and use scalepack2mtz for the conversion. That seems to be the least hasslesome way, although I am not totally sure that this procedure preserves the R-free flags set by xprep. Tim On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: Hello Tim, Thank you for the suggestion. I have now tagged the working set as "1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist. Best, Peter Date: Thu, 28 Oct 2010 16:29:31 +0200 From: t...@shelx.uni-ac.gwdg.de Subject: Re: [ccp4bb] Bug in c_truncate? To: CCP4BB@JISCMAIL.AC.UK Hello Peter, I faintly rememeber a similar kind of problem, and think that if you replace "-1" with "0", the problem should go away. It seemed that "-1" is not an allowed flag for (some) ccp4 programs. Please let us know if this resolves the issue. Tim On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: Dear Crystallographers, Thank you all for the emails. Below are some details of the procedures I performed leading up to the problem. The reflection file is my own data, processed in XDS and then flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. I used the GUI version of F2MTZ, with the settings below: - import file in SHELX format - "keep existing FreeR flags" - fortran format (3F4.0,2F8.3,F4.0) - added data label "I other integer" // FreeRflag The hkl file, in SHELX format, output by XPREP look something like this: -26 -3 1 777.48 39.19 26 -3 -1 800.83 36.31 -26 3 -1 782.67 37.97 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 Notice the test set is flagged "-1" and the working set is not flagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi: -26 -3 1 777.48 39.19 1 26 -3 -1 800.83 36.31 1 -26 3 -1 782.67 37.97 1 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 This is the file which gives me the error message: "Problem with FREE column in input file. All flags apparently identical. Check input file.". Apparently, import to mtz works ok when I use old-truncate instead of c-truncate. Best, Peter -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077
Re: [ccp4bb] Bug in c_truncate? - phase mods
did not hear that your problem is solved here my two cents. I did some tests using the ccp4i option "Convert Intensities to SFs" and found that here ctruncate completely ignored the FreeRflags. So my conclusion is that ctruncate does not need FreeRflags and you can use the following procedure: 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz without any special SHELX options. --> mtz 1 Careful: a FreeRflag of 1 means an unfree reflection and the free reflections have a FreeRflag of zero. 2) run ctruncate with the "Convert Intensities to SFs". You will loose your FreeRflags in this stage. --> mtz 2 3) add the FreeRflags from mtz 1 to mtz 2 using cad. If you wish, I can give you a command file which will do this. It is a somewhat roundabout procedure and I hope that this bug (or feature) will be fixed by the next release of ccp4. Best, Herman -Original Message- From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of George M. Sheldrick Sent: Friday, October 29, 2010 12:30 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] Bug in c_truncate? Tim, Although I always like to advocate XPREP, that would not work because the .sca format - most unfortunately - does not know about free R flags. George Prof. George M. Sheldrick FRS Dept. Structural Chemistry, University of Goettingen, Tammannstr. 4, D37077 Goettingen, Germany Tel. +49-551-39-3021 or -3068 Fax. +49-551-39-22582 On Fri, 29 Oct 2010, Tim Gruene wrote: Hello Peter, the easiest way to overcome the problem might be to use xprep to export to sca-format and use scalepack2mtz for the conversion. That seems to be the least hasslesome way, although I am not totally sure that this procedure preserves the R-free flags set by xprep. Tim On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: Hello Tim, Thank you for the suggestion. I have now tagged the working set as "1" and test set as "0". Unfortunately, it still gives the same error about all Rfree being the same, and only in c-truncate but not old-truncate. Perhaps I should install 6.1.3 and see if the problem still persist. Best, Peter Date: Thu, 28 Oct 2010 16:29:31 +0200 From: t...@shelx.uni-ac.gwdg.de Subject: Re: [ccp4bb] Bug in c_truncate? To: CCP4BB@JISCMAIL.AC.UK Hello Peter, I faintly rememeber a similar kind of problem, and think that if you replace "-1" with "0", the problem should go away. It seemed that "-1" is not an allowed flag for (some) ccp4 programs. Please let us know if this resolves the issue. Tim On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: Dear Crystallographers, Thank you all for the emails. Below are some details of the procedures I performed leading up to the problem. The reflection file is my own data, processed in XDS and then flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i version 6.1.2. I tried looking for known/resolved issues/updates in version 6.1.3 but could not find any so I assumed it is the same version of f2mtz/ctruncate/uniqueify. I used the GUI version of F2MTZ, with the settings below: - import file in SHELX format - "keep existing FreeR flags" - fortran format (3F4.0,2F8.3,F4.0) - added data label "I other integer" // FreeRflag The hkl file, in SHELX format, output by XPREP look something like this: -26 -3 1 777.48 39.19 26 -3 -1 800.83 36.31 -26 3 -1 782.67 37.97 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 Notice the test set is flagged "-1" and the working set is not flagged at all. This actually lead to another error message in f2mtz about missing FreeR flags. From my understanding, the SHELX flagging convention is "1" for working and "-1" for test. So I manually tagged the working set with "1" using vi: -26 -3 1 777.48 39.19 1 26 -3 -1 800.83 36.31 1 -26 3 -1 782.67 37.97 1 27 -3 1 45.722 25.711 -1 -27 3 1 -14.20 31.69 -1 This is the file which gives me the error message: "Problem with FREE column in input file. All flags apparently identical. Check input file.". Apparently, import to mtz works ok when I use old-truncate instead of c-truncate. Best, Peter -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A
Re: [ccp4bb] Bug in c_truncate? - phase mods
run truncate as the very last step, e.g.: >>>> >>>> 1. sortmtz >>>> 2. scala Steps 1& 2 only for internally collected or unmerged >>>> data. >>>> 3. refindex External merged data enters pipeline here: >>>> auto-re-index to reference. >>>> 4. cad Sort; put into standard a.u.; add freeR column from >>>> reference if not already present. >>>> 5. rescut My own prog for auto-determination of resolution cutoff >>>> based on shell & completeness. >>>> 6. truncate Apply resolution cutoff; if Is available convert to Fs. >>>> >>>> I always run steps 3-6 in that order. I always check that the >>>> resolution cutoff is sensible& if Is are available I always run >>>> truncate to ensure it's done properly (i.e. correct cell contents are >>>> specified). I'm still using truncate because AFAICS ctruncate >>>> couldn't handle freeR flags (maybe that's fixed now, maybe not). Also >>>> truncate produces a more informative N(Z) plot which shows the >>>> expected distribution for a twinned crystal (I believe this feature >>>> has now been added to ctruncate). >>>> >>>> Cheers >>>> >>>> -- Ian >>>> >>>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans >>>> wrote: >>>>> >>>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>>> wouldn't expect FreeR flags in the file. >>>>> >>>>> I suppose this should be added for other uses of the program >>>>> >>>>> Is this something that is often used? Do people import intensities into >>>>> CCP4 to convert them to Fs? >>>>> >>>>> Phil >>>>> >>>>> >>>>> On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: >>>>> >>>>>> Dear Peter, >>>>>> >>>>>> Since I did not hear that your problem is solved here my two cents. I >>>>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>>>> found that here ctruncate completely ignored the FreeRflags. So my >>>>>> conclusion is that ctruncate does not need FreeRflags and you can use >>>>>> the following procedure: >>>>>> >>>>>> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >>>>>> without any special SHELX options. --> mtz 1 >>>>>> Careful: a FreeRflag of 1 means an unfree reflection and the free >>>>>> reflections have a FreeRflag of zero. >>>>>> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >>>>>> your FreeRflags in this stage. --> mtz 2 >>>>>> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >>>>>> >>>>>> If you wish, I can give you a command file which will do this. It is a >>>>>> somewhat roundabout procedure and I hope that this bug (or feature) >>>>>> will >>>>>> be fixed by the next release of ccp4. >>>>>> >>>>>> Best, >>>>>> Herman >>>>>> >>>>>> -Original Message- >>>>>> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of >>>>>> George M. Sheldrick >>>>>> Sent: Friday, October 29, 2010 12:30 PM >>>>>> To: CCP4BB@JISCMAIL.AC.UK >>>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>>> >>>>>> Tim, >>>>>> >>>>>> Although I always like to advocate XPREP, that would not work because >>>>>> the .sca format - most unfortunately - does not know about free R >>>>>> flags. >>>>>> >>>>>> George >>>>>> >>>>>> Prof. George M. Sheldrick FRS >>>>>> Dept. Structural Chemistry, >>>>>> University of Goettingen, >>>>>> Tammannstr. 4, >>>>>> D37077 Goettingen, Germany >>>>>> Tel. +49-551-39-3021 or -3068 >>>>>> Fax. +49-551-39-22582 >>>>>> >>>>>> >>>>>> On Fri, 29 Oct 2010, Tim Gruene wrote: >>>>>> >>>>>>> Hello Pe
Re: [ccp4bb] Bug in c_truncate? - phase mods
gt;> round to working out appropriate phase shifts on reindexing >>>> >>>> Phil >>>> >>>> >>>> On 1 Nov 2010, at 13:17, Ian Tickle wrote: >>>> >>>>> Phil >>>>> >>>>> Yes our processing pipeline absolutely has to be able to take in data >>>>> from any internal (in-house X-ray or synchrotron) or external (PDB or >>>>> collaborator's) source, including those where I, sigI and freeR flag >>>>> are present. One of the first things I did was to modify truncate so >>>>> it would pass through the freeR flag column. If the I/sigI are >>>>> present I always strip out the F/sigF columns. So it seemed logical >>>>> to run truncate as the very last step, e.g.: >>>>> >>>>> 1. sortmtz >>>>> 2. scala Steps 1& 2 only for internally collected or unmerged >>>>> data. >>>>> 3. refindexExternal merged data enters pipeline here: >>>>> auto-re-index to reference. >>>>> 4. cad Sort; put into standard a.u.; add freeR column from >>>>> reference if not already present. >>>>> 5. rescut My own prog for auto-determination of resolution cutoff >>>>> based on shell & completeness. >>>>> 6. truncate Apply resolution cutoff; if Is available convert to Fs. >>>>> >>>>> I always run steps 3-6 in that order. I always check that the >>>>> resolution cutoff is sensible& if Is are available I always run >>>>> truncate to ensure it's done properly (i.e. correct cell contents are >>>>> specified). I'm still using truncate because AFAICS ctruncate >>>>> couldn't handle freeR flags (maybe that's fixed now, maybe not). Also >>>>> truncate produces a more informative N(Z) plot which shows the >>>>> expected distribution for a twinned crystal (I believe this feature >>>>> has now been added to ctruncate). >>>>> >>>>> Cheers >>>>> >>>>> -- Ian >>>>> >>>>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans >>>>> wrote: >>>>>> >>>>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>>>> wouldn't expect FreeR flags in the file. >>>>>> >>>>>> I suppose this should be added for other uses of the program >>>>>> >>>>>> Is this something that is often used? Do people import intensities into >>>>>> CCP4 to convert them to Fs? >>>>>> >>>>>> Phil >>>>>> >>>>>> >>>>>> On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: >>>>>> >>>>>>> Dear Peter, >>>>>>> >>>>>>> Since I did not hear that your problem is solved here my two cents. I >>>>>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>>>>> found that here ctruncate completely ignored the FreeRflags. So my >>>>>>> conclusion is that ctruncate does not need FreeRflags and you can use >>>>>>> the following procedure: >>>>>>> >>>>>>> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >>>>>>> without any special SHELX options. --> mtz 1 >>>>>>> Careful: a FreeRflag of 1 means an unfree reflection and the free >>>>>>> reflections have a FreeRflag of zero. >>>>>>> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >>>>>>> your FreeRflags in this stage. --> mtz 2 >>>>>>> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >>>>>>> >>>>>>> If you wish, I can give you a command file which will do this. It is a >>>>>>> somewhat roundabout procedure and I hope that this bug (or feature) >>>>>>> will >>>>>>> be fixed by the next release of ccp4. >>>>>>> >>>>>>> Best, >>>>>>> Herman >>>>>>> >>>>>>> -Original Message- >>>>>>> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On B
Re: [ccp4bb] Bug in c_truncate? - phase mods
ts elements change on re-indexing (that's what re-indexing >>>> means!). If the structure were in 1-D or 2-D exactly the same would >>>> apply: the 1- or 2-D elements would still in general change on >>>> transforming the reference frame so would be represented by 1- or 2-D >>>> vectors; the structure factors would still be invariant, thus >>>> illustrating the important difference between a real scalar and a 1-D >>>> vector, and between a complex scalar and a 2-D vector. >>>> >>>> Cheers >>>> >>>> --Ian >>>> >>>> On Mon, Nov 1, 2010 at 1:28 PM, Phil Evans wrote: >>>>> >>>>> I can see we need to make sure that data can come in at any point, as Is >>>>> of Fs >>>>> >>>>> Pointless can do automatic reindexing to a reference, and will preserve >>>>> all columns from a merged file, but can't cope with phases, as I've not >>>>> got >>>>> round to working out appropriate phase shifts on reindexing >>>>> >>>>> Phil >>>>> >>>>> >>>>> On 1 Nov 2010, at 13:17, Ian Tickle wrote: >>>>> >>>>>> Phil >>>>>> >>>>>> Yes our processing pipeline absolutely has to be able to take in data >>>>>> from any internal (in-house X-ray or synchrotron) or external (PDB or >>>>>> collaborator's) source, including those where I, sigI and freeR flag >>>>>> are present. One of the first things I did was to modify truncate so >>>>>> it would pass through the freeR flag column. If the I/sigI are >>>>>> present I always strip out the F/sigF columns. So it seemed logical >>>>>> to run truncate as the very last step, e.g.: >>>>>> >>>>>> 1. sortmtz >>>>>> 2. scala Steps 1& 2 only for internally collected or unmerged >>>>>> data. >>>>>> 3. refindex External merged data enters pipeline here: >>>>>> auto-re-index to reference. >>>>>> 4. cad Sort; put into standard a.u.; add freeR column from >>>>>> reference if not already present. >>>>>> 5. rescut My own prog for auto-determination of resolution cutoff >>>>>> based on shell & completeness. >>>>>> 6. truncate Apply resolution cutoff; if Is available convert to Fs. >>>>>> >>>>>> I always run steps 3-6 in that order. I always check that the >>>>>> resolution cutoff is sensible& if Is are available I always run >>>>>> truncate to ensure it's done properly (i.e. correct cell contents are >>>>>> specified). I'm still using truncate because AFAICS ctruncate >>>>>> couldn't handle freeR flags (maybe that's fixed now, maybe not). Also >>>>>> truncate produces a more informative N(Z) plot which shows the >>>>>> expected distribution for a twinned crystal (I believe this feature >>>>>> has now been added to ctruncate). >>>>>> >>>>>> Cheers >>>>>> >>>>>> -- Ian >>>>>> >>>>>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans >>>>>> wrote: >>>>>>> >>>>>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>>>>> wouldn't expect FreeR flags in the file. >>>>>>> >>>>>>> I suppose this should be added for other uses of the program >>>>>>> >>>>>>> Is this something that is often used? Do people import intensities into >>>>>>> CCP4 to convert them to Fs? >>>>>>> >>>>>>> Phil >>>>>>> >>>>>>> >>>>>>> On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: >>>>>>> >>>>>>>> Dear Peter, >>>>>>>> >>>>>>>> Since I did not hear that your problem is solved here my two cents. I >>>>>>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>>>>>> found that here ctruncate completely ignored the FreeRflags. So my >>>>>>>> conclusion is that ctruncate does not need F
Re: [ccp4bb] Bug in c_truncate? - phase mods
epresented by real scalars). The indices (whether >>>>> reflection or Miller!) obviously form a 3-D vector with integer >>>>> elements (unless of course you're interested in diffuse scattering >>>>> when they have to be reals). Either way, this is a vector because in >>>>> the general case (there will be exceptions for reflections on symmetry >>>>> axes) its elements change on re-indexing (that's what re-indexing >>>>> means!). If the structure were in 1-D or 2-D exactly the same would >>>>> apply: the 1- or 2-D elements would still in general change on >>>>> transforming the reference frame so would be represented by 1- or 2-D >>>>> vectors; the structure factors would still be invariant, thus >>>>> illustrating the important difference between a real scalar and a 1-D >>>>> vector, and between a complex scalar and a 2-D vector. >>>>> >>>>> Cheers >>>>> >>>>> --Ian >>>>> >>>>> On Mon, Nov 1, 2010 at 1:28 PM, Phil Evans wrote: >>>>>> >>>>>> I can see we need to make sure that data can come in at any point, as Is >>>>>> of Fs >>>>>> >>>>>> Pointless can do automatic reindexing to a reference, and will preserve >>>>>> all columns from a merged file, but can't cope with phases, as I've not >>>>>> got >>>>>> round to working out appropriate phase shifts on reindexing >>>>>> >>>>>> Phil >>>>>> >>>>>> >>>>>> On 1 Nov 2010, at 13:17, Ian Tickle wrote: >>>>>> >>>>>>> Phil >>>>>>> >>>>>>> Yes our processing pipeline absolutely has to be able to take in data >>>>>>> from any internal (in-house X-ray or synchrotron) or external (PDB or >>>>>>> collaborator's) source, including those where I, sigI and freeR flag >>>>>>> are present. One of the first things I did was to modify truncate so >>>>>>> it would pass through the freeR flag column. If the I/sigI are >>>>>>> present I always strip out the F/sigF columns. So it seemed logical >>>>>>> to run truncate as the very last step, e.g.: >>>>>>> >>>>>>> 1. sortmtz >>>>>>> 2. scala Steps 1& 2 only for internally collected or unmerged >>>>>>> data. >>>>>>> 3. refindexExternal merged data enters pipeline here: >>>>>>> auto-re-index to reference. >>>>>>> 4. cad Sort; put into standard a.u.; add freeR column from >>>>>>> reference if not already present. >>>>>>> 5. rescut My own prog for auto-determination of resolution cutoff >>>>>>> based on shell & completeness. >>>>>>> 6. truncate Apply resolution cutoff; if Is available convert to Fs. >>>>>>> >>>>>>> I always run steps 3-6 in that order. I always check that the >>>>>>> resolution cutoff is sensible& if Is are available I always run >>>>>>> truncate to ensure it's done properly (i.e. correct cell contents are >>>>>>> specified). I'm still using truncate because AFAICS ctruncate >>>>>>> couldn't handle freeR flags (maybe that's fixed now, maybe not). Also >>>>>>> truncate produces a more informative N(Z) plot which shows the >>>>>>> expected distribution for a twinned crystal (I believe this feature >>>>>>> has now been added to ctruncate). >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>> -- Ian >>>>>>> >>>>>>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans >>>>>>> wrote: >>>>>>>> >>>>>>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>>>>>> wouldn't expect FreeR flags in the file. >>>>>>>> >>>>>>>> I suppose this should be added for other uses of the program >>>>>>>> >>>>>>>> Is this something that is often used? Do people import intensities into >>>>>>>>
