Re: [ccp4bb] R free flag
Hi Rashi, Rather than use the mtz file from EDS you should use he mmCIF reflection file from the PDB directly and convert it. If the authors deposited the test set, you can keep it this way. Otherwise you need to select a new test set. Note that if you want to optimize an existing PDB entry, the pdb-redo databank is a good starting point. It already did (part of) the work for you. Cheers, Robbie Sent from my Windows 10 phone From: Rashi Aggarwal<mailto:agg.rash...@gmail.com> Sent: woensdag 20 september 2017 08:49 To: CCP4BB@JISCMAIL.AC.UK<mailto:CCP4BB@JISCMAIL.AC.UK> Subject: [ccp4bb] R free flag Hello, I am trying to refine a structure that is deposited in PDB. I have downloaded the _sigmaa.mtz file from the electron density server. Trying to run refmac using this mtz file gives me the popup free R label has not been set. How can I set free R label in an existing mtz file so that after refinement I can get the R and Rfree? -Rashi
[ccp4bb] R free flag
Hello, I am trying to refine a structure that is deposited in PDB. I have downloaded the _sigmaa.mtz file from the electron density server. Trying to run refmac using this mtz file gives me the popup free R label has not been set. How can I set free R label in an existing mtz file so that after refinement I can get the R and Rfree? -Rashi
Re: [ccp4bb] R free flag missing after ARP/wARP?
Dear Lu, The MTZ file produced at the end of the ARP/wARP protein model building is created by Refmac and contains the following: 1. New columns created for the built model: (FC, PHIC, FC_ALL, PHIC_ALL, FWT, PHWT, DELFWT, PHDELWT, FOM, FC_ALL_LS, PHIC_ALL_LS) 2. Columns from the input data file that were used for the refinement - structure factor amplitudes and their estimated standard deviations scaled to those calculated from the model. If Rfree flag was used, the FREE column is also copied to the output MTZ file. Since you chose not to use Rfree for model building, the Rfree flag was not copied to the output. Historically, the default for ARP/wARP protein model building is not to use Rfree. Some years ago we did observe a number of cases where setting a part of the data aside for cross-validation resulted in lower observation-to-parameter ratio and poorer model building. We also thought that the auto-traced model (if built) could itself serve as a validation criterion. In contrast, for ARP/wARP building the solvent structure only, the use of Rfree is a default. Over the recent years much has improved, notably Refmac's likelihood targets. I am sure that in many cases the use of Rfree could now be advantageous for protein model building. We will try to repeat extensive benchmarking with different ARP/wARP protocols, so that many default parameters could be improved. With best regards, Victor On 20/10/2014 04:41, luzuok wrote: Dear all, I was using ARP/wARP in ccp4i, the input mtz file certainly has free R flag, but the output mzt file doesn't have the free R flag label? I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell me what's wrong with this? best reagrds! Lu Zuokun -- 卢作焜 南开大学新生物站A202
Re: [ccp4bb] R free flag missing after ARP/wARP?
Dear Lu Zuokun, The option you mention "do not use the Free R flag", tells ARP/wARP to do refinement without using the R-free flags, so it does not seem strange to me that it would output a file without those labels. I don't know why this would be a good idea, though, if you want to continue building on this model with those same R-free flags. The manual says: The default is not to use Rfree, since the number of traced residues serves as excellent indicator of the success of the job. You can certainly turn the use of Rfree on. (http://www.embl-hamburg.de/ARP/Manual/UserGuide7.4.html) Maybe the thought is that the free reflections will be uncoupled during further building and refinement? Best regards, Folmer Fredslund 2014-10-20 4:41 GMT+02:00 luzuok : > Dear all, > I was using ARP/wARP in ccp4i, the input mtz file certainly has free R > flag, but the output mzt file doesn't have the free R flag label? > I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell > me what's wrong with this? > > best reagrds! > Lu Zuokun > > > -- > 卢作焜 > 南开大学新生物站A202 > > > -- Folmer Fredslund
Re: [ccp4bb] R free flag missing after ARP/wARP?
When you chose the option “Do not use the Free R flag”, then you were telling Refmac not to use cross-validation and therefore to use all reflections as the working set. My experience (doubtless very limited compared to the ARP/wARP developers) is that it’s significantly better to use the Rfree flag, which makes sense since you need cross-validation data in order for likelihood targets to be calibrated properly. But maybe the developers have some set of test cases that led them to choose the opposite as the default. Best wishes, Randy Read On 20 Oct 2014, at 14:24, luzuok wrote: > > Dear Colin and Tim, > But ARPwARP uses REFMAC5 for refinement, does this means that REFMAC5 > uses all reflections as working set? > How to validate that the refinement is not over fit? > > Best wishes! > > Lu Zuokun > > > > -- > 卢作焜 > 南开大学新生物站A202 > > > At 2014-10-20 17:38:58, "Tim Gruene" wrote: > >Dear Lu Zuokun, > > > >since you can use the output mtz-file from ArpWarp for model building, > >but not for refinement, there is no need to include the Rfree flag in > >its output file. Maybe the omission is a deliberate caveat to the users > >to pay attention to this. > > > >Best regards, > >Tim > > > > >On 10/20/2014 04:41 AM, luzuok wrote: > >> Dear all, > >> I was using ARP/wARP in ccp4i, the input mtz file certainly has free R > >> flag, but the output mzt file doesn't have the free R flag label? > >> I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell > >> me what's wrong with this? > >> > >> > >> best reagrds! > >> Lu Zuokun > >> > >> > >> > >> -- > >> 卢作焜 > >> 南开大学新生物站A202 > >> > > > >-- > >Dr Tim Gruene > >Institut fuer anorganische Chemie > >Tammannstr. 4 > >D-37077 Goettingen > > > >GPG Key ID = A46BEE1A > > > > -- Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical Research Tel: + 44 1223 336500 Wellcome Trust/MRC Building Fax: + 44 1223 336827 Hills RoadE-mail: rj...@cam.ac.uk Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk
Re: [ccp4bb] R free flag missing after ARP/wARP?
Dear Colin and Tim, But ARPwARP uses REFMAC5 for refinement, does this means that REFMAC5 uses all reflections as working set? How to validate that the refinement is not over fit? Best wishes! Lu Zuokun -- 卢作焜 南开大学新生物站A202 At 2014-10-20 17:38:58, "Tim Gruene" wrote: >Dear Lu Zuokun, > >since you can use the output mtz-file from ArpWarp for model building, >but not for refinement, there is no need to include the Rfree flag in >its output file. Maybe the omission is a deliberate caveat to the users >to pay attention to this. > >Best regards, >Tim > >On 10/20/2014 04:41 AM, luzuok wrote: >> Dear all, >> I was using ARP/wARP in ccp4i, the input mtz file certainly has free R >> flag, but the output mzt file doesn't have the free R flag label? >> I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell >> me what's wrong with this? >> >> >> best reagrds! >> Lu Zuokun >> >> >> >> -- >> 卢作焜 >> 南开大学新生物站A202 >> > >-- >Dr Tim Gruene >Institut fuer anorganische Chemie >Tammannstr. 4 >D-37077 Goettingen > >GPG Key ID = A46BEE1A >
Re: [ccp4bb] R free flag missing after ARP/wARP?
Dear Lu Zuokun, since you can use the output mtz-file from ArpWarp for model building, but not for refinement, there is no need to include the Rfree flag in its output file. Maybe the omission is a deliberate caveat to the users to pay attention to this. Best regards, Tim On 10/20/2014 04:41 AM, luzuok wrote: > Dear all, > I was using ARP/wARP in ccp4i, the input mtz file certainly has free R > flag, but the output mzt file doesn't have the free R flag label? > I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell me > what's wrong with this? > > > best reagrds! > Lu Zuokun > > > > -- > 卢作焜 > 南开大学新生物站A202 > -- Dr Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen GPG Key ID = A46BEE1A signature.asc Description: OpenPGP digital signature
[ccp4bb] R free flag missing after ARP/wARP?
Dear all, I was using ARP/wARP in ccp4i, the input mtz file certainly has free R flag, but the output mzt file doesn't have the free R flag label? I chose " do not use the Free R flag" in the ARP/wARP GUI. Can anyone tell me what's wrong with this? best reagrds! Lu Zuokun -- 卢作焜 南开大学新生物站A202
Re: [ccp4bb] R-free flag problem
Dear Ed, It turns out that the format of the structure file I sent to the PDB guys is not right. I found out this when running phenix.model_vs_data. After I resent the structrue file, sfcheck gives something OK, which is, High_Res Low_Res Compl Num_Ref R_obs R_work R_free Corr(Fo-Fc) Reported PHENIX 1.695 36.657 95.51 46167 0.1488 0.1474 0.1742 N/A SFCHECK without TLS 1.69 39.42 95.9 45158 0.1604 0.162 0.183 0.9546 REFMAC without TLS 1.694 39.426 95.428 46172 0.148 0.1464 0.1736 0.964 PHENIX 1.69 36.66 96.0 46167 N/A 0.1475 0.1741 N/A Thank you all for your suggestions, Best wishes, Vinson 发件人： Ed Pozharski 收件人： CCP4BB@JISCMAIL.AC.UK 发送日期： 2010/10/23 (周六) 12:07:48 上午 主 题： Re: [ccp4bb] R-free flag problem I must note that it is also interesting to know why the sfcheck produced the R-factor that is twice as high. Of course, the R/Rfree are not supposed to match precisely (as I recall, sfcheck uses Babinet bulk solvent correction which is presumably inferior to the mask-based approach), but 28.6 versus 14.7? Something may be seriously wrong. On Fri, 2010-10-22 at 09:37 +0200, Tim Gruene wrote: > Dear Vinson, > > I would still be interested about what was the cause of the missing R-free >flags > and how you resolved it (I hope I did not miss an email). > > Kind regards, Tim > > On Fri, Oct 22, 2010 at 02:33:03PM +0800, Vinson LIANG wrote: > > > > > > Dear all, > > > > > > > > > > Thank you very much for your suggestions. > > > > I try Pavel's easy method. Hopefully, the result from phenix.model_vs_data >would be enough for PDB. > > > > > > Best wishes, > > > > Vinson > > > > > > > > > > > > > > > > > > 发件人： Pavel Afonine > > 收件人： PHENIX user mailing list > > 抄 送： Vinson LIANG > > 发送日期： 2010/10/22 (周五) 12:44:58 上午 > > 主 题： Re: [phenixbb] R-free problem with phenix.refine > > > > Hi, > > > > > > > > > > Recently I deposited a structure to PDB. However, I was told > > >> The R value of SF check are different from your reported value. > > >> > > >> R work in SF check = 0.2859 > > >> your reported R work = 0.1474 > > >> Also there are no R free flags in your sending files, but you reported R > > >> free value in your all deposition. > > well, SF check is not the best tool to check things - it's not just my only >vision, see lots of conversations on this topic on bb. > > > > Just run > > > > phenix.model_vs_data model.pdb data.hkl > > > > and if output statistics matches the one you deposited to PDB, then you are >100% good. If it doesn't match then there is something to worry about. > > > > > > > > So, I rerun my deposited file with phenix.refine, and it gave a >R-work=0.1475, R-free=0.1742. > > > > > Sounds good. > > > > > > > > However, when I tried to run the same file with refmac5, error came with >"Free R label has not been set". I guess phenix.refine gives Rree-R label that >other program could not recognize. > > > > > No surprises. Jumping between programs is tricky. You have to account for a >lot of things to match things that different programs expect differently. > > > > > > > > Could anyone please help me to solve this problem? > > Step-by-step: > > > > 1) Run > > phenix.model_vs_data model.pdb data.hkl > > > > 2) if output statistics matches the one you deposited to PDB, then you are >100% good. Come back to PDB and say this. > > > > > 3) If it doesn't match then there is something that you need to fix. If you >can't fix or have no idea what's wrong, you can send me the data and model and >I >will have a look. > > > > Pavel. > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Edwin Pozharski, PhD, Assistant Professor University of Maryland, Baltimore -- When the Way is forgotten duty and justice appear; Then knowledge and wisdom are born along with hypocrisy. When harmonious relationships dissolve then respect and devotion arise; When a nation falls to chaos then loyalty and patriotism are born. -- / Lao Tse /
Re: [ccp4bb] R-free flag problem
I must note that it is also interesting to know why the sfcheck produced the R-factor that is twice as high. Of course, the R/Rfree are not supposed to match precisely (as I recall, sfcheck uses Babinet bulk solvent correction which is presumably inferior to the mask-based approach), but 28.6 versus 14.7? Something may be seriously wrong. On Fri, 2010-10-22 at 09:37 +0200, Tim Gruene wrote: > Dear Vinson, > > I would still be interested about what was the cause of the missing R-free > flags > and how you resolved it (I hope I did not miss an email). > > Kind regards, Tim > > On Fri, Oct 22, 2010 at 02:33:03PM +0800, Vinson LIANG wrote: > > > > > > Dear all, > > > > > > > > > > Thank you very much for your suggestions. > > > > I try Pavel's easy method. Hopefully, the result from phenix.model_vs_data > > would be enough for PDB. > > > > > > Best wishes, > > > > Vinson > > > > > > > > > > > > > > > > > > 发件人： Pavel Afonine > > 收件人： PHENIX user mailing list > > 抄 送： Vinson LIANG > > 发送日期： 2010/10/22 (周五) 12:44:58 上午 > > 主 题： Re: [phenixbb] R-free problem with phenix.refine > > > > Hi, > > > > > > > > > > Recently I deposited a structure to PDB. However, I was told > > >> The R value of SF check are different from your reported value. > > >> > > >> R work in SF check = 0.2859 > > >> your reported R work = 0.1474 > > >> Also there are no R free flags in your sending files, but you reported R > > >> free value in your all deposition. > > well, SF check is not the best tool to check things - it's not just my only > > vision, see lots of conversations on this topic on bb. > > > > Just run > > > > phenix.model_vs_data model.pdb data.hkl > > > > and if output statistics matches the one you deposited to PDB, then you are > > 100% good. If it doesn't match then there is something to worry about. > > > > > > > > So, I rerun my deposited file with phenix.refine, and it gave a > > R-work=0.1475, R-free=0.1742. > > > > Sounds good. > > > > > > > > However, when I tried to run the same file with refmac5, error came with > > "Free R label has not been set". I guess phenix.refine gives Rree-R label > > that other program could not recognize. > > > > No surprises. Jumping between programs is tricky. You have to account for a > > lot of things to match things that different programs expect differently. > > > > > > > > Could anyone please help me to solve this problem? > > Step-by-step: > > > > 1) Run > > phenix.model_vs_data model.pdb data.hkl > > > > 2) if output statistics matches the one you deposited to PDB, then you are > > 100% good. Come back to PDB and say this. > > > > 3) If it doesn't match then there is something that you need to fix. If you > > can't fix or have no idea what's wrong, you can send me the data and model > > and I will have a look. > > > > Pavel. > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- Edwin Pozharski, PhD, Assistant Professor University of Maryland, Baltimore -- When the Way is forgotten duty and justice appear; Then knowledge and wisdom are born along with hypocrisy. When harmonious relationships dissolve then respect and devotion arise; When a nation falls to chaos then loyalty and patriotism are born. -- / Lao Tse /
Re: [ccp4bb] R-free flag problem
Dear Vinson, I would still be interested about what was the cause of the missing R-free flags and how you resolved it (I hope I did not miss an email). Kind regards, Tim On Fri, Oct 22, 2010 at 02:33:03PM +0800, Vinson LIANG wrote: > > > Dear all, > > > > > Thank you very much for your suggestions. > > I try Pavel's easy method. Hopefully, the result from phenix.model_vs_data > would be enough for PDB. > > > Best wishes, > > Vinson > > > > > > > > > 发件人： Pavel Afonine > 收件人： PHENIX user mailing list > 抄 送： Vinson LIANG > 发送日期： 2010/10/22 (周五) 12:44:58 上午 > 主 题： Re: [phenixbb] R-free problem with phenix.refine > > Hi, > > > > > Recently I deposited a structure to PDB. However, I was told > >> The R value of SF check are different from your reported value. > >> > >> R work in SF check = 0.2859 > >> your reported R work = 0.1474 > >> Also there are no R free flags in your sending files, but you reported R > >> free value in your all deposition. > well, SF check is not the best tool to check things - it's not just my only > vision, see lots of conversations on this topic on bb. > > Just run > > phenix.model_vs_data model.pdb data.hkl > > and if output statistics matches the one you deposited to PDB, then you are > 100% good. If it doesn't match then there is something to worry about. > > > > So, I rerun my deposited file with phenix.refine, and it gave a > R-work=0.1475, R-free=0.1742. > > Sounds good. > > > > However, when I tried to run the same file with refmac5, error came with > "Free R label has not been set". I guess phenix.refine gives Rree-R label > that other program could not recognize. > > No surprises. Jumping between programs is tricky. You have to account for a > lot of things to match things that different programs expect differently. > > > > Could anyone please help me to solve this problem? > Step-by-step: > > 1) Run > phenix.model_vs_data model.pdb data.hkl > > 2) if output statistics matches the one you deposited to PDB, then you are > 100% good. Come back to PDB and say this. > > 3) If it doesn't match then there is something that you need to fix. If you > can't fix or have no idea what's wrong, you can send me the data and model > and I will have a look. > > Pavel. > > > > > > > > > > > > > > -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
Re: [ccp4bb] R-free flag problem
Dear all, Thank you very much for your suggestions. I try Pavel's easy method. Hopefully, the result from phenix.model_vs_data would be enough for PDB. Best wishes, Vinson 发件人： Pavel Afonine 收件人： PHENIX user mailing list 抄 送： Vinson LIANG 发送日期： 2010/10/22 (周五) 12:44:58 上午 主 题： Re: [phenixbb] R-free problem with phenix.refine Hi, Recently I deposited a structure to PDB. However, I was told >> The R value of SF check are different from your reported value. >> >> R work in SF check = 0.2859 >> your reported R work = 0.1474 >> Also there are no R free flags in your sending files, but you reported R >> free value in your all deposition. well, SF check is not the best tool to check things - it's not just my only vision, see lots of conversations on this topic on bb. Just run phenix.model_vs_data model.pdb data.hkl and if output statistics matches the one you deposited to PDB, then you are 100% good. If it doesn't match then there is something to worry about. So, I rerun my deposited file with phenix.refine, and it gave a R-work=0.1475, R-free=0.1742. Sounds good. However, when I tried to run the same file with refmac5, error came with "Free R label has not been set". I guess phenix.refine gives Rree-R label that other program could not recognize. No surprises. Jumping between programs is tricky. You have to account for a lot of things to match things that different programs expect differently. Could anyone please help me to solve this problem? Step-by-step: 1) Run phenix.model_vs_data model.pdb data.hkl 2) if output statistics matches the one you deposited to PDB, then you are 100% good. Come back to PDB and say this. 3) If it doesn't match then there is something that you need to fix. If you can't fix or have no idea what's wrong, you can send me the data and model and I will have a look. Pavel.
Re: [ccp4bb] R-free flag problem.
Dear Vinson, Liang have a look at the penultimate table from mtzdmp your_mtz_file.mtz where your_mtz_file.mtz is the mtz-file you are using to deposit/ for refinement. That second last table lists the columns which are present in your mtz-file, their types and names and should give a clue what phenix uses as Rfree-flag and why refmac and sfcheck do not recognise it. You can send us that table if you are not sure how to interpret the data it presents. Tim On Thu, Oct 21, 2010 at 01:50:04PM +0800, Vinson LIANG wrote: > Dear all, > > Recently I deposited a structure to PDB. However, I was told > >> The R value of SF check are different from your reported value. > >> > >> R work in SF check = 0.2859 > >> your reported R work = 0.1474 > >> Also there are no R free flags in your sending files, but you reported R > >> free value in your all deposition. > > > > So, I rerun my deposited file with phenix.refine, and it gave a > R-work=0.1475, R-free=0.1742. > > However, when I tried to run the same file with refmac5, error came with > "Free R label has not been set". I guess phenix.refine gives Rree-R label > that other program could not recognize. > > Could anyone please help me to solve this problem? > > Thank you very much in advance, > > Best, > > Vinson, Liang > > > > > > > -- -- Tim Gruene Institut fuer anorganische Chemie Tammannstr. 4 D-37077 Goettingen phone: +49 (0)551 39 22149 GPG Key ID = A46BEE1A signature.asc Description: Digital signature
[ccp4bb] R-free flag problem.
Dear all, Recently I deposited a structure to PDB. However, I was told >> The R value of SF check are different from your reported value. >> >> R work in SF check = 0.2859 >> your reported R work = 0.1474 >> Also there are no R free flags in your sending files, but you reported R >> free value in your all deposition. So, I rerun my deposited file with phenix.refine, and it gave a R-work=0.1475, R-free=0.1742. However, when I tried to run the same file with refmac5, error came with "Free R label has not been set". I guess phenix.refine gives Rree-R label that other program could not recognize. Could anyone please help me to solve this problem? Thank you very much in advance, Best, Vinson, Liang
