Dear Dr Cornilescu, thank you very much for the immediate and thorough response. This indeed make a lot of sense now, in fact I was already suspecting that the weighting of RDCs was the culprit. I am glad we sorted this out.
As take home message I would suggest, a small comment to be added in the default refine.py script. The currently available script clearly serves as a platform to grasp how the program works but it is more suitable then for multiple RDCs dataset from multiple media. Given the majority of users use just NH( and maybe NC, CaCo) RDCs from one medium, i would find it helpful if this is stated in the script. In this case, the different weight factor would be crucial to be documented as well. This kind of direct support/development of XPLOR is what makes this software superior than others. Thank you very much. Leonidas On Oct 26, 2016, at 9:39 PM, Gabriel Cornilescu wrote: > Hi Leonidas, > over fitted > I think the refinement worked fine, it's just indeed overfitted single set of > NH RDCs. It's easy to get the gb1 backbone to fit a single set of NH RDCs by > small torsion angle changes (see attached plot refine_0_vs_start_model.png). > You can see how the backbone was locally deformed at residue Glu-15 to fit > the RDC you changed from -9 to +5 Hz. > > If you fit the original set of RDCs to one of the structures you calculated, > you'll see that xplor fitted very well the original RDCs and that the ones > you altered are large outliers: > calcTensor -aveType sum -normType none -plot -showRDCs tmv107_nh_orig.tbl > refine_0.sa > (see attached refine_0_fit_orig_rdc.png) > > Adding the bicelle NH RDCs (attached as gb1_two_media.tar.gz) will > substantially reduce over fitting either set (rmsds are around 0.8-1.0 Hz): > calcTensor -aveType sum -normType none -plot -showRDCs tmv107_nh.tbl > refine_0.sa > (refine_0_with_bicelle_rdc.png attached). Again Lys-31 (with RDC changed from > +8 to -8Hz) stands out as an outlier. > > > The weight of the RDCs was high in that example to enforce agreement with > lots of RDCs in two media (including lots of small long-range ones, with > larger exp. errors). > > When using a single set in a single medium you therefore need to reduce the > weight of the RDCs until the scatter in the calculated structures matches the > experimental errors, i.e. change: > rampedParams.append( MultRamp(0.05,5.0, "rdcs.setScale( VALUE )") ) > > to something like: > rampedParams.append( MultRamp(0.05,0.2, "rdcs.setScale( VALUE )") ) > > So basically you can fit efficiently your X-ray structure to a set of RDCs > (but make sure you don't over-fit it). If you have independent sets of RDCs > you can use them initially to cross validate your results and later include > them all in the refinement. > > Hope it makes sense, > Gabriel > > On 10/26/2016 09:16 AM, Emmanouilidis, Leonidas wrote: >> Dear Dr Cornilescu, >> >> please find attached the folder as you asked. If you need anythin else >> please let me know. >> >> kind regards, >> Leonidas Emmanouilidis >> >> >> >> ________________________________________ >> From: Gabriel Cornilescu <[email protected]> on behalf of Gabriel >> Cornilescu <[email protected]> >> Sent: Tuesday, October 25, 2016 8:53 PM >> To: Emmanouilidis, Leonidas >> Cc: [email protected]; Charles Schwieters >> Subject: Re: [Xplor-nih] refine.py script handling off RDCs >> >> If you can tar and zip the folder containing your test (except the huge >> xplor output file) and email it to me I will take a look. >> >> >> On 10/25/16 10:24 AM, Emmanouilidis, Leonidas wrote: >>> thank you very much for the suggestion. >>> >>> I have tried as you suggested, having random RDC values for residues >>> 16,29,31,39,48,53 of gb1. The result was the same. Converence with perfect >>> correlation obs/calc without really violations. >>> >>> >>> >>> >>> >>> >>> ________________________________________ >>> From: [email protected] [[email protected]] on behalf of Gabriel >>> Cornilescu [[email protected]] >>> Sent: Monday, October 24, 2016 11:18 PM >>> To: Emmanouilidis, Leonidas >>> Cc: [email protected]; Charles Schwieters >>> Subject: Re: [Xplor-nih] refine.py script handling off RDCs >>> >>> That doesn't tell much, it's equivalent to having a 6% larger alignment >>> tensor and the fit has to be identical. Try to either scramble a few >>> assignments or add/subtract a few Hertz from just a handful of RDCs to see >>> a difference. >>> >>> On Oct 24, 2016 3:52 PM, "Emmanouilidis, Leonidas" >>> <[email protected]<mailto:[email protected]>> wrote: >>> by random I mean addition of 6Hz to all HN-N rdcs. >>> >>> >>> >>> >>> On Oct 24, 2016, at 10:43 PM, Gabriel Cornilescu wrote: >>> >>> >>> When you say random, are you scrambling the assignments, changing the RDC >>> values randomly by 6%, or changing their associated errors? >>> >>> On Oct 24, 2016 3:30 PM, "Emmanouilidis, Leonidas" >>> <[email protected]<mailto:[email protected]>> wrote: >>> This is relative simple. >>> If one runs the tutorial-based gb1 RDC refinement protocol (refine.py) but >>> with hn RDCs values completely random (+6Hz noise to each), the outcome is >>> a seemingly successful refinement with no errors. >>> I did the test using just the hn RDCs. >>> >>> >>> >>> >>> >>> On Oct 24, 2016, at 10:11 PM, Charles Schwieters wrote: >>> >>>> Hello Leonidas-- >>>> >>>>> while I am trying to refine a crystal structure with RDCs using the >>>>> refine.py script, I came across a weird result. >>>>> >>>>> I had near perfect correlation between observed and calculated RDCs >>>>> values, which was fishy. After addition of 5Hz systematic noise to all >>>>> RDCs, i still had perfect correlation and nice convergence with no >>>>> violations(!). Initially i thought that there was something wrong with >>>>> my experimental setup (since i only used RDCs+dihedrals restrains), >>>>> but I had exactly the same result when I changed also the gb1 tutorial >>>>> RDCs values. >>>>> So to sum it up, I used the tutorial script (refine.py) and input >>>>> files for RDC refinement, regardless of how much I alter the RDCs >>>>> values (i tried up to +6Hz noise addition) I get always nice >>>>> convergence, perfect correlation obs/calc with no violations. >>>>> >>>>> Is this how it should be or do I miss/mess somthing? >>>>> >>>> Not sure what's going on- we'll need more details. >>>> >>>> best regards-- >>>> Charles >>> _______________________________________________ >>> Xplor-nih mailing list >>> [email protected]<mailto:[email protected]> >>> https://dcb.cit.nih.gov/mailman/listinfo/xplor-nih >>> >> -- >> >> Thanks, >> Gabriel >> >> https://sites.google.com/a/wisc.edu/gabrielc/ >> > > > <refine_0_vs_start_model.png><refine_0_fit_orig_rdc.png><refine_0_with_bicelle_rdc.png><gb1_two_media.tar.gz> _______________________________________________ Xplor-nih mailing list [email protected] https://dcb.cit.nih.gov/mailman/listinfo/xplor-nih
