Hi Troels, Before I delete them, apart from the R2eff file do we need the results files in the test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/check_graphs/mc_2000/*/ directories? These directories take up too much room, mainly because of the interpolated Grace files in the final directory. Is this information required? Or do you plan on using it in the future?
Cheers, Edward On 21 August 2014 10:02, Edward d'Auvergne <[email protected]> wrote: > Hi Troels, > > I've now converted your code to handle the R1 parameter dynamically, > so that is it part of the standard models. Therefore the '* R1 fit' > models have been removed. Some parts using the MODEL_LIST_*_W_R1 and > MODEL_LIST_*_FIT_R1 were a bit more complicated than expected to sort > out. But most things should now operate fine. The > Relax_disp.test_r1rho_kjaergaard_missing_r1 system test is still > problematic as it is using the pre-run directory and loading the saved > states of the models whereby R1 has not been optimised, rather than > using model nesting as it should, but I should have a solution for > that soon. Anyway, this is only a small amount of work which uses the > infrastructure you added to do everything. > > Regards, > > Edward > > On 20 August 2014 19:36, Troels Emtekær Linnet <[email protected]> wrote: >> Hi Edward. >> >> I trust that you know how to better plan programming than me. >> >> Please convert if you feel that this is a necessary step. >> >> Best >> Troels >> >> 2014-08-20 19:25 GMT+02:00 Edward d'Auvergne <[email protected]>: >>> I have just obliterated the wall. I have implemented: >>> >>> 1) The relax_disp.r1_fit user function >>> (http://thread.gmane.org/gmane.science.nmr.relax.scm/22858). >>> 2) The specific_analyses.relax_disp.data.is_r1_optimsed() function >>> for asking if we should optimise R1 or not >>> (http://thread.gmane.org/gmane.science.nmr.relax.scm/22857). >>> 3) The specific_analyses.relax_disp.parameters.r1_setup() function >>> for modifying the model parameter list as needed >>> (http://thread.gmane.org/gmane.science.nmr.relax.scm/22859). >>> 4) Sent the r1_fit flag into the target function >>> (http://thread.gmane.org/gmane.science.nmr.relax.scm/22866). >>> >>> As can be seen, the changes were rather simple. There is only one >>> brick remaining in the ruins of this wall, and that is simply using >>> this functionality. It would require using the r1_fit flag in the >>> target function __init__() method to switch between target functions, >>> and then removing the redundant '* R1 fit' models. Should I remove >>> the last brick and simplify the model list? >>> >>> Regards, >>> >>> Edward >>> >>> >>> >>> On 20 August 2014 17:34, Edward d'Auvergne <[email protected]> wrote: >>>> It's a thin wall that needs to be broken. For the backend of the >>>> relax_disp.select_model, we shouldn't do anything. We make the >>>> decision at the point of the grid_search() and minimise() API methods >>>> if we should prefix the model parameter lists with R1. As >>>> grid_search() calls minimise(), then we do this once in minimise(). >>>> Even better would be in the >>>> specific_analyses.relax_disp.optimisation.Disp_minimise_command class >>>> just after the R1 data is assembled. >>>> >>>> We could create a new function called >>>> specific_analyses.relax_disp.data.r1_optimisation() which decides if >>>> R1 values are to be optimised. This can be easily extended in the >>>> future by someone else to handle CPMG data as well. This function >>>> returns True or False and this can be stored as the self.r1_fit flag >>>> in Disp_minimise_command. If self.r1_fit is True, we can then call >>>> specific_analyses.relax_disp.parameters.prefix_r1() which adds the >>>> required R1 parameters to the spin container 'param' list. Then when >>>> setting up the target function class in Disp_minimise_command.run(), >>>> the self.r1_fit flag is sent in. The target function class __init__() >>>> method then uses the r1_fit flag to choose which target function >>>> method will be aliased to self.func by adding an extra set of if >>>> statements. I.e. change: >>>> >>>> if model == MODEL_DPL94: >>>> self.func = self.func_DPL94 >>>> if model == MODEL_DPL94_FIT_R1: >>>> self.func = self.func_DPL94_fit_r1 >>>> >>>> To: >>>> >>>> if model == MODEL_DPL94: >>>> if r1_fit: >>>> self.func = self.func_DPL94 >>>> else: >>>> self.func = self.func_DPL94_fit_r1 >>>> >>>> The specific_analyses.relax_disp.data.r1_optimisation() function can >>>> be called at any time that the R1 data is accessed, for example the >>>> unpacking of the parameter values (though in this case the r1_fit flag >>>> could be passed into the Disp_result_command to avoid this call). >>>> That's pretty much it. I don't think anything else is required. Am I >>>> missing something important? >>>> >>>> Cheers, >>>> >>>> Edawrd >>>> >>>> On 20 August 2014 17:14, Troels Emtekær Linnet <[email protected]> >>>> wrote: >>>>> Heh. >>>>> >>>>> I think I tried the solution you suggested. >>>>> >>>>> But I think you will run into several problems, that the parameters is >>>>> no longer setup correct. >>>>> >>>>> The issuing of: >>>>> relax_disp.select_model(model=MODEL) >>>>> >>>>> does "alot" of stuff. >>>>> >>>>> The problems starts about the initialisation of the dispersion class. >>>>> >>>>> One hits a wall of problems, which I surely cannot recommend. >>>>> >>>>> Best >>>>> Troels >>>>> >>>>> >>>>> 2014-08-20 17:07 GMT+02:00 Edward d'Auvergne <[email protected]>: >>>>>> Hi, >>>>>> >>>>>> Not loading the R1 data and then fitting it is good, especially if it >>>>>> is automatic. As long as a RelaxWarning is printed out to inform the >>>>>> user. The target function switching is much simpler than the model >>>>>> name translation. For example simply send in a r1_fit flag to the >>>>>> target function __init__() method and then add some more if statements >>>>>> to the end of that method. It would make it even more automatic from >>>>>> the perspective of the user. >>>>>> >>>>>> As I said, you've already done the hard part. The infrastructure you >>>>>> added is there now for handling this. You have implemented all >>>>>> required target functions and associated functions for handling the R1 >>>>>> fitting. The changes we are discussing are trivial in comparison. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Edward >>>>>> >>>>>> >>>>>> >>>>>> On 20 August 2014 16:56, Troels Emtekær Linnet <[email protected]> >>>>>> wrote: >>>>>>> Hi Edward. >>>>>>> >>>>>>> I actually think it is a quite elegant solution, that if one does not >>>>>>> load R1 data, then the R1 parameter is fitted. >>>>>>> >>>>>>> I think it is a better solution than the previous suggestion. >>>>>>> >>>>>>> It involves less coding, and the logic is sound. >>>>>>> >>>>>>> And the system is still "open" for weird stuff, whereby you by >>>>>>> scripting can analyse with and without fitting R1 in same analyses. >>>>>>> >>>>>>> I think this is best. >>>>>>> >>>>>>> But in the end. Potato, tomato. >>>>>>> If this solution prevents any further implementation, then this could >>>>>>> also be done. >>>>>>> >>>>>>> Best >>>>>>> Troels >>>>>>> >>>>>>> >>>>>>> 2014-08-20 16:48 GMT+02:00 Edward d'Auvergne <[email protected]>: >>>>>>>> An easy system test would be to load a set of CPMG data and a set of >>>>>>>> R1rho data, and then see if this can be analysed assuming no exchange >>>>>>>> via the 'No Rex' models. There are no other models for this yet, but >>>>>>>> it would show what is missing. You've added the infrastructure which >>>>>>>> is the hardest part. What remains to allow this is actually not much >>>>>>>> at all now that the infrastructure is in place. Just a bit of gluing >>>>>>>> code. >>>>>>>> >>>>>>>> On a related note, what is your opinion on the suggestions you made in >>>>>>>> sr #3135 at https://gna.org/support/?3135#comment0 "Optimisation of >>>>>>>> the R1 relaxation rate for the off-resonance R1rho relaxation >>>>>>>> dispersion models."? Namely that we let the user choose if R1 values >>>>>>>> are optimised or not for off-resonance data, whether or not R1 values >>>>>>>> were loaded. So then the '* R1 fit' models are merged into the >>>>>>>> non-fitted models, and the target function switching occurs without >>>>>>>> the user knowing. You should note that the target function switching >>>>>>>> is far simpler than the current model name translation system. It >>>>>>>> just requires a few more if statements at the end of the target >>>>>>>> function __init__() method to alias the correct self.func_*() method. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Edward >>>>>>>> >>>>>>>> >>>>>>>> On 20 August 2014 16:36, Troels Emtekær Linnet <[email protected]> >>>>>>>> wrote: >>>>>>>>> Hi Edward. >>>>>>>>> >>>>>>>>> I think I in the start tried following this path. >>>>>>>>> >>>>>>>>> But I realised with all the "R1 fit" models, that the very important, >>>>>>>>> relax_disp.select_model(model) >>>>>>>>> >>>>>>>>> setup all information for this model. >>>>>>>>> >>>>>>>>> As this was also a new model implementation, I also wanted the >>>>>>>>> flexibility to be able to select the desired model, and >>>>>>>>> be able to compare. >>>>>>>>> >>>>>>>>> I would suggest that this functionality is not digged further down the >>>>>>>>> code, before more experience is collected. >>>>>>>>> >>>>>>>>> Best >>>>>>>>> Troels >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2014-08-20 16:22 GMT+02:00 Edward d'Auvergne <[email protected]>: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> What would be even more powerful would be to shift this functionality >>>>>>>>>> even deeper down and merging the 'No Rex' and 'No Rex R1rho off res' >>>>>>>>>> models into one, just as the 'R2eff' model covers both the >>>>>>>>>> exponential >>>>>>>>>> curve-fitting via minimisation and the two-point approximation by >>>>>>>>>> calculation. Rather than switching the model name for the user, the >>>>>>>>>> target function can be switched. The 'No Rex R1 fit' model could >>>>>>>>>> then >>>>>>>>>> be created (by simply renaming 'No Rex R1rho off res R1 fit') and >>>>>>>>>> this >>>>>>>>>> would then eventually handle CPMG data as well. >>>>>>>>>> >>>>>>>>>> The reason for this is simple. In the future, certain interested >>>>>>>>>> parties (you probably know who) plans on developing a combined CPMG >>>>>>>>>> and R1rho model and on analysing this data together. This will need >>>>>>>>>> a >>>>>>>>>> 'No Rex' model for statistical comparisons. The splitting of 'No >>>>>>>>>> Rex' >>>>>>>>>> into two models as is currently coded is incompatible with this. A >>>>>>>>>> 'No Rex' model which has a loop over experiment types "for ei in >>>>>>>>>> range(self.NE):" switching to the different functionality would be a >>>>>>>>>> better solution. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Edward >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 20 August 2014 16:09, <[email protected]> wrote: >>>>>>>>>>> Author: tlinnet >>>>>>>>>>> Date: Wed Aug 20 16:09:40 2014 >>>>>>>>>>> New Revision: 25100 >>>>>>>>>>> >>>>>>>>>>> URL: http://svn.gna.org/viewcvs/relax?rev=25100&view=rev >>>>>>>>>>> Log: >>>>>>>>>>> Made the GUI selection of models for relaxation dispersion more >>>>>>>>>>> simple. >>>>>>>>>>> >>>>>>>>>>> After the implementation of a function which will translate the >>>>>>>>>>> models, the >>>>>>>>>>> 'No Rex' model will be converted to the 'No Rex' model for R1rho >>>>>>>>>>> off-resonance. >>>>>>>>>>> >>>>>>>>>>> Also the corresponding 'R1 fit' model will be chosen istead, if R1 >>>>>>>>>>> data has not been loaded. >>>>>>>>>>> >>>>>>>>>>> This makes the model selection easier in the GUI interface. >>>>>>>>>>> >>>>>>>>>>> Modified: >>>>>>>>>>> trunk/gui/analyses/auto_relax_disp.py >>>>>>>>>>> >>>>>>>>>>> Modified: trunk/gui/analyses/auto_relax_disp.py >>>>>>>>>>> URL: >>>>>>>>>>> http://svn.gna.org/viewcvs/relax/trunk/gui/analyses/auto_relax_disp.py?rev=25100&r1=25099&r2=25100&view=diff >>>>>>>>>>> ============================================================================== >>>>>>>>>>> --- trunk/gui/analyses/auto_relax_disp.py (original) >>>>>>>>>>> +++ trunk/gui/analyses/auto_relax_disp.py Wed Aug 20 16:09:40 >>>>>>>>>>> 2014 >>>>>>>>>>> @@ -715,8 +715,8 @@ >>>>>>>>>>> MODEL_R2EFF, >>>>>>>>>>> None, >>>>>>>>>>> MODEL_NOREX, >>>>>>>>>>> - MODEL_NOREX_R1RHO, >>>>>>>>>>> - MODEL_NOREX_R1RHO_FIT_R1, >>>>>>>>>>> +# MODEL_NOREX_R1RHO, >>>>>>>>>>> +# MODEL_NOREX_R1RHO_FIT_R1, >>>>>>>>>>> None, >>>>>>>>>>> MODEL_LM63, >>>>>>>>>>> MODEL_LM63_3SITE, >>>>>>>>>>> @@ -735,15 +735,15 @@ >>>>>>>>>>> MODEL_M61, >>>>>>>>>>> MODEL_M61B, >>>>>>>>>>> MODEL_DPL94, >>>>>>>>>>> - MODEL_DPL94_FIT_R1, >>>>>>>>>>> +# MODEL_DPL94_FIT_R1, >>>>>>>>>>> MODEL_TP02, >>>>>>>>>>> - MODEL_TP02_FIT_R1, >>>>>>>>>>> +# MODEL_TP02_FIT_R1, >>>>>>>>>>> MODEL_TAP03, >>>>>>>>>>> - MODEL_TAP03_FIT_R1, >>>>>>>>>>> +# MODEL_TAP03_FIT_R1, >>>>>>>>>>> MODEL_MP05, >>>>>>>>>>> - MODEL_MP05_FIT_R1, >>>>>>>>>>> +# MODEL_MP05_FIT_R1, >>>>>>>>>>> MODEL_NS_R1RHO_2SITE, >>>>>>>>>>> - MODEL_NS_R1RHO_2SITE_FIT_R1, >>>>>>>>>>> +# MODEL_NS_R1RHO_2SITE_FIT_R1, >>>>>>>>>>> MODEL_NS_R1RHO_3SITE_LINEAR, >>>>>>>>>>> MODEL_NS_R1RHO_3SITE, >>>>>>>>>>> None, >>>>>>>>>>> @@ -756,8 +756,8 @@ >>>>>>>>>>> "{%s/%s, %s}" % (r2eff, r1rho, i0), >>>>>>>>>>> None, >>>>>>>>>>> "{%s, ...}" % (r2), >>>>>>>>>>> - "{%s, ...}" % (r2), >>>>>>>>>>> - "{%s, %s, ...}" % (r1, r2), >>>>>>>>>>> +# "{%s, ...}" % (r2), >>>>>>>>>>> +# "{%s, %s, ...}" % (r1, r2), >>>>>>>>>>> None, >>>>>>>>>>> "{%s, ..., %s, %s}" % (r2, phi_ex, kex), >>>>>>>>>>> "{%s, ..., %s, kB, %s, kC}" % (r2, phi_exB, phi_exC), >>>>>>>>>>> @@ -776,15 +776,15 @@ >>>>>>>>>>> "{%s, ..., %s, %s}" % (r1rho_prime, phi_ex, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s}" % (r1rho_prime, dw, kex), >>>>>>>>>>> "{%s, ..., %s, %s}" % (r1rho_prime, phi_ex, kex), >>>>>>>>>>> - "{%s, %s, ..., %s, %s}" % (r1, r1rho_prime, phi_ex, kex), >>>>>>>>>>> +# "{%s, %s, ..., %s, %s}" % (r1, r1rho_prime, phi_ex, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s}" % (r1rho_prime, dw, kex), >>>>>>>>>>> - "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> +# "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s}" % (r1rho_prime, dw, kex), >>>>>>>>>>> - "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> +# "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s}" % (r1rho_prime, dw, kex), >>>>>>>>>>> - "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> +# "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s}" % (r1rho_prime, dw, kex), >>>>>>>>>>> - "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> +# "{%s, %s, ..., pA, %s, %s}" % (r1, r1rho_prime, dw, kex), >>>>>>>>>>> "{%s, ..., pA, %s, %s, pB, %s, %s}" % (r1rho_prime, dw_AB, >>>>>>>>>>> kAB, dw_BC, kBC), >>>>>>>>>>> "{%s, ..., pA, %s, %s, pB, %s, %s, %s}" % (r1rho_prime, >>>>>>>>>>> dw_AB, kAB, dw_BC, kBC, kAC), >>>>>>>>>>> None, >>>>>>>>>>> @@ -797,8 +797,8 @@ >>>>>>>>>>> "The base model for determining the %s/%s values and >>>>>>>>>>> errors for all other models." % (r2eff, r1rho), >>>>>>>>>>> None, >>>>>>>>>>> "The model for no chemical exchange relaxation.", >>>>>>>>>>> - "The model for no chemical exchange being present, for >>>>>>>>>>> off-resonance R1rho-type experiments. R1rho = R1 * cos(theta)^2 + >>>>>>>>>>> r1rho_prime * sin(theta)^2.", >>>>>>>>>>> - "The model for no chemical exchange being present, for >>>>>>>>>>> off-resonance R1rho-type experiments whereby R1 is fit. R1rho = R1 >>>>>>>>>>> * cos(theta)^2 + r1rho_prime * sin(theta)^2.", >>>>>>>>>>> +# "The model for no chemical exchange being present, for >>>>>>>>>>> off-resonance R1rho-type experiments. R1rho = R1 * cos(theta)^2 + >>>>>>>>>>> r1rho_prime * sin(theta)^2.", >>>>>>>>>>> +# "The model for no chemical exchange being present, for >>>>>>>>>>> off-resonance R1rho-type experiments whereby R1 is fit. R1rho = R1 >>>>>>>>>>> * cos(theta)^2 + r1rho_prime * sin(theta)^2.", >>>>>>>>>>> None, >>>>>>>>>>> "The original Luz and Meiboom (1963) 2-site fast exchange >>>>>>>>>>> equation.", >>>>>>>>>>> "The original Luz and Meiboom (1963) 3-site fast exchange >>>>>>>>>>> equation.", >>>>>>>>>>> @@ -817,15 +817,15 @@ >>>>>>>>>>> "The Meiboom (1961) 2-site fast exchange equation.", >>>>>>>>>>> "The Meiboom (1961) 2-site equation for all time scales >>>>>>>>>>> with pA >> pB.", >>>>>>>>>>> "The Davis, Perlman and London (1994) 2-site fast exchange >>>>>>>>>>> equation.", >>>>>>>>>>> - "The Davis, Perlman and London (1994) 2-site fast exchange >>>>>>>>>>> equation, whereby R1 is fit.", >>>>>>>>>>> +# "The Davis, Perlman and London (1994) 2-site fast >>>>>>>>>>> exchange equation, whereby R1 is fit.", >>>>>>>>>>> "The Trott and Palmer (2002) 2-site equation for all time >>>>>>>>>>> scales.", >>>>>>>>>>> - "The Trott and Palmer (2002) 2-site equation for all time >>>>>>>>>>> scales, whereby R1 is fit.", >>>>>>>>>>> +# "The Trott and Palmer (2002) 2-site equation for all time >>>>>>>>>>> scales, whereby R1 is fit.", >>>>>>>>>>> "The Trott, Abergel and Palmer (2003) off-resonance 2-site >>>>>>>>>>> equation for all time scales.", >>>>>>>>>>> - "The Trott, Abergel and Palmer (2003) off-resonance 2-site >>>>>>>>>>> equation for all time scales, whereby R1 is fit.", >>>>>>>>>>> +# "The Trott, Abergel and Palmer (2003) off-resonance >>>>>>>>>>> 2-site equation for all time scales, whereby R1 is fit.", >>>>>>>>>>> "The Miloushev and Palmer (2005) off-resonance 2-site >>>>>>>>>>> equation for all time scales.", >>>>>>>>>>> - "The Miloushev and Palmer (2005) off-resonance 2-site >>>>>>>>>>> equation for all time scales, whereby R1 is fit.", >>>>>>>>>>> +# "The Miloushev and Palmer (2005) off-resonance 2-site >>>>>>>>>>> equation for all time scales, whereby R1 is fit.", >>>>>>>>>>> "The 2-site numerical solution using 3D magnetisation >>>>>>>>>>> vectors.", >>>>>>>>>>> - "The 2-site numerical solution using 3D magnetisation >>>>>>>>>>> vectors, whereby R1 is fit.", >>>>>>>>>>> +# "The 2-site numerical solution using 3D magnetisation >>>>>>>>>>> vectors, whereby R1 is fit.", >>>>>>>>>>> "The 3-site linearised numerical solution using 3D >>>>>>>>>>> magnetisation vectors.", >>>>>>>>>>> "The 3-site numerical solution using 3D magnetisation >>>>>>>>>>> vectors.", >>>>>>>>>>> None, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> relax (http://www.nmr-relax.com) >>>>>>>>>>> >>>>>>>>>>> This is the relax-commits mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>> To unsubscribe from this list, get a password >>>>>>>>>>> reminder, or change your subscription options, >>>>>>>>>>> visit the list information page at >>>>>>>>>>> https://mail.gna.org/listinfo/relax-commits >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> relax (http://www.nmr-relax.com) >>>>>>>>>> >>>>>>>>>> This is the relax-devel mailing list >>>>>>>>>> [email protected] >>>>>>>>>> >>>>>>>>>> To unsubscribe from this list, get a password >>>>>>>>>> reminder, or change your subscription options, >>>>>>>>>> visit the list information page at >>>>>>>>>> https://mail.gna.org/listinfo/relax-devel _______________________________________________ relax (http://www.nmr-relax.com) This is the relax-devel mailing list [email protected] To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel
