Re: Writing/displaying output from full_analysis.py
For the sequence input file I think you want Spin_Name to be N if this is 15N relaxation data. I'm also thinking it's not a bad idea to update to the latest build of relax using subversion (can easily find/download this software via google). This looks vaguely similar to some of the problems Edward had debugged for me, so probably worth the time to update if you're going to use this script. The command to execute in the relax directory would be: svn co svn://svn.gna.org/svn/relax/1.3 relax-1.3 And then a newer version of relax is put in a nested folder in the relax directory. You want to make sure that the system path refers to the newer build of relax instead of the one you originally downloaded. When you type 'relax' it should say something like 'system repository checkout' instead of 'relax 1.3.2.' So the new directory would be ../relax-1.3.2/relax-1.3 Edward will probably have a more effective response, but this stuff will probably prevent other problems in any case. Tyler Quoting Brosey, Chris A [EMAIL PROTECTED]: Greetings, I have encountered a problem running the script full_analysis.py using v1.3.2 of relax (and python v2.5). I am currently executing the script with the MI option (DIFF_MODEL = local_tm). I find that the script successfully reads in data files and carries out the grid search and minimization specified under the multi_model function for the very first tm0 model. However, it appears that there is a problem executing the final command of the function writing the results to an output file (see reproduction of log file and traceback below). The proper directory and file are created, but the results file remains empty; and the script returns the error message 'IndexError: list index out of range'. I obtain the same error using various formatted versions of my own data files, as well as the test data files found in /test_suite/shared_data/model_free/S2_0 .0.149/, modified to include None for data values of GLY 1. Running multi_model commands manually through the relax prompt also returns the same error for both 'results.write' and 'results.display.' I also obtain the same result running the analysis with sample_scripts/model-free.py. I am new to python and the relax interface and apologize if I am bringing up a very basic question. I appreciate any help and suggestions you might have. Thanks! Regards, Chris Log File (edited) relax pipe.create(pipe_name='tm0', pipe_type='mf') relax sequence.read(file='noe.500.out', dir='~/relax/relax-1.3.2/sample_scripts/data', mol_name_col=None, res_num_col=0, res_name_col=1, spin_num_col=None, spin_name_col=None, sep=None) Opening the file '/home/broseyca/relax/relax-1.3.2/sample_scripts/data/noe.500.out' for reading. Mol_name Res_numRes_name Spin_num Spin_name None 1 GLYNone None None 2 ALANone None relax relax_data.read(ri_label='R1', frq_label='500', frq=50010.0, file='r1.500.out', dir='~/relax/relax-1.3.2/sample_scripts/data', mol_name_col=None, res_num_col=0, res_name_col=1, spin_num_col=None, spin_name_col=None, data_col=2, error_col=3, sep=None) Opening the file '/home/broseyca/relax/relax-1.3.2/sample_scripts/data/r1.500.out' for reading. #(A total of 6 relaxation files were read in). relax model_free.select_model(model='tm0', spin_id=None) relax grid_search(lower=None, upper=None, inc=11, constraints=True, verbosity=1) relax minimise(*args=('newton',), func_tol=1e-25, max_iterations=1000, constraints=True, scaling=True, verbosity=1) relax results.write(file='results', dir='local_tm/tm0', force=True, format='xml', compress_type=1) Opening the file 'local_tm/tm0/results.bz2' for writing. Traceback Message 77 sweet:/home/broseyca/relax/relax-1.3.2% Traceback (most recent call last): File relax, line 410, in module Relax() File relax, line 127, in __init__ self.interpreter.run(self.script_file) File /home/broseyca/relax/relax-1.3.2/prompt/interpreter.py, line 270, in run return run_script(intro=self.__intro_string, local=self.local, script_file=script_file, quit=self.__quit_flag, show_script=self.__show_script, raise_relax_error=self.__raise_relax_error) File /home/broseyca/relax/relax-1.3.2/prompt/interpreter.py, line 531, in run_script return console.interact(intro, local, script_file, quit, show_script=show_script, raise_relax_error=raise_relax_error) File /home/broseyca/relax/relax-1.3.2/prompt/interpreter.py, line 427, in interact_script execfile(script_file, local) File prompt/full_analysis2.py, line 668, in module Main(self.relax) File prompt/full_analysis2.py, line 221, in __init__ self.multi_model(local_tm=True) File prompt/full_analysis2.py, line 664, in multi_model results.write(file='results', dir=dir, force=True) File /home/broseyca/relax/relax-1.3.2/prompt/results.py,
Re: Model-Free Results Output
/specific_fns/model_free/mf_minimise.py, line 987, in minimise results = generic_minimise(func=self.mf.func, dfunc=self.mf.dfunc, d2func=self.mf.d2func, args=(), x0=param_vector, min_algor=min_algor, min_options=min_options, func_tol=func_tol, grad_tol=grad_tol, maxiter=max_iterations, A=A, b=b, full_output= ag=verbosity) File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/generic.py, line 412, in generic_minimise results = method_of_multipliers(func=func, dfunc=dfunc, d2func=d2func, args=args, x0=x0, min_options=min_options, A=A, b=b, l=l, u=u, c=c, dc=dc, d2c=d2c, func_tol=func_tol, grad_tol=grad_tol, maxiter=maxiter, full_output=full_output, print_flag= File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/method_of_multipliers.py, line 143, in method_of_multipliers results = min.minimise() File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/method_of_multipliers.py, line 403, in minimise results = self.generic_minimise(func=self.func_LA, dfunc=self.func_dLA, d2func=self.func_d2LA, args=self.args, x0=self.xk, min_algor=self.min_algor, min_options=self.min_options, func_tol=None, grad_tol=self.tk, maxiter=maxiter, full_output=1, pr b_print_flag, print_prefix=\t) File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/generic.py, line 343, in generic_minimise results = newton(func=func, dfunc=dfunc, d2func=d2func, args=args, x0=x0, min_options=min_options, func_tol=func_tol, grad_tol=grad_tol, maxiter=maxiter, full_output=full_output, print_flag=print_flag, print_prefix=print_prefix) File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/newton.py, line 44, in newton results = min.minimise() File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/base_classes.py, line 233, in minimise self.new_param_func() File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/newton.py, line 166, in new_param_func self.line_search() File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/base_classes.py, line 339, in backline self.alpha, fc = backtrack(self.func, self.args, self.xk, self.fk, self.dfk, self.pk, a_init=self.a0) TypeError: 'NoneType' object is not iterable Quoting Edward d'Auvergne [EMAIL PROTECTED]: :S It overwrites a pre-existing file! And the important script of all things!!! That'll have to be fixed. Seb, could you submit a bug report for that? Cheers, Edward On Fri, Oct 24, 2008 at 2:12 PM, Sébastien Morin [EMAIL PROTECTED] wrote: Oh... Sorry... It's still early in the morning here (Quebec City)... ... Dont' forget to specify a log file... LOG - relax -l LOG script.py TEE - relax -t LOG script.py If you just specify the script, the script will be erased by the log... Cheers, Séb Sébastien Morin wrote: Hi Tyler, If you want all output from relax to go to a log file, use the option '-l'... (relax -l script.py) If you want all output from relax to go to a file AND the terminal (display), use the option '-t'... (relax -t script.py) These options are visible when typing : ./relax --help (or relax -h) Good luck ! Séb :) Tyler Reddy wrote: Okay, I submitted the report. Let me know if there's more information you need or a better way to submit bug reports in general.. haven't done this before. Also, I had to manually copy/paste the output, and while it looks fine, if there is a problem with the same pdb structure getting read twice or something that's probably just my pasting error from the unix terminal to the browser. On that note, how can I fix this command: relax full_analysis.py STDOUT.txt 2 STDERR.txt tcsh: Ambiguous output redirect. Or better yet, can I merge all the output streams to a single text file and display the output in the terminal at the same time as well (maybe I'm getting too ambitious there..)? Also, I do get the same output for 'ellipsoid' as well, as Seb asked. Yours, Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, This issue looks like a problem with the loaded PDB structure. The 'local_tm' and 'sphere' optimisations do not utilise structural information, but the 'oblate' and 'prolate' spheroids, and the 'ellipsoid' all require this info. Normally relax will catch this problem and deselect the spins without XH bond vectors, but this seems to be broken in the 1.3 versions hence you get the original cryptic Python error. As this is a real bug with the relax source code (and not a yet-to-be converted feature or a sample script), would you be able to submit a bug report for this issue? The relax bug tracker is located at https://gna.org/bugs/?group=relax and the link at https://gna.org/bugs/?func=additemgroup=relax allows you to create
Re: Model-Free Results Output
The aic results file should be attached to the bug report. It looks like it's there when I check it. Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I've tried the PDB file and it seems to be fine. There are no strange formatting issues there. Therefore it is caused by the loaded 'local_tm/aic/results.bz2' file which is somehow not matching the loading of the PDB file. Unfortunately I can't work out what is causing the XH bond vectors from the PDB file to not be extracted. This is what the loading of vectors should look like: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: Either the spin number or name must be set for the spin ':13' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':14' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':15' to identify the corresponding atom in the structure. These are the messages from one of the system tests. Note the Extracted N-H vectors for lines which are missing from your printout. By using the following commands in relax: pipe.create(pipe_name='ellipsoid', pipe_type='mf') structure.read_pdb(file='PDB_file_truncate_random.pdb', dir=None, model=None, parser='scientific') structure.load_spins() structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) I get the following print out from the last user function: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. Extracted N-H vectors for '#A:[EMAIL PROTECTED]@N'. Extracted CA-H vectors for '#A:[EMAIL PROTECTED]@CA'. Extracted C-H vectors for '#A:[EMAIL PROTECTED]@C'. Extracted O-H vectors for '#A:[EMAIL PROTECTED]@O'. Extracted CB-H vectors for '#A:[EMAIL PROTECTED]@CB'. Extracted CG-H vectors for '#A:[EMAIL PROTECTED]@CG'. Extracted CD1-H vectors for '#A:[EMAIL PROTECTED]@CD1'. Extracted CD2-H vectors for '#A:[EMAIL PROTECTED]@CD2'. RelaxWarning: The XH bond vector for residue '#A:[EMAIL PROTECTED]@H' is of zero length. Extracted H-H vectors for '#A:[EMAIL PROTECTED]@H'. Extracted HA-H vectors for '#A:[EMAIL PROTECTED]@HA'. Extracted HB2-H vectors for '#A:[EMAIL PROTECTED]@HB2'. Extracted HB3-H vectors for '#A:[EMAIL PROTECTED]@HB3'. Extracted HG-H vectors for '#A:[EMAIL PROTECTED]@HG'. Extracted HD11-H vectors for '#A:[EMAIL PROTECTED]@HD11'. Extracted HD12-H vectors for '#A:[EMAIL PROTECTED]@HD12'. Extracted HD13-H vectors for '#A:[EMAIL PROTECTED]@HD13'. Extracted HD21-H vectors for '#A:[EMAIL PROTECTED]@HD21'. Extracted HD22-H vectors for '#A:[EMAIL PROTECTED]@HD22'. Extracted HD23-H vectors for '#A:[EMAIL PROTECTED]@HD23'. Obviously some of these vectors are a bit strange, but this test shows that there is no problem extracting them from your PDB file. Therefore I think I'd need the loaded results file (truncated and randomised if you wish) to be able to identify the problem. Regards, Edward On Wed, Oct 22, 2008 at 3:02 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Okay, I submitted the report. Let me know if there's more information you need or a better way to submit bug reports in general.. haven't done this before. Also, I had to manually copy/paste the output, and while it looks fine, if there is a problem with the same pdb structure getting read twice or something that's probably just my pasting error from the unix terminal to the browser. On that note, how can I fix this command: relax full_analysis.py STDOUT.txt 2 STDERR.txt tcsh: Ambiguous output redirect. Or better yet, can I merge all the output streams to a single text file and display the output in the terminal at the same time as well (maybe I'm getting too ambitious there..)? Also, I do get the same output for 'ellipsoid' as well, as Seb asked. Yours, Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED
Re: Model-Free Results Output
Ok, I updated to the latest build and made the adjustment. Presumably since the heteronucleus is set to 'N', 'H' is enough info for that column in the sequence file? It's working now for 'prolate' in any case. I see that you can change the bug report display to show 'closed' reports, sorry about that! Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Ok, one bug is because of the HETNUC variable. It is set twice. The first should be renamed, but I'll worry about that later. The real reason is because you need to name the spins. This can be done by setting the spin_name column of the file containing the sequence to the PDB atom name. I will also modify the script to use the first HETNUC value to set the spin name if not set by the sequence file. Regards, Edward On Thu, Oct 23, 2008 at 5:57 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Yeah, go ahead. Also, you closed the bug report so I can't actually attach the file to it. I'll try attaching the full_analysis.py script to this message.. hopefully the mailing list allows that. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Right, got it and can reproduce the bug. I think I know what the issue is, but would you be able to attach the full_analysis.py script to the bug report so I can confirm it? Well, actually there are probably two issues, the first being a problem with the full analysis script not setting the spin names (hence it can't find the unnamed atoms in the PDB file) and the second is that relax isn't catching this. Also, would it be ok if I added the truncated results.bz2 and PDB_file_truncate_random.pdb files to the relax test suite? This will allow me to fix the second issue (for the first, the full_analysis.py script needs checking). Cheers, Edward On Thu, Oct 23, 2008 at 4:48 PM, Tyler Reddy [EMAIL PROTECTED] wrote: The aic results file should be attached to the bug report. It looks like it's there when I check it. Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I've tried the PDB file and it seems to be fine. There are no strange formatting issues there. Therefore it is caused by the loaded 'local_tm/aic/results.bz2' file which is somehow not matching the loading of the PDB file. Unfortunately I can't work out what is causing the XH bond vectors from the PDB file to not be extracted. This is what the loading of vectors should look like: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: Either the spin number or name must be set for the spin ':13' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':14' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':15' to identify the corresponding atom in the structure. These are the messages from one of the system tests. Note the Extracted N-H vectors for lines which are missing from your printout. By using the following commands in relax: pipe.create(pipe_name='ellipsoid', pipe_type='mf') structure.read_pdb(file='PDB_file_truncate_random.pdb', dir=None, model=None, parser='scientific') structure.load_spins() structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) I get the following print out from the last user function: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. Extracted N-H vectors for '#A:[EMAIL PROTECTED]@N'. Extracted CA-H vectors for '#A:[EMAIL PROTECTED]@CA'. Extracted C-H vectors for '#A:[EMAIL PROTECTED]@C'. Extracted O-H vectors for '#A:[EMAIL PROTECTED]@O'. Extracted CB-H vectors for '#A:[EMAIL PROTECTED]@CB'. Extracted CG-H vectors for '#A:[EMAIL PROTECTED]@CG'. Extracted CD1-H vectors for '#A:[EMAIL PROTECTED]@CD1'. Extracted CD2-H vectors for '#A:[EMAIL PROTECTED]@CD2'. RelaxWarning: The XH bond vector for residue
Re: Model-Free Results Output
Maybe I need to use 'N' for the spin_name column because the extracted vector is H-H, which is very weird. It took 10 rounds for the 'prolate' script to converge: RelaxWarning: The XH bond vector for residue ':[EMAIL PROTECTED]' is of zero length. Extracted H-H vectors for ':[EMAIL PROTECTED]'. Quoting Edward d'Auvergne [EMAIL PROTECTED]: Ok, one bug is because of the HETNUC variable. It is set twice. The first should be renamed, but I'll worry about that later. The real reason is because you need to name the spins. This can be done by setting the spin_name column of the file containing the sequence to the PDB atom name. I will also modify the script to use the first HETNUC value to set the spin name if not set by the sequence file. Regards, Edward On Thu, Oct 23, 2008 at 5:57 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Yeah, go ahead. Also, you closed the bug report so I can't actually attach the file to it. I'll try attaching the full_analysis.py script to this message.. hopefully the mailing list allows that. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Right, got it and can reproduce the bug. I think I know what the issue is, but would you be able to attach the full_analysis.py script to the bug report so I can confirm it? Well, actually there are probably two issues, the first being a problem with the full analysis script not setting the spin names (hence it can't find the unnamed atoms in the PDB file) and the second is that relax isn't catching this. Also, would it be ok if I added the truncated results.bz2 and PDB_file_truncate_random.pdb files to the relax test suite? This will allow me to fix the second issue (for the first, the full_analysis.py script needs checking). Cheers, Edward On Thu, Oct 23, 2008 at 4:48 PM, Tyler Reddy [EMAIL PROTECTED] wrote: The aic results file should be attached to the bug report. It looks like it's there when I check it. Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I've tried the PDB file and it seems to be fine. There are no strange formatting issues there. Therefore it is caused by the loaded 'local_tm/aic/results.bz2' file which is somehow not matching the loading of the PDB file. Unfortunately I can't work out what is causing the XH bond vectors from the PDB file to not be extracted. This is what the loading of vectors should look like: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). RelaxWarning: No attached atom could be found (atom ID ':[EMAIL PROTECTED]@N'). Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. Extracted N-H vectors for ':[EMAIL PROTECTED]@N'. RelaxWarning: Either the spin number or name must be set for the spin ':13' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':14' to identify the corresponding atom in the structure. RelaxWarning: Either the spin number or name must be set for the spin ':15' to identify the corresponding atom in the structure. These are the messages from one of the system tests. Note the Extracted N-H vectors for lines which are missing from your printout. By using the following commands in relax: pipe.create(pipe_name='ellipsoid', pipe_type='mf') structure.read_pdb(file='PDB_file_truncate_random.pdb', dir=None, model=None, parser='scientific') structure.load_spins() structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) I get the following print out from the last user function: relax structure.vectors(attached='H', spin_id=None, struct_index=None, verbosity=1, ave=True, unit=True) Extracting vectors from the single structure. Calculating the unit vectors. The attached atom is a proton. Extracted N-H vectors for '#A:[EMAIL PROTECTED]@N'. Extracted CA-H vectors for '#A:[EMAIL PROTECTED]@CA'. Extracted C-H vectors for '#A:[EMAIL PROTECTED]@C'. Extracted O-H vectors for '#A:[EMAIL PROTECTED]@O'. Extracted CB-H vectors for '#A:[EMAIL PROTECTED]@CB'. Extracted CG-H vectors for '#A:[EMAIL PROTECTED]@CG'. Extracted CD1-H vectors for '#A:[EMAIL PROTECTED]@CD1'. Extracted CD2-H vectors for '#A:[EMAIL PROTECTED]@CD2'. RelaxWarning: The XH bond vector for residue '#A:[EMAIL
Re: Model-Free Results Output
Okay, I submitted the report. Let me know if there's more information you need or a better way to submit bug reports in general.. haven't done this before. Also, I had to manually copy/paste the output, and while it looks fine, if there is a problem with the same pdb structure getting read twice or something that's probably just my pasting error from the unix terminal to the browser. On that note, how can I fix this command: relax full_analysis.py STDOUT.txt 2 STDERR.txt tcsh: Ambiguous output redirect. Or better yet, can I merge all the output streams to a single text file and display the output in the terminal at the same time as well (maybe I'm getting too ambitious there..)? Also, I do get the same output for 'ellipsoid' as well, as Seb asked. Yours, Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, This issue looks like a problem with the loaded PDB structure. The 'local_tm' and 'sphere' optimisations do not utilise structural information, but the 'oblate' and 'prolate' spheroids, and the 'ellipsoid' all require this info. Normally relax will catch this problem and deselect the spins without XH bond vectors, but this seems to be broken in the 1.3 versions hence you get the original cryptic Python error. As this is a real bug with the relax source code (and not a yet-to-be converted feature or a sample script), would you be able to submit a bug report for this issue? The relax bug tracker is located at https://gna.org/bugs/?group=relax and the link at https://gna.org/bugs/?func=additemgroup=relax allows you to create a report. That would be much appreciated. The bug report will be useful for having this error caught by the relax test suite - if I can replicate the bug there then I can play with the bug and fix it. Note the the actual problem occurred earlier starting with the loading of the PDB file at the structure.read_pdb() user function. So if you could include all print out from that user function to the end of the traceback error message, that would be very useful. Cheers, Edward On Wed, Oct 22, 2008 at 3:17 AM, Tyler Reddy [EMAIL PROTECTED] wrote: Hi Seb, I just tested 'oblate' and it produces the same error output. Tyler Quoting Sébastien Morin [EMAIL PROTECTED]: Hi Tyler, Do you get the same error when trying the 'oblate' or 'ellipsoid' diffusion tensors ? Sébastien Tyler Reddy wrote: I'm having an issue with the full_analysis.py script. It seems to work fine when DIFF_MODEL = 'local_tm' or 'sphere' (converges on 4th round) but for 'prolate' I get the following output: Grid search ~~~ Searching the grid. Traceback (most recent call last): File /Applications/relax-1.3.1/relax-1.3/relax, line 408, in module Relax() File /Applications/relax-1.3.1/relax-1.3/relax, line 125, in __init__ self.interpreter.run(self.script_file) File /Applications/relax-1.3.1/relax-1.3/prompt/interpreter.py, line 270, in run return run_script(intro=self.__intro_string, local=self.local, script_file=script_file, quit=self.__quit_flag, show_script=self.__show_script, raise_relax_error=self.__raise_relax_er File /Applications/relax-1.3.1/relax-1.3/prompt/interpreter.py, line 531, in run_script return console.interact(intro, local, script_file, quit, show_script=show_script, raise_relax_error=raise_relax_error) File /Applications/relax-1.3.1/relax-1.3/prompt/interpreter.py, line 427, in interact_script execfile(script_file, local) File full_analysis.py, line 673, in module Main(self.relax) File full_analysis.py, line 284, in __init__ grid_search(inc=inc) File /Applications/relax-1.3.1/relax-1.3/prompt/minimisation.py, line 152, in grid_search minimise.grid_search(lower=lower, upper=upper, inc=inc, constraints=constraints, verbosity=verbosity) File /Applications/relax-1.3.1/relax-1.3/generic_fns/minimise.py, line 185, in grid_search grid_search(lower=lower, upper=upper, inc=inc, constraints=constraints, verbosity=verbosity) File /Applications/relax-1.3.1/relax-1.3/specific_fns/model_free/mf_minimise.py, line 479, in grid_search self.minimise(min_algor='grid', lower=lower, upper=upper, inc=inc, constraints=constraints, verbosity=verbosity, sim_index=sim_index) File /Applications/relax-1.3.1/relax-1.3/specific_fns/model_free/mf_minimise.py, line 987, in minimise results = generic_minimise(func=self.mf.func, dfunc=self.mf.dfunc, d2func=self.mf.d2func, args=(), x0=param_vector, min_algor=min_algor, min_options=min_options, func_tol=func_tol, g l, maxiter=max_iterations, A=A, b=b, full_output=1, print_flag=verbosity) File /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/minfx/generic.py, line 319, in generic_minimise xk, fk, k = grid(func=func, args=args, grid_ops=min_options, A=A, b=b, l=l, u=u, c=c, print_flag=print_flag) File /Library/Frameworks/Python.framework/Versions/2.5/lib
Re: Relax_fit.py problem
Hi, I'll try to dig up those references. The other thing I find confusing is that some groups use the curve fit error for the parameters. So, the errors in R1 and R2 per residue are actually from the nonlinear curve fitting process itself. In theory, if there is no error in peak height then the fit is perfect. So I wonder if there is yet another relationship to think about if you want to use those values?! I have these values already for T1 and T2 parameters and their curve fitting errors (though I haven't figured out how to propagate these errors to the reciprocal rate constants, or if that will even be meaningful), but I'm not sure how they compare to the other 2 'error types' we are talking about. Certainly, S/N = peak height/RMS baseline noise (From Cavanagh textbook) And while there are many references that throw around the sqrt(2) in various equations, I haven't seen a comprehensive explanation yet. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, That was the reference I used many years ago when I first added these abilities to relax. The text is a little confusing, but the important line is the first of that paragraph you mention: The uncertainties in the measured peak heights, sigma_h, were set equal to the root-mean-square baseline noise in the spectra. So if one looks at the code in relax, there is no multiplication by sqrt(2). As this was a long time ago, I'm not sure if this is the most correct approach. The confusing chi-squared tests between the sigma_h and sqrt(2)*sigma_h may not statistically significant but considering that the parameter number is identical in both cases, the weighting constant simply changes, then no statistically significant difference doesn't mean that one weight is better than the other or that both weights are correct. There is another early reference (or two) in which the NOE error formula is given. I think that may have more information, but I'm struggling to remember what that reference is and cannot find it at the moment. And there may be more recent papers performing a much more thorough noise analysis. It could even be done using synthetic spectra with white noise added (I recently did this to test the effect of white noise on the uncertainty in peak chemical shift position to validate Ad Bax's RDC error formula LW/SN - strangely the results were far more complex than this formula). There is a bit of time to find the correct baseplane RMSD to peak height uncertainty as I need to wait for Sebastien to finish the work with the loading of NMRView (as well as Sparky and XEasy) peak list intensities. The rearrangements I plan to do will affect the code he is working on. Regards, Edward On Mon, Oct 13, 2008 at 7:44 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hi Edward, Palmer et al. (1991) JACS. 113: 4371-4380 is a nice reference for the error conversion. It looks like the value for standard deviation between peaks in paired spectra is sqrt(2) multiplied by the base plane RMS value (in particular, see the short paragraph at the top right of page 4375 in this manuscript). However, the authors seem to use the base plane RMS values regardless, and then verify that the qualitative conclusions do not change when using the more conservative error estimates (i.e. multiplying by 1.4). There's an extensive discussion of using chi-square critical values to verify the validity of this relationship between the noise types, though I must concede that I don't grasp all the details after the first reading. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, There are three ways that an error analysis can be done for relaxation curve fitting, although one of those is only partly implemented in relax at the moment (that means it won't work until I write some computer code). These are: 1. Collect all spectra in duplicate, triplicate, or more if you really have lot of NMR time to kill, for absolutely no reason. The peak intensity error for a single spin is calculated as the standard deviation for each peak. Because this is inaccurate for a low replica number, this error is averaged for all peaks to give one error per spectrum. This error is then used in the Monte Carlo simulations. 2. If only some spectra are duplicated, then the average of the errors for all spectra is calculated. This gives a single error value for all spins and all spectra. This is then used in the Monte Carlo simulations. 3. This is the error analysis technique which is not fully implemented yet. If no spectra are recorded in duplicate, then one needs to use the RMSD of the base plane noise. This is similar to what relax uses for the NOE analysis (hence shouldn't be too hard to implement for relaxation curve fitting). I would need to find the reference, but I think this value needs to be divided or multiplied by root 2 to convert it to a peak height uncertainty. Does anyone know
Re: Relax_fit.py problem
Hey, Farrow et al. (1994) Biochemistry, 33: 5984-6003 also draw a similar conclusion (paragraph starting at bottom left of p. 5988) and apply the RMS value of the noise as an estimate of the standard deviation of peak intensity. If I'm not mistaken this is the exact assumption made by relax for steady-state NOE error propagation by the sum of squares equation from this paper as well. Also of interest on p. 5988, The distribution of the difference in intensities of identical peaks in duplicate spectra should have a standard deviation [sqrt(2)] times greater than the standard deviation of the individual peaks. They again conclude that duplicate and RMS baseline data errors are consistent within those bounds. If the Kay and Palmer labs are going with this conclusion (even if it doesn't really tell us which error is more appropriate), it seems like it's a good bet that you can estimate standard deviation this way. However, I'm sill not clear on the relationship between curve fit errors and the errors measured directly from the spectra. I'm not sure how the nonlinear fitting error factors in for relax R1, R2 curve-fitting scripts. Certainly, if curve-fit error alone could be used that would make things easier since no error measurement on the T1/T2 experiment spectra would be needed, you could just dump the peak heights to relax. Tyler Quoting Tyler Reddy [EMAIL PROTECTED]: Hi, I'll try to dig up those references. The other thing I find confusing is that some groups use the curve fit error for the parameters. So, the errors in R1 and R2 per residue are actually from the nonlinear curve fitting process itself. In theory, if there is no error in peak height then the fit is perfect. So I wonder if there is yet another relationship to think about if you want to use those values?! I have these values already for T1 and T2 parameters and their curve fitting errors (though I haven't figured out how to propagate these errors to the reciprocal rate constants, or if that will even be meaningful), but I'm not sure how they compare to the other 2 'error types' we are talking about. Certainly, S/N = peak height/RMS baseline noise (From Cavanagh textbook) And while there are many references that throw around the sqrt(2) in various equations, I haven't seen a comprehensive explanation yet. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, That was the reference I used many years ago when I first added these abilities to relax. The text is a little confusing, but the important line is the first of that paragraph you mention: The uncertainties in the measured peak heights, sigma_h, were set equal to the root-mean-square baseline noise in the spectra. So if one looks at the code in relax, there is no multiplication by sqrt(2). As this was a long time ago, I'm not sure if this is the most correct approach. The confusing chi-squared tests between the sigma_h and sqrt(2)*sigma_h may not statistically significant but considering that the parameter number is identical in both cases, the weighting constant simply changes, then no statistically significant difference doesn't mean that one weight is better than the other or that both weights are correct. There is another early reference (or two) in which the NOE error formula is given. I think that may have more information, but I'm struggling to remember what that reference is and cannot find it at the moment. And there may be more recent papers performing a much more thorough noise analysis. It could even be done using synthetic spectra with white noise added (I recently did this to test the effect of white noise on the uncertainty in peak chemical shift position to validate Ad Bax's RDC error formula LW/SN - strangely the results were far more complex than this formula). There is a bit of time to find the correct baseplane RMSD to peak height uncertainty as I need to wait for Sebastien to finish the work with the loading of NMRView (as well as Sparky and XEasy) peak list intensities. The rearrangements I plan to do will affect the code he is working on. Regards, Edward On Mon, Oct 13, 2008 at 7:44 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hi Edward, Palmer et al. (1991) JACS. 113: 4371-4380 is a nice reference for the error conversion. It looks like the value for standard deviation between peaks in paired spectra is sqrt(2) multiplied by the base plane RMS value (in particular, see the short paragraph at the top right of page 4375 in this manuscript). However, the authors seem to use the base plane RMS values regardless, and then verify that the qualitative conclusions do not change when using the more conservative error estimates (i.e. multiplying by 1.4). There's an extensive discussion of using chi-square critical values to verify the validity of this relationship between the noise types, though I must concede that I don't grasp all the details after the first reading
Re: Relax_fit.py problem
Okay, so I'll basically just need to get the rms noise (NOT S/N) for my T1 and T2 spectra at various fields. I've been using S/N values for the NOE calculations so that explains why those errors seemed so large. I'd like to find a reference for the rms equation which seems to be: [sqrt(2)]*(baseline noise height)/4 Ignoring the stuff with duplicate spectra, it's actually a rather nice situation with only peak height and baseline noise height being the information required for the calculation of NOE, R1, R2, and the propagation of their respective errors. Presumably this would then be enough to begin the model-free analysis in relax? Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: On Wed, Oct 15, 2008 at 4:56 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hey, Farrow et al. (1994) Biochemistry, 33: 5984-6003 also draw a similar conclusion (paragraph starting at bottom left of p. 5988) and apply the RMS value of the noise as an estimate of the standard deviation of peak intensity. If I'm not mistaken this is the exact assumption made by relax for steady-state NOE error propagation by the sum of squares equation from this paper as well. That is another reference I am very familiar with. It influenced my choice of not using the sqrt(2) factor for the NOE. And there is the NOE error equation I used! There is another publication which has this equation, but in quite a different form (it almost looks like a different equation). It's also given in general form at http://en.wikipedia.org/wiki/Error_propagation as the variance of the ratio of two random variables. Also of interest on p. 5988, The distribution of the difference in intensities of identical peaks in duplicate spectra should have a standard deviation [sqrt(2)] times greater than the standard deviation of the individual peaks. I'll have to check the source code, but from memory this factor was not used when using duplicate spectra. Let me see... Ok, relax is not dividing by sqrt(2) when calculating one error value from a duplicated spectrum. I have to think about this because relax is not calculating the standard deviation of a distribution of differences, as talked about in Palmer et al., 1991 and Farrow et al., 1994. It's calculating the population standard deviation for each spin - this allows for triplicate spectra - and averaging this value for all spins. It's all described in the relax_fit.mean_and_error() user function documentation. I think this may not be the correct method and that this needs more investigation! relax is averaging the standard deviations whereas I think that in reality we should be averaging the variances (the square root of the sum of squared standard deviations). This should occur for the single duplicated (or triplicated) spectrum and for the averaging across all spectra when not all are in duplicate. This might be seen as R1 and R2 error differences between relax and Art Palmer's curvefit program, although Jackknife vs. Monte Carlo simulation differences are also present. They again conclude that duplicate and RMS baseline data errors are consistent within those bounds. If the Kay and Palmer labs are going with this conclusion (even if it doesn't really tell us which error is more appropriate), it seems like it's a good bet that you can estimate standard deviation this way. For base plane RMSD, this is what the NOE is doing and what the new code will do for the relaxation curve fitting. I think I need to revisit the statistics of the duplicated spectra though. However, I'm sill not clear on the relationship between curve fit errors and the errors measured directly from the spectra. I'm not sure how the nonlinear fitting error factors in for relax R1, R2 curve-fitting scripts. Certainly, if curve-fit error alone could be used that would make things easier since no error measurement on the T1/T2 experiment spectra would be needed, you could just dump the peak heights to relax. It could be used, but it is less accurate and much more work to implement. Monte Carlo simulations are the gold standard for error propagation in non-linear problems. A reference is http://en.wikipedia.org/wiki/Error_propagation#Caveats_and_warnings, but this description of the problem is technical and not very good. Wikipedia's description of error propagation is interesting, but is missing the descriptions of using the covariance matrix, Jackknife simulations, Bootstrapping simulations, and Monte Carlo simulations (these are the main techniques, but others exist). The Numerical Receipes book is much clearer on the subject. Regards, Edward ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Re: Relax_fit.py problem
Hi Edward, Palmer et al. (1991) JACS. 113: 4371-4380 is a nice reference for the error conversion. It looks like the value for standard deviation between peaks in paired spectra is sqrt(2) multiplied by the base plane RMS value (in particular, see the short paragraph at the top right of page 4375 in this manuscript). However, the authors seem to use the base plane RMS values regardless, and then verify that the qualitative conclusions do not change when using the more conservative error estimates (i.e. multiplying by 1.4). There's an extensive discussion of using chi-square critical values to verify the validity of this relationship between the noise types, though I must concede that I don't grasp all the details after the first reading. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, There are three ways that an error analysis can be done for relaxation curve fitting, although one of those is only partly implemented in relax at the moment (that means it won't work until I write some computer code). These are: 1. Collect all spectra in duplicate, triplicate, or more if you really have lot of NMR time to kill, for absolutely no reason. The peak intensity error for a single spin is calculated as the standard deviation for each peak. Because this is inaccurate for a low replica number, this error is averaged for all peaks to give one error per spectrum. This error is then used in the Monte Carlo simulations. 2. If only some spectra are duplicated, then the average of the errors for all spectra is calculated. This gives a single error value for all spins and all spectra. This is then used in the Monte Carlo simulations. 3. This is the error analysis technique which is not fully implemented yet. If no spectra are recorded in duplicate, then one needs to use the RMSD of the base plane noise. This is similar to what relax uses for the NOE analysis (hence shouldn't be too hard to implement for relaxation curve fitting). I would need to find the reference, but I think this value needs to be divided or multiplied by root 2 to convert it to a peak height uncertainty. Does anyone know a reference for this? Then a separate error value for all spins and all spectra can be used in the Monte Carlo simulations. Wei Xia has recently asked the same question (https://mail.gna.org/public/relax-users/2008-09/msg0.html). It might be worth reading my reply at https://mail.gna.org/public/relax-users/2008-09/msg2.html. So this feature will be added to relax, but the question is how long will that take. I'd first need the error conversion factor from RMSD of base plane noise to peak height, and then add the ability to use the RMSD value in relaxation curve fitting. The first part will be the hardest, but you'll need that to do a proper Monte Carlo simulation error analysis for the curve fitting. To do the second part I would set up a mini analysis, lets call it a 'system test' because it tests the system - relax - to see if the analysis works, and then make this system test pass - i.e. implement the feature. Don't forget that the errors in a complex analysis (e.g model-free and reduced spectral density mapping) are just as important as the values themselves, if not more. Getting these wrong will really damage optimisation, model selection, and error propagation to the final parameters via Monte Carlo simulations. So both your model-free values and errors will be incorrect. Regards, Edward On Wed, Oct 8, 2008 at 5:07 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hello, It seems that Relax_fit.py requires replicate data because average and standard deviation values are used downstream in the analysis. With no replicate data (since I don't have any) the output is shown below. Also, commenting out the average and error propagation across multiple spectra #relax_fit.mean_and_error() doesn't work either, and I get another error output that is looking for an averaged value. I'll probably try using a duplicate data set to circumvent this for now (unless this is actually another problem). Tyler Output: relax relax_fit.mean_and_error() Calculating the average intensity and standard deviation of all spectra. Time point: 0.01 s Number of spectra: 1 Standard deviation for time point 0: 0.0 Time point: 0.050003 s Number of spectra: 1 Standard deviation for time point 1: 0.0 Time point: 0.10001 s Number of spectra: 1 Standard deviation for time point 2: 0.0 Time point: 0.20001 s Number of spectra: 1 Standard deviation for time point 3: 0.0 Time point: 0.2 s Number of spectra: 1 Standard deviation for time point 4: 0.0 Time point: 0.5 s Number of spectra: 1 Standard deviation for time point 5: 0.0 Time point: 0.80004 s Number of spectra: 1 Standard deviation for time point 6: 0.0 Traceback (most recent call last
Re: Difficulty with noe.py sample script
Of the suggested solutions only installing 1.2.12 and running noe.py from that actually seems to work. I installed a Mac OS X binary version of SVN, and it seemed to work just fine, but maybe there's another build step or something I don't know about after using the commands outlined below. Anyways, the steady-state NOE values from relax match perfectly with the parallel calculations in excel, so that's good. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I've now found and fixed the bug in the 1.2 relax line (see https://gna.org/bugs/?12418). I will release relax 1.2.15 with this fix soonish, but please be patient as it takes up to 6 solid hours to do a release. If you can't wait, you have two alternatives. The best would be to use the subversion program to get the most up-to-date copy of the 1.2 relax line by typing: $ svn co svn://svn.gna.org/svn/relax/1.2 relax-1.2 or if this doesn't work: $ svn co http://svn.gna.org/svn/relax/1.2 relax-1.2 Otherwise you could download and use relax.1.2.12. Regards, Edward On Tue, Oct 7, 2008 at 9:29 PM, Edward d'Auvergne [EMAIL PROTECTED] wrote: On Tue, Oct 7, 2008 at 7:48 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hi Edward, Thanks for your continued patience. That's okay, it's not a problem. Okay, it seems that aligning the data height column by the left most digit rather than the right most digit is not tolerated. Now the script (1.2) almost gets to the end, but there is still an error output, where the program opens noe.out for writing (see below). The noe.out file is created, but it's blank. Assuming I can get functional input for the 1.2 version (which is supposed to work), if this input doesn't work for the 1.3 version I'm happy to submit the bug report at that stage. relax simply splits the text file by whitespace so alignment is only an issue if columns fuse. I think there is another issue somewhere. I've tried the analysis with some of my old data, but I get the following error: relax calc(run='noe', print_flag=1) relax value.write(run='noe', param='noe', file='noe.out', dir=None, force=1) Opening the file 'noe.out' for writing. Traceback (most recent call last): File /data/relax/releases/relax-1.2.14/relax, line 457, in module Relax() File /data/relax/releases/relax-1.2.14/relax, line 167, in __init__ self.interpreter.run() File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 216, in run run_script(intro=self.relax.intro_string, local=self.local, script_file=self.relax.script_file, quit=1) File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 391, in run_script console.interact(intro, local, script_file, quit) File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 343, in interact_script execfile(script_file, local) File noe.py, line 29, in module value.write(name, param='noe', file='noe.out', force=1) File /data/relax/releases/relax-1.2.14/prompt/value.py, line 506, in write self.__relax__.generic.value.write(run=run, param=param, file=file, dir=dir, force=force) File /data/relax/releases/relax-1.2.14/generic_fns/value.py, line 537, in write self.write_data(file, return_value) File /data/relax/releases/relax-1.2.14/generic_fns/value.py, line 549, in write_data self.function_type = self.relax.data.run_types[self.relax.data.run_names.index(run)] ValueError: list.index(x): x not in list This is exactly what you see. So I then tested this on many previous relax releases and found that all versions up to 1.2.12 function properly. This error is a regression which appeared in the 1.2.13 version and still exists in 1.2.14. So a temporary but quick fix would be to do the NOE analysis using relax-1.2.12. For relaxation curve fitting and model-free analysis, the 1.3.1 version can be used (there are system tests for these analyses there so this is much less likely to fail). I'm sorry if this caused inconveniences, I have no idea how this error was introduced. I've created a bug report for this issue (https://gna.org/bugs/index.php). Also, I think it would be very useful if a sample of functional input data, as you supplied for me in the previous email, would be incorporated in the relax manual. Different labs can have different naming conventions and formats, even within a Sparky list, and the column alignment can be tricky (even if trivial once you know about it). Also, if there is sensitivity to the nomenclature system employed (as some of your previous responses on this topic attest), then an example directly in the manual would definitely be appreciated. I know that part of the design philosophy is that it's very easy to incorporate new format tolerance into relax, but it's even faster for a new user who uses a slightly different format to just read the relevant section of the manual and format that way. Done, I've added exactly the text you suggest
Relax_fit.py problem
Hello, It seems that Relax_fit.py requires replicate data because average and standard deviation values are used downstream in the analysis. With no replicate data (since I don't have any) the output is shown below. Also, commenting out the average and error propagation across multiple spectra #relax_fit.mean_and_error() doesn't work either, and I get another error output that is looking for an averaged value. I'll probably try using a duplicate data set to circumvent this for now (unless this is actually another problem). Tyler Output: relax relax_fit.mean_and_error() Calculating the average intensity and standard deviation of all spectra. Time point: 0.01 s Number of spectra: 1 Standard deviation for time point 0: 0.0 Time point: 0.050003 s Number of spectra: 1 Standard deviation for time point 1: 0.0 Time point: 0.10001 s Number of spectra: 1 Standard deviation for time point 2: 0.0 Time point: 0.20001 s Number of spectra: 1 Standard deviation for time point 3: 0.0 Time point: 0.2 s Number of spectra: 1 Standard deviation for time point 4: 0.0 Time point: 0.5 s Number of spectra: 1 Standard deviation for time point 5: 0.0 Time point: 0.80004 s Number of spectra: 1 Standard deviation for time point 6: 0.0 Traceback (most recent call last): File /Applications/relax-1.3.1/relax, line 408, in module Relax() File /Applications/relax-1.3.1/relax, line 125, in __init__ self.interpreter.run(self.script_file) File /Applications/relax-1.3.1/prompt/interpreter.py, line 270, in run return run_script(intro=self.__intro_string, local=self.local, script_file=script_file, quit=self.__quit_flag, show_script=self.__show_script, raise_relax_error=self.__raise_relax_error) File /Applications/relax-1.3.1/prompt/interpreter.py, line 531, in run_script return console.interact(intro, local, script_file, quit, show_script=show_script, raise_relax_error=raise_relax_error) File /Applications/relax-1.3.1/prompt/interpreter.py, line 427, in interact_script execfile(script_file, local) File relax_fit_T1_500Mhz.py, line 45, in module relax_fit.mean_and_error() File /Applications/relax-1.3.1/prompt/relax_fit.py, line 96, in mean_and_error relax_fit_obj.mean_and_error() File /Applications/relax-1.3.1/specific_fns/relax_fit.py, line 729, in mean_and_error sd = sd / float(num_dups) ZeroDivisionError: float division ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Re: Difficulty with noe.py sample script
Yes, the standard deviation of the steady-state NOEs match to 15 decimal places (ignoring the occasional rounding error in excel). The svn command definitely worked, and picked up a bunch of files, but the updated software just didn't fix the problems (1.2.14 or 1.3.1 versions). But, as you say, not all that important anymore since 1.2.12 is just fine for the NOE. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: That's good to hear, that the values match. I hope the errors match as well. If subversion is installed, that then the only way that typing: svn co http://svn.gna.org/svn/relax/1.2 relax-1.2 in a terminal can fail is if the Gna! (http://gna.org) open source infrastructure is down, which doesn't happen, or if the internet isn't working. But if it's working for you, then you don't need to use SVN. Regards, Edward On Wed, Oct 8, 2008 at 3:46 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Of the suggested solutions only installing 1.2.12 and running noe.py from that actually seems to work. I installed a Mac OS X binary version of SVN, and it seemed to work just fine, but maybe there's another build step or something I don't know about after using the commands outlined below. Anyways, the steady-state NOE values from relax match perfectly with the parallel calculations in excel, so that's good. Tyler Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I've now found and fixed the bug in the 1.2 relax line (see https://gna.org/bugs/?12418). I will release relax 1.2.15 with this fix soonish, but please be patient as it takes up to 6 solid hours to do a release. If you can't wait, you have two alternatives. The best would be to use the subversion program to get the most up-to-date copy of the 1.2 relax line by typing: $ svn co svn://svn.gna.org/svn/relax/1.2 relax-1.2 or if this doesn't work: $ svn co http://svn.gna.org/svn/relax/1.2 relax-1.2 Otherwise you could download and use relax.1.2.12. Regards, Edward On Tue, Oct 7, 2008 at 9:29 PM, Edward d'Auvergne [EMAIL PROTECTED] wrote: On Tue, Oct 7, 2008 at 7:48 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hi Edward, Thanks for your continued patience. That's okay, it's not a problem. Okay, it seems that aligning the data height column by the left most digit rather than the right most digit is not tolerated. Now the script (1.2) almost gets to the end, but there is still an error output, where the program opens noe.out for writing (see below). The noe.out file is created, but it's blank. Assuming I can get functional input for the 1.2 version (which is supposed to work), if this input doesn't work for the 1.3 version I'm happy to submit the bug report at that stage. relax simply splits the text file by whitespace so alignment is only an issue if columns fuse. I think there is another issue somewhere. I've tried the analysis with some of my old data, but I get the following error: relax calc(run='noe', print_flag=1) relax value.write(run='noe', param='noe', file='noe.out', dir=None, force=1) Opening the file 'noe.out' for writing. Traceback (most recent call last): File /data/relax/releases/relax-1.2.14/relax, line 457, in module Relax() File /data/relax/releases/relax-1.2.14/relax, line 167, in __init__ self.interpreter.run() File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 216, in run run_script(intro=self.relax.intro_string, local=self.local, script_file=self.relax.script_file, quit=1) File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 391, in run_script console.interact(intro, local, script_file, quit) File /data/relax/releases/relax-1.2.14/prompt/interpreter.py, line 343, in interact_script execfile(script_file, local) File noe.py, line 29, in module value.write(name, param='noe', file='noe.out', force=1) File /data/relax/releases/relax-1.2.14/prompt/value.py, line 506, in write self.__relax__.generic.value.write(run=run, param=param, file=file, dir=dir, force=force) File /data/relax/releases/relax-1.2.14/generic_fns/value.py, line 537, in write self.write_data(file, return_value) File /data/relax/releases/relax-1.2.14/generic_fns/value.py, line 549, in write_data self.function_type = self.relax.data.run_types[self.relax.data.run_names.index(run)] ValueError: list.index(x): x not in list This is exactly what you see. So I then tested this on many previous relax releases and found that all versions up to 1.2.12 function properly. This error is a regression which appeared in the 1.2.13 version and still exists in 1.2.14. So a temporary but quick fix would be to do the NOE analysis using relax-1.2.12. For relaxation curve fitting and model-free analysis, the 1.3.1 version can be used (there are system tests for these analyses there so this is much less likely to fail). I'm sorry if this caused inconveniences, I have no idea how this error was introduced. I've
Re: Relax_fit.py problem
Interesting. I'll try to look for the reference described for the conversion of base plane noise to peak height noise. Quoting Edward d'Auvergne [EMAIL PROTECTED]: On Wed, Oct 8, 2008 at 6:07 PM, Tyler Reddy [EMAIL PROTECTED] wrote: To get past this requirement I added or subtracted the last digit for each peak height for the data set to make a pseudo-replicate data set (2 identical files will still get a 0 value division problem). The program goes through the Monte Carlo simulations, but doesn't quite finish, with the follow output: This will give you the R1 and R2 values, but there errors will be incorrect. These errors must be dead accurate for you to use the R1 and R2 in any type of analysis, as the curvature of the optimisation space and statistical measures are highly dependent on these values. If you need instant results with non-replicated spectra, then I would recommend you not to use relax. There may be other software which will do this analysis for you, but remember that the errors are very important and must be propagated by the only accurate technique - Monte Carlo simulations. The covariance matrix or Jackknife simulation techniques are used at times, but just read the Monte Carlo section of the Numerical Recepies book to see why I recommend to only use MC sims. relax monte_carlo.error_analysis(prune=0.0) relax value.write(run=None, param='rx', file='rx.out', dir=None, force=True) RelaxError: The run argument None must be a string. This one I fixed while implementing the NOE analysis in the 1.3 line for you. You will have to wait until I release relax 1.3.2 to have this fix, unless you get subversion running and download the fixed code ;) Regards, Edward ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Re: Difficulty with noe.py sample script
I ran the 1.2 version of the noe.py script with the 1.2 version of relax on the same machine and had a slightly different error output (below). If this is simply a version-change problem with the script as you say, then I'm not sure why I'd get a convoluted error with version 1.2 as well. Output: Calculating and averaging the unit XH vectors from all structures. relax noe.read(run='noe', file='NOnoe500.list', dir=None, spectrum_type='ref', format='sparky', heteronuc='N', proton='HN', int_col=None) Reference spectrum. Sparky formatted data file. Opening the file 'NOnoe500.list' for reading. Number of header lines found: 2 Traceback (most recent call last): File /Applications/relax-1.2.14/relax, line 457, in module Relax() File /Applications/relax-1.2.14/relax, line 167, in __init__ self.interpreter.run() File /Applications/relax-1.2.14/prompt/interpreter.py, line 216, in run run_script(intro=self.relax.intro_string, local=self.local, script_file=self.relax.script_file, quit=1) File /Applications/relax-1.2.14/prompt/interpreter.py, line 391, in run_script console.interact(intro, local, script_file, quit) File /Applications/relax-1.2.14/prompt/interpreter.py, line 343, in interact_script execfile(script_file, local) File noe_12.py, line 35, in module noe.read(name, file='NOnoe500.list', spectrum_type='ref') File /Applications/relax-1.2.14/prompt/noe.py, line 226, in read self.__relax__.specific.noe.read(run=run, file=file, dir=dir, spectrum_type=spectrum_type, format=format, heteronuc=heteronuc, proton=proton, int_col=int_col) File /Applications/relax-1.2.14/specific_fns/noe.py, line 110, in read self.relax.generic.intensity.read(run=run, file=file, dir=dir, format=format, heteronuc=heteronuc, proton=proton, int_col=int_col, assign_func=self.assign_function) File /Applications/relax-1.2.14/generic_fns/intensity.py, line 232, in read res_num, H_name, X_name, intensity = self.intensity(self.file_data[i]) File /Applications/relax-1.2.14/generic_fns/intensity.py, line 92, in intensity_sparky intensity = float(line[self.int_col]) IndexError: list index out of range Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, This is definitely a bug! If you could file a bug report (https://gna.org/bugs/?func=additemgroup=relax), that would be much appreciated. If you would like a description of how this is done, please see section 3.3 of the relax manual (version 1.3) or the online HTML version (currently at http://www.nmr-relax.com/manual/1.3/node49.html). The reason this is failing is because this code in the 1.3 relax versions has not been converted to the new design yet. For me or the relax developers to implement this, first a system test should be added to the test suite. The test suite can be run by typing: $ relax --test-suite The system test is simply a very tiny, although complete, version of the analysis you would like to do. This can include a small subset of the base data, randomised if you'd like to protect the data prior to publication. With the small data subset, I can include your full analysis script into the test suite and execute it with the data. Then once the test passes, an NOE analysis will be possible. The data and script can be attached to the bug report. If you would like to help in getting this test implemented, that would be much appreciated. Thanks, Edward On Mon, Oct 6, 2008 at 3:31 PM, Tyler Reddy [EMAIL PROTECTED] wrote: Hello, I am having trouble with the noe.py script. I have peptide data with 6 backbone 15N labels, and I suspect it's a problem with input. I've tried several permutations of full PDB file and full .list Sparky format files as well as files trimmed down to include only the labeled residues. I'm not sure which is preferable/required for input, and if unlabeled residues can simply be left with a blank column value or if there is a special way to treat this? Here is a sample of the output I get with any of my tries so far: Scientific Python PDB parser. Loading all structures from the PDB file. Structure('select.pdb'): Peptide chain A of length 6 relax structure.load_spins(spin_id='@N', ave_pos=True) Adding the following spins to the relax data store. Mol_name Res_numRes_name Spin_num Spin_name A 254LEU68 N A 258LEU142N A 261GLY197N A 264LEU230N A 268ALA294N A 273LEU366N relax noe.read(file='NOnoe500_true.list', dir=None, spectrum_type='ref', format='sparky', heteronuc='N', proton='HN', int_col=None) Reference spectrum. Traceback (most recent call last): File /Applications/relax-1.3.1/relax, line 408, in module Relax() File /Applications/relax-1.3.1/relax, line 125, in __init__ self.interpreter.run(self.script_file
Re: Difficulty with noe.py sample script
Hi Edward, Thanks for your continued patience. Okay, it seems that aligning the data height column by the left most digit rather than the right most digit is not tolerated. Now the script (1.2) almost gets to the end, but there is still an error output, where the program opens noe.out for writing (see below). The noe.out file is created, but it's blank. Assuming I can get functional input for the 1.2 version (which is supposed to work), if this input doesn't work for the 1.3 version I'm happy to submit the bug report at that stage. Also, I think it would be very useful if a sample of functional input data, as you supplied for me in the previous email, would be incorporated in the relax manual. Different labs can have different naming conventions and formats, even within a Sparky list, and the column alignment can be tricky (even if trivial once you know about it). Also, if there is sensitivity to the nomenclature system employed (as some of your previous responses on this topic attest), then an example directly in the manual would definitely be appreciated. I know that part of the design philosophy is that it's very easy to incorporate new format tolerance into relax, but it's even faster for a new user who uses a slightly different format to just read the relevant section of the manual and format that way. Output: Calculating the unit XH vectors from the structure. relax noe.read(run='noe', file='NOnoe500_select.list', dir=None, spectrum_type='ref', format='sparky ', heteronuc='N', proton='HN', int_col=None) Reference spectrum. Sparky formatted data file. Opening the file 'NOnoe500_select.list' for reading. Number of header lines found: 2 relax noe.read(run='noe', file='noe500_select.list', dir=None, spectrum_type='sat', format='sparky', heteronuc='N', proton='HN', int_col=None) Saturated spectrum. Sparky formatted data file. Opening the file 'noe500_select.list' for reading. Number of header lines found: 2 relax noe.error(run='noe', error=15.4933, spectrum_type='ref', res_num=None, res_name=None) relax noe.error(run='noe', error=10.1449, spectrum_type='sat', res_num=None, res_name=None) relax calc(run='noe', print_flag=1) relax value.write(run='noe', param='noe', file='noe.out', dir=None, force=1) Opening the file 'noe.out' for writing. Traceback (most recent call last): File /Applications/relax-1.2.14/relax, line 457, in module Relax() File /Applications/relax-1.2.14/relax, line 167, in __init__ self.interpreter.run() File /Applications/relax-1.2.14/prompt/interpreter.py, line 216, in run run_script(intro=self.relax.intro_string, local=self.local, script_file=self.relax.script_file, q uit=1) File /Applications/relax-1.2.14/prompt/interpreter.py, line 391, in run_script console.interact(intro, local, script_file, quit) File /Applications/relax-1.2.14/prompt/interpreter.py, line 343, in interact_script execfile(script_file, local) File noe_12.py, line 75, in module value.write(name, param='noe', file='noe.out', force=1) File /Applications/relax-1.2.14/prompt/value.py, line 506, in write self.__relax__.generic.value.write(run=run, param=param, file=file, dir=dir, force=force) File /Applications/relax-1.2.14/generic_fns/value.py, line 537, in write self.write_data(file, return_value) File /Applications/relax-1.2.14/generic_fns/value.py, line 549, in write_data self.function_type = self.relax.data.run_types[self.relax.data.run_names.index(run)] ValueError: list.index(x): x not in list Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, This is again a bug, as relax hasn't managed to catch a file formatted in a way it cannot recognise and then respond with a error message which is understandable. The error indicates to me that relax is thinking this is a Sparky formatted peak list, but that the intensity column is missing (at least in one position). The file should look something like: Assignment w1 w2 Data Height LEU3N-HN122.454 8.397 129722 GLY4N-HN111.999 8.719 422375 SER5N-HN115.085 8.176 384180 MET6N-HN120.934 8.812 272100 ASP7N-HN122.394 8.750 174970 SER8N-HN113.916 7.836 218762 GLU11N-HN122.194 8.60430412 GLY12N-HN110.525 9.02890144 The NOE analysis works in version 1.2.14, but not in 1.3.1 yet. The idea for filing the bug report and then me creating a system test within relax using a tiny subset of the data was simply a suggestion that would allow me to quickly make the NOE analysis functional in the next relax 1.3 version. Then other relax users would benefit from having this code functional again too. But if you need the NOE data quickly, relax 1.2.14 will do the job with a Sparky peak list possessing all four columns. Regards, Edward On Tue, Oct 7, 2008 at 4:12 PM, Tyler Reddy [EMAIL
Difficulty with noe.py sample script
Hello, I am having trouble with the noe.py script. I have peptide data with 6 backbone 15N labels, and I suspect it's a problem with input. I've tried several permutations of full PDB file and full .list Sparky format files as well as files trimmed down to include only the labeled residues. I'm not sure which is preferable/required for input, and if unlabeled residues can simply be left with a blank column value or if there is a special way to treat this? Here is a sample of the output I get with any of my tries so far: Scientific Python PDB parser. Loading all structures from the PDB file. Structure('select.pdb'): Peptide chain A of length 6 relax structure.load_spins(spin_id='@N', ave_pos=True) Adding the following spins to the relax data store. Mol_name Res_numRes_name Spin_num Spin_name A 254LEU68 N A 258LEU142N A 261GLY197N A 264LEU230N A 268ALA294N A 273LEU366N relax noe.read(file='NOnoe500_true.list', dir=None, spectrum_type='ref', format='sparky', heteronuc='N', proton='HN', int_col=None) Reference spectrum. Traceback (most recent call last): File /Applications/relax-1.3.1/relax, line 408, in module Relax() File /Applications/relax-1.3.1/relax, line 125, in __init__ self.interpreter.run(self.script_file) File /Applications/relax-1.3.1/prompt/interpreter.py, line 270, in run return run_script(intro=self.__intro_string, local=self.local, script_file=script_file, quit=self.__quit_flag, show_script=self.__show_script, raise_relax_error=self.__raise_relax_error) File /Applications/relax-1.3.1/prompt/interpreter.py, line 531, in run_script return console.interact(intro, local, script_file, quit, show_script=show_script, raise_relax_error=raise_relax_error) File /Applications/relax-1.3.1/prompt/interpreter.py, line 427, in interact_script execfile(script_file, local) File noe.py, line 36, in module noe.read(file='NOnoe500_true.list', spectrum_type='ref') File /Applications/relax-1.3.1/prompt/noe.py, line 216, in read noe_obj.read(file=file, dir=dir, spectrum_type=spectrum_type, format=format, heteronuc=heteronuc, proton=proton, int_col=int_col) File /Applications/relax-1.3.1/specific_fns/noe.py, line 114, in read self.relax.generic.intensity.read(run=run, file=file, dir=dir, format=format, heteronuc=heteronuc, proton=proton, int_col=int_col, assign_func=self.assign_function) AttributeError: Noe instance has no attribute 'relax' ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Re: Compiling on Mac OS 10.4
I finally got the compilation to work. The compiler (gcc) apparently did not have the header file in its search path. I'm not sure it's the most elegant solution, but I just put the Python.h header file associated with Sparky in the path in my .cshrc file. set path=($path /Applications/Sparky.app/Contents/Resources/python2.4/include/python2.4/) Quoting Edward d'Auvergne [EMAIL PROTECTED]: Hi, I forgot to ask but if you could report if this was successful, or the steps you took to fix this, that would be much appreciated as the acheived message will help other users in the future. Regards, Edward On Thu, Oct 2, 2008 at 10:19 AM, Edward d'Auvergne [EMAIL PROTECTED] wrote: Hi, Welcome to the relax users mailing list. I've seen this problem before but can't remember exactly what is causing it. We should be able to fix this though. I'm pretty sure that the suggestions on the internet of using distutils for this are out of place. I use distutils for the minfx package which is a dependency for relax 1.3.1 (it used to be part of relax 1.2.x but was spun out as its own project for other people to use). But relax doesn't use distutils, so I think these suggestions for distutils on the internet are rubbish - just trolls trying to convince innocent users that the developers of the software should use distutils rather than the currently used compilation and installation methods. This problem has been encountered before by Douglas Kojetin douglas dot kojetin att gmail dot com (https://mail.gna.org/public/relax-devel/2006-01/msg00011.html). I think you'll find a solution here: https://mail.gna.org/public/relax-devel/2006-01/msg00014.html. The thread continues into the next month here: https://mail.gna.org/public/relax-devel/2006-02/msg0.html. Don't worry about the descriptions about using 'make', we've switched everything in relax to using 'scons'. Anyway, I hope this helps. Otherwise we'll have to find another solution. Regards, Edward On Wed, Oct 1, 2008 at 9:43 PM, Tyler Reddy [EMAIL PROTECTED] wrote: I've install all the dependencies that are documented, and I'd like to get the curve fitting portion of relax working on Mac OS 10.4. When I type 'scons' in the relax directory, I get an error with some undefined symbols, as below. A brief internet search seems to suggest that I need to use something call 'distutils,' but I'm not sure how this works! Any help would be appreciated. Tyler Output: scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... gcc -o maths_fns/relax_fit.so -bundle -bundle_loader /Library/Frameworks/Python.framework/Versions/2.5/bin/python -dynamic math s_fns/c_chi2.os maths_fns/exponential.os maths_fns/relax_fit.os /usr/libexec/gcc/i686-apple-darwin8/4.0.1/ld: Undefined symbols: _PyCObject_AsVoidPtr _PyCObject_Type _PyErr_Format _PyExc_RuntimeError _PyImport_ImportModule _PyObject_GetAttrString _PyArg_ParseTuple _PyArg_ParseTupleAndKeywords _PyErr_Print _PyErr_SetString _PyExc_ImportError _PyList_New _PyList_SetItem _PyList_Type _PyType_IsSubtype _Py_BuildValue _Py_InitModule4 __Py_NoneStruct collect2: ld returned 1 exit status scons: *** [maths_fns/relax_fit.so] Error 1 scons: building terminated because of errors. ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Compiling on Mac OS 10.4
I've install all the dependencies that are documented, and I'd like to get the curve fitting portion of relax working on Mac OS 10.4. When I type 'scons' in the relax directory, I get an error with some undefined symbols, as below. A brief internet search seems to suggest that I need to use something call 'distutils,' but I'm not sure how this works! Any help would be appreciated. Tyler Output: scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... gcc -o maths_fns/relax_fit.so -bundle -bundle_loader /Library/Frameworks/Python.framework/Versions/2.5/bin/python -dynamic math s_fns/c_chi2.os maths_fns/exponential.os maths_fns/relax_fit.os /usr/libexec/gcc/i686-apple-darwin8/4.0.1/ld: Undefined symbols: _PyCObject_AsVoidPtr _PyCObject_Type _PyErr_Format _PyExc_RuntimeError _PyImport_ImportModule _PyObject_GetAttrString _PyArg_ParseTuple _PyArg_ParseTupleAndKeywords _PyErr_Print _PyErr_SetString _PyExc_ImportError _PyList_New _PyList_SetItem _PyList_Type _PyType_IsSubtype _Py_BuildValue _Py_InitModule4 __Py_NoneStruct collect2: ld returned 1 exit status scons: *** [maths_fns/relax_fit.so] Error 1 scons: building terminated because of errors. ___ relax (http://nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users