[ccp4bb] refmac output query

[Emmanuel Saridakis]

Dear All,

A quick question about Refmac on ccp4i. When refining an RNA oligonucleotide 
structure using the findwaters option, the pdb output labels all the atoms as 
HETATM (inluding the original RNA atoms which were not so labelled in the input 
pdb). It is of course trivial to correct this, but I was wondering if there is 
a hidden problem somewhere, that might influence more crucial things.

Many thanks,
Emmanuel

-- 
Dr. Emmanuel Saridakis
Institute of Nanoscience and Nanotechnology
National Centre for Scientific Research "DEMOKRITOS"
15310 Athens
GREECE



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] refmac output

[Eleanor Dodson]

the CCP4 gui does just that - provides map coeffs by default..
You can seek out the full REFMAC output if needed but it is not there by
default.
Eleanor

On 3 August 2017 at 02:04, Pavel Afonine  wrote:

> Hi Ed,
>
> your suggestion makes perfect sense to me, and it's trivial to add an
> option to do what you want. This will be available in next Phenix nightly
> build (clearly not tomorrow given today's power outage).
>
> Command line: use "write_map_coefficients_only=True" (by default is is
> False).
>
> Refinement GUI: Configure -> Output -> Other options -> tick "Write map
> coefficients only" box.
>
> Pavel
>
>
> On Wed, Aug 2, 2017 at 1:12 PM, Edwin Pozharski 
> wrote:
>
>>
>> Just to clarify, how do you use the extra columns in this scenario?  My
>> suggestion was to have the output file that includes only the map
>> coefficient columns, so you still can look at the map.  IIRC, FP/SIGFP
>> columns from refmac output are actually modified from the input (scaled
>> with Boverall), so it was not recommended to use refmac output as input of
>> any kind.
>>
>> Also, to provide context, my comment resulted from dealing with a large
>> chunk of data that included ~200 output mtz files - which is Gb-sized
>> tarball that had to be uploaded/downloaded.  Not the end of the world, but
>> cutting it in half seemed like a good idea at the time. :)  Not hard to
>> script that, of course.
>>
>> I am not necessarily advocating for skinny files to be the default, but
>> as it stands, refmac/buster/phenix do not even provide the option of doing
>> it (It's entirely possible that I am wrong here on specifics and will get
>> corrected by Garib, Gerard and Pavel).
>>
>> Cheers,
>>
>> Ed.
>>
>> Edwin Pozharski, PhD, Assistant Professor
>> University of Maryland, Baltimore
>> --
>> When the Way is forgotten duty and justice appear;
>> Then knowledge and wisdom are born along with hypocrisy.
>> When harmonious relationships dissolve then respect and devotion arise;
>> When a nation falls to chaos then loyalty and patriotism are born.
>> -- / Lao Tse /
>>
>>
>

Re: [ccp4bb] refmac output

[Pavel Afonine]

Hi Ed,

your suggestion makes perfect sense to me, and it's trivial to add an
option to do what you want. This will be available in next Phenix nightly
build (clearly not tomorrow given today's power outage).

Command line: use "write_map_coefficients_only=True" (by default is is
False).

Refinement GUI: Configure -> Output -> Other options -> tick "Write map
coefficients only" box.

Pavel

On Wed, Aug 2, 2017 at 1:12 PM, Edwin Pozharski 
wrote:

>
> Just to clarify, how do you use the extra columns in this scenario?  My
> suggestion was to have the output file that includes only the map
> coefficient columns, so you still can look at the map.  IIRC, FP/SIGFP
> columns from refmac output are actually modified from the input (scaled
> with Boverall), so it was not recommended to use refmac output as input of
> any kind.
>
> Also, to provide context, my comment resulted from dealing with a large
> chunk of data that included ~200 output mtz files - which is Gb-sized
> tarball that had to be uploaded/downloaded.  Not the end of the world, but
> cutting it in half seemed like a good idea at the time. :)  Not hard to
> script that, of course.
>
> I am not necessarily advocating for skinny files to be the default, but as
> it stands, refmac/buster/phenix do not even provide the option of doing it
> (It's entirely possible that I am wrong here on specifics and will get
> corrected by Garib, Gerard and Pavel).
>
> Cheers,
>
> Ed.
>
> Edwin Pozharski, PhD, Assistant Professor
> University of Maryland, Baltimore
> --
> When the Way is forgotten duty and justice appear;
> Then knowledge and wisdom are born along with hypocrisy.
> When harmonious relationships dissolve then respect and devotion arise;
> When a nation falls to chaos then loyalty and patriotism are born.
> -- / Lao Tse /
>
>

Re: [ccp4bb] refmac output

[Ethan A Merritt]

On Wednesday, 02 August, 2017 16:12:30 Edwin Pozharski wrote:
> Just to clarify, how do you use the extra columns in this scenario?  My
> suggestion was to have the output file that includes only the map
> coefficient columns, so you still can look at the map.  IIRC, FP/SIGFP
> columns from refmac output are actually modified from the input (scaled
> with Boverall), so it was not recommended to use refmac output as input of
> any kind.

The scaled Fs are useful to recalculate R as a function of whatever, and
to calculate alternative R values (e.g. for Hamilton R test or comparison
to shelxl R1 R2). It's nice to carry though DANO and SIGDANO so that one
can generate anomalous maps. In general I'd rather err on the side of
including everything rather than discarding something that may be useful
later.  In any case it's trivial to run one additional script to filter
unwanted columns if file size really is a controlling factor.

Ethan




> 
> Also, to provide context, my comment resulted from dealing with a large
> chunk of data that included ~200 output mtz files - which is Gb-sized
> tarball that had to be uploaded/downloaded.  Not the end of the world, but
> cutting it in half seemed like a good idea at the time. :)  Not hard to
> script that, of course.
> 
> I am not necessarily advocating for skinny files to be the default, but as
> it stands, refmac/buster/phenix do not even provide the option of doing it
> (It's entirely possible that I am wrong here on specifics and will get
> corrected by Garib, Gerard and Pavel).
> 
> Cheers,
> 
> Ed.
> 
> Edwin Pozharski, PhD, Assistant Professor
> University of Maryland, Baltimore
> --
> When the Way is forgotten duty and justice appear;
> Then knowledge and wisdom are born along with hypocrisy.
> When harmonious relationships dissolve then respect and devotion arise;
> When a nation falls to chaos then loyalty and patriotism are born.
> -- / Lao Tse /

-- 
Ethan A Merritt
Biomolecular Structure Center,  K-428 Health Sciences Bldg
MS 357742,   University of Washington, Seattle 98195-7742

Re: [ccp4bb] refmac output

[Edwin Pozharski]

Just to clarify, how do you use the extra columns in this scenario?  My
suggestion was to have the output file that includes only the map
coefficient columns, so you still can look at the map.  IIRC, FP/SIGFP
columns from refmac output are actually modified from the input (scaled
with Boverall), so it was not recommended to use refmac output as input of
any kind.

Also, to provide context, my comment resulted from dealing with a large
chunk of data that included ~200 output mtz files - which is Gb-sized
tarball that had to be uploaded/downloaded.  Not the end of the world, but
cutting it in half seemed like a good idea at the time. :)  Not hard to
script that, of course.

I am not necessarily advocating for skinny files to be the default, but as
it stands, refmac/buster/phenix do not even provide the option of doing it
(It's entirely possible that I am wrong here on specifics and will get
corrected by Garib, Gerard and Pavel).

Cheers,

Ed.

Edwin Pozharski, PhD, Assistant Professor
University of Maryland, Baltimore
--
When the Way is forgotten duty and justice appear;
Then knowledge and wisdom are born along with hypocrisy.
When harmonious relationships dissolve then respect and devotion arise;
When a nation falls to chaos then loyalty and patriotism are born.
-- / Lao Tse /

Re: [ccp4bb] refmac output

[Diana Tomchick]

Yes, I agree! This (“Please look at my structure, and here are my files from 
the last cycle of refinement") happens to me almost every week. :)

Diana

**
Diana R. Tomchick
Professor
Departments of Biophysics and Biochemistry
University of Texas Southwestern Medical Center
5323 Harry Hines Blvd.
Rm. ND10.214A
Dallas, TX 75390-8816
diana.tomch...@utsouthwestern.edu
(214) 645-6383 (phone)
(214) 645-6353 (fax)

On Aug 1, 2017, at 10:48 AM, James Holton  wrote:

As someone who uses those "superfluous" columns all the time, I would like to 
chime in in favor of keeping the default output columns of refmac.  If only I 
had a nickle for every time someone asked me to "look at" a structure and only 
gave me the output files of refinement.  Kind of ties your hands.

I have always been a fan of erring on the side of providing information in 
output files.  How hard is it to delete something? How hard is it to get it 
back after you deleted it?

My two cents,

-James Holton
MAD Scientist

On 7/31/2017 8:57 AM, Edwin Pozharski wrote:
> I know space is cheap these days, but is there a reason for Refmac to 
> generate all those extra columns in the output mtz file?  Refmac (as well as 
> phenix.refine and buster-tnt) output mtz file is almost always used for only 
> one purpose - look at the map in coot.  You only need 4 columns for that, not 
> 14.  Other columns are useful for testing, but why not make them optional?
>
> This would certainly be a low priority - one can easily delete extra columns 
> using, say, sftools.
>
> Cheers,
>
> Ed.
>
> ---
> Hurry up, before we all come to our senses!
> Julien, King of Lemurs
>




UT Southwestern


Medical Center



The future of medicine, today.

Re: [ccp4bb] refmac output

[James Holton]

As someone who uses those "superfluous" columns all the time, I would 
like to chime in in favor of keeping the default output columns of 
refmac.  If only I had a nickle for every time someone asked me to "look 
at" a structure and only gave me the output files of refinement.  Kind 
of ties your hands.


I have always been a fan of erring on the side of providing information 
in output files.  How hard is it to delete something? How hard is it to 
get it back after you deleted it?


My two cents,

-James Holton
MAD Scientist

On 7/31/2017 8:57 AM, Edwin Pozharski wrote:
I know space is cheap these days, but is there a reason for Refmac to 
generate all those extra columns in the output mtz file?  Refmac (as 
well as phenix.refine and buster-tnt) output mtz file is almost always 
used for only one purpose - look at the map in coot.  You only need 4 
columns for that, not 14.  Other columns are useful for testing, but 
why not make them optional?


This would certainly be a low priority - one can easily delete extra 
columns using, say, sftools.


Cheers,

Ed.

---
Hurry up, before we all come to our senses!
Julien, King of Lemurs

Re: [ccp4bb] refmac output

[Pavel Afonine]

>
> I know space is cheap these days, but is there a reason for Refmac to
> generate all those extra columns in the output mtz file?  Refmac (as well
> as phenix.refine and buster-tnt) output mtz file is almost always used for
> only one purpose - look at the map in coot.  You only need 4 columns for
> that, not 14.  Other columns are useful for testing, but why not make them
> optional?
>

a phenix.refine run creates an MTZ file with four kinds of data:

1) Copy of input data. Why? For convenience and consistency. Inputs may not
necessarily be in MTZ format and may be spread across multiple files of
different format.

2) Data that were actually used in refinement. Why? A user has options to
cut resolution from both ends, as well as apply cutting by sigma. Plus,
phenix.refine may choose not to use a handful of reflections as outliers
(Read, 1999). So it may be good to have set of reflections that were used
in given refinement run.

3) Model in reciprocal space: Fmodel. Why? This is a reciprocal space
representation of what's in PDB file except that it is richer because
contains not only atomic model (Fcalc) but also solvent contributions (bulk
and non-uniform) as well as all scales. Fmodel taken from this array and
Fobs from "2)" are expected to reproduce reported R-factor exactly.

4) Fourier map coefficients (2mFobs-DFmodel, mFobs-DFmodel, anomalous map
if applicable).

"1)" and "3)" can be optional:
- with trivial scripting one can obtain Fmodel using data from "2)" and
"4)".
- "1)" duplicates inputs. It's not unreasonable to assume they are
still available by the time you finalize your structure.
But.. as you pointed out space is cheap and personally I find it much
easier to have relevant arrays of data centralized in one place (file)
rather than scattered across hard drive.

All the best,
Pavel

Re: [ccp4bb] refmac output

[Jon Agirre]

Perhaps a good opportunity for getting rid of (scaled) F and SIGF too?

Certain pipelines need Refmac's phase estimates (Buccaneer and Crank2 off
the top of my head), but I can't see how activating an 'expert mode' or
'developer mode' in order to get them would be a problem for their authors.

Cheers,

Jon


On 31 July 2017 at 16:57, Edwin Pozharski  wrote:

> I know space is cheap these days, but is there a reason for Refmac to
> generate all those extra columns in the output mtz file?  Refmac (as well
> as phenix.refine and buster-tnt) output mtz file is almost always used for
> only one purpose - look at the map in coot.  You only need 4 columns for
> that, not 14.  Other columns are useful for testing, but why not make them
> optional?
>
> This would certainly be a low priority - one can easily delete extra
> columns using, say, sftools.
>
> Cheers,
>
> Ed.
>
> ---
> Hurry up, before we all come to our senses!
> Julien, King of Lemurs
>
>


-- 
Dr Jon Agirre
York Structural Biology Laboratory / Department of Chemistry
University of York, Heslington, YO10 5DD, York, England
http://www.york.ac.uk/chemistry/research/ysbl/people/staff/jagirre/
Twitter: @alwaysonthejazz
+44 (0) 1904 32 8270

[ccp4bb] refmac output

[Edwin Pozharski]

I know space is cheap these days, but is there a reason for Refmac to
generate all those extra columns in the output mtz file?  Refmac (as well
as phenix.refine and buster-tnt) output mtz file is almost always used for
only one purpose - look at the map in coot.  You only need 4 columns for
that, not 14.  Other columns are useful for testing, but why not make them
optional?

This would certainly be a low priority - one can easily delete extra
columns using, say, sftools.

Cheers,

Ed.

---
Hurry up, before we all come to our senses!
Julien, King of Lemurs