Re: [ccp4bb] Refinement with native and anomalous data

[Piotr Wilk]

 >My advice is to simply cycle back-and-forth between the two data sets.
Refine against your high-resolution data until that converges, then simply
>switch to the other mtz file and use F+ F-, along with "anomalous
wavelength" and probably "refine orefine no", unless you want to refine the
>occupancies of Mn and other heavies.  Personally, I prefer to define
occupancy refinement explicitly using the REFMAC "occupancy group" etc.
>commands.  Once the SAD refinement has converged (as in positions,
occupancies, and B factors stop moving), then switch back to the high-   >
res.  Once that converges, shift back to anomalous again.  Then high-res,
then anomalous, then high-res, etc.

I haven't try that yet, but this definitely looks worth trying. I believe,
that this will give me a chance to use both sources of information
"sequentially" rather than "simultaneously". And what about rebuilding of
the model in-between refinement cycles (coot)?

>I understand that it is tempting to desire software that can somehow
incorporate data from disparate crystals and give you a single "right"
model, >but that can only happen if your data sets are highly isomorphous.
Think about it.  What if one data set has a serine side chain in the +
rotamer >and another data set has the same residue is - or trans ?  What
would you expect to find in your "right" model?

It is indeed tempting, but probably unrealistic as of today. Nevertheless
there is more and more parameters included in refinement protocols (for
example NMR data derived restraints, see
https://doi.org/10.1107/S2059798318000979). I admit that we only work with
the models which are only partly correct, but this should not stop us from
trying to get them ever better and newer protocols often aid this pursuit
(f.e. PDB_REDO).
In case of using data from two different crystals I'd expect the putative
serine to be in two alternative conformations with their occupancies
proportional to the strength of the signal from each crystal.
In my case the crystal belongs to C222(1) with 103.56 106.69 216.70 90.0
90.0 90.0 cell parameters, diffracting to ~1.85A. The crystal itself was a
big one with ~100um x 70um x 70um (on average). First the native data were
collected at 13.5keV with 2000 images x 0.1deg starting from the optimized
angle and resulted in redundancy of ~7.3 . Edge scan was performed and
beamline re-tuned to the peak of Mn edge. Second data set was 3600 x 0.1
deg from a different position of the same crystal (lucky to grow big ones).
The unit cell parameters differed by a fraction of an angstrom (as far as I
recall, but I will check exactly) so they are quite isomorphous. I believe
even more than some of the room-temperature data collected from multiple
crystals which were successful in phasing.

>Oscillating between data sets, however, allows you to have a well-defined
final refinement target, but for the things that are not well-defined by
>your final data set (such as anomalous signal or occupancy), you can "fill
in" those blank spaces by creating bias in the starting structure.

Isn't model bias what we usually try to minimize?

>In the present case, however, both data sets are from the same crystal, so
structural differences like rotamer shifts are not expected.  In fact, >why
not just merge the two data sets?  If you take your two sets of FP, put
them together with CAD, and scale them with SCALEIT, what is the >R-factor
between them?  If the R is less than 5-7% then there is seldom any reason
not to merge them, but if your two wavelengths are 10-15% >different you
should really ask yourself why.  Always a good idea to make an Fo-Fo map
and see where the largest difference peaks are.  If they >are on top of
heavy atoms then you might just be seeing the f' difference, and that is
OK, but you could also be seeing radiation-induced >changes, and those are
perhaps not what you are looking for?  Which data set was collected first?

Instead of merging I just copied some columns (these corresponding somehow
to anomalous differences) from the low energy mtz to the high energy one. I
can use this file sequentially alternating the refmac protocol, taking only
the relevant columns, but keeping the same Rfree flags.
I will also try merging both data sets for comparison. Admittedly I hope
for minimal radiation damage and there were no obvious indication for it
from the data processing, but I could check for it f.e. calculating Fo-Fo
map with first whatever 120deg- last 120deg. But this is a separate issue.

I will try both merging of the data sets and "alternating" protocol first
thing on Monday (I need my linux machine for it).

Thank you for advice,
Best regards,
  Piotrek

pt., 4 sty 2019 o 19:42 James Holton 
napisał(a):

> My advice is to simply cycle back-and-forth between the two data sets.
> Refine against your high-resolution data until that converges, then simply
> switch to the other mtz file and use F+ F-, along with "anomalous
> wavelength" and prob

Re: [ccp4bb] Refinement with native and anomalous data

[James Holton]

My advice is to simply cycle back-and-forth between the two data sets.  
Refine against your high-resolution data until that converges, then 
simply switch to the other mtz file and use F+ F-, along with "anomalous 
wavelength" and probably "refine orefine no", unless you want to refine 
the occupancies of Mn and other heavies. Personally, I prefer to define 
occupancy refinement explicitly using the REFMAC "occupancy group" etc. 
commands.  Once the SAD refinement has converged (as in positions, 
occupancies, and B factors stop moving), then switch back to the 
high-res.  Once that converges, shift back to anomalous again.  Then 
high-res, then anomalous, then high-res, etc.


Obviously, you want to have the same Free-R flags in both mtz files.

You might think that all this back-and-forth would be redundant. Doesn't 
"refinement" of a given model and given data set always converge to the 
same final coordinate set?  No, it does not.  All you need to do is 
"jiggle" your model and re-refine and you will get something new.  RMSDs 
usually around 0.4 A or so, depending on resolution.  What this means 
pragmatically is that any given refinement has a lot of "slop" in its 
final result, and is therefore quite sensitive to initial conditions.  
You can therefore easily incorporate information from multiple data sets 
by doing this switching back-and-forth.  The information from one data 
set then manifests as "bias" for the start of the next refinement.  No 
special software required!


I understand that it is tempting to desire software that can somehow 
incorporate data from disparate crystals and give you a single "right" 
model, but that can only happen if your data sets are highly 
isomorphous.  Think about it.  What if one data set has a serine side 
chain in the + rotamer and another data set has the same residue is - or 
trans ?  What would you expect to find in your "right" model?


Oscillating between data sets, however, allows you to have a 
well-defined final refinement target, but for the things that are not 
well-defined by your final data set (such as anomalous signal or 
occupancy), you can "fill in" those blank spaces by creating bias in the 
starting structure.


In the present case, however, both data sets are from the same crystal, 
so structural differences like rotamer shifts are not expected.  In 
fact, why not just merge the two data sets?  If you take your two sets 
of FP, put them together with CAD, and scale them with SCALEIT, what is 
the R-factor between them?  If the R is less than 5-7% then there is 
seldom any reason not to merge them, but if your two wavelengths are 
10-15% different you should really ask yourself why.  Always a good idea 
to make an Fo-Fo map and see where the largest difference peaks are.  If 
they are on top of heavy atoms then you might just be seeing the f' 
difference, and that is OK, but you could also be seeing 
radiation-induced changes, and those are perhaps not what you are 
looking for?  Which data set was collected first?


-James Holton
MAD Scientist


On 1/4/2019 2:16 AM, Steiner, Roberto wrote:

Dear Piotrek

You can find some info on the use of prior phase information in 
refinement with Refmac in


Acta Crystallogr D Biol Crystallogr. 
 2011 
Apr;67(Pt 4):355-67. doi: 10.1107/S0907444911001314. Epub 2011 Mar 18.



  REFMAC5 for the refinement of macromolecular crystal structures.

Murshudov GN 
1, 
Skubák P 
, 
Lebedev AA 
, 
Pannu NS 
, 
Steiner RA 
, 
Nicholls RA 
, 
Winn MD 
, 
Long F 
, 
Vagin AA 
.


and references therein.
As stated in the paper (paragraph 2.2.2), the incorporation of prior 
phase information by the refinement function is especially useful in 
the early and middle stages of model building and at all stages of 
structure solution at lower resolutions, owing to the improvement in 
the observation-to-parameter ratio.
Refinement in Refmac is very fast therefore the best thing (as you 
just did) is to try both options and see.


With best wishes
Roberto



On 4 Jan 2019, a

Re: [ccp4bb] Refinement with native and anomalous data

[Steiner, Roberto]

Dear Piotrek

You can find some info on the use of prior phase information in refinement with 
Refmac in

Acta Crystallogr D Biol 
Crystallogr. 2011 
Apr;67(Pt 4):355-67. doi: 10.1107/S0907444911001314. Epub 2011 Mar 18.
REFMAC5 for the refinement of macromolecular crystal structures.
Murshudov 
GN1,
 Skubák 
P,
 Lebedev 
AA,
 Pannu 
NS,
 Steiner 
RA,
 Nicholls 
RA,
 Winn 
MD,
 Long 
F,
 Vagin 
AA.

and references therein.

As stated in the paper (paragraph 2.2.2), the incorporation of prior phase 
information by the refinement function is especially useful in the early and 
middle stages of model building and at all stages of structure solution at 
lower resolutions, owing to the improvement in the observation-to-parameter 
ratio.
Refinement in Refmac is very fast therefore the best thing (as you just did) is 
to try both options and see.

With best wishes
Roberto



On 4 Jan 2019, at 09:55, Piotr Wilk 
mailto:wilk.piot...@gmail.com>> wrote:

Dear Eleanor,

I have used the ACORN previously for structure solution but I will have to read 
more about its functionality in structure refinement.
I have run two Refmac jobs using either native or anomalous data with otherwise 
default parameters resulting in R/Rfree of 0.2013/0.2458 and 0.1862/0.2274 
respectively for crystal diffracting to ~1.85A. This seems to me, that using 
anomalous signal in refinement can be useful at least in some cases.

Regards,
  Piotrek


Hmm - you can certainly generate "MAD" phases using the anom signal from one 
data set, and then PARROT or some such density modification tool to extend 
those phases for the higher resolution reflections..
Or ACORN can work well if he data resolution is high enough to give good phases 
for all the data


Then you can use those phases in the initial refinement procedure - the usual 
idea is to use them till the R factor drops below 30% or 35% then just refine 
against the Fobs, phasing just from the model .

But I dont think it is ever worth working at a limited resolution..


Dear Eleanor,

thank you for your comment. My crystals of interest diffract with dmin usually 
between 1.4 and 1.9 A (in high energy data set) and significant anomalous 
signal extends usually to approx. 4 A (in low energy data set).
Certainly I do agree, that in "EITHER, OR" situation one can check both 
approaches, compare the results and take the more convincing one. I can easily 
do that in Refmac running one job against data with Friedel pairs merged and 
parallel one against data with Friedel pairs unmerged. I was considering rather 
an "AND" scenario in which in addition to high resolution data (FP) I'd include 
information from anomalous signal (F+ F-, DANO). I understand that this should 
increase number of observations from a given sample and therefore help to 
refine positions, occupancies and perhaps ADPs for at least a fraction of atoms 
in a model (Mn ions and S in my case). I imagine it as somehow analogical to 
adding geometrical restrains derived from very high resolution data to 
refinement protocols.
I was wondering first of all if my reasoning is sensible and if there is an 
existing protocol to try this?

With kind regards,
 Piotrek

czw., 3 sty 2019 o 22:00 Eleanor Dodson 
mailto:eleanor.dod...@york.ac.uk>> napisał(a):
Hmm - you can certainly generate "MAD" phases using the anom signal from one 
data set, and then PARROT or some such density modification tool to extend 
those phases for the higher resolution reflections..
Or ACORN can work well if he data resolution is high enough to give good phases 
for all the data


Then you can use those phases in the initial refinement procedure - the usual 
idea is to use them till the R factor drops below 30% or 35% then just refine 
against the Fobs, phasing just from the model .

But I dont think it is ever worth working at a limited resolution..

eleanor

On Thu, 3 Jan 2019 at 20:40, Piotr Wilk 
mailto:wilk.piot...@gmail.com>> wrote:
Dear Eleanor,

thank you for your com

Re: [ccp4bb] Refinement with native and anomalous data

[Piotr Wilk]

Dear Eleanor,

I have used the ACORN previously for structure solution but I will have to
read more about its functionality in structure refinement.
I have run two Refmac jobs using either native or anomalous data with
otherwise default parameters resulting in R/Rfree of 0.2013/0.2458 and
0.1862/0.2274 respectively for crystal diffracting to ~1.85A. This seems to
me, that using anomalous signal in refinement can be useful at least in
some cases.

Regards,
  Piotrek


Hmm - you can certainly generate "MAD" phases using the anom signal from
one data set, and then PARROT or some such density modification tool to
extend those phases for the higher resolution reflections..
Or ACORN can work well if he data resolution is high enough to give good
phases for all the data


Then you can use those phases in the initial refinement procedure - the
usual idea is to use them till the R factor drops below 30% or 35% then
just refine against the Fobs, phasing just from the model .

But I dont think it is ever worth working at a limited resolution..


Dear Eleanor,

thank you for your comment. My crystals of interest diffract with dmin
usually between 1.4 and 1.9 A (in high energy data set) and significant
anomalous signal extends usually to approx. 4 A (in low energy data set).
Certainly I do agree, that in "EITHER, OR" situation one can check both
approaches, compare the results and take the more convincing one. I can
easily do that in Refmac running one job against data with Friedel pairs
merged and parallel one against data with Friedel pairs unmerged. I was
considering rather an "AND" scenario in which in addition to high
resolution data (FP) I'd include information from anomalous signal (F+ F-,
DANO). I understand that this should increase number of observations from a
given sample and therefore help to refine positions, occupancies and
perhaps ADPs for at least a fraction of atoms in a model (Mn ions and S in
my case). I imagine it as somehow analogical to adding geometrical
restrains derived from very high resolution data to refinement protocols.
I was wondering first of all if my reasoning is sensible and if there is an
existing protocol to try this?

With kind regards,
 Piotrek

czw., 3 sty 2019 o 22:00 Eleanor Dodson 
napisał(a):

> Hmm - you can certainly generate "MAD" phases using the anom signal from
> one data set, and then PARROT or some such density modification tool to
> extend those phases for the higher resolution reflections..
> Or ACORN can work well if he data resolution is high enough to give good
> phases for all the data
>
>
> Then you can use those phases in the initial refinement procedure - the
> usual idea is to use them till the R factor drops below 30% or 35% then
> just refine against the Fobs, phasing just from the model .
>
> But I dont think it is ever worth working at a limited resolution..
>
> eleanor
>
> On Thu, 3 Jan 2019 at 20:40, Piotr Wilk  wrote:
>
>> Dear Eleanor,
>>
>> thank you for your comment. My crystals of interest diffract with dmin
>> usually between 1.4 and 1.9 A (in high energy data set) and significant
>> anomalous signal extends usually to approx. 4 A (in low energy data set).
>> Certainly I do agree, that in "EITHER, OR" situation one can check both
>> approaches, compare the results and take the more convincing one. I can
>> easily do that in Refmac running one job against data with Friedel pairs
>> merged and parallel one against data with Friedel pairs unmerged. I was
>> considering rather an "AND" scenario in which in addition to high
>> resolution data (FP) I'd include information from anomalous signal (F+ F-,
>> DANO). I understand that this should increase number of observations from a
>> given sample and therefore help to refine positions, occupancies and
>> perhaps ADPs for at least a fraction of atoms in a model (Mn ions and S in
>> my case). I imagine it as somehow analogical to adding geometrical
>> restrains derived from very high resolution data to refinement protocols.
>> I was wondering first of all if my reasoning is sensible and if there is
>> an existing protocol to try this?
>>
>> With kind regards,
>>  Piotrek
>>
>> czw., 3 sty 2019 o 17:02 Eleanor Dodson 
>> napisał(a):
>>
>>> I think any decision depends on the resolution of your two data sets. If
>>> they are very different I would choose the higher resolution one.
>>>
>>> If that is the Anom data then I would use the anom signal at least in
>>> the first cycles to improve the phases..
>>>
>>> Eleanor
>>>
>>> On Thu, 3 Jan 2019 at 14:59, Piotr Wilk  wrote:
>>>
 Dear CCP4 experts,

 I'd like to ask your opinion about using anomalous signal in refinement
 of crystal structures in addition to using high resolution native data.
 I am working on a series of structures for which I have collected two
 data sets (from the same crystal):
 1 - native with higher resolution
 2 - anomalous at MN absorptio

Re: [ccp4bb] Refinement with native and anomalous data

[Eleanor Dodson]

I think any decision depends on the resolution of your two data sets. If
they are very different I would choose the higher resolution one.

If that is the Anom data then I would use the anom signal at least in the
first cycles to improve the phases..

Eleanor

On Thu, 3 Jan 2019 at 14:59, Piotr Wilk  wrote:

> Dear CCP4 experts,
>
> I'd like to ask your opinion about using anomalous signal in refinement of
> crystal structures in addition to using high resolution native data.
> I am working on a series of structures for which I have collected two data
> sets (from the same crystal):
> 1 - native with higher resolution
> 2 - anomalous at MN absorption edge peak.
> The structures were solved with MR and preliminary refinement using the
> native data only yields decent statistics, but I also use anomalous data to
> verify presence and position of manganese ions. For this I used ANODE which
> lists four strong peaks (~30 sigma) as expected for manganese ions and
> around 45 weaker peaks (~9-5 sigma) for sulfur atoms in Cys and Met. I am
> happy to use this information in model building but I was also wondering if
> (and how) beneficial would it be to use both high resolution structure
> factors and somehow lower resolution yet highly specific anomalous signal
> in the same round of refinement?
> In Refmac5 I can use either refinement with "no prior phase information"
> taking FP and SIGFP or "SAD data directly" with SIGFP F(+) SIGF(+) F(-)
> SIGF(-), but I didn't find any "MAD" option to use both.
> I have the following columns in my mtz files:
> for native data: H K L FP SIGFP FreeRflag
> for anomalous data : H K L FP SIGFP F(+) SIGF(+) F(-) SIGF(-) FreeRflag
> or :   H K L FP SIGFP DANO SIGDANO ISYM FreeRflag
>
> I could use CAD to merge the interesting columns into a single mtz file
> containing:
>  H K L FP SIGFP FreeRflag F(+) SIGF(+) F(-) SIGF(-) DANO SIGDANO
>
> I'd appreciate any comments or advise how to use both sources of
> information in the refinement.
>
> I wish you all a Happy New Year.
> Kind regards,
> Piotrek
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1

[ccp4bb] Refinement with native and anomalous data

[Piotr Wilk]

Dear CCP4 experts,

I'd like to ask your opinion about using anomalous signal in refinement of
crystal structures in addition to using high resolution native data.
I am working on a series of structures for which I have collected two data
sets (from the same crystal):
1 - native with higher resolution
2 - anomalous at MN absorption edge peak.
The structures were solved with MR and preliminary refinement using the
native data only yields decent statistics, but I also use anomalous data to
verify presence and position of manganese ions. For this I used ANODE which
lists four strong peaks (~30 sigma) as expected for manganese ions and
around 45 weaker peaks (~9-5 sigma) for sulfur atoms in Cys and Met. I am
happy to use this information in model building but I was also wondering if
(and how) beneficial would it be to use both high resolution structure
factors and somehow lower resolution yet highly specific anomalous signal
in the same round of refinement?
In Refmac5 I can use either refinement with "no prior phase information"
taking FP and SIGFP or "SAD data directly" with SIGFP F(+) SIGF(+) F(-)
SIGF(-), but I didn't find any "MAD" option to use both.
I have the following columns in my mtz files:
for native data: H K L FP SIGFP FreeRflag
for anomalous data : H K L FP SIGFP F(+) SIGF(+) F(-) SIGF(-) FreeRflag
or :   H K L FP SIGFP DANO SIGDANO ISYM FreeRflag

I could use CAD to merge the interesting columns into a single mtz file
containing:
 H K L FP SIGFP FreeRflag F(+) SIGF(+) F(-) SIGF(-) DANO SIGDANO

I'd appreciate any comments or advise how to use both sources of
information in the refinement.

I wish you all a Happy New Year.
Kind regards,
Piotrek



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1