Re: [ccp4bb] quantifying electron density

[Wim Burmeister]

Hello, 

I think, in principle this is possible if you use about internal controls. I 
used this approach a long time ago : 

Burmeister WP, Guilligay D, Cusack S, Wadell G, Arnberg N. Crystal structure of 
species D adenovirus fiber knobs and their sialic acid binding sites. J Virol. 
2004 Jul;78(14):7727-36. doi: 10.1128/JVI.78.14.7727-7736.2004. PMID: 15220447; 
PMCID: PMC434083. 

Cheers 
Wim 



De: "Hughes, Jonathan"  
À: "CCP4BB"  
Envoyé: Mercredi 28 Juin 2023 18:16:22 
Objet: [ccp4bb] quantifying electron density 

hello everyone, 
is there software that can use an electron density map to quantify the number 
of electrons in a selected volume somewhere in a protein? 
cheers 
jon 

-- 
Professor Jon Hughes, BSc, PhD 
Department of Physics 
Free University of Berlin 
Arnimallee 14 
14195 Berlin 
Germany 
mobile: (+49/0)1757929098 
email: lv...@posteo.de 
homepage: 
http://www.uni-giessen.de/fbz/fb08/Inst/pflphys 
Sent without the use of Apple products 

 

To unsubscribe from the CCP4BB list, click the following link: 
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/ 

-- 
Wim Burmeister 
Professor 
Institut de Biologie Structurale (IBS) CIBB 
71 avenue des Martyrs / CS 20192 
38044 Grenoble Cedex 9, FRANCE 
E-mail: [ mailto:wim.burmeis...@ibs.fr | wim.burmeis...@ibs.fr ] 
Mobile: +33 (0) 7 50 49 19 91 
[ 
http://www.ibs.fr/research/research-groups/viral-replication-machines-group-m-jamin/team-03/article/poxvirus-replication-machinery-presentation/
 | website ] 




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] quantifying electron density

[Alexandre Ourjoumtsev]

Dear Sriram, dear Jon, 

The sum of values of a map (always calculated at some finite resolution) is not 
the same as a sum of density values. 
In such maps, a contribution of each atom is a function oscillating with the 
distance to the atomic center. 
Even when the height of these ripples is relatively small, surprisingly, the 
sum (integral) of the map values converges very-very slowly (not like for a 
density itself!) with the radius at which you truncate you sum. See for example 
§3.2 and Figs. 3-4 in a our recent article in Acta Cryst D (2022), D78, 
1451-1468. 

So one should be very careful with interpretations of such formally calculated 
sums (again, even when you have F000, absolute scale etc). 

With best wishes, 

Sacha Urzhumtsev 

- Le 29 Juin 23, à 3:10, sriram raghavan  a écrit 
: 

> Dear Pavel,

> I came across an example of calculating "electron_sums_around_atom" in [
> https://github.com/cctbx/cctbx_project/blob/master/cctbx/examples/fft_map_electron_density_around_atom.py
> | cctbx ] . If we don't zero the F000 or if we normalize the density of the
> real-space map by the unit cell volume and the scattering factor of the unit
> cell (F000), we can obtain the value of the density and sum the values of the
> positions of the obtained grid points on the map. This approach will yield the
> electron density sum across the volume, correct?

> Additionally, there is a tool called [
> https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0236894 |
> PDB-EDA ] that performs a similar calculation but not on selected volume.

> Best regards,
> S. Sriram

> On Thu, Jun 29, 2023 at 4:03 AM Pavel Afonine < [ mailto:pafon...@gmail.com |
> pafon...@gmail.com ] > wrote:

>> Hi Jon,
>> not really the answer to your question but.. This may be very tricky to do
>> because what you look at is not an electron density map but its Fourier image
>> of finite resolution phased by crystal model (that has errors), plus
>> experimental errors, and missing F000 (which is not measured as part of your
>> diffraction experiment). So.. if such a software exists I'd be very cautious
>> interpreting the results you get from it!
>> Pavel

>> On Wed, Jun 28, 2023 at 9:16 AM Hughes, Jonathan < [
>> mailto:jon.hug...@bot3.bio.uni-giessen.de | 
>> jon.hug...@bot3.bio.uni-giessen.de
>> ] > wrote:

>>> hello everyone,
>>> is there software that can use an electron density map to quantify the 
>>> number of
>>> electrons in a selected volume somewhere in a protein?
>>> cheers
>>> jon

>>> --
>>> Professor Jon Hughes, BSc, PhD
>>> Department of Physics
>>> Free University of Berlin
>>> Arnimallee 14
>>> 14195 Berlin
>>> Germany
>>> mobile: (+49/0)1757929098
>>> email: [ mailto:lv...@posteo.de | lv...@posteo.de ]
>>> homepage:
>>> [ http://www.uni-giessen.de/fbz/fb08/Inst/pflphys |
>>> http://www.uni-giessen.de/fbz/fb08/Inst/pflphys ]
>>> Sent without the use of Apple products

>>> 

>>> To unsubscribe from the CCP4BB list, click the following link:
>>> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 |
>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ]

>>> This message was issued to members of [ http://www.jiscmail.ac.uk/CCP4BB |
>>> www.jiscmail.ac.uk/CCP4BB ] , a mailing list hosted by [
>>> http://www.jiscmail.ac.uk/ | www.jiscmail.ac.uk ] , terms & conditions are
>>> available at [ https://www.jiscmail.ac.uk/policyandsecurity/ |
>>> https://www.jiscmail.ac.uk/policyandsecurity/ ]

>> To unsubscribe from the CCP4BB list, click the following link:
>> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 |
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ]
> --
> With Regards

> S.Sriram

> To unsubscribe from the CCP4BB list, click the following link:
> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 |
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ]



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] quantifying electron density

[sriram raghavan]

Dear Pavel,

I came across an example of calculating "electron_sums_around_atom" in cctbx
.
If we don't zero the F000 or if we normalize the density of the real-space
map by the unit cell volume and the scattering factor of the unit cell
(F000), we can obtain the value of the density and sum the values of the
positions of the obtained grid points on the map. This approach will yield
the electron density sum across the volume, correct?

Additionally, there is a tool called PDB-EDA

that performs a similar calculation but not on selected volume.

Best regards,
S. Sriram


On Thu, Jun 29, 2023 at 4:03 AM Pavel Afonine  wrote:

> Hi Jon,
> not really the answer to your question but.. This may be very tricky to do
> because what you look at is not an electron density map but its Fourier
> image of finite resolution phased by crystal model (that has errors), plus
> experimental errors, and missing F000 (which is not measured as part of
> your diffraction experiment). So.. if such a software exists I'd be very
> cautious interpreting the results you get from it!
> Pavel
>
>
> On Wed, Jun 28, 2023 at 9:16 AM Hughes, Jonathan <
> jon.hug...@bot3.bio.uni-giessen.de> wrote:
>
>> hello everyone,
>> is there software that can use an electron density map to quantify the
>> number of electrons in a selected volume somewhere in a protein?
>> cheers
>> jon
>>
>> --
>> Professor Jon Hughes, BSc, PhD
>> Department of Physics
>> Free University of Berlin
>> Arnimallee 14
>> 14195 Berlin
>> Germany
>> mobile:   (+49/0)1757929098
>> email: lv...@posteo.de
>> homepage:
>> http://www.uni-giessen.de/fbz/fb08/Inst/pflphys
>> Sent without the use of Apple products
>>
>> 
>>
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>>
>> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a
>> mailing list hosted by www.jiscmail.ac.uk, terms & conditions are
>> available at https://www.jiscmail.ac.uk/policyandsecurity/
>>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>
-- 
With Regards

S.Sriram



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] quantifying electron density

[Pavel Afonine]

Hi Jon,
not really the answer to your question but.. This may be very tricky to do
because what you look at is not an electron density map but its Fourier
image of finite resolution phased by crystal model (that has errors), plus
experimental errors, and missing F000 (which is not measured as part of
your diffraction experiment). So.. if such a software exists I'd be very
cautious interpreting the results you get from it!
Pavel


On Wed, Jun 28, 2023 at 9:16 AM Hughes, Jonathan <
jon.hug...@bot3.bio.uni-giessen.de> wrote:

> hello everyone,
> is there software that can use an electron density map to quantify the
> number of electrons in a selected volume somewhere in a protein?
> cheers
> jon
>
> --
> Professor Jon Hughes, BSc, PhD
> Department of Physics
> Free University of Berlin
> Arnimallee 14
> 14195 Berlin
> Germany
> mobile:   (+49/0)1757929098
> email: lv...@posteo.de
> homepage:
> http://www.uni-giessen.de/fbz/fb08/Inst/pflphys
> Sent without the use of Apple products
>
> 
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a
> mailing list hosted by www.jiscmail.ac.uk, terms & conditions are
> available at https://www.jiscmail.ac.uk/policyandsecurity/
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

[ccp4bb] quantifying electron density

[Hughes, Jonathan]

hello everyone,
is there software that can use an electron density map to quantify the number 
of electrons in a selected volume somewhere in a protein?
cheers
jon 

--
Professor Jon Hughes, BSc, PhD
Department of Physics
Free University of Berlin
Arnimallee 14
14195 Berlin
Germany
mobile:   (+49/0)1757929098
email: lv...@posteo.de 
homepage:
http://www.uni-giessen.de/fbz/fb08/Inst/pflphys
Sent without the use of Apple products



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Tim Gruene]

Dear Neno,

when I asked the same question a few months ago, I ended up writing my
own program. We used it for our study of coking in the zeolite ZSM5
(https://doi.org/10.1002/anie.202205413). It is not at all user
friendly, and it reads in FCF files, rather than MTZ files. I can
send you the source code nevertheless in case you are interested, or a
linux binary. It is easy to create an FCF file from your MTZ file with
e.g. pdb2ins and a quick run of SHELXL with 'CGLS 0'.

Best,
Tim


On Wed, 10 Aug 2022 09:59:55 -0400 Neno Vuksanovic
 wrote:

>  Dear All,
> 
> I would like to quantify electron density inside of positive Fo-Fc
> blobs in active sites of multiple protomers in the map and compare
> them. I am aware that I can interpolate maps and obtain density
> values at coordinate points using either MapMan, Chimera or Coot, but
> I would like to know if there is a tool that would let me designate a
> sphere of a certain volume and calculate total electron density
> inside it?
> 
> Best Regards,
> Neno
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a
> mailing list hosted by www.jiscmail.ac.uk, terms & conditions are
> available at https://www.jiscmail.ac.uk/policyandsecurity/



-- 
--
Tim Gruene
Head of the Centre for X-ray Structure Analysis
Faculty of Chemistry
University of Vienna

Phone: +43-1-4277-70202

GPG Key ID = A46BEE1A



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/


pgp7oYTz8Cc4z.pgp
Description: OpenPGP digital signature

Re: [ccp4bb] Quantifying electron density inside of a given volume

[James Holton]

Hey Pavel,

Thank you for your thoughtful comments and excellent references.

 I also just realized I made a mistake in an earlier message.  When you 
"integrate" a map using its average value you want to multiply by the 
volume of the asymmetric unit, not the volume of the unit cell. If you 
multiply by the cell volume you are integrating over the symmetry mates 
as well. You probably just want to know the number of electrons in one 
blob (one ASU).


One quick response to Pavel's first comment below too.

On 8/22/2022 9:48 PM, Pavel Afonine wrote:

Hi James,


- Where exactly inside the blob of density do you place these
dummy atoms?

Where? At the peaks.


Peaks? This means you need to have atomic resolution data and also 
blobs representing ordered atoms, so you actually have peaks!


Not so!  Unless the map is completely featureless there is always a 
highest point.  That is where you put the first atom. Once placed, you 
subtract (or otherwise remove) the density of that atom from the map. In 
this new, modified, map somewhere else is now the new highest point. 
This is where you put atom #2. Etc. With each iteration you transfer 
density from the map into a model. I've tried to adopt a strategy that 
keeps the total number of electrons (model + map) fixed, but that 
creates some interesting problems. The trick is that as you remove 
positive density you don't want to introduce negative density.  And by 
"negative" I mean dipping below the vacuum level. This vacuum level may 
seem arbitrary at first, but in the calculated map it is a very real 
thing that cannot be neglected. No amount of adjustment in xyz, 
occupancy or B factor can generate a negative peak in the calculated 
map. In a way, this positivity constraint is another reason why 
occupancy refinement is a good way to integrate density.


Truth be told, although my "divot" approach seems to work fairly well, 
I'm still not entirely happy with it. The general problem of finding a 
minimally complex constellation of atoms that explain a given blob I 
don't think is solved. But I imagine this would make for a good AI project?


Cheers,

-James Holton
MAD Scientist



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Pavel Afonine]

Hi James,

- Where exactly inside the blob of density do you place these dummy atoms?
>
> Where? At the peaks.
>

Peaks? This means you need to have atomic resolution data and also blobs
representing ordered atoms, so you actually have peaks!


> What I usually do is pick peaks, put atoms at the highest ones, then
> either refine for a bit or simply subtract a "divot" of density around each
> added atom and then look for new peaks. Once the height of after-refinement
> positive difference features drops to an insignificant level I stop.  NB
> that "significance" of any sigma level depends on the number of voxels in
> the map, but its usually between 3.5 and 4.5 sigmas.
>  One can also start with a 3D "grid" of dummy atoms.  Gaussians spaced
> within 0.8*fwhm of each other form a pretty flat density profile.  In such
> cases refining just one overall B factor and individual occupancies with no
> xyz motion is pretty effective all by itself.
>

Dummy atoms on a 3D grid approach in fact has a documented instance of a
successful application (although in a slightly different context):

 "Local improvement of electron density maps". (1997).  Acta Cryst., D53,
540-543.

- I guess O or C as an atom type should do it, but what about B factor
> (would you refine B as well?)?
>
> I find oxygen will do in peak-picking cases, but for grid atoms under very
> smooth density and closely-spaced atoms I have found the individual
> occupancies get rather small, the round-off error of occupancy from 0.01 to
> 0.00 creates a granularity of ~0.1 electron, and this can start creating
> artificial noise in the fit.  For cases like this I have gone to "liquid
> helium", or modelling dummy atoms as He. Yes, you might think that hydrogen
> would be better, but H atoms have so many special checks and whiz-bangs for
> how they are treated I eventually gave up and went to He.  (One could also
> argue that at low enough temperature He atoms are allowed to "overlap"
> anyway. ;) )
>

Yes, H (Acta Cryst. 1997. D53, 540-543) or O (Acta Cryst. 1993. D49,
129-147) depending on whether you do peak picking or refine DAs on the grid.
And I see why He may be a better choice, yes, hydrogens are too special
these days!

Pavel



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Quantifying electron density inside of a given volume

[James Holton]

Thank you Pavel, responses inline below.  There are indeed a lot of 
possible approaches, and my experience is far from complete, but I will 
share my impressions.


On 8/15/2022 8:15 PM, Pavel Afonine wrote:

Hi James,

I like your approach with dummy atoms and occupancy refinement. 
Dealing with actual maps sounds like hell to me indeed (especially 
given that we deal with weighted Fourier maps!).
I agree that dealing with maps directly can be tricky, but I have found 
that CCP4 support for this is extensive, if you know the right keywords!


Yes, the standard maps are weighted, but I've found the weighted maps 
are closer to the "true density" than any other kind of map. At least, 
in terms of correlation coefficients. I have tried scaling the 2mFo-DFc 
map to an Fcalc map (with solvent) to try and recover the true, absolute 
scale. Usually the scale factor is not worse than ~0.8 overall. This is 
probably because the average value of "m" is close to that?  One could 
argue that you should scale up all maps by the FOM, but then again isn't 
the maximum-likelihood number of electrons reflected in the weighted 
map?  Such re-scaling is not possible with the mFo-DFc map, of course.  
So, yes, another good argument for doing occupancy refinement instead of 
integrating density!
Reading this as someone who immediately translates this into a 
computer code (in my mind), a few things that aren't clear to me:

- Where exactly inside the blob of density do you place these dummy atoms?
Where? At the peaks. What I usually do is pick peaks, put atoms at the 
highest ones, then either refine for a bit or simply subtract a "divot" 
of density around each added atom and then look for new peaks. Once the 
height of after-refinement positive difference features drops to an 
insignificant level I stop.  NB that "significance" of any sigma level 
depends on the number of voxels in the map, but its usually between 3.5 
and 4.5 sigmas.
 One can also start with a 3D "grid" of dummy atoms.  Gaussians spaced 
within 0.8*fwhm of each other form a pretty flat density profile.  In 
such cases refining just one overall B factor and individual occupancies 
with no xyz motion is pretty effective all by itself.

- How many do you place?
Enough for the largest remaining peak to become insignificant.  In the 
case of occupancy refinement one might also us the final total recovered 
occupancy ad a convergence criterion.  I find it best to use an 
iterative process, but this is also slower. 10-sigma peaks are pretty 
obviously not noise, but once you get down into the ~4 sigma level the 
likelihood that one of them is a "noise peak" gets higher. Eventually, 
it is hard to avoid having at least a few "noise peaks".
This is why I wrote my "watershed" program. This is run after the 
initial adding/refining procedure has converged.  Watershed procedure 
is: take each and every occupancy-refined atoms one at a time, delete 
it, and re-refine everything else to see if losing that one atom made 
any difference.  Using multiple CPUs, you can do all the "minus one" 
refinements in parallel. In some cases Rwork will actually improve after 
deleting an atom, so in that case you should definitely throw it out.  
Once the least-useful dummy atom has been eliminated, you can repeat the 
process with the new, improved constellation of atoms.  In the end, you 
have established that every single atom in your dummy constellation is 
"needed".  By "needed" I mean without it the Rwork goes up. You may want 
to use other criteria, such as the total occupancy drops, or you get a 
significant difference peak.  In general, this "watershed" procedure 
finds at least a few waters in most structures that don't need to be 
there.  I have found that just looking at Rwork is a pretty good way to 
eliminate waters that were built into "noise peaks".  So, if I know I'm 
going to "watershed" anyway, I can be pretty aggressive with how low I 
go in peak height in the water adding protocol.
- Is there a risk of placing them too close to the boundary of the 
blob (in which case the question remains: what is the boundary?)?
Placing at only the tallest peak is one way to avoid this.  Grid layout 
is not.  However, one hopes that atoms placed in vacuum and refined in 
occupancy will drop to occ=0.  Either that or B=500.  In practice with 
"grid of atoms" I tend to refine occupancy first, no B nor xyz. The 
atoms then act like a pinscreen, capturing the density shape rather 
well. Adding B factors then lets the slope of the edges of the blob 
match better.  Doing occ, B and xyz all at the same time on a grid 
usually blows up.
And by "blow up", I mean that Rwork and Rfree rapidly rise into the 
50%s. I can stabilize it by using heavy damping and lots of cycles, but 
this is very slow.  I think this blowup happens because the default step 
size in refinement programs is too large for this "crowded atom" situation.
- I guess O or C as an atom type should do it, but what about B 

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Jessica Bruhn]

Hi all,

There are lots of great suggestions in this thread. I will just add a
little trick from small molecule crystallography: when trying to estimate
how many atoms can fit in a given volume, you can use the Rule of 18. Take
the volume of interest in Angstroms^3 and divide it by 18 Angstroms^3. This
will give you an estimate of the number of atoms that can fit in this
volume (ignoring hydrogens of course). This calculation assumes tight
packing, like we would see in a small molecule structure, so it should be a
good approximation of the maximum number of atoms that can fit in this void
and a good place to start for occupancy refinement.

Best of luck!

Cheers,
Jessica

On Mon, Aug 15, 2022 at 5:17 PM Pavel Afonine  wrote:

> Hi James,
>
> I like your approach with dummy atoms and occupancy refinement. Dealing
> with actual maps sounds like hell to me indeed (especially given that we
> deal with weighted Fourier maps!). Reading this as someone who immediately
> translates this into a computer code (in my mind), a few things that aren't
> clear to me:
> - Where exactly inside the blob of density do you place these dummy atoms?
> - How many do you place?
> - Is there a risk of placing them too close to the boundary of the blob
> (in which case the question remains: what is the boundary?)?
> - I guess O or C as an atom type should do it, but what about B factor
> (would you refine B as well?)?
> - if you refine B, how do you deconvolute occupancy from refined B values
> (and eventually from effects of positional errors of your DAs)?
> -
> - How all these choices are going to affect the result?
>
> All the best!
> Pavel
>
>
> On Mon, Aug 15, 2022 at 4:38 PM James Holton  wrote:
>
>> There are several programs for integrating electron density, but please
>> let me assure you that it is almost always the wrong thing to do.
>>
>> A much better strategy is occupancy refinement.  Throw in dummy atoms,
>> turn off non-bonded interactions to them, and refine their occupancy until
>> it a) stops changing (may be more than one round), and b) there are no
>> Fo-Fc differences left in the region of interest.  Then all you do is add
>> up the occupancies, multiply by the relevant atomic number (usually 8), and
>> voila! you get the best-fit number of electrons in your blob. You may want
>> to try re-running with random starting points to get an idea of the error
>> bars.
>>
>> What is wrong with integrating density?  Well, for one, it is hard to
>> know where to set the boundaries. Integrated density can be VERY sensitive
>> to the choice of radius, making your arbitrary decision of which radius to
>> use a source of error. Too small and you miss stuff. Too big and you add
>> unnecessary noise. Also, neighboring atoms have tails, and if you don't
>> subtract them properly, that is another source of error. Also, because of
>> the missing F000 term, there is an offset, which adds a term proportional
>> to the integration volume.  For example, an integral resulting in zero
>> "electrons" does NOT mean you have vacuum. It just means that the area you
>> integrated has the same average density as the entire map. This may not be
>> the number you want.
>>
>> The beauty of occupancy refinement is that it automatically handles all
>> these problems. The "vacuum level" and F000 are known quantities in the
>> calculated map. The B factors given to the dummy atoms als o allow the
>> borders of your integration region to be "soft": down-weighting the
>> contribution of map noise far from your region of interest.  And, finally,
>> by including atoms in the green density, neighboring atoms won't be sucked
>> into it.
>>
>> Think of it as fitting a smooth curve to noisy data and the number of
>> electrons is just a parameter in that fit, rather than trying to integrate
>> the noisy data itself.  This is not an analogy. Refinement programs are
>> really just very sophisticated curve-fitting programs.  And if you have a
>> forest of overlapping peaks and you are trying to de-convolute the
>> area/volume of one peak, it is best to include that peak in the fit, rather
>> than leave it out. Shoulder peaks especially tend to get "eaten" by large
>> neighboring peaks.
>>
>> How do you turn off non-bonds? Well, there is documentation for refmac:
>> http://www.ysbl.york.ac.uk/refmac/data/refmac_keywords.html
>> and phenix:
>> https://phenix-online.org/documentation/reference/refinement.html
>>
>> All that said, to answer the original question:
>>  One very easy thing to do within the CCP4 suite is to use "mapmask" to
>> make a mask corresponding to your "sphere", or other region of interest.
>> Perhaps place a water at the center of your peak, and either use the
>> "border" feature of mapmask, or use "sfall" to compute a calculated map and
>> convert that to a mask using "threshold" in mapmask.  This mask should have
>> values of 0 or 1 at every voxel. (or, if you feel like being clever,
>> something between 0 and 1 to reflect how much 

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Pavel Afonine]

Hi James,

I like your approach with dummy atoms and occupancy refinement. Dealing
with actual maps sounds like hell to me indeed (especially given that we
deal with weighted Fourier maps!). Reading this as someone who immediately
translates this into a computer code (in my mind), a few things that aren't
clear to me:
- Where exactly inside the blob of density do you place these dummy atoms?
- How many do you place?
- Is there a risk of placing them too close to the boundary of the blob (in
which case the question remains: what is the boundary?)?
- I guess O or C as an atom type should do it, but what about B factor
(would you refine B as well?)?
- if you refine B, how do you deconvolute occupancy from refined B values
(and eventually from effects of positional errors of your DAs)?
-
- How all these choices are going to affect the result?

All the best!
Pavel


On Mon, Aug 15, 2022 at 4:38 PM James Holton  wrote:

> There are several programs for integrating electron density, but please
> let me assure you that it is almost always the wrong thing to do.
>
> A much better strategy is occupancy refinement.  Throw in dummy atoms,
> turn off non-bonded interactions to them, and refine their occupancy until
> it a) stops changing (may be more than one round), and b) there are no
> Fo-Fc differences left in the region of interest.  Then all you do is add
> up the occupancies, multiply by the relevant atomic number (usually 8), and
> voila! you get the best-fit number of electrons in your blob. You may want
> to try re-running with random starting points to get an idea of the error
> bars.
>
> What is wrong with integrating density?  Well, for one, it is hard to know
> where to set the boundaries. Integrated density can be VERY sensitive to
> the choice of radius, making your arbitrary decision of which radius to use
> a source of error. Too small and you miss stuff. Too big and you add
> unnecessary noise. Also, neighboring atoms have tails, and if you don't
> subtract them properly, that is another source of error. Also, because of
> the missing F000 term, there is an offset, which adds a term proportional
> to the integration volume.  For example, an integral resulting in zero
> "electrons" does NOT mean you have vacuum. It just means that the area you
> integrated has the same average density as the entire map. This may not be
> the number you want.
>
> The beauty of occupancy refinement is that it automatically handles all
> these problems. The "vacuum level" and F000 are known quantities in the
> calculated map. The B factors given to the dummy atoms als o allow the
> borders of your integration region to be "soft": down-weighting the
> contribution of map noise far from your region of interest.  And, finally,
> by including atoms in the green density, neighboring atoms won't be sucked
> into it.
>
> Think of it as fitting a smooth curve to noisy data and the number of
> electrons is just a parameter in that fit, rather than trying to integrate
> the noisy data itself.  This is not an analogy. Refinement programs are
> really just very sophisticated curve-fitting programs.  And if you have a
> forest of overlapping peaks and you are trying to de-convolute the
> area/volume of one peak, it is best to include that peak in the fit, rather
> than leave it out. Shoulder peaks especially tend to get "eaten" by large
> neighboring peaks.
>
> How do you turn off non-bonds? Well, there is documentation for refmac:
> http://www.ysbl.york.ac.uk/refmac/data/refmac_keywords.html
> and phenix:
> https://phenix-online.org/documentation/reference/refinement.html
>
> All that said, to answer the original question:
>  One very easy thing to do within the CCP4 suite is to use "mapmask" to
> make a mask corresponding to your "sphere", or other region of interest.
> Perhaps place a water at the center of your peak, and either use the
> "border" feature of mapmask, or use "sfall" to compute a calculated map and
> convert that to a mask using "threshold" in mapmask.  This mask should have
> values of 0 or 1 at every voxel. (or, if you feel like being clever,
> something between 0 and 1 to reflect how much you want to weight a given
> voxel). You can check it in coot. If you then multiply this mask by your
> mFo-DFc map the result will have a non-zero average value. This will be
> printed out in the log file. Multiply this average value by the volume of
> the unit cell and you have your integrated number of electrons. Yes, its
> that simple.
> One issue you may have is map parameter compatibility (grid spacing, axis
> order, xyz limits, etc.). You get around these by using the same grid in
> all your fft or sfall runs, and then use mapmask to make the axis and
> limits match before you multiply the map and mask.  The only other issue
> here might be the average value being a very small number and rounded off
> by the default print precision. You can fix this by multiplying the map by
> a large constant (again, using mapmask), 

Re: [ccp4bb] Quantifying electron density inside of a given volume

[James Holton]

There are several programs for integrating electron density, but please 
let me assure you that it is almost always the wrong thing to do.


A much better strategy is occupancy refinement.  Throw in dummy atoms, 
turn off non-bonded interactions to them, and refine their occupancy 
until it a) stops changing (may be more than one round), and b) there 
are no Fo-Fc differences left in the region of interest.  Then all you 
do is add up the occupancies, multiply by the relevant atomic number 
(usually 8), and voila! you get the best-fit number of electrons in your 
blob. You may want to try re-running with random starting points to get 
an idea of the error bars.


What is wrong with integrating density?  Well, for one, it is hard to 
know where to set the boundaries. Integrated density can be VERY 
sensitive to the choice of radius, making your arbitrary decision of 
which radius to use a source of error. Too small and you miss stuff. Too 
big and you add unnecessary noise. Also, neighboring atoms have tails, 
and if you don't subtract them properly, that is another source of 
error. Also, because of the missing F000 term, there is an offset, which 
adds a term proportional to the integration volume. For example, an 
integral resulting in zero "electrons" does NOT mean you have vacuum. It 
just means that the area you integrated has the same average density as 
the entire map. This may not be the number you want.


The beauty of occupancy refinement is that it automatically handles all 
these problems. The "vacuum level" and F000 are known quantities in the 
calculated map. The B factors given to the dummy atoms als o allow the 
borders of your integration region to be "soft": down-weighting the 
contribution of map noise far from your region of interest.  And, 
finally, by including atoms in the green density, neighboring atoms 
won't be sucked into it.


Think of it as fitting a smooth curve to noisy data and the number of 
electrons is just a parameter in that fit, rather than trying to 
integrate the noisy data itself.  This is not an analogy. Refinement 
programs are really just very sophisticated curve-fitting programs. And 
if you have a forest of overlapping peaks and you are trying to 
de-convolute the area/volume of one peak, it is best to include that 
peak in the fit, rather than leave it out. Shoulder peaks especially 
tend to get "eaten" by large neighboring peaks.


How do you turn off non-bonds? Well, there is documentation for refmac:
http://www.ysbl.york.ac.uk/refmac/data/refmac_keywords.html
and phenix:
https://phenix-online.org/documentation/reference/refinement.html

All that said, to answer the original question:
 One very easy thing to do within the CCP4 suite is to use "mapmask" to 
make a mask corresponding to your "sphere", or other region of 
interest.  Perhaps place a water at the center of your peak, and either 
use the "border" feature of mapmask, or use "sfall" to compute a 
calculated map and convert that to a mask using "threshold" in mapmask.  
This mask should have values of 0 or 1 at every voxel. (or, if you feel 
like being clever, something between 0 and 1 to reflect how much you 
want to weight a given voxel). You can check it in coot. If you then 
multiply this mask by your mFo-DFc map the result will have a non-zero 
average value. This will be printed out in the log file. Multiply this 
average value by the volume of the unit cell and you have your 
integrated number of electrons. Yes, its that simple.
One issue you may have is map parameter compatibility (grid spacing, 
axis order, xyz limits, etc.). You get around these by using the same 
grid in all your fft or sfall runs, and then use mapmask to make the 
axis and limits match before you multiply the map and mask.  The only 
other issue here might be the average value being a very small number 
and rounded off by the default print precision. You can fix this by 
multiplying the map by a large constant (again, using mapmask), then the 
printed value will have lots of digits.


This may seem complicated, but the use of masks can be a very valuable 
skill to develop.  In fact, one way to simplify, stabilize and 
accelerate the occupancy refinement described above is to use a mask to 
isolate the region of interest. That is, take the mFo-DFc map, zero out 
everything far away from your peak, and convert the result to structure 
factors. You can then call these structure factors "Fobs" (alongside the 
original sigma(Fobs)) in a new refinement. The Rwork/Rfree then becomes 
a local statistic, indicative of the % error in your refined total 
occupancy. One caveat is that if every atom in the new refinement is 
having its occupancy refined you will lose the absolute scale. To fix 
this, you need to add back at least one well-ordered atom into "Fobs", 
and also include it in the model.  For example, take a well-ordered 
helix, extract those atoms, calculate a map using "sfall", and add it to 
the masked-off difference 

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Ian Tickle]

Hi Neno

I think you mean the total number of electrons in the volume, i.e. the
volume integral of the electron density, or perhaps you mean the average
electron density, i.e. the total number of electrons in the volume divided
by the volume.

Cheers

-- Ian



On Wed, 10 Aug 2022, 15:00 Neno Vuksanovic,  wrote:

> Dear All,
>
> I would like to quantify electron density inside of positive Fo-Fc blobs
> in active sites of multiple protomers in the map and compare them. I am
> aware that I can interpolate maps and obtain density values at coordinate
> points using either MapMan, Chimera or Coot, but I would like to know if
> there is a tool that would let me designate a sphere of a certain volume
> and calculate total electron density inside it?
>
> Best Regards,
> Neno
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Quantifying electron density inside of a given volume

[Julia Griese]

Dear Neno,

Mapman can do exactly that too. Shameless plug: How to do it is described in 
this paper: http://scripts.iucr.org/cgi-bin/paper?S2059798319009926

Mapman appears to available on github now: 
https://github.com/martynwinn/Uppsala-Software-Factory

Best,

Julia

--
Dr. Julia Griese
Assistant Professor, Docent
Department of Cell and Molecular Biology
Uppsala University
BMC, Box 596
SE-75124 Uppsala
Sweden

email: julia.gri...@icm.uu.se
phone: +46-(0)18-471 4982
http://www.icm.uu.se/structural-biology/griese-lab/

From: CCP4 bulletin board  on behalf of Neno Vuksanovic 

Reply-To: Neno Vuksanovic 
Date: Wednesday, 10 August 2022 at 16:00
To: "CCP4BB@JISCMAIL.AC.UK" 
Subject: [ccp4bb] Quantifying electron density inside of a given volume

Dear All,

I would like to quantify electron density inside of positive Fo-Fc blobs in 
active sites of multiple protomers in the map and compare them. I am aware that 
I can interpolate maps and obtain density values at coordinate points using 
either MapMan, Chimera or Coot, but I would like to know if there is a tool 
that would let me designate a sphere of a certain volume and calculate total 
electron density inside it?

Best Regards,
Neno



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1








När du har kontakt med oss på Uppsala universitet med e-post så innebär det att 
vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du 
läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For 
more information on how this is performed, please read here: 
http://www.uu.se/en/about-uu/data-protection-policy



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

[ccp4bb] Quantifying electron density inside of a given volume

[Neno Vuksanovic]

 Dear All,

I would like to quantify electron density inside of positive Fo-Fc blobs in
active sites of multiple protomers in the map and compare them. I am aware
that I can interpolate maps and obtain density values at coordinate points
using either MapMan, Chimera or Coot, but I would like to know if there is
a tool that would let me designate a sphere of a certain volume and
calculate total electron density inside it?

Best Regards,
Neno



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/