Dear Petr,
Thank you for your reply and for inviting us to address some points!
Please find our replies below, "interleaved" with your questions.
On Tue, Oct 04, 2022 at 04:41:58PM +0000, Petr Kolenko wrote:
> Dear Gerard,
> A great remark! Thank you for starting this thread separately. I think
> that you are absolutely right in many of the points. However, it also
> raises several questions. Here are my favourite ones:
> 1) If STARANISO generally suggests "the same" resolution as PAIREF, there
> are fewer reflection in the final dataset because of the ellipsoidal cut
> in the worst diffraction direction. Is it good to avoid these data?
Your are right that the STARANISO output contains fewer reflections,
but what is a "reflection" in this context, other than a triplet of Miller
indices with a few numbers associated with it? The latter do not necessarily
qualify as "data". This is the essential point that is made in the STARANISO
documentation, found at
https://staraniso.globalphasing.org/anisotropy_about.html
and https://staraniso.globalphasing.org/staraniso_glossary.html ,
where a distinction is made between *measurements* (just the same as what
you call a "reflection": an HKL triplet, with values for I and sig(I)) and
*observations*, defined as measurements deemed (according to a certain
criterion) to contain information and not just noise. STARANISO takes great
care in using a criterion that is based not on the I and sig(I) of
individual reflections, but on a local average of I/sig(I) that detects a
trend rather than just point-wise values, so as to retain weak reflections
that occur surrounded by stronger neighbouring ones, and whose weakness
conveys information. The input reflections to STARANISO that are not present
are those that do not qualify as observations according to that criterion,
and therefore contain only noise. You ask: "Is it good to avoid these
data?": our answer is that the reflections not carried through by STARANISO
are not data but noise, so that avoiding them not only does not lose
information but eliminates noise. We justify this by disagreeing with the
widespread belief, voiced by Eleanor, that noisy measurements cannot harm
refinement if properly weighted, and pointing out that such measurements are
*toxic* in a variety of ways - not least, in the case of anisotropic data,
by biasing all sorts of statistics, or error-model parametrisations, that
assume isotropic distributions of errors or compute certain estimators in
spherical "bins".
A minor point about terminology here: STARANISO does not apply an
"ellipsoidal cut-off". It determines an anisotropic cut-off surface that is
not constrained to being an ellipsoid (it is displayed as it comes in one of
the panels in the Reciprocal Lattice Viewer). An ellipsoid that gives the
best fit to that surface is then determined, but is used only in calculating
the diffraction limits in its three principal directions. The cut-off
applied is based on the surface itself, not on that best-fitting ellipsoid.
> 2) A very general one: Are we doing well when touching the experimental
> data with anisotropic scaling? Should we rather modify the model to fit
> the "raw" data? If so, how about to do it? I may be wrong, but would it be
> possible to refine the structure using all-atom based general anisotropic
> ADP (similar to TLS refinement)?
One could indeed limit the treatment of anisotropy to the determination
and enforcement of the cut-off surface, leaving the data with its original
decay pattern. Refining against such data would yield a model whose
corresponding electron density map would reflect the anisotropy of the data,
i.e. would be blurred differently in different directions. This is what
Michael Sawaya described in
https://www.pnas.org/doi/full/10.1073/pnas.0602606103
but found unsatisfactory when it came to the interpretability of 2Fo-Fc maps
produced in these conditions. He then proposed sharpening the data in the
weaker directions to the same fall-off profile as the best direction after
applying the anisotropic cut-off, and reported that the resulting map was
much more interpretable. STARANISO does the same by default, as specified by
a pre-checked button on the server's submission page; however this button
carries the following annotation:
"Perform anisotropic correction of output intensities and amplitudes
(default): uncheck only if anisotropic correction is not required (not
recommended)."
and thus allows a user to override that default.
> Just a short comment to your table in the attachment. In case of BO from
> our publication, STARANISO resolution goes to 2.3, but our suggestion was
> to stop at the original resolution of 2.6 AA. The reason for that is
> sub-optimal refinement of the geometry. Once restrained in PAIREF, there
> is no improvement anymore.
The entry in the PDB for BO (6I3J) seems to have undergone a number of
revisions, one of them being a carbohydrate remediation. Reprocessing the
raw images with autoPROC/STARANISO indicated diffraction limits of (2.31A,
2.43A, 2.56A) along (a*,b*,c*). Examination of these images shows spot
shapes indicating a possibly cracked crystal; the 0.5-degree image width,
with two unit-cell axis lengths above 200A, could cause worries about
possible angular overlaps, but the F-centering probably saved the game. We
should perhaps follow up separately about this particular dataset, but the
general suggestion we would make is that, when conducting investigations on
such matters as the benefits of PAIREF, it would be advisable to either use
a much larger number of datasets, or, if using only a few as was done in
your paper, to carefully vet them, so as to minimise possible interference
from raw data peculiarities.
With best wishes,
Clemens, Claus, Ian and Gerard.
> ________________________________________
> From: CCP4 bulletin board <CCP4BB@JISCMAIL.AC.UK> on behalf of Gerard
> Bricogne <g...@globalphasing.com>
> Sent: Tuesday, October 4, 2022 6:01:10 PM
> To: CCP4BB@JISCMAIL.AC.UK
> Subject: [ccp4bb] PAIREF, Anisotropy and STARANISO
>
> Dear all,
>
> First of all, apologies for breaking the threads entitled "PAIREF -
> Warning - not enough free reflections in resolution bin" and "Anisotropy" by
> merging them into a new one, but it somehow felt rather against nature to
> keep them separate.
>
> Since the early days of the availability of STARANISO [1] (the actual
> starting year for the Web server [2] was 2016), we had a hunch that much of
> what was happening in the PAIREF procedure might simply be the detection of
> the existence of significant data beyond an initially chosen resolution
> cut-off not only as a result of an excessively conservative criterion having
> been applied in that initial choice, but as a consequence of anisotropy in
> the data. The latter would give rise to different diffraction limits in
> different directions, and the choice of a single value for "the resolution"
> at which the data were cut off would necessarily yield a compromise value
> between the best and the worse diffraction limits. This would imply that
> significant data would be excluded in the best diffracting directions, that
> would subsequently drive PAIREF towards increasing the estimated resolution
> compared to its compromise value.
>
> This "hunch" was validated by a detailed comparison carried out on the
> exact same examples that are considered in the 2020 paper by Maly et al.,
> that is summarised in the attached PDF. In other words, whenever anisotropy
> is present in the data, PAIREF will tend to indicate a higher value for an
> isotropic cut-off than would have been estimated for the initial dataset.
> The problem with taking the PAIREF result as the final answer is that the
> higher cut-off it indicates is applied *isotropically*. The inclusion of the
> significant data thus reclaimed is therefore unavoidably accompanied by that
> of noisy data in the worst diffracting direction(s), resulting in alarmingly
> poor statistics in the outermost shell (as pointed out in Eleanor's message)
> that may cast doubts on the usefulness of the procedure. This consideration
> was the basis of the rationale for implementing an *anisotropic* cut-off
> surface in STARANISO, so that one could thus reclaim the significant data in
> the best-diffracting direction(s) while avoiding the simultaneous inclusion
> of the pure-noise measurements in the worse one(s). While this is clearly
> and extensively explained in the documentation provided on the STARANISO
> server [2], it seems to be far from having been assimilated. Of course this
> would be perfect material for a publication, but life is somehow too short,
> and our to-do list has remained too long, to leave us room for spending the
> necessary time to go through the process of putting a paper together. The
> truly important matter is to get our picture in front of the user community.
>
> Now that the combined topics of PAIREF and anisotropy are being brought
> to the foreground of the community's attention, this seems like the perfect
> opportunity to present our analysis and position: what PAIREF achieves in
> terms of an upward revision of an initial isotropic resolution cut-off is
> likely to be achieved more straightforwardly by submitting the same data to
> the STARANISO server (or using it within autoPROC [3]); and the STARANISO
> output will have the advantage of being devoid of the large extra amount of
> purely noisy, uninformative data that are retained in the output from PAIREF
> according to its revised isotropic cut-off.
>
> We would very much welcome feedback on this position: indeed we would
> like to *crowd-source* the validation (or refutation) of this conclusion. In
> our view, continuing to use the PAIREF procedure to revise an isotropic
> resolution cut off misses the point about the consequences of anisotropy.
> The only sensible use of a PAIREF-like procedure would be to adjust the
> cut-off threshold for the local average of I/sig(I) in STARANISO, whose
> default value is currently 1.2 but can be reset by the user through the Web
> server's GUI. We occasionally see datasets of very high quality for which
> the CC_1/2 value in the outermost shell stays above 0.6 or even 0.7, and it
> is quite plausible that further useful data could be rescued if the local
> I/sig(I) cut-off threshold were lowered below 1.2.
>
> Concerning Eleanor's view that noisy data can't hurt refinement because
> they are properly down-weighted by the consideration of e.g. Rfree values in
> resolution shells, we would point out that any criterion based on statistics
> in resolution shells will be polluted if the data are anisotropic and if the
> noisy data that STARANISO would reject are retained. That will result in
> excessive down-weighting of the significant data that STARANISO retains,
> hence in losing the information they contain. Perhaps this is a matter for
> later discussion, but the main idea is that retaining pure-noise data is not
> neutral in refinement, and that every "isotropic thinking habit" on which
> many views are based needs to be revisited.
>
>
> With best wishes,
>
> Clemens, Claus, Ian and Gerard.
>
>
> [1] Tickle, I.J., Flensburg, C., Keller, P., Paciorek, W., Sharff, A.,
> Vonrhein, C., Bricogne, G. (2018). STARANISO. Cambridge, United
> Kingdom: Global Phasing Ltd.
>
> https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=ind1806&L=CCP4BB&O=D&P=3971
>
> [2] https://staraniso.globalphasing.org/
>
> [3] https://doi.org/10.1107/s0907444911007773
> https://www.globalphasing.com/autoproc/
>
>
> ########################################################################
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
>
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at
> https://www.jiscmail.ac.uk/policyandsecurity/
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list
hosted by www.jiscmail.ac.uk, terms & conditions are available at
https://www.jiscmail.ac.uk/policyandsecurity/
