On 21 June 2013 13:36, Ed Pozharski <epozh...@umaryland.edu> wrote: > Replacing Iobs with E(J) is not only unnecessary, it's ill-advised as it will distort intensity statistics.
On 21 June 2013 18:40, Ed Pozharski <epozh...@umaryland.edu> wrote:
> I think this is exactly what I was trying to emphasize, that applying
some conversion to raw intensities may have negative impact when conversion
is based on incorrect or incomplete assumptions.
Ed, I think you may have missed the point I was trying to make (or more
likely I didn't make it sufficiently explicit).
Let me re-phrase your first response above slightly (I know you didn't say
this, but it's equivalent to what you did say): "Replacing sqrt(I) with
E(F) is not only unnecessary, it's ill-advised as it will distort the
structure refinement.". Does that make sense? I assume you're using
(c)truncate for all datasets, or do you only use it where t-NCS is absent?
If you use (c)truncate with t-NCS then you are already having a "negative
impact" via use of an incorrectly estimated Epost(F) ('post' = 'posterior'
to distinguish from the prior). If not (i.e. you only use (c)truncate when
t-NCS is absent) then clearly the use of Epost(J) in place of J cannot
"distort intensity statistics".
The negative impact comes from the failure to properly account for t-NCS,
not from use of Epost(J), since Epost(F) is equally affected. AFAIK all
current software that performs F & W conversion take no account of t-NCS
and averages Iobs in spherical shells of constant d-spacing (or possibly
more sophisticated ellipsoidal shells, but that doesn't affect the
argument), in order to estimate Eprior(J) as <Iobs/e> for use in the Wilson
prior (e = symmetry enhancement factor). T-NCS will affect both Eprior(J)
and Epost(J) (and Epost(F)) equally, since the only difference between
these is the factor P(I|J,sigmaI) (Gaussian experimental error), which
doesn't depend on t-NCS. So if you are using (c)truncate with t-NCS you
are already using incorrect estimates of Eprior(J) and hence Epost(F)!
Your statement "replacing Iobs with E(J) is not only unnecessary, it's
ill-advised" neglects the fact that there are always 2 sides to an
argument, both pros and cons. Let me illustrate a couple of examples of
severe negative impacts of the use of Iobs in place of Epost(J) in
intensity stats: I leave you to judge which will "distort intensity
statistics" more!
The first concerns the P & Y L test for twinning I mentioned previously as
an example where problems arise from use of Iobs. L is defined as |I1 -
I2| / (I1 + I2) where I1 & I2 are unrelated intensities close in reciprocal
space (i.e. where |h1-h2| + |k1-k2| + |l1-l2| <= 4 and no |index
difference| equals 1 to try to avoid t-NCS issues). The distribution of L
is confined to the range 0 to 1 so clearly if you have an L outside that
you have a problem. Let's say we have a -ve intensity I2 = -1 and we vary
I1. The output from gnuplot (x = I1, y = L) is attached (Ltest-1.png).
Note that L never falls _inside_ the allowed range (and there's a
singularity going off to - & +inf at I1 = 1). Now say we use E(J) in place
of Iobs. Now I2 = -1 will become (say) E(J2) = 0.1 and E(J1) can't be <=
0. See attached plot (Ltest-2.png) for the result: no value of L is now
_outside_ the allowed range. Of course you could "fix" the first case by
ignoring all I <= 0 but then you wouldn't need to use F & W!
The second example concerns the moments of Iobs (or the moments of Z where
Z = (Iobs/e)/<Iobs/e>). The n'th moment of Z is <Z^n> and (c)truncate
calculates it for n = 1 to 4. Lets say we have 2 reflections Z = -2 and Z
= 2. So the 4th moments are both +16. Does that make any intuitive
sense? A -ve intensity contributes the same as the corresponding +ve one?
Now say we use E(J) or E(Z) instead. Z = -2 will become (say) +0.2 and Z
=2 will become 2.1. Now the moments are intuitive & sensible: the -ve
intensity barely contributes to the stats.
Finally a point arising from a response to one of Douglas's posts:
On 21 June 2013 19:48, Ed Pozharski <epozh...@umaryland.edu> wrote:
> If you replace negative Iobs with E(J), you would systematically inflate
the averages, which may turn problematic in some cases. It is probably
better to stick with "raw intensities" and construct theoretical
predictions properly to account for their properties.
This is simply wrong: the corrected intensities are unbiased, so the
average E(J) is exactly equal to the average I (as I demonstrated some time
back in a previous discussion of this topic).
Cheers
-- Ian
<<attachment: Ltest-2.png>>
