<[EMAIL PROTECTED]>
<[EMAIL PROTECTED]> <[EMAIL PROTECTED]>
From: "Ian Tickle" <[EMAIL PROTECTED]>
To: "Frank von Delft" <[EMAIL PROTECTED]>, "Pavel Afonine"
<[EMAIL PROTECTED]>
Cc: <CCP4BB@JISCMAIL.AC.UK>
Return-Path: [EMAIL PROTECTED]
X-OriginalArrivalTime: 03 Oct 2008 16:38:18.0609 (UTC)
FILETIME=[70D42610:01C92576]
I agree completely with Frank, IMO this is not something you should be
doing, particularly as the likelihood function for intensities can
handle negative & zero intensities perfectly well (Randy assures me).
Out of interest I've updated my simulation where I calculate the average
intensity after correction by the various methods in use, to include the
case where you simply drop I <= 0. I didn't include this case before
because I didn't think anyone would be using it!
Here:
S is the Wilson distribution parameter (true intensity), assuming an
acentric distribution;
Iav is the average uncorrected (raw) intensity;
I'av is the average intensity after thresholding at zero (i.e. I' =
max(I,0) );
I"av is the new case, the average ignoring I <= 0;
<Ja>av is the average of the Bayesian estimate assuming a uniform
distribution of the true intensity as the prior (Sivia & David);
<Jb>av is the average of the Bayesian estimate assuming an acentric
Wilson distribution of the true intensity as the prior (French & Wilson
a la TRUNCATE);
rmsE are the respective RMS errors (RMS difference between the
respective 'corrected' intensity and the true intensity).
sigma(I) = 1 throughout.
S Iav rmsE I'av rmsE I"av rmsE <Ja>av rmsE <Jb>av
rmsE
0.0 0.00 1.00 0.40 0.71 0.80 1.00 0.90 1.00 0.00
0.00
0.5 0.50 1.01 0.74 0.78 1.06 0.92 1.17 0.89 0.51
0.43
1.0 1.00 1.00 1.16 0.84 1.40 0.90 1.50 0.86 1.00
0.64
1.5 1.50 0.99 1.62 0.87 1.80 0.91 1.90 0.86 1.49
0.74
2.0 2.00 1.00 2.10 0.90 2.25 0.93 2.34 0.88 2.00
0.80
2.5 2.50 1.00 2.58 0.91 2.71 0.94 2.79 0.89 2.50
0.83
3.0 3.00 1.00 3.07 0.93 3.19 0.95 3.26 0.91 3.00
0.86
3.5 3.50 1.00 3.56 0.93 3.66 0.95 3.72 0.91 3.50
0.88
4.0 4.00 0.99 4.05 0.94 4.13 0.95 4.20 0.91 4.00
0.89
4.5 4.50 0.99 4.55 0.94 4.62 0.95 4.68 0.92 4.50
0.90
5.0 5.00 1.00 5.04 0.95 5.08 0.96 5.17 0.93 5.00
0.91
It can be seen that the new case (I"av) is worse than using all the
uncorrected intensities (Iav) in terms of average bias (difference
between average corrected I and true average S), and only marginally
better in terms of rmsE, and is significantly worse than including the
negative intensities as zero (I'av) on both counts, particularly for low
average intensity (<= 1 sigma). The Bayesian-corrected intensities are
not needed in practice for refinement (but may be better for other
purposes such as twinning tests) because the likelihood function can
handle the uncorrected negative & zero intensities.
Cheers
-- Ian
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf Of Frank von Delft
> Sent: 03 October 2008 10:41
> To: Pavel Afonine
> Cc: CCP4BB@JISCMAIL.AC.UK
> Subject: Re: [ccp4bb] Reading the old literature / truncate /
> refinement programs
>
>
> >>
> >> I mentioned previously phenix.refine tosses your weak data
> if IMEAN,
> >> SIGIMEAN are chosen during refinement.
> >>
> >
> > phenix.refine does not automatically remove the data based on sigma
> > (it does it by user's request only). phenix.refine removes only
> > negative or zero values for Iobs (Fobs).
> That is in fact the same as removing based on sigma:
> standard practice
> has been for some time that no data is removed, ever.
>
> At the refinement stage, that is. Of course, we do remove data at
> merging, for various reasons which probably also need investigating
> (e.g. "high res cutoff" = truncation; cf .Free Lunch).
>
> phx.
>
>
Disclaimer
This communication is confidential and may contain privileged information
intended solely for the named addressee(s). It may not be used or disclosed
except for the purpose for which it has been sent. If you are not the intended
recipient you must not review, use, disclose, copy, distribute or take any
action in reliance upon it. If you have received this communication in error,
please notify Astex Therapeutics Ltd by emailing [EMAIL PROTECTED] and destroy
all copies of the message and any attached documents.
Astex Therapeutics Ltd monitors, controls and protects all its messaging
traffic in compliance with its corporate email policy. The Company accepts no
liability or responsibility for any onward transmission or use of emails and
attachments having left the Astex Therapeutics domain. Unless expressly
stated, opinions in this message are those of the individual sender and not of
Astex Therapeutics Ltd. The recipient should check this email and any
attachments for the presence of computer viruses. Astex Therapeutics Ltd
accepts no liability for damage caused by any virus transmitted by this email.
E-mail is susceptible to data corruption, interception, unauthorized amendment,
and tampering, Astex Therapeutics Ltd only send and receive e-mails on the
basis that the Company is not liable for any such alteration or any
consequences thereof.
Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park,
Cambridge CB4 0QA under number 3751674
