Hi Peter It has nothing to do with linear vs. non-linear models, it has everything to do with errors in the data and the model. Sure, if there are no errors you can in general solve for N unknowns exactly from N equations (obs/param ratio = 1) whether there's a linear relationship or not (though of course there may be specific problems with non-linearity, such as multiple and/or imaginary solutions). If there are errors in the observations (which is always true for any experimental data) then the equations must be over-determined (obs/param ratio > 1) in order to minimise the effect of the errors on the derived parameters. Then you are into estimation theory and I would say that as good a place as any to start is the Wikipedia page on maximum likelihood estimation and references therein:
http://en.wikipedia.org/wiki/Maximum_likelihood -- Ian > -----Original Message----- > From: Peter Adrian Meyer [mailto:[EMAIL PROTECTED] > Sent: 01 October 2007 16:50 > To: Ian Tickle > Cc: ccp4bb@jiscmail.ac.uk > Subject: Re: [ccp4bb] R-sleep > > This raises a slightly tangential question though - how do we know how > what obs/param ratio is good enough? My understanding was > that obs/param > of 1 was sufficient for linear systems; but it doesn't seem > that any of > the objective functions used for refinement are linear (and I > haven't been > able to track down anything on this topic for non-linear systems). > > With my level of math background (aka low), I'm not even sure if I'm > asking the right question...it seems like it might make more > sense to ask > how many observations are needed to define a unique optima for the > refinement function(s) that's convex in all dimensions of the > model. But > if this made sense, that's probably what we'd be talking about. > > Any suggestions for reading material? > > Thanks, > > Pete > > > > The question is how significant is this bias, and is the cure (i.e. > leaving out more reflections from the working set) worse than the > disease? > > For refinements at 'medium' typical resolution (around 2.5 > to 2 Ang) we > > are working with an observation/parameter count ratio of say < 3 > (naturally I'm counting the geometric restraints with the X-ray > > observations). The amount of bias in Rwork and other > statistics derived > from the working set depends critically on how close the > obs/param ratio > is to 1. The Rfree optimisation is used only to determine weighting > parameters (including sigma-A) and it's unlikely there will > be more than > say 20 of these. Typically there are at least 1000 refls in > the test set, > > so for the Rfree optimisation the obs/param ratio will be around 50. > This > > is much larger than the obs/param ratio for Rwork and may > well mean that > the biasing effect on Rfree is negligible. It should be easy > to do some > tests comparing Rfree with Rsleep to check the bias (taking > into account > errors to limited sample sizes of course), and also to see > what are the > effects of leaving out the sleeping set on the refinement and > the maps. I > > don't think it would be wise to rush into this until we > have done proper > evaluations. > > > > -- Ian > > > >> -----Original Message----- > >> From: [EMAIL PROTECTED] > >> [mailto:[EMAIL PROTECTED] On Behalf Of Mark J. van Raaij > Sent: 01 October 2007 14:58 > >> To: CCP4BB@JISCMAIL.AC.UK > >> Subject: R-sleep > >> Dear All, > >> the short paper by Gerard Kleywegt (ActaD 63, 939-940) treats > >> an interesting subject (at least I think so...). I agree that > >> what we are now doing in many cases is effectively refining > >> against Rfree. For example, the standard CNS torsion angle > >> refinement does n refinement trials with randomised starting > >> points. If you then take the one with lowest Rfree (or let a > >> script do this for you), you are biasing Rfree! > >> Therefore, his proposal to put an extra set of reflections in > >> a dormant "vault" (R-sleep) sounds like a good idea to me. > >> However, how would the "vault" be implemented to be > >> effective? If left to the experimenter, it would be very > >> tempting to check R-sleep once in a while (or often) during > >> refinement, rendering it useless as an unbiased validator. > >> or am I being paranoid and too pessimistic? > >> Mark J. van Raaij > >> Unidad de Bioquímica Estructural > >> Dpto de Bioquímica, Facultad de Farmacia > >> and > >> Unidad de Rayos X, Edificio CACTUS > >> Universidad de Santiago > >> 15782 Santiago de Compostela > >> Spain > >> http://web.usc.es/~vanraaij/ > > > > > > Disclaimer > > This communication is confidential and may contain privileged > information > > intended solely for the named addressee(s). It may not be used or > disclosed except for the purpose for which it has been sent. > If you are > not the intended recipient you must not review, use, disclose, copy, > distribute or take any action in reliance upon it. If you > have received > this communication in error, please notify Astex Therapeutics Ltd by > emailing [EMAIL PROTECTED] and destroy all copies of the > message and any attached documents. > > Astex Therapeutics Ltd monitors, controls and protects all > its messaging > traffic in compliance with its corporate email policy. The > Company accepts > > no liability or responsibility for any onward transmission or use of > emails and attachments having left the Astex Therapeutics > domain. Unless > > expressly stated, opinions in this message are those of the > individual > sender and not of Astex Therapeutics Ltd. The recipient > should check this > > email and any attachments for the presence of computer > viruses. Astex > Therapeutics Ltd accepts no liability for damage caused by any virus > transmitted by this email. E-mail is susceptible to data corruption, > interception, unauthorized amendment, and tampering, Astex > Therapeutics > Ltd only send and receive e-mails on the basis that the Company is not > liable for any such alteration or any consequences thereof. > > Astex Therapeutics Ltd., Registered in England at 436 > Cambridge Science > Park, Cambridge CB4 0QA under number 3751674 > > > > > Pete Meyer > Fu Lab > BMCB grad student > Cornell University > > > > > > > > Disclaimer This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [EMAIL PROTECTED] and destroy all copies of the message and any attached documents. Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain. Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof. Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park, Cambridge CB4 0QA under number 3751674
