Hi Phil,
Here I will disagree. R-free rewards you for putting in atom in density
which an atom belongs in. It doesn't necessarily reward you for putting
the *right* atom in that density, but it does become difficult to do
that under normal circumstances unless you have approximately the right
structure.
However in the case of multi-copy refinement at low resolution, the
refinement is perfectly capable of shoving any old atom in density
corresponding to any other old atom if you give it enough leeway.
...
So there's evidence, w/o simulation, that the 12-fold or 16-fold
multicopy refinements are worth 7-8% in R-free, and I'm doubtful that
NCS can generate that sort of gain in either crystal form. I've
certainly never seen that in my own experience at low resolution.
Remember that there are two things at work here: putting atoms into real
density (which does reduce Rfree) and putting atoms into "noise"
(overfitting, which shouldn't help Rfree). At low res, there's a lot of
noise.
If you think about it, there is an analogy to relaxing geometrical
constraints, which also allows the refinement to put atoms into
"density". The reason it usually doesn't help Rfree is that the density
is spurious. At least some of the incorrect structure determinations of
the early 90's (that spurred the introduction of Rfree etc.) had high
rms deviations, suggesting that this is how the overfitting occurred.
Nevertheless, once hit with a bit of simulated annealing, the Rfree
values of such models deteriorated significantly.
I would argue that 12-fold or 16-fold multicopy refinements simply
permitted overfitting of noise. In other words, it is worth 7-8% in
R*work*, but not Rfree. In this case, the main reason Rfree also dropped
is because the test set was coupled *by NCS* to the overfit working set.
Use of a random test set in the presence of NCS could easily prevent the
Rfree value from serving as a warning of overfitting.
Of course, to be absolutely sure, one would have to repeat the multicopy
refinements of the inverted structures with a test set chosen in thin
shells, and then see if Rfree dropped as before. I think only the
original authors would be in a position to do that properly.
Dean
--
Dean R. Madden, Ph.D.
Department of Biochemistry
Dartmouth Medical School
7200 Vail Building
Hanover, NH 03755-3844 USA
tel: +1 (603) 650-1164
fax: +1 (603) 650-1128
e-mail: [EMAIL PROTECTED]
