> If the original cold fusion researchers had used proper statisti-
> cal methods for detecting relationships (i.e., statistical tests,
> a proper taking account of negative results, and experimental de-
> sign considerations), it seems likely that the repeated high or
> borderline p-values would have quickly alerted them to the fact
> that that the sought-after relationships between the variables
> were probably not present. Or if the relationships WERE present,
> they were not properly detectable with the current experimental
> methods, and thus further "more powerful" research must be per-
> formed before a responsible positive conclusion could be drawn.
> This might have saved the original researchers considerable em-
> barrassment and might have saved the physical science community
> substantial costs.
Surely here the problem was not poor statistical procedures per se but
poor experimental design. Individual results were claimed that were
completely at odds with existing theory; if these were not real it was
because the experiment was designed badly, and perhaps because
insufficient control runs (omitting one or more supposedly critical
components) were performed.
Even a single run in which unexplained energy is released in
significant amounts would be important if it were properly designed;
whether or not one could prove anything about the mean over a dozen runs
would be irrelevant. Statistical analysis becomes important later on
when one is fine-tuning the process.
Conversely, a process in which many runs and statistical anaysis were
needed to show the existence of an effect would be likely to remain a
laboratory curiosity. (Given the world's limited supply of palladium,
the process as described seemed like a bad joke on the part of Nature
anyway; what use to the world is perpetual free power in strictly
limited quantities?)
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
