On Sun, 30 Dec 2001 18:07:16 -0500, [EMAIL PROTECTED] (Stan Brown) wrote:
> Rich Ulrich <[EMAIL PROTECTED]> wrote in sci.stat.edu: [ ... ] RU > > We should > >not overlook the chance to teach our budding statisticians: > >*Always* pay attention to the distinction between random trials > >or careful controls, on the one hand; and grab-samples on the other. > >[Maybe your teacher asked the question that way, in order to > >lead up to that in class?] > SB > [ snip; was in a book of homework problems ...] RU > > >The numbers do not *prove* that one gas gives better mileage; > >the mileage was, indeed, better for one gas than another -- for > >reasons yet to be discussed. Different cars? drivers? routes? SB > > All good points for discussion. But I wouldn't focus too much on > that off-the-cuff word "prove". (I'm not being defensive since I > didn't write the exercise. :-) My students did understand that > nothing is ever proved; that there's still a p-value chance of > getting the sample results you got even if you did perfect random > selection an d the null hypothesis is true. Maybe I'm being UNDER- > scrupulous here, but I think it a pardonable bit of sloppy language. > I read this, and I re-read it, and it still seems like Stan has missed the gist. Type I error ? Stan sees all the problem (so it seems to me) as being one of Type I error, "there's still a p-value chance..." Yes, you *might* construe *some* of the problem that way, some of the time. But I am concerned with the fundamental difference, randomized trials versus grab-sample (observational): For the former, you get to start with the assumption that outside variables are not important. For the latter, you *have to* start with the assumption -- if you are a properly trained scientist -- that they are *apt* to be. It is not enough that you have a 'single, important hypothesis' like you have in a design; you have to explain away the other possible causes. RA Fisher (along with a few others) defended tobacco for years. The case against tobacco was not solidly made until *other*, competing explanations had been laid to rest. That included a convergence with biomedical evidence. I was annoyed by sloppiness when I was in grad-school. If I remember right, it was the epidemiologists who regularly were sloppy, and failed to qualify their tests, while the math-trained statisticians were apt to be clear. "We conclude that these *numbers* are different..." instead of "... conclude that these *samples* are different." - I mention this because it suggests to me that such sloppiness is *not* pardonable; that the people who care most about the 'actual outcomes' are the ones who are least able to keep in mind the technical reservations. That's my advice, for what it is worth -- I have never taught any regular course. -- Rich Ulrich, [EMAIL PROTECTED] http://www.pitt.edu/~wpilib/index.html ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =================================================================
