I agree with Jim that we often see patterns in randomness -- it doesn't take a psychologist to find this out. The stars are pretty randomly distributed (although the concentration varies, esp. around the Milky Way), but people have found patterns (constellations) there for millenia. And visual observations are never totally reliable, as tests of eyewitness testamony has shown. Still, given a choice between plots with no stats and stats with no plots, I would trust the plots. However, as I posted earlier, once you see a pattern in the plots you generally need to confirm it with stats, if only to make it past the referees.
The most common error I have seen in statistical analysis is the use of linear models for nonlinear systems. Plotting the data is the easiest way to see whether the data are linear and I think it is at least as reliable as analysis of the residuals. But many ecologists are totally committed to the use of linear models, and some statisticians are too. I have seen presentations where someone shows data that are clearly curvilinear and resolutely proceeds with linear analysis. In one discouraging case a post-doc came to ask me about how to analyse a large data set from a toxicology experiment where the dose-response curves all had the same very non-linear shape. I helped her set up anappropriate model, but her supervisor absolutely refused to let her mess with such a "complicated" mathematical analysis and insisted that she use linear regressions. This was not a unique event, and I think that many ecologists simply refuse to see the need for any element of common sense in the application of statistics. That is part of the reason I refer to this as a religious controversy! I assure Jim that I have no problem with the correct application of statistics. I just haven't seen much evidence that the correct application of statistics is widespread. Judging by what he writes about the papers his students read, there is not much disagreement there. Bill Silvert ----- Original Message ----- From: "James J. Roper" <[EMAIL PROTECTED]> To: <ECOLOG-L@LISTSERV.UMD.EDU> Sent: Saturday, July 21, 2007 1:31 AM Subject: Re: ECOLOGY Mathematics and the metamathematics of evasive ecology? Re: Request: Data sets for biocalculus project >I am sure that we would all hope that the real "issue" is scientific, and > not religious.... > > Just to clear up some confusion - plots are often absolutely necessary to > convey results and to see patterns. Statistics are often absolutely > necessary to understand or confirm pattern. People have been known to see > patterns in randomness, so our "intuition" about pattern can mislead. > Psychologists have tested people to look and see pattern in a random > collection of circles of various colors and sizes. People found patterns > where none existed. This only tells us that we should be careful when we > think we perceive pattern. If we think we perceive pattern in data that > wa= > s > not collected to test that pattern, we should generate new tests to > specifically test for those patterns. > > Statisticians are often not biologists, so while they may know what what > they are doing, they often do not know biology. Also, I am finding more > an= > d > more that people are forgetting how to do simple math! If they can't do > th= > e > calculation on their computer, they can't do the calculation.... > > Finally, I have been having my biostatistics students read papers that > include the topic of the week - we have found so many errors in analysis > that the poor Brazilian students who do not read English well, and assume > that not the author erred, but rather they erred in thinking that the > stats > was bad. I would guess that as many as 50% of the papers have from mild > to > serious mistakes. Scarey! > > So, the moral of the story is, look at well-done figures, then check your > conclusions with appropriate statistics, if necessary. But always doubt. > > Cheers, > > Jim > > On 7/19/07, William Silvert <[EMAIL PROTECTED]> wrote: >> >> Although this is really a religious rather than scientific debate which >> i= > s >> unlikely to lead to any concensus, I want to respond to some of Jim >> Roper's >> comments. >> >> The fact that you can learn a lot by looking at plots does not mean the >> that >> "results are so glaringly obvious". Humans are very good at pattern >> recognition and often can see what is present in a plot better than they >> can >> analyse numerical data. Also, plots often show unexpected features which >> lead to new knowledge - they are not just for hypothesis testing. >> >> On several occasions I have been consulted by people who are quite expert >> at >> statistics who cannot interpret their data, and who were surprised that >> b= > y >> plotting the results in the right way a clear answer leaped out at them. >> Of >> course they then had to confirm the results with statistics, but that is >> mainly to get the paper past referees. >> >> Jim ends with the usual comment that if the statistics are carried out by >> someone who is really good at stats, the results will be good. That may >> b= > e >> true, but good statisticians are pretty rare beasts, and in the average >> lab >> the plotting method is just as reliable as textbook stats. Some of you >> ma= > y >> recall a post of mine a couple of years ago where I surveyed a lot of >> statistically sophisticated fisheries scientists to see if they could add >> two numbers (what is 100+-3 + 100 +-4?) and only one person came up with >> the >> answer - but he was very unsure of himself, and couldn't figure out how >> t= > o >> multiply the numbers. >> >> Just a glance through any journal will quickly show that most biologists >> have little idea of significance and represent their results with >> exaggerated precision. In a perfect world maybe we could trust all >> statistical analyses, but we ain't there yet. >> >> Bill Silvert >> >> >> ----- Original Message ----- >> From: "James J. Roper" <[EMAIL PROTECTED]> >> To: <ECOLOG-L@LISTSERV.UMD.EDU> >> Sent: Wednesday, July 18, 2007 3:43 PM >> Subject: Re: ECOLOGY Mathematics and the metamathematics of evasive >> ecology? >> Re: Request: Data sets for biocalculus project >> >> >> > Mattheus, >> > >> > You are showing some misunderstanding of the use of statistics. A few >> > observations. >> > >> > 1. If your results are so glaringly obvious, then the question was >> > probably not very interesting, or a logical consequence of the methods. >> > >> > 2. Questions that are not so simple need statistics to discover the >> > probability of something happening when it is not obligatory that it >> > happen. >> > >> >> statistical tests when you can simply draw a plot and >> >> your conclusion comes? >> > 3. A plot can mislead. >> >> I need to learn that populations must >> >> be normal, they must be homoscedastic, there are at >> >> least 3 models for ANOVA, there is something out there >> >> with the name of ANCOVA, and I have no single idea if >> >> this is useful for me or not... >> > > > > --=20 > James J. Roper, Ph.D. > Ecologia e Din=E2micas Populacionais > de Vertebrados Terrestres > ------------------------------ > > Caixa Postal 19034 > 81531-990 Curitiba, Paran=E1, Brasil > ------------------------------ > > E-mail: [EMAIL PROTECTED] > Telefone: 55 41 33857249 > Mobile: 55 41 99870543 > ------------------------------ > > Ecologia e Conserva=E7=E3o na UFPR <http://www.bio.ufpr.br/ecologia/> > Econci=EAncia - Consultoria e Tradu=E7=F5es > <http://jjroper.googlespages.co= > m> > >
