Re: [ccp4bb] PNAS on fraud
Just a few additional ideas on the significance of the presented values of the correlation coefficient. For samples of size N from a bivariate normal with correlation r, its standard deviation is approximately StDev(R) = (1 - R^2)/sqrt(N – 1) - note that it depends on the number of points used to calculate CC. One good reference is Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, Series B, 15(2), 193-232. For the three cases mentioned in Figure 3, fraud, error and duplicates the correlation coefficients and their standard deviationa are respectively: 0.294 0.031 (almost 10 times above its standard deviation) 0.338 0.042 (also well above standard deviation) 0.119 0.045 (2.5 times above standard deviation, what the authors call 'slight' correlation) A distribution of CC is not Gaussian, which is often inconvenient. One can do a Fischer's z-transformation to obtain z-statistics z = (1/2)ln((1+R)/(1-R)) which is approximately normally distributed with standard deviation 1/sqrt(N-3) and then do z-score tests on it. For the same three cases the values of normally distributed z and their standard deviation are very similar to the values obtained from the approximation above. 0.303 0.034 0.352 0.048 0.120 0.045 With best regards, Victor On 18/10/2012 19:52, DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Philippe Dumas Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ -
[ccp4bb] Fwd: [ccp4bb] PNAS on fraud
Begin forwarded message: Date: October 19, 2012 4:40:35 AM EDT To: Randy Read rj...@cam.ac.ukmailto:rj...@cam.ac.uk Subject: Re: [ccp4bb] PNAS on fraud This thread has been quite interesting to me. I've had a long interest in scientific fraud, which I've generally held to be victimless. While that view is unsupportable in a fundamental sense, I feel strongly that we should understand that error correction costs exponentially more, the smaller the tolerance for errors. In protein synthesis, evolution has settled on error rates of ~1 in 4000-1. Ensuring those rates is already costly in terms of NTPs hydrolyzed. NASA peer review provided me another shock: budgets for microgravity experiments were an order of magnitude higher than those for ground-based experiments, and most of the increase came via NASA's insistence on higher quality control. Informally, I've concluded that the rate of scientific fraud in all journals is probably less than the 1 in 10,000 that (mother) nature settled on. I concur with Randy. Charlie On Oct 18, 2012, at 2:43 PM, Randy Read wrote: In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell structures would be worse!
Re: [ccp4bb] PNAS on fraud
Dear Charlie, Do you mean that small doses of fraud should be accepted as a form of natural evolution? Or perhaps you were suggesting that genuine errors/mistakes are acceptable in 1/1 due to the high costs of spotting them? D From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 13:09 To: ccp4bb Subject: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud Begin forwarded message: Date: October 19, 2012 4:40:35 AM EDT To: Randy Read rj...@cam.ac.ukmailto:rj...@cam.ac.uk Subject: Re: [ccp4bb] PNAS on fraud This thread has been quite interesting to me. I've had a long interest in scientific fraud, which I've generally held to be victimless. While that view is unsupportable in a fundamental sense, I feel strongly that we should understand that error correction costs exponentially more, the smaller the tolerance for errors. In protein synthesis, evolution has settled on error rates of ~1 in 4000-1. Ensuring those rates is already costly in terms of NTPs hydrolyzed. NASA peer review provided me another shock: budgets for microgravity experiments were an order of magnitude higher than those for ground-based experiments, and most of the increase came via NASA's insistence on higher quality control. Informally, I've concluded that the rate of scientific fraud in all journals is probably less than the 1 in 10,000 that (mother) nature settled on. I concur with Randy. Charlie On Oct 18, 2012, at 2:43 PM, Randy Read wrote: In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell structures would be worse! -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Re: [ccp4bb] PNAS on fraud
Hi Phillippe, If looking only at the figs. 3A,B,C in the PNAS paper alone, yes, I would agree with you that the proposed correlation is quite weak. Without the help of the trend lines, I would probably conclude that there is no correlation between the IF and number of retractions by a simple look. Of course the R squares are quite telling on the quality of the fitting already. On the other hand, the same authors had done similar analysis on a smaller pool of samples (17 journals) in an earlier study: http://iai.asm.org/content/79/10/3855.full, figure 1. It seems that when including way less journals, the trend stood out quite nicely - leading the authors to say a strong correlation in the earlier publication. I am not sure if the earlier clearer trend was a result of cherry picking, as the choice of journals looks quite normal – like a standard pool of journals one particular lab would consider to publish papers on. It would also be interesting for us on the CCP4BB to try picking only the journals that we would consider to publish structures on, and plot the RI:IF graph to see what would happen. Compared to other fields of biology, frauds in crystallography is probably easier to detect, thus we need worry less about the false negatives: the low impact papers that were fraudulent or erroneous, but nobody cared to spend their effort battling.( I think when taking consideration of this, drawing conclusions from figs. 3A,B,C would be even more dangerous.) I would also like to bring two more issues for discussion: One, in the 2011 IAI paper's fig. 1, the authors plotted Retraction Index, which is the total # of Retractions multiplied by 1000 then divided by total number of publication, whereas in the 2012 PNAS paper figures 3A,B,C, the plots simply used number of retractions to plot against the IFs. I wonder what they will look like if the figures 3A,B,C were plotted as RI vs IF – considering that many low or moderately-low IF journals publish huge numbers of papers. The second issue is, in the PNAS figures 3A,B,C, at the lower left corner, although the dots have a dense looking, the viewers have to realize that most of them only represent 1 to 3 retractions. Ten of such points contain the same number of retractions that one point at the upper halves of the panels A and B contains. Maybe simple bar graphs for numbers of retractions in each IF bin would provide more help. Also, the fact that the averaged IFs landed at ~8 and ~12 for the fraud and errors cases (fig 3D) suggests that the absolute number of retractions occurred in high IF journals is quite significant, especially considering that there are way fewer journals with IF10 than the ones with IF10 in the 200-300 journals. So in my view, maybe trying to fit a straight line to the distribution is overly idealistic, some sort of partition does exist. Zhijie -- From: DUMAS Philippe (UDS) p.du...@ibmc-cnrs.unistra.fr Sent: Friday, October 19, 2012 6:15 AM To: Zhijie Li zhijie...@utoronto.ca Subject: Re: [ccp4bb] PNAS on fraud Le Jeudi 18 Octobre 2012 22:52 CEST, Zhijie Li zhijie...@utoronto.ca a écrit: Thank you for this funny (and yet significant) comment. But I do not see clearly whether you agree with me or with the PNAS paper For me, this conclusion in the PNAS paper is just ridiculous. Philippe D On curve fitting: http://twitpic.com/8jd081 -- From: DUMAS Philippe (UDS) p.du...@ibmc-cnrs.unistra.fr Sent: Thursday, October 18, 2012 1:52 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Philippe Dumas
Re: [ccp4bb] PNAS on fraud
Dom, You've opened a pandora's box here, which I won't try to contain. The short answer is both of the above. I feel it is becoming increasingly difficult as a referee to be on top of every paper I review, and as an editor it is becoming increasingly difficult to find willing referees. Both phenomena are diagnostic of the cost of eliminating fraudulent publications, which gall me pretty much as much as they do many others, but which do not drive me apoplectic, either. I've been amused over the years by the frantic efforts to bring crystallographic charlatans to justice, even as I've been angered by publication in high-impact journals of material I myself view as fraudulent, but which obviously survives peer review. On the second of your alternatives, I'll give you two examples of highly celebrated frauds that wound up moving science forward, despite their scurrilous background. The first is the story of Hasko Paradies, whose only legitimate publication, as far as I know, was a first-author paper on the crystallization of tRNA. In that paper, he was, I think, the first author to describe the use of spermine/spermidine and Mg++ ions in improving crystallization conditions. These two contributions proved useful in the actual generation by others of suitable crystals. Paradies apparently went on to make a habit of filching precession photographs from dark rooms and then presenting them elsewhere and at meetings as if he had taken them and as if they were from hot problems of the day. His story was chronicled by Wayne Hendrickson, Ed Lattman, and others in Nature many years later. He dropped out of science and became a pediatrician, I believe in Munich, where, despite not having attended medical school, he was much beloved by his patients and their families. Paradies had been an associate of my own post-doctoral mentor, Sir Aaron Klug. I've no way of knowing whether or not he actually faked the data in his report of tRNA crystallization. His crystals did not diffract in any case, which may have driven him to short cuts. The other celebrated Fraud was Mark Spector, who embarrassed (and indeed victimized) Ephraim Racker at Cornell by using 125Iodine to construct gel autoradiographs to support his remarkable notion of the use of phosphorylation and dephosphorylation in cell signaling. His data were entirely fictitious, but it turned out that his ideas were pregnant indeed. I still view the cross-checking he provoked in serious students of signaling as having stimulated the entire field and actually accelerated it. Both Paradies and Spector are gifted fakes. Their work deserves appreciation for the intelligence that went into the tales they told. A lay homolog was Ferdinand Waldo Demara, who had very little formal education, but who established himself as outstanding in several fields, including open heart surgery, which he performed on a Japanese sailor rescued from after a battle, and who had shrapnel very close to his heart. Apparently, the sailor lived, and Demara saved his life. His story is told in a wonderful film with Tony Curtis in the roll, called The Great Imposter. I hope I've answered your question about what I meant to say on the subject. Charlie On Oct 19, 2012, at 11:25 AM, dom.bell...@diamond.ac.ukmailto:dom.bell...@diamond.ac.uk wrote: Dear Charlie, Do you mean that small doses of fraud should be accepted as a form of natural evolution? Or perhaps you were suggesting that genuine errors/mistakes are acceptable in 1/1 due to the high costs of spotting them? D From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 13:09 To: ccp4bb Subject: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud Begin forwarded message: Date: October 19, 2012 4:40:35 AM EDT To: Randy Read rj...@cam.ac.ukmailto:rj...@cam.ac.uk Subject: Re: [ccp4bb] PNAS on fraud This thread has been quite interesting to me. I've had a long interest in scientific fraud, which I've generally held to be victimless. While that view is unsupportable in a fundamental sense, I feel strongly that we should understand that error correction costs exponentially more, the smaller the tolerance for errors. In protein synthesis, evolution has settled on error rates of ~1 in 4000-1. Ensuring those rates is already costly in terms of NTPs hydrolyzed. NASA peer review provided me another shock: budgets for microgravity experiments were an order of magnitude higher than those for ground-based experiments, and most of the increase came via NASA's insistence on higher quality control. Informally, I've concluded that the rate of scientific fraud in all journals is probably less than the 1 in 10,000 that (mother) nature settled on. I concur with Randy. Charlie On Oct 18, 2012, at 2:43 PM, Randy Read wrote: In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell
Re: [ccp4bb] PNAS on fraud
This is worth looking at as well. Suggests most papers should be retracted! http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 Colin From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 17:55 To: ccp4bb Subject: Re: [ccp4bb] PNAS on fraud Dom, You've opened a pandora's box here, which I won't try to contain. The short answer is both of the above. I feel it is becoming increasingly difficult as a referee to be on top of every paper I review, and as an editor it is becoming increasingly difficult to find willing referees. Both phenomena are diagnostic of the cost of eliminating fraudulent publications, which gall me pretty much as much as they do many others, but which do not drive me apoplectic, either. I've been amused over the years by the frantic efforts to bring crystallographic charlatans to justice, even as I've been angered by publication in high-impact journals of material I myself view as fraudulent, but which obviously survives peer review. On the second of your alternatives, I'll give you two examples of highly celebrated frauds that wound up moving science forward, despite their scurrilous background. The first is the story of Hasko Paradies, whose only legitimate publication, as far as I know, was a first-author paper on the crystallization of tRNA. In that paper, he was, I think, the first author to describe the use of spermine/spermidine and Mg++ ions in improving crystallization conditions. These two contributions proved useful in the actual generation by others of suitable crystals. Paradies apparently went on to make a habit of filching precession photographs from dark rooms and then presenting them elsewhere and at meetings as if he had taken them and as if they were from hot problems of the day. His story was chronicled by Wayne Hendrickson, Ed Lattman, and others in Nature many years later. He dropped out of science and became a pediatrician, I believe in Munich, where, despite not having attended medical school, he was much beloved by his patients and their families. Paradies had been an associate of my own post-doctoral mentor, Sir Aaron Klug. I've no way of knowing whether or not he actually faked the data in his report of tRNA crystallization. His crystals did not diffract in any case, which may have driven him to short cuts. The other celebrated Fraud was Mark Spector, who embarrassed (and indeed victimized) Ephraim Racker at Cornell by using 125Iodine to construct gel autoradiographs to support his remarkable notion of the use of phosphorylation and dephosphorylation in cell signaling. His data were entirely fictitious, but it turned out that his ideas were pregnant indeed. I still view the cross-checking he provoked in serious students of signaling as having stimulated the entire field and actually accelerated it. Both Paradies and Spector are gifted fakes. Their work deserves appreciation for the intelligence that went into the tales they told. A lay homolog was Ferdinand Waldo Demara, who had very little formal education, but who established himself as outstanding in several fields, including open heart surgery, which he performed on a Japanese sailor rescued from after a battle, and who had shrapnel very close to his heart. Apparently, the sailor lived, and Demara saved his life. His story is told in a wonderful film with Tony Curtis in the roll, called The Great Imposter. I hope I've answered your question about what I meant to say on the subject. Charlie On Oct 19, 2012, at 11:25 AM, dom.bell...@diamond.ac.ukmailto:dom.bell...@diamond.ac.uk wrote: Dear Charlie, Do you mean that small doses of fraud should be accepted as a form of natural evolution? Or perhaps you were suggesting that genuine errors/mistakes are acceptable in 1/1 due to the high costs of spotting them? D From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 13:09 To: ccp4bb Subject: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud Begin forwarded message: Date: October 19, 2012 4:40:35 AM EDT To: Randy Read rj...@cam.ac.ukmailto:rj...@cam.ac.uk Subject: Re: [ccp4bb] PNAS on fraud This thread has been quite interesting to me. I've had a long interest in scientific fraud, which I've generally held to be victimless. While that view is unsupportable in a fundamental sense, I feel strongly that we should understand that error correction costs exponentially more, the smaller the tolerance for errors. In protein synthesis, evolution has settled on error rates of ~1 in 4000-1. Ensuring those rates is already costly in terms of NTPs hydrolyzed. NASA peer review provided me another shock: budgets for microgravity experiments were an order of magnitude higher than those for ground-based experiments, and most of the increase came via NASA's insistence on higher quality control. Informally, I've concluded
Re: [ccp4bb] PNAS on fraud
On Friday, October 19, 2012 10:12:44 am Colin Nave wrote: This is worth looking at as well. Suggests most papers should be retracted! http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 A paper claiming that all papers are false, by someone named Ioannidis? I wonder if he is from Crete :-) E for channeling Epimenides Merritt -- Ethan A Merritt Biomolecular Structure Center, K-428 Health Sciences Bldg University of Washington, Seattle 98195-7742
Re: [ccp4bb] PNAS on fraud
This gets us more into the philosophy of science but I've always felt authors had a right to speculate in the discussion sections of their papers on what it all means. And speculate even past the information in the actual data (see for example the wonderfully prescient final lines of the Watson Crick paper). As long as the experiments are fully described and the confidence of the data is clearly spelled out. George T. DeTitta, Ph.D. Principal Research Scientist Hauptman-Woodward Institute Professor Department of Structural Biology SUNY at Buffalo 700 Ellicott Street Buffalo NY 14203-1102 USA (716) 898-8611 (voice) (716) 480-8615 (mobile) (716) 898-8660 (fax) deti...@hwi.buffalo.edu www.hwi.buffalo.edu -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Colin Nave Sent: Friday, October 19, 2012 1:13 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud This is worth looking at as well. Suggests most papers should be retracted! http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 Colin From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 17:55 To: ccp4bb Subject: Re: [ccp4bb] PNAS on fraud Dom, You've opened a pandora's box here, which I won't try to contain. The short answer is both of the above. I feel it is becoming increasingly difficult as a referee to be on top of every paper I review, and as an editor it is becoming increasingly difficult to find willing referees. Both phenomena are diagnostic of the cost of eliminating fraudulent publications, which gall me pretty much as much as they do many others, but which do not drive me apoplectic, either. I've been amused over the years by the frantic efforts to bring crystallographic charlatans to justice, even as I've been angered by publication in high-impact journals of material I myself view as fraudulent, but which obviously survives peer review. On the second of your alternatives, I'll give you two examples of highly celebrated frauds that wound up moving science forward, despite their scurrilous background. The first is the story of Hasko Paradies, whose only legitimate publication, as far as I know, was a first-author paper on the crystallization of tRNA. In that paper, he was, I think, the first author to describe the use of spermine/spermidine and Mg++ ions in improving crystallization conditions. These two contributions proved useful in the actual generation by others of suitable crystals. Paradies apparently went on to make a habit of filching precession photographs from dark rooms and then presenting them elsewhere and at meetings as if he had taken them and as if they were from hot problems of the day. His story was chronicled by Wayne Hendrickson, Ed Lattman, and others in Nature many years later. He dropped out of science and became a pediatrician, I believe in Munich, where, despite not having attended medical school, he was much beloved by his patients and their families. Paradies had been an associate of my own post-doctoral mentor, Sir Aaron Klug. I've no way of knowing whether or not he actually faked the data in his report of tRNA crystallization. His crystals did not diffract in any case, which may have driven him to short cuts. The other celebrated Fraud was Mark Spector, who embarrassed (and indeed victimized) Ephraim Racker at Cornell by using 125Iodine to construct gel autoradiographs to support his remarkable notion of the use of phosphorylation and dephosphorylation in cell signaling. His data were entirely fictitious, but it turned out that his ideas were pregnant indeed. I still view the cross-checking he provoked in serious students of signaling as having stimulated the entire field and actually accelerated it. Both Paradies and Spector are gifted fakes. Their work deserves appreciation for the intelligence that went into the tales they told. A lay homolog was Ferdinand Waldo Demara, who had very little formal education, but who established himself as outstanding in several fields, including open heart surgery, which he performed on a Japanese sailor rescued from after a battle, and who had shrapnel very close to his heart. Apparently, the sailor lived, and Demara saved his life. His story is told in a wonderful film with Tony Curtis in the roll, called The Great Imposter. I hope I've answered your question about what I meant to say on the subject. Charlie On Oct 19, 2012, at 11:25 AM, dom.bell...@diamond.ac.ukmailto:dom.bell...@diamond.ac.uk wrote: Dear Charlie, Do you mean that small doses of fraud should be accepted as a form of natural evolution? Or perhaps you were suggesting that genuine errors/mistakes are acceptable in 1/1 due to the high costs of spotting them? D From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 13:09 To: ccp4bb Subject: [ccp4bb] Fwd
[ccp4bb] A case of post publication fraud...Re: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud
Dear Colleagues, A different type of, post publication, fraud is the case of the discovery of streptomycin. See :- http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)61202-1/fulltext I am just returned from the ICSTI Conference on Science, Law and Ethics in Washington DC representing IUCr where I learnt of this very disturbing case. I explained to the book's author and speaker Peter Pringle that, on behalf of Universities today, procedures are now in place, at least in the University of Manchester, for graduate students and supervisors to both sign within 'eprog' that they have discussed matters of authorship etiquette and rules as well as IP sharing formalities. Have a good weekend, John Prof John R Helliwell DSc On 19 Oct 2012, at 13:08, Carter, Charlie car...@med.unc.edu wrote: Begin forwarded message: Date: October 19, 2012 4:40:35 AM EDT To: Randy Read rj...@cam.ac.uk Subject: Re: [ccp4bb] PNAS on fraud This thread has been quite interesting to me. I've had a long interest in scientific fraud, which I've generally held to be victimless. While that view is unsupportable in a fundamental sense, I feel strongly that we should understand that error correction costs exponentially more, the smaller the tolerance for errors. In protein synthesis, evolution has settled on error rates of ~1 in 4000-1. Ensuring those rates is already costly in terms of NTPs hydrolyzed. NASA peer review provided me another shock: budgets for microgravity experiments were an order of magnitude higher than those for ground-based experiments, and most of the increase came via NASA's insistence on higher quality control. Informally, I've concluded that the rate of scientific fraud in all journals is probably less than the 1 in 10,000 that (mother) nature settled on. I concur with Randy. Charlie On Oct 18, 2012, at 2:43 PM, Randy Read wrote: In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell structures would be worse!
Re: [ccp4bb] PNAS on fraud
I think the real point here is that a difference exits between divergent interpretation of legitimate evidence - which is normal scientific epistemology - or whether the presented 'evidence' is in some fashion tampered with. The former is healthy procedure and (I hope) not subject of disagreement - we all have been wrong a few times at least and corrected either by better insight or new evidence (or actually useful reviews) - and the question boils down to where 'tampering' with evidence starts. Is willful neglect of contrary results tinkering? Is looking only for reinforcing data already tinkering (aka expectation and confirmation bias)? It is easy to judge in the case of poorly fabricated stuff like bet V1 or c3b, but I think the borderline cases are potentially much more damaging. Btw, I have few more references to the psychology of science Koehler JJ (1993) The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality. Organizational Behavior and Human Decision Processes 56(1): 28-55. Simmons JP, Nelson LD and Simonsohn U (2011) False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science: DOI: 10.1177/0956797611417632. Frey BS (2003) Publishing as Prostitution? Choosing Between Ones Own Ideas and Academic Failure. Public Choice 116, 205-223 (ETHZ) Nice weekend reading. Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of George DeTitta Sent: Friday, October 19, 2012 10:46 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud This gets us more into the philosophy of science but I've always felt authors had a right to speculate in the discussion sections of their papers on what it all means. And speculate even past the information in the actual data (see for example the wonderfully prescient final lines of the Watson Crick paper). As long as the experiments are fully described and the confidence of the data is clearly spelled out. George T. DeTitta, Ph.D. Principal Research Scientist Hauptman-Woodward Institute Professor Department of Structural Biology SUNY at Buffalo 700 Ellicott Street Buffalo NY 14203-1102 USA (716) 898-8611 (voice) (716) 480-8615 (mobile) (716) 898-8660 (fax) deti...@hwi.buffalo.edu www.hwi.buffalo.edu -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Colin Nave Sent: Friday, October 19, 2012 1:13 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud This is worth looking at as well. Suggests most papers should be retracted! http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 Colin From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Carter, Charlie Sent: 19 October 2012 17:55 To: ccp4bb Subject: Re: [ccp4bb] PNAS on fraud Dom, You've opened a pandora's box here, which I won't try to contain. The short answer is both of the above. I feel it is becoming increasingly difficult as a referee to be on top of every paper I review, and as an editor it is becoming increasingly difficult to find willing referees. Both phenomena are diagnostic of the cost of eliminating fraudulent publications, which gall me pretty much as much as they do many others, but which do not drive me apoplectic, either. I've been amused over the years by the frantic efforts to bring crystallographic charlatans to justice, even as I've been angered by publication in high-impact journals of material I myself view as fraudulent, but which obviously survives peer review. On the second of your alternatives, I'll give you two examples of highly celebrated frauds that wound up moving science forward, despite their scurrilous background. The first is the story of Hasko Paradies, whose only legitimate publication, as far as I know, was a first-author paper on the crystallization of tRNA. In that paper, he was, I think, the first author to describe the use of spermine/spermidine and Mg++ ions in improving crystallization conditions. These two contributions proved useful in the actual generation by others of suitable crystals. Paradies apparently went on to make a habit of filching precession photographs from dark rooms and then presenting them elsewhere and at meetings as if he had taken them and as if they were from hot problems of the day. His story was chronicled by Wayne Hendrickson, Ed Lattman, and others in Nature many years later. He dropped out of science and became a pediatrician, I believe in Munich, where, despite not having attended medical school, he was much beloved by his patients and their families. Paradies had been an associate of my own post-doctoral mentor, Sir Aaron Klug. I've no way of knowing whether or not he actually faked the data in his report of tRNA crystallization. His crystals did not diffract in any case, which may have driven him to short cuts. The other celebrated Fraud was Mark Spector, who embarrassed
Re: [ccp4bb] PNAS on fraud
CCP4- intetesting topic and many off-target topics veing discussed. John Ionnandis' work, who is an epidemiologist and statistician, addresses issues in the design and interpretation of GWAS studies for SNPs and disease associations in one hand and clinical studies, especially Phase III intervention studies on the other. His primary interest is in statistic interpretation of these studies rather than 'fabrication' or purposeful introduction of erroneous information into study reports. He is a useful if controversial advocate for clarity in statistical concepts for these works. Dexter Kennedy, MD Thumbed from my iPhone On Oct 19, 2012, at 10:40 AM, Ethan Merritt merr...@u.washington.edu wrote: On Friday, October 19, 2012 10:12:44 am Colin Nave wrote: This is worth looking at as well. Suggests most papers should be retracted! http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124 A paper claiming that all papers are false, by someone named Ioannidis? I wonder if he is from Crete :-) E for channeling Epimenides Merritt -- Ethan A Merritt Biomolecular Structure Center, K-428 Health Sciences Bldg University of Washington, Seattle 98195-7742
Re: [ccp4bb] A case of post publication fraud...Re: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud
On Fri, Oct 19, 2012 at 2:20 PM, Jrh jrhelliw...@gmail.com wrote: Dear Colleagues, A different type of, post publication, fraud is the case of the discovery of streptomycin. See :- http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)61202-1/fulltext I can't resist posting this quite interesting tangential fact that is unrelated to fraud : penicillin was discovered by Ernest Duchesne. -Bryan
[ccp4bb] Jrh further Re: [ccp4bb] A case of post publication fraud...Re: [ccp4bb] Fwd: [ccp4bb] PNAS on fraud
One of the hardest things for an author, and a handling Editor, is making sure that the references list of a submitted article is complete, but is an easier task now with our e-tools than in the days of the penicillin discovery. Another case is that of Einstein's special theory article of 1905 where Lorentz was not cited. Abraham Paix explored this in the biography of Einstein noting that Einstein did finally acknowledge that they, Lorentz and Einstein, had been in correspondence on the topic. Prof John R Helliwell DSc On 19 Oct 2012, at 19:43, Bryan Lepore bryanlep...@gmail.com wrote: On Fri, Oct 19, 2012 at 2:20 PM, Jrh jrhelliw...@gmail.com wrote: Dear Colleagues, A different type of, post publication, fraud is the case of the discovery of streptomycin. See :- http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)61202-1/fulltext I can't resist posting this quite interesting tangential fact that is unrelated to fraud : penicillin was discovered by Ernest Duchesne. -Bryan
[ccp4bb] PNAS on fraud
Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ -
Re: [ccp4bb] PNAS on fraud
Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Philippe Dumas Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ -
Re: [ccp4bb] PNAS on fraud
The fit seems to be driven by the high number of points in the area of the graph where many points overlap. The points that catch your eye and establish the visible balance probably do not contribute much. Maybe this one should have been plotted as log in the abscissa for appearances. James On Oct 18, 2012, at 11:52 AM, DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Philippe Dumas Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ -
Re: [ccp4bb] PNAS on fraud
On Thursday, October 18, 2012 10:52:48 am DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan Philippe Dumas Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ - -- Ethan A Merritt Biomolecular Structure Center, K-428 Health Sciences Bldg University of Washington, Seattle 98195-7742
Re: [ccp4bb] PNAS on fraud
One might include independent prior evidence (Kleywegt, Brown @ Ramaswami) showing that in general most other quality indicators are worse for high impact journals. So, as a frequentist I agree that his correlation is significantly weak, as a Bayesian I say it is reasonably probable. Cheers, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud On Thursday, October 18, 2012 10:52:48 am DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan
Re: [ccp4bb] PNAS on fraud
As much fun as it is to bash Nature, Science and Cell, the evidence that they publish poorer quality structures doesn't actually hold up well. Gerard Kleywegt (cited below) and I tried to use that supposition as the basis of a positive control for our case-controlled validation paper in Acta D, but we were surprised that once you account for the fact that the high-profile journals tend to publish papers on bigger structures that generally diffract to lower resolution, there's actually very little evidence that those structures are worse than comparable lower-resolution structures in lower-impact journals. They probably do have more than their fair share of retractions -- but then it's hard to control for the varying level of scrutiny applied to papers published in different journals. In support of Bayesian reasoning, it's good to see that the data could over-rule our prior belief that Nature/Science/Cell structures would be worse! - Randy J. Read Department of Haematology, University of Cambridge Cambridge Institute for Medical ResearchTel: +44 1223 336500 Wellcome Trust/MRC Building Fax: +44 1223 336827 Hills RoadE-mail: rj...@cam.ac.uk Cambridge CB2 0XY, U.K. www-structmed.cimr.cam.ac.uk On 18 Oct 2012, at 19:31, Bernhard Rupp (Hofkristallrat a.D.) wrote: One might include independent prior evidence (Kleywegt, Brown @ Ramaswami) showing that in general most other quality indicators are worse for high impact journals. So, as a frequentist I agree that his correlation is significantly weak, as a Bayesian I say it is reasonably probable. Cheers, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud On Thursday, October 18, 2012 10:52:48 am DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan
Re: [ccp4bb] PNAS on fraud
My two cents: the R-squared for figure 3A is 9%, therefore only a minor proportion of the variation (or random noise) in the data was explained by the fitted model, taking a log scale may reduce that random scatter look but the fit is essentially the same. On Thu, Oct 18, 2012 at 12:10 PM, Ethan Merritt merr...@u.washington.eduwrote: On Thursday, October 18, 2012 10:52:48 am DUMAS Philippe (UDS) wrote: Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan Philippe Dumas Dear CCP4 followers, Maybe you are already aware of this interesting study in PNAS regarding the prevalence of fraud vs. 'real' error in paper retractions: Fang FC, Steen RG and Casadevall A (2012) Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 109(42): 17028-33. http://www.pnas.org/content/109/42/17028.abstract There were also a few comments on related stuff such as fake peer review in the Chronicle of Higher Education. As not all may have access to that journal, I have put the 3 relevant pdf links on my web http://www.ruppweb.org/CHE_Misconduct_PNAS_Stuft_Oct_2012.pdf http://www.ruppweb.org/CHE_DYI_reviews_Sept_30_2012.pdf http://www.ruppweb.org/CHE_The-Great-Pretender_Oct_8_2012.pdf Best regards, BR - Bernhard Rupp 001 (925) 209-7429 +43 (676) 571-0536 b...@ruppweb.org hofkristall...@gmail.com http://www.ruppweb.org/ - -- Ethan A Merritt Biomolecular Structure Center, K-428 Health Sciences Bldg University of Washington, Seattle 98195-7742
Re: [ccp4bb] PNAS on fraud
I think that the jump between fraud and other quality indicators is a bit too steep for me. Poor quality indicators may suggest poor data that the xtal was willing to diffract, a concept that to me is very orthogonal to fraud. Fred [32m*** Fred Dyda, Ph.D. Phone:301-402-4496 Laboratory of Molecular BiologyFax: 301-496-0201 DHHS/NIH/NIDDK e-mail:fred.d...@nih.gov Bldg. 5. Room 303 Bethesda, MD 20892-0560 URGENT message e-mail: 2022476...@mms.att.net Google maps coords: 39.000597, -77.102102 http://www2.niddk.nih.gov/NIDDKLabs/IntramuralFaculty/DydaFred ***[m
Re: [ccp4bb] PNAS on fraud
Randy Read just pointed out to me that in their case-controlled analysis paper http://journals.iucr.org/d/issues/2009/02/00/ba5130/index.html when considering lower resolution and other factors, the vanity journals seem to come out no worse than the rest. In any case I suspect any retractions are underrepresented in those journals because they fight it harder ;-) Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan
Re: [ccp4bb] PNAS on fraud
Just to add in the controversy, with a somewhat related issue: Current crystallographic ethic presumes that a structure is deposited just before the submission of the paper. In a survey we did, we found that while in one journal only 2% of structures are deposited after the paper submission date, on another thats 5%, on another one that is 29% and in yet another one close to 50%. The journals are Nature, Science, ActaD and Proteins in order of decreasing IF. Is there any correlation? To get some guesses first, Robbie can send the answer tomorrow at around noon (as I will be unavailable travelling ...) Tassos On 18 Oct 2012, at 21:13, Bernhard Rupp (Hofkristallrat a.D.) wrote: Randy Read just pointed out to me that in their case-controlled analysis paper http://journals.iucr.org/d/issues/2009/02/00/ba5130/index.html when considering lower resolution and other factors, the vanity journals seem to come out no worse than the rest. In any case I suspect any retractions are underrepresented in those journals because they fight it harder ;-) Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan
Re: [ccp4bb] PNAS on fraud
Tassos, just to clarify what you are saying in the Journal with 2% deposition after submission, 98% have been deposited prior to submission (the way it should be). Is that what you are saying or am I reading that wrong ? Or are you saying only 2% of structures are deposited in that journal ? Jürgen On Oct 18, 2012, at 3:24 PM, Anastassis Perrakis wrote: Just to add in the controversy, with a somewhat related issue: Current crystallographic ethic presumes that a structure is deposited just before the submission of the paper. In a survey we did, we found that while in one journal only 2% of structures are deposited after the paper submission date, on another thats 5%, on another one that is 29% and in yet another one close to 50%. The journals are Nature, Science, ActaD and Proteins in order of decreasing IF. Is there any correlation? To get some guesses first, Robbie can send the answer tomorrow at around noon (as I will be unavailable travelling ...) Tassos On 18 Oct 2012, at 21:13, Bernhard Rupp (Hofkristallrat a.D.) wrote: Randy Read just pointed out to me that in their case-controlled analysis paper http://journals.iucr.org/d/issues/2009/02/00/ba5130/index.html when considering lower resolution and other factors, the vanity journals seem to come out no worse than the rest. In any case I suspect any retractions are underrepresented in those journals because they fight it harder ;-) Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UKmailto:CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan .. Jürgen Bosch Johns Hopkins University Bloomberg School of Public Health Department of Biochemistry Molecular Biology Johns Hopkins Malaria Research Institute 615 North Wolfe Street, W8708 Baltimore, MD 21205 Office: +1-410-614-4742 Lab: +1-410-614-4894 Fax: +1-410-955-2926 http://lupo.jhsph.edu
Re: [ccp4bb] PNAS on fraud
On 18 Oct 2012, at 21:30, Bosch, Juergen wrote: Tassos, just to clarify what you are saying in the Journal with 2% deposition after submission, 98% have been deposited prior to submission (the way it should be). Is that what you are saying or am I reading that wrong ? Yes, that is what I am saying! 2% is good, 50% is bad. (btw, the 'worse' is close to 70% - any guesses?) A. Or are you saying only 2% of structures are deposited in that journal ? Jürgen On Oct 18, 2012, at 3:24 PM, Anastassis Perrakis wrote: Just to add in the controversy, with a somewhat related issue: Current crystallographic ethic presumes that a structure is deposited just before the submission of the paper. In a survey we did, we found that while in one journal only 2% of structures are deposited after the paper submission date, on another thats 5%, on another one that is 29% and in yet another one close to 50%. The journals are Nature, Science, ActaD and Proteins in order of decreasing IF. Is there any correlation? To get some guesses first, Robbie can send the answer tomorrow at around noon (as I will be unavailable travelling ...) Tassos On 18 Oct 2012, at 21:13, Bernhard Rupp (Hofkristallrat a.D.) wrote: Randy Read just pointed out to me that in their case-controlled analysis paper http://journals.iucr.org/d/issues/2009/02/00/ba5130/index.html when considering lower resolution and other factors, the vanity journals seem to come out no worse than the rest. In any case I suspect any retractions are underrepresented in those journals because they fight it harder ;-) Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan .. Jürgen Bosch Johns Hopkins University Bloomberg School of Public Health Department of Biochemistry Molecular Biology Johns Hopkins Malaria Research Institute 615 North Wolfe Street, W8708 Baltimore, MD 21205 Office: +1-410-614-4742 Lab: +1-410-614-4894 Fax: +1-410-955-2926 http://lupo.jhsph.edu
Re: [ccp4bb] PNAS on fraud
That must be an NMR journal :-) Jürgen On Oct 18, 2012, at 3:34 PM, Anastassis Perrakis wrote: On 18 Oct 2012, at 21:30, Bosch, Juergen wrote: Tassos, just to clarify what you are saying in the Journal with 2% deposition after submission, 98% have been deposited prior to submission (the way it should be). Is that what you are saying or am I reading that wrong ? Yes, that is what I am saying! 2% is good, 50% is bad. (btw, the 'worse' is close to 70% - any guesses?) A. Or are you saying only 2% of structures are deposited in that journal ? Jürgen On Oct 18, 2012, at 3:24 PM, Anastassis Perrakis wrote: Just to add in the controversy, with a somewhat related issue: Current crystallographic ethic presumes that a structure is deposited just before the submission of the paper. In a survey we did, we found that while in one journal only 2% of structures are deposited after the paper submission date, on another thats 5%, on another one that is 29% and in yet another one close to 50%. The journals are Nature, Science, ActaD and Proteins in order of decreasing IF. Is there any correlation? To get some guesses first, Robbie can send the answer tomorrow at around noon (as I will be unavailable travelling ...) Tassos On 18 Oct 2012, at 21:13, Bernhard Rupp (Hofkristallrat a.D.) wrote: Randy Read just pointed out to me that in their case-controlled analysis paper http://journals.iucr.org/d/issues/2009/02/00/ba5130/index.html when considering lower resolution and other factors, the vanity journals seem to come out no worse than the rest. In any case I suspect any retractions are underrepresented in those journals because they fight it harder ;-) Best, BR -Original Message- From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Ethan Merritt Sent: Thursday, October 18, 2012 11:11 AM To: CCP4BB@JISCMAIL.AC.UKmailto:CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Fang et al. claim that R^2 = 0.0866, which means that CC = 0.29. While a correlation coefficient of less than 0.3 is not a complete lack of correlation, it's still rather weak. The highly significant must be taken in a purely statistical sense. That is, it doesn't mean the measures are highly correlated, it means the evidence for non-zero correlation is very strong. Ethan .. Jürgen Bosch Johns Hopkins University Bloomberg School of Public Health Department of Biochemistry Molecular Biology Johns Hopkins Malaria Research Institute 615 North Wolfe Street, W8708 Baltimore, MD 21205 Office: +1-410-614-4742 Lab: +1-410-614-4894 Fax: +1-410-955-2926 http://lupo.jhsph.eduhttp://lupo.jhsph.edu/ .. Jürgen Bosch Johns Hopkins University Bloomberg School of Public Health Department of Biochemistry Molecular Biology Johns Hopkins Malaria Research Institute 615 North Wolfe Street, W8708 Baltimore, MD 21205 Office: +1-410-614-4742 Lab: +1-410-614-4894 Fax: +1-410-955-2926 http://lupo.jhsph.edu
Re: [ccp4bb] PNAS on fraud
On curve fitting: http://twitpic.com/8jd081 -- From: DUMAS Philippe (UDS) p.du...@ibmc-cnrs.unistra.fr Sent: Thursday, October 18, 2012 1:52 PM To: CCP4BB@JISCMAIL.AC.UK Subject: Re: [ccp4bb] PNAS on fraud Le Jeudi 18 Octobre 2012 19:16 CEST, Bernhard Rupp (Hofkristallrat a.D.) hofkristall...@gmail.com a écrit: I had a look to this PNAS paper by Fang et al. I am a bit surprised by their interpretation of their Fig. 3: they claim that here exists a highly signficant correlation between Impact factor and number of retractations. Personnaly, I would have concluded to a complete lack of correlation... Should I retract this judgment? Philippe Dumas
