Re: [R] Potencial bug in R.adjust (holm method)
On 06 Aug 2015, at 19:20 , Iker Vaquero Alba karrasp...@yahoo.es wrote: Hello all, I am doing some Bonferroni correction analyses with R.adjust function. I have a spreadsheet with 24 columns, each with 5 values. When I use the holm method, it gives me adjusted figures for all the original values except from the ones in the 4th row of each column. I mean, the value on the 4th row for every column is exactly the same either in the original data or in the corrected one. I've tried using another algorithm just to see what happens (bonferroni, for example) and everything is fine, I get corrected figures for all the values, even the ones on the 4th row. Does anyone know whether this is any kind of known bug of the holm algorithm of P.adjust function. If so, should I worry about it? If so, can anybody suggest any possible solution? Thank you very much. Iker p.adjust(), I presume? The Holm procedure is essentially to sort p-values in decreasing order and mutiplying by 1:n plus a little fiddling to keep the order and prevent p 1. The logic is that if you reject the hypothesis corresponding to the smallest p after Bonferroni-correction by N, you only have N-1 simultaneous tests to consider, etc. The smallest multiplier will be 1, so it's not strange that one value appears uncorrected. It's curious that it is always in the 4th row, but it might be that the p-values in the 4 other rows are all considerably smaller. You do realize that if you run p.adjust for each column, you are not actually adjusting for the total of 120 tests, only for 5 of them, effectively ignoring the 23 other columns every time? -pd __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln,Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Potencial bug in R.adjust (holm method)
Please send a reproducible example. Best, Uwe Ligges On 06.08.2015 19:20, Iker Vaquero Alba wrote: Hello all, I am doing some Bonferroni correction analyses with R.adjust function. I have a spreadsheet with 24 columns, each with 5 values. When I use the holm method, it gives me adjusted figures for all the original values except from the ones in the 4th row of each column. I mean, the value on the 4th row for every column is exactly the same either in the original data or in the corrected one. I've tried using another algorithm just to see what happens (bonferroni, for example) and everything is fine, I get corrected figures for all the values, even the ones on the 4th row. Does anyone know whether this is any kind of known bug of the holm algorithm of P.adjust function. If so, should I worry about it? If so, can anybody suggest any possible solution? Thank you very much. Iker __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln,Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Potencial bug in R.adjust (holm method)
On 07 Aug 2015, at 16:20 , Iker Vaquero Alba karrasp...@yahoo.es wrote: Hello, Peter, and thank you for your clarifying reply. Actually, that was another doubt I had. As I take my data from different files (every bunch of 5 values is taken from a different file), I assumed that, if I wanted to adjust for the error due to repeated measures, it should be enough to do it inside each file, right? I mean, the possible error due to repeated measures shouldn't go beyond each file. Once you close one file, open and attach a new one, the count, let's say, starts from scratch. Am I right? And if I'm not, what is the reason why should I pool all the p-values in a massive column and adjust them in bulk? There's no clear answer to that kind of question. It depends on which kind of error rate you want to control, but in principle multiple response variables are not different than multiple comparisons. If you test 20 response variables at level 5%, on average 1 will come out significant even if there are no actual effects (if the variables are highly correlated, this may mean that all 20 come out significant with 5% probability, though). Thank you very much for your help. Iker __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln, Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 De: peter dalgaard pda...@gmail.com Para: Iker Vaquero Alba karrasp...@yahoo.es CC: r-help@r-project.org r-help@r-project.org Enviado: Viernes 7 de agosto de 2015 14:40 Asunto: Re: [R] Potencial bug in R.adjust (holm method) On 06 Aug 2015, at 19:20 , Iker Vaquero Alba karrasp...@yahoo.es wrote: Hello all, I am doing some Bonferroni correction analyses with R.adjust function. I have a spreadsheet with 24 columns, each with 5 values. When I use the holm method, it gives me adjusted figures for all the original values except from the ones in the 4th row of each column. I mean, the value on the 4th row for every column is exactly the same either in the original data or in the corrected one. I've tried using another algorithm just to see what happens (bonferroni, for example) and everything is fine, I get corrected figures for all the values, even the ones on the 4th row. Does anyone know whether this is any kind of known bug of the holm algorithm of P.adjust function. If so, should I worry about it? If so, can anybody suggest any possible solution? Thank you very much. Iker p.adjust(), I presume? The Holm procedure is essentially to sort p-values in decreasing order and mutiplying by 1:n plus a little fiddling to keep the order and prevent p 1. The logic is that if you reject the hypothesis corresponding to the smallest p after Bonferroni-correction by N, you only have N-1 simultaneous tests to consider, etc. The smallest multiplier will be 1, so it's not strange that one value appears uncorrected. It's curious that it is always in the 4th row, but it might be that the p-values in the 4 other rows are all considerably smaller. You do realize that if you run p.adjust for each column, you are not actually adjusting for the total of 120 tests, only for 5 of them, effectively ignoring the 23 other columns every time? -pd __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln,Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd@cbs.dk Priv: pda...@gmail.com -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
Re: [R] Potencial bug in R.adjust (holm method)
Hello, Peter, and thank you for your clarifying reply. Actually, that was another doubt I had. As I take my data from different files (every bunch of 5 values is taken from a different file), I assumed that, if I wanted to adjust for the error due to repeated measures, it should be enough to do it inside each file, right? I mean, the possible error due to repeated measures shouldn't go beyond each file. Once you close one file, open and attach a new one, the count, let's say, starts from scratch. Am I right? And if I'm not, what is the reason why should I pool all the p-values in a massive column and adjust them in bulk? Thank you very much for your help. Iker __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln, Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 De: peter dalgaard pda...@gmail.com CC: r-help@r-project.org r-help@r-project.org Enviado: Viernes 7 de agosto de 2015 14:40 Asunto: Re: [R] Potencial bug in R.adjust (holm method) Hello all, I am doing some Bonferroni correction analyses with R.adjust function. I have a spreadsheet with 24 columns, each with 5 values. When I use the holm method, it gives me adjusted figures for all the original values except from the ones in the 4th row of each column. I mean, the value on the 4th row for every column is exactly the same either in the original data or in the corrected one. I've tried using another algorithm just to see what happens (bonferroni, for example) and everything is fine, I get corrected figures for all the values, even the ones on the 4th row. Does anyone know whether this is any kind of known bug of the holm algorithm of P.adjust function. If so, should I worry about it? If so, can anybody suggest any possible solution? Thank you very much. Iker p.adjust(), I presume? The Holm procedure is essentially to sort p-values in decreasing order and mutiplying by 1:n plus a little fiddling to keep the order and prevent p 1. The logic is that if you reject the hypothesis corresponding to the smallest p after Bonferroni-correction by N, you only have N-1 simultaneous tests to consider, etc. The smallest multiplier will be 1, so it's not strange that one value appears uncorrected. It's curious that it is always in the 4th row, but it might be that the p-values in the 4 other rows are all considerably smaller. You do realize that if you run p.adjust for each column, you are not actually adjusting for the total of 120 tests, only for 5 of them, effectively ignoring the 23 other columns every time? -pd __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln, Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: pd@cbs.dk Priv: pda...@gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Potencial bug in R.adjust (holm method)
Hello all, I am doing some Bonferroni correction analyses with R.adjust function. I have a spreadsheet with 24 columns, each with 5 values. When I use the holm method, it gives me adjusted figures for all the original values except from the ones in the 4th row of each column. I mean, the value on the 4th row for every column is exactly the same either in the original data or in the corrected one. I've tried using another algorithm just to see what happens (bonferroni, for example) and everything is fine, I get corrected figures for all the values, even the ones on the 4th row. Does anyone know whether this is any kind of known bug of the holm algorithm of P.adjust function. If so, should I worry about it? If so, can anybody suggest any possible solution? Thank you very much. Iker __ Dr. Iker Vaquero-Alba Visiting Postdoctoral Research Associate Laboratory of Evolutionary Ecology of Adaptations, School of Life Sciences, University of Lincoln, Riseholme Park Campus, Lincoln LN2 2LG, UK. https://eric.exeter.ac.uk/repository/handle/10036/3381 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
