Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
Dear Mr. Dalgaard, Could you help me know the name of post-hoc multi-comparaison test mentioned in kruskal function of agricolae package? Thank you in advance. Tham Tran -- View this message in context: http://r.789695.n4.nabble.com/Multiple-Comparisons-Kruskal-Wallis-Test-kruskal-agricolae-and-kruskalmc-pgirmess-don-t-yield-the-sa-tp4639004p4690739.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
On May 16, 2014, at 2:07 PM, Tham Tran wrote: Dear Mr. Dalgaard, You do realize that was a posting from 2012, right? Could you help me know the name of post-hoc multi-comparaison test mentioned in kruskal function of agricolae package? There are multiple such tests mentioned on that function help page. Did you review the material above the Nabble posting? (This is why including the context in every posting is the recommended way to pose a question on r-help.) See also the rest of the Posting Guide. PLEASE do read the posting guide http://www.R-project.org/posting-guide.html Thank you in advance. Tham Tran -- View this message in context: http://r.789695.n4.nabble.com/Multiple-Comparisons-Kruskal-Wallis-Test-kruskal-agricolae-and-kruskalmc-pgirmess-don-t-yield-the-sa-tp4639004p4690739.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help and provide commented, minimal, self-contained, reproducible code. David Winsemius Alameda, CA, USA __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
I see.. So apparently the different functions are doing the same thing! :-) Besides I didn't know the groups should have about the same size. Thank you four your time Mr. Dalgaard. -- View this message in context: http://r.789695.n4.nabble.com/Multiple-Comparisons-Kruskal-Wallis-Test-kruskal-agricolae-and-kruskalmc-pgirmess-don-t-yield-the-sa-tp4639004p4639431.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
On Aug 3, 2012, at 07:42 , greatest.possible.newbie wrote:
I am doing multiple comparisons for data that is not normally distributed.
For this purpose I tried both functions kruskal{agricolae} and
kruskalmc{pgirmess}. It confuses me that these functions do not yield the
same results although they are doing the same thing, don't they? Can anyone
tell my why this happens and which function I can trust?
kruskalmc() tells me that there are no differences between any of the groups
(i.e. the difference column of the results is filled only with FALSE).
kruskal() tells me that there are indeed differences (between group 4 and
11).
Trust nothing if you don't understand the issues involved. There's a reason
that special code is required for multiple comparisons, and a rather
complicated and inexact theory for dealing with it. There are special
complications with rank tests because the standard theory assumes a global null
(no group differences at all).
One of the approaches is to perform the testing pairwise and adjust the
p-values for multiple comparisons. Notice that the kruskal() function has a
p.adj argument which defaults to none!
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
Thank you for your answer. The p.adj argument in the kruskal()-function doesn't seem to change anything... Not even the bonferroni-method although it is described as the most conservative one (multiplying all p-values with the number of comparisons). I suppose the kruskal()-function is not working properly... On the other hand I doubt the method behind the kruskalmc()-function as this function doesn't even turn out to detect significant differences between the grouping variable (which is obviously a severe error). Do you think it is justifiable to use the kruskal()-function without p-adjustment, i.e. doing only pairwise tests like you can do with the kruskal.test()-function although I obviously want to do multiple comparisons? kruskal(x[,1],x[,2],p.adj=bonferroni) #Yields exactely the same results. #Groups, Treatments and mean of the ranks #a 11 304.4 #ab 9 296 #ab 7 286.6 #ab 8 278.2 #ab 10 268.7 #ab 2 250.6 #ab 6 242.9 #ab 1 242.1 #ab 3 239.4 #ab 5 228.8 #b 4 219.5 kruskalmc(x[,2],x[,2]) #Multiple comparison test after Kruskal-Wallis #p.value: 0.05 #Comparisons # obs.dif critical.dif difference #[..] #6-9 54.0162.02688 FALSE #6-10 69.5159.04584 FALSE #6-11 94.5133.02196 FALSE #7-8 18.0160.00778 FALSE #7-9 35.0169.78370 FALSE #7-10 50.5166.94123 FALSE #7-11 75.5142.36796 FALSE #8-9 17.0165.54197 FALSE #8-10 32.5162.62538 FALSE #8-11 57.5137.28174 FALSE #9-10 15.5172.25281 FALSE #9-11 40.5148.56074 FALSE #10-1125.0145.30369 FALSE -- View this message in context: http://r.789695.n4.nabble.com/Multiple-Comparisons-Kruskal-Wallis-Test-kruskal-agricolae-and-kruskalmc-pgirmess-don-t-yield-the-sa-tp4639004p4639027.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
On Aug 3, 2012, at 11:33 , greatest.possible.newbie wrote: Thank you for your answer. The p.adj argument in the kruskal()-function doesn't seem to change anything... Not even the bonferroni-method although it is described as the most conservative one (multiplying all p-values with the number of comparisons). I suppose the kruskal()-function is not working properly... Apparently, the grouping logic doesn't care about p.adj. Not the most fortunate design in my opinion, but try looking at the output with group=FALSE. On the other hand I doubt the method behind the kruskalmc()-function as this function doesn't even turn out to detect significant differences between the grouping variable (which is obviously a severe error). That's not obvious! Did you check all group comparisons? How big are the groups? Do you think it is justifiable to use the kruskal()-function without p-adjustment, i.e. doing only pairwise tests like you can do with the kruskal.test()-function although I obviously want to do multiple comparisons? kruskal(x[,1],x[,2],p.adj=bonferroni) #Yields exactely the same results. #Groups, Treatments and mean of the ranks #a 11 304.4 #ab9 296 #ab7 286.6 #ab8 278.2 #ab10 268.7 #ab2 250.6 #ab6 242.9 #ab1 242.1 #ab3 239.4 #ab5 228.8 #b 4 219.5 kruskalmc(x[,2],x[,2]) #Multiple comparison test after Kruskal-Wallis #p.value: 0.05 #Comparisons # obs.dif critical.dif difference #[..] #6-9 54.0162.02688 FALSE #6-10 69.5159.04584 FALSE #6-11 94.5133.02196 FALSE #7-8 18.0160.00778 FALSE #7-9 35.0169.78370 FALSE #7-10 50.5166.94123 FALSE #7-11 75.5142.36796 FALSE #8-9 17.0165.54197 FALSE #8-10 32.5162.62538 FALSE #8-11 57.5137.28174 FALSE #9-10 15.5172.25281 FALSE #9-11 40.5148.56074 FALSE #10-1125.0145.30369 FALSE -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
Generally multiple comparisons are conducted after a test for a significant
difference among any of the groups. For your data
kruskal.test(x[,1]~x[,2])
Kruskal-Wallis rank sum test
data: x[, 1] by x[, 2]
Kruskal-Wallis chi-squared = 11.0098, df = 10, p-value = 0.3568
There are no significant differences between the groups, so there is no
reason to use a multiple comparison test.
--
David L Carlson
Associate Professor of Anthropology
Texas AM University
College Station, TX 77843-4352
-Original Message-
From: r-help-boun...@r-project.org [mailto:r-help-bounces@r-
project.org] On Behalf Of peter dalgaard
Sent: Friday, August 03, 2012 5:59 AM
To: greatest.possible.newbie
Cc: r-help@r-project.org
Subject: Re: [R] Multiple Comparisons-Kruskal-Wallis-Test:
kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results
although they should do (?)
On Aug 3, 2012, at 11:33 , greatest.possible.newbie wrote:
Thank you for your answer.
The p.adj argument in the kruskal()-function doesn't seem to change
anything... Not even the bonferroni-method although it is described
as the
most conservative one (multiplying all p-values with the number of
comparisons). I suppose the kruskal()-function is not working
properly...
Apparently, the grouping logic doesn't care about p.adj. Not the most
fortunate design in my opinion, but try looking at the output with
group=FALSE.
On the other hand I doubt the method behind the kruskalmc()-function
as this
function doesn't even turn out to detect significant differences
between the
grouping variable (which is obviously a severe error).
That's not obvious! Did you check all group comparisons? How big are
the groups?
Do you think it is justifiable to use the kruskal()-function without
p-adjustment, i.e. doing only pairwise tests like you can do with the
kruskal.test()-function although I obviously want to do multiple
comparisons?
kruskal(x[,1],x[,2],p.adj=bonferroni)
#Yields exactely the same results.
#Groups, Treatments and mean of the ranks
#a 11 304.4
#ab 9 296
#ab 7 286.6
#ab 8 278.2
#ab 10 268.7
#ab 2 250.6
#ab 6 242.9
#ab 1 242.1
#ab 3 239.4
#ab 5 228.8
#b 4 219.5
kruskalmc(x[,2],x[,2])
#Multiple comparison test after Kruskal-Wallis
#p.value: 0.05
#Comparisons
# obs.dif critical.dif difference
#[..]
#6-9 54.0162.02688 FALSE
#6-10 69.5159.04584 FALSE
#6-11 94.5133.02196 FALSE
#7-8 18.0160.00778 FALSE
#7-9 35.0169.78370 FALSE
#7-10 50.5166.94123 FALSE
#7-11 75.5142.36796 FALSE
#8-9 17.0165.54197 FALSE
#8-10 32.5162.62538 FALSE
#8-11 57.5137.28174 FALSE
#9-10 15.5172.25281 FALSE
#9-11 40.5148.56074 FALSE
#10-1125.0145.30369 FALSE
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd@cbs.dk Priv: pda...@gmail.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-
guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Multiple Comparisons-Kruskal-Wallis-Test: kruskal{agricolae} and kruskalmc{pgirmess} don't yield the same results although they should do (?)
On Aug 3, 2012, at 18:49 , David L Carlson wrote: Generally multiple comparisons are conducted after a test for a significant difference among any of the groups. For your data kruskal.test(x[,1]~x[,2]) Kruskal-Wallis rank sum test data: x[, 1] by x[, 2] Kruskal-Wallis chi-squared = 11.0098, df = 10, p-value = 0.3568 There are no significant differences between the groups, so there is no reason to use a multiple comparison test. There's a point to that, but on the other hand, if multiple comparison methods control the familywise error rate by themselves, further guarding by a global test really just complicates things even further. I forgot that we actually had the data... A few points can be made. First, this is a really unbalanced design: table(x[,2]) 1 2 3 4 5 6 7 8 9 10 11 267 39 23 24 25 21 17 19 15 16 34 This makes both the grouping method inherently suspect since it assumes at least roughly similar group sizes. However, since it doesn't even attempt to correct for multiple tests, the point is a bit moot. If we forget the grouping technique, kruskal() gives kruskal(x[,1], x[,2], group=F) Comparison between treatments mean of the ranks Difference pvalue sig LCL UCL 1 - 10 -26.6163390 0.473942 -99.5913928 46.35871 1 - 11 -62.3553095 0.018026 * -113.9832721 -10.72735 ... 11 - 3 64.9916880 0.095914 . -11.5557888 141.53916 11 - 4 84.9056373 0.027780 *9.3153935 160.49588 11 - 5 75.6064706 0.047290 *0.9077143 150.30523 11 - 6 61.4978992 0.125302 -17.1938261 140.18962 so three comparisons are formally significant at level 0.05, but this is without correction for the multiple comparisons. This roughly amounts to multiplying all p values by 55 (actually 55, 54, 53, ... in the Holm method), which of course doesn't leave anything significant: kruskal(x[,1], x[,2], group=F, p.adj=holm) P value adjustment method: holm Comparison between treatments mean of the ranks Difference pvalue sigLCL UCL 1 - 10 -26.6163390 1.0 -150.58158 97.34890 1 - 11 -62.3553095 0.99143 -150.05751 25.34689 ... 11 - 3 64.9916880 1.0 -65.04215 195.02552 11 - 4 84.9056373 1.0 -43.50211 213.31339 11 - 5 75.6064706 1.0 -51.28688 202.49982 11 - 6 61.4978992 1.0 -72.17844 195.17424 ... which is in no way at variance with the global test or the fact that kruskalmc shows no differences to be significant. It also roughly fits results of the stock pairwise.wilcox.test: pairwise.wilcox.test(x[,1], x[,2]) Pairwise comparisons using Wilcoxon rank sum test data: x[, 1] and x[, 2] 12345678910 2 1.00 --------- 3 1.00 1.00 -------- 4 1.00 1.00 1.00 ------- 5 1.00 1.00 1.00 1.00 ------ 6 1.00 1.00 1.00 1.00 1.00 ----- 7 1.00 1.00 1.00 1.00 1.00 1.00 ---- 8 1.00 1.00 1.00 1.00 1.00 1.00 1.00 --- 9 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 -- 10 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 - 11 0.91 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 P value adjustment method: holm pairwise.wilcox.test(x[,1], x[,2], p.adj=none) Pairwise comparisons using Wilcoxon rank sum test data: x[, 1] and x[, 2] 1 2 3 4 5 6 7 8 9 10 2 0.705 - - - - - - - - - 3 0.916 0.748 - - - - - - - - 4 0.469 0.557 0.343 - - - - - - - 5 0.675 0.587 0.733 0.920 - - - - - - 6 0.985 0.727 0.805 0.608 0.869 - - - - - 7 0.226 0.349 0.311 0.153 0.187 0.362 - - - - 8 0.282 0.562 0.288 0.135 0.325 0.524 0.874 - - - 9 0.173 0.246 0.296 0.105 0.162 0.351 0.820 0.728 - - 10 0.519 0.650 0.539 0.172 0.407 0.759 0.857 0.855 0.737 - 11 0.016 0.105 0.091 0.041 0.055 0.121 0.727 0.420 0.922 0.499 P value adjustment method: none (They don't get exactly the same p-values because pairwise.wilcox.test is based on ranks recomputed separately for each pair of groups.) -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
