Re: [R] do faster ANOVAS
Thank you, with the matrix for the responses (here my 101 timepoints), it takes
less than 30 minutes for 1000 pemutations, whereas before it takes 2h30!
Best regards,
Mélissa
> Message du 10/06/10 18:52
> De : "Douglas Bates"
> A : "melissa"
> Copie à : r-help@r-project.org
> Objet : Re: [R] do faster ANOVAS
>
>
> The lm and aov functions can take a matrix response allowing you to
> fit all of the responses for a single attribute simultaneously.
>
>
> On Thu, Jun 10, 2010 at 8:47 AM, melissa wrote:
> > Dear all R users,
> > I want to realize 800 000 ANOVAS and to store Sum of Squares of the
> > effects. Here is an extract of my table data
> > Product attribute subject rep t1 t2 t3 ⦠t101
> > P1 A1 S1 R1 1 0 0 ⦠1
> > I want to realize 1 ANOVA per timepoint and per attribute, there are 101
> > timepoints and 8 attributes so I want to realize 808 ANOVAS. This will be
> > an ANOVA with two factors :
> > Here is one example:
> > Aov(t1~Subject*Product,data[data$attribute==âA1â,])
> > I want to store for each ANOVA SSprod,SSsujet,SSerreur,SSinter and SStotal.
> > In fact I want the result in several matrices:
> > Ssprod matrice:
> > T1 t2 t3 t4 ⦠t101
> > A1 ssprod(A1,T1)
> > A2
> > A3
> > â¦
> > A8
> > So I would like a matrice like that for ssprod, ssujet,sserreur,ssinter and
> > sstotal.
> > And this is for one permutation, and I want to do 1000 permutations
> > Here is my code:
> > SSmatrixglobal<-function(k){
> >
> > daten.temp<-data
> > daten.temp$product=permutations[[k]]
> > listmat<-apply(daten.temp[,5:105],2,function(x,y){
> > tab2<-as.data.frame(cbind(x,y))
> > tab.class<-by(tab2[,1:3],tab2[,4],function(x){
> > f <- formula(paste(names(x)[1],"~",names(x)[2],"*",names(x)[3],sep=""))
> > anovas <- aov(f, data=x)
> > anovas$call$formula <-f
> > s1 <- summary(anovas)
> > qa <- s1[[1]][,2]
> > return(qa)
> > })
> > return(tab.class)
> > },y=daten.temp[,1:3]
> > )
> > ar <-
> > array(unlist(listmat),dim=c(length(listmat[[1]][[1]]),length(listmat[[1]]),length(listmat)))
> > l=lapply(1:4,function(i) ar[i,,])
> > sssujet=l[[1]]
> > ssprod=l[[2]]
> > ssinter=l[[3]]
> > sserreur=l[[4]]
> > ss=rbind(sssujet,ssprod,ssinter,sserreur,sstotal)
> > ss=as.data.frame(ss)
> > sqlSave(channel,ss,"SS1000",append=T)
> > rm(ss,numperm,daten.temp)
> > }
> >
> > system.time(por <- lapply(c(1:1000), SSmatrixglobal))
> >
> > But it takes time about 90seconds for a permutation so *1000, how can I do
> > in order to do faster ANOVAS?
> >
> > Many thanks
> > Best regards
> > Mélissa
> >
> > PS: I think that I can gain a lot of time in the aov function but I don't
> > know how to do
> > [[alternative HTML version deleted]]
> >
> >
> > __
> > R-help@r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> >
>
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] do faster ANOVAS
The lm and aov functions can take a matrix response allowing you to
fit all of the responses for a single attribute simultaneously.
On Thu, Jun 10, 2010 at 8:47 AM, melissa wrote:
> Dear all R users,
> I want to realize 800 000 ANOVAS and to store Sum of Squares of the effects.
> Here is an extract of my table data
> Product attribute subject rep t1 t2 t3 … t101
> P1 A1 S1 R1 1 0 0 … 1
> I want to realize 1 ANOVA per timepoint and per attribute, there are 101
> timepoints and 8 attributes so I want to realize 808 ANOVAS. This will be an
> ANOVA with two factors :
> Here is one example:
> Aov(t1~Subject*Product,data[data$attribute==”A1”,])
> I want to store for each ANOVA SSprod,SSsujet,SSerreur,SSinter and SStotal.
> In fact I want the result in several matrices:
> Ssprod matrice:
> T1 t2 t3 t4 … t101
> A1 ssprod(A1,T1)
> A2
> A3
> …
> A8
> So I would like a matrice like that for ssprod, ssujet,sserreur,ssinter and
> sstotal.
> And this is for one permutation, and I want to do 1000 permutations
> Here is my code:
> SSmatrixglobal<-function(k){
>
> daten.temp<-data
> daten.temp$product=permutations[[k]]
> listmat<-apply(daten.temp[,5:105],2,function(x,y){
> tab2<-as.data.frame(cbind(x,y))
> tab.class<-by(tab2[,1:3],tab2[,4],function(x){
> f <- formula(paste(names(x)[1],"~",names(x)[2],"*",names(x)[3],sep=""))
> anovas <- aov(f, data=x)
> anovas$call$formula <-f
> s1 <- summary(anovas)
> qa <- s1[[1]][,2]
> return(qa)
> })
> return(tab.class)
> },y=daten.temp[,1:3]
> )
> ar <-
> array(unlist(listmat),dim=c(length(listmat[[1]][[1]]),length(listmat[[1]]),length(listmat)))
> l=lapply(1:4,function(i) ar[i,,])
> sssujet=l[[1]]
> ssprod=l[[2]]
> ssinter=l[[3]]
> sserreur=l[[4]]
> ss=rbind(sssujet,ssprod,ssinter,sserreur,sstotal)
> ss=as.data.frame(ss)
> sqlSave(channel,ss,"SS1000",append=T)
> rm(ss,numperm,daten.temp)
> }
>
> system.time(por <- lapply(c(1:1000), SSmatrixglobal))
>
> But it takes time about 90seconds for a permutation so *1000, how can I do in
> order to do faster ANOVAS?
>
> Many thanks
> Best regards
> Mélissa
>
> PS: I think that I can gain a lot of time in the aov function but I don't
> know how to do
> [[alternative HTML version deleted]]
>
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] do faster ANOVAS
Dear all R users,
I want to realize 800 000 ANOVAS and to store Sum of Squares of the effects.
Here is an extract of my table data
Product attribute subject rep t1 t2 t3 ⦠t101
P1 A1 S1 R1 1 0 0 ⦠1
I want to realize 1 ANOVA per timepoint and per attribute, there are 101
timepoints and 8 attributes so I want to realize 808 ANOVAS. This will be an
ANOVA with two factors :
Here is one example:
Aov(t1~Subject*Product,data[data$attribute==âA1â,])
I want to store for each ANOVA SSprod,SSsujet,SSerreur,SSinter and SStotal.
In fact I want the result in several matrices:
Ssprod matrice:
T1 t2 t3 t4 ⦠t101
A1 ssprod(A1,T1)
A2
A3
â¦
A8
So I would like a matrice like that for ssprod, ssujet,sserreur,ssinter and
sstotal.
And this is for one permutation, and I want to do 1000 permutations
Here is my code:
SSmatrixglobal<-function(k){
daten.temp<-data
daten.temp$product=permutations[[k]]
listmat<-apply(daten.temp[,5:105],2,function(x,y){
tab2<-as.data.frame(cbind(x,y))
tab.class<-by(tab2[,1:3],tab2[,4],function(x){
f <- formula(paste(names(x)[1],"~",names(x)[2],"*",names(x)[3],sep=""))
anovas <- aov(f, data=x)
anovas$call$formula <-f
s1 <- summary(anovas)
qa <- s1[[1]][,2]
return(qa)
})
return(tab.class)
},y=daten.temp[,1:3]
)
ar <-
array(unlist(listmat),dim=c(length(listmat[[1]][[1]]),length(listmat[[1]]),length(listmat)))
l=lapply(1:4,function(i) ar[i,,])
sssujet=l[[1]]
ssprod=l[[2]]
ssinter=l[[3]]
sserreur=l[[4]]
ss=rbind(sssujet,ssprod,ssinter,sserreur,sstotal)
ss=as.data.frame(ss)
sqlSave(channel,ss,"SS1000",append=T)
rm(ss,numperm,daten.temp)
}
system.time(por <- lapply(c(1:1000), SSmatrixglobal))
But it takes time about 90seconds for a permutation so *1000, how can I do in
order to do faster ANOVAS?
Many thanks
Best regards
Mélissa
PS: I think that I can gain a lot of time in the aov function but I don't know
how to do
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
