Re: [R] “For” calculation is so slow
I'm not sure what you are trying to prove with that example - the
loopless
versions are massively faster, no?
In some languages loops are integral part of programming habits. In R you
can many things do with whole objects without looping - vectorisation
approach. See R-Inferno from Patrick Burns - circle 3.
I don't disagree that loops are sometimes unavoidable, and I suppose
sometimes loops can be faster when the non-loop version e.g. breaks your
Hm. I am not sure what it has to do with memory budget.
memory budget, or performs tons of needless computations. But I think
avoiding for loops whenever you can is a good rule of thumb in R coding.
I constantly use loops when I create pictures of dataframe values.
This
library(ggplot2)
pdf(konc.pdf, 8,8, useDingbats=F)
for (i in columns) {
p-ggplot(df.name, aes(x=x.value, y=df.name[,i], colour=other.column))
print(p+geom_smooth(method=lm)+geom_point(aes(shape=some.factor,
size=5))+scale_y_continuous(names(df.name)[i]))
}
dev.off()
can easily create pdf file with many plots of selected columns against one
column. I know that I could use lapply, but the loop seems to me clean and
efficient even when I plot several hundreds graphs.
Regards
Petr
--
View this message in context: http://r.789695.n4.nabble.com/For-
calculation-is-so-slow-tp4630830p4630897.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] “For” calculation is so slow
Dear All,
The function I wrote can run well with the small data, but with the large
data, the function runs very very slowly. How can I correct it? Thank you
very much. My function as below:
a-c(1:240)
b-c(1:240)
l=function(a,b){
v=0
u=0
uv=0
v[1]=0
u[1]=0
uv[1]=0
for (i in 1:(length(s)-1)){
v[i]-((gx[[i]][b,(gx[[i]][a,1]+1)])-(gx[[i]][a,gx[[i]][a,1]+1]))/(gx[[i]][a,gx[[i]][a,1]+1])
u[i]-((gy[[i]][a,(gy[[i]][a,1]+1)])-(gy[[i]][b,gy[[i]][a,1]+1]))/(gy[[i]][a,gy[[i]][a,1]+1])
uv[i]-v[i]+u[i]
}
w=0
w=mean(uv)
}
kk-data.frame()
for (a in 1:240){
for (b in 1:240){
if (ab)
kk[a,b]=l(a,b)
else kk[a,b]=0
}}
max(kk)
--
View this message in context:
http://r.789695.n4.nabble.com/For-calculation-is-so-slow-tp4630830.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
On Tue, May 22, 2012 at 9:01 AM, jiangxijixzy jiangxiji...@163.com wrote:
The function I wrote can run well with the small data, but with the large
data, the function runs very very slowly. How can I correct it? Thank you
very much. My function as below:
I guess this is a classic loops vs vectorization problem.
fortune('treat')
Contrary to popular belief the speed of R's interpreter is rarely the limiting
factor to R's speed. People treating R like C is typically the limiting factor.
You have vector operations, USE THEM.
-- Byron Ellis
R-help (October 2005)
I would suggest that you read up on vectorization in R to understand
why it is often preferred to loops. Two obvious places are an Rnews
article by John Fox (if I remember well) and his Companion to Applied
Regression book.
Moreover, please sign your messages with your real name. Regards
Liviu
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
Hi
Dear All,
The function I wrote can run well with the small data, but with the
large
data, the function runs very very slowly. How can I correct it? Thank
you
Your function does not run slowly, it does not run at all.
l(10,20)
Error in l(10, 20) : object 's' not found
No s object found. I presume that also gx and gy will not be found.
And besides I am pretty sure that you probably does not need any loop at
all.
very much. My function as below:
a-c(1:240)
b-c(1:240)
l=function(a,b){
v=0
u=0
uv=0
v[1]=0
Why?
v-0
v
[1] 0
v[1]-0
v
[1] 0
So there is no difference between first v and second v
u[1]=0
uv[1]=0
for (i in 1:(length(s)-1)){
v[i]-((gx[[i]][b,(gx[[i]][a,1]+1)])-(gx[[i]][a,gx[[i]][a,1]+1]))/(gx[[i]]
[a,gx[[i]][a,1]+1])
u[i]-((gy[[i]][a,(gy[[i]][a,1]+1)])-(gy[[i]][b,gy[[i]][a,1]+1]))/(gy[[i]]
[a,gy[[i]][a,1]+1])
uv[i]-v[i]+u[i]
}
w=0
w=mean(uv)
Your function does not return anything. Here you can test it by yourself
on shortened version
l=function(a,b){
v=0
u=0
uv=0
v-10
u-20
uv-v+u
w=0
w=mean(uv)
}
Regards
Petr
}
kk-data.frame()
for (a in 1:240){
for (b in 1:240){
if (ab)
kk[a,b]=l(a,b)
else kk[a,b]=0
}}
max(kk)
--
View this message in context: http://r.789695.n4.nabble.com/For-
calculation-is-so-slow-tp4630830.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
For loops are really, really slow in R. In general, you want to avoid them like the plague. If you absolutely must insist on using them in large, computationally intense and complex code, consider implementing the relevant parts in C, say, and calling that from R. Staying within R, you can probably considerably speed up that code by storing gx and gy as a multi-dimensional arrays. (e.g. for sample data, something like rawGy = sample( 1:240, 240^2* 241, replace = T) rawGx = sample( 1:240, 240^2 *241, replace = T) gx = array(rawGx, dim = c(length(s) - 1, 240, max(rawGx)+1 ) ) gy = array(rawGy, dim = c(length(s) - 1, 240, max(rawGy)+1 ) ) ), in which case, you can easily do the computation without loops by gxa = (gx[ ,a,1]+ 1) gya =(gy[ ,a, 1] +1) uv = gx[cbind(1:(length(s) - 1) , b, gxa)] / gx[cbind(1:(length(s) - 1) , a, gxa)] - gy[cbind(1:(length(s) - 1) ,b, gya)]/gy[cbind(1:(length(s) - 1) ,a, gya)] or similar, which will be enormously faster (on my computer, there's an over 30x speed up). With a bit of thought, I'm sure you can also figure out how to let it vectorise in a, as well... Zhou -- View this message in context: http://r.789695.n4.nabble.com/For-calculation-is-so-slow-tp4630830p4630855.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
Hi
For loops are really, really slow in R. In general, you want to avoid
them
I strongly disagree. ***Proper*** use of looping is quite convenient and
reasonably fast.
Consider
system.time( {
+ a=0
+ for (i in 1:1000) {
+ a -a+i
+ }
+ a
+ })
user system elapsed
10.220.02 10.28
system.time(b-sum(as.numeric(1:1000)))
user system elapsed
0.090.010.11
identical(a,b)
[1] TRUE
It is usually implementing C habits into R code what makes looping slow.
The slowest part of a program is usually programming and it is influenced
mainly by programmer.
Regards
Petr
like the plague. If you absolutely must insist on using them in large,
computationally intense and complex code, consider implementing the
relevant
parts in C, say, and calling that from R.
Staying within R, you can probably considerably speed up that code by
storing gx and gy as a multi-dimensional arrays. (e.g. for sample data,
something like
rawGy = sample( 1:240, 240^2* 241, replace = T)
rawGx = sample( 1:240, 240^2 *241, replace = T)
gx = array(rawGx, dim = c(length(s) - 1, 240, max(rawGx)+1 ) )
gy = array(rawGy, dim = c(length(s) - 1, 240, max(rawGy)+1 ) )
), in which case, you can easily do the computation without loops by
gxa = (gx[ ,a,1]+ 1)
gya =(gy[ ,a, 1] +1)
uv = gx[cbind(1:(length(s) - 1) , b, gxa)] / gx[cbind(1:(length(s) - 1)
, a,
gxa)] - gy[cbind(1:(length(s) - 1) ,b, gya)]/gy[cbind(1:(length(s) - 1)
,a,
gya)]
or similar, which will be enormously faster (on my computer, there's an
over
30x speed up). With a bit of thought, I'm sure you can also figure out
how
to let it vectorise in a, as well...
Zhou
--
View this message in context: http://r.789695.n4.nabble.com/For-
calculation-is-so-slow-tp4630830p4630855.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
And as followup
system.time(d-1000*1001/2)
user system elapsed
0.020.000.02
identical(a,b,d)
[1] TRUE
Regards
Petr
Hi
For loops are really, really slow in R. In general, you want to avoid
them
I strongly disagree. ***Proper*** use of looping is quite convenient and
reasonably fast.
Consider
system.time( {
+ a=0
+ for (i in 1:1000) {
+ a -a+i
+ }
+ a
+ })
user system elapsed
10.220.02 10.28
system.time(b-sum(as.numeric(1:1000)))
user system elapsed
0.090.010.11
identical(a,b)
[1] TRUE
It is usually implementing C habits into R code what makes looping slow.
The slowest part of a program is usually programming and it is
influenced
mainly by programmer.
Regards
Petr
like the plague. If you absolutely must insist on using them in large,
computationally intense and complex code, consider implementing the
relevant
parts in C, say, and calling that from R.
Staying within R, you can probably considerably speed up that code by
storing gx and gy as a multi-dimensional arrays. (e.g. for sample
data,
something like
rawGy = sample( 1:240, 240^2* 241, replace = T)
rawGx = sample( 1:240, 240^2 *241, replace = T)
gx = array(rawGx, dim = c(length(s) - 1, 240, max(rawGx)+1 ) )
gy = array(rawGy, dim = c(length(s) - 1, 240, max(rawGy)+1 ) )
), in which case, you can easily do the computation without loops by
gxa = (gx[ ,a,1]+ 1)
gya =(gy[ ,a, 1] +1)
uv = gx[cbind(1:(length(s) - 1) , b, gxa)] / gx[cbind(1:(length(s) -
1)
, a,
gxa)] - gy[cbind(1:(length(s) - 1) ,b, gya)]/gy[cbind(1:(length(s) -
1)
,a,
gya)]
or similar, which will be enormously faster (on my computer, there's
an
over
30x speed up). With a bit of thought, I'm sure you can also figure out
how
to let it vectorise in a, as well...
Zhou
--
View this message in context: http://r.789695.n4.nabble.com/For-
calculation-is-so-slow-tp4630830p4630855.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
Thanks, I will take that factor of 100 anytime rather than keep some syntactic
familiarity from other languages.
Not to say I don't ever use loops, but I always try to vectorize the inner
loop, if not the inner two loops.
---
Jeff NewmillerThe . . Go Live...
DCN:jdnew...@dcn.davis.ca.usBasics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/BatteriesO.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---
Sent from my phone. Please excuse my brevity.
Petr PIKAL petr.pi...@precheza.cz wrote:
Hi
For loops are really, really slow in R. In general, you want to avoid
them
I strongly disagree. ***Proper*** use of looping is quite convenient
and
reasonably fast.
Consider
system.time( {
+ a=0
+ for (i in 1:1000) {
+ a -a+i
+ }
+ a
+ })
user system elapsed
10.220.02 10.28
system.time(b-sum(as.numeric(1:1000)))
user system elapsed
0.090.010.11
identical(a,b)
[1] TRUE
It is usually implementing C habits into R code what makes looping
slow.
The slowest part of a program is usually programming and it is
influenced
mainly by programmer.
Regards
Petr
like the plague. If you absolutely must insist on using them in
large,
computationally intense and complex code, consider implementing the
relevant
parts in C, say, and calling that from R.
Staying within R, you can probably considerably speed up that code by
storing gx and gy as a multi-dimensional arrays. (e.g. for sample
data,
something like
rawGy = sample( 1:240, 240^2* 241, replace = T)
rawGx = sample( 1:240, 240^2 *241, replace = T)
gx = array(rawGx, dim = c(length(s) - 1, 240, max(rawGx)+1 ) )
gy = array(rawGy, dim = c(length(s) - 1, 240, max(rawGy)+1 ) )
), in which case, you can easily do the computation without loops by
gxa = (gx[ ,a,1]+ 1)
gya =(gy[ ,a, 1] +1)
uv = gx[cbind(1:(length(s) - 1) , b, gxa)] / gx[cbind(1:(length(s) -
1)
, a,
gxa)] - gy[cbind(1:(length(s) - 1) ,b, gya)]/gy[cbind(1:(length(s) -
1)
,a,
gya)]
or similar, which will be enormously faster (on my computer, there's
an
over
30x speed up). With a bit of thought, I'm sure you can also figure
out
how
to let it vectorise in a, as well...
Zhou
--
View this message in context: http://r.789695.n4.nabble.com/For-
calculation-is-so-slow-tp4630830p4630855.html
Sent from the R help mailing list archive at Nabble.com.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] “For” calculation is so slow
I'm not sure what you are trying to prove with that example - the loopless versions are massively faster, no? I don't disagree that loops are sometimes unavoidable, and I suppose sometimes loops can be faster when the non-loop version e.g. breaks your memory budget, or performs tons of needless computations. But I think avoiding for loops whenever you can is a good rule of thumb in R coding. -- View this message in context: http://r.789695.n4.nabble.com/For-calculation-is-so-slow-tp4630830p4630897.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
