Re: [R] Avoiding for Loop for moving average
The 'filter' function should be able
to do what you want efficiently.
On 02/09/2011 18:06, Noah Silverman wrote:
Joshua,
Thanks for the tip.
I need to "roll my own" code on this. But perhaps I can borrow some code from
the package you mentioned.
Is the package just performing the loop, but in a faster language?
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
On Sep 2, 2011, at 9:58 AM, Joshua Ulrich wrote:
On Fri, Sep 2, 2011 at 11:47 AM, R. Michael Weylandt
wrote:
Have you looked at SMA/EMA from the TTR package? That's a pretty quick
implementation.
runmean from caTools is even better for the SMA but I don't think there's an
easy way to turn that into an EWMA.
SMA still calls Fortran code, so that's why it's slower than
caTools::runmean. I've moved the EMA code to C, so it's about as fast
as it can be.
Noah, use EMA's ratio argument to replicate your for loop.
Hope this helps,
Michael Weylandt
Best,
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
On Fri, Sep 2, 2011 at 12:43 PM, Noah Silvermanwrote:
Hello,
I need to calculate a moving average and an exponentially weighted moving
average over a fairly large data set (500K rows).
Doing this in a for loop works nicely, but is slow.
ewma<- data$col[1]
N<- dim(data)[1]
for(i in 2:N){
data$ewma<- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
}
Since the moving average "accumulates" as we move through the data, I'm not
sure on the best/fastest way to do this.
Does anyone have any suggestions on how to avoid a loop doing this?
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Patrick Burns
pbu...@pburns.seanet.com
twitter: @portfolioprobe
http://www.portfolioprobe.com/blog
http://www.burns-stat.com
(home of 'Some hints for the R beginner'
and 'The R Inferno')
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
Thanks Joshua,
I really like the example given in the blog post that Abhijit pointed me to.
Doing it in C++ using the Inline seems like an easy way to get a massive
improvement in speed without the hassle of writing a package.
I'm working on coding that now.
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
On Sep 2, 2011, at 10:32 AM, Joshua Ulrich wrote:
> On Fri, Sep 2, 2011 at 12:06 PM, Noah Silverman
> wrote:
>> Joshua,
>>
>> Thanks for the tip.
>>
>> I need to "roll my own" code on this. But perhaps I can borrow some code
>> from the package you mentioned.
>>
>> Is the package just performing the loop, but in a faster language?
>>
> As I said, the function is in C. You could also use the compiler
> package to compile your pure R function for a 3-4x speedup.
>
> Best,
> --
> Joshua Ulrich | FOSS Trading: www.fosstrading.com
>
>
>>
>> --
>> Noah Silverman
>> UCLA Department of Statistics
>> 8117 Math Sciences Building #8208
>> Los Angeles, CA 90095
>>
>> On Sep 2, 2011, at 9:58 AM, Joshua Ulrich wrote:
>>
>>> On Fri, Sep 2, 2011 at 11:47 AM, R. Michael Weylandt
>>> wrote:
Have you looked at SMA/EMA from the TTR package? That's a pretty quick
implementation.
runmean from caTools is even better for the SMA but I don't think there's
an
easy way to turn that into an EWMA.
>>> SMA still calls Fortran code, so that's why it's slower than
>>> caTools::runmean. I've moved the EMA code to C, so it's about as fast
>>> as it can be.
>>>
>>> Noah, use EMA's ratio argument to replicate your for loop.
>>>
Hope this helps,
Michael Weylandt
>>>
>>> Best,
>>> --
>>> Joshua Ulrich | FOSS Trading: www.fosstrading.com
>>>
>>>
>>>
On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman
wrote:
> Hello,
>
> I need to calculate a moving average and an exponentially weighted moving
> average over a fairly large data set (500K rows).
>
> Doing this in a for loop works nicely, but is slow.
>
> ewma <- data$col[1]
> N <- dim(data)[1]
> for(i in 2:N){
>data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
> }
>
>
> Since the moving average "accumulates" as we move through the data, I'm
> not
> sure on the best/fastest way to do this.
>
> Does anyone have any suggestions on how to avoid a loop doing this?
>
>
>
>
> --
> Noah Silverman
> UCLA Department of Statistics
> 8117 Math Sciences Building #8208
> Los Angeles, CA 90095
>
>
>[[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>[[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
On Fri, Sep 2, 2011 at 12:06 PM, Noah Silverman wrote:
> Joshua,
>
> Thanks for the tip.
>
> I need to "roll my own" code on this. But perhaps I can borrow some code
> from the package you mentioned.
>
> Is the package just performing the loop, but in a faster language?
>
As I said, the function is in C. You could also use the compiler
package to compile your pure R function for a 3-4x speedup.
Best,
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
>
> --
> Noah Silverman
> UCLA Department of Statistics
> 8117 Math Sciences Building #8208
> Los Angeles, CA 90095
>
> On Sep 2, 2011, at 9:58 AM, Joshua Ulrich wrote:
>
>> On Fri, Sep 2, 2011 at 11:47 AM, R. Michael Weylandt
>> wrote:
>>> Have you looked at SMA/EMA from the TTR package? That's a pretty quick
>>> implementation.
>>>
>>> runmean from caTools is even better for the SMA but I don't think there's an
>>> easy way to turn that into an EWMA.
>>>
>> SMA still calls Fortran code, so that's why it's slower than
>> caTools::runmean. I've moved the EMA code to C, so it's about as fast
>> as it can be.
>>
>> Noah, use EMA's ratio argument to replicate your for loop.
>>
>>> Hope this helps,
>>>
>>> Michael Weylandt
>>>
>>
>> Best,
>> --
>> Joshua Ulrich | FOSS Trading: www.fosstrading.com
>>
>>
>>
>>> On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman
>>> wrote:
>>>
Hello,
I need to calculate a moving average and an exponentially weighted moving
average over a fairly large data set (500K rows).
Doing this in a for loop works nicely, but is slow.
ewma <- data$col[1]
N <- dim(data)[1]
for(i in 2:N){
data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
}
Since the moving average "accumulates" as we move through the data, I'm not
sure on the best/fastest way to do this.
Does anyone have any suggestions on how to avoid a loop doing this?
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
>>>
>>> [[alternative HTML version deleted]]
>>>
>>> __
>>> R-help@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
There is a recent blog post by Dirk Eddelbeutel on how to do something
similar using his Rcpp package and C++, with massive time improvements.
http://dirk.eddelbuettel.com/blog/
On 9/2/2011 12:43 PM, Noah Silverman wrote:
> Hello,
>
> I need to calculate a moving average and an exponentially weighted moving
> average over a fairly large data set (500K rows).
>
> Doing this in a for loop works nicely, but is slow.
>
> ewma<- data$col[1]
> N<- dim(data)[1]
> for(i in 2:N){
> data$ewma<- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
> }
>
>
> Since the moving average "accumulates" as we move through the data, I'm not
> sure on the best/fastest way to do this.
>
> Does anyone have any suggestions on how to avoid a loop doing this?
>
>
>
>
> --
> Noah Silverman
> UCLA Department of Statistics
> 8117 Math Sciences Building #8208
> Los Angeles, CA 90095
>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
Joshua,
Thanks for the tip.
I need to "roll my own" code on this. But perhaps I can borrow some code from
the package you mentioned.
Is the package just performing the loop, but in a faster language?
--
Noah Silverman
UCLA Department of Statistics
8117 Math Sciences Building #8208
Los Angeles, CA 90095
On Sep 2, 2011, at 9:58 AM, Joshua Ulrich wrote:
> On Fri, Sep 2, 2011 at 11:47 AM, R. Michael Weylandt
> wrote:
>> Have you looked at SMA/EMA from the TTR package? That's a pretty quick
>> implementation.
>>
>> runmean from caTools is even better for the SMA but I don't think there's an
>> easy way to turn that into an EWMA.
>>
> SMA still calls Fortran code, so that's why it's slower than
> caTools::runmean. I've moved the EMA code to C, so it's about as fast
> as it can be.
>
> Noah, use EMA's ratio argument to replicate your for loop.
>
>> Hope this helps,
>>
>> Michael Weylandt
>>
>
> Best,
> --
> Joshua Ulrich | FOSS Trading: www.fosstrading.com
>
>
>
>> On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman
>> wrote:
>>
>>> Hello,
>>>
>>> I need to calculate a moving average and an exponentially weighted moving
>>> average over a fairly large data set (500K rows).
>>>
>>> Doing this in a for loop works nicely, but is slow.
>>>
>>> ewma <- data$col[1]
>>> N <- dim(data)[1]
>>> for(i in 2:N){
>>>data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
>>> }
>>>
>>>
>>> Since the moving average "accumulates" as we move through the data, I'm not
>>> sure on the best/fastest way to do this.
>>>
>>> Does anyone have any suggestions on how to avoid a loop doing this?
>>>
>>>
>>>
>>>
>>> --
>>> Noah Silverman
>>> UCLA Department of Statistics
>>> 8117 Math Sciences Building #8208
>>> Los Angeles, CA 90095
>>>
>>>
>>>[[alternative HTML version deleted]]
>>>
>>> __
>>> R-help@r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>[[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
On Fri, Sep 2, 2011 at 11:47 AM, R. Michael Weylandt
wrote:
> Have you looked at SMA/EMA from the TTR package? That's a pretty quick
> implementation.
>
> runmean from caTools is even better for the SMA but I don't think there's an
> easy way to turn that into an EWMA.
>
SMA still calls Fortran code, so that's why it's slower than
caTools::runmean. I've moved the EMA code to C, so it's about as fast
as it can be.
Noah, use EMA's ratio argument to replicate your for loop.
> Hope this helps,
>
> Michael Weylandt
>
Best,
--
Joshua Ulrich | FOSS Trading: www.fosstrading.com
> On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman wrote:
>
>> Hello,
>>
>> I need to calculate a moving average and an exponentially weighted moving
>> average over a fairly large data set (500K rows).
>>
>> Doing this in a for loop works nicely, but is slow.
>>
>> ewma <- data$col[1]
>> N <- dim(data)[1]
>> for(i in 2:N){
>> data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
>> }
>>
>>
>> Since the moving average "accumulates" as we move through the data, I'm not
>> sure on the best/fastest way to do this.
>>
>> Does anyone have any suggestions on how to avoid a loop doing this?
>>
>>
>>
>>
>> --
>> Noah Silverman
>> UCLA Department of Statistics
>> 8117 Math Sciences Building #8208
>> Los Angeles, CA 90095
>>
>>
>> [[alternative HTML version deleted]]
>>
>> __
>> R-help@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Avoiding for Loop for moving average
Have you looked at SMA/EMA from the TTR package? That's a pretty quick
implementation.
runmean from caTools is even better for the SMA but I don't think there's an
easy way to turn that into an EWMA.
Hope this helps,
Michael Weylandt
On Fri, Sep 2, 2011 at 12:43 PM, Noah Silverman wrote:
> Hello,
>
> I need to calculate a moving average and an exponentially weighted moving
> average over a fairly large data set (500K rows).
>
> Doing this in a for loop works nicely, but is slow.
>
> ewma <- data$col[1]
> N <- dim(data)[1]
> for(i in 2:N){
>data$ewma <- alpha * data$ewma[i-1] + (1-alpha) * data$value[i]
> }
>
>
> Since the moving average "accumulates" as we move through the data, I'm not
> sure on the best/fastest way to do this.
>
> Does anyone have any suggestions on how to avoid a loop doing this?
>
>
>
>
> --
> Noah Silverman
> UCLA Department of Statistics
> 8117 Math Sciences Building #8208
> Los Angeles, CA 90095
>
>
>[[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
