Hi:
The key phrase in your mail was 'data.table'. Given the size of the object,
it is very likely to be a data.table, which (oddly enough) comes from
package
data.table. It is designed to quickly process information in very large
datasets. 3M rows is an 'average' sized data.table :)
Your request
I am not exactly sure how your filtering code is working, but take a look at
?na.omit
You will probably need a few additional steps if you want to remove
all rows related to a particular id. Also look at ?subset which is a
good general way to subset your data.
Josh
On Thu, Jun 3, 2010 at 11:
Hey Jeff,
I have a few ideas. Each has some different requirements, and to help
you choose, I bench marked them.
###START###
##Basic data
> test <- data.frame(totret=rnorm(10^7), id=rep(1:10^4, each=10^3),
> time=rep(c(1, rep(0, 999)), 10^4))
##Option 1: probably the most general, but also t
Hey Josh,
Thanks for the quick response!
I guess I have to switch from the Java mindset to the matrix/vector mindset
of R.
Your code worked very well, but I just have one problem:
Essentially I have a time series of stock A, followed by a time series of
stock B, etc.
So there are break points
Hey Josh,
Thanks for the quick response!
I guess I have to switch from the Java mindset to the matrix/vector mindset
of R.
Your code worked very well, but I just have one problem:
Essentially I have a time series of stock A, followed by a time series of
stock B, etc.
So there are break points
Hello Jeff,
Try this:
test <- data.frame(totret=rnorm(10^7)) #create some sample data
test[-1,"dailyreturn"] <- test[-1,"totret"]/test[-nrow(test),"totret"]
The general idea is to take the column "totret" excluding the first 1,
dividided by "totret" exluding the last row. This gives in effect t
Hello Everyone,
I just started a new job & it requires heavy use of R to analyze datasets.
I have a data.table that looks like this. It is sorted by ID & Date, there
are about 150 different IDs & the dataset spans 3 million rows. The main
columns of concern are ID, date, and totret. What I need
7 matches
Mail list logo
