Yesterday I posted the following question (my apologies for not putting a subject line):
=================question====================== Hello -- I would like to know of a more efficient way of writing the following piece of code. Thanks. options(stringsAsFactors=FALSE) orig <- c(rep('11111111',100000),rep('22222222',200000),rep('33333333' ,300000),rep('44444444',400000)) orig.unique <- unique(orig) system.time(df <- as.data.frame(sapply(orig.unique, function(x) ifelse(orig==x, 1, 0)))) ============================================ I received a response via e-mail which was **extremely** useful. =================answer====================== Using sapply instead of lapply here is a waste. sapply() calls lapply(), which returns a list that sapply() turns into a list by making each list element a column of the matrix. data.frame(matrix) then makes a list from the columns of the matrix. The one thing that sapply gives you and lapply doesn't is column names. If you attach names to orig.unique then lapply's output will have them. Also ifelse(orig==x,1,0) slower than the equivalent as.numeric(orig==x). I wrote functions g0 (containing your code), g1 (using lapply), and g2 (ifelse->as.numeric). I parameterized them by the number of '1111111' elements and they each return the data.frame created and the time it took to do it: > g0 function(n = 1e+05) { orig <- c(rep("11111111", n), rep("22222222", 2*n), rep("33333333", 3*n), rep("44444444", 4*n)) orig.unique <- unique(orig) time <- system.time(df <- as.data.frame(sapply(orig.unique, function(x) ifelse(orig == x, 1, 0)))) list(time = time, df = df) } > g1 function (n = 1e+05) { orig <- c(rep("11111111", n), rep("22222222", 2*n), rep("33333333", 3*n), rep("44444444", 4*n)) orig.unique <- unique(orig) names(orig.unique) <- orig.unique time <- system.time(df <- data.frame(check.names=FALSE, lapply(orig.unique, function(x) ifelse(orig == x, 1, 0)))) list(time = time, df = df) } > g2 function (n = 1e+05) { orig <- c(rep("11111111", n), rep("22222222", 2*n), rep("33333333", 3*n), rep("44444444", 4*n)) orig.unique <- unique(orig) names(orig.unique) <- orig.unique time <- system.time(df <- data.frame(check.names=FALSE, lapply(orig.unique, function(x) as.numeric(orig == x)))) list(time = time, df = df) } For n=10^5 the times were > g0(1e5)$time user system elapsed 20.65 0.41 20.64 > g1(1e5)$time user system elapsed 2.35 0.05 2.36 > g2(1e5)$time user system elapsed 0.73 0.10 0.77 and the data.frames each produced were identical. Another approach is to use outer() to make a matrix that gets passed to data.frame(). It seems slightly slower than g2, but small changes might make it faster. > g3 function (n = 1e+05) { orig <- c(rep("11111111", n), rep("22222222", 2 * n), rep("33333333", 3 * n), rep("44444444", 4 * n)) orig.unique <- unique(orig) names(orig.unique) <- orig.unique time <- system.time(df <- data.frame(check.names=FALSE, outer(orig, orig.unique, function(x, y) as.numeric(x==y)))) list(time = time, df = df) } > g3(1e5)$time user system elapsed 1.02 0.00 0.97 When you want to optimize code it is often handy to write functions like this to do the timing for various problem sizes. You can quickly experiment with small versions of the problem to make sure the results are correct and the time looks reasonable and later see if the times scale up as hoped to your desired problem size. ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.