-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi,
I'm doing some basic operations on large sparse matrices, for example getting a row. it takes close to 30 seconds on a 3Ghz machine, and shots the memory usage up to the sky. I suspect there are dense intermediary steps (which, if true would defeat the purpose of trying to use sparse representaitons). As much as I try understanding the hierarchy of Matrix classes, it's a mystery to me. Is subsetting sparse matrices memory-intensive? Does it have to do with features of the language, such as pass-by-value bu default? Or am I doing something inneficent without knowing? (Note: example below would only work with 64bit R and lots of memory; reduce size of matrix 2-3 orders of magnitude for 32-bit R) ## libraries library(Matrix) rSpMatrix <- function(nrow, ncol, nnz, rand.x = function(n) round(rnorm(nnz), 2)) { ## Purpose: random sparse matrix ## ---------------------------------------------------------------------- ## Arguments: (nrow,ncol): dimension ## nnz : number of non-zero entries ## rand.x: random number generator for 'x' slot ## ---------------------------------------------------------------------- ## Author: Martin Maechler, Date: 14.-16. May 2007 stopifnot((nnz <- as.integer(nnz)) >= 0, nrow >= 0, ncol >= 0, nnz <= nrow * ncol) spMatrix(nrow, ncol, i = sample(nrow, nnz, replace = TRUE), j = sample(ncol, nnz, replace = TRUE), x = rand.x(nnz)) } ## (original ir a term x doc matrix of the entire wikipedia) mm <- rSpMatrix(793251, 1027355, nnz = 205746204) # make it column based: mm <- as(mm, "CsparseMatrix") a=mm[1,,drop=F]#this takes close to 30 seconds on a 3Ghz machine Thanks, - -Jose - -- === I'm learning a new keyboard layout. I type slowly. Please excuse typos and slower-that-usual responses. Thanks for your patience=== Jose Quesada, PhD. Max Planck Institute, Center for Adaptive Behavior and Cognition -ABC-, Lentzeallee 94, office 224, 14195 Berlin http://www.josequesada.name/ http://twitter.com/Quesada -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) iQIcBAEBAgAGBQJK43ZUAAoJEGMobUVGH+HKsZEP/1+R/x2Qv9Kc1LYNP01Hvirz n8ZVz/fGm+uCt/n4F+cXdXntZ3yoUVZcPy7LwMIo5W4x/saS5cMjBg6Z8hsDvXEQ 2KiiPYJ46C6hEOGJwrpYfOZwm7t5aIMMhaP182P19ziKhpp/Gn4joTnXUzigNHzt vIPBOBgdZdPvC0h9ByvtCmMSX3roQPv1nMIojrPC0+EzfIVVNIcsfQYYacOyhcuM /RR97aOzHvcdCln7FbrIT2I0SeeVH3scFGN7q7KFi6Sy+KZmQsv7FxfnUAvf5R+l DAcm0ekeThksSmJE/Td1220ZguaORjyMHwAKfJH+wiXei24+N+Xf22469g2gCEWb hiNUzhLHXOSY6mKZ80LKMbhUM4JhKs7K1HImwQmVDa/1UU1WwsjrzZ2fHHqnsjQi Uysrttu1nbTT5Yvn9CT8gedM7A78sIddjpi1PavbRVJl7/eDN5PGgwilQ70DNetJ PY8QvLGlA4GGtdvTzxFVP2VK0QgfxmRedrEwqxR1AlIZRn9iK8jrZCXTGLeXe5LX BX7PQXs11ZfuW/kzDBstqobCrxSERRb/HP5BlY+mKmZ0SieoVkpLeJ7PdeHu+r31 8bsZPkeHaO5MR1KrG155JOK1vgmHPpfSiq0lNUT99hncoFKBvweDnE3Etx+tBra7 h9lxbDeHl6xkbCJj/LDb =ZVWO -----END PGP SIGNATURE----- ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.