[R] Re : growing a list sequentially -- memory management

[justin bem]

you may try something like  

myres <-tapply(1:n, as.factor(1:n) gen)

but n in very large in your case. 

you may create an external database and fill it

connect to you database

library(RODC)
con<-odbcConnect(...)

st<-seq(1,n, by=h)  
# since n=157347, h can be set to 1573 to have around that 100 loops

for ( i in 1: length(sh)){
    index<-st(i):st(i+1)
    myres(tapply(index, factor(index),gen
    sqlwrite(myres,connexion,append=(i>1))
}

It will be more faster .
 Justin BEM
BP 1917 Yaoundé
Tél (237) 99597295
(237) 22040246 




________________________________
De : Patrick Burns <[EMAIL PROTECTED]>
À : erwann rogard <[EMAIL PROTECTED]>
Cc : r-help@r-project.org
Envoyé le : Vendredi, 14 Novembre 2008, 12h37mn 35s
Objet : Re: [R] growing a list sequentially -- memory management

The recommended technique is to create objects
as their final size and then subscript into them
with your data.

My intuition (which is often brutally wrong) tells
me that your case should not be overly traumatic.
So I'm suspicious that you are fragmenting memory
in other ways as well.

Patrick Burns
[EMAIL PROTECTED]
+44 (0)20 8525 0696
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")


erwann rogard wrote:
> hello,
>
> i have something like:
>
> out<-list()
>
> for(i in 1:n){
>  data<-gen(...) #fixed size data
>  out[[i]]<- fun(data)
> }
>
>  
>> object.size(out[[1]])
>>    
> 6824
>
> In principle 1 GB should allow
>
> n = 1024^3/6824 = 157347?
>
> i have about 2GB are not taken by other processes. however, I can see the
> memory shrinking quite rapidly on my system monitor and have to stop the
> simulation after only n=300. why such a discrepancy? any remedy?
>
> x86_64-pc-linux/RKWard/R2.8.0/ 4GB
>
> thanks.
>
>     [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
>

______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



      
        [[alternative HTML version deleted]]


______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.