[Rd] memory problem read.table v array (PR#9526)

2007-02-22 Thread wlangdon
Full_Name: bill langdon
Version: 2.4.1
OS: ubuntu
Submission from: (NULL) (155.245.58.159)


#WBL 22 Feb 2007 ubuntu 
R.version
#platform   i486-pc-linux-gnu
#arch   i486
#os linux-gnu
#system i486, linux-gnu
#status
#major  2
#minor  4.1
#year   2006
#month  12
#day18
#svn rev40228
#language   R
#version.string R version 2.4.1 (2006-12-18)

#if matrix "a" is created by array "vals" is created ok

#if matrix "a" is created by read.table,
#peak resource use (CPU, memory) by "array()" is excessive

#a = array(0,dim=c(409600,1));#ok
#a = read.table("big.txt",header=FALSE);  #all memory used 
#a = read.table("639x639.txt",header=FALSE);  #all memory used 
#a = read.table("4096.txt",header=FALSE); #all memory used 
#a = read.table("4096nocomment.txt",header=FALSE);#all memory used 
#a = read.table("tiny.txt",header=FALSE); #ten lines ok
#a = read.table("1000.txt",header=FALSE); #all memory used 
a = read.table("639.txt",header=FALSE);   #uses 1.6047029GB
dim(a)
dd = 639;
vals = array(a,dim=c(dd,dd));

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel


Re: [Rd] memory problem read.table v array (PR#9526)

2007-02-22 Thread Prof Brian Ripley
There is nothing to reproduce here: we do not have 639.txt.

But note that read.table returns a data frame, and ?array has

 data: a vector (including a list) giving data to fill the array.

so I do wonder if this is what you intended: you seem to have tried to 
create an array list with 639*639 elements.  It is certainly not the same 
sort of object as array(0,dim=c(409600,1)).

Had 639.txt contained 639 rows of reals, your 'a' would be about
639*639*639*8 bytes, beyond the address space of your machine.

as.matrix(read.table("639.txt", header=FALSE)) might have been what you 
are looking for.


On Thu, 22 Feb 2007, [EMAIL PROTECTED] wrote:

> Full_Name: bill langdon
> Version: 2.4.1
> OS: ubuntu
> Submission from: (NULL) (155.245.58.159)
>
>
> #WBL 22 Feb 2007 ubuntu
> R.version
> #platform   i486-pc-linux-gnu
> #arch   i486
> #os linux-gnu
> #system i486, linux-gnu
> #status
> #major  2
> #minor  4.1
> #year   2006
> #month  12
> #day18
> #svn rev40228
> #language   R
> #version.string R version 2.4.1 (2006-12-18)
>
> #if matrix "a" is created by array "vals" is created ok
>
> #if matrix "a" is created by read.table,
> #peak resource use (CPU, memory) by "array()" is excessive
>
> #a = array(0,dim=c(409600,1));#ok
> #a = read.table("big.txt",header=FALSE);  #all memory used
> #a = read.table("639x639.txt",header=FALSE);  #all memory used
> #a = read.table("4096.txt",header=FALSE); #all memory used
> #a = read.table("4096nocomment.txt",header=FALSE);#all memory used
> #a = read.table("tiny.txt",header=FALSE); #ten lines ok
> #a = read.table("1000.txt",header=FALSE); #all memory used
> a = read.table("639.txt",header=FALSE);   #uses 1.6047029GB
> dim(a)
> dd = 639;
> vals = array(a,dim=c(dd,dd));


-- 
Brian D. Ripley,  [EMAIL PROTECTED]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel