Thomas Lumley wrote:
On Mon, 23 Aug 2004, Tony Plate wrote:
One idea I was thinking about was to have a new class of object that
referred to data in a file on disk, and which had all the standard methods
of matrices and arrays, i.e., subsetting ("["), dim, dimnames, etc.
This is what RPgSql does w
We've seen something very similar before. A file is just a database writ
small, and RPgSQL did essentially this in ca 2001. Fei Chen's research
system (see his talk at DSC2003) again does similar things.
Both of those were for virtual data frames as I recall. It's easier for a
matrix/array, b
On Mon, 23 Aug 2004, Tony Plate wrote:
>
> One idea I was thinking about was to have a new class of object that
> referred to data in a file on disk, and which had all the standard methods
> of matrices and arrays, i.e., subsetting ("["), dim, dimnames, etc. The
> object in memory would only store
> Does anyone have any opinions as to the merits of this idea? Would there
> be any interest in seeing such a class in R?
Have you looked at the 'externalVector' package in Bioconductor? I'm
admittedly not super familiar with it, although my understanding of how it
works and what it does seems
I've been wondering how to work with more data than can fit in memory, in a
way that allows it to be worked with conveniently and quickly. Of course,
a database can be used for this purpose, but extracting data from a
database is much slower and somewhat less convenient than extracting data
fr
