Dear Henrik Please allow me to add also my opinion: Databases may be the most straightforward way to expand, but in my opinion there is a better solution, which was developed for high energy physics, where people need to work with huge amounts of data including very large vectors.
The solution is called ROOT (see: http://root.cern.ch/), especially objects, called TTree, were developed for this purpose. A TTree loads only a defined subset into memory, but given enough space can be stored completely in memory (see: http://root.cern.ch/root/html/TTree.html). Thus, ROOT is exactly the framework, from which you say: "it is hard to beat a framework that stores (contiguous) data that is optimized for reading and/or writing". Best regards Christian _._._._._._._._._._._._._._._._ C.h.i.s.t.i.a.n S.t.r.a.t.o.w.a V.i.e.n.n.a A.u.s.t.r.i.a e.m.a.i.l: cstrato at aon.at _._._._._._._._._._._._._._._._ ______________________________________________ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel