Thanks Jay for the lightning fast insightful response! Didn't realize that this limitation was intrinsic to R itself. I'll have to go look at bigmemory and some of the other projects.
Thanks, -Chuck On Wed, 13 Apr 2011 14:11:25 -0400, Jay Emerson <jayemer...@gmail.com> wrote: > Chuck, > > Internally, R is using 4-byte integers for indexing, and the length of > a vector is thus constrained to 2-billion-ish elements. Two ways > around this include packages ff and > bigmemory, for example, or relying on database-like queries. However, > the resulting objects can't be used with standard R functions (with > some special exceptions). > > Jay > > > On Wed, Apr 13, 2011 at 2:05 PM, <ch...@chuckcode.com> wrote: >> >> Hi All, >> >> Thanks to Dirk Eddelbuettel and the other contributors for such a >> wonderful package. Rcpp really transforms the way that I've been able to >> incorporate c++ code into R. Makes it possible to speed up the critical >> computations while keeping all the great flexibility and features of R. >> >> I've run into a problem lately with particularly large vectors in Rcpp. I >> seem to be overflowing when my vectors get larger than 2^31 elements on a >> 64 bit system. It looks from the code of both the classic (included >> below) >> and the newer versions as though this is due to using ints rather than >> something like size_t (http://en.wikipedia.org/wiki/Size_t) as the type >> for >> size and indexing into the vector. It looks like RcppResultsSet's add() >> functions may also be using ints. >> >> Curious to know if there is a particular reason for using ints rather >> than >> something like size_t and if the project managers would be open to >> changing >> to accomodate larger vectors. >> >> Thanks, >> -Chuck Sugnet >> >> template <typename T> >> class RcppVector { >> public: >> typedef T* iterator ; >> typedef const T* const_iterator ; >> >> RcppVector(SEXP vec); >> RcppVector(int len); >> int size() const; >> T& operator()(int i) const; >> T *cVector() const; >> std::vector<T> stlVector() const; >> >> inline const_iterator begin() const { return v ; } >> inline const_iterator end() const { return v + len ; } >> >> inline iterator begin(){ return v ; } >> inline iterator end(){ return v + len ; } >> >> private: >> int len; >> T *v; >> }; >> _______________________________________________ >> Rcpp-devel mailing list >> Rcpp-devel@lists.r-forge.r-project.org >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> _______________________________________________ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
