Hi Chuck, Thanks for posting here.
On 13 April 2011 at 12:22, ch...@chuckcode.com wrote: | Thanks Jay for the lightning fast insightful response! Didn't realize that | this limitation was intrinsic to R itself. I'll have to go look at | bigmemory and some of the other projects. Yes, Rcpp uses 'proxy model' classes around the underlying SEXP objects. It really is just R under it, and we inherit all its advantages and oddities. Such as int32 indexing for vectors. The bigmemory package can be a way out, and as it interfaces R using external pointers, you should also be able to hook into this via the Rcpp::XPtr class. That would make for a great new demo or example. Feel like cooking one up while you work through this? ;-) Cheers, Dirk | | Thanks, | -Chuck | | On Wed, 13 Apr 2011 14:11:25 -0400, Jay Emerson <jayemer...@gmail.com> | wrote: | > Chuck, | > | > Internally, R is using 4-byte integers for indexing, and the length of | > a vector is thus constrained to 2-billion-ish elements. Two ways | > around this include packages ff and | > bigmemory, for example, or relying on database-like queries. However, | > the resulting objects can't be used with standard R functions (with | > some special exceptions). | > | > Jay | > | > | > On Wed, Apr 13, 2011 at 2:05 PM, <ch...@chuckcode.com> wrote: | >> | >> Hi All, | >> | >> Thanks to Dirk Eddelbuettel and the other contributors for such a | >> wonderful package. Rcpp really transforms the way that I've been able | to | >> incorporate c++ code into R. Makes it possible to speed up the critical | >> computations while keeping all the great flexibility and features of R. | >> | >> I've run into a problem lately with particularly large vectors in Rcpp. | I | >> seem to be overflowing when my vectors get larger than 2^31 elements on | a | >> 64 bit system. It looks from the code of both the classic (included | >> below) | >> and the newer versions as though this is due to using ints rather than | >> something like size_t (http://en.wikipedia.org/wiki/Size_t) as the type | >> for | >> size and indexing into the vector. It looks like RcppResultsSet's add() | >> functions may also be using ints. | >> | >> Curious to know if there is a particular reason for using ints rather | >> than | >> something like size_t and if the project managers would be open to | >> changing | >> to accomodate larger vectors. | >> | >> Thanks, | >> -Chuck Sugnet | >> | >> template <typename T> | >> class RcppVector { | >> public: | >> typedef T* iterator ; | >> typedef const T* const_iterator ; | >> | >> RcppVector(SEXP vec); | >> RcppVector(int len); | >> int size() const; | >> T& operator()(int i) const; | >> T *cVector() const; | >> std::vector<T> stlVector() const; | >> | >> inline const_iterator begin() const { return v ; } | >> inline const_iterator end() const { return v + len ; } | >> | >> inline iterator begin(){ return v ; } | >> inline iterator end(){ return v + len ; } | >> | >> private: | >> int len; | >> T *v; | >> }; | >> _______________________________________________ | >> Rcpp-devel mailing list | >> Rcpp-devel@lists.r-forge.r-project.org | >> https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel | >> | _______________________________________________ | Rcpp-devel mailing list | Rcpp-devel@lists.r-forge.r-project.org | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel -- Dirk Eddelbuettel | e...@debian.org | http://dirk.eddelbuettel.com _______________________________________________ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
