Doug, I think the problem is resolved. I wasted a lot of time trying to debug the new Module code, and didn't think to try gctorture() with older code that has worked for a long time. That code failed too!
The attached file tortureFix.cpp shows the kind of fix that is needed in some of the Rcpp code, for example, in Reference.cpp, and perhaps in other files. The attached script torture.R tests this function with/without the suggested fix. The unsafe code may or may not fail for you. It depends on the machine/compiler, but because it can fail it is wrong. It fails consistently under Windows using Visual C++, for example, which is a good reason why software should be tested using compilers other than GCC, and testing with gctorture() is obviously a good idea as well. Thanks, Dominick On Sat, Jan 8, 2011 at 2:06 PM, Douglas Bates <ba...@stat.wisc.edu> wrote: > On Sat, Jan 8, 2011 at 12:57 PM, Dominick Samperi <djsamp...@gmail.com> > wrote: > > > > > > On Sat, Jan 8, 2011 at 9:48 AM, Douglas Bates <ba...@stat.wisc.edu> > wrote: > >> > >> On Sat, Jan 8, 2011 at 7:49 AM, Douglas Bates <ba...@stat.wisc.edu> > wrote: > >> > On Sat, Jan 8, 2011 at 12:45 AM, Dominick Samperi < > djsamp...@gmail.com> > >> > wrote: > >> >> I checked things under Linux and Windows (using GCC and VC++ DLL's) > and > >> >> the > >> >> same problem occurs at the same place, which is a good sign when it > >> >> comes to > >> >> memory issues. Basically, the Rcpp::Reference(std::string) > constructor > >> >> that > >> >> is > >> >> part of S4_field, or S4_CppOverloadedMethods constructors fails, > >> >> depending > >> >> on > >> >> which comes first (whether there are fields or not). This only > happens > >> >> when > >> >> gctorture() is turned on, so R must be clobbering an unprotected SEXP > >> >> somewhere... > >> > > >> > Thanks, Dominick. I too have been working on tracking this down and > >> > got to that point. If you follow a little further you will find that > >> > it is the evaluation of the R function getCurrentErrorMessage in the > >> > Rcpp::Evaluator::run method where things start to go bad, as far as I > >> > can see. I hope to be able to isolate this today as I need a working > >> > version of lme4a by tomorrow. > >> > >> My current theory is that Rcpp_cache is being blown away by the > >> garbage collector. Because we want this to be persistent I think the > >> simplest thing to do is to assign it to a name like .Rcpp_cache in the > >> namespace during the .onLoad function. I'll try that. > > > > I tested this theory by cutting out the Evaluator code and it appears > that > > the cache is not harmed by gctorture(). I suspect that the problem may > > be earlier in the chain of construction, either a SEXP that was not > > preserved, > > or an aliasing problem. It appears that init_Rcpp_cache() is called twice > > at start-up, which does not seem right, but suppressing the second > attempt > > does not fix the problem. > > The second call was eliminated in the SVN archive a couple of days > ago. I have gotten around some of the problems with the garbage > collection by redefining how the Rcpp_cache variable is created (not > yet checked in) so it progresses further. I still think that there is > a point where a promise in not being evaluated before being used but > haven't found it yet. > > With lazyLoad enabled the first time that you find a variable in the > namespace it will have a PROMSXP typeof field and you need to invoke > Rf_eval on it to actually get the value. There are several variables > from the Rcpp namespace being used here, which is why I am suspicious > that an unevaluated promise is somehow getting used. > > >> > >> >> On Fri, Jan 7, 2011 at 9:47 AM, Romain Francois > >> >> <rom...@r-enthusiasts.com> > >> >> wrote: > >> >>> > >> >>> Hmm. I commited 2845 and 2846 today. > >> >>> > >> >>> Anyway, if you see it also with 0.9.0 this means more detective > work. > >> >>> > >> >>> Le 07/01/11 15:05, Douglas Bates a écrit : > >> >>>> > >> >>>> On Fri, Jan 7, 2011 at 5:34 AM, Romain Francois > >> >>>> <rom...@r-enthusiasts.com> wrote: > >> >>>>> > >> >>>>> Le 05/01/11 18:52, Douglas Bates a écrit : > >> >>>>>> > >> >>>>>> I don't know whether this is through error on my part or because > of > >> >>>>>> an > >> >>>>>> "infelicity" in the Rcpp module code but the lme4a package, which > >> >>>>>> now > >> >>>>>> uses Rcpp modules extensively, ends up with some > difficult-to-trace > >> >>>>>> memory corruption issues. Yesterday i finally bit the bullet and > >> >>>>>> ran > >> >>>>>> a test with gctorture(TRUE) and valgrind enabled. It takes a > very > >> >>>>>> long time and results in a segfault when trying to load the > >> >>>>>> package. > >> >>>>>> I enclose the transcript. I should say that this is using > >> >>>>>> Rcpp_0.9.0 > >> >>>>>> from CRAN, not the SVN version of Rcpp. > >> >>>>>> > >> >>>>>> I just got these results this morning (it was running overnight) > >> >>>>>> and > >> >>>>>> haven't looked at the code in Module.cpp and cache.cpp yet. If > it > >> >>>>>> seems likely that the code is beyond me I can try to work out a > >> >>>>>> simpler example that triggers the problem. > >> >>>>> > >> >>>>> Hi Doug, > >> >>>>> > >> >>>>> Sorry for the delay, I'm not fully operational yet. > >> >>>>> > >> >>>>> All this might be related to some code I put in during holidays > and > >> >>>>> did > >> >>>>> not > >> >>>>> have a chance to fully test. > >> >>>>> > >> >>>>> Can you try with rev 2845 and let me know if you still see the > >> >>>>> problem. > >> >>>>> > >> >>>>> Romain > >> >>>> > >> >>>> Regrettably the problem persists with rev 2845 (which was from > >> >>>> 2011-01-04, is that the one you meant?) but it is also present when > >> >>>> using Rcpp_0.9.0 > >> >>> > >> >>> > >> >>> -- > >> >>> Romain Francois > >> >>> Professional R Enthusiast > >> >>> +33(0) 6 28 91 30 30 > >> >>> http://romainfrancois.blog.free.fr > >> >>> |- http://bit.ly/fT2rZM : highlight 0.2-5 > >> >>> |- http://bit.ly/gpCSpH : Evolution of Rcpp code size > >> >>> `- http://bit.ly/hovakS : RcppGSL initial release > >> >>> > >> >>> > >> >>> _______________________________________________ > >> >>> Rcpp-devel mailing list > >> >>> Rcpp-devel@lists.r-forge.r-project.org > >> >>> > >> >>> > https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel > >> >> > >> >> > >> > > > > > >
#include "Rcpp.h" RcppExport SEXP testfft(SEXP vec, SEXP env) { //#define UNSAFE_CODE #ifdef UNSAFE_CODE // may not survive gctorture(), sometimes! SEXP call = ::Rf_lcons( Rf_install("fft"), Rf_cons(vec , R_NilValue)); SEXP res = PROTECT(Rf_eval(call, env)); UNPROTECT(1); #else // always survives gctorture() SEXP call = PROTECT(::Rf_lcons( Rf_install("fft"), Rf_cons(vec , R_NilValue))); SEXP res = PROTECT(Rf_eval(call, env)); UNPROTECT(2); #endif return res; }
torture.R
Description: Binary data
_______________________________________________ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel