Hi all, We have a simulator, 'simmer', implemented as a C++ object (with Rcpp). This simulator generates a bunch of data, and there are some functions defined to bring this data to R space as data frames. The memory usage can be a problem depending on the simulation model, so we would like to add support for logging data directly to disk while being as transparent as possible from the user perspective.
Our idea is to define a reusable chunk size and periodically append the chunk to a DBIConnection by calling DBI::dbWriteTable from C++. In this way, the subsequent analysis may be performed with dplyr no matter which storage method is selected, memory or database. The question is whether there is a better, more efficient, way of achieving this goal. Is there any package already doing something similar? Ideas and alternatives welcome. Thanks in advance. Regards, -- Iñaki Úcar http://www.enchufa2.es @Enchufa2 [[alternative HTML version deleted]] ______________________________________________ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel