Thanks!
I want to save data from a matrix to a file and then retrieve it later while running R code, all within R. As long as the compression doesn't result in data loss, it should be suitable for my needs. Best, --Sameh From: Dirk Eddelbuettel <e...@debian.org> Date: Thursday, May 9, 2024 at 6:35 AM To: Sameh Abdulah <sameh.abdu...@kaust.edu.sa> Cc: r-package-devel@r-project.org <r-package-devel@r-project.org> Subject: [EXTERNAL] Re: [R-pkg-devel] Fast Matrix Serialization in R? On 9 May 2024 at 03:20, Sameh Abdulah wrote: | I need to serialize and save a 20K x 20K matrix as a binary file. Hm that is an incomplete specification: _what_ do you want to do with it? Read it back in R? Share it with other languages (like Python) ? I.e. what really is your use case? Also, you only seem to use readBin / writeBin. Why not readRDS / saveRDS which at least give you compression? If it is to read/write from / to R look into the qs package. It is good. The README.md at its repo has benchmarks: https://urldefense.com/v3/__https://github.com/traversc/qs__;!!Nmw4Hv0!zZitnMd5aMqDWhLShhC5Npmd8pkisFVejC3grX2YGSGWFZihVaHAezYyo5nXbAwIdqCj2T8Nbk-Yofxa-y8$<https://urldefense.com/v3/__https:/github.com/traversc/qs__;!!Nmw4Hv0!zZitnMd5aMqDWhLShhC5Npmd8pkisFVejC3grX2YGSGWFZihVaHAezYyo5nXbAwIdqCj2T8Nbk-Yofxa-y8$> If you want to index into the stored data look into fst. Else also look at databases Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org -- This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email. [[alternative HTML version deleted]] ______________________________________________ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel