On 6 Jun 2013, at 10:45am, Philip Bennefall <phi...@blastbay.com> wrote:
> I have a bunch of data structures in memory that I am looking to replace with > an SqLite database, primarily for the purpose of avoiding reinventing the > wheel with various sorts etc. I would then like to serialize the data into a > memory buffer and do additional processing before finally rendering it to > disk. The additional processing might include compression, encryption, and a > few other things specific to my application. Two problems: Unlike the SQLite file format, the format SQLite uses when it keeps things in memory is not published, and changes from version to version. Because the writers of SQLite expect the in-memory format to be accessed only by things built into the SQLite API, you have to read the source code to know what's going on. So any routines you come up will have to just deal with whatever they find rather than trying to understand its structure. Also your data will be able to restored only back to versions of SQLite where the internal data format hasn't changed. SQLite does not, by its nature, keep everything in one long block of memory. It allocates and frees smaller blocks of memory as data is stored or deleted, and also as it needs to create temporary structures such as indexes needed to speed up a specific command. So turning a stored database into one stream of octets takes more than just reading a section of memory. Rather than try to mess with the internals of SQLite I suspect you would be better served by doing the following: 1) Using SQLite's existing in-memory databases to keep your data in memory while your app executes. 2) Writing your own routine in your preferred programming language to dump your data into text or octets in memory or disk in whatever format you want. One standard way to do this is to generate the SQL commands needed to reproduce your database. Since these are very repetitive standard ASCII commands they compress down extremely well and you can do encryption at the same time using any of a number of standard libraries. Data in this format has the added advantages that it is human-readable (after decompression) and can be passed straight to sqlite3_exec() to rebuild the database. However, you might prefer to invent your own format, perhaps more like CSV, that makes implicit use of your data structures. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
