It actually does make sense to add chunking to sqlite. There would be some computational overhead, but, that all depends on the chunk size and the cache size of the database. It makes no sense to implement YAFS (Yet Another File System) inside of SQLite.
While many here view SQLite only in terms of desktop applications, the reality is it gets used in embedded systems for data gathering and image processing. Some of these systems gather data in real time, and others poll devices periodically to obtain a "unit" of work. The devices generating the data have completely different operating systems and even different Endianism than the data collection system. These units of work are eventually uploaded to yet another system (usually midrange or mainframe) where they are processed into industrial strength database systems in a much more granular fashion. Chunking of data allows for units of work to be arbitrary sizes. I understand that many of you reading this may not grasp the application so I will bastardize some real life stuff you might be able to picture. Many of you probably run BOINC and participate in some noble research project with the idle time of your computer. (If you don't, you should.) No matter the project, they bust up massive amounts of data into chunks. Somewhere a table in a database identifies each chunk, the date it was collected, processed, who processed it, a corresponding results chunk, and some summary result information fields. When your BOINC client connects with the server it scans the database to identify the next available chunk or chunks, assigns them to you, then sends the chunks to your client for processing. The database and the client do not care about the content of the chunk, just its size and transmission CRC. For lack of a better description, the client plug-in for the project is the only piece which knows about the content of the chunk and how to process it. It should be possible to add chunking to the database itself in such a manner that any user who does not actually use blobs in their database does not pay a computational penalty for the feature. There is an ever increasing number of embedded systems which would like to use a "linkable" database, but stumble when it comes to raw data storage. Just my 0.02. On Fri, 2013-04-26 at 18:16 +0100, Simon Slavin wrote: > On 26 Apr 2013, at 5:26pm, Stephen Chrzanowski <pontia...@gmail.com> wrote: > > > ALL THAT SAID, I doubt it'd get implemented > > I'm also in this category. In fact I hope it doesn't get implemented. Yes, > technically it can be done. But it's the sort of thing people assign as > Computer Science homework. > -- Roland Hughes, President Logikal Solutions (630)-205-1593 http://www.theminimumyouneedtoknow.com http://www.infiniteexposure.net No U.S. troops have ever lost their lives defending our ethanol reserves. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
