I'm not a developer for SQLite, but, this is my take on it. This, by
absolutely NO MEANS is meant to be snubbing anyone. This is just how my
brain works and I'm outputting a blob. {smirk}
I've never touched, looked, or even tried to wrap my mind around what
Oracle is as an entity (Same for PostreSQL), and maybe Oracle can handle
blobs as file type as you describe, but, this is why I've been told Oracle
DBMS is a file system away from being an OS, and why the compiled
amalgamation Windows DLL I have compiled is about 800k in size, versus a 2
gig download install package for Oracle.
SQLite is 'lite' for a reason. Its intelligent in a lot of places, but
pretty dumb* by design* (rightfully so) in others. I really doubt it'd
ever be able to handle taking in a large volume of data for one field, do a
comparison, byte for byte, then figure out what has been changed, then find
pages that need to be removed then re-added. There is no guarantee that
the data would be different at all, so, it'd be wasted process time doing
the comparison. I also don't know of any formula that would take, say, a
meg of data, summarize it into a few bytes to represent a truly 100% unique
value. Eventually, you will encounter two completely different chunks of
data that somehow manage to break down into the same 'unique' value. I
suppose you could throw more than one algo at figuring out the unique
value, but again, you can't ever say that the values computed for a large
chunk of data is ALWAYS going to be unique. But even with all that said,
now you're throwing much more overhead at a lite package.
If you look at the chances that size of the blob changes more often than
just a random single byte, you're looking at doing a complete dump of that
old data anyways because something significant within that data has
changed. The loss or gain of a single byte of data shifts the data to the
left or right, and now the engine has to be aware of the changes, handle
the data appropriately, and output a computed result instead of just
`belching` the data out by just streaming the data out by whatever means it
does it by.
I am by no means saying that any this CAN'T be done. What I'm saying is
that even forgetting the time it takes to code this kind of functionality
to properly handle this kind of data management is a SIGNIFICANT overhead.
Even though the overall purpose of SQLite is to handle and manage data, as
I understand it, the projects primary goal is to be slim to fit in
low-memory devices, and secondary be fast. Adding the additional overhead
of doing checks like this just, IMO, doesn't fit its calling. If this kind
of comparison/functionality is needed, I'd say make it a business rule
instead of a database rule, which means leave this kind of functionality at
the application layer somewhere. To me, it makes more sense that the
application level should handle the presentation of the data to the data
management system, and the data management system just be able to handle
that data for fast and easy recall. I also think handling large volume
chunks of data should not be included in a database, but left to the file
system.
In conclusion:
http://www.u-neak.com/three_kinds_service.htm
http://www.sixside.com/fast_good_cheap.asp
http://en.wikipedia.org/wiki/Project_management_triangle
On Tue, Apr 23, 2013 at 3:50 AM, Dominique Devienne <ddevie...@gmail.com>wrote:
>
> I also really wish SQLite blobs would map directly to the usual FILE*
> semantic, both in being able to grow a blob via writing (and truncate it
> too), but also and more importantly not rewriting the whole row or blob
> when modifying only a few bytes of the blob, but only affected pages.
> Basically another level of indirection, where the row holds only a blob
> locator (like in Oracle), and the blob value is in separate, not
> necessarily contiguous pages, as described here for example:
> http://jonathanlewis.wordpress.com/2013/03/22/lob-update/. That way only
> modified blob pages would need to participate in the transaction. SQLite is
> not MVCC like Oracle, but the ability to at least not overwrite the whole
> blob when changing 1 byte would be great. (I'm assuming there isn't, but
> I'm no SQLite expert). My $0.02. --DD
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
