Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread Stephen Chrzanowski
@Jens; I'd take what David suggested one step further and just remove the BLOB from the tables of concern and keep the blob on a 1:1 PK:FK relationship and only look at the BLOB tables when required. That way if you do an occasional [select * from ...] you're not reading the BLOB because it's liv

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread Richard Hipp
On 3/29/17, David Raymond wrote: > Remember to make the blob field the very last field in your table schema, > and avoid "select * from", otherwise you can create performance problems. > The data for a record is stored in the same order as the fields are defined, > and overflow pages are a linked

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread David Raymond
Remember to make the blob field the very last field in your table schema, and avoid "select * from", otherwise you can create performance problems. The data for a record is stored in the same order as the fields are defined, and overflow pages are a linked list. So if you have your 100 MB blob f

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread Dan Kennedy
On 03/30/2017 12:52 AM, Olivier Mascia wrote: Le 29 mars 2017 à 19:40, Simon Slavin a écrit : Since I’m using WAL, will this db handle keep a snapshot of the time before the blob’s row was deleted, allowing the blob handle to keep working? As Simon said wisely, a word of the developers would

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread Olivier Mascia
> Le 29 mars 2017 à 19:40, Simon Slavin a écrit : > >> Since I’m using WAL, will this db handle keep a snapshot of the time before >> the blob’s row was deleted, allowing the blob handle to keep working? As Simon said wisely, a word of the developers would clarify this better. Yet, since you'r

Re: [sqlite] Questions on big blobs and blob I/O

2017-03-29 Thread Simon Slavin
On 29 Mar 2017, at 6:14pm, Jens Alfke wrote: > * Do big blobs cause any performance problems for the db as a whole, like > fragmentation? In particular, is the blob data stored contiguously in the > file? Blobs are stored in the same pages that other values are stored in. If a blob is bigge