Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Keith Medcalf
You still have not explained what you think COW is and of what possible use it could be. If you want a "snapshot + changes" then why not just enable WAL mode and disable checkpointing? -- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traff

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Fredrik Larsen
Applying the dump from session extension will modify the db, so you need to copy all data to the node that will run sqlite, then apply changes, then start db. A COW system would allow you to to store the bulk of the data in a shared and network-mounted folder, and only copy over the changes since t

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Fredrik Larsen
A file-system with COW support would work, but that is a big dependency/constraint to bring into a project, and not always possible/practical. A file based version (snapshot + changes) will be more practical and easier to manage, and also very doable IMHO. Anyway, I was just wondering if anyone el

[sqlite] sqlite3 doesn't build with -DSQLITE_ENABLE_DBPAGE_VTAB with bitbake

2019-10-04 Thread Kevin Dankwardt
I need sqlite3 to be built with -DSQLITE_ENABLE_DBPAGE_VTAB so that I can use the ".recover" command. The ".dbinfo" command doesn't work either. I add -DSQLITE_ENABLE_DBPAGE_VTAB to the sqlite recipe CFLAGS in various ways but I never get sqlite3 built with the option. Example error: the ".dbinfo

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Roman Fleysher
Maybe this for this scenario: You have in-memory database, used mostly for reading and you want to save its copy to disk when update on in-memory is performed? Otherwise, what is copied and what is to write in "copy on write"? From: sqlite-users [sqlit

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Keith Medcalf
On Friday, 4 October, 2019 05:18, Fredrik Larsen wrote: >A copy-on-write IO-path where data is split into static and dynamic parts >(think snapshots for storage) would be very helpful for our project, . What do you mean? Useful how? >This would simplify backups, testing, moving data around in

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread Simon Slavin
On 4 Oct 2019, at 12:17pm, Fredrik Larsen wrote: > A copy-on-write IO-path where data is split into static and dynamic parts > (think snapshots for storage) would be very helpful for our project. SQLite abstracts changes-only tracking at the transaction level, and provides it as the session ex

Re: [sqlite] Copy-on-write VFS

2019-10-04 Thread test user
Hello Fredrik, Why does it need to be part of a VFS instead of using a file system with COW like ZFS? On Fri, 4 Oct 2019 at 12:18, Fredrik Larsen wrote: > Hi > > A copy-on-write IO-path where data is split into static and dynamic parts > (think snapshots for storage) would be very helpful for o

[sqlite] Copy-on-write VFS

2019-10-04 Thread Fredrik Larsen
Hi A copy-on-write IO-path where data is split into static and dynamic parts (think snapshots for storage) would be very helpful for our project, . This would simplify backups, testing, moving data around in a multinode environment, etc. Does something like this exist for sqlite? In my head this