[sqlite] How to read data from WAL?

Sairam, Just in case it's useful and you've not see it, Simon Slavin pointed out this "rqlite" project for replicated sqlite in an email from 25 Feb. Donald Simon Slavin slavins at bigfraud.org via

[sqlite] How to read data from WAL?

> I have done some manual WAL decoding for my forensic software that can > identifiy a previous DB state - its fun :)... > > (From memory) > > To determine which pages belong to the last transaction, you need to : > > Read the WAL header to obtain the current salt, then read each wal > frame to det

[sqlite] How to read data from WAL?

> > Sounds to me you want to implement logical decoding, that is to > extract an SQL form from the binary WAL format. > I don't believe SQLite provides such a feature, that is something > other databases use for bidirectional replication. I just want to know which tables and rows are changed from

[sqlite] How to read data from WAL?

> > To achieve what goal? > > In any case, perhaps https://www.sqlite.org/rbu.html is of interest to > you. > I want to make note of those changes and replicate in another DB. And I found that WAL is the correct place where changes to the database are present. So my main aim is to get those change

[sqlite] How to read data from WAL?

> > If you're really sure that the database files are identical, the easiest > way to do the replication would be to copy the WAL file. > > What processing do you want to do? > I want to take note of changes and depending on data, I might replicate in some other DB. The database files are not nece

[sqlite] How to read data from WAL?

On 7 Mar 2016, at 12:24pm, Sairam Gaddam wrote: > I want to make note of those changes and replicate in another DB. Okay. Reading changes from the WAL file is perhaps a poor way to do this. First, not all SQLite databases have a WAL file. Second, SQLite can put changes in the WAL file and

[sqlite] How to read data from WAL?

> > This requires detailed knowledge of how SQLite works. If the file format > page I referred you to does not help you may have to study the SQLite > source code to find out what you need. > Yeah I read the source file wal.c and based on what I read, SQLite syncs the frames which have page data

[sqlite] How to read data from WAL?

My main aim is to find the fresh changes which are to be made to database through WAL module. How is data stored in WAL? If the data is stored in pages inside WAL then how to see the changes which are presently made to the database? Should I decrypt the information from WAL to know the changes wh

[sqlite] How to read data from WAL?

On Mon, Mar 7, 2016 at 6:52 AM, Sairam Gaddam wrote: > My main aim is to find the fresh changes which are to be made to database > through WAL module. > To achieve what goal? In any case, perhaps https://www.sqlite.org/rbu.html is of interest to you. --DD

[sqlite] How to read data from WAL?

On Mon, Mar 7, 2016 at 7:55 AM, Sairam Gaddam wrote: > And the decryption here I mean is to get information from those pages(like > on which table, which column a change is made etc.,) which involves reading > the page to get the changes which are presently made. Sounds to me you want to implemen

[sqlite] How to read data from WAL?

I have done some manual WAL decoding for my forensic software that can identifiy a previous DB state - its fun :)... (From memory) To determine which pages belong to the last transaction, you need to : Read the WAL header to obtain the current salt, then read each wal frame to determine which fr

[sqlite] How to read data from WAL?

Sairam Gaddam wrote: > I want to make note of those changes and replicate someplace else. And I > found that WAL is the correct place where changes to the database are > present. So my main aim is to get those changes and replicate after further > processing. If you're really sure that the databas

[sqlite] How to read data from WAL?

On 7 Mar 2016, at 5:52am, Sairam Gaddam wrote: > How is data stored in WAL? You can learn about the data format for the WAL file in section 4.0 of this page: > If the data is stored in pages inside WAL then how to see the changes which > are presentl