On Tue, Mar 12, 2019 at 11:45 AM James K. Lowden <jklow...@schemamania.org> wrote:
> On Tue, 12 Mar 2019 10:36:37 -0400 > ted.goldbl...@gmail.com wrote: > > > The problem is basically that as part of a test, the customer wants > > to power fail the device, and then to be able to recover (after the > > device restarts) data that was generated/stored as close to the power > > failure as possible. > ... > > Unfortunately, when doing this, it appears that significant amounts > > of data that should be in the database are missing and/or corrupted, > > and this doesn?t appear particularly dependent on when the failure > > occurs. > > Only 3 possibilities exist: > > 1. SQLite finished its transaction before the power cut, and the data > were committed and do appear in the database later. > This is not the case > > 2. SQLite did not finish its transaction, and the transaction was > rolled back as part of database initialization after power was > restored. This is also not the case > > 3. The hardware or driver reported the data were written when they > were not. > This *may* be the case > > We can dismiss as statistically insignificant possibility #4, a bug in > SQLite, because of SQLite's excellent testing regimen and gigantic user > base. I have been writing software for too many decades to casually dismiss the possibilities of software bugs. If there couldn't be bugs in SQLite, there would have been no bug fixes since the version being used here, and having briefly perused the revision history, it is obvious that isn't the case. Further, different users run in different environments and do different things, both of which can shake loose bugs or unexpected behaviors. That said, I don't think what is going on here is a "oops" type bug, though it could be an interaction (or configuration) issue - possibly something that could have been addressed in one way or other over the last 8 years, hence my question. > > > If you can confirm that SQLite finished the transaction whose data the > database does not reflect on restart, you really must suspect the > driver or device. I don't know much about USB drives, but consumer > grade hard drives *normally* lie about committed data for performance > reasons. "It's easy to make it fast if it doesn't have to be right." > USB devices face at least as much temptation to misrepresent their > performance. > I can confirm the state of the database after restart. As far as the backing device - this is an industrial device. The storage is not USB, consumer grade, or a hard drive. It is NAND flash parts soldered to the PC board and directly addressed by the (locally written) low-level device driver. I can assure you that there was no one to be impressed by any performance numbers - that isn't an issue. What may be an issue, however, is the (real-time OS supplied) file system driver, which *could* do buffering and *might *not honor sync() properly. I am currently looking into that. > > _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
