Excellent! I was hoping/assuming it would be something like this. In this case, there is one process and two threads, but this is almost certainly what is happening.
Is this explained in a code comment somewhere? If not, would an sqlite committer be willing to add it? It is always nice when benign data races are explained. I assume accesses within the function "sqlite3StatusSet" may also be benign data races. I think this issue is now solved, but just for completeness, the platform is Linux (Ubuntu), although as explained below, sqlite runs inside an interpreter (but an implementation of pthread mutexes are used). The trace, test case and stack traces are here: https://www.doc.ic.ac.uk/~pt1110/files/file_race.tar.gz The tool is a modified version of KLEE: http://klee.llvm.org/ KLEE is a symbolic execution tool that interprets/executes LLVM bytecode. My modifications are actually a fork of Cloud9, which is a fork of KLEE that supports threads and includes a more complete POSIX model. https://cloud9.epfl.ch/projects/cloud9 I have added some schedule reductions, symbolic data race detection and some POSIX function implementations. My changes are not yet released, but they will be once I have an interesting study. Thanks, Paul On 5 May 2012 06:14, Dan Kennedy <danielk1...@gmail.com> wrote: > On 05/04/2012 11:21 PM, Paul Thomson wrote: >> >> I am working on a tool that (among other things) can detect data >> races, including file access races. I have detected a file race in >> SQLite on the database file that appears to be real, although I am not >> certain - I have no experience with SQLite. I compiled SQLite with: >> #define SQLITE_THREADSAFE 2 >> >> I use a simple test case where two threads call opendb on the same >> file, write into the same table and then close the database. The file >> race appears to occur due to sqlite3PagerReadFileheader (as the >> database is opened?) in one thread and pager_write_pagelist in the >> other. It looks as though the page that was written was pgno 1. Can >> any experts explain whether these two accesses are in fact >> synchronised, or if the race is benign? > > > Is the issue that PagerReadFileheader() may be called to read the > file-header at the same time as the first page of the database > (which contains the file-header being read) is being written by the > second connection? > > If so, it's a known issue. Immediately after opening a db file, > SQLite reads the first 100 bytes of it. Since it holds no locks > on the file at this point, some other process may be writing at > the same time. So the data read cannot be trusted. > > SQLite knows this. The only field it uses from the file-header > read at this point is the page-size field (as in "PRAGMA page_size", > default 1024 bytes). Later on, when it actually accesses the database, > it obtains a SHARED lock and reads the first page of data from > the file - using the page-size garnered from the unsafe read made > of the file-header. Once the first page of the db is loaded, it takes > another look at the page-size field in the header (part of the first > page). If it turns out that the page-size is not as expected, it > throws away any data in the cache and tries to read the first page > again, this time hopefully with the correct page size. > > So the call to sqlite4PagerReadFileheader() is just an optimization > that increases the chances that when the database is first read > SQLite will guess the correct page-size - and not have to load, > discard, and reload the first page of the db. > > The proprietary zipvfs extension (implemented as a VFS layer that > intercepts all IO calls) always returns a buffer full of zeroes > when SQLite attempts the initial 100 byte read. And it seems to > work Ok. > > The tool you're working on sounds pretty cool. Is it going to > be open-source? > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
