Re: [sqlite] Data race (file race) in pager
Excellent! I was hoping/assuming it would be something like this. In this case, there is one process and two threads, but this is almost certainly what is happening. Is this explained in a code comment somewhere? If not, would an sqlite committer be willing to add it? It is always nice when benign data races are explained. I assume accesses within the function "sqlite3StatusSet" may also be benign data races. I think this issue is now solved, but just for completeness, the platform is Linux (Ubuntu), although as explained below, sqlite runs inside an interpreter (but an implementation of pthread mutexes are used). The trace, test case and stack traces are here: https://www.doc.ic.ac.uk/~pt1110/files/file_race.tar.gz The tool is a modified version of KLEE: http://klee.llvm.org/ KLEE is a symbolic execution tool that interprets/executes LLVM bytecode. My modifications are actually a fork of Cloud9, which is a fork of KLEE that supports threads and includes a more complete POSIX model. https://cloud9.epfl.ch/projects/cloud9 I have added some schedule reductions, symbolic data race detection and some POSIX function implementations. My changes are not yet released, but they will be once I have an interesting study. Thanks, Paul On 5 May 2012 06:14, Dan Kennedy wrote: > On 05/04/2012 11:21 PM, Paul Thomson wrote: >> >> I am working on a tool that (among other things) can detect data >> races, including file access races. I have detected a file race in >> SQLite on the database file that appears to be real, although I am not >> certain - I have no experience with SQLite. I compiled SQLite with: >> #define SQLITE_THREADSAFE 2 >> >> I use a simple test case where two threads call opendb on the same >> file, write into the same table and then close the database. The file >> race appears to occur due to sqlite3PagerReadFileheader (as the >> database is opened?) in one thread and pager_write_pagelist in the >> other. It looks as though the page that was written was pgno 1. Can >> any experts explain whether these two accesses are in fact >> synchronised, or if the race is benign? > > > Is the issue that PagerReadFileheader() may be called to read the > file-header at the same time as the first page of the database > (which contains the file-header being read) is being written by the > second connection? > > If so, it's a known issue. Immediately after opening a db file, > SQLite reads the first 100 bytes of it. Since it holds no locks > on the file at this point, some other process may be writing at > the same time. So the data read cannot be trusted. > > SQLite knows this. The only field it uses from the file-header > read at this point is the page-size field (as in "PRAGMA page_size", > default 1024 bytes). Later on, when it actually accesses the database, > it obtains a SHARED lock and reads the first page of data from > the file - using the page-size garnered from the unsafe read made > of the file-header. Once the first page of the db is loaded, it takes > another look at the page-size field in the header (part of the first > page). If it turns out that the page-size is not as expected, it > throws away any data in the cache and tries to read the first page > again, this time hopefully with the correct page size. > > So the call to sqlite4PagerReadFileheader() is just an optimization > that increases the chances that when the database is first read > SQLite will guess the correct page-size - and not have to load, > discard, and reload the first page of the db. > > The proprietary zipvfs extension (implemented as a VFS layer that > intercepts all IO calls) always returns a buffer full of zeroes > when SQLite attempts the initial 100 byte read. And it seems to > work Ok. > > The tool you're working on sounds pretty cool. Is it going to > be open-source? > > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Data race (file race) in pager
On 05/04/2012 11:21 PM, Paul Thomson wrote: I am working on a tool that (among other things) can detect data races, including file access races. I have detected a file race in SQLite on the database file that appears to be real, although I am not certain - I have no experience with SQLite. I compiled SQLite with: #define SQLITE_THREADSAFE 2 I use a simple test case where two threads call opendb on the same file, write into the same table and then close the database. The file race appears to occur due to sqlite3PagerReadFileheader (as the database is opened?) in one thread and pager_write_pagelist in the other. It looks as though the page that was written was pgno 1. Can any experts explain whether these two accesses are in fact synchronised, or if the race is benign? Is the issue that PagerReadFileheader() may be called to read the file-header at the same time as the first page of the database (which contains the file-header being read) is being written by the second connection? If so, it's a known issue. Immediately after opening a db file, SQLite reads the first 100 bytes of it. Since it holds no locks on the file at this point, some other process may be writing at the same time. So the data read cannot be trusted. SQLite knows this. The only field it uses from the file-header read at this point is the page-size field (as in "PRAGMA page_size", default 1024 bytes). Later on, when it actually accesses the database, it obtains a SHARED lock and reads the first page of data from the file - using the page-size garnered from the unsafe read made of the file-header. Once the first page of the db is loaded, it takes another look at the page-size field in the header (part of the first page). If it turns out that the page-size is not as expected, it throws away any data in the cache and tries to read the first page again, this time hopefully with the correct page size. So the call to sqlite4PagerReadFileheader() is just an optimization that increases the chances that when the database is first read SQLite will guess the correct page-size - and not have to load, discard, and reload the first page of the db. The proprietary zipvfs extension (implemented as a VFS layer that intercepts all IO calls) always returns a buffer full of zeroes when SQLite attempts the initial 100 byte read. And it seems to work Ok. The tool you're working on sounds pretty cool. Is it going to be open-source? ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Data race (file race) in pager
On Fri, May 4, 2012 at 12:21 PM, Paul Thomson wrote: > I am working on a tool that (among other things) can detect data > races, including file access races. I have detected a file race in > SQLite on the database file that appears to be real, although I am not > certain - I have no experience with SQLite. I compiled SQLite with: > #define SQLITE_THREADSAFE 2 > > I use a simple test case where two threads call opendb There is no such function "opendb()" in SQLite, either as an API or internal. Did you mean sqlite3_open()? > on the same > file, write into the same table and then close the database. The file > race appears to occur due to sqlite3PagerReadFileheader (as the > database is opened?) in one thread and pager_write_pagelist in the > other. It looks as though the page that was written was pgno 1. Can > any experts explain whether these two accesses are in fact > synchronised, or if the race is benign? > Each database connection will have its own page cache, so access to internal data structures need not be synchronized. Access to the disk file is synchronized using file locks - the exact workings of which depends on the operating system, which you have left unspecified. > > I am using sqlite-amalgamation-3071100. I will later upload the test > case, trace (showing all calls that are made by each thread) and the > stack trace for each thread when the race occurs, unless it becomes > obvious that the above functions cannot race. What is best way to > provide these files? (attachments? upload to a website and provide a > link?) > > Thanks, > Paul > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
