Hi,
The SQLite database in my application becomes corrupted quite often under
high test load (error 'database disk image is malformed' is returned).
Some information:
SQLite version: 3.3.8, but a similar crash occurred at previous versions as
well.
SQLite configuration: --enable-threadsafe --enable-cross-thread-connections
--disable-tcl
OS: Linux Fedora Core 6, kernel 2.6.18
CPU: Intel dual core 2.13 GHz
The database is created with PRAGMA synchronous = OFF and PRAGMA
auto_vacuum=ON.
Application details: The application is a message queue server implemented
over a multi-process Apache http server.
The corruption occurs both when there's a single thread per process or
multiple threads per process.
In any case, each process uses a *single* SQLite session serialized between
all threads with a mutex. The test application adds, retrieves and deletes
entries concurrently.
The corruption is reproduced quite fast if auto-vacuum is set, and probably
does not occur with auto-vacuum disabled.
In order to further investigate the problem, I compiled SQLite with
--enable-debug and set a breakpoint a sqlite3Corrupt(). The debugger hit the
breakpoint and heres some information I gathered:
(gdb) bt
#0 sqlite3Corrupt () at /home/ron/sqlite-3.3.8/src/main.c:1153
#1 0x00aff2aa in ptrmapPut (pBt=0xa5b0f6a8, key=0, eType=5 '\005',
parent=16)
at /home/ron/sqlite-3.3.8/src/btree.c:826
#2 0x00b006d8 in relocatePage (pBt=0xa5b0f6a8, pDbPage=0xa5b53468,
eType=5 '\005', iPtrPage=3, iFreePage=16)
at /home/ron/sqlite-3.3.8/src/btree.c:2192
#3 0x00b009a6 in sqlite3BtreeSync (p=0xa5b11100, zMaster=0x0)
at /home/ron/sqlite-3.3.8/src/btree.c:2452
#4 0x00b47e9b in sqlite3VdbeHalt (p=0xa5b2c1c0)
at /home/ron/sqlite-3.3.8/src/vdbeaux.c:1037
#5 0x00b3e3cf in sqlite3VdbeExec (p=0xa5b2c1c0)
at /home/ron/sqlite-3.3.8/src/vdbe.c:641
#6 0x00b45a7a in sqlite3_step (pStmt=0xa5b2c1c0)
at /home/ron/sqlite-3.3.8/src/vdbeapi.c:231
#7 0x00cc359f in sql_step (db_sess=0x87023a8, stmt=0xa5b2c1c0)
at q_utils.c:371
#8 0x00cc5b3f in Q_Delete_entry (db_sess=0x87023a8,
msgID=0x87942e8 "3498-3023096720-1316816218-729045",
pChanges=0xb430c208)
at q_utils.c:1364
...
The corruption was identified due to key == 0 at btree.c:826.
Note that theres no direct call to relocatePage() from sqlite3BtreeSync(),
as it appears from the stack trace. Instead, the compiler probably optimized
and relocatePage() was called from autoVacuumCommit().
I have a feeling the corruption is related to the PRAGMA auto_vacuum=ON.
Some more variable information:
(gdb) up
#1 0x00aff2aa in ptrmapPut (pBt=0xa5b0f6a8, key=0, eType=5 '\005',
parent=16)
at /home/ron/sqlite-3.3.8/src/btree.c:826
826 return SQLITE_CORRUPT_BKPT;
(gdb) p *pBt
$7 = {pPager = 0xa5b0e5f0, pCursor = 0x0, pPage1 = 0xa5b50238,
inStmt = 0 '\0', readOnly = 0 '\0', maxEmbedFrac = 64 '@',
minEmbedFrac = 32 ' ', minLeafFrac = 32 ' ', pageSizeFixed = 1 '\001',
autoVacuum = 1 '\001', pageSize = 1024, usableSize = 1024, maxLocal = 230,
minLocal = 103, maxLeaf = 989, minLeaf = 103, pBusyHandler = 0xa5b113dc,
inTransaction = 2 '\002', nRef = 1, nTransaction = 1, pSchema =
0xa5b0e740,
xFreeSchema = 0xb0ff40 <sqlite3SchemaFree>, pLock = 0x0, pNext = 0x0}
(gdb) p *pBt->pPager
$8 = {journalOpen = 1 '\001', journalStarted = 0 '\0', useJournal = 1
'\001',
noReadlock = 0 '\0', stmtOpen = 0 '\0', stmtInUse = 0 '\0',
stmtAutoopen = 0 '\0', noSync = 1 '\001', fullSync = 0 '\0',
full_fsync = 0 '\0', state = 2 '\002', tempFile = 0 '\0', readOnly = 0
'\0',
needSync = 0 '\0', dirtyCache = 1 '\001', alwaysRollback = 0 '\0',
memDb = 0 '\0', setMaster = 0 '\0', errCode = 0, dbSize = 35,
origDbSize = 35, stmtSize = 0, nRec = 9, cksumInit = 2661323079,
stmtNRec = 0, nExtra = 80, pageSize = 1024, nPage = 13, nMaxPage = 48,
nRef = 2, mxPage = 2000, aInJournal = 0xa5b0f788 "&\212\201", aInStmt =
0x0,
zFilename = 0xa5b0e6b8 "/usr/local/apache2/q-db/mq/zz.db",
zJournal = 0xa5b0e6fa "/usr/local/apache2/q-db/mq/zz.db-journal",
zDirectory = 0xa5b0e6d9 "/usr/local/apache2/q-db/mq", fd = 0xa5b10f90,
jfd = 0xa5b0f930, stfd = 0x0, pBusyHandler = 0xa5b113dc,
pFirst = 0xa5b51df0, pLast = 0xa5b52ba0, pFirstSynced = 0xa5b51df0,
pAll = 0xa5b53030, pStmt = 0x0, pDirty = 0xa5b53030, journalOff = 9800,
journalHdr = 0, stmtHdrOff = 0, stmtCksum = 0, stmtJSize = 0,
sectorSize = 512, xDestructor = 0xaff090 <pageDestructor>,
xReiniter = 0xaffe70 <pageReinit>, xCodec = 0, pCodecArg = 0x0, nHash =
256,
aHash = 0xa5b0fc78}
(gdb) p *pBt->pPage1
$9 = {isInit = 0 '\0', idxShift = 0 '\0', nOverflow = 0 '\0', intKey = 0
'\0',
leaf = 0 '\0', zeroData = 0 '\0', leafData = 0 '\0', hasData = 0 '\0',
hdrOffset = 100 'd', childPtrSize = 0 '\0', maxLocal = 0, minLocal = 0,
cellOffset = 0, idxParent = 0, nFree = 0, nCell = 0, aOvfl = {{pCell =
0x0,
idx = 0}, {pCell = 0x0, idx = 0}, {pCell = 0x0, idx = 0}, {pCell =
0x0,
idx = 0}, {pCell = 0x0, idx = 0}}, pBt = 0xa5b0f6a8,
aData = 0xa5b4fe38 "SQLite format 3", pgno = 1, pParent = 0x0}
(gdb)
(gdb) up
#2 0x00b006d8 in relocatePage (pBt=0xa5b0f6a8, pDbPage=0xa5b53468,
eType=5 '\005', iPtrPage=3, iFreePage=16)
at /home/ron/sqlite-3.3.8/src/btree.c:2192
2192 Pgno childPgno = get4byte(&pPage->aData[pPage->hdrOffset+8]);
(gdb) p *pDbPage
$10 = {isInit = 1 '\001', idxShift = 0 '\0', nOverflow = 0 '\0',
intKey = 0 '\0', leaf = 0 '\0', zeroData = 0 '\0', leafData = 0 '\0',
hasData = 1 '\001', hdrOffset = 0 '\0', childPtrSize = 4 '\004',
maxLocal = 230, minLocal = 103, cellOffset = 12, idxParent = 0,
nFree = 65524, nCell = 0, aOvfl = {{pCell = 0x0, idx = 0}, {pCell = 0x0,
idx = 0}, {pCell = 0x0, idx = 0}, {pCell = 0x0, idx = 0}, {pCell =
0x0,
idx = 0}}, pBt = 0xa5b0f6a8, aData = 0xa5b53068 "", pgno = 16,
pParent = 0x0}
(gdb) up
#3 0x00b009a6 in sqlite3BtreeSync (p=0xa5b11100, zMaster=0x0)
at /home/ron/sqlite-3.3.8/src/btree.c:2452
2452 rc = relocatePage(pBt, pDbMemPage, eType, iPtrPage, iFreePage);
(gdb) p *p
$12 = {pSqlite = 0xa5b112c8, pBt = 0xa5b0f6a8, inTrans = 2 '\002'}
(gdb)
Any help will be greatly appreciated.
Thanks,
Ron
_________________________________________________________________
Don't just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
