What happens when you run your program through Valgrind, or any other
memory debugger? Even the crudest memory debugger that fills free()'d
memory with garbage can be very helpful in debugging multi-threaded
program crashes.
Regarding Apache 2, I've had problems with their memory pools before,
specifically mixing Apache Portable Runtime (APR) memory allocation
with traditional malloc-based allocation. You might want to doublecheck
your memory freeing logic.
--- Ron Avriel <[EMAIL PROTECTED]> wrote:
> Hi,
>
> The SQLite database in my application becomes corrupted quite often under
> high test load (error 'database disk image is malformed' is returned).
>
> Some information:
> SQLite version: 3.3.8, but a similar crash occurred at previous versions as
> well.
> SQLite configuration: --enable-threadsafe --enable-cross-thread-connections
> --disable-tcl
> OS: Linux Fedora Core 6, kernel 2.6.18
> CPU: Intel dual core 2.13 GHz
>
> The database is created with PRAGMA synchronous = OFF and PRAGMA
> auto_vacuum=ON.
>
> Application details: The application is a message queue server implemented
> over a multi-process Apache http server.
> The corruption occurs both when there's a single thread per process or
> multiple threads per process.
> In any case, each process uses a *single* SQLite session serialized between
> all threads with a mutex. The test application adds, retrieves and deletes
> entries concurrently.
>
> The corruption is reproduced quite fast if auto-vacuum is set, and probably
> does not occur with auto-vacuum disabled.
>
> In order to further investigate the problem, I compiled SQLite with
> --enable-debug and set a breakpoint a sqlite3Corrupt(). The debugger hit the
> breakpoint and heres some information I gathered:
>
> (gdb) bt
> #0 sqlite3Corrupt () at /home/ron/sqlite-3.3.8/src/main.c:1153
> #1 0x00aff2aa in ptrmapPut (pBt=0xa5b0f6a8, key=0, eType=5 '\005',
> parent=16)
> at /home/ron/sqlite-3.3.8/src/btree.c:826
> #2 0x00b006d8 in relocatePage (pBt=0xa5b0f6a8, pDbPage=0xa5b53468,
> eType=5 '\005', iPtrPage=3, iFreePage=16)
> at /home/ron/sqlite-3.3.8/src/btree.c:2192
> #3 0x00b009a6 in sqlite3BtreeSync (p=0xa5b11100, zMaster=0x0)
> at /home/ron/sqlite-3.3.8/src/btree.c:2452
> #4 0x00b47e9b in sqlite3VdbeHalt (p=0xa5b2c1c0)
> at /home/ron/sqlite-3.3.8/src/vdbeaux.c:1037
> #5 0x00b3e3cf in sqlite3VdbeExec (p=0xa5b2c1c0)
> at /home/ron/sqlite-3.3.8/src/vdbe.c:641
> #6 0x00b45a7a in sqlite3_step (pStmt=0xa5b2c1c0)
> at /home/ron/sqlite-3.3.8/src/vdbeapi.c:231
> #7 0x00cc359f in sql_step (db_sess=0x87023a8, stmt=0xa5b2c1c0)
> at q_utils.c:371
> #8 0x00cc5b3f in Q_Delete_entry (db_sess=0x87023a8,
> msgID=0x87942e8 "3498-3023096720-1316816218-729045",
> pChanges=0xb430c208)
> at q_utils.c:1364
>
> ...
>
> The corruption was identified due to key == 0 at btree.c:826.
>
> Note that theres no direct call to relocatePage() from sqlite3BtreeSync(),
> as it appears from the stack trace. Instead, the compiler probably optimized
> and relocatePage() was called from autoVacuumCommit().
> I have a feeling the corruption is related to the PRAGMA auto_vacuum=ON.
>
> Some more variable information:
> (gdb) up
> #1 0x00aff2aa in ptrmapPut (pBt=0xa5b0f6a8, key=0, eType=5 '\005',
> parent=16)
> at /home/ron/sqlite-3.3.8/src/btree.c:826
> 826 return SQLITE_CORRUPT_BKPT;
> (gdb) p *pBt
> $7 = {pPager = 0xa5b0e5f0, pCursor = 0x0, pPage1 = 0xa5b50238,
> inStmt = 0 '\0', readOnly = 0 '\0', maxEmbedFrac = 64 '@',
> minEmbedFrac = 32 ' ', minLeafFrac = 32 ' ', pageSizeFixed = 1 '\001',
> autoVacuum = 1 '\001', pageSize = 1024, usableSize = 1024, maxLocal = 230,
> minLocal = 103, maxLeaf = 989, minLeaf = 103, pBusyHandler = 0xa5b113dc,
> inTransaction = 2 '\002', nRef = 1, nTransaction = 1, pSchema =
> 0xa5b0e740,
> xFreeSchema = 0xb0ff40 <sqlite3SchemaFree>, pLock = 0x0, pNext = 0x0}
> (gdb) p *pBt->pPager
> $8 = {journalOpen = 1 '\001', journalStarted = 0 '\0', useJournal = 1
> '\001',
> noReadlock = 0 '\0', stmtOpen = 0 '\0', stmtInUse = 0 '\0',
> stmtAutoopen = 0 '\0', noSync = 1 '\001', fullSync = 0 '\0',
> full_fsync = 0 '\0', state = 2 '\002', tempFile = 0 '\0', readOnly = 0
> '\0',
> needSync = 0 '\0', dirtyCache = 1 '\001', alwaysRollback = 0 '\0',
> memDb = 0 '\0', setMaster = 0 '\0', errCode = 0, dbSize = 35,
> origDbSize = 35, stmtSize = 0, nRec = 9, cksumInit = 2661323079,
> stmtNRec = 0, nExtra = 80, pageSize = 1024, nPage = 13, nMaxPage = 48,
> nRef = 2, mxPage = 2000, aInJournal = 0xa5b0f788 "&\212\201", aInStmt =
> 0x0,
> zFilename = 0xa5b0e6b8 "/usr/local/apache2/q-db/mq/zz.db",
> zJournal = 0xa5b0e6fa "/usr/local/apache2/q-db/mq/zz.db-journal",
> zDirectory = 0xa5b0e6d9 "/usr/local/apache2/q-db/mq", fd = 0xa5b10f90,
> jfd = 0xa5b0f930, stfd = 0x0, pBusyHandler = 0xa5b113dc,
> pFirst = 0xa5b51df0, pLast = 0xa5b52ba0, pFirstSynced = 0xa5b51df0,
> pAll = 0xa5b53030, pStmt = 0x0, pDirty = 0xa5b53030, journalOff = 9800,
> journalHdr = 0, stmtHdrOff = 0, stmtCksum = 0, stmtJSize = 0,
> sectorSize = 512, xDestructor = 0xaff090 <pageDestructor>,
> xReiniter = 0xaffe70 <pageReinit>, xCodec = 0, pCodecArg = 0x0, nHash =
> 256,
> aHash = 0xa5b0fc78}
> (gdb) p *pBt->pPage1
> $9 = {isInit = 0 '\0', idxShift = 0 '\0', nOverflow = 0 '\0', intKey = 0
> '\0',
> leaf = 0 '\0', zeroData = 0 '\0', leafData = 0 '\0', hasData = 0 '\0',
> hdrOffset = 100 'd', childPtrSize = 0 '\0', maxLocal = 0, minLocal = 0,
> cellOffset = 0, idxParent = 0, nFree = 0, nCell = 0, aOvfl = {{pCell =
> 0x0,
> idx = 0}, {pCell = 0x0, idx = 0}, {pCell = 0x0, idx = 0}, {pCell =
> 0x0,
> idx = 0}, {pCell = 0x0, idx = 0}}, pBt = 0xa5b0f6a8,
> aData = 0xa5b4fe38 "SQLite format 3", pgno = 1, pParent = 0x0}
> (gdb)
> (gdb) up
> #2 0x00b006d8 in relocatePage (pBt=0xa5b0f6a8, pDbPage=0xa5b53468,
> eType=5 '\005', iPtrPage=3, iFreePage=16)
> at /home/ron/sqlite-3.3.8/src/btree.c:2192
> 2192 Pgno childPgno = get4byte(&pPage->aData[pPage->hdrOffset+8]);
> (gdb) p *pDbPage
> $10 = {isInit = 1 '\001', idxShift = 0 '\0', nOverflow = 0 '\0',
> intKey = 0 '\0', leaf = 0 '\0', zeroData = 0 '\0', leafData = 0 '\0',
> hasData = 1 '\001', hdrOffset = 0 '\0', childPtrSize = 4 '\004',
> maxLocal = 230, minLocal = 103, cellOffset = 12, idxParent = 0,
> nFree = 65524, nCell = 0, aOvfl = {{pCell = 0x0, idx = 0}, {pCell = 0x0,
> idx = 0}, {pCell = 0x0, idx = 0}, {pCell = 0x0, idx = 0}, {pCell =
> 0x0,
> idx = 0}}, pBt = 0xa5b0f6a8, aData = 0xa5b53068 "", pgno = 16,
> pParent = 0x0}
> (gdb) up
> #3 0x00b009a6 in sqlite3BtreeSync (p=0xa5b11100, zMaster=0x0)
> at /home/ron/sqlite-3.3.8/src/btree.c:2452
> 2452 rc = relocatePage(pBt, pDbMemPage, eType, iPtrPage, iFreePage);
> (gdb) p *p
> $12 = {pSqlite = 0xa5b112c8, pBt = 0xa5b0f6a8, inTrans = 2 '\002'}
> (gdb)
>
> Any help will be greatly appreciated.
> Thanks,
> Ron
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
