Hi sqlite experts,
I am currently incorporating the sqlite online backup API (version 3.8.0.2)
into our product. I meet a tricky problem, where the sqlite3_backup_step hangs
at acquiring the BTree’s lock of the destination file.
Specifically, I am backing up in-memory database to NFS disk file. The
in-memory database is manipulated in the main thread (T1) while the back-up
runs in another thread (T2). There are only 2 threads in the process and there
is no other process existing in our product environment.
The stack trace I got in the T2 when the hanging happens is as following:
#0 0x0000003de740d654 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x0000003de7408f4a in _L_lock_1034 () from /lib64/libpthread.so.0
#2 0x0000003de7408e0c in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00002aaaaacc20f6 in pthreadMutexEnter (p=0x23084300) at sqlite3.c:18299
#4 0x00002aaaaacc1f31 in sqlite3_mutex_enter (p=0x23084300) at sqlite3.c:17812
#5 0x00002aaaaacd9e4f in lockBtreeMutex (p=0x2aaad8001040) at sqlite3.c:49323
#6 0x00002aaaaacd9f49 in sqlite3BtreeEnter (p=0x2aaad8001040) at
sqlite3.c:49410
#7 0x00002aaaaacdd970 in sqlite3BtreeBeginTrans (p=0x2aaad8001040, wrflag=2)
at sqlite3.c:52240
#8 0x00002aaaaace71b7 in sqlite3_backup_step (p=0x2aaad8000fc0, nPage=50) at
sqlite3.c:58451
#9 0x00002aaace885b37 in ResDB::backup (this=0x2314c3f0, numBlocks=10,
numPagesPerBlock=50, blockIntervalInMillisec=50) at dbEngine.cpp:659
#10 0x00002aaace977373 in ResDBBackupManager::process (this=0x23030880) at
dbBackupManager.cpp:53
#11 0x00002aaace976e3a in DBBackupEventHandler::event (this=0x23040870,
event=0x2aaad8000990) at dbBackupEventHandler.cpp:35
The sqlite3_backup_step hangs at the very first call and it cannot go any
further so that nothing is actually back-uped and the file on-disk is still
empty, which disappoints me very much.
-rw-rw-r-- 1 yhzhan cadence3 0 2016-07-19 06:59 Interactive.3.rdb
Take a look at the code of sqlite3_backup_step. It looks there are 2 locks for
both source and destination database. One is for the database itself and the
other is for the tables (Btree). The flow hangs at sqlite3BtreeBeginTrans
SQLITE_API int sqlite3_backup_step(sqlite3_backup *p, int nPage){
sqlite3_mutex_enter(p->pSrcDb->mutex);
sqlite3BtreeEnter(p->pSrc);
if( p->pDestDb ){
sqlite3_mutex_enter(p->pDestDb->mutex);
}
……
/* Lock the destination database, if it is not locked already. */
if( SQLITE_OK==rc && p->bDestLocked==0
&& SQLITE_OK==(rc = sqlite3BtreeBeginTrans(p->pDest, 2))
){
p->bDestLocked = 1;
sqlite3BtreeGetMeta(p->pDest, BTREE_SCHEMA_VERSION, &p->iDestSchema);
}
Also tried the latest version of sqlite 3.13.0.0, the problem remains.
My back-upcode is quite simple, almost the duplicate of online backup api
example https://www.sqlite.org/backup.html
bool ResDB::backup(int numBlocks, int numPagesPerBlock, int
blockIntervalInMillisec)
{
if (!m_sqlBackupDestObj) {
int rc = sqlite3_open(m_fileNameOnNFS.c_str(), &m_sqlBackupDestObj);
Q_ASSERT(rc == SQLITE_OK);
}
if (!m_backupHandle) {
m_backupHandle = sqlite3_backup_init(m_sqlBackupDestObj, "main",
m_sqlObj, "main");
Q_ASSERT(m_backupHandle);
}
bool finished = false;
int i = 0;
while (!finished && i < numBlocks) {
while (true) {
int rc = sqlite3_backup_step(m_backupHandle, numPagesPerBlock);
if (rc == SQLITE_OK) {
sqlite3_sleep(blockIntervalInMillisec);
++i;
break;
} else if (rc == SQLITE_DONE) {
finished = true;
break;
} else if (rc == SQLITE_LOCKED) {
sqlite3_sleep(blockIntervalInMillisec);
continue;
} else if (rc == SQLITE_BUSY) {
sqlite3_sleep(blockIntervalInMillisec);
continue;
} else {
finished = true;
Q_ASSERT(false);
break;
}
}
}
if (finished) {
sqlite3_backup_finish(m_backupHandle);
m_backupHandle = NULL;
sqlite3_close(m_sqlBackupDestObj);
m_sqlBackupDestObj = NULL;
}
return finished;
}
So I am wondering what factor will result in lockBtreeMutex hang? I am not sure
whether there is any misuse of the online backup api. Currently I am clueless.
Do you have any suggestions?
_______________________________________________
sqlite-users mailing list
[email protected]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
