Hi! We're seeing the dreaded "Error: database disk image is malformed"
message, even though the database is actually fine. I've read a few
other threads on it and I'd previously concluded that it was just an
outstanding bug. But I'm wondering if it's actually due to a
combination of disabling synchronous writing AND disabling the shared
cache, and then accessing it from a second process. Have you heard of
this? More details follow:
1) First, we have a server with two threads: the primary thread doing
the main activity, and a second thread doing checkpointing.
2) However, we get erratic behavior when using the sqlite3 command-line
tool to just do a basic select on the database: sometimes it works,
sometimes it returns "Error: database disk image malformed". Sometimes
we just run the same command many times until it works.
3) The main server thread opens the database like:
sqlite3_config( SQLITE_CONFIG_MULTITHREAD )
sqlite3_enable_shared_cache( 0 )
#define DB_OPEN_FLAGS SQLITE_OPEN_READWRITE | \
SQLITE_OPEN_CREATE | \
SQLITE_OPEN_NOMUTEX
sqlite3_open_v2( filename.c_str(), &_db, DB_OPEN_FLAGS, NULL )
PRAGMA legacy_file_format = OFF
PRAGMA synchronous = OFF
PRAGMA count_changes = OFF
PRAGMA temp_store = MEMORY
PRAGMA journal_mode = WAL
PRAGMA wal_autocheckpoint = 0
4) All this works great for server threads, and we have the same exact
commits happening in the same exact order on 5 different servers (using
a custom distributed transaction layer).
5) And this has worked great even with the command-line access for a
very long time -- years now, in fact. But as the database grows (>40GB)
and activity increases, we've started to get the malformed error more
and more frequently -- to the point that it's also impossible to query
the database from the command line anymore.
6) I had just accepted this, but recently we did queries on two servers
via the sqlite3 command-line, and found they were giving back *very*
different results -- for several minutes. Then I'm guessing both
flushed their changes to disk, after which they gave the same result.
This made me think "Hm, maybe the command line process isn't able to
access the in-memory cached data of the server process, and maybe that's
what's causing the malformed error?"
Anyway, that's the background -- has anybody encountered this before?
I'd welcome any advice. Thanks!
-david
Founder and CEO of Expensify
Follow us at http://twitter.com/expensify
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
