Re: [sqlite] Database corruption issue

2014-04-14 Thread Grzegorz Sikorski
Hi, Actually we had data=writeback originally and problem also appeared. Our system has to be extremely stable and power loss can not cause any data damage at any time. We may deal with some latest data loss in such case, but no corruption is allowed. Thus, we decided to put journaling filesy

Re: [sqlite] Database corruption issue

2014-04-13 Thread Florian Weimer
* Grzegorz Sikorski: > We do fsck on the startup. Occasionally, there are some errors, so we > decided to do 'fsck -p' to fix them before mounting the > filesystem. Here is how we then mount the actual filesystem: > /dev/mmcblk0p2 on /media/DATA type ext4 > (rw,relatime,barrier=1,journal_checksum,

Re: [sqlite] Database corruption issue

2014-04-11 Thread Grzegorz Sikorski
Hi, Thanks you for your response, that is really helpful. Actually, I will do completely the other way, so setup my database with frequent checkpoints and process exiting occasionally, but for test purposes to track the issue down. At the moment the problem is hardly reproducible (one occurre

Re: [sqlite] Database corruption issue

2014-04-11 Thread Stephan Beal
On Fri, Apr 11, 2014 at 1:38 PM, Simon Slavin wrote: > I seem to recall that the sqlite3_open() call dos not really open the > database. The open actually happens when the data is > first accessed. So to do the above "one process that opens the > database" does one need to do some kind of acces

Re: [sqlite] Database corruption issue

2014-04-11 Thread Simon Slavin
On 11 Apr 2014, at 11:49am, Grzegorz Sikorski wrote: > No, the journal files (-wal/-shm) are not modified at all. The only think > that may be done (but rarely) is changing its privileges to rw-rw-r--. We > need to be able to open database in read-only mode by group and as far as I > know the

Re: [sqlite] Database corruption issue

2014-04-11 Thread Richard Hipp
On Fri, Apr 11, 2014 at 6:55 AM, Grzegorz Sikorski wrote: > Hi Richard, > > We do fsck on the startup. Occasionally, there are some errors, so we > decided to do 'fsck -p' to fix them before mounting the filesystem. Here is > how we then mount the actual filesystem: > /dev/mmcblk0p2 on /media/DATA

Re: [sqlite] Database corruption issue

2014-04-11 Thread Grzegorz Sikorski
Hi Richard, We do fsck on the startup. Occasionally, there are some errors, so we decided to do 'fsck -p' to fix them before mounting the filesystem. Here is how we then mount the actual filesystem: /dev/mmcblk0p2 on /media/DATA type ext4 (rw,relatime,barrier=1,journal_checksum,nodelalloc,data

Re: [sqlite] Database corruption issue

2014-04-11 Thread Grzegorz Sikorski
Hi Simon, See my comments below. Thank you for your time, Greg On 10/04/14 18:25, Simon Slavin wrote: On 10 Apr 2014, at 5:04pm, Grzegorz Sikorski wrote: The problem is we occasionally observe database corruption and whole database becomes rubbish. The problem is very rare and occurs only

Re: [sqlite] Database corruption issue

2014-04-10 Thread Richard Hipp
On Thu, Apr 10, 2014 at 12:04 PM, Grzegorz Sikorski wrote: > Hi, > > We developed an application which base on SQLite3. It is running on ARM > processor with Linux 2.6.37 (no, there is no easy way to upgrade it). We > put our database on microSD card (industrial grade, very reliable) > formatted

Re: [sqlite] Database corruption issue

2014-04-10 Thread Simon Slavin
On 10 Apr 2014, at 5:04pm, Grzegorz Sikorski wrote: > The problem is we occasionally observe database corruption and whole database > becomes rubbish. The problem is very rare and occurs only (as far as we were > able to confirm so far) if there is a power lose during write to the database. A

[sqlite] Database corruption issue

2014-04-10 Thread Grzegorz Sikorski
Hi, We developed an application which base on SQLite3. It is running on ARM processor with Linux 2.6.37 (no, there is no easy way to upgrade it). We put our database on microSD card (industrial grade, very reliable) formatted with ext4 partition. The database is used in WAL mode with default