Re: [sqlite] Database corruption check.

On 4/15/19, Tim Streater wrote: >> >> This command was added to the command-line tool recently. > > 3.19.3 has it - that's almost two years ago. > The .selftest command was added on 2017-03-09 by https://www.sqlite.org/src/timeline?c=f4fcd46f08ba59d2 and hence as likely first in release 3.18.0

Re: [sqlite] Database corruption check.

On 15 Apr 2019, at 11:02, Simon Slavin wrote: > On 15 Apr 2019, at 10:36am, Lullaby Dayal wrote: > >> Thank you very much for your response. The link seems to be helpful. But I >> fail to run the .selftest command from my sqlite3 prompt. I got the error: >> unknown command or invalid arguments

Re: [sqlite] Database corruption check.

On Mon, Apr 15, 2019 at 11:37 AM Lullaby Dayal wrote: > [...]. But I fail to run the .selftest command from my sqlite3 prompt. I > got the error: > unknown command or invalid arguments error. > That code dates back to July 2017. So you must have a very old version. > I am a newbie in SQLite.

Re: [sqlite] Database corruption check.

On 15 Apr 2019, at 10:36am, Lullaby Dayal wrote: > Thank you very much for your response. The link seems to be helpful. But I > fail to run the .selftest command from my sqlite3 prompt. I got the error: > unknown command or invalid arguments error. This command was added to the command-line

Re: [sqlite] Database corruption check.

Hi Richard, Thank you very much for your response. The link seems to be helpful. But I fail to run the .selftest command from my sqlite3 prompt. I got the error: unknown command or invalid arguments error. I am a newbie in SQLite. I am not sure how can I get this to working in our application

Re: [sqlite] Database corruption check.

On 4/14/19, Lullaby Dayal wrote: > > For Sqlite database, as per my understanding, implementing pragma > integrity_check won't guarantee all errors to be detected. Maybe you are confused with "PRAGMA quick_check"? The "PRAGMA integrity_check" takes a little longer, but does a better job. There

[sqlite] Database corruption check.

Hi, We are using sqlite for our embedded automotive system based on QNX. We have a requirement to check whether database is corrupted on start-up and replace it with default database if such a scenario happens. For Sqlite database, as per my understanding, implementing pragma integrity_check

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

I'm using SQLCipher in my main app and it's using SQLite 3.15.2. However, my little sample app where I could also easily demonstrate the problem, is using whatever the built-in SQLite version is in macOS Sierra 10.12.4. I couldn't find SQLITE_MMAP_READWRITE in the Couchbase Lite source anywhere,

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 04/19/2017 02:42 AM, Jens Alfke wrote: On Apr 18, 2017, at 2:20 AM, Deon Brewis wrote: It's not like it was subtle - it's a dead on repro. I was able to repro this by doing a power cycle 2 hours after shutting the app down. OSX didn't seem to have any interest in

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Apr 18, 2017, at 9:46 PM, Deon Brewis wrote: > > I did report it: > http://sqlite.1065341.n5.nabble.com/SQLITE-vs-OSX-mmap-inevitable-catalog-corruption-td85620.html Thanks for the pointer to the thread. There was a reply by Dan Kennedy shortly thereafter: >> As of

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

ct: Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS > On Apr 18, 2017, at 1:50 AM, Deon Brewis <de...@outlook.com> wrote: > > Are you by change using memory mapped IO (MMAP_SIZE something other than 0)? > > This does not work on OSX. Not even remotely. I track

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Apr 18, 2017, at 2:35 PM, Bob Friesenhahn > wrote: > > If the filesystem implementation is not fully-coherent, then data written by > programmed file I/O may not be reflected in the memory mapped space, > resulting in programs using something other than

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On Tue, 18 Apr 2017, Jens Alfke wrote: On Apr 18, 2017, at 2:20 AM, Deon Brewis wrote: It's not like it was subtle - it's a dead on repro. I was able to repro this by doing a power cycle 2 hours after shutting the app down. OSX didn't seem to have any interest in

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hi Jens, It would be a good test if you could independently verify my findings using the sample app I wrote to see if you are able to reproduce the corruption with memory mapped I/O turned ON and the fact that there's no corruption when it's turned OFF. I know it seem strange given the

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Apr 18, 2017, at 12:55 PM, Jeffrey Mattox wrote: > > I'm an iOS and macOS developer. Mac app bundles are special in other ways > beside just having a bit set. Brendan is not storing his database in the app bundle. Both he and Simon have already said that. —Jens

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

I'm an iOS and macOS developer. Mac app bundles are special in other ways beside just having a bit set. For one, there's a security check somewhere that verifies that the app bundle has not been changed, as those files are expected to be read-only. Apple says apps should put their data

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Apr 18, 2017, at 2:20 AM, Deon Brewis wrote: > > It's not like it was subtle - it's a dead on repro. I was able to repro this > by doing a power cycle 2 hours after shutting the app down. OSX didn't seem > to have any interest in flushing mmap files until you soft

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 18 Apr 2017, at 7:37pm, Brendan Duddridge wrote: > Well, perhaps the way I've been bundling the db.sqlite file within a > package triggers this particular bug. Although a package is really just a > sub-folder, so I don't see how that would make a difference. I don’t

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> I’m the architect of Couchbase Lite and the lead developer for iOS and > macOS. I enabled SQLite’s memory-mapped I/O at least two years ago. I never > considered it could be problematic since (a) Brendan is IIRC the only > iOS/Mac developer who’s reported database corruption, and (b) I assumed >

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 18 Apr 2017, at 6:46pm, Jens Alfke wrote: > This is a statement about hard disk controller firmware and is true for any > OS. (I used to work for Apple, and corresponded with filesystem architect > Dominic Giampaolo about this back in the day.) Some disk controllers

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Apr 18, 2017, at 1:50 AM, Deon Brewis wrote: > > Are you by change using memory mapped IO (MMAP_SIZE something other than 0)? > > This does not work on OSX. Not even remotely. I tracked an issue down in > November 2015, and was able to trivially corrupt a database 100%

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 18 Apr 2017, at 11:52am, Richard Hipp wrote: > I was unaware that memory-mapped I/O was busted on MacOS. I will > investigate further, but probably the solution will be that we will > completely disable memory-mapped I/O on MacOS, just as we have had to > do for OpenBSD.

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 4/18/17, Brendan Duddridge wrote: > > I commented out this line of code in the Couchbase lite CBL_SQliteStorage.m > source file and no more corruption > > //int err = sqlite3_config(SQLITE_CONFIG_MMAP_SIZE, > (SInt64)kSQLiteMMapSize, (SInt64)-1); > > > It would be

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Behalf Of Simon Slavin > Sent: Tuesday, April 18, 2017 1:57 AM > To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> > Subject: Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS > > > On 18 Apr 2017, at 9:50am, Deon Brewis <de...@outlook.com> wrote

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

org> Subject: Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS On 18 Apr 2017, at 9:50am, Deon Brewis <de...@outlook.com> wrote: > “From the OSX documentation: > > Note that while fsync() will flush all data from the host to the drive (i.e. > the "perm

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 18 Apr 2017, at 10:07am, Brendan Duddridge wrote: > >// Enable memory-mapped I/O if available Please try an alternative version of that code which definitely does not use memory mapping in any way. You may be able to do it by changing that IFDEF sequence. Or you

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Message- > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > On Behalf Of Brendan Duddridge > Sent: Tuesday, April 18, 2017 1:36 AM > To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> > Subject: Re: [sqlite] Database corruption, and PRAGMA fullfsync

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

emonstration code which uses the SQLite API directly we can attempt > to debug. > > If the problem you need solving is actually with Couchbase, you can solve > it by upgrading to any version from 2.0 onwards. Since that no longer uses > SQLite, SQLite database corruption can no longer be a problem.

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 18 Apr 2017, at 9:50am, Deon Brewis wrote: > “From the OSX documentation: > > Note that while fsync() will flush all data from the host to the drive (i.e. > the "permanent storage device"), Deon, I’m not sure this is related, but have you seen

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hi Richard, I just did another test with my PowerFailureTest app. I launched it and the count of rows in it was 4402. Without even running the function to import my data that writes to the SQLite file, I cut the power (holding down the power button on my MBP). When my Mac finally rebooted, I saw

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

y just a folder with one extra bit set. Any demonstration code which uses the SQLite API directly we can attempt to debug. If the problem you need solving is actually with Couchbase, you can solve it by upgrading to any version from 2.0 onwards. Since that no longer uses SQLite, SQLite

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

lite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Brendan Duddridge Sent: Tuesday, April 18, 2017 1:36 AM To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS Hi Rowan, My apologies. My

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hi Rowan, My apologies. My sample app does use NSDocument. But my production app doesn't. And they both exhibit the same problem. On Tue, Apr 18, 2017 at 12:29 AM, Rowan Worth wrote: > Hey Brendan, > > I'm no OSX expert, but from what I've read this afternoon about NSDocument >

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hey Rowan, In my Mac app I'm actually not using NSDocument. I used to and then I changed my document subclass to extend NSObject instead, just in case NSDocument was doing bad things behind my back. I had also disabled auto-save and versions anyway. For a while though I tried extending

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 4/18/17, Brendan Duddridge wrote: > In both cases though I can cause SQLite file corruption by cutting the > power on my MacBook Pro. Corruption-by-power-loss problems tend to be very sensitive to timing. If you are able to reliably generate the problem by leisurely

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hey Brendan, I'm no OSX expert, but from what I've read this afternoon about NSDocument and friends combined with what I know about sqlite I have to say you are completely mad to continue passing sqlite databases to NSDocument, *especially* as you don't define your own sub-class to do any of the

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Thanks Richard for your reply. Sorry about the COLLATE problem. That's a Couchbase Lite thing. I find it weird that a different WAL file is getting in there somehow when a power failure occurs. I'm a bit stumped at how it can even write to the file system the moment the power shuts down so I

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 4/16/17, Richard Hipp wrote: > I cannot seem to find the implementation for COLLATE JSON anywhere in > your source code. Can you give me a hint as to which source file I > should be looking in? I worked around the "COLLATE JSON" problem (by writing my own JSON collation).

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

I cannot seem to find the implementation for COLLATE JSON anywhere in your source code. Can you give me a hint as to which source file I should be looking in? On 4/16/17, Brendan Duddridge wrote: > Hi, > > I know this is an old thread, but I just found it now when I was

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

Hi, I know this is an old thread, but I just found it now when I was doing some research on this topic. Thanks Jens for starting this thread. When Jens said he knew a developer who could create a corrupted database by turning off the power, I'm pretty sure he was talking about me. This has been

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 11/16/16, Simon Slavin wrote: > > There's also a tool from the SQLite team which can analyze a corrupt SQLite > file and tell some things about how it is corrupt. Unfortunately I can't > remember what it's called or where to find it. But I heard about it on this > list

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 16 Nov 2016, at 7:25am, Jens Alfke wrote: > The case I’m concerned about happens on Macs, as I said, and the developer > says he can trigger it in his app via a power failure or a forced shutdown > (holding the power button for 5 seconds.) This version of our library is

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Nov 15, 2016, at 1:46 PM, Simon Slavin wrote: > > Do these people admit they're letting their phones run out of power ? There’s nothing wrong with letting your phone run out of power, and software should be resilient to it. I don’t think that’s the problem,

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 2016/11/15 10:34 PM, Jens Alfke wrote: On Nov 15, 2016, at 10:57 AM, Simon Slavin wrote: sqlite> PRAGMA checkpoint_fullfsync; 1 I wasn’t aware of that pragma. Just tried it on my Mac (10.12.1), and its value is 1 even if I don’t first set pragma fullfsync; i.e. it

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

At 22:41 15/11/2016, you wrote: So if you're truly worried about flush-to-disk what do you do ? Solution 1 is to buy hard disks rated for servers -- sometimes called "enterprise-class hard drives" -- and to set the DIP switches to tell them they're being used on a server. Those things are

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On Tue, 15 Nov 2016, Simon Slavin wrote: Modern storage subsystems (hard disk or SSD) intended for use in a normal user computer always lie to the OS about flushing to disk. The apparent increase in speed from doing this is so big that every manufacturer has to do it, or risk having every

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 15 Nov 2016, at 8:34pm, Jens Alfke wrote: > (Sorry to be frothing at the mouth about this; but my team’s dealing with a > few users/customers whose apps encounter db corruption, on Android as well as > macOS, and we’re getting really frustrated trying to figure out

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 15 Nov 2016, at 8:18pm, Jens Alfke wrote: > The only way to guarantee a true barrier is to really-and-truly flush the > disk controller, which requires not simply flushing but resetting it. That’s > what F_FULLFSYNC on macOS does. (Unfortunately it makes the disk

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 15 Nov 2016, at 8:34pm, Jens Alfke wrote: > On Nov 15, 2016, at 10:57 AM, Simon Slavin wrote: > >> sqlite> PRAGMA checkpoint_fullfsync; >> 1 > > I wasn’t aware of that pragma. Just tried it on my Mac (10.12.1), and its > value is 1 even if I

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Nov 15, 2016, at 10:57 AM, Simon Slavin wrote: > > sqlite> PRAGMA checkpoint_fullfsync; > 1 I wasn’t aware of that pragma. Just tried it on my Mac (10.12.1), and its value is 1 even if I don’t first set pragma fullfsync; i.e. it defaults to 1. (Contradicting the

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

> On Nov 15, 2016, at 10:57 AM, Simon Slavin wrote: > > My understanding is that F_FULLFSYNC still works the way you describe on a > Mac and SQLite still uses it the way the documentation says. But I'm not in > touch with either development group. This seems like a

Re: [sqlite] Database corruption, and PRAGMA fullfsync on macOS

On 15 Nov 2016, at 6:11pm, Jens Alfke wrote: > I verified in the built-in sqlite3 tool on macOS 10.12.1 that the result of > `pragma fullfsync` is 0. The default setting is 0. But you can change it. On my Mac running 10.12.x, SQLite version 3.14.0 2016-07-26 15:17:14

[sqlite] Database corruption, and PRAGMA fullfsync on macOS

I’m seeing conflicting information about SQLite’s use of F_FULLFSYNC on macOS when committing transactions. This is making me nervous about durability and the possibility of database corruption. The SQLite docs for PRAGMA fullfsync (https://www.sqlite.org/pragma.html#pragma_fullfsync

Re: [sqlite] Database corruption question

On 20 Jul 2016 at 20:29, Richard Hipp wrote: > On 7/19/16, Doug Nebeker wrote: >> there is only one process using the database file (though multiple >> threads, but they each have their own database handle). > > That threading mode

Re: [sqlite] Database corruption question

On 7/19/16, Doug Nebeker wrote: > there is only one process using the database file (though multiple > threads, but they each have their own database handle). That threading mode (https://www.sqlite.org/threadsafe.html) are you using? Are you sure that you are using the

Re: [sqlite] Database corruption question

On 19 Jul 2016, at 5:43pm, Doug Nebeker wrote: > Would it be possible for a file scanning process (anti-virus, backup, etc) to > grab hold of a database file at just the right moment, momentarily blocking a > write or delete, and causing corruption? It might prevent the

[sqlite] Database corruption question

Been using SQLite for a long time and a huge fan. We occasionally see database corruption on a local NTFS Windows drive and I've been trying to figure it out. I finally have some logs from the SQLITE_CONFIG_LOG callback that may be of help: (11) database corruption at line 78267 of

[sqlite] Database corruption using zipvfs on android 64 bits (x86_64)

On 11/06/2015 11:57 PM, Alexandre Mainville wrote: > Hi, > > I am experiencing database corruption using zipvfs on android 64 > bits (x86_64). > The same program executing on 32 bits runs without problems. > > The program creates a new database on every run and does a series of > inserts and

[sqlite] Database corruption using zipvfs on android 64 bits (x86_64)

Alexandre Mainville wrote: > Hi, > > I am experiencing database corruption using zipvfs on android 64 > bits (x86_64). > The same program executing on 32 bits runs without problems. > > The program creates a new database on every run and does a series of > inserts and updates (always the same

[sqlite] Database corruption using zipvfs on android 64 bits (x86_64)

Hi, I am experiencing database corruption using zipvfs on android 64 bits (x86_64). The same program executing on 32 bits runs without problems. The program creates a new database on every run and does a series of inserts and updates (always the same series of operations). Both programs produce

Re: [sqlite] Database corruption issue

Hi, Actually we had data=writeback originally and problem also appeared. Our system has to be extremely stable and power loss can not cause any data damage at any time. We may deal with some latest data loss in such case, but no corruption is allowed. Thus, we decided to put journaling

Re: [sqlite] Database corruption issue

* Grzegorz Sikorski: > We do fsck on the startup. Occasionally, there are some errors, so we > decided to do 'fsck -p' to fix them before mounting the > filesystem. Here is how we then mount the actual filesystem: > /dev/mmcblk0p2 on /media/DATA type ext4 >

Re: [sqlite] Database corruption issue

Hi, Thanks you for your response, that is really helpful. Actually, I will do completely the other way, so setup my database with frequent checkpoints and process exiting occasionally, but for test purposes to track the issue down. At the moment the problem is hardly reproducible (one

Re: [sqlite] Database corruption issue

On Fri, Apr 11, 2014 at 1:38 PM, Simon Slavin wrote: > I seem to recall that the sqlite3_open() call dos not really open the > database. The open actually happens when the data is > first accessed. So to do the above "one process that opens the > database" does one need

Re: [sqlite] Database corruption issue

On 11 Apr 2014, at 11:49am, Grzegorz Sikorski wrote: > No, the journal files (-wal/-shm) are not modified at all. The only think > that may be done (but rarely) is changing its privileges to rw-rw-r--. We > need to be able to open database in read-only mode by group

Re: [sqlite] Database corruption issue

On Fri, Apr 11, 2014 at 6:55 AM, Grzegorz Sikorski wrote: > Hi Richard, > > We do fsck on the startup. Occasionally, there are some errors, so we > decided to do 'fsck -p' to fix them before mounting the filesystem. Here is > how we then mount the actual filesystem: >

Re: [sqlite] Database corruption issue

Hi Richard, We do fsck on the startup. Occasionally, there are some errors, so we decided to do 'fsck -p' to fix them before mounting the filesystem. Here is how we then mount the actual filesystem: /dev/mmcblk0p2 on /media/DATA type ext4

Re: [sqlite] Database corruption issue

Hi Simon, See my comments below. Thank you for your time, Greg On 10/04/14 18:25, Simon Slavin wrote: On 10 Apr 2014, at 5:04pm, Grzegorz Sikorski wrote: The problem is we occasionally observe database corruption and whole database becomes rubbish. The problem is

Re: [sqlite] Database corruption issue

On Thu, Apr 10, 2014 at 12:04 PM, Grzegorz Sikorski wrote: > Hi, > > We developed an application which base on SQLite3. It is running on ARM > processor with Linux 2.6.37 (no, there is no easy way to upgrade it). We > put our database on microSD card (industrial grade,

Re: [sqlite] Database corruption issue

On 10 Apr 2014, at 5:04pm, Grzegorz Sikorski wrote: > The problem is we occasionally observe database corruption and whole database > becomes rubbish. The problem is very rare and occurs only (as far as we were > able to confirm so far) if there is a power lose during

[sqlite] Database corruption issue

Hi, We developed an application which base on SQLite3. It is running on ARM processor with Linux 2.6.37 (no, there is no easy way to upgrade it). We put our database on microSD card (industrial grade, very reliable) formatted with ext4 partition. The database is used in WAL mode with default

Re: [sqlite] Database Corruption - Table data being overwritten

ating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of herb...@gdls.com [herb...@gdls.com] Sent: Tuesday, June 05, 2012 1:38 PM To: sqlite-users@sqlite.org Subject: EXT :[sqlite] Database

Re: [sqlite] Database Corruption - Table data being overwritten

On 6 Jun 2012, at 1:03pm, herb...@gdls.com wrote: > Even more bizarre, a .dump of "table1" will show: > INSERT INTO "table1" VALUES(28274, 6, 10308); > > But this query returns nothing: > SELECT * FROM table1 WHERE table1_id=28274; There should be (at least) two copies of that number in the

[sqlite] Database Corruption - Table data being overwritten

Hello, I have run into a problem using SQLite in an embedded environment. Currently using version 3.7.5. (I know, I want to update, just have been locked in at this version for the time being) The power is routinely cut from the system, so I have JOURNAL_MODE = PERSIST because there were

Re: [sqlite] Database Corruption - Table data being overwritten

On Wed, Jun 6, 2012 at 8:03 AM, wrote: > Hello, > > I have run into a problem using SQLite in an embedded environment. > > Currently using version 3.7.5. (I know, I want to update, just have been > locked in at this version for the time being) > > The power is routinely cut

[sqlite] Database Corruption - Table data being overwritten

Hello, I have run into a problem using SQLite in an embedded environment. Currently using version 3.7.5. (I know, I want to update, just have been locked in at this version for the time being) The power is routinely cut from the system, so I have JOURNAL_MODE = PERSIST because there were

Re: [sqlite] Database corruption with fts table after closing/reopening the database

On 10 Feb 2012, at 11:01am, Julien LF wrote: >> Did you try surrounding them strings with single quotes, instead ? > > I got the same problem using single quotes. > However the machine I was performing those tests on is a vmware virtual > machine. > Performing the same steps on physical

Re: [sqlite] Database corruption with fts table after closing/reopening the database

> Did you try surrounding them strings with single quotes, instead ? I got the same problem using single quotes. However the machine I was performing those tests on is a vmware virtual machine. Performing the same steps on physical hardware (identical os & sqlite version) worked fine, so I guess

Re: [sqlite] Database corruption with fts table after closing/reopening the database

On Fri, Feb 10, 2012 at 5:20 AM, Julien LF wrote: > Hello, > > I'm testing the fts extensions and experiencing a 'database disk image > is malformed' problem that I can reproduce the following way: > - Create a database > - Open the database to insert/update rows.

[sqlite] Database corruption with fts table after closing/reopening the database

Hello, I'm testing the fts extensions and experiencing a 'database disk image is malformed' problem that I can reproduce the following way: - Create a database - Open the database to insert/update rows. 'match' queries work at this point - Close the database - Open the database. 'match' queries

Re: [sqlite] Help sqlite database corruption

...@sqlite.org] on behalf of Singh, Manpreet [manpreet.sing...@ca.com] Sent: Tuesday, June 21, 2011 2:55 AM To: sqlite-...@sqlite.org; sqlite-users@sqlite.org Subject: EXT :Re: [sqlite] Help sqlite database corruption Anyone with clues to the problem "malformed disk image"? F

Re: [sqlite] Help sqlite database corruption

Anyone with clues to the problem "malformed disk image"? From: Singh, Manpreet Sent: Tuesday, May 24, 2011 3:10 PM To: sqlite-...@sqlite.org; d...@hwaci.com; sqlite-users@sqlite.org Cc: Singh, Abhijeet; Singh, Satbeer; Prasanth, Neelapalem Subject: RE: Help sqlite database

Re: [sqlite] Help sqlite database corruption

On Tue, May 24, 2011 at 5:39 AM, Singh, Manpreet wrote: > Hello, > > > > Resending with some more finding any help would be appreciated. One of > the tables in the database is getting corrupted. > http://www.sqlite.org/howtocorrupt.html

Re: [sqlite] Help sqlite database corruption

ubject Matter Expert SQL Certified (Oracle 9i) MCP (SQL 2000 Administration) MS Certified Technology Specialist - SQL Server 2005 From: Singh, Manpreet Sent: Wednesday, May 18, 2011 12:04 PM To: 'sqlite-...@sqlite.org' Subject: Help sqlite database corruption Hello, Looking for help,

[sqlite] SQLite database corruption

Hi, We're experiencing strange database corruptions using SQLite database (we are using it for over three years). We are reporting this now because one of databases got corrupted on less sensitive data which we can give you for analysis and because we wanted to reject all other possibilities of

Re: [sqlite] Database corruption on Linux ext3

On Thu, Jul 15, 2010 at 10:20:22PM +0200, Florian Weimer wrote: > It's a generic Linux problem, not an ext3-specific issue. Until > recently, the Linux block layer had no concept of a sync operation. > Linux basically assumed that all writes were synchronous and ordered, > which they are not if

Re: [sqlite] Database corruption on Linux ext3

* D. Richard Hipp: > An appliance manufacturer has discovered a database corruption issue > on Linux using ext3. The issue is documented here: > > http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem It's a generic Linux problem, not an ext3-specific issue. Until recently,

Re: [sqlite] Database corruption on Linux ext3

On Wed, Jul 14, 2010 at 1:35 AM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07/13/2010 05:30 PM, Jim Wilcoxson wrote: > > I don't think this would work, because the problem described is that the > > writes aren't making it to disk. If

Re: [sqlite] Database corruption on Linux ext3

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 05:30 PM, Jim Wilcoxson wrote: > I don't think this would work, because the problem described is that the > writes aren't making it to disk. If pages don't make it to disk, the old > pages will be present, with the old, and valid

Re: [sqlite] Database corruption on Linux ext3

On Tue, Jul 13, 2010 at 10:03 PM, Scott Hess wrote: > On Tue, Jul 13, 2010 at 8:24 PM, Simon Slavin wrote: >> It might be useful to figure out whether we're aiming for >> detection or correction.  By 'correction' I don't mean recovery >> of all

Re: [sqlite] Database corruption on Linux ext3

On Tue, Jul 13, 2010 at 8:24 PM, Simon Slavin wrote: > It might be useful to figure out whether we're aiming for > detection or correction. By 'correction' I don't mean recovery > of all information, I mean restoring the database to some state > it was in just after a

Re: [sqlite] Database corruption on Linux ext3

On 14 Jul 2010, at 1:30am, Jim Wilcoxson wrote: > This problem of not doing writes, or doing them in > the wrong order, is a different animal IMO. If writes are not happening, or are happening in the wrong order, you're in trouble. It's almost impossible to figure out how to even detect that

Re: [sqlite] Database corruption on Linux ext3

On Tue, Jul 13, 2010 at 8:06 PM, Roger Binns wrote: > > On 07/13/2010 04:57 PM, Simon Slavin wrote: > > One on each page and one for the entire file that checksums the page > checksums ? > > One for each page plus one of the header would make the most sense, but the > I

Re: [sqlite] Database corruption on Linux ext3

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 04:57 PM, Simon Slavin wrote: > One on each page and one for the entire file that checksums the page > checksums ? One for each page plus one of the header would make the most sense, but the overriding concern would be something that

Re: [sqlite] Database corruption on Linux ext3

On 13 Jul 2010, at 9:26pm, Roger Binns wrote: > On 07/13/2010 12:59 PM, D. Richard Hipp wrote: >> You are encouraged to submit comments, insights, criticism, and >> analysis to this mailing list. Thanks. > > Have you considered adding internal checksums to SQLite files so that at the > very

Re: [sqlite] Database corruption on Linux ext3

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 12:59 PM, D. Richard Hipp wrote: > You are encouraged to submit comments, insights, criticism, and > analysis to this mailing list. Thanks. Have you considered adding internal checksums to SQLite files so that at the very least

[sqlite] Database corruption on Linux ext3

An appliance manufacturer has discovered a database corruption issue on Linux using ext3. The issue is documented here: http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem You are encouraged to submit comments, insights, criticism, and analysis to this mailing list.

Re: [sqlite] database corruption problem

On Jun 9, 2010, at 12:51 AM, Dave Segleau wrote: > > On 6/8/2010 9:25 AM, Dan Kennedy wrote: >> >> Those pragmas should not cause a problem. Simon's referring >> to "PRAGMA synchronous". The docs for which explain the >> risks assumed by changing the default setting. >> >>

Re: [sqlite] database corruption problem

On 6/8/2010 9:25 AM, Dan Kennedy wrote: > > Those pragmas should not cause a problem. Simon's referring > to "PRAGMA synchronous". The docs for which explain the > risks assumed by changing the default setting. > > http://www.sqlite.org/pragma.html#pragma_synchronous > > As Dan said,

Re: [sqlite] database corruption problem

On 8 Jun 2010, at 5:25pm, Dan Kennedy wrote: > Those pragmas should not cause a problem. Simon's referring > to "PRAGMA synchronous". The docs for which explain the > risks assumed by changing the default setting. > > http://www.sqlite.org/pragma.html#pragma_synchronous You're both right. I

  1   2   >