Re: [sqlite] Database corruption on Linux ext3

2010-07-17 Thread Chris Wedgwood
On Thu, Jul 15, 2010 at 10:20:22PM +0200, Florian Weimer wrote: > It's a generic Linux problem, not an ext3-specific issue. Until > recently, the Linux block layer had no concept of a sync operation. > Linux basically assumed that all writes were synchronous and ordered, > which they are not if y

Re: [sqlite] Database corruption on Linux ext3

2010-07-15 Thread Florian Weimer
* D. Richard Hipp: > An appliance manufacturer has discovered a database corruption issue > on Linux using ext3. The issue is documented here: > > http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem It's a generic Linux problem, not an ext3-specific issue. Until recently, th

Re: [sqlite] Database corruption on Linux ext3

2010-07-14 Thread Jim Wilcoxson
On Wed, Jul 14, 2010 at 1:35 AM, Roger Binns wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 07/13/2010 05:30 PM, Jim Wilcoxson wrote: > > I don't think this would work, because the problem described is that the > > writes aren't making it to disk. If pages don't make it to disk

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 05:30 PM, Jim Wilcoxson wrote: > I don't think this would work, because the problem described is that the > writes aren't making it to disk. If pages don't make it to disk, the old > pages will be present, with the old, and valid checks

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Scott Hess
On Tue, Jul 13, 2010 at 10:03 PM, Scott Hess wrote: > On Tue, Jul 13, 2010 at 8:24 PM, Simon Slavin wrote: >> It might be useful to figure out whether we're aiming for >> detection or correction.  By 'correction' I don't mean recovery >> of all information, I mean restoring the database to some s

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Scott Hess
On Tue, Jul 13, 2010 at 8:24 PM, Simon Slavin wrote: > It might be useful to figure out whether we're aiming for > detection or correction. By 'correction' I don't mean recovery > of all information, I mean restoring the database to some state > it was in just after a COMMIT took effect. There's

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Simon Slavin
On 14 Jul 2010, at 1:30am, Jim Wilcoxson wrote: > This problem of not doing writes, or doing them in > the wrong order, is a different animal IMO. If writes are not happening, or are happening in the wrong order, you're in trouble. It's almost impossible to figure out how to even detect that h

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Jim Wilcoxson
On Tue, Jul 13, 2010 at 8:06 PM, Roger Binns wrote: > > On 07/13/2010 04:57 PM, Simon Slavin wrote: > > One on each page and one for the entire file that checksums the page > checksums ? > > One for each page plus one of the header would make the most sense, but the > I don't think this would wo

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 04:57 PM, Simon Slavin wrote: > One on each page and one for the entire file that checksums the page > checksums ? One for each page plus one of the header would make the most sense, but the overriding concern would be something that is

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Simon Slavin
On 13 Jul 2010, at 9:26pm, Roger Binns wrote: > On 07/13/2010 12:59 PM, D. Richard Hipp wrote: >> You are encouraged to submit comments, insights, criticism, and >> analysis to this mailing list. Thanks. > > Have you considered adding internal checksums to SQLite files so that at the > very l

Re: [sqlite] Database corruption on Linux ext3

2010-07-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/13/2010 12:59 PM, D. Richard Hipp wrote: > You are encouraged to submit comments, insights, criticism, and > analysis to this mailing list. Thanks. Have you considered adding internal checksums to SQLite files so that at the very least corrup

[sqlite] Database corruption on Linux ext3

2010-07-13 Thread D. Richard Hipp
An appliance manufacturer has discovered a database corruption issue on Linux using ext3. The issue is documented here: http://www.sqlite.org/draft/lockingv3.html#ext3-barrier-problem You are encouraged to submit comments, insights, criticism, and analysis to this mailing list. Thanks