Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
#x27;, instead of a 'while do.' -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Thursday, May 28, 2009 7:19 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recov

Re: [sqlite] corrupt database recovery

2009-05-28 Thread John Machin
On 29/05/2009 10:18 AM, John Machin wrote: > On 29/05/2009 9:34 AM, Gene Allen wrote: >> Yeah. >> >> Since my code works in blocks, read/compress/encrypt/write, loop. Almost >> all the real data was being written to the compressed file, however any >> finalization and flushing of the stream wasn

Re: [sqlite] corrupt database recovery

2009-05-28 Thread John Machin
On 29/05/2009 9:34 AM, Gene Allen wrote: > Yeah. > > Since my code works in blocks, read/compress/encrypt/write, loop. Almost > all the real data was being written to the compressed file, however any > finalization and flushing of the stream wasn't occurring (since the encrypt > was failing) a

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Allen Fowler
> > Since my code works in blocks, read/compress/encrypt/write, loop. Almost > all the real data was being written to the compressed file, however any > finalization and flushing of the stream wasn't occurring (since the encrypt > was failing) so the last bit of any SQLite database wouldn't be

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
SQLite Database; kennethinbox-sql...@yahoo.com Subject: Re: [sqlite] corrupt database recovery > I think I found my defect: my old stress tests was based on doing > compression/encryptions/decryption/decompression passes on files of random > sizes; so I would do about a 10 million passes or so

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Allen Fowler
> I think I found my defect: my old stress tests was based on doing > compression/encryptions/decryption/decompression passes on files of random > sizes; so I would do about a 10 million passes or so and say...that's pretty > good. > > Well...a more structured test exposed the problem and it w

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Gene Allen
ur help and advice. Gene -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Ken Sent: Thursday, May 28, 2009 11:32 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery Gene, Im sure others have sugg

Re: [sqlite] corrupt database recovery

2009-05-28 Thread Ken
Gene, Im sure others have suggested, but have you tried running your code through valgrind? Can you remove the custom VFS ? --- On Wed, 5/27/09, Gene Allen wrote: > From: Gene Allen > Subject: Re: [sqlite] corrupt database recovery > To: mgr...@medcom-online.de, "'Ge

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
x27;m running a large test on the encryption/compression code to > see > if I can get it to break. > > -Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm > Sent: Wednesday, May 27, 2009 5:09

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
'General Discussion of SQLite Database' Subject: Re: [sqlite] corrupt database recovery I'm sorry that I can't help with the SQLite part, but couldn't you write a simple test harness that would create a file (of random size?) with random contents, encrypt/compress to a

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
alf Of Marcus Grimm Sent: Wednesday, May 27, 2009 5:09 PM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] corrupt database recovery sorry, I don't know where sqlite stores any kind of data like index or raw table, my suspect is that, since sqlite usually looks for a

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Doug
n it. > > > > I'm sure that it's a coding error on my part. SQLite is very stable, > in > > my > > opinion. I'm just trying to get a rough idea on where I'm screwing > up the > > database. > > > > > > -Original Mess

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
rethink my approach. > > > -Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm > Sent: Wednesday, May 27, 2009 2:44 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt da

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
ll have to rethink my approach. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Marcus Grimm Sent: Wednesday, May 27, 2009 2:44 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery so, if you thin

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Marcus Grimm
John Elrick > Sent: Wednesday, May 27, 2009 12:58 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > What platform? Any chance they are using a network drive? > > > John > > Gene wrote: >> My code is outside th

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
-Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Elrick Sent: Wednesday, May 27, 2009 12:58 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery What platform? Any chance they

Re: [sqlite] corrupt database recovery

2009-05-27 Thread John Elrick
---Original Message- > From: sqlite-users-boun...@sqlite.org > [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Elrick > Sent: Wednesday, May 27, 2009 10:59 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > Gene All

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Jim Wilcoxson
John Elrick > Sent: Wednesday, May 27, 2009 10:59 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] corrupt database recovery > > Gene Allen wrote: >> Ok...it's happened again and I've decided that I need to track this down >> once and for all! >

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene
Sent: Wednesday, May 27, 2009 10:59 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery Gene Allen wrote: > Ok...it's happened again and I've decided that I need to track this down > once and for all! > > Here is what I'm seeing

Re: [sqlite] corrupt database recovery

2009-05-27 Thread John Elrick
Gene Allen wrote: > Ok...it's happened again and I've decided that I need to track this down > once and for all! > > Here is what I'm seeing: I get errors when I do a integrity_check (see > below), but I can .dump it to a text file and then .read it into another > database ok. > > It seems to me th

Re: [sqlite] corrupt database recovery

2009-05-27 Thread Gene Allen
.@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 8:15 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery On 26/04/2009 5:47 AM, Gene wrote: > Every now and again, we have a database that gets corrupt in the field (bad > coding o

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Jim Dodgen
Cool solution. On Sun, Apr 26, 2009 at 6:39 AM, Jim Wilcoxson wrote: > You could do a binary search to find the highest accessible rowid: > >    select rowid where rowid = 2147483647 (fails) >    select rowid where rowid = 1073741824 (fails) >    ... >    select rowid where rowid = 65536 (fails)

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Jim Wilcoxson
You could do a binary search to find the highest accessible rowid: select rowid where rowid = 2147483647 (fails) select rowid where rowid = 1073741824 (fails) ... select rowid where rowid = 65536 (fails) select rowid where rowid = 32768 (works!) select rowid where rowid = 4

Re: [sqlite] corrupt database recovery

2009-04-26 Thread Tito Ciuro
Hi Gene, On Apr 25, 2009, at 3:47 PM, Gene wrote: > Every now and again, we have a database that gets corrupt in the > field (bad > coding on our end, not sqlite). How do you corrupt a database with bad coding? Just curious... -- Tito ___ sqlite-use

Re: [sqlite] corrupt database recovery

2009-04-25 Thread Gene Allen
t build my insert on the fly. Thank you. Gene -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 9:30 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt data

Re: [sqlite] corrupt database recovery

2009-04-25 Thread John Machin
On 26/04/2009 11:28 AM, Gene wrote: > You are exactly right John...that is indeed what the code looks > like...except we have over 25 columns (it's a flat table). I presume that you are referring to this: """ Write a script that loops around doing 'select * from mytable where rowid = ?' on a c

Re: [sqlite] corrupt database recovery

2009-04-25 Thread Gene
hanks for you comments John! -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of John Machin Sent: Saturday, April 25, 2009 8:15 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] corrupt database recovery On 26/

Re: [sqlite] corrupt database recovery

2009-04-25 Thread John Machin
On 26/04/2009 5:47 AM, Gene wrote: > Every now and again, we have a database that gets corrupt in the field (bad > coding on our end, not sqlite). > Hi Gene, The obvious question: why not fix the bad code? What does 'PRAGMA integrity_check;' say about these corrupt databases? > When we get on