Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2015-01-08 Thread David Barrett
Sorry for the slow response -- yes, this is great logic. We're just disabling vacuum. Thanks! -david On Mon, Dec 8, 2014 at 6:18 PM, Simon Slavin wrote: > > On 9 Dec 2014, at 1:36am, David Barrett wrote: > > > *Re: Why VACUUM.* We vacuum weekly. This particular database is a > > "rolling j

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-09 Thread Hadley Wickham
> Also, SSD drives wear out fast. We don't have good figures yet for > mass-produced drives (manufacturers introduce new models faster than the old > ones wear out, so it's hard to gather stats) but typical figures show a drive > failing in from 2,000 to 3,000 write cycles of each single block.

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-09 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/2014 05:36 PM, David Barrett wrote: > *Re: "a simple way is to sleep in the progress callback"* -- Can > you tell me more about this? Are you referring to the callback > provided to sqlite3_exec(), or something else? https://sqlite.org/c3ref

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-09 Thread RSmith
On 2014/12/09 03:36, David Barrett wrote: Hi all, great questions: *Re: Why VACUUM.* We vacuum weekly. This particular database is a "rolling journal" -- we are constantly adding new rows to the end of the table, and every week we truncate off the head of the journal to only keep 3M rows at t

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Simon Slavin
On 9 Dec 2014, at 1:36am, David Barrett wrote: > *Re: Why VACUUM.* We vacuum weekly. This particular database is a > "rolling journal" -- we are constantly adding new rows to the end of the > table, and every week we truncate off the head of the journal to only keep > 3M rows at the "tail". G

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread David Barrett
Hi all, great questions: *Re: Why VACUUM.* We vacuum weekly. This particular database is a "rolling journal" -- we are constantly adding new rows to the end of the table, and every week we truncate off the head of the journal to only keep 3M rows at the "tail". Given that we're truncating the "

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Keith Medcalf
ne knows why. >-Original Message- >From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- >boun...@sqlite.org] On Behalf Of Roger Binns >Sent: Monday, 8 December, 2014 16:16 >To: General Discussion of SQLite Database >Subject: Re: [sqlite] Artificially slow VACUUM by i

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/08/2014 01:35 PM, Max Vlasov wrote: > I wonder whether I/O "sleeping" possible in the first place. In this particular case the OP wants to vacuum while the machine is doing other I/O activity unrelated to the vacuum. Having more sleeping during

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Max Vlasov
On Mon, Dec 8, 2014 at 5:56 PM, Roger Binns wrote: > On 12/07/2014 04:43 PM, David Barrett wrote: >> so I'm curious if you can think of a way using the API (or any >> other way) to essentially "nice" the process by inserting a short >> "sleep" into whatever loop runs inside the VACUUM command. > >

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/07/2014 04:43 PM, David Barrett wrote: > so I'm curious if you can think of a way using the API (or any > other way) to essentially "nice" the process by inserting a short > "sleep" into whatever loop runs inside the VACUUM command. Using OS pro

Re: [sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-07 Thread Simon Slavin
On 8 Dec 2014, at 12:43am, David Barrett wrote: > Other alternatives we're considering are to fork and nice the process, or > to call ioprio_set() directly, but I'm curious if there's a simpler way to > do it. Thanks! VACUUM does the same job (in a very different way) as copying all the TABLEs

[sqlite] Artificially slow VACUUM by injecting a sleep() somewhere?

2014-12-07 Thread David Barrett
Hi! I have a large database, and running the VACUUM pragma hammers disk IO so badly that it dramatically affects performance on the box. I'm not in a hurry for the results, so I'm curious if you can think of a way using the API (or any other way) to essentially "nice" the process by inserting a s