Re: [sqlite] Very, very slow commits - Possibly solved

2018-07-31 Thread Rob Willett
Chris, I'll try and summarise. 1. We have a 60GB database collecting data. This database is accessed by a single process once every five mins and around 5MB of data (approx 600-800 rows) is added. Data has never been deleted. 2. The database is getting too big for the server it's hosted on.

Re: [sqlite] Very, very slow commits - Possibly solved

2018-07-31 Thread Chris Locke
I've been following this thread with interest, but this just doesn't make sense... > Logically speaking SQLite shouldn't notice the difference in row order, but things do slow down, > even with analyse. Are you accessing each row via its ID? Even so, that should still be indexed. I thought you

Re: [sqlite] Very, very slow commits - Possibly solved

2018-07-31 Thread Rob Willett
Dear all, We think we have now found the issue with the slow commits. We believe this is due to an inherent (and old) defect in our database design. We think our original design has an implicit ordering of rows in a table, when the table is only increasing this flaw in the design isn't appare