Stuart Johnston wrote:
Writes are slow if you do one insert per transaction.
http://search.cpan.org/perldoc?DBD::SQLite#Performance
Just think of the number of round trips you are doing in this case so
27min is not too bad.
We have 300000+ reads then which each insert there could be I think at
least
3 round trips
1) prepare
2) bind
3) execute
that is allot of round trips;).
cheers
John Scoles
Owen wrote:
I have a sqlite (version 3.6.13) database with 300000+ rows
1500 of these rows have faulty data.
So I read the main database row by row, tested the validity of one
column, and inserted the good rows into a new database.
This process took 27 minutes on a fairly recent desktop computer.
( about 10000 rows a minute or 170 a second )
I thought the whole process would take a few seconds, so I wonder if 27
minutes is a reasonable time for this database creation.
As the database was being created, I noticed a journal file was in use
though I haven't worked out what that did, yet.
TIA
Owen
