Phil,
Removing the journalling will certainly cause you lots of grief in the event of
a "crash"...
You could do the following,
The write code (inserts) will queue incoming data into an "array/storage
in memory etc..."
When the first row is captured set a timer.
When either the timer expires or you reach a row limit threshold, write
the data to sqlite in a batched transaction.
This way you get good performance and reliablity! But if you code crashes and
you don't keep persistent what was in memory you'll loose that data.
You'll probably need to do some tuning of the timer/row limits to get a
balance between performance
and potential data loss.
HTH,
Ken
Phil Sherrod <[EMAIL PROTECTED]> wrote: > Do you know that the performance
without doing anything special is
unacceptable?
If I do insertions of a test set of 2000 records using a BEGIN
TRANSACTION/COMMIT around each one, the speed is 20 times slower than doing
additions in a single transaction. I hacked the winSync routine and removed
the call to FlushFileBuffers which forces Windows to write data from its
cache to disk; this immediately sped up the run by a factor of 20 without
making any other changes. In other words, removing the call of
FlushFileBuffers for each transaction made my application run 20 times
faster. A speed factor of 20 is significant.
One solution is to batch up many insertions within the same transaction.
The problem is that in my application insertions come in sporadically along
with read accesses to the data that must be serviced quickly. So it is
tricky to try to batch up insertions within transactions when I don't know
how long it will be before another insertion arrives, and I must deal with a
continuous flow of read accesses interleaved with the insertions.
While transactions and rollback are a wonderful facility, I would like to
see some option to turn them off when performance is the primary
consideration.
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
