On 26/03/17 22:53, Werner Pamler via Lazarus wrote:
Trying to extend the import/export example of fpspreadsheet from a
dBase table to a TBufDataset I came across this issue with
TBufDataset: While data are posted to the database as quickly as usual
writing to file takes extremely long if there are more than a few
thousand records.
Run the demo attached below. On my system, I measure these
(non-linearly scaling) execution times for writing the TBufDataset
table to file:
1000 records -- 0.9 seconds
2000 records -- 8.8 seconds
3000 records -- 31.1 seconds
etc.
Compared to that, writing of the same data to a dbf file is a wink of
an eye. Is there anything which I am doing wrong? Or should I report a
bug?
I don't think you do anything wrong.
I can get small performance increases by
- avoiding FieldByName() calls and using AppendRecord
- using SaveToFile and avoiding an intermediate memory stream
- increasing the value of PacketRecords
but the speedups are insignificant.
Clearly either the insertion algorithm should be improved, or the
buffering, or the way the buffered records are written to disk. Maybe
all three areas of TBufDataset can be optimised for better performance.
--
_______________________________________________
Lazarus mailing list
Lazarus@lists.lazarus-ide.org
http://lists.lazarus-ide.org/listinfo/lazarus
