Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread LacaK via Lazarus
But now another issue is coming up: If I increase the number of records to 40,000 population of records slows down after about 10,000 records, speeds up again, and comes to an apparant stand-still for 32900 records. After waiting some time the record counter (which is incremented in steps of

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread Werner Pamler via Lazarus
Am 27.03.2017 um 10:59 schrieb LacaK via Lazarus: But now another issue is coming up: If I increase the number of records to 40,000 population of records slows down after about 10,000 records, speeds up again, and comes to an apparant stand-still for 32900 records. After waiting some time the

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread LacaK via Lazarus
Try call FExportDataset.MergeChangeLog before: WriteLn('Saving...'); Does anything in your timing changed ? Ah - that's it. TBufDataset saves the records instantly now. Probably, this should go into the official wiki site for TBufDataset. But now another issue is coming up: If I

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread Werner Pamler via Lazarus
Am 27.03.2017 um 10:13 schrieb LacaK via Lazarus: Try call FExportDataset.MergeChangeLog before: WriteLn('Saving...'); Does anything in your timing changed ? Ah - that's it. TBufDataset saves the records instantly now. Probably, this should go into the official wiki site for TBufDataset.

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread LacaK via Lazarus
Try call FExportDataset.MergeChangeLog before: WriteLn('Saving...'); Does anything in your timing changed ? -Laco. -- ___ Lazarus mailing list Lazarus@lists.lazarus-ide.org http://lists.lazarus-ide.org/listinfo/lazarus

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread Werner Pamler via Lazarus
Am 27.03.2017 um 00:53 schrieb Howard Page-Clark via Lazarus: I can get small performance increases by - avoiding FieldByName() calls and using AppendRecord No, at least not for the issue I am referring to. Like in the answer to Marc's comment: This happens while the table is populated, but the

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread Werner Pamler via Lazarus
Am 27.03.2017 um 09:07 schrieb Marc Santhoff via Lazarus: I didn't count, but you make extensive use of the Random() function. Could that be the cause of slowness? No, Random() is called only while records are populated - this step is completed without any noticable delay. Time is measured

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-27 Thread Marc Santhoff via Lazarus
On So, 2017-03-26 at 23:53 +0200, Werner Pamler via Lazarus wrote: > Trying to extend the import/export example of fpspreadsheet from a dBase > table to a TBufDataset I came across this issue with TBufDataset: While > data are posted to the database as quickly as usual writing to file > takes

Re: [Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-26 Thread Howard Page-Clark via Lazarus
On 26/03/17 22:53, Werner Pamler via Lazarus wrote: Trying to extend the import/export example of fpspreadsheet from a dBase table to a TBufDataset I came across this issue with TBufDataset: While data are posted to the database as quickly as usual writing to file takes extremely long if there

[Lazarus] Writing >1000 TBufDataset records to file is extremely slow

2017-03-26 Thread Werner Pamler via Lazarus
Trying to extend the import/export example of fpspreadsheet from a dBase table to a TBufDataset I came across this issue with TBufDataset: While data are posted to the database as quickly as usual writing to file takes extremely long if there are more than a few thousand records. Run the demo