Re: [Qgis-developer] memory layer performance while adding many features?

2015-08-05 Thread Matthias Kuhn
Hi all, in general I think we should advice to use the layer.addFeature() method with a layer.commitChanges() in the end. It has the advantage that it does batch updates on the provider automatically and proper signals are sent so the UI (canvas, attribute table...) can reflect changes immediately

Re: [Qgis-developer] memory layer performance while adding many features?

2015-08-05 Thread Raymond Nijssen
Hi Denis, Adding all features at once, or 1000 by 1000, speeds up enormously. It only takes a few minutes, at a steady speed of 480 fpm. Where adding the same dataset 1 by 1 took about 5 hours. Thanks for pointing this out! Raymond On 30-07-15 09:34, Denis Rouzaud wrote: From your code [0

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-30 Thread Raymond Nijssen
Merci Denis, I will try both options later today. Raymond On 30-07-15 09:34, Denis Rouzaud wrote: From your code [0] , I would see 2 options: * add features all at once on the provider using QgsVectorDataProvider::addFeatures (the method you currently use) * work on the layer level rather

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-30 Thread Denis Rouzaud
From your code [0] , I would see 2 options: * add features all at once on the provider using QgsVectorDataProvider::addFeatures (the method you currently use) * work on the layer level rather than the provider, which I believe is the recommended way for plugins. Make the layer editable, and a

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-30 Thread Raymond Nijssen
Hi Hugo, I'm not specifically adding an index. So if you are right, I'm not using that. Regards, Raymond On 30-07-15 09:10, Hugo Mercier wrote: Hi Raymond, On 29/07/2015 09:04, Raymond Nijssen wrote: Dear developers, ;) A plugin of mine imports data from a gml file into a memory layer. W

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-30 Thread Raymond Nijssen
Hi Denis, The code is in the ImaerReader plugin in the qgis repo and in github: https://github.com/opengeogroep/AERIUS-QGIS-plugins I'm adding the features one by one. Regards, Raymond On 30-07-15 08:46, Denis Rouzaud wrote: Hi Raymond, Can you show us the code doing this? Do you commit fea

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-30 Thread Hugo Mercier
Hi Raymond, On 29/07/2015 09:04, Raymond Nijssen wrote: > Dear developers, ;) > > A plugin of mine imports data from a gml file into a memory layer. Works > fine for tiny gml files, but takes forever on huge ones. Sounds > plausible maybe, but the relation is not linear. > > So I did did some te

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-29 Thread Denis Rouzaud
Hi Raymond, Can you show us the code doing this? Do you commit features one by one ar all at once? Best wishes, Denis On 07/29/2015 09:04 AM, Raymond Nijssen wrote: Dear developers, ;) A plugin of mine imports data from a gml file into a memory layer. Works fine for tiny gml files, but takes

Re: [Qgis-developer] memory layer performance while adding many features?

2015-07-29 Thread Victor Olaya
No idea why that might be happeningbut hats off to you for such a well documented report. I had to say it :-) 2015-07-29 9:04 GMT+02:00 Raymond Nijssen : > Dear developers, ;) > > A plugin of mine imports data from a gml file into a memory layer. Works > fine for tiny gml files, but takes f

[Qgis-developer] memory layer performance while adding many features?

2015-07-29 Thread Raymond Nijssen
Dear developers, ;) A plugin of mine imports data from a gml file into a memory layer. Works fine for tiny gml files, but takes forever on huge ones. Sounds plausible maybe, but the relation is not linear. So I did did some tests and found out that reading and parsing the gml and creating th