Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi Nyall I'm still a bit unclear as to where you are taking these classes though. Is the intention that eventually ALL operations will be done using QgsGeos? In the end, yes, everything should go through QgsGeometryEngine interface. There are two possibilities: 1. client code creates QgsGeos, calls prepare() on it and makes repeaded intersection(), etc. directly on QgsGeometryEngine. This can be done with master branch as is. 2. client code uses QgsGeometry and calls intersection() on it. To benefit from prepared geometries, we would need to enhance QgsGeometry to cache QgsGeos and call prepare() the first time it is used. Thinking about it, version 1 might be better. We cannot know if the cached QgsGeos is really needed a second time and it might use more memory if not. In the first case, the client code knows if it is intended to do repeaded operations on the same geometry. What do you think? Could we put a giant NOT STABLE API AND MAY CHANGE for all these classes, so that we can address any unforeseen issues in 2.12? That would be ok. Regards, Marco On 23.06.2015 13:59, Nyall Dawson wrote: On 23 June 2015 at 16:15, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: Hi Nyall This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. Are you really sure the impact is huge? If you select features on the map, you get a new one from provider each time. So also with the old geometry class, it was necessary to convert to geos for each intersect. That's why the subject line says potentially ;) But yes, if several geos operations are done in sequence on really the same geometry instance, there is an overhead now. However, I would assume the geos conversion takes far less time than the geos operation itself. Did you observe cases where the geos conversion really takes very long time? My biggest concern was operations such as the Select by location operations, where a single feature is compared against all the other features in a second layer. In this case there would be benefit from retaining the geos representation between each comparison. However, I've just run some tests and there's no appreciable difference in the time required for selecting intersecting features from large layers using this algorithm between 2.8-2.10, so you're correct in that the geos conversion isn't very expensive. (Unfortunately, as Martin has pointed out, memory usage in 2.10 was up by 50%). Lets assume the geos conversion indeed takes a long time. Couldn't we just store the pointer to QgsGeos (or better QgsGeometryEngine class) inside QgsGeometry? This would be straightforward to do. Furthermore, it is possible to prepare QgsGeos. A prepared geos geometry can do _much_ faster intersects/touches/ etc. operations if called repeatedly. We can cache a prepared QgsGeos now with the new engine, but we couldn't do that with the old QgsGeometry class. So for repeated predicate calculations, the new geometry is much better, also performance wise. Agreed - when PostGIS flipped their intersects operation to utilise prepared geometries the difference was huge. I'm excited to see this taken advantage of in QGIS too! I'm still a bit unclear as to where you are taking these classes though. Is the intention that eventually ALL operations will be done using QgsGeos? Because if not, then I can't see anyway to actually utilise the prepared geometries as it stands now - none of the methods in QgsGeometry utilise these. Please don't take my reservations as attacking this work. I think it's great stuff, and generally has been quite painless since it landed, which is surprising given the extent of the work. I just want to make sure 2.10 is in the best shape it can be in, and I'm doubtful that we are release ready yet. My biggest concern is honestly having the current QgsGeometryV2 API locked in while there are still questions regarding how it's implemented. Could we put a giant NOT STABLE API AND MAY CHANGE for all these classes, so that we can address any unforeseen issues in 2.12? Nyall Regards, Marco On 23.06.2015 04:09, Nyall Dawson wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return
Re: [Qgis-developer] Use the style tab widget in a Python plugin
Hi Michael The QgsRendererV2Widget is really just abstract base class meant to be implemented by custom implementations of renderers. I think you are looking for this: from qgis.gui import * w=QgsRendererV2PropertiesDialog(iface.activeLayer(), QgsStyleV2.defaultStyle(), True) w.show() Cheers Martin On Tue, Jun 23, 2015 at 11:09 PM, kimaidou kimai...@gmail.com wrote: Ok, it seems python bindings are missing here. Here is the console output from qgis.gui import * from qgis.core import * rw = QgsRendererV2Widget( iface.activeLayer(), QgsStyleV2.defaultStyle() ) Traceback (most recent call last): File input, line 1, in module TypeError: qgis._gui.QgsRendererV2Widget represents a C++ abstract class and cannot be instantiated Too bad ;) 2015-06-23 16:38 GMT+02:00 kimaidou kimai...@gmail.com: Hum... After digging a little deeper, I found something interesting in the Python Cookbook [1] It seems I need to use the class QgsRendererV2Widget [1] http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/vector.html#creating-custom-renderers 2015-06-23 16:29 GMT+02:00 kimaidou kimai...@gmail.com: Hi all, The LayerBoard plugin I made during the HF in Nodebo shows a table of all vector (and raster) layers and allows to edit some properties directly by editing the table cells, such as max and min scale, layer name, title and abstract, assign SRS, etc. I would like to display the content of the Style tab of the layer properties dialog in a panel right to the table. This tab will show the style interface refreshed whenever the user clicks on a layer in the tab, and let her/him change the style for this layer the same way we do via the layer properties dialog. I have searched a bit in the api and in the source code, but cannot find the right way to do so. I have not found a custom Widget which I can instantiate with the layer and wich will provide me all the logic underneath. Is this even possible ? Has anyone an idea of how to to this without rewriting all the code ( connection between ui elements and slots) Thanks in advance Michaël ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Zoom in more than 1:2,256 borken
Hello, In my project test I can't zoom in more than 1:2,256. When I try to zoom more, the map moves (ie pan). I built from the last commit today (few mn ago). Does someone have the same behaviour? Thanks, Y. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] DXF/DWG import requirements document
Hi, To all who have an interest in contributing to a DXF/DWG import function in QGIS: I have started working on a requirements document - you can view it here: https://docs.google.com/a/qgis.org/document/d/1NJv1lwd5TnWdV-fTuW7g5ts1O8r3vb46xYeAIQ_BwGw/edit?usp=sharing If you want to contribute to the document, please send me your gmail account so I can give you write access. After a couple of days (maybe 1.5-2 weeks) we can hopefully finish this document and ask for a quote. I will then ask again for organizations or companies who can contribute financially. Thank you in advance for your support! Andreas ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] Use the style tab widget in a Python plugin
Hi all, The LayerBoard plugin I made during the HF in Nodebo shows a table of all vector (and raster) layers and allows to edit some properties directly by editing the table cells, such as max and min scale, layer name, title and abstract, assign SRS, etc. I would like to display the content of the Style tab of the layer properties dialog in a panel right to the table. This tab will show the style interface refreshed whenever the user clicks on a layer in the tab, and let her/him change the style for this layer the same way we do via the layer properties dialog. I have searched a bit in the api and in the source code, but cannot find the right way to do so. I have not found a custom Widget which I can instantiate with the layer and wich will provide me all the logic underneath. Is this even possible ? Has anyone an idea of how to to this without rewriting all the code ( connection between ui elements and slots) Thanks in advance Michaël ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Use the style tab widget in a Python plugin
Hum... After digging a little deeper, I found something interesting in the Python Cookbook [1] It seems I need to use the class QgsRendererV2Widget [1] http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/vector.html#creating-custom-renderers 2015-06-23 16:29 GMT+02:00 kimaidou kimai...@gmail.com: Hi all, The LayerBoard plugin I made during the HF in Nodebo shows a table of all vector (and raster) layers and allows to edit some properties directly by editing the table cells, such as max and min scale, layer name, title and abstract, assign SRS, etc. I would like to display the content of the Style tab of the layer properties dialog in a panel right to the table. This tab will show the style interface refreshed whenever the user clicks on a layer in the tab, and let her/him change the style for this layer the same way we do via the layer properties dialog. I have searched a bit in the api and in the source code, but cannot find the right way to do so. I have not found a custom Widget which I can instantiate with the layer and wich will provide me all the logic underneath. Is this even possible ? Has anyone an idea of how to to this without rewriting all the code ( connection between ui elements and slots) Thanks in advance Michaël ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Use the style tab widget in a Python plugin
Ok, it seems python bindings are missing here. Here is the console output from qgis.gui import * from qgis.core import * rw = QgsRendererV2Widget( iface.activeLayer(), QgsStyleV2.defaultStyle() ) Traceback (most recent call last): File input, line 1, in module TypeError: qgis._gui.QgsRendererV2Widget represents a C++ abstract class and cannot be instantiated Too bad ;) 2015-06-23 16:38 GMT+02:00 kimaidou kimai...@gmail.com: Hum... After digging a little deeper, I found something interesting in the Python Cookbook [1] It seems I need to use the class QgsRendererV2Widget [1] http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/vector.html#creating-custom-renderers 2015-06-23 16:29 GMT+02:00 kimaidou kimai...@gmail.com: Hi all, The LayerBoard plugin I made during the HF in Nodebo shows a table of all vector (and raster) layers and allows to edit some properties directly by editing the table cells, such as max and min scale, layer name, title and abstract, assign SRS, etc. I would like to display the content of the Style tab of the layer properties dialog in a panel right to the table. This tab will show the style interface refreshed whenever the user clicks on a layer in the tab, and let her/him change the style for this layer the same way we do via the layer properties dialog. I have searched a bit in the api and in the source code, but cannot find the right way to do so. I have not found a custom Widget which I can instantiate with the layer and wich will provide me all the logic underneath. Is this even possible ? Has anyone an idea of how to to this without rewriting all the code ( connection between ui elements and slots) Thanks in advance Michaël ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] How to provide a Processing model in a plugin ?
Hi, I don't know exactly what happened, but it works now. I can launch my own model from the plugin. This is the final code : model = ModelerAlgorithm.ModelerAlgorithm.fromFile(file_path) model.provider = Processing.modeler Processing.modeler.algs.append(model) command_line = model.commandLineName() Processing.algs['model'][command_line] = model return command_line runalg shows by default the QgsMessageBar. We will see if we need it or if we need to change processing. Thanks again Victor. 2015-06-22 17:28 GMT+02:00 Etienne Trimaille etienne.trimai...@gmail.com: Hi Victor, Thanks for the answer. I maybe the first one, but I need to launch a model in the InaSAFE plugin and I don't want to write by hand the equivalent in python :) I tried your code above : model = ModelerAlgorithm.fromFile(Path/to/your/model/file) model.provider = Processing.modeler Processing.modeler.algs.append(model) But I can't launch it with runalg, I got 'Algorithm not found', so I add this : Processing.algs['model'][model.commandLineName()] = model I can see it with processing.alglist now. But I can't launch it with processing.runalg : Traceback (most recent call last): File /home/etienne/.qgis2/python/plugins/inasafe/safe/routing/gui/routing_dialog.py, line 210, in accept None) File /home/etienne/.qgis2/python/plugins/processing/tools/general.py, line 71, in runalg alg = Processing.runAlgorithm(algOrName, None, *args) File /home/etienne/.qgis2/python/plugins/processing/core/Processing.py, line 277, in runAlgorithm alg = alg.getCopy() File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 169, in getCopy newone.defineCharacteristics() File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 208, in defineCharacteristics modelOutput = copy.deepcopy(alg.algorithm.getOutputFromName(out)) File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 110, in algorithm self._algInstance = ModelerUtils.getAlgorithm(self.consoleName).getCopy() AttributeError: 'NoneType' object has no attribute 'getCopy' Do you have an idea ? Regards, Etienne 2015-06-08 16:28 GMT+02:00 Victor Olaya vola...@gmail.com: You can use the instance of ModelerAlgorithmProvider that is stored in the Processing object. Like this: model = ModelerAlgorithm.fromFile(Path/to/your/model/file) model.provider = Processing.modeler Processing.modeler.algs.append(model) That will add your model to the list of available ones I guess you are the first one asking for this...and this is the best solution i can think of so far :-) Let me know if that works... I hope it helps 2015-06-08 15:35 GMT+02:00 Etienne Trimaille etienne.trimai...@gmail.com : Hi, I'm working on a plugin and I need to use my own model inside. I know how to add a new algorithm in processing but I didn't find something on how to add a model to processing. Is there a way to ship my model with the plugin ? Or do I need to copy manually my model into the processing models folder with Python ? I saw the ModelerAlgorithmProvider (like the AlgorithmProvider) but I not sure if I can do what I want with it. Thanks, Etienne ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] How to provide a Processing model in a plugin ?
Etienne Good to see that it works :-) For the record, and in case you are interested in checking it, it might be worth having a look at the add model from file action. It is a different approach, just adding the file to the model folder and refreshing the model provider, but just in case it is useful for you. It will also make the model available for the user, outside of your plugin context https://github.com/qgis/QGIS/blob/master/python/plugins/processing/modeler/AddModelFromFileAction.py Regarding the progress indication, once you have your algorithm with parameters set, you can execute it using the AlgorithmExecutor class, which accepts a progress object as parameter. That should let you redirect the output messages and progress indications. Notice that this might change soon, since there is a multithreading implementation of that class being written at the moment. Let me know if you have more questions Cheers 2015-06-23 14:28 GMT+02:00 Etienne Trimaille etienne.trimai...@gmail.com: Hi, I don't know exactly what happened, but it works now. I can launch my own model from the plugin. This is the final code : model = ModelerAlgorithm.ModelerAlgorithm.fromFile(file_path) model.provider = Processing.modeler Processing.modeler.algs.append(model) command_line = model.commandLineName() Processing.algs['model'][command_line] = model return command_line runalg shows by default the QgsMessageBar. We will see if we need it or if we need to change processing. Thanks again Victor. 2015-06-22 17:28 GMT+02:00 Etienne Trimaille etienne.trimai...@gmail.com: Hi Victor, Thanks for the answer. I maybe the first one, but I need to launch a model in the InaSAFE plugin and I don't want to write by hand the equivalent in python :) I tried your code above : model = ModelerAlgorithm.fromFile(Path/to/your/model/file) model.provider = Processing.modeler Processing.modeler.algs.append(model) But I can't launch it with runalg, I got 'Algorithm not found', so I add this : Processing.algs['model'][model.commandLineName()] = model I can see it with processing.alglist now. But I can't launch it with processing.runalg : Traceback (most recent call last): File /home/etienne/.qgis2/python/plugins/inasafe/safe/routing/gui/routing_dialog.py, line 210, in accept None) File /home/etienne/.qgis2/python/plugins/processing/tools/general.py, line 71, in runalg alg = Processing.runAlgorithm(algOrName, None, *args) File /home/etienne/.qgis2/python/plugins/processing/core/Processing.py, line 277, in runAlgorithm alg = alg.getCopy() File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 169, in getCopy newone.defineCharacteristics() File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 208, in defineCharacteristics modelOutput = copy.deepcopy(alg.algorithm.getOutputFromName(out)) File /home/etienne/.qgis2/python/plugins/processing/modeler/ModelerAlgorithm.py, line 110, in algorithm self._algInstance = ModelerUtils.getAlgorithm(self.consoleName).getCopy() AttributeError: 'NoneType' object has no attribute 'getCopy' Do you have an idea ? Regards, Etienne 2015-06-08 16:28 GMT+02:00 Victor Olaya vola...@gmail.com: You can use the instance of ModelerAlgorithmProvider that is stored in the Processing object. Like this: model = ModelerAlgorithm.fromFile(Path/to/your/model/file) model.provider = Processing.modeler Processing.modeler.algs.append(model) That will add your model to the list of available ones I guess you are the first one asking for this...and this is the best solution i can think of so far :-) Let me know if that works... I hope it helps 2015-06-08 15:35 GMT+02:00 Etienne Trimaille etienne.trimai...@gmail.com: Hi, I'm working on a plugin and I need to use my own model inside. I know how to add a new algorithm in processing but I didn't find something on how to add a model to processing. Is there a way to ship my model with the plugin ? Or do I need to copy manually my model into the processing models folder with Python ? I saw the ModelerAlgorithmProvider (like the AlgorithmProvider) but I not sure if I can do what I want with it. Thanks, Etienne ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] DXF/DWG import requirements document
Hi Andreas and everyone, We're very interested in contributing to a DXF/DWG import and export function. This is mine gmail: carlos.p...@gmail.com Perhaps, We have some money to cofinance or help to develop this function! Thank you for your offer. Best regards, * http://www.psig.es* *Carlos López Quintanilla* www.psig.es carlos.lo...@psig.es +34 699.680.261 2015-06-23 16:42 GMT+02:00 Andreas Neumann a.neum...@carto.net: Hi, To all who have an interest in contributing to a DXF/DWG import function in QGIS: I have started working on a requirements document - you can view it here: https://docs.google.com/a/qgis.org/document/d/1NJv1lwd5TnWdV-fTuW7g5ts1O8r3vb46xYeAIQ_BwGw/edit?usp=sharing If you want to contribute to the document, please send me your gmail account so I can give you write access. After a couple of days (maybe 1.5-2 weeks) we can hopefully finish this document and ask for a quote. I will then ask again for organizations or companies who can contribute financially. Thank you in advance for your support! Andreas ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Zoom in more than 1:2,256 borken
On 23-06-15 14:05, Yves Jacolin wrote: Hello, In my project test I can't zoom in more than 1:2,256. When I try to zoom more, the map moves (ie pan). I built from the last commit today (few mn ago). Does someone have the same behaviour? Hi Yves, just rebuild, and cannot reproduce this, both in our national crs and 3856 with OpenLayers layers. did you try to disable all plugins, or try a fresh configuration? Regards, Richard Duivenvoorde ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] DXF/DWG import requirements document
Dear Andreas, beside Import of layer names as attributes I would prefer 1. setting layer filter in the import dialog (e.g. as a regexp or a selection list from the available CAD layers), unselected layers are skipped in input. It is not clear for me if the layer name rules can be used for this purpose. 2. import each CAD layer into separate QGIS layers by feature type (point, annotation, polyline, polygon), it could be a checkbox in the import dialog. Optionally during the import polygon layers sould be cheched for islands (polygons inside other polygon) and remove the area of islands from the container polygon. Regards, Zoltan On Tue, 23 Jun 2015, Andreas Neumann wrote: Hi, To all who have an interest in contributing to a DXF/DWG import function in QGIS: I have started working on a requirements document - you can view it here: https://docs.google.com/a/qgis.org/document/d/1NJv1lwd5TnWdV-fTuW7g5ts1O8r3vb46xYeAIQ_BwGw/edit?usp=sharing If you want to contribute to the document, please send me your gmail account so I can give you write access. After a couple of days (maybe 1.5-2 weeks) we can hopefully finish this document and ask for a quote. I will then ask again for organizations or companies who can contribute financially. Thank you in advance for your support! Andreas ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Zoom in more than 1:2,256 borken
On Tue, Jun 23, 2015 at 02:05:14PM +0200, Yves Jacolin wrote: Hello, In my project test I can't zoom in more than 1:2,256. When I try to zoom more, the map moves (ie pan). I built from the last commit today (few mn ago). Does someone have the same behaviour? I remember having had troubles with high zoom values, with rendering going wild. Did you try with an old version ? --strk; ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Use the style tab widget in a Python plugin
Hi Martin, Thanks for the help. Now I can display the widget in my scrollArea. I have still have some crashes, but this is the way to go. I will then need to set the layer style via a custom apply button Cheers, Michael 2015-06-23 17:23 GMT+02:00 Martin Dobias wonder...@gmail.com: Hi Michael The QgsRendererV2Widget is really just abstract base class meant to be implemented by custom implementations of renderers. I think you are looking for this: from qgis.gui import * w=QgsRendererV2PropertiesDialog(iface.activeLayer(), QgsStyleV2.defaultStyle(), True) w.show() Cheers Martin On Tue, Jun 23, 2015 at 11:09 PM, kimaidou kimai...@gmail.com wrote: Ok, it seems python bindings are missing here. Here is the console output from qgis.gui import * from qgis.core import * rw = QgsRendererV2Widget( iface.activeLayer(), QgsStyleV2.defaultStyle() ) Traceback (most recent call last): File input, line 1, in module TypeError: qgis._gui.QgsRendererV2Widget represents a C++ abstract class and cannot be instantiated Too bad ;) 2015-06-23 16:38 GMT+02:00 kimaidou kimai...@gmail.com: Hum... After digging a little deeper, I found something interesting in the Python Cookbook [1] It seems I need to use the class QgsRendererV2Widget [1] http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/vector.html#creating-custom-renderers 2015-06-23 16:29 GMT+02:00 kimaidou kimai...@gmail.com: Hi all, The LayerBoard plugin I made during the HF in Nodebo shows a table of all vector (and raster) layers and allows to edit some properties directly by editing the table cells, such as max and min scale, layer name, title and abstract, assign SRS, etc. I would like to display the content of the Style tab of the layer properties dialog in a panel right to the table. This tab will show the style interface refreshed whenever the user clicks on a layer in the tab, and let her/him change the style for this layer the same way we do via the layer properties dialog. I have searched a bit in the api and in the source code, but cannot find the right way to do so. I have not found a custom Widget which I can instantiate with the layer and wich will provide me all the logic underneath. Is this even possible ? Has anyone an idea of how to to this without rewriting all the code ( connection between ui elements and slots) Thanks in advance Michaël ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Zoom in more than 1:2,256 borken
Hi Yves, any project scales defined? I was tricked by this once.. Cheers Régis -- View this message in context: http://osgeo-org.1560.x6.nabble.com/Zoom-in-more-than-1-2-256-borken-tp5212529p5212567.html Sent from the Quantum GIS - Developer mailing list archive at Nabble.com. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] DXF/DWG import requirements document
Hi Zoltan, I added these two requirements in the document. However, with the new legend options introduced in QGIS 2.8 you can easily toggle the visibility of classes or rules or delete certain rules - so I am not 100% convinced if this option is really necessary. Thanks for your reply! Andreas On 23.06.2015 19:26, Siki Zoltan wrote: Dear Andreas, beside Import of layer names as attributes I would prefer 1. setting layer filter in the import dialog (e.g. as a regexp or a selection list from the available CAD layers), unselected layers are skipped in input. It is not clear for me if the layer name rules can be used for this purpose. 2. import each CAD layer into separate QGIS layers by feature type (point, annotation, polyline, polygon), it could be a checkbox in the import dialog. Optionally during the import polygon layers sould be cheched for islands (polygons inside other polygon) and remove the area of islands from the container polygon. Regards, Zoltan On Tue, 23 Jun 2015, Andreas Neumann wrote: Hi, To all who have an interest in contributing to a DXF/DWG import function in QGIS: I have started working on a requirements document - you can view it here: https://docs.google.com/a/qgis.org/document/d/1NJv1lwd5TnWdV-fTuW7g5ts1O8r3vb46xYeAIQ_BwGw/edit?usp=sharing If you want to contribute to the document, please send me your gmail account so I can give you write access. After a couple of days (maybe 1.5-2 weeks) we can hopefully finish this document and ask for a quote. I will then ask again for organizations or companies who can contribute financially. Thank you in advance for your support! Andreas ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
On 23 June 2015 at 16:15, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: Hi Nyall This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. Are you really sure the impact is huge? If you select features on the map, you get a new one from provider each time. So also with the old geometry class, it was necessary to convert to geos for each intersect. That's why the subject line says potentially ;) But yes, if several geos operations are done in sequence on really the same geometry instance, there is an overhead now. However, I would assume the geos conversion takes far less time than the geos operation itself. Did you observe cases where the geos conversion really takes very long time? My biggest concern was operations such as the Select by location operations, where a single feature is compared against all the other features in a second layer. In this case there would be benefit from retaining the geos representation between each comparison. However, I've just run some tests and there's no appreciable difference in the time required for selecting intersecting features from large layers using this algorithm between 2.8-2.10, so you're correct in that the geos conversion isn't very expensive. (Unfortunately, as Martin has pointed out, memory usage in 2.10 was up by 50%). Lets assume the geos conversion indeed takes a long time. Couldn't we just store the pointer to QgsGeos (or better QgsGeometryEngine class) inside QgsGeometry? This would be straightforward to do. Furthermore, it is possible to prepare QgsGeos. A prepared geos geometry can do _much_ faster intersects/touches/ etc. operations if called repeatedly. We can cache a prepared QgsGeos now with the new engine, but we couldn't do that with the old QgsGeometry class. So for repeated predicate calculations, the new geometry is much better, also performance wise. Agreed - when PostGIS flipped their intersects operation to utilise prepared geometries the difference was huge. I'm excited to see this taken advantage of in QGIS too! I'm still a bit unclear as to where you are taking these classes though. Is the intention that eventually ALL operations will be done using QgsGeos? Because if not, then I can't see anyway to actually utilise the prepared geometries as it stands now - none of the methods in QgsGeometry utilise these. Please don't take my reservations as attacking this work. I think it's great stuff, and generally has been quite painless since it landed, which is surprising given the extent of the work. I just want to make sure 2.10 is in the best shape it can be in, and I'm doubtful that we are release ready yet. My biggest concern is honestly having the current QgsGeometryV2 API locked in while there are still questions regarding how it's implemented. Could we put a giant NOT STABLE API AND MAY CHANGE for all these classes, so that we can address any unforeseen issues in 2.12? Nyall Regards, Marco On 23.06.2015 04:09, Nyall Dawson wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
On 23 June 2015 at 16:25, Werner Macho werner.ma...@gmail.com wrote: Hi! I would be in favor of releasing it as is. 1. This is not the _LTS_ release (I thought this is exactly why we introduced LTS) - so people should know that there are new features which sometimes can cause something 2. releasing it with the changes means more feedback if this is really a problem and more time with lots of testing to fix it until the next LTS ... I'm not in favour of treating non LTS releases as beta type releases though. There's been potentially 10s of thousands of dollars of investment in 2.10, and people who have sponsored features are still expecting a stable release when 2.10 comes out. Nyall Though there should probably a release note to make people aware and ask them to test this very well. kind regards Werner On Tue, Jun 23, 2015 at 8:15 AM, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: Hi Nyall This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. Are you really sure the impact is huge? If you select features on the map, you get a new one from provider each time. So also with the old geometry class, it was necessary to convert to geos for each intersect. But yes, if several geos operations are done in sequence on really the same geometry instance, there is an overhead now. However, I would assume the geos conversion takes far less time than the geos operation itself. Did you observe cases where the geos conversion really takes very long time? Lets assume the geos conversion indeed takes a long time. Couldn't we just store the pointer to QgsGeos (or better QgsGeometryEngine class) inside QgsGeometry? This would be straightforward to do. Furthermore, it is possible to prepare QgsGeos. A prepared geos geometry can do _much_ faster intersects/touches/ etc. operations if called repeatedly. We can cache a prepared QgsGeos now with the new engine, but we couldn't do that with the old QgsGeometry class. So for repeated predicate calculations, the new geometry is much better, also performance wise. Regards, Marco On 23.06.2015 04:09, Nyall Dawson wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to address this? Release with the regression? Or am I missing something and there's an easier solution we could implement? Or even possibly this additional cost of recalculating the geos representation is trivial and can be ignored (maybe someone could test this with a little repeated intersection script)? Thoughts? Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
On 23 June 2015 at 20:07, Matthias Kuhn matth...@opengis.ch wrote: So the question is, is this something that we can fix later on, or would fixing it require an API break and therefore we need to wait for 3.0 if we don't fix it now? As I mentioned in an earlier reply, could we mark this component of the API as non-stable for 2.10? That would give us the possibility of later revision. IIRC it wouldn't be the first time this has been done - I think the labeling API was initially marked as experimental and non stable too... Nyall Regards, Matthias On 06/23/2015 11:37 AM, Marco Hugentobler wrote: Hi Martin The new geometries uses more memory for backward compatibility. In the mid/long term, the cached wkb will be removed, so QgsAbstractGeometry will be the only represention. The provider can feed wkb (e.g. database providers) or construct the geometry by feeding QgsPointV2s. I'm however surprised that this should have such an impact with the attached dataset. Because the shp has size 588MByte, you could load that in memory several times (and we don't load everything into memory in QGIS in case the dataset is even bigger). Looking on the attached file, I think the problem is not the memory, but rather some round trips in the closestVertex function. But it is far from something that would justify a release delay or even a geometry rollback. Regards, Marco On 23.06.2015 05:42, Martin Dobias wrote: Hi Agreed with Nyall. Apart from the above mentioned problems, there are newly introduced performance regressions with snapping: the cached geometries in 2.10 take double the amount of memory they used in 2.8, and snapping got much slower with more complex layers (compared to 2.8) - e.g. with [1]. The extra memory consumption is caused by the fact that providers provide geometry in WKB representation, which is then converted to new representation and WKB is kept. We should use just one representation and drop the usage of the other one - that means stop using WKB in providers and common code paths, so the WKB representation does not even get created (and cached). One heretic idea at the end - what others think about postponing the release of the new geometry architecture to 2.12 so that there is more time to address the current issues (fix performance, fix high memory consumption, clean up API, write unit tests). It seems to me that some of the issues would be difficult to address even if the release of 2.10 is moved by another week or so. Regards Martin [1] http://www.iucnredlist.org/spatial-data/MAMMALS_TERRESTRIAL.zip On Tue, Jun 23, 2015 at 10:09 AM, Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to address this? Release with the regression? Or am I missing something and there's an easier solution we could implement? Or even possibly this additional cost of recalculating the geos representation is trivial and can be ignored (maybe someone could test this with a little repeated intersection script)? Thoughts? Nyall
[Qgis-developer] request information on how to upgrade an old project
hello all i have an old project based on the 1.6 qgis version i need to update to the last version 2.8 and a new system ubuntu 14 i have installed all dependencies requiered by qgis , everything works fine , ( qgiss, grass, python, qt, etc ... ) but when i try to build my old project i have a lot of error like no such file or directy because it seems that some classes or header file have been removed or renamed here is the header liste of my project : // QGIS Includes // #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include some of them doesn't appear in the class list on the web site of qgis for the 2.8 : http://qgis.org/api/2.8/annotated.html for the 1.6 : http://qgis.org/api/1.6/classes.html by exemple : qgssinglesymbolrenderer, qgsrenderer, qgsuniquevaluerenderer, and qgssymbologyutils give an error message no such file or directory i don't know how to build correctly my project because i don't know and i don't find the list of change , what files have been removed, renamed, fusionned ? thanks for your help Regards Florent -- - Florent Ainardi-Wieloszynski florent.ainardi-wieloszyn...@c-s.fr Tel : (+33) 4 94 08 75 75 GSM : (+33) 6 10 51 26 23 Fax : (+33) 4 94 08 09 38 CS Toulon 230, rue Marcellin Berthelot ZI Toulon-Est - La Garde - BP 68 83 079 Toulon Cedex 09 - FRANCE http://www.c-s.fr - ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi Martin The new geometries uses more memory for backward compatibility. In the mid/long term, the cached wkb will be removed, so QgsAbstractGeometry will be the only represention. The provider can feed wkb (e.g. database providers) or construct the geometry by feeding QgsPointV2s. I'm however surprised that this should have such an impact with the attached dataset. Because the shp has size 588MByte, you could load that in memory several times (and we don't load everything into memory in QGIS in case the dataset is even bigger). Looking on the attached file, I think the problem is not the memory, but rather some round trips in the closestVertex function. But it is far from something that would justify a release delay or even a geometry rollback. Regards, Marco On 23.06.2015 05:42, Martin Dobias wrote: Hi Agreed with Nyall. Apart from the above mentioned problems, there are newly introduced performance regressions with snapping: the cached geometries in 2.10 take double the amount of memory they used in 2.8, and snapping got much slower with more complex layers (compared to 2.8) - e.g. with [1]. The extra memory consumption is caused by the fact that providers provide geometry in WKB representation, which is then converted to new representation and WKB is kept. We should use just one representation and drop the usage of the other one - that means stop using WKB in providers and common code paths, so the WKB representation does not even get created (and cached). One heretic idea at the end - what others think about postponing the release of the new geometry architecture to 2.12 so that there is more time to address the current issues (fix performance, fix high memory consumption, clean up API, write unit tests). It seems to me that some of the issues would be difficult to address even if the release of 2.10 is moved by another week or so. Regards Martin [1] http://www.iucnredlist.org/spatial-data/MAMMALS_TERRESTRIAL.zip On Tue, Jun 23, 2015 at 10:09 AM, Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to address this? Release with the regression? Or am I missing something and there's an easier solution we could implement? Or even possibly this additional cost of recalculating the geos representation is trivial and can be ignored (maybe someone could test this with a little repeated intersection script)? Thoughts? Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Hierarchy of new geometry classes
http://osgeo-org.1560.x6.nabble.com/Questions-regarding-new-geometry-engine-td5208770.html#a5209162 On 23.06.2015 05:58, Martin Dobias wrote: Hi Recently I have been looking a bit at the new geometry classes and there is one thing I do not understand: if QgsLineStringV2 is derived from QgsCurveV2 and QgsPolygonV2 is derived from QgsSurfaceV2, why is not the same approach used for multi-part geometries? Currently QgsMultiLineStringV2 and QgsMultiPolygonV2 are subclasses of QgsGeometryCollectionV2 - but why they are not derived from QgsMultiCurveV2 and QgsMultiSurfaceV2 respectively? It looks like OGR uses that approach too (multi-polygon from multi-surface and multi-linestring from multi-curve): http://www.gdal.org/classOGRGeometry.html Regards Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi Marco, On Tue, 23. Jun 2015 at 08:15:27 +0200, Marco Hugentobler wrote: Are you really sure the impact is huge? Well, if you have to ask and Nyall apparently is the first to notice - the impact can't be huge. I think reverting is no option - there have been numerous things that were adapted to the new engine. The indexing when snapping on a large layer is noticable however. But I don't think that's a huge issue either. Some will complain - but there are other things that don't work nicely on large layers either. It's a reminder that new stuff should go in early in the development cycle and not shortly before the feature freeze. But that's just a future note. I'd release what we have. Jürgen -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS release manager (PSC) GermanyIRC: jef on FreeNode signature.asc Description: Digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Crash in new GRASS plugin?
On Thu, May 21, 2015 at 10:53 AM, Radim Blazek radim.bla...@gmail.com wrote:
On Thu, May 21, 2015 at 10:27 AM, Paolo Cavallini cavall...@faunalia.it
wrote:
AFAICT, it can only happen if you have GISRC set, but the file does
not exist or it is corrupted. Try to unset GISRC envar and it should
run.
I think this is due to Processing, that sets temp grass variables.
I see:
algs/grass/GrassUtils.py 250: os.putenv('GISRC', gisrc)
I think that Processing should not set GISRC globally, it should
rather start modules with that envar. QGIS may be started from GRASS
shell, in that case the plugin is using GISRC to setup active mapset
so it must not be changed. The plugin is also setting GISRC globally
when a mapset is opened, but that is correct, because it also locks
the mapset and in that case it works like a GRASS shell.
Victor, could you please remove
os.putenv('GISRC', gisrc)
os.putenv('GRASS_MESSAGE_FORMAT', 'gui')
os.putenv('GRASS_BATCH_JOB'...
is not it enough to set them in createGrassScript()?
Victor, can you look at it please?
Radim
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi Nyall This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. Are you really sure the impact is huge? If you select features on the map, you get a new one from provider each time. So also with the old geometry class, it was necessary to convert to geos for each intersect. But yes, if several geos operations are done in sequence on really the same geometry instance, there is an overhead now. However, I would assume the geos conversion takes far less time than the geos operation itself. Did you observe cases where the geos conversion really takes very long time? Lets assume the geos conversion indeed takes a long time. Couldn't we just store the pointer to QgsGeos (or better QgsGeometryEngine class) inside QgsGeometry? This would be straightforward to do. Furthermore, it is possible to prepare QgsGeos. A prepared geos geometry can do _much_ faster intersects/touches/ etc. operations if called repeatedly. We can cache a prepared QgsGeos now with the new engine, but we couldn't do that with the old QgsGeometry class. So for repeated predicate calculations, the new geometry is much better, also performance wise. Regards, Marco On 23.06.2015 04:09, Nyall Dawson wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to address this? Release with the regression? Or am I missing something and there's an easier solution we could implement? Or even possibly this additional cost of recalculating the geos representation is trivial and can be ignored (maybe someone could test this with a little repeated intersection script)? Thoughts? Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi! I would be in favor of releasing it as is. 1. This is not the _LTS_ release (I thought this is exactly why we introduced LTS) - so people should know that there are new features which sometimes can cause something 2. releasing it with the changes means more feedback if this is really a problem and more time with lots of testing to fix it until the next LTS Though there should probably a release note to make people aware and ask them to test this very well. kind regards Werner On Tue, Jun 23, 2015 at 8:15 AM, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: Hi Nyall This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. Are you really sure the impact is huge? If you select features on the map, you get a new one from provider each time. So also with the old geometry class, it was necessary to convert to geos for each intersect. But yes, if several geos operations are done in sequence on really the same geometry instance, there is an overhead now. However, I would assume the geos conversion takes far less time than the geos operation itself. Did you observe cases where the geos conversion really takes very long time? Lets assume the geos conversion indeed takes a long time. Couldn't we just store the pointer to QgsGeos (or better QgsGeometryEngine class) inside QgsGeometry? This would be straightforward to do. Furthermore, it is possible to prepare QgsGeos. A prepared geos geometry can do _much_ faster intersects/touches/ etc. operations if called repeatedly. We can cache a prepared QgsGeos now with the new engine, but we couldn't do that with the old QgsGeometry class. So for repeated predicate calculations, the new geometry is much better, also performance wise. Regards, Marco On 23.06.2015 04:09, Nyall Dawson wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to address this? Release with the regression? Or am I missing something and there's an easier solution we could implement? Or even possibly this additional cost of recalculating the geos representation is trivial and can be ignored (maybe someone could test this with a little repeated intersection script)? Thoughts? Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Hierarchy of new geometry classes
Hi Marco On Tue, Jun 23, 2015 at 2:27 PM, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: http://osgeo-org.1560.x6.nabble.com/Questions-regarding-new-geometry-engine-td5208770.html#a5209162 Thanks - I thought this was already asked, but couldn't find where... IMO the hierarchy in QGIS should follow the one in ISO and OGC models. Maybe right now there is no practical advantage for the implementation to follow the hierarchy, but is there a strong reason to deviate from it? It seems confusing that multi-polygon cannot be cast to multi-surface. I am afraid that things like this can bite us later... Regards Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Hierarchy of new geometry classes
Hi Martin but is there a strong reason to deviate from it? Only a week reason ( clone() method ). But I think you are right and it might be better to follow the ISO model to be more compatible with other libs. Regards, Marco On 23.06.2015 08:39, Martin Dobias wrote: Hi Marco On Tue, Jun 23, 2015 at 2:27 PM, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: http://osgeo-org.1560.x6.nabble.com/Questions-regarding-new-geometry-engine-td5208770.html#a5209162 Thanks - I thought this was already asked, but couldn't find where... IMO the hierarchy in QGIS should follow the one in ISO and OGC models. Maybe right now there is no practical advantage for the implementation to follow the hierarchy, but is there a strong reason to deviate from it? It seems confusing that multi-polygon cannot be cast to multi-surface. I am afraid that things like this can bite us later... Regards Martin -- Dr. Marco Hugentobler Sourcepole - Linux Open Source Solutions Weberstrasse 5, CH-8004 Zürich, Switzerland marco.hugentob...@sourcepole.ch http://www.sourcepole.ch Technical Advisor QGIS Project Steering Committee ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
Hi, I don't think reverting the geometry work is an option. It works well in general and offers really nice new possibilities with z and m values and circular arcs despite it was merged a bit late, did not receive a lot of attention in the freeze period and lacks some unit tests. But then, should we spoil CPU cycles and memory just because it's present on modern machines? The way I see it, QGIS should also be able to run on limited hardware like portable devices or older hardware. IIRC implicitly shared geometries have been one of the features which were an integral part of this from the beginning. If we don't have it now but are able to add it later on I don't mind so much but I do mind if we miss the oportunity to add it now. So the question is, is this something that we can fix later on, or would fixing it require an API break and therefore we need to wait for 3.0 if we don't fix it now? Regards, Matthias On 06/23/2015 11:37 AM, Marco Hugentobler wrote: Hi Martin The new geometries uses more memory for backward compatibility. In the mid/long term, the cached wkb will be removed, so QgsAbstractGeometry will be the only represention. The provider can feed wkb (e.g. database providers) or construct the geometry by feeding QgsPointV2s. I'm however surprised that this should have such an impact with the attached dataset. Because the shp has size 588MByte, you could load that in memory several times (and we don't load everything into memory in QGIS in case the dataset is even bigger). Looking on the attached file, I think the problem is not the memory, but rather some round trips in the closestVertex function. But it is far from something that would justify a release delay or even a geometry rollback. Regards, Marco On 23.06.2015 05:42, Martin Dobias wrote: Hi Agreed with Nyall. Apart from the above mentioned problems, there are newly introduced performance regressions with snapping: the cached geometries in 2.10 take double the amount of memory they used in 2.8, and snapping got much slower with more complex layers (compared to 2.8) - e.g. with [1]. The extra memory consumption is caused by the fact that providers provide geometry in WKB representation, which is then converted to new representation and WKB is kept. We should use just one representation and drop the usage of the other one - that means stop using WKB in providers and common code paths, so the WKB representation does not even get created (and cached). One heretic idea at the end - what others think about postponing the release of the new geometry architecture to 2.12 so that there is more time to address the current issues (fix performance, fix high memory consumption, clean up API, write unit tests). It seems to me that some of the issues would be difficult to address even if the release of 2.10 is moved by another week or so. Regards Martin [1] http://www.iucnredlist.org/spatial-data/MAMMALS_TERRESTRIAL.zip On Tue, Jun 23, 2015 at 10:09 AM, Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, Unfortunately, we've become aware of a serious performance regression caused by the new geometry engine. Basically, the situation is that for all geometry operations which rely on geos (think buffers, splits, spatial relation operations such as intersects and within,... ) the geometry now needs to be converted into a geos representation with *every* operation. In the old geometry engine this conversion was done once, and the result stored so that follow up operations would not need to recalculate it. This potentially has huge impacts on the performance of common tasks such as selecting all features which intersect a geometry. I've had a look, and unfortunately it's not trivial to fix this. I think the correct solution to this is to: - make QgsGeometryEngine accept and return QgsGeometry containers, not QgsAbstractGeometryV2 - store the generated geos representation of geometries within QgsGeometryPrivate inside the QgsGeometry container. This way it will be reusable between different geometry operations, and shared when QgsGeometry objects are copied. This will also have the benefit that if a geometry is prepared using geos then subsequent geos operations performed on that QgsGeometry and its shared copies will be much faster. - make QgsGeometry a friend class of QgsGeo, so that it can access QgsGeometryPrivate to retrieve or set the geos representation of the geometry as required An alternative (short term) solution would be to just cache the geos representation when geometry operations are called through the older QgsGeometry modification/relationship operations. This would be easier, but means that the API of QgsGeometryEngine will be stuck with the current design, and we won't be able to properly fix this until breaking the api for 3.0. Either way, I doubt this can be addressed within the remaining 3 days we have until release. Should we delay to
[Qgis-developer] beforeCommitChanges signal : equivalent to SQL BEFORE triggers' OLD and NEW values
Hi, In SQL, using triggers, it's possible to pre-process values before inserting/deleting/updating them. That's a very powerful feature for things such as validation, formatting, logging modifications, timestamping, caching, synchronization across table/databases, running other scripts, etc. Is it possible to emulate this in QGIS with Python (so purely on client side) ? There is the QgsVectorLayer beforeCommitChanges signal, which is triggered exactly at the correct time, but I didn't find a way to iterate through the modified features, getting their OLD and NEW state. I think it may be possible to implement it using the attributeValueChanged (QgsFeatureId fid, int idx, const QVariant) and the geometryChanged (QgsFeatureId fid, QgsGeometry geom) signals to store a list of NEW states, and then make a new query to the data provider to get the OLD states (using QgsVectorDataProvider getFeatures). Would that work ? It's a bit sad that there's a new query to make to the provider. The best would be to have the attributeValueChanged and the geometryChanged signals to return the OLD value as well directly. Would this be a reasonable feature request ? Or is there another way to achieve the same result that I didn't think of ? Thanks a lot, Olivier ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Potentially serious performance regression in new geometry - should 2.10 be delayed?
On 24 June 2015 at 00:44, Marco Hugentobler marco.hugentob...@sourcepole.ch wrote: Hi Nyall I'm still a bit unclear as to where you are taking these classes though. Is the intention that eventually ALL operations will be done using QgsGeos? In the end, yes, everything should go through QgsGeometryEngine interface. There are two possibilities: 1. client code creates QgsGeos, calls prepare() on it and makes repeaded intersection(), etc. directly on QgsGeometryEngine. This can be done with master branch as is. This sounds like the right approach to me, but I have a couple of further questions: - For methods which don't have a geos implementation (eg, QgsGeometry::smooth ), should they just be implemented in the QgsGeometryEngine base class? - I'm a bit concerned about the usability of this approach for plugin developers. It means that they'd need to be aware that QGIS uses geos for these geometry operations and use that QgsGeos is the class required for geometry relation/modification operations. That's not obvious and could be a stumbling block for new PyQGIS developers. It also locks these plugins then into using the geos engine, and if we ever decided to switch to a different engine then the plugins would also need to be updated. Just thinking aloud here... is there a good reason we couldn't instead move these geos implementations to QgsGeometryEngine itself and do away with QgsGeos? Then, in the (unlikely?) situation that a geos implementation is inferior to a native QGIS method we can mix-and-match between geos and alternative implementations as required. 2. client code uses QgsGeometry and calls intersection() on it. To benefit from prepared geometries, we would need to enhance QgsGeometry to cache QgsGeos and call prepare() the first time it is used. Thinking about it, version 1 might be better. We cannot know if the cached QgsGeos is really needed a second time and it might use more memory if not. In the first case, the client code knows if it is intended to do repeaded operations on the same geometry. What do you think? Could we put a giant NOT STABLE API AND MAY CHANGE for all these classes, so that we can address any unforeseen issues in 2.12? That would be ok. Great! That's my biggest reservation - if we've got the opportunity to tweak the API for 2.12 then my fears have been unfounded :) Thanks! Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
