Re: [QGIS-Developer] Performance issues while loading a big QGIS project
Thank you very much for the idea. This is also a very interesting approach we will consider. Kind regards, Josef Von: QGIS-Developer Im Auftrag von Bo Victor Thomsen Gesendet: Mittwoch, 25. September 2019 08:55 An: qgis-developer@lists.osgeo.org Betreff: Re: [QGIS-Developer] Performance issues while loading a big QGIS project That's a bloody huge project... I can't fathom you need all these layers at once. You could try to split the project in a "base" project with a minimum set of layers and split the rest in of the layers/groups into a number of qlr files. And use the "QLR Browser" plugin to load/unload the different qlr files as you need them. I personally think this method is better than using other plugins with similar functionality and certainly is better than loading 1080 layers at once. The QLR browser plugin is used in a number of Danish municipalities which typically uses a very large number of layers in their daily use of QGIS. -- Med venlig hilsen / Kind regards Bo Victor Thomsen Den 23-09-2019 kl. 16:34 skrev bauer.jo...@swm.de<mailto:bauer.jo...@swm.de>: Dear QGIS developers, we are facing some crucial performance issues while loading a big QGIS project. The project ... ... has 1080 layers ... has 167 layer groups ... is heavily styled (in fact, most of the information in the .qgs file are stylings) ... has a size of 53 MB (the .gqs file) ... loads the data from a PostGIS database The main problem we are facing is, that the loading time of the .qgs file is between 10 and 15 Minutes (depending on the network traffic). We don't need all the layers which are defined in the .qgs file immediately. In most cases, we just need a bunch of those layers at once. Therefore, the question is, if QGIS provides a mechanism to postpone the initial loading of all layers from QGIS startup to the point in time when we actually need the layers? I.e., the layer structure is loaded but not the layer data. Is there a build-in QGIS mechanism that allows loading only the needed layers from the .qgs file and write the appropriate styles to it or is it planned to implement such a mechanism in QGIS? Kind regards, Josef ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Performance issues while loading a big QGIS project
That's a bloody huge project... I can't fathom you need all these layers at once. You could try to split the project in a "base" project with a minimum set of layers and split the rest in of the layers/groups into a number of qlr files. And use the "QLR Browser" plugin to load/unload the different qlr files as you need them. I personally think this method is better than using other plugins with similar functionality and certainly is better than loading 1080 layers at once. The QLR browser plugin is used in a number of Danish municipalities which typically uses a very large number of layers in their daily use of QGIS. -- Med venlig hilsen / Kind regards Bo Victor Thomsen Den 23-09-2019 kl. 16:34 skrev bauer.jo...@swm.de: Dear QGIS developers, we are facing some crucial performance issues while loading a big QGIS project. The project ... ... has 1080 layers ... has 167 layer groups ... is heavily styled (in fact, most of the information in the .qgs file are stylings) ... has a size of 53 MB (the .gqs file) ... loads the data from a PostGIS database The main problem we are facing is, that the loading time of the .qgs file is between 10 and 15 Minutes (depending on the network traffic). We don't need all the layers which are defined in the .qgs file immediately. In most cases, we just need a bunch of those layers at once. Therefore, the question is, if QGIS provides a mechanism to postpone the initial loading of all layers from QGIS startup to the point in time when we actually need the layers? I.e., the layer structure is loaded but not the layer data. Is there a build-in QGIS mechanism that allows loading only the needed layers from the .qgs file and write the appropriate styles to it or is it planned to implement such a mechanism in QGIS? Kind regards, Josef ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Performance issues while loading a big QGIS project
Thanks for your feedback. Bye! Le mar. 24 sept. 2019 à 18:09, a écrit : > Hello Nyall, hello Régis, > > > > thank you for your hints. The option “Trust project when data source has > no metadata” improved the loading time a lot. The project still needs about > 6 minutes to load, but at least it is already an improvement. However, I > guess, that is normal for such a large project. The plug-in “Layers menu > from project” also looks very interesting. > > > > Kind regards, > > Josef > > > > *Von:* Régis Haubourg > *Gesendet:* Dienstag, 24. September 2019 08:30 > *An:* Nyall Dawson > *Cc:* Bauer.Josef extern IT-DS-TS ; QGIS Developers > List > *Betreff:* Re: [QGIS-Developer] Performance issues while loading a big > QGIS project > > > > Agreed, this should speed up the loading time of layers without metadata > (probably postgresql views mainly in your case) > > > > If your need is to access some layers only, I recommend the > menu_layers_from_project plugin. It will only parse the xml but not load > the data until you choose to import some layers into a session. > > > > I also experienced that lowering the network timeout can speed up ogc > webservice layers when the service is not responding. > > Regards > > Régis > > > > Le mar. 24 sept. 2019 à 00:49, Nyall Dawson a > écrit : > > On Tue, 24 Sep 2019 at 00:41, wrote: > > > > > Therefore, the question is, if QGIS provides a mechanism to postpone the > initial loading of all layers from QGIS startup to the point in time when > we actually need the layers? I.e., the layer structure is loaded but not > the layer data. > > There's no such existing mechanism. > > Have you experiment with the setting under project properties -- data > sources -- "Trust project when data source has no metadata"? I believe > that was added to assist with cases like this. > > Nyall > > > > > > > > > Is there a build-in QGIS mechanism that allows loading only the needed > layers from the .qgs file and write the appropriate styles to it or is it > planned to implement such a mechanism in QGIS? > > > > > > > > Kind regards, > > > > Josef > > > > ___ > > QGIS-Developer mailing list > > QGIS-Developer@lists.osgeo.org > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > ___ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Performance issues while loading a big QGIS project
Hello Nyall, hello Régis, thank you for your hints. The option “Trust project when data source has no metadata” improved the loading time a lot. The project still needs about 6 minutes to load, but at least it is already an improvement. However, I guess, that is normal for such a large project. The plug-in “Layers menu from project” also looks very interesting. Kind regards, Josef Von: Régis Haubourg Gesendet: Dienstag, 24. September 2019 08:30 An: Nyall Dawson Cc: Bauer.Josef extern IT-DS-TS ; QGIS Developers List Betreff: Re: [QGIS-Developer] Performance issues while loading a big QGIS project Agreed, this should speed up the loading time of layers without metadata (probably postgresql views mainly in your case) If your need is to access some layers only, I recommend the menu_layers_from_project plugin. It will only parse the xml but not load the data until you choose to import some layers into a session. I also experienced that lowering the network timeout can speed up ogc webservice layers when the service is not responding. Regards Régis Le mar. 24 sept. 2019 à 00:49, Nyall Dawson mailto:nyall.daw...@gmail.com>> a écrit : On Tue, 24 Sep 2019 at 00:41, mailto:bauer.jo...@swm.de>> wrote: > > Therefore, the question is, if QGIS provides a mechanism to postpone the > initial loading of all layers from QGIS startup to the point in time when we > actually need the layers? I.e., the layer structure is loaded but not the > layer data. There's no such existing mechanism. Have you experiment with the setting under project properties -- data sources -- "Trust project when data source has no metadata"? I believe that was added to assist with cases like this. Nyall > > > > Is there a build-in QGIS mechanism that allows loading only the needed layers > from the .qgs file and write the appropriate styles to it or is it planned to > implement such a mechanism in QGIS? > > > > Kind regards, > > Josef > > ___ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org<mailto:QGIS-Developer@lists.osgeo.org> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Performance issues while loading a big QGIS project
Agreed, this should speed up the loading time of layers without metadata (probably postgresql views mainly in your case) If your need is to access some layers only, I recommend the menu_layers_from_project plugin. It will only parse the xml but not load the data until you choose to import some layers into a session. I also experienced that lowering the network timeout can speed up ogc webservice layers when the service is not responding. Regards Régis Le mar. 24 sept. 2019 à 00:49, Nyall Dawson a écrit : > On Tue, 24 Sep 2019 at 00:41, wrote: > > > > > Therefore, the question is, if QGIS provides a mechanism to postpone the > initial loading of all layers from QGIS startup to the point in time when > we actually need the layers? I.e., the layer structure is loaded but not > the layer data. > > There's no such existing mechanism. > > Have you experiment with the setting under project properties -- data > sources -- "Trust project when data source has no metadata"? I believe > that was added to assist with cases like this. > > Nyall > > > > > > > > > Is there a build-in QGIS mechanism that allows loading only the needed > layers from the .qgs file and write the appropriate styles to it or is it > planned to implement such a mechanism in QGIS? > > > > > > > > Kind regards, > > > > Josef > > > > ___ > > QGIS-Developer mailing list > > QGIS-Developer@lists.osgeo.org > > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer > ___ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Performance issues while loading a big QGIS project
On Tue, 24 Sep 2019 at 00:41, wrote: > > Therefore, the question is, if QGIS provides a mechanism to postpone the > initial loading of all layers from QGIS startup to the point in time when we > actually need the layers? I.e., the layer structure is loaded but not the > layer data. There's no such existing mechanism. Have you experiment with the setting under project properties -- data sources -- "Trust project when data source has no metadata"? I believe that was added to assist with cases like this. Nyall > > > > Is there a build-in QGIS mechanism that allows loading only the needed layers > from the .qgs file and write the appropriate styles to it or is it planned to > implement such a mechanism in QGIS? > > > > Kind regards, > > Josef > > ___ > QGIS-Developer mailing list > QGIS-Developer@lists.osgeo.org > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
[QGIS-Developer] Performance issues while loading a big QGIS project
Dear QGIS developers, we are facing some crucial performance issues while loading a big QGIS project. The project ... ... has 1080 layers ... has 167 layer groups ... is heavily styled (in fact, most of the information in the .qgs file are stylings) ... has a size of 53 MB (the .gqs file) ... loads the data from a PostGIS database The main problem we are facing is, that the loading time of the .qgs file is between 10 and 15 Minutes (depending on the network traffic). We don't need all the layers which are defined in the .qgs file immediately. In most cases, we just need a bunch of those layers at once. Therefore, the question is, if QGIS provides a mechanism to postpone the initial loading of all layers from QGIS startup to the point in time when we actually need the layers? I.e., the layer structure is loaded but not the layer data. Is there a build-in QGIS mechanism that allows loading only the needed layers from the .qgs file and write the appropriate styles to it or is it planned to implement such a mechanism in QGIS? Kind regards, Josef ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
