Re: [Qgis-user] Snapping tools really slow

[Paul Wittle]

Hi,

I looks like it was something to do with the hybrid indexing method. I used the 
python code in the previous reply and presto...it works.

We had already checked and there was nothing wrong with the speed of the data 
load. You turn the layer on and it is pretty much instantly displayed so I'm 
guessing something with the way it calculates the index was the issue.

I'm told it was voted that the option should not be added to the settings 
because it would confuse people. Personally I think you should be more 
concerned about users out there who might experience similar performance and 
never mention it; potentially deciding to use other software as well.

We had been putting up with the slow performance till now but that single 
setting makes QGIS runs super-fast and smooth.

My opinion is just that, my opinion, but I hope that those discussing it will 
consider making the default something that can be configured in the global 
settings ui file. Alternatively it might be worth investigating why the hybrid 
method is so slow and just fixing that. They really are not comparable in our 
install, one is instantly available and the other takes literally 5 minutes!

Many thanks for the help everyone; I'm really really pleased with the 
performance now! 😊

Paul

-Original Message-
From: Philip Barlow 
Sent: 22 January 2021 13:39
To: Paul Wittle ; Julien Cabieces 

Cc: qgis-user@lists.osgeo.org
Subject: RE: [Qgis-user] Snapping tools really slow

Hi Paul,

What sort of database are you using?  MSSQL or Postgres?

We're on Postgres13 and PostGIS 3.0. A quick test with QGIS 3.10.14 digitising 
to OSMM... there is a small delay but its only around a second or two. You may 
have a larger extent than me in your database though? Mine is just the whole of 
the County of Pembrokeshire.  I haven't tried 3.16 to see if there is any 
noticeable difference.

Regards

Phil



-Original Message-
From: Qgis-user  On Behalf Of Paul Wittle
Sent: 22 January 2021 08:10
To: Julien Cabieces 
Cc: qgis-user@lists.osgeo.org
Subject: Re: [Qgis-user] Snapping tools really slow

Hi,

It might be best if I screen record you a video of it loading so you can see 
the performance issue as it is always difficult to discuss in text. I'll have 
to send that to you offline from this thread as I believe videos aren't 
supported by this group.

We are snapping to Ordnance Survey MasterMap because we work in the public 
sector and are permitted to do so for council business. Depending on licencing 
you may not be able to access the same level of OS data as you would probably 
have to pay for it (assuming your company is private) so I guess that would not 
be helpful in terms of replicating the issue. That said; I also tried it using 
the OS Maps API (WFS) and it was also slow so OS may be able to share that with 
you for test purposes.

In the older version 3.10 the progress bar always disappeared for some reason 
so it appeared to just hand whilst the index builds. In the new version a blue 
bar appears at the bottom but it is not a loading bar it is the one you get 
when the application doesn't know how long is left. I tried to stop the process 
because it was long running and it said 'Indexing' was running did I want to 
try and stop the process so I'm pretty sure it is the indexing that is not 
working for some reason.

Is there a way to pre-build the snapping index?

It works fine for smaller layers, things we create etc. If I can resolve this 
one layer then all my problems go away so we would be really happy to pre-cache 
the snapping index if that is possible.

Thanks,
Paul

-Original Message-
From: Julien Cabieces 
Sent: 22 January 2021 05:32
To: Paul Wittle 
Cc: qgis-user@lists.osgeo.org
Subject: Re: [Qgis-user] Snapping tools really slow


Hi,

I made quick tests with some dataset and I failed to see any performance issues.

Could you create an issue with some data sample and project?

Is it the index building that take a long time (there is a progress bar 
displayed at the bottom of the map close to the mouse coordinate information 
while index is currently building) or is it when you move your cursor on the 
map that the snaping information takes time to appear. Or is it when you pan 
the map?

You can enable snapping for specific range of scale, avoiding building index at 
very large scale where snapping is useless.

Regards,
Julien



> Hi,
>
> I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
> extremely slow and oddly significantly slower on 3.16 than 3.10. We thought 
> it might be downloading the dataset on screen but discovered the data is back 
> from the database within seconds and the delay is because QGIS is building an 
> index.
>
> Why is it so inefficient and is there anything that can be done to get better 
> performance?
>
> It is really slow and we are trying to tran

Re: [Qgis-user] Snapping tools really slow

[Philip Barlow]

Hi Paul,

What sort of database are you using?  MSSQL or Postgres? 

We're on Postgres13 and PostGIS 3.0. A quick test with QGIS 3.10.14 digitising 
to OSMM... there is a small delay but its only around a second or two. You may 
have a larger extent than me in your database though? Mine is just the whole of 
the County of Pembrokeshire.  I haven't tried 3.16 to see if there is any 
noticeable difference.

Regards

Phil



-Original Message-
From: Qgis-user  On Behalf Of Paul Wittle
Sent: 22 January 2021 08:10
To: Julien Cabieces 
Cc: qgis-user@lists.osgeo.org
Subject: Re: [Qgis-user] Snapping tools really slow

Hi,

It might be best if I screen record you a video of it loading so you can see 
the performance issue as it is always difficult to discuss in text. I'll have 
to send that to you offline from this thread as I believe videos aren't 
supported by this group.

We are snapping to Ordnance Survey MasterMap because we work in the public 
sector and are permitted to do so for council business. Depending on licencing 
you may not be able to access the same level of OS data as you would probably 
have to pay for it (assuming your company is private) so I guess that would not 
be helpful in terms of replicating the issue. That said; I also tried it using 
the OS Maps API (WFS) and it was also slow so OS may be able to share that with 
you for test purposes.

In the older version 3.10 the progress bar always disappeared for some reason 
so it appeared to just hand whilst the index builds. In the new version a blue 
bar appears at the bottom but it is not a loading bar it is the one you get 
when the application doesn't know how long is left. I tried to stop the process 
because it was long running and it said 'Indexing' was running did I want to 
try and stop the process so I'm pretty sure it is the indexing that is not 
working for some reason.

Is there a way to pre-build the snapping index?

It works fine for smaller layers, things we create etc. If I can resolve this 
one layer then all my problems go away so we would be really happy to pre-cache 
the snapping index if that is possible.

Thanks,
Paul

-Original Message-
From: Julien Cabieces 
Sent: 22 January 2021 05:32
To: Paul Wittle 
Cc: qgis-user@lists.osgeo.org
Subject: Re: [Qgis-user] Snapping tools really slow


Hi,

I made quick tests with some dataset and I failed to see any performance issues.

Could you create an issue with some data sample and project?

Is it the index building that take a long time (there is a progress bar 
displayed at the bottom of the map close to the mouse coordinate information 
while index is currently building) or is it when you move your cursor on the 
map that the snaping information takes time to appear. Or is it when you pan 
the map?

You can enable snapping for specific range of scale, avoiding building index at 
very large scale where snapping is useless.

Regards,
Julien



> Hi,
>
> I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
> extremely slow and oddly significantly slower on 3.16 than 3.10. We thought 
> it might be downloading the dataset on screen but discovered the data is back 
> from the database within seconds and the delay is because QGIS is building an 
> index.
>
> Why is it so inefficient and is there anything that can be done to get better 
> performance?
>
> It is really slow and we are trying to transfer users from MapInfo / ESRI so 
> they will not like the performance at the moment.
>
> To be clear the data has a primary key, spatial indices and all is of good 
> quality. As I say the data downloads and displays within seconds on the map; 
> is it because I've selected snapping to segment as well as vertex?
>
> Thanks,
>
> Paul Wittle
> [cid:image001.jpg@01D6F016.C23D1CA0]<https://www.dorsetcouncil.gov.uk/
> >
> Business Solutions Analyst (GIS)
> ICT Operations
> Dorset Council
> 01305 228473
>  %20%20%20> dorsetcouncil.gov.uk<https://www.dorsetcouncil.gov.uk>
>
> [cid:image002.png@01D6F016.C23D1CA0]<https://www.facebook.com/DorsetCo
> uncilUK>
> [cid:image003.png@01D6F016.C23D1CA0]<https://instagram.com/DorsetCounc
> ilUK>
> [cid:image004.png@01D6F016.C23D1CA0]<https://twitter.com/DorsetCouncil
> UK>
>
> This e-mail and any files transmitted with it are intended solely for 
> the use of the individual or entity to whom they are addressed. It may 
> contain unclassified but sensitive or protectively marked material and 
> should be handled accordingly. Unless you are the named addressee (or 
> authorised to receive it for the addressee) you may not copy or use 
> it, or disclose it to anyone else. If you have received this 
> transmission in error please notify the sender immediately. All 
> traffic may be subject to recording 

Re: [Qgis-user] Snapping tools really slow

[Julien Cabieces]

Hi

> Hi,
>
> It might be best if I screen record you a video of it loading so you can see 
> the performance issue as it is always difficult to discuss in text. I'll have 
> to send that to you offline from this thread as I believe videos aren't 
> supported by this group.
>

I agree, I'll wait for a video.

> We are snapping to Ordnance Survey MasterMap because we work in the
> public sector and are permitted to do so for council
> business. Depending on licencing you may not be able to access the
> same level of OS data as you would probably have to pay for it
> (assuming your company is private) so I guess that would not be
> helpful in terms of replicating the issue. That said; I also tried it
> using the OS Maps API (WFS) and it was also slow so OS may be able to
> share that with you for test purposes.

I try with some WFS, and indexing is as slow as rendering, meaning that
this probably data retrieval which slow down things under the hood.

>
> In the older version 3.10 the progress bar always disappeared for some
> reason so it appeared to just hand whilst the index builds. In the new
> version a blue bar appears at the bottom but it is not a loading bar
> it is the one you get when the application doesn't know how long is
> left. I tried to stop the process because it was long running and it
> said 'Indexing' was running did I want to try and stop the process so
> I'm pretty sure it is the indexing that is not working for some
> reason.

The progress bar disappears so the indexing is made in parallel for all
your layers and you can edit even if the index is not ready yet (snapping
would be not available until the index is ready though).

>
> Is there a way to pre-build the snapping index?
>

No. For some huge layers (millions of vertex), it's not possible to
index everything. QGIS try to guess the best reasonnable area to index
and rebuild it everytime you move outside this area.

That's why I told you about snapping range scale. QGIS would trigger
index building only in certain range scale (the relevant one for
edition), which decrease the size of index to build.

For some users who have huge amount of data, I also advice them to
change the indexing strategy using Python macros (Project > Properties >
Macro) with this code.


from qgis.utils import iface
from qgis.core import QgsSnappingUtils

def openProject():

iface.mapCanvas().snappingUtils().setIndexingStrategy(QgsSnappingUtils.IndexExtent)

def saveProject():
pass

def closeProject():

iface.mapCanvas().snappingUtils().setIndexingStrategy(QgsSnappingUtils.IndexHybrid)


It would build index only for the canvas extent. With the appropriate
snapping range scale, you manage to avoid this long time waiting for
indexing (but it built index everytime you move...).

I'm ok it's not ideal, but so far this is the only way I know to manage
spapping on huge amount of data.


> It works fine for smaller layers, things we create etc. If I can resolve this 
> one layer then all my problems go away so we would be really happy to 
> pre-cache the snapping index if that is possible.
>
> Thanks,
> Paul
>
> -----Original Message-
> From: Julien Cabieces 
> Sent: 22 January 2021 05:32
> To: Paul Wittle 
> Cc: qgis-user@lists.osgeo.org
> Subject: Re: [Qgis-user] Snapping tools really slow
>
>
> Hi,
>
> I made quick tests with some dataset and I failed to see any performance 
> issues.
>
> Could you create an issue with some data sample and project?
>
> Is it the index building that take a long time (there is a progress bar 
> displayed at the bottom of the map close to the mouse coordinate information 
> while index is currently building) or is it when you move your cursor on the 
> map that the snaping information takes time to appear. Or is it when you pan 
> the map?
>
> You can enable snapping for specific range of scale, avoiding building index 
> at very large scale where snapping is useless.
>
> Regards,
> Julien
>
>
>
>> Hi,
>>
>> I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
>> extremely slow and oddly significantly slower on 3.16 than 3.10. We thought 
>> it might be downloading the dataset on screen but discovered the data is 
>> back from the database within seconds and the delay is because QGIS is 
>> building an index.
>>
>> Why is it so inefficient and is there anything that can be done to get 
>> better performance?
>>
>> It is really slow and we are trying to transfer users from MapInfo / ESRI so 
>> they will not like the performance at the moment.
>>
>> To be clear the data has a primary key, spatial indices and all is of good 
>> quality. A

Re: [Qgis-user] Snapping tools really slow

[Paul Wittle]

Hi,

It might be best if I screen record you a video of it loading so you can see 
the performance issue as it is always difficult to discuss in text. I'll have 
to send that to you offline from this thread as I believe videos aren't 
supported by this group.

We are snapping to Ordnance Survey MasterMap because we work in the public 
sector and are permitted to do so for council business. Depending on licencing 
you may not be able to access the same level of OS data as you would probably 
have to pay for it (assuming your company is private) so I guess that would not 
be helpful in terms of replicating the issue. That said; I also tried it using 
the OS Maps API (WFS) and it was also slow so OS may be able to share that with 
you for test purposes.

In the older version 3.10 the progress bar always disappeared for some reason 
so it appeared to just hand whilst the index builds. In the new version a blue 
bar appears at the bottom but it is not a loading bar it is the one you get 
when the application doesn't know how long is left. I tried to stop the process 
because it was long running and it said 'Indexing' was running did I want to 
try and stop the process so I'm pretty sure it is the indexing that is not 
working for some reason.

Is there a way to pre-build the snapping index?

It works fine for smaller layers, things we create etc. If I can resolve this 
one layer then all my problems go away so we would be really happy to pre-cache 
the snapping index if that is possible.

Thanks,
Paul

-Original Message-
From: Julien Cabieces 
Sent: 22 January 2021 05:32
To: Paul Wittle 
Cc: qgis-user@lists.osgeo.org
Subject: Re: [Qgis-user] Snapping tools really slow


Hi,

I made quick tests with some dataset and I failed to see any performance issues.

Could you create an issue with some data sample and project?

Is it the index building that take a long time (there is a progress bar 
displayed at the bottom of the map close to the mouse coordinate information 
while index is currently building) or is it when you move your cursor on the 
map that the snaping information takes time to appear. Or is it when you pan 
the map?

You can enable snapping for specific range of scale, avoiding building index at 
very large scale where snapping is useless.

Regards,
Julien



> Hi,
>
> I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
> extremely slow and oddly significantly slower on 3.16 than 3.10. We thought 
> it might be downloading the dataset on screen but discovered the data is back 
> from the database within seconds and the delay is because QGIS is building an 
> index.
>
> Why is it so inefficient and is there anything that can be done to get better 
> performance?
>
> It is really slow and we are trying to transfer users from MapInfo / ESRI so 
> they will not like the performance at the moment.
>
> To be clear the data has a primary key, spatial indices and all is of good 
> quality. As I say the data downloads and displays within seconds on the map; 
> is it because I've selected snapping to segment as well as vertex?
>
> Thanks,
>
> Paul Wittle
> [cid:image001.jpg@01D6F016.C23D1CA0]<https://www.dorsetcouncil.gov.uk/
> >
> Business Solutions Analyst (GIS)
> ICT Operations
> Dorset Council
> 01305 228473
>  %20%20%20> dorsetcouncil.gov.uk<https://www.dorsetcouncil.gov.uk>
>
> [cid:image002.png@01D6F016.C23D1CA0]<https://www.facebook.com/DorsetCo
> uncilUK>
> [cid:image003.png@01D6F016.C23D1CA0]<https://instagram.com/DorsetCounc
> ilUK>
> [cid:image004.png@01D6F016.C23D1CA0]<https://twitter.com/DorsetCouncil
> UK>
>
> This e-mail and any files transmitted with it are intended solely for
> the use of the individual or entity to whom they are addressed. It may
> contain unclassified but sensitive or protectively marked material and
> should be handled accordingly. Unless you are the named addressee (or
> authorised to receive it for the addressee) you may not copy or use
> it, or disclose it to anyone else. If you have received this
> transmission in error please notify the sender immediately. All
> traffic may be subject to recording and/or monitoring in accordance
> with relevant legislation. Any views expressed in this message are
> those of the individual sender, except where the sender specifies and
> with authority, states them to be the views of Dorset Council. Dorset
> Council does not accept service of documents by fax or other
> electronic means. Virus checking: Whilst all reasonable steps have
> been taken to ensure that this electronic communication and its
> attachments whether encoded, encrypted or otherwise supplied are free
> from computer viruses, Dorset Council accepts no liability in respec

Re: [Qgis-user] Snapping tools really slow

[Julien Cabieces]

Hi,

I made quick tests with some dataset and I failed to see any performance
issues.

Could you create an issue with some data sample and project?

Is it the index building that take a long time (there is a progress bar
displayed at the bottom of the map close to the mouse coordinate information
while index is currently building) or is it when you move your cursor on the 
map that the
snaping information takes time to appear. Or is it when you pan the map?

You can enable snapping for specific range of scale, avoiding building
index at very large scale where snapping is useless.

Regards,
Julien



> Hi,
>
> I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
> extremely slow and oddly significantly slower on 3.16 than 3.10. We thought 
> it might be downloading the dataset on screen but discovered the data is back 
> from the database within seconds and the delay is because QGIS is building an 
> index.
>
> Why is it so inefficient and is there anything that can be done to get better 
> performance?
>
> It is really slow and we are trying to transfer users from MapInfo / ESRI so 
> they will not like the performance at the moment.
>
> To be clear the data has a primary key, spatial indices and all is of good 
> quality. As I say the data downloads and displays within seconds on the map; 
> is it because I've selected snapping to segment as well as vertex?
>
> Thanks,
>
> Paul Wittle
> [cid:image001.jpg@01D6F016.C23D1CA0]
> Business Solutions Analyst (GIS)
> ICT Operations
> Dorset Council
> 01305 228473 
> 
> dorsetcouncil.gov.uk
>
> [cid:image002.png@01D6F016.C23D1CA0]
> [cid:image003.png@01D6F016.C23D1CA0]
> [cid:image004.png@01D6F016.C23D1CA0]
>
> This e-mail and any files transmitted with it are intended solely for the use 
> of the individual or entity to whom they are addressed. It may contain 
> unclassified but sensitive or protectively marked material and should be 
> handled accordingly. Unless you are the named addressee (or authorised to 
> receive it for the addressee) you may not copy or use it, or disclose it to 
> anyone else. If you have received this transmission in error please notify 
> the sender immediately. All traffic may be subject to recording and/or 
> monitoring in accordance with relevant legislation. Any views expressed in 
> this message are those of the individual sender, except where the sender 
> specifies and with authority, states them to be the views of Dorset Council. 
> Dorset Council does not accept service of documents by fax or other 
> electronic means. Virus checking: Whilst all reasonable steps have been taken 
> to ensure that this electronic communication and its attachments whether 
> encoded, encrypted or otherwise su
 pplied are free from computer viruses, Dorset Council accepts no liability in 
respect of any loss, cost, damage or expense suffered as a result of accessing 
this message or any of its attachments. For information on how Dorset Council 
processes your information, please see www.dorsetcouncil.gov.uk/416433
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

[Qgis-user] Snapping tools really slow

[Paul Wittle]

Hi,

I've been using version 3.10 and 3.16 and I notice that enabling snapping is 
extremely slow and oddly significantly slower on 3.16 than 3.10. We thought it 
might be downloading the dataset on screen but discovered the data is back from 
the database within seconds and the delay is because QGIS is building an index.

Why is it so inefficient and is there anything that can be done to get better 
performance?

It is really slow and we are trying to transfer users from MapInfo / ESRI so 
they will not like the performance at the moment.

To be clear the data has a primary key, spatial indices and all is of good 
quality. As I say the data downloads and displays within seconds on the map; is 
it because I've selected snapping to segment as well as vertex?

Thanks,

Paul Wittle
[cid:image001.jpg@01D6F016.C23D1CA0]
Business Solutions Analyst (GIS)
ICT Operations
Dorset Council
01305 228473 

dorsetcouncil.gov.uk

[cid:image002.png@01D6F016.C23D1CA0]
[cid:image003.png@01D6F016.C23D1CA0]
[cid:image004.png@01D6F016.C23D1CA0]

This e-mail and any files transmitted with it are intended solely for the use 
of the individual or entity to whom they are addressed. It may contain 
unclassified but sensitive or protectively marked material and should be 
handled accordingly. Unless you are the named addressee (or authorised to 
receive it for the addressee) you may not copy or use it, or disclose it to 
anyone else. If you have received this transmission in error please notify the 
sender immediately. All traffic may be subject to recording and/or monitoring 
in accordance with relevant legislation. Any views expressed in this message 
are those of the individual sender, except where the sender specifies and with 
authority, states them to be the views of Dorset Council. Dorset Council does 
not accept service of documents by fax or other electronic means. Virus 
checking: Whilst all reasonable steps have been taken to ensure that this 
electronic communication and its attachments whether encoded, encrypted or 
otherwise supplied are free from computer viruses, Dorset Council accepts no 
liability in respect of any loss, cost, damage or expense suffered as a result 
of accessing this message or any of its attachments. For information on how 
Dorset Council processes your information, please see 
www.dorsetcouncil.gov.uk/416433
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user