Re: [Qgis-developer] [Server] performance questions

[Marco Hugentobler]

Hi all

This is not a server specific problem. It is that QGIS may need a long 
time to get metadata for some layers. So the best solution is (as it has 
been proposed already in this thread) to enhance QGIS to store the 
metadata (in the project file) and not read it from the db the first 
time. This will also speed up loading of large projects, it is not only 
usefull for the server. We should avoid to make any server specific 
things in core for that.


>- there was a bug in QGIS Server which did not use cache whenever a 
GetMap request was called with an empty STYLE parameter


No, the opposite. It did not use cache with a non-empty style parameter.

Regards,
Marco

On 27.02.2016 10:32, kimaidou wrote:

Hi all,

I have made some comparison on requests done when QGIS Server is 
called for a simple GetMap, on the first initialization and aftewards 
when the cache is found.


Here are the logs

When QGIS builds the maplayer object, ie when there is no cache for 
this layer yet (first call, for example with a getcapabilities). This 
is only done if no cache is found for this layer in QGIS Server


1/ Log when estimated metadata is used (and stats are available in 
PostGIS)

https://paste.sh/yguqyRWL#EPcHHVH3uuijGgEtzH0pAlHb

2/ Log when QGIS does not use the metadata for a layer . Which means 
this is done for any view at present, since no estimated metadata is 
available, or also for a table if the user forgot to check the option 
"use metadata" when he first created the Postgis connection (which can 
be often the case because this option is not checked by default in 
PostgreSQL connection dialog )

https://paste.sh/frCglZ9A#A1nQ6BXtdBeEiZWQmTJP8v9l

3/ Log when a layer is already in QGIS Server cache, for example for a 
GetMap request. This means for table and views, since it does not 
depend on metadata existence or not, but only on cache

https://paste.sh/Rkvqf757#K1MCRitovIxMblBVils1VF7r

My conclusions here :

In the first example 1/, we see all the queries QGIS does to get all 
the needed information for building the maplayer object

* postgis information : postgis_version() , postgis_geos_version()
* search for topology features -> not really usefull server side I 
think ( ? )

* feature count . In 1, estimated metadata is used, so this query is fast
* layer extent calculation . In 1/ QGIS uses st_estimatedextent, so it 
also should be fast



In the second example 2/, without available stats (no use of estimated 
metadata for views or when the option is not checked in layer ), we 
can see the performance killing queries (for complex or big datasets) 
done by QGIS to get the needed information :


* check if the field used as primary key is unique :
SELECT count(distinct ""nb"")=count(""nb"") FROM (SELECT row_number() 
OVER() AS nb, * FROM ""2013"".v_geo_parcelle) AS ""subQuery_0"""


* Get the extent :
SELECT st_extent(""geom"") FROM (SELECT row_number() OVER() AS nb, * 
FROM ""2013"".v_geo_parcelle) AS ""subQuery_0""



In 3/, QGIS has already built the cache for the maplayer, and only 
fetches the features, which is very fast.



My thoughts on how to improve QGIS server performance with PostgreSQL 
provider (and possibly for other providers as well )


* check that layer cache in QGIS server is well used.
- I do not know for example if the 100 default number is for the whole 
server (which is then reached very fast and will become useless if 
there are many projects served by QGIS), or for each projects.
- there was a bug in QGIS Server which did not use cache whenever a 
GetMap request was called with an empty STYLE parameter. It has been 
fixed but check your version is not older to this commit : 
https://github.com/qgis/QGIS/commit/a5450a78ffcbba9a75afa8be43460ad741fea050 



* Add an option in QGIS ( in OWS server properties ) to tell QGIS 
Server to not get some information if the user who publish the project 
knows what he does, meaning

  - use the extent stored in  and do not ask for it server side
  - do NOT check that the primary key is unique -> the person who 
built the project and publish it does know that his layer has a unique 
primary key.
Note that there is a limit in the number of layers QGIS Server will 
cache ( 100 by default).


* Add a even more hard way of telling QGIS Server "Never delete the 
cache for this layer, as the extent and the other properties wont 
change (even if I allow editing inside my known layer extent )" . This 
means the cache will be store permanently for this layer and would be 
lost only if the apache / nginx server is reloaded . I do not know if 
it is feasible


* Store layers metadata "the QGIS way" in the database, as we do for 
styles. This means the users building the project has enough rights on 
the postgreSQL database, but could be a handy QGIS solution for 
production ready projects. This way we could store the metadata for 
tables AND views, in a format close to what QGIS needs to build the 
maplayer object. The user would define some op

Re: [Qgis-developer] Build error (local issue)

[Matthias Kuhn]

Yves,

Integer64 attribute support was added to GDAL with version 2.0 and QGIS
only uses it with GDAL >= 2.0. It seems you were using a (development)
version that pretends to be >=2.0 but does not yet include Integer64
attribute support.
Solution: downgrade or upgrade to a stable GDAL release.

Matthias

On 02/28/2016 08:46 PM, Yves Jacolin wrote:
> $ gdalinfo --version
> GDAL 2.0.0dev, released 2014/04/16
>
> This is too recent?
>
> I have probably an official gdal package installed also :/
>
> Y.
> Le dimanche 28 février 2016, 20:23:26 Matthias Kuhn a écrit :
>> Hi Yves,
>>
>> Which GDAL version is installed?
>>
>> Matthias
>>
>> On 02/28/2016 06:56 PM, Yves Jacolin wrote:
>>> Hello,
>>>
>>> From several weeks/monthes I get this error when building QGIS in my home
>>> computer:
>>>
>>> [ 11%] Building CXX object
>>> src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o
>>> /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp: In member function
>>> ‘void* QgsVectorFileWriter::createFeature(QgsFeature&)’:
>>> /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp:1778:78: error:
>>> ‘OGR_F_SetFieldInteger64’ was not declared in this scope
>>>
>>>  OGR_F_SetFieldInteger64( poFeature, ogrField,
>>>  attrValue.toLongLong()
>>>
>>> );
>>>
>>> 
>>>   
> ^
>>> make[2]: *** [src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o]
>>> Erreur 1
>>> make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Erreur 2
>>> make: *** [all] Erreur 2
>>>
>>> Do you have any idea on this?
>>>
>>> Thanks,
>>>
>>> Y.

-- 
Matthias Kuhn
OPENGIS.ch - https://www.opengis.ch
Spatial • (Q)GIS • PostGIS • Open Source




signature.asc
Description: OpenPGP digital signature
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Build error (local issue)

[Yves Jacolin]

Matthias,

Seems better now that I updated my gdal build.

Thanks,

Y.
Yves JacolinLe dimanche 28 février 2016, 20:46:10  a écrit :
> $ gdalinfo --version
> GDAL 2.0.0dev, released 2014/04/16
> 
> This is too recent?
> 
> I have probably an official gdal package installed also :/
> 
> Y.
> 
> Le dimanche 28 février 2016, 20:23:26 Matthias Kuhn a écrit :
> > Hi Yves,
> > 
> > Which GDAL version is installed?
> > 
> > Matthias
> > 
> > On 02/28/2016 06:56 PM, Yves Jacolin wrote:
> > > Hello,
> > > 
> > > From several weeks/monthes I get this error when building QGIS in my
> > > home
> > > computer:
> > > 
> > > [ 11%] Building CXX object
> > > src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o
> > > /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp: In member function
> > > ‘void* QgsVectorFileWriter::createFeature(QgsFeature&)’:
> > > /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp:1778:78: error:
> > > ‘OGR_F_SetFieldInteger64’ was not declared in this scope
> > > 
> > >  OGR_F_SetFieldInteger64( poFeature, ogrField,
> > >  attrValue.toLongLong()
> > > 
> > > );
> 
> ^
> 
> > > make[2]: ***
> > > [src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o]
> > > Erreur 1
> > > make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Erreur 2
> > > make: *** [all] Erreur 2
> > > 
> > > Do you have any idea on this?
> > > 
> > > Thanks,
> > > 
> > > Y.

-- 
Yves Jacolin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Build error (local issue)

[Yves Jacolin]

$ gdalinfo --version
GDAL 2.0.0dev, released 2014/04/16

This is too recent?

I have probably an official gdal package installed also :/

Y.
Le dimanche 28 février 2016, 20:23:26 Matthias Kuhn a écrit :
> Hi Yves,
> 
> Which GDAL version is installed?
> 
> Matthias
> 
> On 02/28/2016 06:56 PM, Yves Jacolin wrote:
> > Hello,
> > 
> > From several weeks/monthes I get this error when building QGIS in my home
> > computer:
> > 
> > [ 11%] Building CXX object
> > src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o
> > /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp: In member function
> > ‘void* QgsVectorFileWriter::createFeature(QgsFeature&)’:
> > /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp:1778:78: error:
> > ‘OGR_F_SetFieldInteger64’ was not declared in this scope
> > 
> >  OGR_F_SetFieldInteger64( poFeature, ogrField,
> >  attrValue.toLongLong()
> > 
> > );
> > 
> > 
> >   
^
> > 
> > make[2]: *** [src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o]
> > Erreur 1
> > make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Erreur 2
> > make: *** [all] Erreur 2
> > 
> > Do you have any idea on this?
> > 
> > Thanks,
> > 
> > Y.

-- 
Yves Jacolin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Build error (local issue)

[Matthias Kuhn]

Hi Yves,

Which GDAL version is installed?

Matthias

On 02/28/2016 06:56 PM, Yves Jacolin wrote:
> Hello,
>
> From several weeks/monthes I get this error when building QGIS in my home 
> computer:
>
> [ 11%] Building CXX object 
> src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o
> /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp: In member function 
> ‘void* 
> QgsVectorFileWriter::createFeature(QgsFeature&)’:
> /usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp:1778:78: error: 
> ‘OGR_F_SetFieldInteger64’ was not declared in this scope
>  OGR_F_SetFieldInteger64( poFeature, ogrField, attrValue.toLongLong() 
> );
>   
> ^
> make[2]: *** [src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o] 
> Erreur 1
> make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Erreur 2
> make: *** [all] Erreur 2
>
> Do you have any idea on this?
>
> Thanks,
>
> Y.

-- 
Matthias Kuhn
OPENGIS.ch - https://www.opengis.ch
Spatial • (Q)GIS • PostGIS • Open Source




signature.asc
Description: OpenPGP digital signature
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Build error (local issue)

[Yves Jacolin]

Hello,

From several weeks/monthes I get this error when building QGIS in my home 
computer:

[ 11%] Building CXX object 
src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o
/usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp: In member function ‘void* 
QgsVectorFileWriter::createFeature(QgsFeature&)’:
/usr/local/src/QGIS/src/core/qgsvectorfilewriter.cpp:1778:78: error: 
‘OGR_F_SetFieldInteger64’ was not declared in this scope
 OGR_F_SetFieldInteger64( poFeature, ogrField, attrValue.toLongLong() 
);
  ^
make[2]: *** [src/core/CMakeFiles/qgis_core.dir/qgsvectorfilewriter.cpp.o] 
Erreur 1
make[1]: *** [src/core/CMakeFiles/qgis_core.dir/all] Erreur 2
make: *** [all] Erreur 2

Do you have any idea on this?

Thanks,

Y.
-- 
Yves Jacolin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] A perfect day to write a new test!

[Matthias Kuhn]

Hi Sachin,

I forward this mail to the QGIS developer list, where many more
developers are around who may be able to answer your question.

Regards,
Matthias

On 02/27/2016 10:30 AM, Sachin Chowdhary wrote:
>
> Hi sir,
>
> Do you know the procedure to make exe after successful build of
> Qgis2.8.3 in msvs2010.
>
> On 26-Feb-2016 5:10 PM, "Matthias Kuhn"  > wrote:
>
> Hi all, today is a great day to add a new test to QGIS.
>
> This is the start of an almost 4 year old email [1]. But despite its age, 
> its headline is more valid than ever.
>
> It's Friday again, so in the spirit of "Test Friday", it's a great day to 
> write a test. If you don't have much time or if you don't feel comfortable 
> with programming, there is a new infrastructure for processing tests in place 
> which requires GIS experts and not programmers to write tests [2]. Go check 
> it out!
>
> I have already spent some time today and created a pull request to 
> improve the behavior of epected failures [3].
>
> Happy Test Friday!
>
> Regards
> Matthias
>
> [1] 
> https://lists.osgeo.org/pipermail/qgis-developer/2012-September/022325.html
> [2] 
> https://github.com/qgis/QGIS/tree/master/python/plugins/processing/tests
> [3] https://github.com/qgis/QGIS/pull/2846
>
> -- 
> Matthias Kuhn
> OPENGIS.ch - https://www.opengis.ch
> Spatial • (Q)GIS • PostGIS • Open Source
>
>
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org 
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>

-- 
Matthias Kuhn
OPENGIS.ch - https://www.opengis.ch
Spatial • (Q)GIS • PostGIS • Open Source



signature.asc
Description: OpenPGP digital signature
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] [QGIS-UX] Key modifiers with selection

[Anita Graser]

On Sun, Dec 20, 2015 at 4:41 PM, Anita Graser  wrote:

> Hi,
>
> On Thu, Dec 3, 2015 at 10:59 AM, Matthias Kuhn 
> wrote:
>
>>
>>
>> On 12/03/2015 10:36 AM, Anita Graser wrote:
>>
>>
>>
>> On Thu, Dec 3, 2015 at 1:19 AM, Nyall Dawson 
>> wrote:
>>>
>>> Actually, composer is:
>>> - holding shift = add to selection
>>> - holiding ctrl = substract from selection
>>> - holding alt (can be used with shift/ctrl too) = change from
>>> "intersects" to "fully contains" selection mode
>>>
>>> But +1 for making canvas follow this same behaviour!
>>>
>>
>> ​+1 (with shift = add)
>>
>> It's actually:
>>
>> - shift and mouse move (multiple selection): add
>> - shift and click: toggle
>>
>> Right?
>>
>>
> ​Just wondering what's the current status of the discussion. I tried
> today's master and shift + mouse move (multiple selection) currently
> toggles.
>
> Are you still planning to change this to add?
>
> Also: When I hold down shift, the selection only affect features which are
> completely within the drawn rectangle rather than everything that the
> rectangle touches as is the case without key modifier.
>
> ... quite confusing actually ...
>
>

​Hi,

Where can I find a description of the currently implemented behavior? I've
experimented a bit (with 2.14) but I'm more confused than ever ..

Best wishes,
Anita​
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Last polish for the 2.14 changelog

[Tim Sutton]

Hi

> On 28 Feb 2016, at 08:34, Nyall Dawson  wrote:
> 
> 
> On 28 Feb 2016 4:51 PM, "Tim Sutton"  > wrote:
> >
> > Hi All
> >
> >
> > Over the last weeks we have been working hard on getting the visual 
> > changelog ready for 2.14. A huge thanks to those that have been 
> > contributing to it, most appreciated! Soon it will be published live on 
> > http://qgis.org  and I want to ask if you could all help 
> > with some peer review. There are still a few entries without detailed 
> > descriptions, credits to authors and funders, images etc. I will be making 
> > a last few tweaks to the reporting layout (e.g. to sort sponsors by 
> > sponsorship level). Could I ask you all to take a look here:
> >
> > http://changelog.qgis.org/en/qgis/version/2.14.0/ 
> > 
> >
> >
> > and give me any feedback, improvements etc. Or feel free to edit directly 
> > if you have the needed access. Please don’t tweet or otherwise publicise 
> > the above link until the official release announcements are out (from 
> > Jürgen).
> 
> Thanks for your hard work with this Tim!
> 
> Just wondering, is there any facility at the moment for setting the display 
> order of entries in the log?
> 

Its on our roadmap but sorry we don’t have it in place yet. I will have it done 
for the next release.

Regards

Tim


> Nyall
> 
> >
> > Regards
> >
> > Tim
> > —
> >
> >
> >
> >
> > Tim Sutton
> >
> > Visit http://kartoza.com  to find out about open 
> > source:
> >
> > * Desktop GIS programming services
> > * Geospatial web development
> > * GIS Training
> > * Consulting Services
> >
> > Skype: timlinux Irc: timlinux on #qgis at freenode.net 
> > 
> > Tim is a member of the QGIS Project Steering Committee
> >
> > Kartoza is a merger between Linfiniti and Afrispatial
> >
> >
> > ___
> > Qgis-developer mailing list
> > Qgis-developer@lists.osgeo.org 
> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer 
> > 
> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer 
> > 

—





Tim Sutton

Visit http://kartoza.com  to find out about open source:

* Desktop GIS programming services
* Geospatial web development
* GIS Training
* Consulting Services

Skype: timlinux Irc: timlinux on #qgis at freenode.net
Tim is a member of the QGIS Project Steering Committee

Kartoza is a merger between Linfiniti and Afrispatial



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer