Re: [Geoserver-users] Performance issue with large polygon
Thanks for the comments, a few things for me to investigate, I'll try using a shape file and also add some metrics to the clipping (I'm already running a slightly modified version of geoserver). Also glad to hear it's common to chop up large polygons like I was thinking. Kind Regards, Vic From: andrea.a...@gmail.com [andrea.a...@gmail.com] on behalf of Andrea Aime [andrea.a...@geo-solutions.it] Sent: 09 August 2015 20:25 To: Martin Davis Cc: Kirk, Victor; geoserver-users@lists.sourceforge.net Subject: Re: [Geoserver-users] Performance issue with large polygon On Sun, Aug 9, 2015 at 5:19 PM, Martin Davis mailto:mtncl...@gmail.com>> wrote: AFAIK there is no way to cache a layer in memory. This would be a useful feature to add to GeoServer. Perhaps you could cache the geometry as a shapefile - which should provide fastest performance off disk? It would be interesting to see what the performance boost is at any rate. I also wonder if clipping small parts out of a very large geometry is causing a performance hit. We have a fast rectangular clipping algorithm geared towards rendering that is significantly faster than JTS intersection (as we don't require the result to be topologically valid, which greatly simplifies things). It could still be that it's clipping, mind, but there are other possible issues in play, besides the obvious data transfer cost, which is going to be high regardless, if the DB is postgis, loading from tables with large polygons might result in the spatial index not being used at all: http://postgis.net/docs/performance_tips.html#small_tables_large_objects Other spatial databases might have their own specific issues against geometries with large number of vertices. Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it AVVERTENZE AI SENSI DEL D.Lgs. 196/2003 Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Does Geoserver support the ellipsoidal Equidistant Cylindrical projection?
Jonathan Moules hrwallingford.com> writes: > > Hi Robin, > As you don't seem to have any responses, I'd suggest reporting it as a bug with all the details you have. > Cheers, > Jonathan Thanks Jonathan. I've logged this as GEOS-7149. Regards, Robin > > -Original Message- > From: robkhe [mailto:robinhoutmeyers gmail.com] > Sent: Thursday, July 30, 2015 4:38 PM > To: geoserver-users lists.sourceforge.net > Subject: [Geoserver-users] Does Geoserver support the ellipsoidal Equidistant Cylindrical projection? > > Hi, > > A clean Geoserver installation advertises EPSG:4087 in its OGC WMS capabilities, which corresponds to an > equidistant cylindrical projection with ellipsoidal calculations. > > It seems however that Geoserver treats this reference the same as EPSG:32662, which is the equidistant > cylindrical projection with spherical calculations. > > This can be verified using the Transformation console available in the Demos section on the Geoserver > portal page. > > For instance, the point (10, 5) in EPSG:4326 results in the point > (1113194.9079327357 556597.4539663679) when transformed to EPSG:4087 - which is the same as for > EPSG:32662 - while I would expect (1113194.9079327357, > 552885.4510584074) for EPSG:4087. > > Could this be a bug or am I overlooking something? > > Kind regards, > Robin > > -- > View this message in context: http://osgeo- org.1560.x6.nabble.com/Does-Geoserver-support-the-ellipsoidal- Equidistant-Cylindrical-projection-tp5217819.html > Sent from the GeoServer - User mailing list archive at Nabble.com. > > -- > ___ > Geoserver-users mailing list > Geoserver-users lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > This message has been scanned for viruses by MailControl - www.mailcontrol.com > > Click > https://www.mailcontrol.com/sr/A3Fy2cJmdgTGX2PQPOmvUrikelaf+VTnw+AX4tuo8 GGVpp4hFmdwvpm1t5MUD6ihSOpEbUBhyl3sVggzUgwWUw== > to report this email as spam. > > > > HR Wallingford and its subsidiaries uses faxes and emails for confidential and legally privileged > business communications. They do not of themselves create legal commitments. Disclosure to parties > other than addressees requires our specific consent. We are not liable for unauthorised disclosures nor > reliance upon them. > If you have received this message in error please advise us immediately and destroy all copies of it. > > HR Wallingford Limited > Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom > Registered in England No. 02562099 > > > > -- > -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Code not closing CloseableIterator
On Sun, Aug 9, 2015 at 9:09 PM, sikeoka wrote:
> Even if all code that is calling the list() function always closes the
> returned CloseableIterator, the final CloseableIteratorAdapter instance
> that
> is created in SecureCatalogImpl.list() does not contain the reference to
> close the filteredWrapped CloseableIterator so filteredWrapped is left
> unclosed when the garbage collector gets to it. The code change in my
> previous messages fixes that so closing the CloseableIterator that is
> returned by SecureCatalogImpl.list() will close the filteredWrapped
> instance.
>
Ah... I was focusing on that extra in the row, and did not notice
the real change was switching from constructor to factory method
>
> This does not generate the warning message in most cases because
> CloseableIteratorAdapter.hasNext() will call close() on itself before
> returning false but calling hasNext() until it returns false is not a
> requirement. I was able to find an instance in the WMS 1.3.0
> GetCapabilities code where hasNext() may not be called until it returns
> false although this is likely to happen anywhere where an exception is
> thrown while processing the iterator.
>
> This is a portion of the code from Capabilities_1_3_0_Transformer:
> private void handleRootBbox(Iterator layers) {
> ...
> while(layers.hasNext()) {
> ...
> if(latlonBbox.contains(world)){
> break;
> }
> }
> ...
> }
>
> I did not see a JIRA ticket about this.
Can someone please do it? Nothing gets done without a ticket to remind
us, because it's almost impossible that we have time to work it right away,
and when we find some time a week or a month later, only what's on Jira
gets considered
Cheers
Andrea
--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
*AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.
The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
---
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Performance issue with large polygon
On Sun, Aug 9, 2015 at 5:19 PM, Martin Davis wrote: > AFAIK there is no way to cache a layer in memory. This would be a useful > feature to add to GeoServer. > > Perhaps you could cache the geometry as a shapefile - which should provide > fastest performance off disk? It would be interesting to see what the > performance boost is at any rate. I also wonder if clipping small parts > out of a very large geometry is causing a performance hit. > We have a fast rectangular clipping algorithm geared towards rendering that is significantly faster than JTS intersection (as we don't require the result to be topologically valid, which greatly simplifies things). It could still be that it's clipping, mind, but there are other possible issues in play, besides the obvious data transfer cost, which is going to be high regardless, if the DB is postgis, loading from tables with large polygons might result in the spatial index not being used at all: http://postgis.net/docs/performance_tips.html#small_tables_large_objects Other spatial databases might have their own specific issues against geometries with large number of vertices. Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Code not closing CloseableIterator
Even if all code that is calling the list() function always closes the
returned CloseableIterator, the final CloseableIteratorAdapter instance that
is created in SecureCatalogImpl.list() does not contain the reference to
close the filteredWrapped CloseableIterator so filteredWrapped is left
unclosed when the garbage collector gets to it. The code change in my
previous messages fixes that so closing the CloseableIterator that is
returned by SecureCatalogImpl.list() will close the filteredWrapped
instance.
This does not generate the warning message in most cases because
CloseableIteratorAdapter.hasNext() will call close() on itself before
returning false but calling hasNext() until it returns false is not a
requirement. I was able to find an instance in the WMS 1.3.0
GetCapabilities code where hasNext() may not be called until it returns
false although this is likely to happen anywhere where an exception is
thrown while processing the iterator.
This is a portion of the code from Capabilities_1_3_0_Transformer:
private void handleRootBbox(Iterator layers) {
...
while(layers.hasNext()) {
...
if(latlonBbox.contains(world)){
break;
}
}
...
}
I did not see a JIRA ticket about this. I can reproduce this with a WMS
1.3.0 GetCapabilities request although the warning message won't appear
until the garbage collector runs which may not always happen with a single
request.
--
View this message in context:
http://osgeo-org.1560.x6.nabble.com/Code-not-closing-CloseableIterator-tp5073066p5219096.html
Sent from the GeoServer - User mailing list archive at Nabble.com.
--
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Performance issue with large polygon
AFAIK there is no way to cache a layer in memory. This would be a useful feature to add to GeoServer. Perhaps you could cache the geometry as a shapefile - which should provide fastest performance off disk? It would be interesting to see what the performance boost is at any rate. I also wonder if clipping small parts out of a very large geometry is causing a performance hit. On Fri, Aug 7, 2015 at 1:28 AM, Kirk, Victor wrote: > Hi, > > We have a polygon layer used as a backdrop for a WMS based layer group. > The layer contains the outlines of the UK, one of which is the mainland and > is obviously quite large. I have generalised versions of the geometries to > use when zoomed out, however when zoomed in, encoding the whole mainland > for every tile introduces a bottleneck and requests quickly build up and > time out as every request. The layer group contains around 20 different > layers, without the polygon layer the group renders very quickly add the > layer back in and it becomes very unsresposive. > > The layer is used to provide the coast and a background colour for land. > I'm looking at "chopping" up the polygons on a grid to create smaller > geometries that can be used to provide the background colour and extract > the coast into a number of smaller lines. > > But before I do this I was wondering if there was some way I could get > geoserver to cache the features for this layer in memory rather than having > to hit the database for every request. > > Regards, Vic > -- > > > > > > This document is intended for, and should only be read by, those persons > to whom it is addressed. Its contents are confidential and if you have > received this message in error, please delete it. Any form of reproduction, > dissemination, copying, disclosure, modification, distribution and / or > publication of this message without our prior written consent is strictly > prohibited. > > Any views expressed in this message are those of the individual sender, > and do not necessarily represent the position of Cubic Transportation > Systems (ITMS) Limited (‘CTS’). Furthermore CTS does not authorise or use > e-mail for official contractual correspondence. Nothing received in e-mail > has any contractual validity. > > CTSL and each legal entity in Cubic Corporation reserve the right to > monitor all e-mail communications through its networks. > > Registered Office: > Cubic Transportation Systems Ltd > AFC House > Honeycrock Lane > Salfords > Surrey > RH1 5LA > United Kingdom > > Registered in England under number 8498086 > > > > -- > ___ > Geoserver-users mailing list > Geoserver-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-users > -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Code not closing CloseableIterator
On Thu, Aug 6, 2015 at 10:02 PM, sikeoka wrote: > This warning might be coming from around line 1415 in the method: > org.geoserver.security.SecureCatalogImpl.list(Class, Filter, Integer, > Integer, SortBy) > > The following lines: > return new > CloseableIteratorAdapter(Iterators.filter(filteredWrapped, > com.google.common.base.Predicates.notNull())); > > should be changed to: > return CloseableIteratorAdapter.filter(filteredWrapped, > com.google.common.base.Predicates. notNull()); > I don't see how this can possibly make the iterator get closed, the close operation requires a explicting "close" call in the code using the iterator. Was a ticket ever opened, with steps to reproduce? Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Performance issue with large polygon
On Fri, Aug 7, 2015 at 10:28 AM, Kirk, Victor wrote: > Hi, > > We have a polygon layer used as a backdrop for a WMS based layer group. > The layer contains the outlines of the UK, one of which is the mainland and > is obviously quite large. I have generalised versions of the geometries to > use when zoomed out, however when zoomed in, encoding the whole mainland > for every tile introduces a bottleneck and requests quickly build up and > time out as every request. The layer group contains around 20 different > layers, without the polygon layer the group renders very quickly add the > layer back in and it becomes very unsresposive. > > The layer is used to provide the coast and a background colour for land. > I'm looking at "chopping" up the polygons on a grid to create smaller > geometries that can be used to provide the background colour and extract > the coast into a number of smaller lines. > Yes, this is the standard trick to handle large polygons. > > But before I do this I was wondering if there was some way I could get > geoserver to cache the features for this layer in memory rather than having > to hit the database for every request. > No, there is no facility to keep features in memory, we do no data caching whatsoever. Full layer caching would be relatively easy to implement though, partial/usage sensitive one would be harder (only keep in memory the hot areas), both would require either dedicated resources to work it, or funding to one of the commercial support providers towards implementing that specific feature. Cheers Andrea -- == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003* Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003. The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc. --- -- ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
