Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

Good point.  Reduced precision should only be used for data which is being
used in a read-only way.  I guess Geoserver could try to enforce this, or
else leave it up to the caller to make sure they are doing the right thing.

On Wed, Oct 18, 2017 at 10:13 AM, Rahkonen Jukka (MML) <
jukka.rahko...@maanmittauslaitos.fi> wrote:

> Hi,
>
>
> Reducing precision could lead to corrupted data with WFS-T. Perhaps server
> should accept the parameter, if it comes true, only when it is
> not configured to support transactions.
>
>
> -Jukka Rahkonen-
> --
> *Lähettäjä:* Martin Davis <mtncl...@gmail.com>
> *Lähetetty:* 18. lokakuuta 2017 19:53
> *Vastaanottaja:* Andrea Aime
> *Kopio:* geoserver-users@lists.sourceforge.net
> *Aihe:* Re: [Geoserver-users] Reduce precision of WFS output to decrease
> response size?
>
> Well, isn't this the Discussion phase of the feature request?
>
> And as I mentioned, although it would be great to see this done,
> unfortunately we may not be able to make a good case to fund it.
>
> On Wed, Oct 18, 2017 at 9:43 AM, Andrea Aime <andrea.a...@geo-solutions.it
> > wrote:
>
>> On Wed, Oct 18, 2017 at 6:09 PM, Martin Davis <mtncl...@gmail.com> wrote:
>>
>>> Here's another reason why reducing precision of responses on-the-fly
>>> would be useful.  It turn out the data as stored in EPGS:3005 essentially
>>> has 1 m precision.  However, we are requesting it reprojected to
>>> EPSG:3857.  The reprojection creates values using full double precision,
>>> which is output by GeoServer.  So in this case it is perfectly safe to
>>> round back to 1 m precision.
>>>
>>> We will explore requesting the data in the native CRS, since we can
>>> project on the client-side.  But it would be nice to be able to do this
>>> fully in GeoServer.
>>>
>>
>> By now you should know that just being right about the usefulness of a
>> feature request achieves exactly nothing ;-)
>> See: https://github.com/geoserver/geoserver/wiki/Successfull
>> y-requesting-and-integrating-new-features-and-improvements-in-GeoServer
>>
>>
>>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Rahkonen Jukka (MML)]

Hi,


Reducing precision could lead to corrupted data with WFS-T. Perhaps server 
should accept the parameter, if it comes true, only when it is not configured 
to support transactions.


-Jukka Rahkonen-


Lähettäjä: Martin Davis <mtncl...@gmail.com>
Lähetetty: 18. lokakuuta 2017 19:53
Vastaanottaja: Andrea Aime
Kopio: geoserver-users@lists.sourceforge.net
Aihe: Re: [Geoserver-users] Reduce precision of WFS output to decrease response 
size?

Well, isn't this the Discussion phase of the feature request?

And as I mentioned, although it would be great to see this done, unfortunately 
we may not be able to make a good case to fund it.

On Wed, Oct 18, 2017 at 9:43 AM, Andrea Aime 
<andrea.a...@geo-solutions.it<mailto:andrea.a...@geo-solutions.it>> wrote:
On Wed, Oct 18, 2017 at 6:09 PM, Martin Davis 
<mtncl...@gmail.com<mailto:mtncl...@gmail.com>> wrote:
Here's another reason why reducing precision of responses on-the-fly would be 
useful.  It turn out the data as stored in EPGS:3005 essentially has 1 m 
precision.  However, we are requesting it reprojected to EPSG:3857.  The 
reprojection creates values using full double precision, which is output by 
GeoServer.  So in this case it is perfectly safe to round back to 1 m precision.

We will explore requesting the data in the native CRS, since we can project on 
the client-side.  But it would be nice to be able to do this fully in GeoServer.

By now you should know that just being right about the usefulness of a feature 
request achieves exactly nothing ;-)
See: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

Well, isn't this the Discussion phase of the feature request?

And as I mentioned, although it would be great to see this done,
unfortunately we may not be able to make a good case to fund it.

On Wed, Oct 18, 2017 at 9:43 AM, Andrea Aime 
wrote:

> On Wed, Oct 18, 2017 at 6:09 PM, Martin Davis  wrote:
>
>> Here's another reason why reducing precision of responses on-the-fly
>> would be useful.  It turn out the data as stored in EPGS:3005 essentially
>> has 1 m precision.  However, we are requesting it reprojected to
>> EPSG:3857.  The reprojection creates values using full double precision,
>> which is output by GeoServer.  So in this case it is perfectly safe to
>> round back to 1 m precision.
>>
>> We will explore requesting the data in the native CRS, since we can
>> project on the client-side.  But it would be nice to be able to do this
>> fully in GeoServer.
>>
>
> By now you should know that just being right about the usefulness of a
> feature request achieves exactly nothing ;-)
> See: https://github.com/geoserver/geoserver/wiki/
> Successfully-requesting-and-integrating-new-features-and-
> improvements-in-GeoServer
>
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Andrea Aime]

On Wed, Oct 18, 2017 at 6:09 PM, Martin Davis  wrote:

> Here's another reason why reducing precision of responses on-the-fly would
> be useful.  It turn out the data as stored in EPGS:3005 essentially has 1 m
> precision.  However, we are requesting it reprojected to EPSG:3857.  The
> reprojection creates values using full double precision, which is output by
> GeoServer.  So in this case it is perfectly safe to round back to 1 m
> precision.
>
> We will explore requesting the data in the native CRS, since we can
> project on the client-side.  But it would be nice to be able to do this
> fully in GeoServer.
>

By now you should know that just being right about the usefulness of a
feature request achieves exactly nothing ;-)
See:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Cheers
Andrea


-- 

Regards,

Andrea Aime

==
GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o
nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
loro utilizzo è consentito esclusivamente al destinatario del messaggio,
per le finalità indicate nel messaggio stesso. Qualora riceviate questo
messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
darcene notizia via e-mail e di procedere alla distruzione del messaggio
stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
utilizzarlo per finalità diverse, costituisce comportamento contrario ai
principi dettati dal D.Lgs. 196/2003.

The information in this message and/or attachments, is intended solely for
the attention and use of the named addressee(s) and may be confidential or
proprietary in nature or covered by the provisions of privacy act
(Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
Code).Any use not in accord with its purpose, any disclosure, reproduction,
copying, distribution, or either dissemination, either whole or partial, is
strictly forbidden except previous formal approval of the named
addressee(s). If you are not the intended recipient, please contact
immediately the sender by telephone, fax or e-mail and delete the
information in this message that has been received in error. The sender
does not give any warranty or accept liability as the content, accuracy or
completeness of sent messages and accepts no responsibility  for changes
made after they were sent or for other risks which arise as a result of
e-mail transmission, viruses, etc.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

Here's another reason why reducing precision of responses on-the-fly would
be useful.  It turn out the data as stored in EPGS:3005 essentially has 1 m
precision.  However, we are requesting it reprojected to EPSG:3857.  The
reprojection creates values using full double precision, which is output by
GeoServer.  So in this case it is perfectly safe to round back to 1 m
precision.

We will explore requesting the data in the native CRS, since we can project
on the client-side.  But it would be nice to be able to do this fully in
GeoServer.

On Tue, Oct 17, 2017 at 3:55 PM, Martin Davis  wrote:

> Is there any way in GeoServer to request that the precision of WFS output
> be reduced?  (I.e. remove decimal places from the ordinates?).
>
> We have a large dataset which has far too much precision on the data.
> Removing unecessary decimal digits seems like it could significantly reduce
> WFS response size.
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

I was thinking that the paging would be exposed to the user - but of course
as you point out it could just be used internally to lower the per-request
response size.  I'm not sure we'd want to flood the server that heavily,
but improving performance via concurrent queries is an interesting option,
for sure.  We'll give this some thought.  We're also exploring reducing the
precision of the data itself, which is the ultimate best solution.

Agreed that using WMS to visualize the data might be better, atlhough the
query filter is fairly large (essentially a giant IN expression), so I
wonder how that would perform in practice.  The real impediment is that the
mapping framework we are using doesn't support this explicitly (it uses
OpenLayers at its core, but heavily encapsulated for purposes of client
code longevity).  But we might explore this approach if we have to.

Thanks for all your helpful suggestions.

Martin

On Tue, Oct 17, 2017 at 5:29 PM, Ben Caradoc-Davies 
wrote:

> Why an awkward user experience? Can you not assemble the features on the
> client side? You could even make the requests in parallel if you do not
> mind the risk of saturating the server.
>
> If you are visualising all the features on a map, WMS might be a better
> solution. Why do you need the geometries? Interaction? Could this be
> implemented with a WMS GetFeatureInfo and then a WFS GetFeature to get only
> the geometries you need?
>
> Kind regards,
> Ben.
>
>
> On 18/10/17 13:20, Martin Davis wrote:
>
>> There are many geometries in a reponse (for example, one response has 2240
>> geometries, and has an uncompressed size of 123 MB!).
>>
>> We're aware of WFS paging, but in this particular use case it would result
>> an awkward user experience.   The use case is that they want to visualize
>> *all* the features on a map, to help decide where to locate a particular
>> biological sampling station. (The query features are biogeoclimatic zones
>> in BC, which is why they are so huuuge).
>>
>> Yes, GZIP is happening on the wire, but we could still realize a 50%
>> savings if the decimals were dropped.
>>
>> On Tue, Oct 17, 2017 at 5:08 PM, Ben Caradoc-Davies 
>> wrote:
>>
>> Martin,
>>>
>>> do you have one giant geometry or many features? If you have multiple
>>> features, you can use WFS 2.0.0 paging to select a subset so that
>>> individual requests do not time out. This example requests one at a time
>>> but you can try larger increments:
>>> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
>>> 0=GetFeature=topp:states=json=1&
>>> startindex=0
>>> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
>>> 0=GetFeature=topp:states=json=1&
>>> startindex=1
>>> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
>>> 0=GetFeature=topp:states=json=1&
>>> startindex=2
>>> [...]
>>>
>>> Note that paging enables sorting to make requests stable; this imposes a
>>> small server overhead.
>>>
>>> GeoServer also supports paging for WFS 1.1.0 if you need it (extension,
>>> not part of the spec); just use maxfeatures instead of count and you
>>> should
>>> be good to go.
>>>
>>> GZIP compression should already be used on the wire.
>>>
>>> Kind regards,
>>> Ben.
>>>
>>>
>>> On 18/10/17 12:41, Martin Davis wrote:
>>>
>>> Thanks, Ben.  Unfortunately that setting won't work for us, because (a)
 we
 don't have access to the server to set it, (b) we want to use GeoJSON,
 and
 (c) we wouldn't want this setting to impact other layers (and in fact we
 would like it to apply only to specific requests).

 And yes, am aware we can request responses with no geometry, but in this
 case we need to display the geometry.  That's actually the key problem -
 we
 are requesting a large amount of geometry for display, and the requests
 are
 very slow or failing completely because the response size is so large.

 On Tue, Oct 17, 2017 at 4:29 PM, Ben Caradoc-Davies 
 wrote:

 Martin,

>
> try Settings / Global / Number of decimals. This works for GML 2 and
> GML
> 3.1 but does not seem to work for GML 3.2 output.
>
> Do you need the geometries? If not, you can use the propertyname
> parameter
> to request only the properties you need for even smaller responses:
> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
> 0=GetFeature=topp:states=topp:PERSONS,
> topp:WORKERS
>
> Kind regards,
> Ben.
>
>
> On 18/10/17 11:55, Martin Davis wrote:
>
> Is there any way in GeoServer to request that the precision of WFS
> output
>
>> be reduced?  (I.e. remove decimal places from the ordinates?).
>>
>> We have a large dataset which has far too much precision on the data.
>> Removing unecessary decimal digits seems like it could significantly
>> reduce
>> WFS response size.
>>
>>
>>
>> 

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Ben Caradoc-Davies]

Why an awkward user experience? Can you not assemble the features on the 
client side? You could even make the requests in parallel if you do not 
mind the risk of saturating the server.


If you are visualising all the features on a map, WMS might be a better 
solution. Why do you need the geometries? Interaction? Could this be 
implemented with a WMS GetFeatureInfo and then a WFS GetFeature to get 
only the geometries you need?


Kind regards,
Ben.

On 18/10/17 13:20, Martin Davis wrote:

There are many geometries in a reponse (for example, one response has 2240
geometries, and has an uncompressed size of 123 MB!).

We're aware of WFS paging, but in this particular use case it would result
an awkward user experience.   The use case is that they want to visualize
*all* the features on a map, to help decide where to locate a particular
biological sampling station. (The query features are biogeoclimatic zones
in BC, which is why they are so huuuge).

Yes, GZIP is happening on the wire, but we could still realize a 50%
savings if the decimals were dropped.

On Tue, Oct 17, 2017 at 5:08 PM, Ben Caradoc-Davies 
wrote:


Martin,

do you have one giant geometry or many features? If you have multiple
features, you can use WFS 2.0.0 paging to select a subset so that
individual requests do not time out. This example requests one at a time
but you can try larger increments:
http://localhost:8080/geoserver/wfs?service=WFS=2.0.
0=GetFeature=topp:states=json=1&
startindex=0
http://localhost:8080/geoserver/wfs?service=WFS=2.0.
0=GetFeature=topp:states=json=1&
startindex=1
http://localhost:8080/geoserver/wfs?service=WFS=2.0.
0=GetFeature=topp:states=json=1&
startindex=2
[...]

Note that paging enables sorting to make requests stable; this imposes a
small server overhead.

GeoServer also supports paging for WFS 1.1.0 if you need it (extension,
not part of the spec); just use maxfeatures instead of count and you should
be good to go.

GZIP compression should already be used on the wire.

Kind regards,
Ben.


On 18/10/17 12:41, Martin Davis wrote:


Thanks, Ben.  Unfortunately that setting won't work for us, because (a) we
don't have access to the server to set it, (b) we want to use GeoJSON, and
(c) we wouldn't want this setting to impact other layers (and in fact we
would like it to apply only to specific requests).

And yes, am aware we can request responses with no geometry, but in this
case we need to display the geometry.  That's actually the key problem -
we
are requesting a large amount of geometry for display, and the requests
are
very slow or failing completely because the response size is so large.

On Tue, Oct 17, 2017 at 4:29 PM, Ben Caradoc-Davies 
wrote:

Martin,


try Settings / Global / Number of decimals. This works for GML 2 and GML
3.1 but does not seem to work for GML 3.2 output.

Do you need the geometries? If not, you can use the propertyname
parameter
to request only the properties you need for even smaller responses:
http://localhost:8080/geoserver/wfs?service=WFS=2.0.
0=GetFeature=topp:states=topp:PERSONS,
topp:WORKERS

Kind regards,
Ben.


On 18/10/17 11:55, Martin Davis wrote:

Is there any way in GeoServer to request that the precision of WFS output

be reduced?  (I.e. remove decimal places from the ordinates?).

We have a large dataset which has far too much precision on the data.
Removing unecessary decimal digits seems like it could significantly
reduce
WFS response size.




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot



___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to
this list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users


--

Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand





--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand





--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The 

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

There are many geometries in a reponse (for example, one response has 2240
geometries, and has an uncompressed size of 123 MB!).

We're aware of WFS paging, but in this particular use case it would result
an awkward user experience.   The use case is that they want to visualize
*all* the features on a map, to help decide where to locate a particular
biological sampling station. (The query features are biogeoclimatic zones
in BC, which is why they are so huuuge).

Yes, GZIP is happening on the wire, but we could still realize a 50%
savings if the decimals were dropped.

On Tue, Oct 17, 2017 at 5:08 PM, Ben Caradoc-Davies 
wrote:

> Martin,
>
> do you have one giant geometry or many features? If you have multiple
> features, you can use WFS 2.0.0 paging to select a subset so that
> individual requests do not time out. This example requests one at a time
> but you can try larger increments:
> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
> 0=GetFeature=topp:states=json=1&
> startindex=0
> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
> 0=GetFeature=topp:states=json=1&
> startindex=1
> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
> 0=GetFeature=topp:states=json=1&
> startindex=2
> [...]
>
> Note that paging enables sorting to make requests stable; this imposes a
> small server overhead.
>
> GeoServer also supports paging for WFS 1.1.0 if you need it (extension,
> not part of the spec); just use maxfeatures instead of count and you should
> be good to go.
>
> GZIP compression should already be used on the wire.
>
> Kind regards,
> Ben.
>
>
> On 18/10/17 12:41, Martin Davis wrote:
>
>> Thanks, Ben.  Unfortunately that setting won't work for us, because (a) we
>> don't have access to the server to set it, (b) we want to use GeoJSON, and
>> (c) we wouldn't want this setting to impact other layers (and in fact we
>> would like it to apply only to specific requests).
>>
>> And yes, am aware we can request responses with no geometry, but in this
>> case we need to display the geometry.  That's actually the key problem -
>> we
>> are requesting a large amount of geometry for display, and the requests
>> are
>> very slow or failing completely because the response size is so large.
>>
>> On Tue, Oct 17, 2017 at 4:29 PM, Ben Caradoc-Davies 
>> wrote:
>>
>> Martin,
>>>
>>> try Settings / Global / Number of decimals. This works for GML 2 and GML
>>> 3.1 but does not seem to work for GML 3.2 output.
>>>
>>> Do you need the geometries? If not, you can use the propertyname
>>> parameter
>>> to request only the properties you need for even smaller responses:
>>> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
>>> 0=GetFeature=topp:states=topp:PERSONS,
>>> topp:WORKERS
>>>
>>> Kind regards,
>>> Ben.
>>>
>>>
>>> On 18/10/17 11:55, Martin Davis wrote:
>>>
>>> Is there any way in GeoServer to request that the precision of WFS output
 be reduced?  (I.e. remove decimal places from the ordinates?).

 We have a large dataset which has far too much precision on the data.
 Removing unecessary decimal digits seems like it could significantly
 reduce
 WFS response size.



 
 --
 Check out the vibrant tech community on one of the world's most
 engaging tech sites, Slashdot.org! http://sdm.link/slashdot



 ___
 Geoserver-users mailing list

 Please make sure you read the following two resources before posting to
 this list:
 - Earning your support instead of buying it, but Ian Turton:
 http://www.ianturton.com/talks/foss4g.html#/
 - The GeoServer user list posting guidelines:
 http://geoserver.org/comm/userlist-guidelines.html

 Geoserver-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/geoserver-users


 --
>>> Ben Caradoc-Davies 
>>> Director
>>> Transient Software Limited 
>>> New Zealand
>>>
>>>
>>
> --
> Ben Caradoc-Davies 
> Director
> Transient Software Limited 
> New Zealand
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Ben Caradoc-Davies]

Yes, that would be nice.

On 18/10/17 12:17, Martin Davis wrote:

So it would be nice to see this added as a WFS vendor parameter
"maxDecimals" [1]
[1]http://docs.geoserver.org/stable/en/user/services/wfs/vendor.html


--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Ben Caradoc-Davies]

Martin,

do you have one giant geometry or many features? If you have multiple 
features, you can use WFS 2.0.0 paging to select a subset so that 
individual requests do not time out. This example requests one at a time 
but you can try larger increments:

http://localhost:8080/geoserver/wfs?service=WFS=2.0.0=GetFeature=topp:states=json=1=0
http://localhost:8080/geoserver/wfs?service=WFS=2.0.0=GetFeature=topp:states=json=1=1
http://localhost:8080/geoserver/wfs?service=WFS=2.0.0=GetFeature=topp:states=json=1=2
[...]

Note that paging enables sorting to make requests stable; this imposes a 
small server overhead.


GeoServer also supports paging for WFS 1.1.0 if you need it (extension, 
not part of the spec); just use maxfeatures instead of count and you 
should be good to go.


GZIP compression should already be used on the wire.

Kind regards,
Ben.

On 18/10/17 12:41, Martin Davis wrote:

Thanks, Ben.  Unfortunately that setting won't work for us, because (a) we
don't have access to the server to set it, (b) we want to use GeoJSON, and
(c) we wouldn't want this setting to impact other layers (and in fact we
would like it to apply only to specific requests).

And yes, am aware we can request responses with no geometry, but in this
case we need to display the geometry.  That's actually the key problem - we
are requesting a large amount of geometry for display, and the requests are
very slow or failing completely because the response size is so large.

On Tue, Oct 17, 2017 at 4:29 PM, Ben Caradoc-Davies 
wrote:


Martin,

try Settings / Global / Number of decimals. This works for GML 2 and GML
3.1 but does not seem to work for GML 3.2 output.

Do you need the geometries? If not, you can use the propertyname parameter
to request only the properties you need for even smaller responses:
http://localhost:8080/geoserver/wfs?service=WFS=2.0.
0=GetFeature=topp:states=topp:PERSONS,
topp:WORKERS

Kind regards,
Ben.


On 18/10/17 11:55, Martin Davis wrote:


Is there any way in GeoServer to request that the precision of WFS output
be reduced?  (I.e. remove decimal places from the ordinates?).

We have a large dataset which has far too much precision on the data.
Removing unecessary decimal digits seems like it could significantly
reduce
WFS response size.




--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot



___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to
this list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users



--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand





--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

Thanks, Ben.  Unfortunately that setting won't work for us, because (a) we
don't have access to the server to set it, (b) we want to use GeoJSON, and
(c) we wouldn't want this setting to impact other layers (and in fact we
would like it to apply only to specific requests).

And yes, am aware we can request responses with no geometry, but in this
case we need to display the geometry.  That's actually the key problem - we
are requesting a large amount of geometry for display, and the requests are
very slow or failing completely because the response size is so large.

On Tue, Oct 17, 2017 at 4:29 PM, Ben Caradoc-Davies 
wrote:

> Martin,
>
> try Settings / Global / Number of decimals. This works for GML 2 and GML
> 3.1 but does not seem to work for GML 3.2 output.
>
> Do you need the geometries? If not, you can use the propertyname parameter
> to request only the properties you need for even smaller responses:
> http://localhost:8080/geoserver/wfs?service=WFS=2.0.
> 0=GetFeature=topp:states=topp:PERSONS,
> topp:WORKERS
>
> Kind regards,
> Ben.
>
>
> On 18/10/17 11:55, Martin Davis wrote:
>
>> Is there any way in GeoServer to request that the precision of WFS output
>> be reduced?  (I.e. remove decimal places from the ordinates?).
>>
>> We have a large dataset which has far too much precision on the data.
>> Removing unecessary decimal digits seems like it could significantly
>> reduce
>> WFS response size.
>>
>>
>>
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> ___
>> Geoserver-users mailing list
>>
>> Please make sure you read the following two resources before posting to
>> this list:
>> - Earning your support instead of buying it, but Ian Turton:
>> http://www.ianturton.com/talks/foss4g.html#/
>> - The GeoServer user list posting guidelines:
>> http://geoserver.org/comm/userlist-guidelines.html
>>
>> Geoserver-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>>
> --
> Ben Caradoc-Davies 
> Director
> Transient Software Limited 
> New Zealand
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Ben Caradoc-Davies]

Martin,

try Settings / Global / Number of decimals. This works for GML 2 and GML 
3.1 but does not seem to work for GML 3.2 output.


Do you need the geometries? If not, you can use the propertyname 
parameter to request only the properties you need for even smaller 
responses:

http://localhost:8080/geoserver/wfs?service=WFS=2.0.0=GetFeature=topp:states=topp:PERSONS,topp:WORKERS

Kind regards,
Ben.

On 18/10/17 11:55, Martin Davis wrote:

Is there any way in GeoServer to request that the precision of WFS output
be reduced?  (I.e. remove decimal places from the ordinates?).

We have a large dataset which has far too much precision on the data.
Removing unecessary decimal digits seems like it could significantly reduce
WFS response size.



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot



___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users



--
Ben Caradoc-Davies 
Director
Transient Software Limited 
New Zealand

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

I guess this is my answer:

https://gis.stackexchange.com/questions/228454/reduce-wfs-response-size-by-reducing-feature-detail

Too bad.  WPS is so much harder to use than WFS (at least my impression -
happy to shown otherwise).  The WFS Global setting is too broad to use in
this case.  The per-layer one would be fine, but apparently does not work
for GeoJSON (which kind of defeats the purpose, since GML is much fluffier
to begin with).  And in this case we don't have access to the server.

So it would be nice to see this added as a WFS vendor parameter
"maxDecimals" [1]

[1] http://docs.geoserver.org/stable/en/user/services/wfs/vendor.html

On Tue, Oct 17, 2017 at 3:55 PM, Martin Davis  wrote:

> Is there any way in GeoServer to request that the precision of WFS output
> be reduced?  (I.e. remove decimal places from the ordinates?).
>
> We have a large dataset which has far too much precision on the data.
> Removing unecessary decimal digits seems like it could significantly reduce
> WFS response size.
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

[Geoserver-users] Reduce precision of WFS output to decrease response size?

[Martin Davis]

Is there any way in GeoServer to request that the precision of WFS output
be reduced?  (I.e. remove decimal places from the ordinates?).

We have a large dataset which has far too much precision on the data.
Removing unecessary decimal digits seems like it could significantly reduce
WFS response size.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users