Re: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

[Andrea Aime]

Hi,
Peter's example works sometimes, other times it does not, depends on
whether the layer bounding box is
aligned with the cached grid bounds, or not (I don't remember what is done
for default scale denominator, but that one
also has to match a cached zoom level).

If you want to make sure that tiles are getting cached, go to "tiled
layers" and pick a OpenLayers preview
there, those are build around the right bounds and set of zoom levels.

WMS per se is not meant to be cacheable and often it is not. The "direct
integration" between WMS and caching
subsystem is there because often people set a basemap like OSM in their web
clients, and then overlay a WMS
that gets also similarly tiled as a result.
This make those WMS request match exactly the web mercator gridset. In
other words, direct integration is leveraging
a common "accident" to make the server perform less work.

If you want to make sure you're using tiles instead, use a tile oriented
protocol, like WMTS.

Cheers
Andrea

On Sat, Sep 1, 2018 at 3:43 PM Peter Smythe  wrote:

> Hi Jan
>
> > Should it be possible to cache tiles using an application (Open Layers)
> and not using direct WMS requests with exact BBox extents for each tile?
> How can that be achieved? (how to use the mandatory &tiled=true
> 
>  parameter).
>
>
> Yes, it is.  On the Layer Preview page, click on the 3 dots (top left) to
> expand the menu.  Then change Tiling: to Tiled.  You can see that the map
> now loads in tiles, not all at once.  Verify &tiled=true in Developer Tools.
>
> [image: image.png]
>
> Regards
>
> Peter
>
> On Fri, 31 Aug 2018 at 10:46, Jan Donner  wrote:
>
>> Thank you Jukka and Andrea, this at least partly solved my problem!
>> Sorry about that large log. Did not wan't to loose any possibly important
>> information.
>>
>> I disabled the In memory BlobStore and used the first of the two URL:s
>> Andrea provided.
>> - Now it did cache that 0/0.png tile onto S3! (The second URL did not
>> work).
>> - Seeding also worked!
>>
>> Using the Layer preview did not cache tiles on S3! Should it be possible
>> to cache tiles using an application (Open Layers) and not using direct WMS
>> requests with exact BBox extents for each tile? How can that be achieved?
>> (how to use the mandatory &tiled=true
>> 
>>  parameter).
>>
>>
>> br.
>> Jan
>>
>>
>>
>>
>>
>> to 30. elok. 2018 klo 22.35 Andrea Aime (andrea.a...@geo-solutions.it)
>> kirjoitti:
>>
>>> On Thu, Aug 30, 2018 at 5:13 PM Jan Donner 
>>> wrote:
>>>
 Hi!
 I'm experiencing problems with using Geoserver and GeoWebCache with a
 S3 blob store.
 I have managed to create a blob store on S3 (the new folder appeared in
 s3 as defined by S3 Object Key Prefix) using Geoserver
 I try to cache tiles onto it by making a WMS request from the example
 layer topp:states, but no files are cached into the S3 blob store:


 http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,180,90&width=256&height=256&format=image/png8&tiled=true


>>> This is just a partial answer, but I'm at a conference and there is not
>>> much time to dedicate to the user list.
>>> The request you're using does not conform to any cacheable gridset, so
>>> it's not cached. The WGS84 gridset does not have one tile
>>> at the top level, but two. Could you try these instead?:
>>>
>>>
>>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,0,90&width=256&height=256&format=image/png8&tiled=true
>>>
>>>
>>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=0,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>>>
>>> Also, did you enable the "transparent integration" in the "caching
>>> defaults" page? Otherwise just making WMS requests
>>> won't create any tiles. Maybe try to use the tiled layers preview too.
>>>
>>> Cheers
>>> Andrea
>>>
>>>
>>> == GeoServer Professional Services from the experts! Visit
>>> http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf
>>> Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa
>>> (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
>>> http://www.geo-solutions.it http://twitter.com/geosolutions_it
>>> --- *Con
>>> riferimento alla normativa sul trattamento dei dati personali (Reg. UE
>>> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
>>> precisa che

Re: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

[Peter Smythe]

Hi Jan

> Should it be possible to cache tiles using an application (Open Layers)
and not using direct WMS requests with exact BBox extents for each tile?
How can that be achieved? (how to use the mandatory &tiled=true

parameter).


Yes, it is.  On the Layer Preview page, click on the 3 dots (top left) to
expand the menu.  Then change Tiling: to Tiled.  You can see that the map
now loads in tiles, not all at once.  Verify &tiled=true in Developer Tools.

[image: image.png]

Regards

Peter

On Fri, 31 Aug 2018 at 10:46, Jan Donner  wrote:

> Thank you Jukka and Andrea, this at least partly solved my problem!
> Sorry about that large log. Did not wan't to loose any possibly important
> information.
>
> I disabled the In memory BlobStore and used the first of the two URL:s
> Andrea provided.
> - Now it did cache that 0/0.png tile onto S3! (The second URL did not
> work).
> - Seeding also worked!
>
> Using the Layer preview did not cache tiles on S3! Should it be possible
> to cache tiles using an application (Open Layers) and not using direct WMS
> requests with exact BBox extents for each tile? How can that be achieved?
> (how to use the mandatory &tiled=true
> 
>  parameter).
>
>
> br.
> Jan
>
>
>
>
>
> to 30. elok. 2018 klo 22.35 Andrea Aime (andrea.a...@geo-solutions.it)
> kirjoitti:
>
>> On Thu, Aug 30, 2018 at 5:13 PM Jan Donner 
>> wrote:
>>
>>> Hi!
>>> I'm experiencing problems with using Geoserver and GeoWebCache with a S3
>>> blob store.
>>> I have managed to create a blob store on S3 (the new folder appeared in
>>> s3 as defined by S3 Object Key Prefix) using Geoserver
>>> I try to cache tiles onto it by making a WMS request from the example
>>> layer topp:states, but no files are cached into the S3 blob store:
>>>
>>>
>>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>>>
>>>
>> This is just a partial answer, but I'm at a conference and there is not
>> much time to dedicate to the user list.
>> The request you're using does not conform to any cacheable gridset, so
>> it's not cached. The WGS84 gridset does not have one tile
>> at the top level, but two. Could you try these instead?:
>>
>>
>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,0,90&width=256&height=256&format=image/png8&tiled=true
>>
>>
>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=0,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>>
>> Also, did you enable the "transparent integration" in the "caching
>> defaults" page? Otherwise just making WMS requests
>> won't create any tiles. Maybe try to use the tiled layers preview too.
>>
>> Cheers
>> Andrea
>>
>>
>> == GeoServer Professional Services from the experts! Visit
>> http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf
>> Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa
>> (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
>> http://www.geo-solutions.it http://twitter.com/geosolutions_it
>> --- *Con riferimento
>> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
>> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
>> circostanza inerente alla presente email (il suo contenuto, gli eventuali
>> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
>> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
>> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
>> sarei comunque grato se potesse darmene notizia. This email is intended
>> only for the person or entity to which it is addressed and may contain
>> information that is privileged, confidential or otherwise protected from
>> disclosure. We remind that - as provided by European Regulation 2016/679
>> “GDPR” - copying, dissemination or use of this e-mail or the information
>> herein by anyone other than the intended recipient is prohibited. If you
>> have received this email by mistake, please notify us immediately by
>> telephone or e-mail.*
>>
>
>
> --
>
> Jan Donner
> Senior GIS Architect,  *Trimble Forestry*
> Revontulentie 8 C |
> 02100
> Espoo, Finland
> +358 40 538  <+358+40+774+3592>5483 Mobile
> jan.don...@trimble.com
>
> *Deliver

Re: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

[Jan Donner]

Thank you Jukka and Andrea, this at least partly solved my problem!
Sorry about that large log. Did not wan't to loose any possibly important
information.

I disabled the In memory BlobStore and used the first of the two URL:s
Andrea provided.
- Now it did cache that 0/0.png tile onto S3! (The second URL did not work).
- Seeding also worked!

Using the Layer preview did not cache tiles on S3! Should it be possible to
cache tiles using an application (Open Layers) and not using direct WMS
requests with exact BBox extents for each tile? How can that be achieved?
(how to use the mandatory &tiled=true

 parameter).


br.
Jan





to 30. elok. 2018 klo 22.35 Andrea Aime (andrea.a...@geo-solutions.it)
kirjoitti:

> On Thu, Aug 30, 2018 at 5:13 PM Jan Donner  wrote:
>
>> Hi!
>> I'm experiencing problems with using Geoserver and GeoWebCache with a S3
>> blob store.
>> I have managed to create a blob store on S3 (the new folder appeared in
>> s3 as defined by S3 Object Key Prefix) using Geoserver
>> I try to cache tiles onto it by making a WMS request from the example
>> layer topp:states, but no files are cached into the S3 blob store:
>>
>>
>> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>>
>>
> This is just a partial answer, but I'm at a conference and there is not
> much time to dedicate to the user list.
> The request you're using does not conform to any cacheable gridset, so
> it's not cached. The WGS84 gridset does not have one tile
> at the top level, but two. Could you try these instead?:
>
>
> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,0,90&width=256&height=256&format=image/png8&tiled=true
>
>
> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=0,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>
> Also, did you enable the "transparent integration" in the "caching
> defaults" page? Otherwise just making WMS requests
> won't create any tiles. Maybe try to use the tiled layers preview too.
>
> Cheers
> Andrea
>
>
> == GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf
> Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa
> (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
> http://www.geo-solutions.it http://twitter.com/geosolutions_it
> --- *Con riferimento
> alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
> Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
> circostanza inerente alla presente email (il suo contenuto, gli eventuali
> allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
> destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
> errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
> sarei comunque grato se potesse darmene notizia. This email is intended
> only for the person or entity to which it is addressed and may contain
> information that is privileged, confidential or otherwise protected from
> disclosure. We remind that - as provided by European Regulation 2016/679
> “GDPR” - copying, dissemination or use of this e-mail or the information
> herein by anyone other than the intended recipient is prohibited. If you
> have received this email by mistake, please notify us immediately by
> telephone or e-mail.*
>


-- 

Jan Donner
Senior GIS Architect,  *Trimble Forestry*
Revontulentie 8 C |
02100
Espoo, Finland
+358 40 538  <+358+40+774+3592>5483 Mobile
jan.don...@trimble.com

*Delivering the Connected Forest*
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.n

Re: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

[Andrea Aime]

On Thu, Aug 30, 2018 at 5:13 PM Jan Donner  wrote:

> Hi!
> I'm experiencing problems with using Geoserver and GeoWebCache with a S3
> blob store.
> I have managed to create a blob store on S3 (the new folder appeared in s3
> as defined by S3 Object Key Prefix) using Geoserver
> I try to cache tiles onto it by making a WMS request from the example
> layer topp:states, but no files are cached into the S3 blob store:
>
>
> http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,180,90&width=256&height=256&format=image/png8&tiled=true
>
>
This is just a partial answer, but I'm at a conference and there is not
much time to dedicate to the user list.
The request you're using does not conform to any cacheable gridset, so it's
not cached. The WGS84 gridset does not have one tile
at the top level, but two. Could you try these instead?:

http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,0,90&width=256&height=256&format=image/png8&tiled=true

http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=0,-90,180,90&width=256&height=256&format=image/png8&tiled=true

Also, did you enable the "transparent integration" in the "caching
defaults" page? Otherwise just making WMS requests
won't create any tiles. Maybe try to use the tiled layers preview too.

Cheers
Andrea


== GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf
Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa
(LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549
http://www.geo-solutions.it http://twitter.com/geosolutions_it
--- *Con riferimento
alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 -
Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni
circostanza inerente alla presente email (il suo contenuto, gli eventuali
allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i
destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per
errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le
sarei comunque grato se potesse darmene notizia. This email is intended
only for the person or entity to which it is addressed and may contain
information that is privileged, confidential or otherwise protected from
disclosure. We remind that - as provided by European Regulation 2016/679
“GDPR” - copying, dissemination or use of this e-mail or the information
herein by anyone other than the intended recipient is prohibited. If you
have received this email by mistake, please notify us immediately by
telephone or e-mail.*
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Re: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

[Rahkonen Jukka (MML)]

Hi,

There is almost nothing about S3 in your huge logfile (delete obviously 
irrelevant and repeating contents next time) but I can see rows referring to 
[memory.MemoryBlobStore] and  [guava.GuavaCacheProvider]. I gues that you have 
the In memory BlobStore enabled. Try if disabling it would change anything.

-Jukka Rahkonen-

Lähettäjä: Jan Donner [mailto:jan.don...@trimble.com]
Lähetetty: 30. elokuuta 2018 12:17
Vastaanottaja: geoserver-users@lists.sourceforge.net
Aihe: [Geoserver-users] Geoserver/GeoWebCache and S3 Blobstore

Hi!
I'm experiencing problems with using Geoserver and GeoWebCache with a S3 blob 
store.
I have managed to create a blob store on S3 (the new folder appeared in s3 as 
defined by S3 Object Key Prefix) using Geoserver
I try to cache tiles onto it by making a WMS request from the example layer 
topp:states, but no files are cached into the S3 blob store:

http://localhost:8080/geoserver/gwc/service/wms?layers=topp:states&styles=&version=1.1.1&srs=EPSG:4326&Request=GetMap&bbox=-180,-90,180,90&width=256&height=256&format=image/png8&tiled=true

I have also tried seeding the tiles of that same layer, but nothing happens. 
Nor are there any hints in the log as far as I can see.
What could be the problem? Should caching and / or seeding onto S3 work or are 
there known issues?
Attached is the geoserver log.
Any help would be appreciated!

--
·GeoServer Version2.13.2
·Git Revisionafacec24a28f35313f7712325be0ff8bc33ac423
·Build Date23-Jul-2018 21:50
·GeoTools Version19.2 (rev 65573d34f5a7da06abb3b793026d4b1c09af616a)
·GeoWebCache Version1.13.2 (rev 
1.13.x/78018a44f070339647f3c302b868adb5f1b08290)

best regards
Jan
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Geoserver-users mailing list

Please make sure you read the following two resources before posting to this 
list:
- Earning your support instead of buying it, but Ian Turton: 
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines: 
http://geoserver.org/comm/userlist-guidelines.html

If you want to request a feature or an improvement, also see this: 
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer


Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users