Hello,
We have been working on this, but still couldn't make it work. As we now
have a better understanding of the problem, we decided to start from the
beginning and give a more detailed explanation. With a bit of luck
someone has run through the same or similar problem and can give us some
clue.
1.
What we are trying to do:
- Seed the cache for a vector layer that is stored in PostGIS.
- GeoServer is connected to PostGIS via JNDI (although we have also
tested via plain JBDC)
- Seed the cache for mapbox vector tile format (PBF)
- We have set Meta-tiling to 1x1, as suggested by Olivier in a
previous message (also explained in GEO-8228).
2.
What happens when we try to seed the cache in the setup that fails:
- It seeds the cache successfully if we use image/jpg, image/png or
image/png8 formats.
- The seeding process gets stuck (until the task is apparently
killed by GeoServer), stating that -1 tiles have been created.
- GeoServer logs (we enable verbose logging) show that there are
StackOverflowError exceptions when we try to seed the case (with the
default stack size of 1 MB).
- We have raised the stack size to 2MB (using -Xss2m) and those
exceptions disappear. But now we get "Undeclared thowable exception"
exceptions, whose root cause seem to be some kind of timeout
somewhere in GeoServer (see attache log file).
3.
What happens when we try to seed the cache in the setup that works:
- It seeds the cache successfully if we use image/jpg, image/png or
image/png8 formats.
- It seeds the cache successfully if we use "mapbox vector tile"
format, or GeoJSON format
4.
What happens if we try to preview the cached layer in the
configuration that fails, using openlayer library
- We can successfully display the pre-seeded tiles in the cache
preview tool in GeoServer, for image/jpg, image/png and image/png8
formats.
- No tiles are displayed, and we get HTTP status codes 503 (Service
Unavailable) when we try to preview the PBF or GeoJSON formats.
- Once we've tried to preview PBF or GeoJSON formats and got 503
errors, we also get the same errors (and nothing is displayed) if we
try to preview image/jpg, image/png and image/png8 formats again
(even if the cache is already pre-seeded). The only way to clear
this situation is by restarting GeoServer.
5.
What happens if we try to preview the cached layer in the
configuration that works, using openlayer library
- We can successfully display the pre-seeded tiles in the cache
preview tool in GeoServer, for image/jpg, image/png and image/png8
formats.
- We can successfully display the pre-seeded tiles in the cache
preview tool in GeoServer for "mapbox vector tile" format.
- We can't display the pre-seeded tiles in the cache preview tool in
GeoServer for GeoJSON format. There are not HTTP errors, but nothing
is shown.
6.
Details on the configuration that fails:
- We are using https://github.com/kartoza/docker-geoserver
<https://github.com/kartoza/docker-geoserver> Docker image for GeoServer
- We are using the default controlflow values configured in the
above Docker image.
- We have added and enabled the following plugins to the base
configuration:
- importer plugin,
- gwc-s3-plugin,
- wps-jdbc-plugin,
- imagemosaic-jdbc-plugin,
- jdbcstore-plugin
- The machine where the Docker container runs is an AWS EC2 instance
with 4 cores and 16 GB of RAM
- The EC2 instance and the Docker container are managed by AWS
ElasticBeanstalk
- The blobstore we configure for the place on a directory on an EFS
filesystem (AWS service that implements NFS). But we have also tried
temporarily using a directory on a local disk of the EC2 instance
with the exact same results.
- PostGIS database is a AWS RDS instance, running Postgresql 12.5
with PostGIS 3.0.0
7.
Details on the configuration that works:
- Same Docker container as above, with same plugins-
- Running on local develop machine with 4 cores and 12 GB of RAM.
- Using docker-compose to run the containers
- Cache blobstore on local machine disk (NVMe)
- PostGIS database running locally, using
"postgis/postgis:12-3.1-alpine" container (Postgresql 12.6, con
Postgis 3.1.1)
If you need any additional details, don't hesitate to contact me.
Thank you
Bingen Galartza Iparragirre
magnet.coop
21/7/27 11:41(e)an, Bingen Galartza Iparragirre igorleak idatzi zuen:
Hi all,
After further investigation we think that the problem might not be in
the S3 blobstore plugin. We found that there is an open issue [1] for
the extension that says that the caching throws an exception when a
tile is empty/blank. We were getting the very same exception, buf
after patching the code (just a workaround, so that's why we didn't
PR), we still get some exceptions in the Geoserver side. That's why we
think that the problem is when generating the tiles before uploading
them to S3. So, the GWC S3 extension might be getting empty tiles when
it shouldn't.
_When using the cache layer preview we get the following exceptions
(full stacktrace attached):_
java.lang.NullPointerException
at
org.geoserver.gwc.wms.CachingWebMapService.invoke(CachingWebMapService.java:75)
at
org.geoserver.gwc.wms.CachingWebMapService.invoke(CachingWebMapService.java:41)
And all the requests fail with a 503 http status code. The normal
Geoserver preview works without any problem.
_And if we use the seeding tool:_
Error dispatching tile request to GeoServer
org.geowebcache.GeoWebCacheException: Problem communicating with GeoServer
at
org.geoserver.gwc.layer.GeoServerTileLayer.getMetatilingReponse(GeoServerTileLayer.java:657)
at
org.geoserver.gwc.layer.GeoServerTileLayer.getTile(GeoServerTileLayer.java:592)
Caused by: java.lang.IllegalStateException: Expected:
RenderedImageMap, got null
at
org.geoserver.gwc.layer.GeoServerTileLayer.dispatchGetMap(GeoServerTileLayer.java:709)
at
org.geoserver.gwc.layer.GeoServerTileLayer.getMetatilingReponse(GeoServerTileLayer.java:651)
[1] "S3 storage: GeoWebCacheException/NullPointerException for blank
tiles" https://github.com/GeoWebCache/geowebcache/issues/976
Any idea what we might be missing?
Thank you
Bingen
21/7/23 09:02(e)an, Lucas Sousa igorleak idatzi zuen:
Hi Olivier,
Thanks for the advice but no luck. I had the same problem again but I
tried using a different blobstore, a local filesystem in this case,
and it worked without problems.
So I think the problem is related with the S3 blobstore plugin.
Thank you again,
Lucas.
On 22/7/21 21:23, Olivier Gagnon wrote:
Hi,
we cache vector tiles without any problems now but I remember that
we had to change a few cache settings for pbf.
Metatile size 1 X 1
gutter size 10
Also try removing all gridsets and re-add them.
See if it helps
Good luck !
Envoyé à partir d’Outlook <http://aka.ms/weboutlook>
------------------------------------------------------------------------
*De :* Lucas Sousa <lucas.so...@magnet.coop>
*Envoyé :* 22 juillet 2021 11:32
*À :* geoserver-users@lists.sourceforge.net
<geoserver-users@lists.sourceforge.net>
*Objet :* [Geoserver-users] Error when caching mapbox-vector-tile
format tiles
Hi everyone,
We are having a problem when caching tiles using the PBF format in
Geoserver (v2.19.0) using the GWC (v1.19.0) integration, Java
OpenJDK 11 and Debian 10. The thing is if we cache using a different
tile format such as PNG or GeoJSON everything works as expected.
However, if try to cache tiles using PBF (mapbox-vector-tile) format
the process throws an exception in Geoserver and caches just a few a
tiles (we are using S3 blob store plugin to cache tiles). After the
first try it doesn't cache anymore. We checked the Geoserver logs
using the VERBOSE logging profile and we get the following exceptions:
Error dispatching tile request to GeoServer
org.geowebcache.GeoWebCacheException: Problem communicating with
GeoServer
Caused by: java.lang.IllegalStateException: Expected:
RenderedImageMap, got null
java.lang.NullPointerException
I attached the full log file for the details.
A couple of details about the layers:
- They are stored in PostGIS (JNDI).
- The shape files are about 35.9 MiB.
Does anyone know what could be happening?
Thanks in advance.
--
*LUCAS SOUSA DE FREITAS*
(+34) 688 714 662
magnet.coop
Santa Marina 3, Oñati (20560)
i...@magnet.coop <mailto:i...@magnet.coop>
*www.magnet.coop <http://www.magnet.coop>*
/This e-mail and the documents attached are confidential and
intended solely for the addressee; it may also be privileged. If you
receive this communication in error, please notify the sender
immediately and delete it. If you are not the intended recipient,
you are expressly prohibited from reading, copying, distributing,
disseminating or, in any other way, using any of the information
without the Magnet consent. Please note that Internet e-mail
guarantees neither the confidentiality nor the proper receipt of the
message sent, so that Magnet shall not be liable for any damages
caused. As its integrity cannot be secured on the Internet, the
Magnet liability cannot be triggered for the message content.
Although the sender endeavors to maintain a computer virus-free
network, the sender does not warrant that this transmission is
virus-free and will not be liable for any damages resulting from any
virus transmitted./
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian
Turton:http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting
guidelines:http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see
this:https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
--
*LUCAS SOUSA DE FREITAS*
(+34) 688 714 662
magnet.coop
Santa Marina 3, Oñati (20560)
i...@magnet.coop
*www.magnet.coop*
/This e-mail and the documents attached are confidential and intended
solely for the addressee; it may also be privileged. If you receive
this communication in error, please notify the sender immediately and
delete it. If you are not the intended recipient, you are expressly
prohibited from reading, copying, distributing, disseminating or, in
any other way, using any of the information without the Magnet
consent. Please note that Internet e-mail guarantees neither the
confidentiality nor the proper receipt of the message sent, so that
Magnet shall not be liable for any damages caused. As its integrity
cannot be secured on the Internet, the Magnet liability cannot be
triggered for the message content. Although the sender endeavors to
maintain a computer virus-free network, the sender does not warrant
that this transmission is virus-free and will not be liable for any
damages resulting from any virus transmitted./
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian
Turton:http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting
guidelines:http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see
this:https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
If you want to request a feature or an improvement, also see this:
https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
