On Tue, Jul 9, 2013 at 6:27 AM, Craig Jones <craig.jo...@utas.edu.au> wrote:
> Hi All,
>
> We are getting the exception below when downloading a large wfs response
> in csv or gml format using gzip compression. Were are using geoserver
> 2.1.1 in production but this also happens in 2.3.3.
>
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOf(Arrays.java:2271)
> java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
>
> java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
> java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
>
> java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:253)
> java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:211)
> java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:146)
>
> org.geoserver.filters.GZIPResponseStream.write(GZIPResponseStream.java:86)
>
> org.geoserver.filters.GZIPResponseStream.write(GZIPResponseStream.java:79)
>
> org.geoserver.filters.AlternativesResponseStream.write(AlternativesResponseStream.java:53)
>
> org.geoserver.wfs.response.WfsExceptionHandler.handle1_0(WfsExceptionHandler.java:132)
>
> org.geoserver.wfs.response.WfsExceptionHandler.handleDefault(WfsExceptionHandler.java:81)
>
> org.geoserver.wfs.response.WfsExceptionHandler.handleServiceException(WfsExceptionHandler.java:59)
>
> org.geoserver.ows.Dispatcher.handleServiceException(Dispatcher.java:1638)
> org.geoserver.ows.Dispatcher.exception(Dispatcher.java:1583)
> org.geoserver.ows.Dispatcher.handleRequestInternal(Dispatcher.java:282)
>
> org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
>
> org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
>
>
> From the log message above, geoserver is falling over writing all the
> gzipped output to a ByteArrayOutputStream which in this case filled up all
> available memory.
>
>
This is a serious issue, because it goes against the basic design
principles of GeoServer, where each data access is performed
in a streaming way, read a bit of input, encode a bit out output, repeat.
That said, can you describe a bit more your scenario? There is one data
source that unfortunately cannot do streaming (the database is not JDBC
compliant, ignores the fetch size we set), which is mysql, is that your
case?
> I note that this was the subject of a previous issue GEOS-2626, but it
> ooks to me like the changes made there were reverted as a part of the
> changes made in GEOS-4845 r16568.
>
I'm confused about the reference to GEOS-4845, it just moved the existing
files from web/app to main to make it easier building a custom version
of GeoServer without having to recur to maven war overlays?
Mind, the history in git is only 2 years deep, using "git blame" often
results in surprising or incomplete outputs
Anyways, the gzip compression filter can be disabled by manipulating the
web.xml file, have you tried removing it? Does that fix your issue?
Cheers
Andrea
--
==
Our support, Your Success! Visit http://opensdi.geo-solutions.it for more
information.
==
Ing. Andrea Aime
@geowolf
Technical Lead
GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549
http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------
------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
