Hi,
we are using a batch program to update GeoServer with the REST API. Every hour
a couple of new GeoTIFFs are registered as new image mosaic layers, and old
layers and coverage stores are deleted. This works well, but after a couple of
days we get this exception while registering new GeoTIFFs, and GeoServer stops
working properly:
08 Aug 13:50:05 ERROR [imagemosaic.ImageMosaicUtils] - Too many open files
java.io.IOException: Too many open files
(full stacktrace see below)
Each image mosaic has only one GeoTIFF (only used for transparent background),
and the Server had virtually no load while the error appeared, so the problem
seems to be due to our updates. In the log output, you can also see that there
still is a lock on some mosaic's shape file.
What can we do to release the lock? Will that solve our problem?
Best regards,
Udo
This is our environment:
Java 1.6.0_24
No native JAI
Tomcat 6.0.18
GeoServer 2.0.2
OS: 32 bit Linux (CentOS 5.8)
Here is a log output when we delete a layer / coverage store with the REST API:
08 Aug 13:50:04 INFO [catalog.rest] - DELETE layer
WMS_MSG-RGB-natural_RGB-3-2-1_1208031115
08 Aug 13:50:04 INFO [gwc.GWCCleanser] - Deleting GWC cache for
MET9_RGB-3-2-1_msg:WMS_MSG-RGB-natural_RGB-3-2-1_1208031115
08 Aug 13:50:04 INFO [file.FileBlobStore] -
/mnt/eumetsat/data/data_dir/gwc/MET9_RGB-3-2-1_msg_WMS_MSG-RGB-natural_RGB-3-2-1_1208031115
does not exist or is not writable
08 Aug 13:50:04 INFO [gwc.GWCCleanser] - GWC cache for
MET9_RGB-3-2-1_msg:WMS_MSG-RGB-natural_RGB-3-2-1_1208031115 deleted successfully
08 Aug 13:50:04 INFO [catalog.rest] - DELETE coverage
WMS_MSG-RGB-natural_RGB-3-2-1_1208031115,WMS_MSG-RGB-natural_RGB-3-2-1_1208031115
08 Aug 13:50:04 INFO [catalog.rest] - DELETE coverage store
MET9_RGB-3-2-1_msg,WMS_MSG-RGB-natural_RGB-3-2-1_1208031115
Here is a log output when we register a new GeoTIFF with the REST API:
08 Aug 13:50:04 INFO [catalog.rest] - PUT file, mimetype: text/plain
08 Aug 13:50:04 INFO [catalog.rest] - Auto-configuring coverage store:
WMS_MSG-RGB-natural_RGB-3-2-1_1208081030
08 Aug 13:50:04 INFO [org.geoserver] - Loaded store 'null', enabled
08 Aug 13:50:04 INFO [catalog.rest] - PUT coverage store
MET9_RGB-3-2-1_msg,WMS_MSG-RGB-natural_RGB-3-2-1_1208081030
08 Aug 13:50:04 INFO [org.geoserver] - Loaded coverage
'WMS_MSG-RGB-natural_RGB-3-2-1_1208081030', enabled
08 Aug 13:50:04 INFO [catalog.rest] - PUT coverage
WMS_MSG-RGB-natural_RGB-3-2-1_1208081030,WMS_MSG-RGB-natural_RGB-3-2-1_1208081030
Here is a log output with the error:
08 Aug 13:50:04 INFO [catalog.rest] - PUT file, mimetype: text/plain
08 Aug 13:50:04 INFO [catalog.rest] - Auto-configuring coverage store:
WMS_MSG-RGB-natural_RGB-3-2-1_1208081115
08 Aug 13:50:05 ERROR [imagemosaic.ImageMosaicUtils] - Too many open files
java.io.IOException: Too many open files
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.checkAndCreate(File.java:1704)
at java.io.File.createTempFile(File.java:1792)
at java.io.File.createTempFile(File.java:1828)
at
org.geotools.data.shapefile.ShpFiles.getStorageFile(ShpFiles.java:943)
at
org.geotools.data.shapefile.ShapefileFeatureWriter.<init>(ShapefileFeatureWriter.java:118)
at
org.geotools.data.shapefile.ShapefileDataStore.createFeatureWriter(ShapefileDataStore.java:614)
at
org.geotools.data.AbstractDataStore.getFeatureWriter(AbstractDataStore.java:486)
at
org.geotools.data.AbstractDataStore.getFeatureWriter(AbstractDataStore.java:525)
at
org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.handleFile(IndexBuilder.java:729)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:335)
at
org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.<init>(IndexBuilder.java:880)
at org.geotools.gce.imagemosaic.IndexBuilder.run(IndexBuilder.java:1099)
at
org.geotools.gce.imagemosaic.ImageMosaicUtils.createMosaic(ImageMosaicUtils.java:314)
at
org.geotools.gce.imagemosaic.ImageMosaicUtils.checkSource(ImageMosaicUtils.java:416)
at
org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageMosaicReader.java:171)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:272)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:186)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:87)
at
org.geoserver.catalog.rest.CoverageStoreFileResource.handlePut(CoverageStoreFileResource.java:132)
at org.restlet.Finder.handle(Finder.java:298)
at
org.geoserver.rest.BeanDelegatingRestlet.handle(BeanDelegatingRestlet.java:38)
at org.restlet.Filter.doHandle(Filter.java:105)
at org.restlet.Filter.handle(Filter.java:134)
at org.restlet.Router.handle(Router.java:444)
at
com.noelios.restlet.ext.servlet.ServletConverter.service(ServletConverter.java:129)
at
org.geoserver.rest.RESTDispatcher.handleRequestInternal(RESTDispatcher.java:79)
at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at
org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:521)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:640)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
at
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:183)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:662)
08 Aug 13:50:05 ERROR [data.shapefile] - The following locker still has a
lock� read on
file:/mnt/eumetsat/data/work/MET9_RGB-3-2-1_msg/WMS_MSG-RGB-natural_RGB-3-2-1_1208081115.tiff/WMS_MSG-RGB-natural_RGB-3-2-1_1208081115.tiff.shp
by org.geotools.data.shapefile.shp.ShapefileReader
it was created with the following stack trace
org.geotools.data.shapefile.ShpFilesLocker$Trace: Locking
file:/mnt/eumetsat/data/work/MET9_RGB-3-2-1_msg/WMS_MSG-RGB-natural_RGB-3-2-1_1208081115.tiff/WMS_MSG-RGB-natural_RGB-3-2-1_1208081115.tiff.shp
for read by org.geotools.data.shapefile.shp.ShapefileReader in thread
http-80-32
at
org.geotools.data.shapefile.ShpFilesLocker.setTraceException(ShpFilesLocker.java:72)
at
org.geotools.data.shapefile.ShpFilesLocker.<init>(ShpFilesLocker.java:36)
at org.geotools.data.shapefile.ShpFiles.acquireRead(ShpFiles.java:365)
at
org.geotools.data.shapefile.ShpFiles.getReadChannel(ShpFiles.java:813)
at
org.geotools.data.shapefile.shp.ShapefileReader.<init>(ShapefileReader.java:213)
at
org.geotools.data.shapefile.ShapefileDataStore.openShapeReader(ShapefileDataStore.java:453)
at
org.geotools.data.shapefile.ShapefileDataStore.getAttributesReader(ShapefileDataStore.java:439)
at
org.geotools.data.shapefile.ShapefileDataStore.createFeatureWriter(ShapefileDataStore.java:600)
at
org.geotools.data.AbstractDataStore.getFeatureWriter(AbstractDataStore.java:486)
at
org.geotools.data.AbstractDataStore.getFeatureWriter(AbstractDataStore.java:525)
at
org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.handleFile(IndexBuilder.java:729)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:335)
at
org.geotools.gce.imagemosaic.IndexBuilder$MosaicDirectoryWalker.<init>(IndexBuilder.java:880)
at org.geotools.gce.imagemosaic.IndexBuilder.run(IndexBuilder.java:1099)
at
org.geotools.gce.imagemosaic.ImageMosaicUtils.createMosaic(ImageMosaicUtils.java:314)
at
org.geotools.gce.imagemosaic.ImageMosaicUtils.checkSource(ImageMosaicUtils.java:416)
at
org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageMosaicReader.java:171)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:272)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:186)
at
org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:87)
at
org.geoserver.catalog.rest.CoverageStoreFileResource.handlePut(CoverageStoreFileResource.java:132)
at org.restlet.Finder.handle(Finder.java:298)
at
org.geoserver.rest.BeanDelegatingRestlet.handle(BeanDelegatingRestlet.java:38)
at org.restlet.Filter.doHandle(Filter.java:105)
at org.restlet.Filter.handle(Filter.java:134)
at org.restlet.Router.handle(Router.java:444)
at
com.noelios.restlet.ext.servlet.ServletConverter.service(ServletConverter.java:129)
at
org.geoserver.rest.RESTDispatcher.handleRequestInternal(RESTDispatcher.java:79)
at
org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:875)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:809)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:571)
at
org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:521)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:640)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
at
org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at
org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
at
org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
at
org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:73)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.geoserver.filters.ReverseProxyFilter.doFilter(ReverseProxyFilter.java:183)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:47)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:662)
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
