Hi Daniele, Thanks ! With your help and my friends at www.java-monitor.com the issue was resolved. See also this thread: http://java-monitor.com/forum/showthread.php?p=2764#post2764 I can recommend using java-monitor when running Java GIS services.
All in all two actions were needed to resolve the problem: 1- delete and re-add the 3 mosaicking datastores/layers (total of about 1000 GeoTIFFs). This resolved the original "WMS hangs for Image..." 2- increase open files limit from 1024 (!) to 20480. This resolved the "Too many open files.." issue (which eventually also appeared on 2.1-beta3). I used USE_JAI_IMAGEREAD = true on all 3 layers, accessed them from multiple browsers and monitored open files : lsof | grep <tiffdir> | wc -l. The number of open files increases in in cases (zooming out) above 1024 but always settles back down to a few hundred, i.e. I haven't noticed any open files leakage. So I now keep running 2.1-rc1. Again thanks. best regards, Just van den Broecke www.justobjects.nl On 19-01-11 18:09, Daniele Romagnoli wrote: > Hi Just, > > On Wed, Jan 19, 2011 at 5:10 PM, Just van den Broecke > <j...@justobjects.nl <mailto:j...@justobjects.nl>> wrote: > > Hi Daniele, > > Please read below. > > On 19-01-11 11:55, Daniele Romagnoli wrote: > > Hi, > > please read below... > > > > > > On Tue, Jan 18, 2011 at 8:30 PM, Just van den Broecke > > <j...@justobjects.nl <mailto:j...@justobjects.nl> > <mailto:j...@justobjects.nl <mailto:j...@justobjects.nl>>> wrote: > > > > Hi, > > > > I upgraded 2.1-beta3 to 2.1-rc1 and all (PostGIS, Shapefile > layers etc) > > looked bright except for an issue with the Image mosaicking > plugin: any > > WMS request to one mosaicking layer just hangs forever > without any error > > in the log nor a CPU spin. Moving back to 2.1-beta3 it works > again. > > > > > > can you share the workspace for that coverage? > > You can find it within your > > $GEOSERVER_DATA_DIR/workspaces/$WORKSPACE/$COVERAGE > > where $WORKSPACE is the workspace you have used to configure the > > coverage, as an instance, "cite", "it.geosolutions", ... > > and $COVERAGE is the name you used for your coverage. > > In that dir you should find a coveragestore.xml as well as a > folder with > > the name of your layer containing a coverage.xml as well as a > layer.xml. > > Can you zip it and send it to me? > Ok, done > > Thank you. I'm checking it right now. > > > > > > > > > This happens on the following environment: > > Ubuntu 9.10 32b, 12 GB RAM, Sun JDK 1.6_22, native > JAI+ImageIO, Tomcat > > 6.0.29, no Gdal bindings. Tomcat JAVA_OPTS="-server -Xmx2048M > -Xms256M > > -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:MaxPermSize=512m > > -XX:+UseParallelGC". The original GS 2.1-beta3 was patched with a > > gt-shapefile-2.7-SNAPSHOT.jar provided by Andrea on > 22/12/2010 to fix > > http://jira.codehaus.org/browse/GEOS-4268 (should be in 2.1-rc1). > > GS 2.1-rc1 was dated 2011-01-15 01:22 > > > > The source data is a directory with about 380 GeoTIFF files > each about > > 14MB, referenced for EPSG:28992 and overviews added: > > gdaladdo -r average <tifffile> 2 4 8 16 32 64 128. > > None of the generated index shapefiles was changed and are > readable. > > > > What happens if you try to configure a new coveragestore using > 2.1-RC1 > > with the same datasets (I suggest to do a backup or work to a > different > > mosaic)? > Initially this seemed to work (e.g. in the layer preview) but the all > kinds of problems appeared: > - not always the entire coverage was shown, sometimes it was mostly > black. Usually after zooming in and then zooming out. > - errors in the log: "Too many open files" for any file and for > connections > > > This usually happens when there are no more file descriptors available > which usually occurs when the max number of open files has been reached. > Once there are no more file descriptors available, any kind of error may > happen (as reported by your stack trace and crashes). > A first thing you can try, is reconfiguring the imageMosaic coverages by > setting the USE_JAI_IMAGEREAD = false. Basically, by doing this, you are > forcing the imageMosaic to deal with standard ImageReader.read > operations (Immediate loading), followed by ImageReader.close operations. > When using instead JAI ImageRead operations, the imageMosaic leverages > on the deferred execution model, which load data only when it is really > needed (by scheduling and caching tiles). This approach don't explicity > close the readers. So, if you have several mosaic composed of > hundreds/thousands of tiff files, I would try configuring them to > leverage on USE_JAI_IMAGEREAD = false. Do to this, you can edit > coverage.xml (the ones you sent to me) for your coverages before > starting GeoServer or you can edit this property through the GUI. > > Please, let me know if this suggestion partially solves your issues. > Then, I will investigate again based on your feedbacks. > > Regards, > Daniele > > > Since I have 3 similar coverage stores I tried removing the Shape files > first before adding the new store. But now I couldn't addd the store at > all: GUI error: > "Could not list layers for this store, an error occurred retrieving > them: Unable to acquire a reader for this coverage with format: > ImageMosaic" and an errortrace in the log: > 19 Jan 17:01:28 ERROR [imagemosaic.Utils] - Unable to build mosaic > java.lang.IllegalArgumentException: No input stream for the provided > source > at > org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:201) > at > org.geotools.gce.geotiff.GeoTiffFormat.getReader(GeoTiffFormat.java:271) > at > org.geotools.gce.geotiff.GeoTiffFormat.getReader(GeoTiffFormat.java:77) > at > > org.geotools.gce.imagemosaic.catalogbuilder.CatalogBuilder$CatalogBuilderDirectoryWalker.handleFile(CatalogBuilder.java:455) > at > org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367) > at > org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:335) > at > > org.geotools.gce.imagemosaic.catalogbuilder.CatalogBuilder$CatalogBuilderDirectoryWalker.<init>(CatalogBuilder.java:724) > . > . > 19 Jan 17:01:28 WARN [imagemosaic.ImageMosaicFormat] - This plugin > accepts File, URL or String. The string may describe a File or an URL > org.geotools.data.DataSourceException: This plugin accepts File, URL or > String. The string may describe a File or an URL > at > > org.geotools.gce.imagemosaic.ImageMosaicReader.initReaderFromURL(ImageMosaicReader.java:230) > at > > org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageMosaicReader.java:192) > at > > org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:451) > at > > org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:103) > . > . > 19 Jan 17:01:28 INFO [geoserver.web] - Getting list of coverages for > saved store file:/var/kademo/data/bonne/tiffperiod/1901-1925 > java.lang.RuntimeException: Could not list layers for this store, an > error occurred retrieving them: Unable to acquire a reader for this > coverage with format: ImageMosaic > at > > org.geoserver.web.data.layer.NewLayerPageProvider.getItems(NewLayerPageProvider.java:119) > at > > org.geoserver.web.wicket.GeoServerDataProvider.getFilteredItems(GeoServerDataProvider.java:201) > at > > org.geoserver.web.data.layer.NewLayerPageProvider.getFilteredItems(NewLayerPageProvider.java:135) > at > > org.geoserver.web.wicket.GeoServerDataProvider.size(GeoServerDataProvider.java:216) > at > > org.geoserver.web.wicket.GeoServerTablePanel$Pager.first(GeoServerTablePanel.java:502) > at > > org.geoserver.web.wicket.GeoServerTablePanel$Pager.updateMatched(GeoServerTablePanel.java:492) > . > . > Caused by: java.lang.Exception: Unable to acquire a reader for this > coverage with format: ImageMosaic > at > > org.geoserver.catalog.CatalogBuilder.buildCoverage(CatalogBuilder.java:627) > at > > org.geoserver.web.data.layer.NewLayerPageProvider.getItems(NewLayerPageProvider.java:86) > ... 164 more > > Hmm tried some other action in the GUI (deliberately a wrong dir name to > check if the dir name was richt) and then even the JVM crashed ! > > > ava frames: (J=compiled Java code, j=interpreted, Vv=VM code) > J > > java.lang.ClassLoader.findBootstrapClass(Ljava/lang/String;)Ljava/lang/Class; > J java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; > J java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class; > v ~StubRoutines::call_stub > J > > java.lang.Class.forName0(Ljava/lang/String;ZLjava/lang/ClassLoader;)Ljava/lang/Class; > J > > java.lang.Class.forName(Ljava/lang/String;ZLjava/lang/ClassLoader;)Ljava/lang/Class; > j sun.misc.Service$LazyIterator.next()Ljava/lang/Object;+31 > j java.nio.charset.Charset$1.getNext()Z+26 > j java.nio.charset.Charset$1.hasNext()Z+1 > j java.nio.charset.Charset$4.run()Ljava/lang/Object;+26 > v ~StubRoutines::call_stub > J > > java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object; > j java.nio.charset.Charset.availableCharsets()Ljava/util/SortedMap;+7 > j > > org.geoserver.web.data.store.panel.CharsetPanel.<init>(Ljava/lang/String;Lorg/apache/wicket/model/IModel;Lorg/apache/wicket/model/IModel;Z)V+73 > j > > org.geoserver.web.data.store.shape.ShapefileStoreEditPanel.<init>(Ljava/lang/String;Lorg/apache/wicket/markup/html/form/Form;)V+94 > j > > org.geoserver.web.data.store.shape.ShapefileDirectoryEditPanel.<init>(Ljava/lang/String;Lorg/apache/wicket/markup/html/form/Form;)V+3 > v ~StubRoutines::call_stub > > > > > Other mosaicking layers prepared in the same way do work. > > > > Just to make sure I have understood: > > Do you have other previously (2.1-beta3) configured mosaics which > still > > work on 2.1-RC1 whilst the one composed of 380 GeoTIFF EPSG:28992 > is the > > only one which doesn't work? > Yes, but it those mosaics were configured in a much earlier GS version > (think 2.0.2). But these appear to be quite stable in any version. > > best, > > Just van den broecke > www.justobjects.nl <http://www.justobjects.nl> > > > > > Please, let me know. > > Regards, > > Daniele > > > > > > best, > > > > Just van den Broecke > > www.justobjects.nl <http://www.justobjects.nl> > <http://www.justobjects.nl> > > > > On 18-01-11 17:07, Justin Deoliveira wrote: > > > The GeoServer team is happy to announce the first release > > candidate of > > > 2.1. Now available for download: > > > > > > http://geoserver.org/display/GEOS/GeoServer+2.1-RC1 > > > > > > Check out the blog for what's new in this release: > > > > > > http://blog.geoserver.org/2011/01/18/geoserver-21rc1-released/ > > > > > > The entire change log is also available courtesy of JIRA: > > > > > > > > > > http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982 > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982> > > > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982 > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982>> > > > > > > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982 > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982> > > > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982 > > <http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=10311&version=16982>>> > > > > > > Download the release and help us get to the final 2.1 release by > > trying > > > it out and reporting any issues in the bug tracker or on the > > mailing list. > > > > > > Thanks for using GeoServer! > > > > > > -- > > > The GeoServer Team > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > Protect Your Site and Customers from Malware Attacks > > > Learn about various malware tactics and how to avoid them. > Understand > > > malware threats, the impact they can have on your business, and > > how you > > > can protect your company and customers by using code signing. > > > http://p.sf.net/sfu/oracle-sfdevnl > > > > > > > > > > > > _______________________________________________ > > > Geoserver-users mailing list > > > Geoserver-users@lists.sourceforge.net > <mailto:Geoserver-users@lists.sourceforge.net> > > <mailto:Geoserver-users@lists.sourceforge.net > <mailto:Geoserver-users@lists.sourceforge.net>> > > > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > Protect Your Site and Customers from Malware Attacks > > Learn about various malware tactics and how to avoid them. > Understand > > malware threats, the impact they can have on your business, > and how you > > can protect your company and customers by using code signing. > > http://p.sf.net/sfu/oracle-sfdevnl > > _______________________________________________ > > Geoserver-users mailing list > > Geoserver-users@lists.sourceforge.net > <mailto:Geoserver-users@lists.sourceforge.net> > > <mailto:Geoserver-users@lists.sourceforge.net > <mailto:Geoserver-users@lists.sourceforge.net>> > > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > > > > > > > > > -- > > ------------------------------------------------------- > > Ing. Daniele Romagnoli > > Software Engineer > > > > GeoSolutions S.A.S. > > Via Poggio alle Viti 1187 > > 55054 Massarosa (LU) > > Italy > > > > phone: +39 0584962313 > > fax: +39 0584962313 > > mob: +39 328 0559267 > > > > > > http://www.geo-solutions.it > > http://geo-solutions.blogspot.com/ > > > > ------------------------------------------------------- > > > > > > > > > ------------------------------------------------------------------------------ > Protect Your Site and Customers from Malware Attacks > Learn about various malware tactics and how to avoid them. Understand > malware threats, the impact they can have on your business, and how you > can protect your company and customers by using code signing. > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > Geoserver-users mailing list > Geoserver-users@lists.sourceforge.net > <mailto:Geoserver-users@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/geoserver-users > > > > > -- > ------------------------------------------------------- > Ing. Daniele Romagnoli > Software Engineer > > GeoSolutions S.A.S. > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > > phone: +39 0584962313 > fax: +39 0584962313 > mob: +39 328 0559267 > > > http://www.geo-solutions.it > http://geo-solutions.blogspot.com/ > > ------------------------------------------------------- > ------------------------------------------------------------------------------ Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
