Hi Peter,
Thank you for the examples, very helpful! Using a indexer.xml indeed seems
like a better set-up. Have you ever tried this in combination with the
imagepyramid plugin? I know that the imagepyramid are basically just
imagemoasics per folder level. However as the folder structure is different
it causes some problems. My folder structure looks like:
/geoserver/data/workspace/<IMAGEMOSAIC_DIRECTORY>/0/.tiffs
/geoserver/data/workspace/<IMAGEMOSAIC_DIRECTORY>/1/.tiffs
/geoserver/data/workspace/<IMAGEMOSAIC_DIRECTORY>/2/.tiffs
etc..
Where the numbers stand for the pyramid levels which are filled with
multiple tiffs, the indexer.properties and the foldername.properties. I put
tiffs from different dates in the same folder and use a unique date in the
filename. Should I put the indexer.xml and datastore.properties in each
pyramid directory? I can try it later this week and will let you know if it
works.
Kind regards,
Darell
On Fri, Sep 29, 2017 at 11:43 AM, Peter Kovac <peter.ko...@microstep-mis.com
> wrote:
> I usually configure ImageMosaic for time-series plain GeoTIFF granules so
> my advice might not be 100% accurate but I hope it helps anyway.
>
> Firstly, your schema seems wrong to me: you have two types specified for
> ingestion - java.util.Date and a Double after that. Perhaps another
> attribute of double type is missing or the :Double part should be deleted.
>
> Otherwise, I recommend using indexer.xml instead of indexer.properties for
> configuration. In your case it might look like this:
>
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <Indexer>
> <domains>
> <domain name="ingestion">
> <attributes><attribute ref="timeCollector">ingestion<
> /attribute></attributes>
> </domain>
> </domains>
> <schemas>
> <schema name="default">
> <attributes>*the_geom:Polygon,location:String,ingestion:
> java.util.Date</attributes>
> </schema>
> </schemas>
> <coverages>
> <coverage>
> <name>your_coverage_name</name>
> <schema ref="default"></schema>
> <domains>
> <domain ref="ingestion"/>
> </domains>
> </coverage>
> </coverages>
> <collectors>
> <collector name="timeCollector">
>
> <value>regex=([0-9]{14}),format=yyyyMMddHHmmss,fullPath=true</value><!--
> configure according to your time format-->
> <spi>TimestampFileNameExtractorSPI</spi>
> <mapped>ingestion</mapped>
> </collector>
> </collectors>
> <parameters>
> <parameter name="AbsolutePath" value="true" /><!-- if true, then
> database table will contain full paths in location column -->
> <parameter name="Caching" value="false" />
> <parameter name="CanBeEmpty" value="false" />
> <parameter name="IndexingDirectories"
> value="/path/to/directory/to/be/indexed"
> /><!-- this path might be relative if you know current working directory of
> your GeoServer/Tomcat instance -->
> <parameter name="Name" value="your_coverage_name" /><!-- this
> value is used as the name of the database table -->
> <parameter name="Recursive" value="true" /><!-- true if path
> specified in IndexingDirectories needs to be traversed recursively which I
> guess is your case -->
> <parameter name="TimeAttribute" value="ingestion" /><!-- your time
> dimension attribute is called ingestion -->
> </parameters>
> </Indexer>
> Usually my ImageMosaic folder structure is like this (/opt/geoserver_data
> is the GEOSERVER_DATA_DIR)
> * the indexer.xml and datastore.properties files are in a
> /opt/geoserver_data/data/<IMAGEMOSAIC_DIRECTORY>
> * there is a subdirectory, usually with a date tree structure for storing
> granules from separate days in separate directories:
> /opt/geoserver_data/data/<IMAGEMOSAIC_DIRECTORY>/tiff/
> YYYY/MM/DD/some_file_YYYYMMDDHHmss.tiff
> * in indexer.xml I have *Recursive *parameter set to
> *true ** I usually use absolute path to the tiff directory as
> *IndexingDirectories* parameter value, e.g. /opt/geoserver_data/data/<
> IMAGEMOSAIC_DIRECTORY>/tiff
>
> Finally, if I encounter any problems during ImageMosaic creation I ensure
> to:
> * set GeoServer logging to VERBOSE
> * drop any database table created (ImageMosaic will refuse to create store
> if a table with the same name already exists), along with index and
> metadata (if using Oracle)
> * try it again and look into the log (usually the problem is not revealed
> in the last stacktrace in the log, but in some before last)
> * if it doesn't help or I still cannot figure out what is wrong I launch
> Eclipse debugger and examine what EXACTLY is wrong (I used it a lot until I
> figured out how to properly configure time parsing regexes)
>
> Regards,
>
> Peter Kovac
>
>
>
> On 29. 9. 2017 10:40, Darell van der Voort wrote:
>
> Hi Peter,
>
> This is my indexer.properties configuration:
>
> Caching=false
> TimeAttribute=ingestion
> Schema=*the_geom:Polygon,location:String,ingestion:java.util.Date:Double
> PropertyCollectors=TimestampFileNameExtractorSPI[timeregex](ingestion)
>
> And this is my 0.properties which is automatically generated for the
> lowest zoomlevel (0):
>
> #-Automagically created from GeoTools-
> #Wed Sep 27 14:50:18 UTC 2017
> MosaicCRS=EPSG\:32617
> Levels=0.062,0.062
> Heterogeneous=false
> TimeAttribute=ingestion
> AbsolutePath=false
> Name=0
> TypeName=0
> Caching=false
> ExpandToRGB=false
> LocationAttribute=location
> SuggestedSPI=it.geosolutions.imageioimpl.plugins.tiff.TIFFImageReaderSpi
> CheckAuxiliaryMetadata=false
> LevelsNum=1
>
> So like I said, the plugin automatically configures level 0 to the
> PostgreSQL DB if I add the datastore.properties files, but refuses to do so
> for other levels if I add the datastore.properties file to the folder
> levels. If I do not add the datastore.properties file to the additional
> levels, the plugin automatically creates a shapefile for these levels,
> while using the DB connection for level 0. This combination works, even
> with a time dimension. However I would like to use the DB connection for
> all levels and use unique table names in the DB.
>
> Kind regards,
> Darell
>
>
>
>
> On Fri, Sep 29, 2017 at 9:59 AM, Peter Kovac <
> peter.ko...@microstep-mis.com> wrote:
>
>> Hi Darell,
>>
>> can you post your indexer.xml configuration for the imagemosaic?
>>
>> Regards,
>>
>> Peter Kovac
>>
>> On 28. 9. 2017 9:59, Darell van der Voort wrote:
>>
>> Hello everybody,
>>
>> I'm running Geoserver 2.11.2 on Java 1.8.0_144 32 bits on Amazon Linux
>> together with Tomcat 9. The plugins I have installed are imagemosaic,
>> imagepyramid, GeowebCache (internal) and Geofence.
>>
>> When using the imagemosaic plugin I am able to use a Postgresql DB for
>> storing the granules and time dimension via a datastore.properties file.
>> The file looks like:
>>
>> SPI=org.geotools.data.postgis.PostgisNGJNDIDataStoreFactory
>> schema=database_schema_name
>> Loose\ bbox=true
>> Estimated\ extends=false
>> validate\ connections=true
>> Connection\ timeout=10
>> preparedStatements=true
>> jndiReferenceName=java:comp/env/jdbc/postgis
>>
>> Which work as expected. However when I add the same file in the
>> imagepyramid datastructure, this setup fails. When I add this
>> datastore.properties in just the lowest level (0) then that level gets
>> configured in a database table, while the higher levels are configured
>> using a shapefile. When I add a datastore.properties to all levels,
>> geoserver returns the following error:
>>
>> 2017-09-28 07:39:40,780 ERROR [gce.imagemosaic] -
>> java.io.IOException
>> at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.create
>> GranuleCatalogFromDatastore(ImageMosaicConfigHandler.java:406)
>> at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.create
>> Catalog(ImageMosaicConfigHandler.java:329)
>> at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.buildC
>> atalog(ImageMosaicConfigHandler.java:962)
>> at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.indexi
>> ngPreamble(ImageMosaicConfigHandler.java:941)
>> at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker$Mosa
>> icDirectoryWalker.<init>(ImageMosaicDirectoryWalker.java:109)
>> at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker.run(
>> ImageMosaicDirectoryWalker.java:197)
>> at org.geotools.gce.imagemosaic.Utils.createMosaic(Utils.java:460)
>> at org.geotools.gce.imagemosaic.Utils.checkSource(Utils.java:1336)
>> at org.geotools.gce.imagemosaic.ImageMosaicReader.initReaderFro
>> mURL(ImageMosaicReader.java:532)
>> at org.geotools.gce.imagemosaic.ImageMosaicReader.<init>(ImageM
>> osaicReader.java:479)
>> at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(Ima
>> geMosaicFormat.java:504)
>> at org.geotools.gce.imagepyramid.Utils.checkSource(Utils.java:200)
>> at org.geotools.gce.imagepyramid.ImagePyramidReader.<init>(Imag
>> ePyramidReader.java:177)
>> at org.geotools.gce.imagepyramid.ImagePyramidFormat.getReader(I
>> magePyramidFormat.java:249)
>> at org.geotools.gce.imagepyramid.ImagePyramidFormat.getReader(I
>> magePyramidFormat.java:56)
>> at org.geoserver.catalog.ResourcePool.getGridCoverageReader(Res
>> ourcePool.java:1511)
>> at org.geoserver.catalog.ResourcePool.getGridCoverageReader(Res
>> ourcePool.java:1441)
>> at org.geoserver.catalog.impl.CoverageStoreInfoImpl.getGridCove
>> rageReader(CoverageStoreInfoImpl.java:59)
>> .....
>> at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.
>> run(TaskThread.java:61)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: java.lang.NullPointerException
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:264)
>> at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.create
>> GranuleCatalogFromDatastore(ImageMosaicConfigHandler.java:389)
>> ... 160 more
>>
>> Is it possible to configure the granules of all levels in the database?
>> And if so, how would my datastore.properties file look like? Like I said,
>> it works fine for the lowest level (0) but it fails for all the levels
>> above. If it may not be possible, is it sufficient to use the shapefiles as
>> an index for the granules? The geoserver is used in a production
>> environment and I am worried about performance issues when using the
>> shapefiles as index.
>>
>> Furthermore the imagepyramid automatically names the table after the
>> level folder (0). This folder and table name are automatically created when
>> configuring the datastore. Is there a way to custom name the folder and
>> thus the table name? Otherwise I could end up with multiple '0' tables in
>> the same schema.
>>
>> Many thanks!
>>
>> Kind regards,
>> Darell
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Geoserver-users mailing list
>>
>> Please make sure you read the following two resources before posting to this
>> list:
>> - Earning your support instead of buying it, but Ian Turton:
>> http://www.ianturton.com/talks/foss4g.html#/
>> - The GeoServer user list posting guidelines:
>> http://geoserver.org/comm/userlist-guidelines.html
>> Geoserver-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/geoserver-users
>>
>>
>> --
>> Peter Kovac
>> IMS Programmer
>> microstep-mispeter.ko...@microstep-mis.com
>>
>>
>
> --
> Peter Kovac
> IMS Programmer
> microstep-mispeter.ko...@microstep-mis.com
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Geoserver-users mailing list
Please make sure you read the following two resources before posting to this
list:
- Earning your support instead of buying it, but Ian Turton:
http://www.ianturton.com/talks/foss4g.html#/
- The GeoServer user list posting guidelines:
http://geoserver.org/comm/userlist-guidelines.html
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
