Re: [Geoserver-users] Issues with image pyramid
On Fri, 2 Nov 2012 07:16:59 AM Stricker, Scott wrote: > Apologies if this is not the right forum for this question. > > I'm trying to setup geoserver environment for demo purposes to display > satellite imagery such as truemarble imagery > (http://www.unearthedoutdoors.net/global_data/true_marble/download). > > I'd like to set it up as follows > Scale > A - show one set of tiles > Scale range < A and > B higher res set of tiles Scale range < B and > C even > higher res set of tiles And so on. > Can I build an 'image pyramid' data set manually? For example, configure > geoserver to use 32km TIFF for lowest resolution 1km TIFF tiles for next > 500m tiles for next resolution level 250m tiles for highest resolution > > Seems like it should be fairly straightforward - what am I missing? > > Thanks for any help/pointers/links that can be provided? I build my image pyramids manually (well I've automated it using a bash script - attached for reference). The requirements as I understand them (mostly from trial and error): The root directory has a .properties file and a .prj file which both have the same name as pyramid layer. Each level of the pyramid is in a separate directory which contains an ImageMosaic consisting of a number of images (TIFF files), a shape file (.shp, .shx, .dbf) created by gdaltindex, a .properties file and a .prj file. The properties files is different from the root one while the .prj file is the same (don't know if they have to be, but mine all are). For example I have a layer called bathy_pyramid. In my coverages/bathy_pyrmid directory I have: >more bathy_pyramid.properties Name=bathy_pyramid LevelsNum=10 LevelsDirs=1 2 4 8 16 32 64 128 256 512 Envelope2D=109.8395984985,-64.001972 173.996985,0.1604010028 Levels=2.44618395E-4,2.44618395E-4 4.89236791E-4,4.89236791E-4 9.78473581E-4,9.78473581E-4 0.00195694716,0.00195694716 0.00391389432,0.00391389432 0.00782778865,0.00782778865 0.0156555773,0.0156555773 0.0313111546,0.0313111546 0.0626223092,0.0626223092 0.125244618,0.125244618 >more bathy_pyramid.prj GEOGCS["WGS 84", DATUM["World Geodetic System 1984", SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]], PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]], UNIT["degree", 0.017453292519943295], AXIS["Geodetic longitude", EAST], AXIS["Geodetic latitude", NORTH], AUTHORITY["EPSG","4326"]] >ls 512 bathy_110_174_-64_0_bathy.tif bathy_pyramid.dbf bathy_pyramid.prj bathy_pyramid.properties bathy_pyramid.shp bathy_pyramid.shx >more 512/bathy_pyramid.properties Name=bathy_pyramid LocationAttribute=location LevelsNum=1 Envelope2D=109.8395984985,-64.001972 173.996985,0.1604010028 Levels=0.125245,0.125245 Points to note: - The Name and Envelope2D fields for all the properties files for a pyramid must be identical. - geoserver must have at least read access to all files and write access to the directories (very easy to overlook when you do things manually) - The Levels field in the root properties file is used by geoserver to determine which level to use for the resolution requested, I'm not sure of the algorithm but you can play with the numbers to get it to do what you want. The Levels field in the level directories is the pixels size of the TIFFs, I think this must match the TIFFs and all the TIFFs must have the same resolution. The Levels field in the level properties and the corresponding value in the root properties don't have to be the same, but in my case always are. - The number of (space separated) entries in LevelDirs and Levels in the root properties file must be exactly LevelsNum - The value of the LocationAttribute field of the level properties file matches the field name used in gdaltindex, e.g.: gdaltindex -tileindex location bathy_pyramid.shp *.tif Hope this helps Gordon update_pyramid Description: application/shellscript -- LogMeIn Central: Instant, anywhere, Remote PC access and management. Stay in control, update software, and manage PCs from one command center Diagnose problems and improve visibility into emerging IT issues Automate, monitor and manage. Do more in less time with Central http://p.sf.net/sfu/logmein12331_d2d___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] How to update existing Imagepyramid
On Thu, 5 Jul 2012 10:21:54 PM Casper Børgesen wrote:
> I have an existing ImagePyramid in GeoServer 2.1.4 (but I can use 2.2 if
> that helps).
>
> If it is possible, how can I update the pyramid (add new tiles) without
> deleting and re-creating the ImagePyramid store each time?
I do it by running gdaltindex to add the new tiles to the existing shape file
in the level directory of the image pyramid.
I'm not sure what is required to load the new shape file into memory. I often
restart geoserver - I know that works, but there may be less drastic methods if
restarting is not an option for you (I'd be interested in hearing what does
work if anyone knows).
For example the following extract from my update bash script ensures all tiff
files are indexed in the shape files for the pyramid (this does generate plenty
of warnings that some files are already in the index):
for f in */. ; do
(
cd $f
find . -name "*.tif" -exec gdaltindex -tileindex location $name.shp
'{}' '+'
)
done
Regards
Gordon
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Problem adding an image pyramid
On Tue, 26 Jun 2012 09:35:29 PM johnrobot wrote: > Hi > I am having trouble adding an image pyramid created using FME. I have the > following: > > - A folder (D:\Ortofoto\2010_pyramid) containing subfolders (0, 1, 2...) > with tiled images > - In each of these subfolders, there is a index SHP-file (called > 2010_pyramid). The attribute location contains the name of the corresponding > TIF file for each polygon in the SHP file. The images and the polygons > match spatially. > - In the folder D:\Ortofoto\2010_pyramid, I have added the files > 2010_pyramid.prj and 2010_pyramid.properties. > > When I try to add the pyramid in GeoServer, I get the error message > > "Could not list layers for this store, an error occurred retrieving them: > Unable to create reader for this mosaic since we could not parse the > configuration." > > What am I missing? Do I need any extra files, rename files, move any of the > files or edit the properties file? I am having trouble finding any > documentation besides > http://docs.geoserver.org/stable/en/user/tutorials/imagepyramid/imagepyramid > .html (which doesn´t really document the properties of the image pyramid > data set). I use GeoServer 2.1.4 on Windows Server 2008 R2 Standard, 64 > bit. I have added a zip file containing: > > - Sample data from level 7 of the pyramid (TIF and SHP) > - 2010_pyramid.prj > - 2010_pyramid.properties > http://osgeo-org.1560.n6.nabble.com/file/n4983886/2010_pyramid.zip > 2010_pyramid.zip In each of the subfolders you also need a file called 2010_pyramid.prj and 2010_pyramid.properties. The 2010_pyramid.prj should be a copy of the one in the root directory. The 2010_pyramid.properties should look something like: Name=2010_pyramid LocationAttribute=location LevelsNum=1 Envelope2D= Levels= I haven't found any documentation but this is what geoserver generates when it successfully generates a pyramid and trial and error shows it works. >From experience, the Envelope2D in the subdirectory files must match Envelope2D in the root directory. The Levels in the subdirectory file does not need to match the respective value for Levels in the root directory (I surmise Levels in the subdirectory should match the resolution of the TIFFs while Levels in the root directory is used to select which subdirectory to use). Hope this helps Gordon -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] How to build the indexes of an image-pyramid
On Wed, 20 Jun 2012 10:18:06 PM Casper Børgesen wrote: > As I understand you use gdaltindex to create the tile index for each level. > Further more you create the properties file at the root of the pyramid. I create a properties file at the root and a properties file for each level. The below description is based on trial and error getting things to work. As described it works for me. I haven't looked at the geoserver code at all. Root directory The root properties file specifies the number of levels, the name of the directory containing the level and scale of that level. It also contains the name of the layer and the bounding envelope. In my case (and I think yours) the directory names happen to be numbers. The LevelDirs and Levels properties must have (at least) the number of entries specified by the LevelsNum property. Levels contains a comma separated pairs of numbers giving the x and y size of each pixel in projected units. The root level also contains a .prj file specifying the projection of the data. For example I have a ImagePyramid layer called bathy_pyramid: >more bathy_pyramid.properties Name=bathy_pyramid LevelsNum=10 LevelsDirs=1 2 4 8 16 32 64 128 256 512 Envelope2D=109.8395984985,-64.001972 173.996985,0.1604010028 Levels=2.44618395E-4,2.44618395E-4 4.89236791E-4,4.89236791E-4 9.78473581E-4,9.78473581E-4 0.00195694716,0.00195694716 0.00391389432,0.00391389432 0.00782778865,0.00782778865 0.0156555773,0.0156555773 0.0313111546,0.0313111546 0.0626223092,0.0626223092 0.125244618,0.125244618 >more bathy_pyramid.prj GEOGCS["WGS 84", DATUM["World Geodetic System 1984", SPHEROID["WGS 84", 6378137.0, 298.257223563, AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]], PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]], UNIT["degree", 0.017453292519943295], AXIS["Geodetic longitude", EAST], AXIS["Geodetic latitude", NORTH], AUTHORITY["EPSG","4326"]] Level directory gdaltindex creates the tile index as a shape file (.shp .shx, .dbf). The shape file has the same name as the layer. The properties file for a level specifies only one level and specifies the attribute of the shape file instead of directories (as specified in gdaltindex - default is location). The Name and Envelope2D attributes of the level properties file must match the root properties file. The level also has a projection file which I suspect must be identical to the root one. >ls 512 bathy_110_174_-64_0_bathy.tif bathy_pyramid.dbf bathy_pyramid.prj bathy_pyramid.properties bathy_pyramid.shp bathy_pyramid.shx >more 512/bathy_pyramid.properties Name=bathy_pyramid LocationAttribute=location LevelsNum=1 Envelope2D=109.8395984985,-64.001972 173.996985,0.1604010028 Levels=0.125245,0.125245 > If I try to update an existing ImagePyramid store, restart Geoserver, it > seems like Geoserver is updating all the index files anyway. My experience is that if Geoserver doesn't like the properties file for an ImagePyramid for any reason it will try and generates its own (often unsuccessfully if its assumption aren't met). (Worst case the properties file it generates is invalid and thereafter any GetCapabilities request will fail. This is particularly problematic as this problem can be triggered by a single bad image in the pyramid being read, which in a large pyramid with some uninteresting data it may be some weeks after you make any changes before a user requests that area/scale and breaks the whole server). Good luck Gordon -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] How to build the indexes of an image-pyramid
On Mon, 18 Jun 2012 05:56:11 PM Casper Børgesen wrote: > I have experimented with creating the data for an image pyramid. I have > created a set of files in a folder structure like in the following example: > > Pyramid: > 0: > 12: > GeoTiffs > 13: > GeoTiffs > 14: > GeoTiffs > 1: > 6: > GeoTiffs > 7: > GeoTiffs > 2: > 3: > GeoTiffs > > If I try to create an image pyramid through Geoserver using the ImagePyramid > plugin, it creates a pyramid from level 0 only and returns without errors. > The shape file from level 0 seems to include all the Geotiffs on that > level. I would expect however, that the pyramid included level 1 and 2 > also, but it doesn’t. > > It seems as it doesn’t see the two other levels, so is there a file name > convention for the Geotiffs when using Geoserver to build the pyramid? > Currently my naming scheme is __.tif > > And is there another way to build the pyramid without using Geoserver to > build the index files? I have a rather large set of files and I don’t want > the geoserver to use resources on creating the indexes if I can do it on a > workstation instead. I've attached the script I use to build my image pyramids. The script is designed to update existing pyramids by copying any new geotiffs from the work area where I build them but it will also build the pyramid from scratch. My pyramids have a directory structure with the pyramid name as the base directory name then a bunch of subdirectories having geotiffs. There's lots of stuff you probably don't need but the core bits are there. Hope this helps Gordon update_pyramid Description: application/shellscript -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Image pyramid with sparse tiles gives black boxes.
2.1.x nightly works correctly. I'm now getting other errors that I didn't get with 1.7.6, but they all appear to be legitimate errors and 1.7.6 was too forgiving. (a small number of two band images appearing in a four band pyramid. Thanks Gordon On Sat, 10 Sep 2011 01:48:54 AM Simone Giannecchini wrote: > Ciao Gordon, > please try to upgrade to 2.1.x nightly and report to us if things > improve or not. > > Regards, > Simone Giannecchini > --- > Ing. Simone Giannecchini > GeoSolutions S.A.S. > Founder > > Via Poggio alle Viti 1187 > 55054 Massarosa (LU) > Italy > > phone: +39 0584 962313 > fax: +39 0584 962313 > mob:+39 333 8128928 > > http://www.geo-solutions.it > http://geo-solutions.blogspot.com/ > http://www.youtube.com/user/GeoSolutionsIT > http://www.linkedin.com/in/simonegiannecchini > http://twitter.com/simogeo > > ------- > > On Fri, Sep 9, 2011 at 1:12 AM, Gordon Keith wrote: > > http://jira.codehaus.org/browse/GEOS-4759 > > > > Regards > > Gordon > > > > On Thu, 8 Sep 2011 10:42:48 PM Andrea Aime wrote: > >> On Thu, Sep 8, 2011 at 7:36 AM, Gordon Keith wrote: > >> > Geoserver 2.1.0 and 2.1.1 > >> > > >> > I have an image pyramid build from a number of geotiffs where there > >> > are large gaps in the dataset. > >> > > >> > If I make a GetMap request with a bounding box that doesn't include > >> > any of the geotiffs I get back a black box. > >> > > >> > In Geoserver 1.7.6 I fixed this by setting OutputTransparentColour to > >> > 00. > >> > > >> > With 2.1.x setting OutputTransparentColour to 00 gives me either: > >> > > >> > code="internalError" Rendering process failed > >> > Unsupported data type. > >> > > >> > or > >> > > >> > Error rendering coverage on the fast path > >> > java.lang.IllegalStateException: > >> > Unsupported data type. > >> > Unsupported data type. > >> > > >> > depending on whether -Dorg.geoserver.render.raster.direct.disable=true > >> > is set. > >> > > >> > How can I set no data areas to transparent in Geoserver 2.1.x? > >> > >> Setting the background transparent color just like you did. > >> Please share some bits of that pyramid on a jira bug report so that we > >> can look at what's going on > >> > >> Cheers > >> Andrea > > > > -- > > > > Gordon Keith > > Programmer/Data Analyst > > Marine Acoustics > > CSIRO Marine and Atmospheric Research > > http://www.cmar.csiro.au > > > > The command that Christ has given us is this: > > whoever loves God must love his brother also. > > -- 1 John 4:21 > > > > ----- > >- Doing More with Less: The Next Generation Virtual Desktop > > What are the key obstacles that have prevented many mid-market businesses > > from deploying virtual desktops? How do next-generation virtual > > desktops provide companies an easier-to-deploy, easier-to-manage and more > > affordable virtual desktop > > model.http://www.accelacomm.com/jaw/sfnl/114/51426474/ > > ___ > > Geoserver-users mailing list > > Geoserver-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/geoserver-users > -- Gordon Keith Programmer/Data Analyst Marine Acoustics CSIRO Marine and Atmospheric Research http://www.cmar.csiro.au Hardware n, "The parts of a computer system that can be kicked." -Henri Karrenbeld -- Doing More with Less: The Next Generation Virtual Desktop What are the key obstacles that have prevented many mid-market businesses from deploying virtual desktops? How do next-generation virtual desktops provide companies an easier-to-deploy, easier-to-manage and more affordable virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/ ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
Re: [Geoserver-users] Image pyramid with sparse tiles gives black boxes.
http://jira.codehaus.org/browse/GEOS-4759 Regards Gordon On Thu, 8 Sep 2011 10:42:48 PM Andrea Aime wrote: > On Thu, Sep 8, 2011 at 7:36 AM, Gordon Keith wrote: > > Geoserver 2.1.0 and 2.1.1 > > > > I have an image pyramid build from a number of geotiffs where there are > > large gaps in the dataset. > > > > If I make a GetMap request with a bounding box that doesn't include any > > of the geotiffs I get back a black box. > > > > In Geoserver 1.7.6 I fixed this by setting OutputTransparentColour to > > 00. > > > > With 2.1.x setting OutputTransparentColour to 00 gives me either: > > > > code="internalError" Rendering process failed > > Unsupported data type. > > > > or > > > > Error rendering coverage on the fast path > > java.lang.IllegalStateException: > > Unsupported data type. > > Unsupported data type. > > > > depending on whether -Dorg.geoserver.render.raster.direct.disable=true is > > set. > > > > How can I set no data areas to transparent in Geoserver 2.1.x? > > Setting the background transparent color just like you did. > Please share some bits of that pyramid on a jira bug report so that we can > look at what's going on > > Cheers > Andrea > -- Gordon Keith Programmer/Data Analyst Marine Acoustics CSIRO Marine and Atmospheric Research http://www.cmar.csiro.au The command that Christ has given us is this: whoever loves God must love his brother also. -- 1 John 4:21 -- Doing More with Less: The Next Generation Virtual Desktop What are the key obstacles that have prevented many mid-market businesses from deploying virtual desktops? How do next-generation virtual desktops provide companies an easier-to-deploy, easier-to-manage and more affordable virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/ ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
[Geoserver-users] Image pyramid with sparse tiles gives black boxes.
Geoserver 2.1.0 and 2.1.1 I have an image pyramid build from a number of geotiffs where there are large gaps in the dataset. If I make a GetMap request with a bounding box that doesn't include any of the geotiffs I get back a black box. In Geoserver 1.7.6 I fixed this by setting OutputTransparentColour to 00. With 2.1.x setting OutputTransparentColour to 00 gives me either: code="internalError" Rendering process failed Unsupported data type. or Error rendering coverage on the fast path java.lang.IllegalStateException: Unsupported data type. Unsupported data type. depending on whether -Dorg.geoserver.render.raster.direct.disable=true is set. How can I set no data areas to transparent in Geoserver 2.1.x? Regards Gordon -- Gordon Keith Programmer/Data Analyst Marine Acoustics CSIRO Marine and Atmospheric Research http://www.cmar.csiro.au Computers are like air conditioners. Both stop working, if you open windows. -- Doing More with Less: The Next Generation Virtual Desktop What are the key obstacles that have prevented many mid-market businesses from deploying virtual desktops? How do next-generation virtual desktops provide companies an easier-to-deploy, easier-to-manage and more affordable virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/ ___ Geoserver-users mailing list Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
