Jim, you would think that ;) However, in practice I wouldn't expect
the disk access time for geotiffs to be significantly different from
jpeg if you have properly optimized your geotiffs using gdal_translate
-co "TILED=YES" - the internal structure is efficiently indexed so
that gdal only has to read the minimum number of 256x256 blocks to
cover the requested extent. And using gdaladdo to generate overviews
just makes it that much more efficient.
Even if you are reading less physical data from the disk to get the
equivalent coverage from jpeg, the decompression overhead is enough to
negate the difference in IO time based on Ed's oft quoted advice (and
other's experience too I think). The rules that apply in this case
seem to be 'tile your data', 'do not compress it' and 'buy the fastest
disk you can afford'.
Compression is useful and probably necessary if you hit disk space
limits.
Cheers
Paul
On 15-Sep-08, at 5:48 PM, Jim Klassen wrote:
Just out of curiosity, has anyone tested the performance of Jpegs
vs. GeoTiffs?
I would expect at some point the additional disk access time
required for GeoTiffs (of the same pixel count) as Jpegs would
outweigh the additional processor time required to decompress the
Jpegs. (Also the number of Jpegs that can fit in disk cache is
greater than for similar GeoTiffs.)
For reference we use 1000px by 1000px Jpeg tiles (with world files).
We store multiple resolutions of the dataset, each in its own
directory. We start at the native dataset resolution, and half that
for each step, stopping when there are less than 10 tiles produced
at that particular resolution. (I.e for one of our county wide
datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then
created for each resolution (using gdaltindex followed by shptree)
and a layer is created in the mapfile for each tileindex and
appropriate min/maxscales are set. The outputformat in the mapfile
is set to jpeg.
Our typical tile size is 200KB. There are about 20k tiles in the 6in/
px dataset, 80k tiles in the 3in/px dataset (actually 4in data, but
stored in 3in so it fits with the rest of the datasets well). I have
tested and this large number of files in a directory doesn't seem to
effect performance on our system.
Average access time for a 500x500px request to mapserver is 300ms
measured at the client using perl/LWP and about 220ms with shp2img.
Machine is mapserver 5.2.0/x86-64/2.8GHz Xeon/Linux 2.6.16/ext3
filesystem.
Jim Klassen
City of Saint Paul
"Fawcett, David" <[EMAIL PROTECTED]> 09/15/08 1:10 PM >>>
Better yet,
Add your comments to:
http://mapserver.gis.umn.edu/docs/howto/optimizeraster
and
http://mapserver.gis.umn.edu/docs/howto/optimizevector
I had always thought that all we needed to do to make these pages
great
was to grok the list for all of Ed's posts...
David.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brent
Fraser
Sent: Monday, September 15, 2008 12:55 PM
To: mapserver-users@lists.osgeo.org
Subject: [mapserver-users] Ed's Rules for the Best Raster Performance
In honor of Ed's imminent retirement from the Mapserver Support Group,
I've put together "Ed's List for the Best Raster Performance":
#1. Pyramid the data
- use MAXSCALE and MINSCALE in the LAYER object.
#2. Tile the data (and merge your upper levels of the pyramid for
fewer
files).
- see the TILEINDEX object
#3. Don't compress your data
- avoid jpg, ecw, and mrsid formats.
#4. Don't re-project your data on-the-fly.
#5. Get the fastest disks you can afford.
(Ed, feel free to edit...)
Brent Fraser
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
__________________________________________
Paul Spencer
Chief Technology Officer
DM Solutions Group Inc
http://www.dmsolutions.ca/
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
