Jim, you would think that ;) However, in practice I wouldn't expect the disk access time for geotiffs to be significantly different from jpeg if you have properly optimized your geotiffs using gdal_translate -co "TILED=YES" - the internal structure is efficiently indexed so that gdal only has to read the minimum number of 256x256 blocks to cover the requested extent. And using gdaladdo to generate overviews just makes it that much more efficient.

Even if you are reading less physical data from the disk to get the equivalent coverage from jpeg, the decompression overhead is enough to negate the difference in IO time based on Ed's oft quoted advice (and other's experience too I think). The rules that apply in this case seem to be 'tile your data', 'do not compress it' and 'buy the fastest disk you can afford'.

Compression is useful and probably necessary if you hit disk space limits.

Cheers

Paul

On 15-Sep-08, at 5:48 PM, Jim Klassen wrote:

Just out of curiosity, has anyone tested the performance of Jpegs vs. GeoTiffs?

I would expect at some point the additional disk access time required for GeoTiffs (of the same pixel count) as Jpegs would outweigh the additional processor time required to decompress the Jpegs. (Also the number of Jpegs that can fit in disk cache is greater than for similar GeoTiffs.)

For reference we use 1000px by 1000px Jpeg tiles (with world files). We store multiple resolutions of the dataset, each in its own directory. We start at the native dataset resolution, and half that for each step, stopping when there are less than 10 tiles produced at that particular resolution. (I.e for one of our county wide datasets 6in/px, 1ft/px, 2ft/px, ... 32ft/px). A tileindex is then created for each resolution (using gdaltindex followed by shptree) and a layer is created in the mapfile for each tileindex and appropriate min/maxscales are set. The outputformat in the mapfile is set to jpeg.

Our typical tile size is 200KB. There are about 20k tiles in the 6in/ px dataset, 80k tiles in the 3in/px dataset (actually 4in data, but stored in 3in so it fits with the rest of the datasets well). I have tested and this large number of files in a directory doesn't seem to effect performance on our system.

Average access time for a 500x500px request to mapserver is 300ms measured at the client using perl/LWP and about 220ms with shp2img.

Machine is mapserver 5.2.0/x86-64/2.8GHz Xeon/Linux 2.6.16/ext3 filesystem.

Jim Klassen
City of Saint Paul

"Fawcett, David" <[EMAIL PROTECTED]> 09/15/08 1:10 PM >>>
Better yet,

Add your comments to:

http://mapserver.gis.umn.edu/docs/howto/optimizeraster

and

http://mapserver.gis.umn.edu/docs/howto/optimizevector

I had always thought that all we needed to do to make these pages great
was to grok the list for all of Ed's posts...

David.

-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Brent
Fraser
Sent: Monday, September 15, 2008 12:55 PM
To: mapserver-users@lists.osgeo.org
Subject: [mapserver-users] Ed's Rules for the Best Raster Performance


In honor of Ed's imminent retirement from the Mapserver Support Group,
I've put together "Ed's List for the Best Raster Performance":


#1. Pyramid the data
       - use MAXSCALE and MINSCALE in the LAYER object.

#2. Tile the data (and merge your upper levels of the pyramid for fewer
files).
       - see the TILEINDEX object

#3. Don't compress your data
       - avoid jpg, ecw, and mrsid formats.

#4. Don't re-project your data on-the-fly.

#5. Get the fastest disks you can afford.


(Ed, feel free to edit...)

Brent Fraser
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users


__________________________________________

   Paul Spencer
   Chief Technology Officer
   DM Solutions Group Inc
   http://www.dmsolutions.ca/

_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Reply via email to