On 6/10/2013 12:57 PM, Evans, James R Civ USAF ACC 84 RADES/SCZE wrote:
Hi Stephen,
Thanks, for the reply. I previously got 4 sample images from the USDA, and
was able to get them to work just fine. There was no processing required.
The sample images I got were all from Utah, and they are NAD83, UTM zone 12.
I added the 4 sample images to a shape file using gdaltindex. I used UPSG
26912 and mapserver served them up very quickly for such large files.
Now I have this entire data set, and it stretches from UTM zone 10, to UTM
zone 19. The data is divided into directories by two letter state
abbreviations, and under that into subdirectories. I'm just wondering how
to add this to my mapfile. Do I need a different entry for each UTM Zone?
How is it possible to get a single layer entry that includes multiple
projections? This is looking like a huge job and I just want to know the
best strategy for getting this done.
So now you have a problem. You data is in UTM spread over 10 different
projections. What do you plane to do when have your image is zone 10 and
half is in zone 11 or if you zoom out and you images has 3-4 zones
displayed?
All data in an image must be rendered in the same projection. While I
don't doubt that your test with 4 images worked fine, did you you test
this a multiple zoom levels and at some point you will probably want to
create a super overlay image so you do not have to open multiple files
to just pull a tiny overlay out of each one.
Your use cases will determine how you want to deal with the data. For
example does it HAVE to be in a UTM projection, or can you work with a
Spherical Mercator or geographic projection? The end solution will be
much easier if you can work with one common projection over your whole
data set. Otherwise, you will have to deal with the transitions from one
zone to the next or maybe set up 10 separate servers that only serve one
zone.
Having pushed larges amounts (4-25TB) of imagery data more than once it
is important to make these decisions up front and and prototype up
something like a 4-10 degree square across a UTM boundary and make sure
that the results are going to be what you expect before you process all
the data.
-Steve
Thanks,
James
-----Original Message-----
From: mapserver-users-boun...@lists.osgeo.org
[mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Stephen
Woodbridge
Sent: Friday, June 07, 2013 8:41 AM
To: mapserver-users@lists.osgeo.org
Subject: Re: [mapserver-users] Best way to import 4.5TB of imagery?
On 6/7/2013 10:31 AM, James_in_Utah wrote:
Hi,
We just got 3 hard drive, loaded with 4.5TB of NAIP imagery for all of
CONUS. I think there's a total of about 400,000 jpgs. The data is in
directories, by states. Under each state, there are subfolders,
probably reference by longitude. Other than going through folder by
folder, adding each image to a shape file using gdaltindex, what's the
best strategy for loading a couple of hundred thousand files up to our
server and making the imagery available via our mapserver? Should I
maintain the current directory structure when I copy the imagery to
the server, or just dump all of it into a single directory? Do I want
to stay with 1 shape file, or break it up by state? We eventually
want a contiguous layer for all of CONUS to be served up to our users.
James,
Since imagery data is served via gdal, you might want to also ask this
question on the gdal list.
There are issues with jpg related to the fact that if you only want a small
part of the image you still have to uncompress the whole image. So part of
the answer might be that you need to pre-process all the imagery into
something like a jpg compress tiled geotif or something else.
You also need to consider what projection your imagery is in and what
projection you want to display it in. Because if you need to preprocess the
data, that would also be a good time to reproject it.
Anyway the gdal list can probably ask additional questions to help sort all
that out.
-Steve W
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
