On 1/29/2014 1:29 PM, Rahkonen Jukka (Tike) wrote:
Oh, I thought you had something interesting :)
If that little shapefile was slow you for sure had no qix index on that. One
shapefile will be good for you but ogrtileindex is about as good alternative
and if you have faulty data in some of the 6000 original shapefiles you can
correct just those. I believe that Spatialite db with spatial index and
vacuumed would be as fast or faster.
-Jukka Rahkonen-
As Jukka and others have commented it sounds like you might be missing
your spatial index *.qix file when you test your all in one shapefile.
256 MB shapefile is not that big unless you are trying to load the whole
thing to display the contours for the whole world. If the later is the
case then you probably need to rethink loading all 256 MB of data
because it my guess is that it is not all useful at that resolution.
If you are filtering only some countours using and expression, then you
would be better off separating them into another shapefile and adding a
another layer.
-Steve
EVANS, JAMES R GS-13 USAF ACC 84 RADES/SCZE wrote:
I was off on the size of the shape file. It's 265MB, not GB. Doh! Anyway,
the performance when processing that files was still surprisingly bad. Quad
core server and all 4 core's were maxed out for about 5 minutes after just
loading that layer on a client. I am running Server 2003, and only have 4GB
ram on that server. We are looking at replacing it.
James
-----Original Message-----
From: mapserver-users-boun...@lists.osgeo.org
[mailto:mapserver-users-boun...@lists.osgeo.org] On Behalf Of Even Rouault
Sent: Wednesday, January 29, 2014 11:09 AM
To: mapserver-users@lists.osgeo.org; Brent Wood
Subject: Re: [mapserver-users] Layer with over 6000 shapefiles
Le mercredi 29 janvier 2014 18:40:49, Brent Wood a écrit :
This is less a mapserver issue than a data management issue - and
shapefiles are not a particularly good way to manage data.
I suggest you look at a spatially enabled database, with attribute &
spatial indexes, such as Postgis, instead of shapefiles as a data source.
These are pretty much designed for such use cases.
That said, if you cannot use a spatial database in this case, you can
do things with mapserver to improve things - like have multiple layers
(though 6000 seems a bit excessive) in a single group - so addressable
as a single layer but each with its own shapefile spatial index file.
A spatialite DB with spatial index could also perhaps do it. Not sure if it
scales well up to 260 GB however.
Brent Wood
________________________________
From: "EVANS, JAMES R GS-13 USAF ACC 84 RADES/SCZE"
<james.evan...@us.af.mil> To: "mapserver-users@lists.osgeo.org"
<mapserver-users@lists.osgeo.org> Sent: Thursday, January 30, 2014
5:32 AM
Subject: [mapserver-users] Layer with over 6000 shapefiles
Hi,
Is there a way to add a layer that points to 6000 some odd shapefiles?
I've created a set of contour files for the whole world, and when I
had the info in a single shapefile it was about 260GBs. Mapserver
would serve it up, but performance was horrible. I now have the set
broken up into about 6000 separate shapefiles, but don't know how to
address that in a single layer. I tried using gdaltindex, to create a
single index file that points to all the other shapefiles, but I get
an error saying the shapefiles are an unsupported type. Is there some
other way to create an index that Mapserver can use to point to this
many shapefiles? Thanks, James
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
--
Geospatial professional services
http://even.rouault.free.fr/services.html
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
_______________________________________________
mapserver-users mailing list
mapserver-users@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
