On 11/13/07, Frank Warmerdam <[EMAIL PROTECTED]> wrote: > P Kishor wrote: > > So, I am thinking, Shapefile is the de facto data standard for GIS > > data. That it is open (albeit not Free) along with the deep and wide > > presence of ESRI's products from the beginning of the epoch, it has > > been widely adopted. Existence of shapelib, various language bindings, > > and ready use by products such as MapServer has continued to cement > > Shapefile as the format to use. All this is in spite of Shapefile's > > inherent drawbacks, particularly in the area of attribute data > > management. > > > > What if we came up with a new and improved data format -- call it > > "Open Shapefile" (extension .osh) -- that would be completely Free, > > single-file based (instead of the multiple .shp, .dbf, .shx, etc.), > > and based on SQLite, giving the .osh format complete relational data > > handling capabilities. We would require a new version of Shapelib, > > improved language bindings, make it the default and preferred format > > for MapServer, and provide seamless and painless import of regular > > .shp data into .osh for native rendering. Its adoption would be quick > > in the open source community. The non-opensource community would > > either not give a rat's behind for it, but it wouldn't affect them... > > they would still work with their preferred .shp until they learned > > better. By having a completely open and Free single-file based, built > > on SQLite, fully relational dbms capable spatial data format, it would > > be positioned for continued improvement and development. > > Puneet, > > I've had a similar idea kicking around in my head for a while, but I think > of it as "open geodatabase". I see the goals as providing a similar role > to the "personal geodatabase", including:
I should mention that over on the SQLite list every once a eon someone asks the question about spatlializing SQLite a la PostGIS. That definitely could be one way to go as I have a little experience in this. Last year I had the opportunity to tackle a point-in-polygon overlay problem that was seeming to be intractable for ArcGIS. So, I dumped the data into Shapefile, unpacked the coordinates from the Shapefile and stuffed them into SQLite, then used the bounding box method to narrow my searches. With the clever use of indexes, and a bunch of optimizations, I basically wrote a fast and functional overlay program with Perl/DBI. The overlay task now takes 2 days, which is a huge improvement from the earlier 7-8 days that it used to be. One thing this new "geo database" should not be is that like SQLite it should not require a server. As wonderful as PostGIS is, installing and managing PostGres is a major obstacle to its use. I use SQLite for pretty much everything because of the ease with which I can get started with it. It takes me longer to download it than to start working with it on a new machine. As Larry Wall likes to say, "you can write faster programs in C, but you can program faster in Perl." It is kinda like that... on the one hand, highlighting the "database-ness" of the new format would be a good and powerful thing, but on the other hand, it might lead folks to think that a db server is required. Having a server-less, self-contained, rdbms-capable format would be the key. > > o RDBMS style operations like SQL filtering, joins, etc. > o Get past all the shapefile limitations related to the .dbf format (very > restricted data types, short attribute names, lots of other limits) > o Allow storing many layers in one file. > o Built in spatial indexing and attribute indexing. > o OGC style coordinate system and geometry support. > > I have had some hope that the existing SDF format supported by FDO would > be this new format; however, SDF is quite a complicated format, and the > only available open source implementation is quite heavily tied to FDO. > Once you carry along FDO the whole thing becomes fairly heavy in terms of > the amount of code required, and the interface complexity. But (I think) > it satisfies most of my goals and already exists. > > I do feel that we need to be cautious before launching "yet another format". > I'm also a bit dubious about some aspects of sqlite as a native data store. > In particular, it's typeless "everything is a string" approach strikes me > as potentially being a problem. It also remains to be seen whether we could > build fast spatial indexing directly in, though I suppose with a fat enough > middleware layer it could be done. > > PS. I'm still doubtful it would be faster than shapefiles+qix for most web > mapping needs. We will never know until it is done. And, isn't premature optimization the root of all evil? > > Best regards, > -- > ---------------------------------------+-------------------------------------- > I set the clouds in motion - turn up | Frank Warmerdam, [EMAIL PROTECTED] > light and sound - activate the windows | http://pobox.com/~warmerdam > and watch the world go round - Rush | President OSGeo, http://osgeo.org > > _______________________________________________ Discuss mailing list Discuss@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss