Darren, These limitations are a result of the shapefile reader/writer implementation. While the specification permits up to 8GB, the implementation might not use it fully.
According to the specification a feature offset is stored as a 32-bit integer to a 16-bit word. So, that's (2^32)*16 bits or (2^32)*2 bytes. To fully utilize this, we need to use unsigned 32-bit integers and check for overflows. Originally, the shapelib library used by GDAL had the same limit as ESRI's. If you are reaching the file size limits, I suggest you look into some other file format to store your data. SQLite/SpatiaLite is a good option. A hacky way to split the features across multiple shapefiles is to use the -where option in ogr2ogr. You can filter based on the FID values. On Mon, Nov 25, 2013 at 4:12 PM, CARMAN, Darren < darren.car...@astrium.eads.net> wrote: > Hi List > > > > I notice on the OGR formats page for ESRI Shapefile the following is > mentioned: > > ================ > > Size Issues > > Geometry: The Shapefile format explicitly uses 32bit offsets and so cannot > go over 8GB (it actually uses 32bit offsets to 16bit words). Hence, it is > is not recommended to use a file size over 4GB. > > Attributes: The dbf format does not have any offsets in it, so it can be > arbitrarily large. > > ================ > > > > Yet on the ESRI website: > > ================ > > Geometry limitations > > There is a 2 GB size limit for any shapefile component file, which > translates to a maximum of roughly 70 million point features. The actual > number of line or polygon features you can store in a shapefile depends on > the number of vertices in each line or polygon (a vertex is equivalent to a > point). > > ================ > > > > I assume the OGR web page is wrong, or has a different meaning outside of > ESRI S/W use. > > > > I notice that the PostGIS utility pgsql2shp stops processing with an error > once the dbf file goes over 2GB. Is there any way to get ogr2ogr to do this? > > > > Ideally, something I can’t find a lot of information about online, is > there a way to get ogr2ogr to start writing to a new shapefile at a certain > processing point (number of objects or file size)? > > > > Alternatively, assuming the OGR website text is correct for files it > creates, is there any utility that could be used to split the created > shapefile files into ones with a size less than 2GB? > > > > Thanks in advance > > Darren > > > > > > > > *Darren Carman* > Senior Software Engineer > > GEO-Information Services > > > > *Astrium Services* > > Tel +44 (0)1252 362138 > > http://www.astrium-geo.com <http://www.infoterra.co.uk> > > > > > > Infoterra Ltd. Is part of the Astrium GEO-Information Services Division > and a wholly owned subsidiary of Astrium, Europe's leading space systems > and services specialist. > > Disclaimer. The information contained in this e-mail and its attachments > are confidential and intended only for the use of the named addressee(s). > If you are not the intended addressee, please do not read, copy, use or > disclose this message or its attachments. If you have received this message > in error, please notify the sender immediately and delete or destroy all > copies of this message and attachments in all media. Any views or opinions > expressed are solely those of the author and do not necessarily represent > those of Infoterra Ltd and shall not form part of any binding agreement. > > Infoterra Limited a company registered in England under number 2359955 > and having its registered office at Europa House, Southwood Crescent, > Farnborough, GU14 0NL. VAT number GB 476 0468 27. > > P Before printing, think about the environment > > > > > > > > > _______________________________________________ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > http://lists.osgeo.org/mailman/listinfo/gdal-dev > -- Best regards, Chaitanya kumar CH. +91-9494447584 17.2416N 80.1426E
_______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/gdal-dev