Hi Mike,
thanks for the report. Issue fixed in master. I've refreshed the Docker
image
Note however that this Overture dataset contains 1,390,499,721 features,
and with our estimate of 83 bytes / feature to build the FlatGeobuf
RTree, that means you need at least 115 GB RAM on your machine
(presumably more due to fragmentation etc)
You could potentially save a few minutes by not adding -progress since
GetFeatureCount() on such dataset requires getting some bits in each of
the ~1600 files, but I guess that for a process that's going to take a
few hours, having some display progress justifies a few extra minutes :-)
Even
Le 01/11/2023 à 11:18, Michael Smith a écrit :
Using the ghcr.io/osgeo/gdal:ubuntu-full-latest docker image, I get a
segfault when trying to convert overture maps to flatgeobuf
ogr2ogr -f flatgeobuf /data/overturemaps_2023_10_19.fgb
/vsis3/overturemaps-us-west-2/release/2023-10-19-alpha.0/theme=buildings/
theme=buildings -progress
…
FlatGeobuf: Spatial index requested will write to temp file and do
second pass on close
Segmentation fault (core dumped)
But no core file.
--
Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers
*From: *gdal-dev <gdal-dev-boun...@lists.osgeo.org> on behalf of Even
Rouault via gdal-dev <gdal-dev@lists.osgeo.org>
*Reply-To: *Even Rouault <even.roua...@spatialys.com>
*Date: *Tuesday, October 31, 2023 at 1:14 PM
*To: *Rahkonen Jukka <jukka.rahko...@maanmittauslaitos.fi>,
"gdal-dev@lists.osgeo.org" <gdal-dev@lists.osgeo.org>
*Subject: *Re: [gdal-dev] GDAL 3.8.0beta1 available for testing
Le 31/10/2023 à 17:36, Rahkonen Jukka a écrit :
Hi,
I made a simple test with ogr2ogr and geopackage to geopackage on Windows.
With GDAL 3.8.0dev-3e4dc710a2 (no arrow, old R-Tree) the timing was 36 minutes,
with GDAL 3.8.0dev-6bbd2c080a the same conversion took 21 minutes. The gpkg
file is 75 GB, with 120 layers.
Could it be that the new R-Tree implementation does not print anything into
debug?
Indeed. Debug traces added back in https://github.com/OSGeo/gdal/pull/8640
Even
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________ gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
