Hi,
I have large amount of vector maps (~ 400) which each has hundreds of
thousands of areas and over 150 attribute columns. I'm looking for the
most efficient way to export these to a format I can share with
non-GRASS GIS users. My favourite would probably be Geopackage, amongst
others because the attribute column names are sometimes fairly long,
thus making export to a DBF based format not easily feasible.
I've seen several sources of information about possible performance
increases, either in the SQLite world (as a Geopackage is based on
SQLite) or in the OGR SQLite driver page.
For OGR [1], there is the -gt option to ogr2ogr which allows increasing
the number of features per transaction (or even running the entire
conversion in one single transaction). There is also mention of the
OGR_SQLITE_CACHE and OGR_SQLITE_SYNCHRONOUS configuration options.
However, I don't seem to understand if, and if yes, how I can set any of
these when using v.out.ogr. In r73117, MarkusM added config and dataset
open options to r.in.gdal and v.in.ogr. Would we need something like
this to be able to set the above config options ? How about the
transaction option ?
On the SQLite side, there are some pragma statements that allow
increasing performance (generally at the cost of less data security, but
in some cases this might be acceptable), such as journal_mode [2] or
synchronous [3] (which I suppose is what OGR_SQLITE_SYNCHRONOUS sets as
well), or possibly locking_mode (I don't know how this is handled
internally in OGR).
Again, my question would be how these could be set when exporting data
with v.out.ogr.
Other interesting options (which would also influence performance)
might be a flag in v.out.ogr allowing exporting without attribute data
(although a simple v.db.connect -d before export would probably be
enough) and allowing selecting a subset of columns to export. But that's
something else.
Moritz
[1] https://gdal.org/drv_sqlite.html
[2] https://sqlite.org/pragma.html#pragma_journal_mode
[3] https://sqlite.org/pragma.html#pragma_synchronous
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
