Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Le vendredi 15 janvier 2016 11:55:07, kimaidou a écrit :
> Hi all
>
> I have just tested QGIS master build againt GDAL "trunk" (build via a git
> clone from GDAL github repository & following the doc instructions [1] ).
> It works like a charm with a small roads dataset.
>
> Thanks Even (and other involved devs and funders ) for making this happen !
>
> I have seen one first caveat :
>
> * Export a Shapefile to GeoJSON in QGIS via the "Save as" and choose to
> keep only 1 number for decimal precision
> * Open this GeoJSON file, toggle edition, and add a new feature
> * The new feature geometries are written with default decimal precision
> (for example : 6790978.453365888446569 ) and not like the other original
> features ( for example : 6790978.4 )
>
> We should have a way in QGIS to keep the original decimal precision. Any
> clue on this ? Should I open a ticket ?
This is not something that could get fixed easily. When you open the geojson
file, there's nothing in it explicitly mentionning the decimal precision (and
it could be non uniform). There could be some logic to guess it, but that
wouldn't be trivial since json-c returns a IEEE-754 double and not the
original ASCII representation. A less user friendly but more reliable way
would be to have an open option / env. variable to define the wished precision
when you're in update mode.
>
> Regards
> Michaël
>
> [1] https://trac.osgeo.org/gdal/wiki/BuildingOnUnix
>
> 2016-01-14 20:57 GMT+01:00 Even Rouault :
> > Le jeudi 14 janvier 2016 20:38:04, Nyall Dawson a écrit :
> > > On 15 Jan 2016 2:39 AM, "Even Rouault"
> >
> > wrote:
> > > > A more "streaming-like" approach for the driver not proceeding to
> > > > full ingestion of features could be desirable to remove that
> > > > limitation, but
> > >
> > > that's
> > >
> > > > more involved. A possibility could be to have a "at-hand" parser to
> > >
> > > delimitate
> > >
> > > > JSon "Feature" objects and use only json-c to parse each feature. But
> > >
> > > GeoJSON
> > >
> > > > is certainly not the more appropriate file format to deal with huge
> > >
> > > datasets...
> > >
> > >
> > > Even,
> > >
> > > Editable GeoJSON in OGR is great news!
> > >
> > > I've got a question regarding the geojson driver you may be able to
> >
> > assist
> >
> > > with. Is there any method in the OGR libraries which allow direct
> > > parsing of a string to a layer? (Ie, without first writing it out to
> > > a file).
> > >
> > > I'd like to add the ability to directly paste geojson text into QGIS
> > > and have it inserted as a feature in the current layer (like how you
> > > can currently paste WKT text as a feature). I don't want to have to
> > > manually parse the json (that would be a nightmare).
> >
> > Well, the GeoJSON driver support "filenames" which are in fact GeoJSON
> > content:
> >
> > $ ogrinfo
> > '{"type":"Feature","properties":{"foo":"bar"},"geometry":{"type":"Point",
> > "coordinates":[2,49]}}' -ro -al
> >
> > Layer name: OGRGeoJSON
> > Geometry: Point
> > Feature Count: 1
> > Extent: (2.00, 49.00) - (2.00, 49.00)
> > Layer SRS WKT:
> > GEOGCS["WGS 84",
> >
> > DATUM["WGS_1984",
> >
> > SPHEROID["WGS 84",6378137,298.257223563,
> >
> > AUTHORITY["EPSG","7030"]],
> >
> > AUTHORITY["EPSG","6326"]],
> >
> > PRIMEM["Greenwich",0,
> >
> > AUTHORITY["EPSG","8901"]],
> >
> > UNIT["degree",0.0174532925199433,
> >
> > AUTHORITY["EPSG","9122"]],
> >
> > AUTHORITY["EPSG","4326"]]
> >
> > foo: String (0.0)
> > OGRFeature(OGRGeoJSON):0
> >
> > foo (String) = bar
> > POINT (2 49)
> >
> > Works also with FeatureCollections.
> > Alternatively, you could also put the content in a in-memory GDAL file
> > (/vsimem/ virtual file system) and open it.
> >
> > So you could likely instanciate a temporary layer and get a QGIS feature
> > from that.
> >
> >
> > If you are just interested in geometries and not attributes, there's also
> > the C function :
> >
> > OGRGeometryHOGR_G_CreateGeometryFromJson (const char *);
> >
> > > If I could somehow take advantage of OGR's geojson driver to do the
> > > heavy lifting then this work would be trivial.
> > >
> > > Any ideas?
> > >
> > > Nyall
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com
> > ___
> > Qgis-developer mailing list
> > Qgis-developer@lists.osgeo.org
> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Hi all
I have just tested QGIS master build againt GDAL "trunk" (build via a git
clone from GDAL github repository & following the doc instructions [1] ).
It works like a charm with a small roads dataset.
Thanks Even (and other involved devs and funders ) for making this happen !
I have seen one first caveat :
* Export a Shapefile to GeoJSON in QGIS via the "Save as" and choose to
keep only 1 number for decimal precision
* Open this GeoJSON file, toggle edition, and add a new feature
* The new feature geometries are written with default decimal precision
(for example : 6790978.453365888446569 ) and not like the other original
features ( for example : 6790978.4 )
We should have a way in QGIS to keep the original decimal precision. Any
clue on this ? Should I open a ticket ?
Regards
Michaël
[1] https://trac.osgeo.org/gdal/wiki/BuildingOnUnix
2016-01-14 20:57 GMT+01:00 Even Rouault :
> Le jeudi 14 janvier 2016 20:38:04, Nyall Dawson a écrit :
> > On 15 Jan 2016 2:39 AM, "Even Rouault"
> wrote:
> > > A more "streaming-like" approach for the driver not proceeding to full
> > > ingestion of features could be desirable to remove that limitation, but
> >
> > that's
> >
> > > more involved. A possibility could be to have a "at-hand" parser to
> >
> > delimitate
> >
> > > JSon "Feature" objects and use only json-c to parse each feature. But
> >
> > GeoJSON
> >
> > > is certainly not the more appropriate file format to deal with huge
> >
> > datasets...
> >
> >
> > Even,
> >
> > Editable GeoJSON in OGR is great news!
> >
> > I've got a question regarding the geojson driver you may be able to
> assist
> > with. Is there any method in the OGR libraries which allow direct parsing
> > of a string to a layer? (Ie, without first writing it out to a file).
> >
> > I'd like to add the ability to directly paste geojson text into QGIS and
> > have it inserted as a feature in the current layer (like how you can
> > currently paste WKT text as a feature). I don't want to have to manually
> > parse the json (that would be a nightmare).
>
> Well, the GeoJSON driver support "filenames" which are in fact GeoJSON
> content:
>
> $ ogrinfo
> '{"type":"Feature","properties":{"foo":"bar"},"geometry":{"type":"Point","coordinates":[2,49]}}'
> -ro -al
>
> Layer name: OGRGeoJSON
> Geometry: Point
> Feature Count: 1
> Extent: (2.00, 49.00) - (2.00, 49.00)
> Layer SRS WKT:
> GEOGCS["WGS 84",
> DATUM["WGS_1984",
> SPHEROID["WGS 84",6378137,298.257223563,
> AUTHORITY["EPSG","7030"]],
> AUTHORITY["EPSG","6326"]],
> PRIMEM["Greenwich",0,
> AUTHORITY["EPSG","8901"]],
> UNIT["degree",0.0174532925199433,
> AUTHORITY["EPSG","9122"]],
> AUTHORITY["EPSG","4326"]]
> foo: String (0.0)
> OGRFeature(OGRGeoJSON):0
> foo (String) = bar
> POINT (2 49)
>
> Works also with FeatureCollections.
> Alternatively, you could also put the content in a in-memory GDAL file
> (/vsimem/ virtual file system) and open it.
>
> So you could likely instanciate a temporary layer and get a QGIS feature
> from that.
>
>
> If you are just interested in geometries and not attributes, there's also
> the C function :
>
> OGRGeometryHOGR_G_CreateGeometryFromJson (const char *);
>
>
> >
> > If I could somehow take advantage of OGR's geojson driver to do the heavy
> > lifting then this work would be trivial.
> >
> > Any ideas?
> >
> > Nyall
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Le jeudi 14 janvier 2016 20:38:04, Nyall Dawson a écrit :
> On 15 Jan 2016 2:39 AM, "Even Rouault" wrote:
> > A more "streaming-like" approach for the driver not proceeding to full
> > ingestion of features could be desirable to remove that limitation, but
>
> that's
>
> > more involved. A possibility could be to have a "at-hand" parser to
>
> delimitate
>
> > JSon "Feature" objects and use only json-c to parse each feature. But
>
> GeoJSON
>
> > is certainly not the more appropriate file format to deal with huge
>
> datasets...
>
>
> Even,
>
> Editable GeoJSON in OGR is great news!
>
> I've got a question regarding the geojson driver you may be able to assist
> with. Is there any method in the OGR libraries which allow direct parsing
> of a string to a layer? (Ie, without first writing it out to a file).
>
> I'd like to add the ability to directly paste geojson text into QGIS and
> have it inserted as a feature in the current layer (like how you can
> currently paste WKT text as a feature). I don't want to have to manually
> parse the json (that would be a nightmare).
Well, the GeoJSON driver support "filenames" which are in fact GeoJSON content:
$ ogrinfo
'{"type":"Feature","properties":{"foo":"bar"},"geometry":{"type":"Point","coordinates":[2,49]}}'
-ro -al
Layer name: OGRGeoJSON
Geometry: Point
Feature Count: 1
Extent: (2.00, 49.00) - (2.00, 49.00)
Layer SRS WKT:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]]
foo: String (0.0)
OGRFeature(OGRGeoJSON):0
foo (String) = bar
POINT (2 49)
Works also with FeatureCollections.
Alternatively, you could also put the content in a in-memory GDAL file
(/vsimem/ virtual file system) and open it.
So you could likely instanciate a temporary layer and get a QGIS feature from
that.
If you are just interested in geometries and not attributes, there's also the C
function :
OGRGeometryHOGR_G_CreateGeometryFromJson (const char *);
>
> If I could somehow take advantage of OGR's geojson driver to do the heavy
> lifting then this work would be trivial.
>
> Any ideas?
>
> Nyall
--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
On 15 Jan 2016 2:39 AM, "Even Rouault" wrote: > > > A more "streaming-like" approach for the driver not proceeding to full > ingestion of features could be desirable to remove that limitation, but that's > more involved. A possibility could be to have a "at-hand" parser to delimitate > JSon "Feature" objects and use only json-c to parse each feature. But GeoJSON > is certainly not the more appropriate file format to deal with huge datasets... > Even, Editable GeoJSON in OGR is great news! I've got a question regarding the geojson driver you may be able to assist with. Is there any method in the OGR libraries which allow direct parsing of a string to a layer? (Ie, without first writing it out to a file). I'd like to add the ability to directly paste geojson text into QGIS and have it inserted as a feature in the current layer (like how you can currently paste WKT text as a feature). I don't want to have to manually parse the json (that would be a nightmare). If I could somehow take advantage of OGR's geojson driver to do the heavy lifting then this work would be trivial. Any ideas? Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
On 01/14/2016 07:07 PM, Even Rouault wrote: > Le jeudi 14 janvier 2016 18:49:40, Matthias Kuhn a écrit : >> On 01/14/2016 06:32 PM, Even Rouault wrote: >>> Le jeudi 14 janvier 2016 18:04:00, Matthias Kuhn a écrit : On 01/14/2016 05:42 PM, Even Rouault wrote: >> It will benefit from expression compiling (for filtering and order by) >> which results in performance improvements when it can be applied. > I'm not completely sure to know what you refer to in the QGIS context > but the OGR SetAttributeFilter() method is directly evaluated by the > SQLite request engine as a WHERE clause, and similarly for the > ExecuteSQL() API. It tries to compile QgsExpressions to native SQL syntax. I think the point of using OGR that it offers an awesome abstraction layer where the consumer doesn't need to care about the backend. As soon as ExecuteSQL is used this abstraction no longer applies and it just adds another layer of uncertainty (different library versions...) and may potentially hide internals of the backends which may be useful in some cases (datatypes, backend metadata, available indexes, ...), so the advantage is gone while it complicates advanced topics. Or did I miss something? >>> - datatypes: all the ones listed in "Table 1. GeoPackage Data Types" >>> http://www.geopackage.org/spec/#_sqlite_container have an OGR equivalent >>> - backend metadata: not sure what you're thinking exactly too. But at >>> some point this has to be translated in the QGIS abstraction too, no ? - >>> available indexes: not present in the OGR abstraction currently (but the >>> abstraction can be extended if needed, as it has been done in the past). >>> Is QGIS aware of which indices exist ? >>> >>> A possibility would be to have a thin GeoPackage provider (I guess it >>> could probably even by generic to most OGR drivers that have native >>> SQL92 capabilities) extending the base OGR one + using its capabilities >>> with the specificities of GPKG (eg adding the SORT BY clause). Or a more >>> quick&dirty way would be to hack that directly in the OGR provider for >>> the vetted OGR drivers. >> I was referring to databases in general (expression compilation is also >> done for MSSQL, Oracle, Postgres) >> >> I do not doubt that for every requirement an interface can also be >> implemented in OGR but it still adds an extra roundtrip in the >> development cycle, its presence has to be guaranteed on the client >> system (we're still at 1.10.x on win, not sure about all linux distros >> and mac) and an extra layer of possible errors, performance loss... > I can understand those reasons (althoug I'm a bit skeptical about the > performance loss. That would mean the translation from OGRFeature to > QGISFeature becomes really significant in performance profiles) Fair enough, this particular point is unlikely to be significant >> What would be the advantage of using OGR when using ExecuteSQL? > Just that the GPKG driver already exists (that was my main point) and in > theory GDAL/OGR is the natural place to deal with file formats. > And it does a good job with it, at least there is GPKG support already! -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Le jeudi 14 janvier 2016 18:49:40, Matthias Kuhn a écrit : > On 01/14/2016 06:32 PM, Even Rouault wrote: > > Le jeudi 14 janvier 2016 18:04:00, Matthias Kuhn a écrit : > >> On 01/14/2016 05:42 PM, Even Rouault wrote: > It will benefit from expression compiling (for filtering and order by) > which results in performance improvements when it can be applied. > >>> > >>> I'm not completely sure to know what you refer to in the QGIS context > >>> but the OGR SetAttributeFilter() method is directly evaluated by the > >>> SQLite request engine as a WHERE clause, and similarly for the > >>> ExecuteSQL() API. > >> > >> It tries to compile QgsExpressions to native SQL syntax. > >> > >> I think the point of using OGR that it offers an awesome abstraction > >> layer where the consumer doesn't need to care about the backend. > >> > >> As soon as ExecuteSQL is used this abstraction no longer applies and it > >> just adds another layer of uncertainty (different library versions...) > >> and may potentially hide internals of the backends which may be useful > >> in some cases (datatypes, backend metadata, available indexes, ...), so > >> the advantage is gone while it complicates advanced topics. > >> Or did I miss something? > > > > - datatypes: all the ones listed in "Table 1. GeoPackage Data Types" > > http://www.geopackage.org/spec/#_sqlite_container have an OGR equivalent > > - backend metadata: not sure what you're thinking exactly too. But at > > some point this has to be translated in the QGIS abstraction too, no ? - > > available indexes: not present in the OGR abstraction currently (but the > > abstraction can be extended if needed, as it has been done in the past). > > Is QGIS aware of which indices exist ? > > > > A possibility would be to have a thin GeoPackage provider (I guess it > > could probably even by generic to most OGR drivers that have native > > SQL92 capabilities) extending the base OGR one + using its capabilities > > with the specificities of GPKG (eg adding the SORT BY clause). Or a more > > quick&dirty way would be to hack that directly in the OGR provider for > > the vetted OGR drivers. > > I was referring to databases in general (expression compilation is also > done for MSSQL, Oracle, Postgres) > > I do not doubt that for every requirement an interface can also be > implemented in OGR but it still adds an extra roundtrip in the > development cycle, its presence has to be guaranteed on the client > system (we're still at 1.10.x on win, not sure about all linux distros > and mac) and an extra layer of possible errors, performance loss... I can understand those reasons (althoug I'm a bit skeptical about the performance loss. That would mean the translation from OGRFeature to QGISFeature becomes really significant in performance profiles) > > What would be the advantage of using OGR when using ExecuteSQL? Just that the GPKG driver already exists (that was my main point) and in theory GDAL/OGR is the natural place to deal with file formats. -- Spatialys - Geospatial professional services http://www.spatialys.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
On 01/14/2016 06:32 PM, Even Rouault wrote: > Le jeudi 14 janvier 2016 18:04:00, Matthias Kuhn a écrit : >> On 01/14/2016 05:42 PM, Even Rouault wrote: It will benefit from expression compiling (for filtering and order by) which results in performance improvements when it can be applied. >>> I'm not completely sure to know what you refer to in the QGIS context but >>> the OGR SetAttributeFilter() method is directly evaluated by the SQLite >>> request engine as a WHERE clause, and similarly for the ExecuteSQL() >>> API. >> It tries to compile QgsExpressions to native SQL syntax. >> >> I think the point of using OGR that it offers an awesome abstraction >> layer where the consumer doesn't need to care about the backend. >> >> As soon as ExecuteSQL is used this abstraction no longer applies and it >> just adds another layer of uncertainty (different library versions...) >> and may potentially hide internals of the backends which may be useful >> in some cases (datatypes, backend metadata, available indexes, ...), so >> the advantage is gone while it complicates advanced topics. >> Or did I miss something? > - datatypes: all the ones listed in "Table 1. GeoPackage Data Types" > http://www.geopackage.org/spec/#_sqlite_container have an OGR equivalent > - backend metadata: not sure what you're thinking exactly too. But at some > point this has to be translated in the QGIS abstraction too, no ? > - available indexes: not present in the OGR abstraction currently (but the > abstraction can be extended if needed, as it has been done in the past). Is > QGIS aware of which indices exist ? > > A possibility would be to have a thin GeoPackage provider (I guess it could > probably even by generic to most OGR drivers that have native SQL92 > capabilities) extending the base OGR one + using its capabilities with the > specificities of GPKG (eg adding the SORT BY clause). Or a more quick&dirty > way > would be to hack that directly in the OGR provider for the vetted OGR drivers. I was referring to databases in general (expression compilation is also done for MSSQL, Oracle, Postgres) I do not doubt that for every requirement an interface can also be implemented in OGR but it still adds an extra roundtrip in the development cycle, its presence has to be guaranteed on the client system (we're still at 1.10.x on win, not sure about all linux distros and mac) and an extra layer of possible errors, performance loss... What would be the advantage of using OGR when using ExecuteSQL? >> All the best -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Le jeudi 14 janvier 2016 18:04:00, Matthias Kuhn a écrit : > On 01/14/2016 05:42 PM, Even Rouault wrote: > >> It will benefit from expression compiling (for filtering and order by) > >> which results in performance improvements when it can be applied. > > > > I'm not completely sure to know what you refer to in the QGIS context but > > the OGR SetAttributeFilter() method is directly evaluated by the SQLite > > request engine as a WHERE clause, and similarly for the ExecuteSQL() > > API. > > It tries to compile QgsExpressions to native SQL syntax. > > I think the point of using OGR that it offers an awesome abstraction > layer where the consumer doesn't need to care about the backend. > > As soon as ExecuteSQL is used this abstraction no longer applies and it > just adds another layer of uncertainty (different library versions...) > and may potentially hide internals of the backends which may be useful > in some cases (datatypes, backend metadata, available indexes, ...), so > the advantage is gone while it complicates advanced topics. > Or did I miss something? - datatypes: all the ones listed in "Table 1. GeoPackage Data Types" http://www.geopackage.org/spec/#_sqlite_container have an OGR equivalent - backend metadata: not sure what you're thinking exactly too. But at some point this has to be translated in the QGIS abstraction too, no ? - available indexes: not present in the OGR abstraction currently (but the abstraction can be extended if needed, as it has been done in the past). Is QGIS aware of which indices exist ? A possibility would be to have a thin GeoPackage provider (I guess it could probably even by generic to most OGR drivers that have native SQL92 capabilities) extending the base OGR one + using its capabilities with the specificities of GPKG (eg adding the SORT BY clause). Or a more quick&dirty way would be to hack that directly in the OGR provider for the vetted OGR drivers. > > All the best -- Spatialys - Geospatial professional services http://www.spatialys.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
On 01/14/2016 05:42 PM, Even Rouault wrote: >> It will benefit from expression compiling (for filtering and order by) >> which results in performance improvements when it can be applied. > I'm not completely sure to know what you refer to in the QGIS context but the > OGR SetAttributeFilter() method is directly evaluated by the SQLite request > engine as a WHERE clause, and similarly for the ExecuteSQL() API. It tries to compile QgsExpressions to native SQL syntax. I think the point of using OGR that it offers an awesome abstraction layer where the consumer doesn't need to care about the backend. As soon as ExecuteSQL is used this abstraction no longer applies and it just adds another layer of uncertainty (different library versions...) and may potentially hide internals of the backends which may be useful in some cases (datatypes, backend metadata, available indexes, ...), so the advantage is gone while it complicates advanced topics. Or did I miss something? All the best -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
> It will benefit from expression compiling (for filtering and order by) > which results in performance improvements when it can be applied. I'm not completely sure to know what you refer to in the QGIS context but the OGR SetAttributeFilter() method is directly evaluated by the SQLite request engine as a WHERE clause, and similarly for the ExecuteSQL() API. > > >> All the best -- Spatialys - Geospatial professional services http://www.spatialys.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
On 01/14/2016 04:47 PM, Even Rouault wrote: > Le jeudi 14 janvier 2016 16:38:43, Matthias Kuhn a écrit : >> Hi Michaël >> >> On 01/14/2016 04:18 PM, kimaidou wrote: >>> Hi QGIS, >>> >>> I have read that GDAL can now update GeoJSON since version 2.1.0. See >>> [1] >>> >>> If I build QGIS with GDAL 2.1, would I be able to edit the GeoJSON as >>> I can for Shapefiles ? For the record, capabilities are defined here [2] >>> >>> Perhaps the GDAL doc about "updating existing GeoJSON files" has a >>> different meaning that the need raised by Even Rouault here [3] >>> >>> Or is the best approach to create a specific QGIS driver for GeoJSON ? >>> I would try to propose a PR for this if this is the best way to go. >> Giving GDAL 2.1 a try will be way less effort than implementing a >> GeoJSON driver inside QGIS. And if there are things still left to be >> fixed, it's probably better to fix things on the GDAL end since this way >> the support for it is in a library shared by many FOSSGIS tools. >> >> (Please note that this recommendation is only meant for non-database >> providers. Feel free to propose a native GeoPackage driver ;) ) > Just curious to know what advantage a native GeoPackage driver would bring > over the one in OGR ? It will benefit from expression compiling (for filtering and order by) which results in performance improvements when it can be applied. >> All the best -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Thanks a lot Even (and Matthias) for your prompt answers, and for pointing out the current limitations due to in-memory layer approach. I am build GDAL trunk to make some tests with some data Regards Michaël 2016-01-14 16:47 GMT+01:00 Even Rouault : > Le jeudi 14 janvier 2016 16:38:43, Matthias Kuhn a écrit : > > Hi Michaël > > > > On 01/14/2016 04:18 PM, kimaidou wrote: > > > Hi QGIS, > > > > > > I have read that GDAL can now update GeoJSON since version 2.1.0. See > > > [1] > > > > > > If I build QGIS with GDAL 2.1, would I be able to edit the GeoJSON as > > > I can for Shapefiles ? For the record, capabilities are defined here > [2] > > > > > > Perhaps the GDAL doc about "updating existing GeoJSON files" has a > > > different meaning that the need raised by Even Rouault here [3] > > > > > > Or is the best approach to create a specific QGIS driver for GeoJSON ? > > > I would try to propose a PR for this if this is the best way to go. > > > > Giving GDAL 2.1 a try will be way less effort than implementing a > > GeoJSON driver inside QGIS. And if there are things still left to be > > fixed, it's probably better to fix things on the GDAL end since this way > > the support for it is in a library shared by many FOSSGIS tools. > > > > (Please note that this recommendation is only meant for non-database > > providers. Feel free to propose a native GeoPackage driver ;) ) > > Just curious to know what advantage a native GeoPackage driver would bring > over the one in OGR ? > > > > > All the best > > -- > Spatialys - Geospatial professional services > http://www.spatialys.com > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Le jeudi 14 janvier 2016 16:38:43, Matthias Kuhn a écrit : > Hi Michaël > > On 01/14/2016 04:18 PM, kimaidou wrote: > > Hi QGIS, > > > > I have read that GDAL can now update GeoJSON since version 2.1.0. See > > [1] > > > > If I build QGIS with GDAL 2.1, would I be able to edit the GeoJSON as > > I can for Shapefiles ? For the record, capabilities are defined here [2] > > > > Perhaps the GDAL doc about "updating existing GeoJSON files" has a > > different meaning that the need raised by Even Rouault here [3] > > > > Or is the best approach to create a specific QGIS driver for GeoJSON ? > > I would try to propose a PR for this if this is the best way to go. > > Giving GDAL 2.1 a try will be way less effort than implementing a > GeoJSON driver inside QGIS. And if there are things still left to be > fixed, it's probably better to fix things on the GDAL end since this way > the support for it is in a library shared by many FOSSGIS tools. > > (Please note that this recommendation is only meant for non-database > providers. Feel free to propose a native GeoPackage driver ;) ) Just curious to know what advantage a native GeoPackage driver would bring over the one in OGR ? > > All the best -- Spatialys - Geospatial professional services http://www.spatialys.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Hi Michael, > > I have read that GDAL can now update GeoJSON since version 2.1.0. See [1] > > If I build QGIS with GDAL 2.1, would I be able to edit the GeoJSON as I can > for Shapefiles ? Yes, that was developed mostly for the QGIS use case and has been tested with it. You have full editing capabilities : adding, deleting, modifying features, adding, deleting fields. > > Another GeoJSON related question. Any idea on the relevance of adding the > ability to create a spatial index file *.qix for GeoJSON layers in order to > improve rendering speed ? The OGR GeoJSON driver proceeds into the ingestion of the full file into memory, which can be a limitation when dealing with huge files ( the json-c lib might typically consume an amount of memory more than 10 times the size of the geojson file). For a in-memory layer like the one used underneath by the GeoJSON driver, the spatial filtering done by OGR uses a basic BBOX filter while iterating over the features (the BBOX isn't stored into the geometry object but recomputed each time it is needed). This could potentially benefit from a in-memory spatial index, but I'd expect that to be only useful for large geojson files (let's say > 100 MB), i.e. the ones that the driver would currently have issues to deal with. A more "streaming-like" approach for the driver not proceeding to full ingestion of features could be desirable to remove that limitation, but that's more involved. A possibility could be to have a "at-hand" parser to delimitate JSon "Feature" objects and use only json-c to parse each feature. But GeoJSON is certainly not the more appropriate file format to deal with huge datasets... Cheers, Even -- Spatialys - Geospatial professional services http://www.spatialys.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] GDAL 2.1 can update Geojson files - What is missing in QGIS to edit GeoJSON files ?
Hi Michaël On 01/14/2016 04:18 PM, kimaidou wrote: > Hi QGIS, > > I have read that GDAL can now update GeoJSON since version 2.1.0. See [1] > > If I build QGIS with GDAL 2.1, would I be able to edit the GeoJSON as > I can for Shapefiles ? For the record, capabilities are defined here [2] > > Perhaps the GDAL doc about "updating existing GeoJSON files" has a > different meaning that the need raised by Even Rouault here [3] > > Or is the best approach to create a specific QGIS driver for GeoJSON ? > I would try to propose a PR for this if this is the best way to go. Giving GDAL 2.1 a try will be way less effort than implementing a GeoJSON driver inside QGIS. And if there are things still left to be fixed, it's probably better to fix things on the GDAL end since this way the support for it is in a library shared by many FOSSGIS tools. (Please note that this recommendation is only meant for non-database providers. Feel free to propose a native GeoPackage driver ;) ) All the best -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
