Re: [QGIS-Developer] qgis / ogr converts INT & DATE values from spatial view to STRING
I've reported the issue to the OGC GeoPackage specification issue
tracker: https://github.com/opengeospatial/geopackage/issues/683
Le 22/07/2024 à 15:57, Even Rouault via QGIS-Developer a écrit :
Benjamin,
Explicit casts like "(CAST(SUM(D.QAI) as INTEGER) as n_qai" did not
solve my problem.
Yeah, I recently came across a similar issue, and the finding is that
it is related to the SQLite version. Such casts apparently work since
SQLite 3.42. So if your QGIS build uses an older SQLite version, they
won't be honored.
$ ogr2ogr poly.gpkg autotest/ogr/data/poly.shp
$ ogrinfo poly.gpkg -sql "create view v as select fid as ogc_fid,
geom, cast(1 as int) as my_int, cast(1 as real) as my_real from poly"
$
LD_LIBRARY_PATH=/home/even/install-sqlite-3.41.0/lib:$LD_LIBRARY_PATH
ogrinfo poly.gpkg -al -so | grep my_
my_int: String (0.0)
my_real: String (0.0)
vs
$
LD_LIBRARY_PATH=/home/even/install-sqlite-3.42.0/lib:$LD_LIBRARY_PATH
ogrinfo poly.gpkg -al -so | grep my_
my_int: Integer64 (0.0)
my_real: Real (0.0)
This is really a SQLite3 issue as demonstrated by:
$ ~/install-sqlite-3.41.0/bin/sqlite3 poly.gpkg "pragma table_info(v)"
0|ogc_fid|INTEGER|0||0
1|geom|POLYGON|0||0
2|my_int|NUM|0||0
3|my_real|NUM|0||0
vs
$ ~/install-sqlite-3.42.0/bin/sqlite3 poly.gpkg "pragma table_info(v)"
0|ogc_fid|INTEGER|0||0
1|geom|POLYGON|0||0
2|my_int|INT|0||0
3|my_real|REAL|0||0
*However* this only works for TEXT, INTEGER and REAL. As there is no
SQLite native DATETIME "affinity" (cf
https://www.sqlite.org/datatype3.html), there's no way currently to
force view columns to DATETIME
Interestingly, the OGRDataSource::ExecuteSQL does returns correct
integer value for COUNT and SUM results. DATE(...) results are
returned as STRINGS as well.
with ds.ExecuteSQL('SELECT * FROM ard_data_byTile') as lyr:
yes, that's because in that mode, the GPKG driver uses the value of
the first row to infer the data type, but this isn't super bullet
proof (particularly if the first row contains NULL values)
Even
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] qgis / ogr converts INT & DATE values from spatial view to STRING
Benjamin,
Explicit casts like "(CAST(SUM(D.QAI) as INTEGER) as n_qai" did not
solve my problem.
Yeah, I recently came across a similar issue, and the finding is that it
is related to the SQLite version. Such casts apparently work since
SQLite 3.42. So if your QGIS build uses an older SQLite version, they
won't be honored.
$ ogr2ogr poly.gpkg autotest/ogr/data/poly.shp
$ ogrinfo poly.gpkg -sql "create view v as select fid as ogc_fid, geom,
cast(1 as int) as my_int, cast(1 as real) as my_real from poly"
$ LD_LIBRARY_PATH=/home/even/install-sqlite-3.41.0/lib:$LD_LIBRARY_PATH
ogrinfo poly.gpkg -al -so | grep my_
my_int: String (0.0)
my_real: String (0.0)
vs
$ LD_LIBRARY_PATH=/home/even/install-sqlite-3.42.0/lib:$LD_LIBRARY_PATH
ogrinfo poly.gpkg -al -so | grep my_
my_int: Integer64 (0.0)
my_real: Real (0.0)
This is really a SQLite3 issue as demonstrated by:
$ ~/install-sqlite-3.41.0/bin/sqlite3 poly.gpkg "pragma table_info(v)"
0|ogc_fid|INTEGER|0||0
1|geom|POLYGON|0||0
2|my_int|NUM|0||0
3|my_real|NUM|0||0
vs
$ ~/install-sqlite-3.42.0/bin/sqlite3 poly.gpkg "pragma table_info(v)"
0|ogc_fid|INTEGER|0||0
1|geom|POLYGON|0||0
2|my_int|INT|0||0
3|my_real|REAL|0||0
*However* this only works for TEXT, INTEGER and REAL. As there is no
SQLite native DATETIME "affinity" (cf
https://www.sqlite.org/datatype3.html), there's no way currently to
force view columns to DATETIME
Interestingly, the OGRDataSource::ExecuteSQL does returns correct
integer value for COUNT and SUM results. DATE(...) results are
returned as STRINGS as well.
with ds.ExecuteSQL('SELECT * FROM ard_data_byTile') as lyr:
yes, that's because in that mode, the GPKG driver uses the value of the
first row to infer the data type, but this isn't super bullet proof
(particularly if the first row contains NULL values)
Even
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] WKT from QgsGeometry incompatible with OGR
I've went ahead and added support for PointZ like WKT strings in OGR in
https://github.com/OSGeo/gdal/pull/10370, but it would be probably
better that QGIS writes conformant WKT by default (I suspect that fixing
this has the potential of breaking a number of tests though)
Le 06/07/2024 à 08:56, Even Rouault via QGIS-Developer a écrit :
Hi,
Both formats are compliant with the WKT specifications.
Are they? I may have missed something in the Simple Features spec, but
looking at https://portal.ogc.org/files/?artifact_id=25355 , I can't
see where it would allow the form {geometryTypeName}Z without a space
between {geometryTypeName} and Z.
For example the BNF at page 56 shows:
::= point z
The examples at page 62 also show a space.
Even
Nyall
A minimal example is below. Executing it will return an exception:
"RuntimeError: OGR Error: Corrupt data"
from osgeo import ogr
qgis_geometyry = QgsGeometry().fromWkt("POINT Z (0 0 0)")
wkt = qgis_geometyry.asWkt() # 'PointZ (0 0 0)'
ogr.CreateGeometryFromWkt(wkt)
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] WKT from QgsGeometry incompatible with OGR
Hi,
Both formats are compliant with the WKT specifications.
Are they? I may have missed something in the Simple Features spec, but
looking at https://portal.ogc.org/files/?artifact_id=25355 , I can't see
where it would allow the form {geometryTypeName}Z without a space
between {geometryTypeName} and Z.
For example the BNF at page 56 shows:
::= point z
The examples at page 62 also show a space.
Even
Nyall
A minimal example is below. Executing it will return an exception:
"RuntimeError: OGR Error: Corrupt data"
from osgeo import ogr
qgis_geometyry = QgsGeometry().fromWkt("POINT Z (0 0 0)")
wkt = qgis_geometyry.asWkt() # 'PointZ (0 0 0)'
ogr.CreateGeometryFromWkt(wkt)
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Build problems using install instructions -building branches
Hi, The procedure at https://github.com/qgis/QGIS/blob/master/INSTALL.md#39-building-different-branches is more for developers to create feature branches and submit pull requests To just built the release-3_36 branch, Assuming that "origin" is your remote pointing to https://github.com/qgis/qgis (if not "git remote add origin https://github.com/qgis/qgis;, or replace origin by the name of the remote that points to https://github.com/qgis/qgis ) from your release-3_36 directory: git fetch origin release-3_36 git reset --hard origin/release-3_36 # make sure you have no local changes before doing that! Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Compiling against homebuilt GDAL
When you run "make -jSOMETHING" the error message may be quite above the end of the standard error stream. Look further above for a "error:" string or something like that. Or just re-run "make", and wait for it to fail. The error message should then just be in the last lines Le 12/03/2024 à 11:44, Thomas Larsen Wessel a écrit : Sorry for the late reply. @Richard; Your advice is certainly useful. I could use some advice on best practices :) After *installing* GDAL, I was able to run cmake without issues. But then make would fail at 56% every time (simply printing "Error 2"). By coincidence I found out, that if I just ran the same make command one more time (without cleaning), it would progress beyond the 56%, and run for another 20 minutes before it failed at 64%. If I ran for a third time it would progress a bit more (or maybe the output just looks different), but still at 64%. Any additional attempts give no further progress. mkdir build; cd build GDAL_DIR=/home/velle/install-gdal-3.7.2 cmake .. -DCMAKE_BUILD_TYPE=DEBUG 2>&1 | tee cmakelog.txt time make -j4 VERBOSE=1 2>&1 | tee make_j4_1.txt time make -j4 VERBOSE=1 2>&1 | tee make_j4_2.txt time make -j4 VERBOSE=1 2>&1 | tee make_j4_3.txt The last part of make_j4_3.txt looks like this: Dependencies file "src/core/CMakeFiles/qgis_core.dir/vectortile/qgsvtpktiles.cpp.o.d" is newer than depends file "/home/velle/b/QGIS/build/src/core/CMakeFiles/qgis_core.dir/compiler_depend.internal". Dependencies file "src/core/CMakeFiles/qgis_core.dir/vectortile/qgsvtpkvectortiledataprovider.cpp.o.d" is newer than depends file "/home/velle/b/QGIS/build/src/core/CMakeFiles/qgis_core.dir/compiler_depend.internal". Dependencies file "src/core/CMakeFiles/qgis_core.dir/vectortile/qgsxyzvectortiledataprovider.cpp.o.d" is newer than depends file "/home/velle/b/QGIS/build/src/core/CMakeFiles/qgis_core.dir/compiler_depend.internal". Consolidate compiler generated dependencies of target qgis_core make[2]: Leaving directory '/home/velle/b/QGIS/build' make -f src/core/CMakeFiles/qgis_core.dir/build.make src/core/CMakeFiles/qgis_core.dir/build make[2]: Entering directory '/home/velle/b/QGIS/build' make[2]: Nothing to be done for 'src/core/CMakeFiles/qgis_core.dir/build'. make[2]: Leaving directory '/home/velle/b/QGIS/build' [ 64%] Built target qgis_core make[1]: Leaving directory '/home/velle/b/QGIS/build' make: *** [Makefile:166: all] Error 2 I don't see anything failing until the point it aborts. Can I get more details about what is failing? It took some time to find the VERBOSE=1 option, but even with verbose output I don't understand what is going on. Is it On Thu, Feb 29, 2024 at 8:52 AM Matthias Kuhn wrote: On Wed, Feb 28, 2024 at 7:45 PM Even Rouault via QGIS-Developer wrote: and then when configuring QGIS, use GDAL_DIR=$HOME/install-gdal-3.7.2 . I'm not totally sure about that last part. may require tweaking. you could alter the PATH to point to $HOME/install-gdal-3.7.2/bin so that "gdal-config" from your installed GDAL is picked up It will first try to locate GDAL via cmake config (https://github.com/qgis/QGIS/blob/master/cmake/FindGDAL.cmake#L16). Only if that fails it will fall back to the gdal-config executable. To help cmake find a package on a custom install prefix, refer to https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-search-procedure in your case I'd suggest using `-D GDAL_ROOT=$HOME/install-gdal-3.7.2` after installing Matthias -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Compiling against homebuilt GDAL
Thomas, Le 28/02/2024 à 19:33, Thomas Larsen Wessel via QGIS-Developer a écrit : I have successfully built QGIS and GDAL (3.7.2) separately. Now I want to build QGIS against this GDAL. This is what I tried: git clone g...@github.com:qgis/QGIS.git cd QGIS mkdir build cd build GDAL_DIR=/home/velle/gdal3.7.2/build cmake .. The output contains the following error for GDAL: CMake Error at /home/velle/gdal-3.7.2/build/GDALConfig.cmake:24 (include): include could not find requested file: /home/velle/gdal-3.7.2/build/GDAL-targets.cmake Call Stack (most recent call first): cmake/FindGDAL.cmake:16 (find_package) CMakeLists.txt:365 (find_package) Could someone please tell me if I'm even doing it right? You need to *install* GDAL first. You can't just point QGIS to the GDAL build directory So in the GDAL build directory, do "cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install-gdal-3.7.2 ; make -j$(nproc); make install" and then when configuring QGIS, use GDAL_DIR=$HOME/install-gdal-3.7.2 . I'm not totally sure about that last part. may require tweaking. you could alter the PATH to point to $HOME/install-gdal-3.7.2/bin so that "gdal-config" from your installed GDAL is picked up Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] GDAL processing algorithms issues due the default output layer extensions weirdly changed
I'm not sure if that explains the family of bugs seen, but as those settings are a *integer* index in the list returned by QgsVectorFileWriter::supportedFormatExtensions() or QgsRasterFileWriter::supportedFormatExtensions() , they depend both on the QGIS list of known formats which may evolve over versions (not that frequent that said), and/or if the GDAL version changes and/or if drivers are enabled/disabled. It would be more robust to capture the extension as a string rather than an index. And even capturing an extension is not super robust, as potentially several drivers could recognize the same extension For example when using the ".xml" extension as a vector output, you'll probably be surprised that the obscure PDS4 vector driver (tabular data for planetary datasets) is used! This is because: $ ogrinfo --formats | grep xml | grep 'w' PDS4 -raster,vector- (rw+vs): NASA Planetary Data System 4 (*.xml) GML -vector- (rw+v): Geography Markup Language (GML) (*.gml, *.xml) Interlis 2 -vector- (rw+v): Interlis 2 (*.xtf, *.xml, *.ili) GMLAS -vector- (rwv): Geography Markup Language (GML) driven by application schemas (*.gml, *.xml) whereas looking at QgsVectorFileWriterMetadataContainer , I strongly suspect that the intent was rather to use the GeoRSS driver for which QGIS registers a .xml extension (the GDAL GeoRSS driver doesn't advertize a particular file extension) ==> Those settings should actually be deprecated and replaced by others that save a GDAL driver name ... plus an extension, because the "MapInfo File" driver produces quite different output if you chose ".tab" or ".mif" (or the GeoJSONSeq driver that will use different separators if you select the ".geojsonl" or ".geojsons" extensions) Even Le 26/02/2024 à 00:23, Nyall Dawson via QGIS-Developer a écrit : On Mon, 26 Feb 2024 at 08:58, Andrea Giudiceandrea via QGIS-Developer wrote: Hi developers, various QGIS users have reported during the latest months (mostly since September 2023, and a couple also back in July 2023) on various channels a strange issue which makes the GDAL processing algorithms to fail when the temporary output is set. Looking at the provided processing logs, it turns out that the issue is due to the fact that the default output vector layer extension was set to .xtf "Interlis 2" (instead of the default .gpkg "GeoPackage") or that the default output raster layer extension was set to .nc "NetCDF" (instead of the default .tif "GeoTIFF"). The weird think is that all the users affected by the issue have reported that they had not deliberately changed these settings. Hmm, I wonder if it's a plugin changing this setting. I've definitely seen plugins before doing unfriendly things like this... Nyall ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Backporting qt6 sip bindings to 3.34?
Le 10/02/2024 à 00:07, Nyall Dawson via QGIS-Developer a écrit : Hey list, What does everyone think about us backporting the python/PyQt6/ directory from master to the 3.34 branch? I'm thinking we could do this as "orphan"/unused files, with the intention that it would make backporting to this branch easier. Obviously it introduces a bunch of "noise" in the repo for this branch. +1 Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] QGIS plugin depends on pypi package
Le 09/02/2024 à 00:45, John Lindsay via QGIS-Developer a écrit : Hello, I believe that I've solved my earlier issue with my QGIS plugin, which previously relied on a pip package. In order to avoid this dependency, I decided to include the Whitebox Workflows wheel in with my plugin. Unfortunately, however, this has increased the size of my plugin fairly substantially. The issue is that I need to included a wheel for each of the four supported operating systems and each are about 10MB. My plugin zip file is now a little less than 40MB. When I try to upload the new version to the QGIS plugin repo it is telling me that the file is too large. Does anyone know what the maximum size of a plugin zip file is? Thanks. Presumably 25 MB if I trust https://github.com/qgis/QGIS-Django/blob/master/qgis-app/plugins/validator.py#L20 . (But in theory, my interpretation of https://plugins.qgis.org/publish/ "Don't include binaries" requirement is that you wouldn't be allowed to include binary wheels.. There seems however to be a discrepancy between that requirement and the actual practice since other plugins do that.) Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] [Qgis-psc] Theoretical discussion: A QGIS paid plugin marketplace?
I’m not sure if I would want this marketplace being hosted by QGIS itself and being access through the current Plugin Manager. I prefer the idea of QGIS being clearly free… but at the same time maybe it’s not that bad if there is a mix of free and paid plugin at the same place, as long as it’s very easy filter. The obvious benefit of a marketplace hosted by QGIS is that a small commission could collected as Tim suggested. With a separate entity managing paid plugins, there is no way for QGIS.org to systematically receive a small donation as Sebastian explained with the way it works with Blender. A viable external market place would at some point need to use the QGIS trademark (in their domain name, etc), so they would have to talk with QGIS.org first to see at which condition QGIS.org accepts they use its trademark. That condition could be to subscribe to a annual sustainable membership + a commission per plugin -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Theoretical discussion: A QGIS paid plugin marketplace?
Hi Nyall, It would seem tricky that qgis.org both offers/promotes a gratis & FOSS "market place" (the current official plugin repository), and a paid one. That could be conceived, but if qgis.org would promote in some way the paid market place, this would create some expectation on the quality of the plugins proposed for a fee, and QGIS itself, which could create some tensions with users. The theoretical question of someone buying a paid (GPL) plugin (or maybe just finding its source code somewhere) and making it a gratis plugin once it has obtained its source code would be quite untainable if qgis.org managed both market places. The spirit & letter of the GPL is that anyone can improve existing code and share it with others, so it would be a hard take to prevent someone from distributing an improved/modified version of a paid GPL plugin in the gratis repository, and obviously having the same entity managing both market places would be a conflict of interest. I'm not sure if the non-profit status of qgis.org would allow it to manage a paid marketplace. I was wondering about the connection between Blender Market, LLC the private company behind blendermarket.com, and Blender.org the OSS organization, since that could be an interesting situation with respect to trademark. https://blendermarket.com/page/about brings interesting clues about why it's working: - "The Blender Market is independently owned and operated and has no affiliation with The Blender Foundation." but: - "The Blender Market is a proud Corporate Gold member of the Blender Development Fund" I don't know if blender.org has some sort of gratis plugin repo, or if not, more generally how they would welcome a code contribution in their software coming from OSS code from the paid market place... I assume they must have some policy on how to handle such situation. All in all, I believe that such organization, with a clear separation of the .org and the .com as different legal entities, with a "win-win" relationship, would probably be the best solution if one wanted to implement that. Even Le 01/02/2024 à 02:28, Nyall Dawson via QGIS-Developer a écrit : Hi lists! I wanted to kick start a (hopefully!) civil, THEORETICAL discussion about the role of a paid plugin marketplace for QGIS plugins. This has been on my mind for a while, and recently was bumped by this email to the list: On Fri, 19 Jan 2024 at 19:38, gam17--- via QGIS-Developer wrote: > > Hi everyone, > like many of you, I have developed and maintained a plugin for many > years completely free of charge. > I have never received any donation or compensation of any kind and now I > would like to find a solution. > Has anyone already found a way to receive donations? > I was thinking of asking for a sponsor that would be displayed during > execution, for example in the window titles or through a specific menu > item like QGIS does (in this way the sponsor would be much less > visible). So again, stressing that this is a THEORETICAL discussion, I'm interested in hearing people's thoughts on the potential role of a paid plugin marketplace for QGIS. Here's a bullet point dump of where I'm currently sitting: - Yes, I'm aware that plugins must be GPL, and that this makes paid plugins a little trickier in that they're obviously still subject to the GPL. - The GPL does NOT prevent charging for software, or mandate making it public to non-paying customers. We could potentially have GPL plugins which are only available to paid users, and only make these plugins available privately to those users. YES, the GPL **DOES** mean that those paying customers can redistribute the plugin publicly and freely without issue if they want (and regardless of whether the original developer wants!) - In fact, there's already likely thousands of private, paid for plugins out there! I'm talking here of plugins made specifically for internal use by one organisation only. Yep, that organisation COULD make the plugin public/freely available, but in many cases they are specific to that one organisation's needs or contain organisation sensitive logic/data. These plugins are completely compliant with the GPL, despite being private and paid for by that organisation. - There's nothing preventing a public GPL QGIS plugin from depending on a subscription based back-end, and offering zero value to anyone not paying for that backend. And there's a growing number of these plugins, which depend on users paying xxx large corporate entity regular high fees to access the backend service. The GPL doesn't (and arguably shouldn't) prevent these large entities from making money off QGIS plugins. - But this means that the current situation is unfairly weighted toward these large entities! A one-person team making an excellent plugin and providing an awesome tool for use in QGIS has a MUCH MUCH harder time finding ANY financial compensation for their
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
Update: actually, instead of completely disabling server side spatial
filtering with geographic SRS, I just found a hack... One can use
ST_SRID() to hack a "random" projected CRS, and then things work
So now the OGR mySQL driver will do things like:
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 32631), ST_SRID(geometry_column, 32631));
Not pretty but probably a bit better than just client side filtering.
On reflection, I suspect that in MySQL the geometry in geographic SRS
are actually what is called a geography in PostGIS, that is a shape on
the ellipsoid/sphere, where the smallest polygon is selected. And thus
the following request actually returns true:
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 4326, 'axis-order=long-lat'),
ST_GeomFromText('POINT(-179.5 0)', 4326, 'axis-order=long-lat'));
Le 28/01/2024 à 19:04, Even Rouault via QGIS-Developer a écrit :
Richard,
what I don't understand is how you manage to create with ogr2ogr a
table without explicit SRS with features whose coordinates are not
compatible of geographic coordinates. As mentioned previously, I do
get a hard error from ogr2ogr when trying that. Which GDAL version do
you use?
The reason for which QGIS doesn't display features is likely it calls
OGR with SetSpatialFilter() , which the mySQL driver turns into a
WHERE MBRIntersects(ST_GeomFromText('POLYGON((.))',
'axis-order=long-lat'), `geom_colum_name`) filter. The
'axis-order=long-lat' qualifier is because OGR (perhaps mistakenly?)
identifies the layer as with a geographic SRS, and if the extents of
the layer aren't compatible of geographic coordinates, that probably
causes the filter to fail.
Actually trying the following on a layer without explicit CRS, which
then causes a undefined geographic CRS to be assigned to the layer
$ ogrinfo
"mysql:test,user=root,password=passwd,port=33060,host=127.0.0.1" test
-spat -180 -90 180 90
I do get:
ERROR 1: MySQL error message:Longitude -180.00 is out of range in
function st_geomfromtext. It must be within (-180.00, 180.00].
Description: SELECT `OGR_FID`, `SHAPE` `SHAPE`, `id` FROM `test` WHERE
MBRIntersects(ST_GeomFromText('POLYGON((-180 -90, 180 -90, 180 90,
-180 90, -180 -90))', 32767, 'axis-order=long-lat'), `SHAPE`)
So MySQL is super picky on the range of values for spatial filters.
OK, fair enough, easy to fix by clamping values on the driver side.
But!!! I also discovered what is to me a MySQL non-sense. It seems
spatial predicates are totally broken when using geographic SRS.
select MBRIntersects(ST_GeomFromText('POLYGON((-90 -90, 90 -90, 90 90,
-90 90, -90 -90))', 4326), ST_GeomFromText('POINT(0 0)', 4326));
returns true as expected
But enlarging that extent to almost the whole world with
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 4326, 'axis-order=long-lat'),
ST_GeomFromText('POINT(0 0)', 4326));
or (to test if winding order has some importance)
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, -179 89, 179
89, 179 -89, -179 -89))', 4326, 'axis-order=long-lat'),
ST_GeomFromText('POINT(0 0)', 4326));
returns false (I've also verified with mysql:8.3.0 image)
And
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 32631), ST_GeomFromText('POINT(0 0)', 32631));
returns true as expected
Consequence, it seems safer to disable spatial filtering on layers
with geographic coordinates with MySQL...
That's just what I've done in https://github.com/OSGeo/gdal/pull/9152
. Hopefully that will fix those issues with geometries not appearing.
Obviously performance will not be optimal as spatial filtering will be
done on OGR client side...
Even
Le 28/01/2024 à 14:52, Richard Duivenvoorde a écrit :
On 1/23/24 23:21, Even Rouault wrote:
So all in all, no issues on my side...
You were off course right :-)
I found out how to fix the missing crs in mysql and wrote some info
(for future reference) in a comment:
https://github.com/qgis/QGIS/issues/55463#issuecomment-1913602140
One thing I'm curious for, is why QGIS even after setting the right
crs, is not able to show the geom (but IS able to show the
attributes). Apparently the features are retrieved. If you receive
features from a shp file without crs, after setting it on the layer,
all is fine. So myql is different?
Did a short try to debug this, but this is above my c++ skills :-)
Anyway thanks! And I hope the comment is helpful to mysql peeps
Regards,
Richard Duivenvoorde
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/m
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
But, my point: why can QGIS apparently show attributes, while not showing a geom.. Apparently the features were fully retrieved. When displaying the attribute table, QGIS doesn't apply a spatial filter. Whereas when displaying features on the canvas, it does. And as I found out, something's weird (broken?) on MySQL side regarding spatial filtering on layers without CRS (or with the implied undefined geographic SRS) -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
Richard,
what I don't understand is how you manage to create with ogr2ogr a table
without explicit SRS with features whose coordinates are not compatible
of geographic coordinates. As mentioned previously, I do get a hard
error from ogr2ogr when trying that. Which GDAL version do you use?
The reason for which QGIS doesn't display features is likely it calls
OGR with SetSpatialFilter() , which the mySQL driver turns into a WHERE
MBRIntersects(ST_GeomFromText('POLYGON((.))',
'axis-order=long-lat'), `geom_colum_name`) filter. The
'axis-order=long-lat' qualifier is because OGR (perhaps mistakenly?)
identifies the layer as with a geographic SRS, and if the extents of the
layer aren't compatible of geographic coordinates, that probably causes
the filter to fail.
Actually trying the following on a layer without explicit CRS, which
then causes a undefined geographic CRS to be assigned to the layer
$ ogrinfo
"mysql:test,user=root,password=passwd,port=33060,host=127.0.0.1" test
-spat -180 -90 180 90
I do get:
ERROR 1: MySQL error message:Longitude -180.00 is out of range in
function st_geomfromtext. It must be within (-180.00, 180.00].
Description: SELECT `OGR_FID`, `SHAPE` `SHAPE`, `id` FROM `test` WHERE
MBRIntersects(ST_GeomFromText('POLYGON((-180 -90, 180 -90, 180 90, -180
90, -180 -90))', 32767, 'axis-order=long-lat'), `SHAPE`)
So MySQL is super picky on the range of values for spatial filters. OK,
fair enough, easy to fix by clamping values on the driver side.
But!!! I also discovered what is to me a MySQL non-sense. It seems
spatial predicates are totally broken when using geographic SRS.
select MBRIntersects(ST_GeomFromText('POLYGON((-90 -90, 90 -90, 90 90,
-90 90, -90 -90))', 4326), ST_GeomFromText('POINT(0 0)', 4326));
returns true as expected
But enlarging that extent to almost the whole world with
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 4326, 'axis-order=long-lat'),
ST_GeomFromText('POINT(0 0)', 4326));
or (to test if winding order has some importance)
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, -179 89, 179
89, 179 -89, -179 -89))', 4326, 'axis-order=long-lat'),
ST_GeomFromText('POINT(0 0)', 4326));
returns false (I've also verified with mysql:8.3.0 image)
And
select MBRIntersects(ST_GeomFromText('POLYGON((-179 -89, 179 -89, 179
89, -179 89, -179 -89))', 32631), ST_GeomFromText('POINT(0 0)', 32631));
returns true as expected
Consequence, it seems safer to disable spatial filtering on layers with
geographic coordinates with MySQL...
That's just what I've done in https://github.com/OSGeo/gdal/pull/9152 .
Hopefully that will fix those issues with geometries not appearing.
Obviously performance will not be optimal as spatial filtering will be
done on OGR client side...
Even
Le 28/01/2024 à 14:52, Richard Duivenvoorde a écrit :
On 1/23/24 23:21, Even Rouault wrote:
So all in all, no issues on my side...
You were off course right :-)
I found out how to fix the missing crs in mysql and wrote some info
(for future reference) in a comment:
https://github.com/qgis/QGIS/issues/55463#issuecomment-1913602140
One thing I'm curious for, is why QGIS even after setting the right
crs, is not able to show the geom (but IS able to show the
attributes). Apparently the features are retrieved. If you receive
features from a shp file without crs, after setting it on the layer,
all is fine. So myql is different?
Did a short try to debug this, but this is above my c++ skills :-)
Anyway thanks! And I hope the comment is helpful to mysql peeps
Regards,
Richard Duivenvoorde
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Qt6 builds and plugins -- ready for testing!
Richard,
There's something weird with your installation. According to
https://salsa.debian.org/qt-kde-team/extras/qwt/-/blob/debian/master/src/qwt_plot_layout.h?ref_type=heads#L84,
this should be
|constQRectF& plotRect, Options options = Options());|
According to
https://packages.debian.org/experimental/amd64/libqwt-headers/filelist ,
this file is provided by the libqwt-header package. Check that it is at
version 6.2.0-1
The version you have is presumably 6.1.4-2:
https://salsa.debian.org/qt-kde-team/extras/qwt/-/blob/debian/6.1.4-2/src/qwt_plot_layout.h?ref_type=tags#L84
Even
Le 27/01/2024 à 10:39, Richard Duivenvoorde via QGIS-Developer a écrit :
On 1/27/24 00:31, Nyall Dawson wrote:
Looks like you'll need https://github.com/qgis/QGIS/pull/56033 too.
Try removing that line for now.
Yep, thanks. Merged/fixed now apparently.
Next issue with qwt:
FAILED:
src/gui/CMakeFiles/qgis_gui.dir/raster/qgsrasterhistogramwidget.cpp.o
...
In file included from
/home/richard/git/qgisqt6/src/gui/raster/qgsrasterhistogramwidget.cpp:45:
/usr/include/qwt/qwt_plot_layout.h:84:51: error: invalid conversion
from ‘int’ to ‘QwtPlotLayout::Option’ [-fpermissive]
84 | const QRectF , Options options = 0x00 );
| ^~~~
| |
| int
In file included from
/usr/include/x86_64-linux-gnu/qt6/QtCore/qglobal.h:1401,
from
/usr/include/x86_64-linux-gnu/qt6/QtGui/qtguiglobal.h:7,
from
/usr/include/x86_64-linux-gnu/qt6/QtWidgets/qtwidgetsglobal.h:7,
from
/usr/include/x86_64-linux-gnu/qt6/QtWidgets/qapplication.h:7,
from
/usr/include/x86_64-linux-gnu/qt6/QtWidgets/QApplication:1,
from
/home/richard/git/qgisqt6/src/core/qgsapplication.h:19,
from
/home/richard/git/qgisqt6/src/gui/raster/qgsrasterhistogramwidget.cpp:18:
/usr/include/x86_64-linux-gnu/qt6/QtCore/qflags.h:74:45: note:
initializing argument 1 of ‘constexpr QFlags::QFlags(Enum) [with
Enum = QwtPlotLayout::Option]’
74 | constexpr inline Q_IMPLICIT QFlags(Enum flags) noexcept :
i(Int(flags)) {}
| ~^
[344/2107] Building CXX object
src/gui/CMakeFiles/qgis_gui.dir/qgis_gui_autogen/mocs_compilation.cpp.o
Not sure if this has something to do with my qwt install from
experimental/sid:
libqwt-qt6-6.2/now 6.2.0-1 amd64 [installed,local]
Qt widgets library for technical applications (runtime, qt6)
I think the version is more or less in line with my qt6 from
testing/trixie?
pyqt6-dev/testing,now 6.6.1-2 all [installed]
development files for PyQt6
libqt6core6/testing,now 6.4.2+dfsg-21 amd64 [installed,automatic]
Qt 6 core module
Any hint to overcome this one?
Regards,
Richard Duivenvoorde
PS: let me know if this is totally not helpful, I'm just eager to know
if QGIS-qt6 is building (or I can make it build) on Debian testing/trixie
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
Le 23/01/2024 à 20:43, Richard Duivenvoorde a écrit :
Ah, oo :-)
I can reproduce now in the docker!
After seeing
src/core/providers/ogr/qgsogrprovider.cpp:3236 : (crs) [0ms] no
spatial reference found
in the debug info of that other db, I thought to try to load the data
WITHOUT telling it EPSG:31370:
So instead of:
ogr2ogr -update
"mysql:test,user=root,password=passwd,port=33060,host=127.0.0.1"
test.csv -select id -nlt point -a_srs EPSG:31370
I removed the test table and did
ogr2ogr -update
"mysql:test,user=root,password=passwd,port=33060,host=127.0.0.1"
test.csv -select id -nlt point
This fails for me with:
ERROR 1: MySQL error message:Longitude 168937.186906 is out of range in
function st_geomfromtext. It must be within (-180.00, 180.00].
Description: INSERT INTO `test` (`SHAPE` , `id`) VALUES
(ST_GeomFromText('POINT (168937.186906043 175180.302889316)',32767,
'axis-order=long-lat') , '1')
ERROR 1: Unable to write feature 1 from layer test.
ERROR 1: Terminating translation prematurely after failed
translation of layer test (use -skipfailures to skip errors)
If you don't specify a SRS at layer creation time, the CreateLayer()
logic in the MySQL driver assigns to it a "Undefined_geographic_SRS",
and thus you must set longitude, latitude in range.
If trying with a point in range for geographic coordinates, ogr2ogr
works and QGIS also happily displays it (with unknown CRS as expected)
printf 'id,WKT\n1,"POINT(2 49)"\n' > test.csv
So all in all, no issues on my side...
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
I'll try to debug the current db, and I've asked the people to create a test db (which hopefully then has the same issue :-) ) I'm not familiar with MySQL but I guess there must be some dump & restore commands that could be used to exactly replicate a setup where the bug triggers? -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Strange Mysql (spatial) behaviour: no points visible
Richard, I've failed to reproduce. Here's my testing procedure: docker run --name gdal-mysql1 -e MYSQL_ROOT_PASSWORD=passwd -e "MYSQL_ROOT_HOST=%" -p 33060:3306 -d mysql:8.0.34 mysqld --default-authentication-plugin=mysql_native_password docker exec gdal-mysql1 sh -c "echo 'CREATE DATABASE test; SELECT Version()' | mysql -uroot -ppasswd" printf 'id,WKT\n1,"POINT(168937.186906043 175180.302889316)"\n' > test.csv ogr2ogr -update "mysql:test,user=root,password=passwd,port=33060,host=127.0.0.1" test.csv -select id -nlt point -a_srs EPSG:31370 then opening it with QGIS displays the point on the canvas and the attribute table. I can also add other points. Even Le 16/01/2024 à 20:07, Richard Duivenvoorde via QGIS-Developer a écrit : No hints :-) ? I now have a live connection to the db, and can reproduce the issue here in master on Linux too. It's really strange: have a local db, I can even copy features from the attribute table to my local table, and THEN they show up ogrinfo shows exactly the same information of the db ( using -so). coordinates are fine too: select coord, ST_AsWKT(coord) from tblQgisProjecten tqp limit 1 coord |ST_AsWKT(coord) | -++ POINT (168937.186906043 175180.302889316)|POINT(168937.186906043 175180.302889316)| could it be in char encoding, or in a strange attribute value? We already tried to create a simple view with only coord and ID, but that did not show something either. As I cannot share the connection info, I'm happy to share my screen or so to show this. SELECT version(); version()| -+ 8.0.34-26| but is an upgraded db, not sure where it came from Regards, Richard Duivenvoorde On 1/12/24 15:06, Richard Duivenvoorde via QGIS-Developer wrote: Hi Devs, I was contacted by a company who after upgrading their db and QGIS did NOT see the points anymore. Note that 'all worked' when they used QGIS 3.10 (and an older mysql db) After a MySQL database upgrade, QGIS 3.10 was not able to connect anymore (apparently TLS issues), so they have to use 3.28 or higher. A small export loaded in my local db was OK in QGIS. So in a online meeting, sharing their screen I tried: - opening the db with ogrinfo: all data is visible (POINT(.)) showing correct coords and attributes - loading the table (25000 records) in QGIS: data: EPSG:31370 project EPSG:31370 : - attribute table shows all records - mapcanvas empty! - able to 'zoom to' records (Belgium), but NO points visible/selectable - creating a tiny table with only id and geom column of 3 records: mapcanvas emtpy - export the loaded (but invisible) layer to a geopackage: points AND attributes VISIBLE! - loading the data in dbeaver: all data is shown in the spatial tab There was an encoding issue when exporting to gpkg, but we tried to create a smaller table (very few columns): nothing. Anybody familiar with QGIS and Mysql has an idea what this not showing of any point/geom could be? Any hint appreciated, I plan to try to get a connection to the db myself, but hoping I just miss something obvious... Regards, Richard Duivenvoorde ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] unsubscribing is unsuccessful
Rhea, I've just unsubscribed you (from the admin interface) Even Le 16/01/2024 à 18:17, Rhea via QGIS-Developer a écrit : Hello, I’ve been trying to unsubscribe its been months now and it’s not working, i keep on receiving emails as if im still part of the list. I followed this link : https://lists.osgeo.org/mailman/listinfo/qgis-developer and i wrote my email, clicked unsubscribe, got and email and still it’s not working. Please help. Thank you. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Enhancing QGIS Development and Security Features Proposition
Rhea, * - Proposition would be a feature that allows users to limit Python console functionality based on their needs. If you have in mind to limit the set of functionality available in the Python console, that's not technically doable, at least with the CPython interpreter. All you can do is to not provide any access at all to the Python console. Python isn't a "safe" language, that is a sufficiently willing user can do arbitrary things, including crashing itself with just core functionality: $ python -c "exec(type((lambda: 0).__code__)(0, 0, 0, 0, 0, 0, b'\x053', (), (), (), '', '', 0, b''))" Segmentation fault (core dumped) cf https://stackoverflow.com/questions/42544487/how-can-i-cause-python-3-5-to-crash People have tried sandboxing Python many times, but under careful examinations, all attempts fail at some point. Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] qgis ubuntu 20.04 dependencies
Le 19/06/2023 à 20:44, Hugh Kelley a écrit : OK thanks, exactly the general guidance I was looking for. I'll look into upgrading. Is there currently a version that makes things easiest? QGIS continuous integration uses: - ubuntu 22.04 for QT5 builds: https://github.com/qgis/QGIS/blob/master/.docker/qgis3-qt5-build-deps.dockerfile - Fedora 38 for QG6 builds: https://github.com/qgis/QGIS/blob/master/.docker/qgis3-qt6-build-deps.dockerfile On Mon, Jun 19, 2023 at 2:34 PM Even Rouault wrote: Building recent QGIS versions on 20.04 is going to be quite a lot of pain since you'll need indeed to have a qt 5.15 build, and all the other auxiliary QT based dependencies build against it. I'd suggest you update to 22.04 at least. Or if you don't want to do development but can live with released versions, try conda-forge builds (cf https://plugins.qgis.org/planet/tag/conda/) Le 19/06/2023 à 20:30, Hugh Kelley via QGIS-Developer a écrit : Hi all, while working on compiling qgis on ubuntu using these instructions [0] , i've run into an error while running cmake that "QRecursiveMutex" is not found. It looks like that became available in QT 5.14. Ubuntu 20.04 installs QT 5.12. During my searches i've seen comments that imply 5.15 might not be available for 20.04. I'm trying to figure out whether I should be trying to upgrade Qt, if I have to upgrade Ubuntu, or if there is some option in the compilation process that can resolve this for me. I've installed Qt 5.15.2 to /opt/ and added it to PATH, but qtdiag still indicates i'm using 5.12.x I thought i'd start by asking here before asking on a Qt general forum in case there's a qgis specific solution. I've also set the cmake 3d option to "no" which got me around the absence of q qt 3d extras pacakge/file. So maybe there's a similar solution for this although this version problem seem more essential. Thanks for any suggestions! [0] https://github.com/qgis/QGIS/blob/master/INSTALL.md -- Hugh Kelley ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. -- Hugh Kelley -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] qgis ubuntu 20.04 dependencies
Building recent QGIS versions on 20.04 is going to be quite a lot of pain since you'll need indeed to have a qt 5.15 build, and all the other auxiliary QT based dependencies build against it. I'd suggest you update to 22.04 at least. Or if you don't want to do development but can live with released versions, try conda-forge builds (cf https://plugins.qgis.org/planet/tag/conda/) Le 19/06/2023 à 20:30, Hugh Kelley via QGIS-Developer a écrit : Hi all, while working on compiling qgis on ubuntu using these instructions [0] , i've run into an error while running cmake that "QRecursiveMutex" is not found. It looks like that became available in QT 5.14. Ubuntu 20.04 installs QT 5.12. During my searches i've seen comments that imply 5.15 might not be available for 20.04. I'm trying to figure out whether I should be trying to upgrade Qt, if I have to upgrade Ubuntu, or if there is some option in the compilation process that can resolve this for me. I've installed Qt 5.15.2 to /opt/ and added it to PATH, but qtdiag still indicates i'm using 5.12.x I thought i'd start by asking here before asking on a Qt general forum in case there's a qgis specific solution. I've also set the cmake 3d option to "no" which got me around the absence of q qt 3d extras pacakge/file. So maybe there's a similar solution for this although this version problem seem more essential. Thanks for any suggestions! [0] https://github.com/qgis/QGIS/blob/master/INSTALL.md -- Hugh Kelley ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] How does QGIS parses the shapefile so quickly
Le 27/03/2023 à 10:06, Alessandro Pasotti via QGIS-Developer a écrit : Hi, QGIS is using the GDAL/OGR library to read the shapefiles. which uses shapelib On Mon, Mar 27, 2023 at 9:51 AM 余 绍健 via QGIS-Developer wrote: Dear All, Sorry to interrupt, I'm newer to QGIS, whe I use QIGS, I find that it is so quick for QGIS to parse shapefile even it is a very big file, I try to decode the file by my own with shapelib library, but the result is disappointed, I try to learn it by myself with QGIS source code but failed, so anyone can help point the entry of decoding process? Thanks. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
[QGIS-Developer] Stricter policy for LTR backport ? (was Re: Call for emergency 3.22 release)
Hi,
I guess one point of improvement is to really implement a policy that
was discussed in the past, that is that after let's say half the total
period of support of the LTR only "critical" bugfixes are allowed into
it, with critical being for example defined as:
- fix a regression introduced in a previous patch release ("patch" as in
"major.minor.patch" version numbering scheme)
- fix a crashing bug
- fix data corruption
And also taking into account the complexity of the fix, how common the
scenario is (a crashing bug in some "obscure" functionality has not the
same severity as one at program opening), the risk of its side effects,
how well covered it is by unit tests, etc.
This will always remain subjective, and there will always be the risk
that a few-liner fix causes unanticipated issues. That's the reality of
software development.
Perhaps half the period is quite restrictive, but maybe the last 3
planned patch releases in the lifetime of a LTR should really enforce
such super strict requirements.
Looking quickly at https://github.com/qgis/QGIS/commits/release-3_22, I
can see this is far from being the case (and I should start by following
my own advice as I can see a few commits of mine that definitely don't
match the above criteria for critical bug fixes)
Even
Le 02/02/2023 à 17:19, Julien Cabieces via QGIS-Developer a écrit :
Hi,
I'm sorry that my PR led to a severe regression and thank you to the
people who take care of fixing/releasing.
I take the decision to merge this one because:
- It was one month old
- It has already been reviewed by someone else
- It was not a critical modification (I believed...)
I have to admit I rarely thoroughly review backport from bot because
they are exactly the same as the one already reviewed. And I'd probably
miss this kind of errors when reviewing too.
If you have any advice on how to behave in the backport/review process,
feel free to share.
Regards,
Julien
On Thu, 2 Feb 2023, 6:12 pm Alessandro Pasotti, wrote:
+1 for immediate release from me.
On a side note (because I sympathize thinking about what I would feel
if it was a commit of mine) : Julien is doing an amazing job, a
mistake can happen to anyone.
100% agree - it could easily have been any of us leading to this.
I think the main lesson for us is "just" that we need to be extra careful with
backports to ltr, and carefully assess whether every backport is really warranted or
not.
Nyall
On Thu, Feb 2, 2023 at 7:38 AM Andreas Neumann via QGIS-Developer
wrote:
>
> Hi Nyall,
>
> I agree, this is a pretty nasty one. Jürgen - can we release another
release soon?
>
> Thank you Even for your reflections on the code patterns involved. I trust
that the core devs will discuss and suggest improvements in our
documentation/guidelines in order to avoid similar cases in the future.
>
> Thank you for your response.
>
> Andreas
>
> On 2023-02-02 02:19, Nyall Dawson via QGIS-Developer wrote:
>
> Hi PSC/list,
>
> I came across this horrible regression during bug hunting today:
> https://github.com/qgis/QGIS/pull/51703
>
> If you load a project with any broken layers and then hover over the
> layer in the layer tree, QGIS will instantly crash. It's a regression
> caused by https://github.com/qgis/QGIS/pull/50256, and unfortunately
> that PR was backported to 3.22 and accordingly the crash present in
> the final release of 3.22.
>
> Given the extreme severity of this crash I believe we should be
> pushing out another unplanned 3.22 patch release, as we cannot leave
> the final 3.22 LTR with such a nasty regression in place.
>
> Nyall
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
Alessandro Pasotti
QCooperative: www.qcooperative.net
ItOpen: www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Call for emergency 3.22 release
Dereferencing a null dataProvider() seems to me to be a recurrent class
of (crashing) bugs (I've for sure fixed such issues several times)
My idea is probably a bit silly, but what about
https://gist.github.com/rouault/ed533d30738bda806ebeaa47f0877e2c ?
Basically:
- deprecate dataProvider() for C++, with a GCC
__attribute__((deprecated()) annotation so we can spot all occurrences
- replace it with dataProviderNullable() which returns a
QgsVectorDataProviderPtr structure that encapsulates the raw pointer and
have a get() method to get it
So you have to replace code like:
mTracksLayer->dataProvider()->addFeature( feature,
QgsFeatureSink::Flag::FastInsert );
with
if ( auto provider = mTracksLayer->dataProviderNullablePtr().get() )
{
provider->addFeature( feature, QgsFeatureSink::Flag::FastInsert );
}
Of course someone could still do
mTracksLayer->dataProviderNullablePtr().get()->addFeature( ... ) but
that should be obvious they are cheating... And I guess some linting
script could catch "dataProviderNullablePtr().get()->" buggy/risky patterns
And of course it forces checking for null in places where it cannot be,
but perhaps be on the safe side...
Another alternative I thought was to have dataProvider() return a dummy
non-null instance if mDataProvider is null, but that's probably not a
good idea.
Even
Le 02/02/2023 à 02:19, Nyall Dawson via QGIS-Developer a écrit :
Hi PSC/list,
I came across this horrible regression during bug hunting today:
https://github.com/qgis/QGIS/pull/51703
If you load a project with any broken layers and then hover over the
layer in the layer tree, QGIS will instantly crash. It's a regression
caused by https://github.com/qgis/QGIS/pull/50256, and unfortunately
that PR was backported to 3.22 and accordingly the crash present in
the final release of 3.22.
Given the extreme severity of this crash I believe we should be
pushing out another unplanned 3.22 patch release, as we cannot leave
the final 3.22 LTR with such a nasty regression in place.
Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
[QGIS-Developer] [solved] weird RuntimeError: qgis._gui cannot import type '����' from PytQT5.QtCore"
Hi, just to avoid other people losing a couple hours like me... If you update to latest master on Debian/Ubuntu, and after rebuild, you get a cryptic error like the following when importing qgis_gui: # PYTHONPATH=output/python QGIS_PREFIX_PATH=output python3 -c "from qgis._gui import *" Traceback (most recent call last): File "", line 1, in RuntimeError: qgis._gui cannot import type '' from PyQt5.QtCore Then the fix is to install python3-pyqt5.qtmultimedia. Cf https://github.com/qgis/QGIS/commit/324dac16696bff43d637920733eb5ee4cb7b3ddf Thanks to Alessandro for hinting me on the right direction. I'd wish PyQt would output a better error message... Even -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Fwd: [gdal-dev] Announcing SOZip: Seek-Optimized profile for the .zip format
> Very cool project, Even! (Why not "Cloud-Optimized" though? ) The term is heavily loaded and quite vague. SOZip doesn't make something cloud unfriendly magically cloud friendly. If you need to seek at 100 different locations in a (uncompressed) file, you'll need to also seek at 100 locations in its SOZip compressed version (which is excruciatingly slow for a regular non-optimized ZIP). SOZip will shine most for local use cases, where this seeking in the underlying compressed stream is a cheap operation. For remote use, your mileage may vary. -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Fwd: [gdal-dev] Announcing SOZip: Seek-Optimized profile for the .zip format
Le 09/01/2023 à 16:54, Richard Duivenvoorde a écrit : Hi Even, Cool! Out of curiosity: I thought sqlite/geopackage was already relatively 'skinny' packed? Am I wrong? Well, SQLite does some packing of integer or floating-point values, but nothing on strings or blobs (which means no compression o the WKB extended geometries of GeoPackage). You can get easily a 3x compression ratio when zipping GeoPackage vector files (of course depends a lot on what you put in them) Anybody has experience with zipping "lrge zipped GeoPackages"? ;-) Is that useful? I just tested a 885MB mbtiles file (I know not geopackage...but still sqlite isnt't it?), and that ended up in 868MB. I have successfully tested a ~30 GB uncompressed GeoPackage vector file compressed as ~10 GB as SOZip. MBTiles holds PNG or JPEG tiles which are already compressed, so no surprise that the Deflate compression of ZIP doesn't help. OR... are we talking about sets of geopackages? The SOZip spec is all about laaarge-files-in-zip. Considerations of many-files-in-zip are not covered by it (they are orthogonal) -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
[QGIS-Developer] Fwd: [gdal-dev] Announcing SOZip: Seek-Optimized profile for the .zip format
Sorry for cross-posting, but very relevant topic for QGIS. To make it short, pending compressing .zip files in the SOZip way, it is possible to directly read lrge zipped GeoPackage files (or Shapefiles for nostalgic) from QGIS without prior decompression Even Message transféré Sujet : [gdal-dev] Announcing SOZip: Seek-Optimized profile for the .zip format Date : Mon, 9 Jan 2023 15:19:07 +0100 De :Even Rouault Pour : gdal-...@lists.osgeo.org Hi, It is my pleasure to announce ( https://github.com/sozip/sozip-spec/blob/master/blog/01-announcement.md ) the initial release of the specification ( https://github.com/sozip/sozip-spec/blob/master/sozip_specification.md ) for the SOZip (Seek-Optimized Zip) profile to the ZIP file format, as well as its GDAL implementation. What is SOZip ? -- A Seek-Optimized ZIP file (SOZip) is a ZIP file that contains one or several Deflate-compressed files that are organized and annotated such that a SOZip-aware reader can perform very fast random access (seek) within a compressed file. SOZip makes it possible to access large compressed files directly from a .zip file without prior decompression. It is not a new file format, but a profile of the existing ZIP format, done in a fully backward compatible way. ZIP readers that are non-SOZip aware can read a SOZip-enabled file normally and ignore the extended features that support efficient seek capability. Use cases -- The SOZip specification is intended to be general purpose / not domain specific. It was first developed to serve geospatial use cases, which commonly have large compressed files inside of ZIP archives. In particular, it makes it possible for users to read large GIS files using the Shapefile, GeoPackage or FlatGeobuf formats (which have no native provision for compression) compressed in .zip files without prior decompression. Efficient random access and selective decompression are a requirement to provide acceptable performance in many usage scenarios: spatial index filtering, access to a feature by its identifier, etc. Performance -- SOZip is efficient: * The overhead of using a file from a SOZip archive, compared to using it uncompressed, is of the order of 10% for common read operations. * Generation of a SOZip file can be much faster than regular ZIP generation when using multithreading. * SOZip files are typically only ~ 5% larger than regular ZIPs (dependent on content, and chunk size) Have a look at benchmarking results: https://github.com/sozip/sozip-spec/blob/master/README.md#benchmarking Other ZIP related specification -- The SOZip GitHub organization also hosts the KeyValuePairs extra-field specification ( https://github.com/sozip/keyvaluepairs-spec/blob/master/zip_keyvalue_extra_field_specification.md ), to be able to encode arbitrary key-value pairs of metadata associated with a file within a ZIP. For example to store the Content-Type of a file. How does this relate to GDAL ? --- Pull request https://github.com/OSGeo/gdal/pull/7042 has been submitted with the following enhancements: * The /vsizip/ virtual file system uses the SOZip index to perform fast random access within a compressed SOZip-enabled file. * The Shapefile and GPKG drivers can directly generate SOZip-enabled .shz/.shp.zip or .gpkg.zip files. * Addition of the CPLAddFileInZip() C function that can compress a file and add it to an new or existing ZIP file, and enable the SOZip optimization when relevant. * The existed VSIGetFileMetadata() method can be called on a filename of the form /vsizip/path/to/the/file.zip/path/inside/the/zip/file and with domain = "ZIP" to get information if a SOZip index is available for that file. * The sozip (https://github.com/rouault/gdal/blob/sozip/doc/source/programs/sozip.rst) new command line utility can be used to create a seek-optimized ZIP file, to append files to an existing ZIP file, list the contents of a ZIP file and display the SOZip optimization status or validate a SOZip file. Best regards, Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-...@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Offering QGIS installers through Windows store
Hi, I've said it a few times, but I'll repeat myself one more time as it can't hurt :-) I'd strongly advise for QGIS Windows installer to rely on the massive Conda-Forge packaging effort. OSGeo4W is a bit of a masochist effort when one sees that it packages everything from scratch, a bit like if QGIS would take the responsibility of creating a Linux distro from scratch, whereas leveraging Conda-Forge would enable QGIS to benefit from > 90% of its dependencies already packages by a much bigger community. The QGIS community could then concentrate on the QGIS build recipee, and probably a few other packages specific to QGIS. Then the QGIS installer itself would mostly bundle a subset of Conda-Forge dependencies pinned at the wished versions. Even Le 16/12/2022 à 16:59, Julien Moura via QGIS-Developer a écrit : Hello, I'm working on the packaging purpose at Oslandia. Because we are committed to pushing QGIS in the most industrial information systems, at Oslandia we have been very interested in QGIS deployment for a few years now. We have reused and actively contributed to many projects: OSGeo4W v1 mirror, NSIS and PowerShell recipes, presentations at FOSS4Gs (or at QGIS conferences), etc. Since this year, we are making efforts to de-correlate software deployment and end-user configurations (profiles): the QGIS Deployment Toolbelt (https://guts.github.io/qgis-deployment-cli/) has been created with this goal. It is still a young project and also it is one project among many others. We are obviously very interested in this project of packaging QGIS on Windows Store, a topic that we already discussed internally last year and that I discussed namely with Régis Haubourg in Florence. We obviously have some question marks and discussion points to make sure that all the process is in line with community and open-source governance. Still, it will be quite a challenge and we think it is a good direction to follow. As noted by others, this seems a huge undertaking though, and uncertainty paves the road and make this goal a distant future. We could start with gathering and referencing all resources dedicated to packaging, which would sum up as a website ( packaging.qgis.org ?) with all documentation regarding OSGeo4W recipes ( beyond the aging OSGeo trac ), and other packaging methods, like customized .deb packaging. The goal is to structure and promote knowledge on this topic to attract packager in addition to developers. And then we would also create packaging process documentation for Windows Store. Regards, Julien M Le 23/11/2022 à 10:41, Even Rouault via QGIS-Developer a écrit : - There is a requirement for every library in the package to be "UWP compatible". At that time, libpq (postgresql) didn't have that, so I stopped. It's hard to say how many other packages will also need special attention. [1] Interesting. I know that people have contributed in the past fixes to make PROJ *compilable* with UWP, but as we don't have a CI target for that, it is unknown how it behaves at runtime. GDAL likely fails to build with UWP according to https://github.com/OSGeo/gdal/issues/1201 Without a CI target that can be hooked into github action (maybe that exists. didn't check), UWP compatibility on the long term is going to be hard to achieve. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] prepare_commit.sh freezes console
Hi Harrissou, "q" should do it, as the diff is displayed with the "less" utility, and it's the shortcut to exit from it Even Le 16/12/2022 à 06:18, Delaz J via QGIS-Developer a écrit : Hi, Whenever I run prepare_commit.sh and there are changes, I fail to find how to get back to the command prompt. Whatever I do (and I can be quite inventive on keyboard shortcuts), nothing more will happen in the tab of the terminal I ran the command. The only option I have is to kill that tab and continue in a new tab. So my question is: How do you take control of your terminal after you run prepare_commit.sh? I'm on Ubuntu 22.04 if it makes any difference. Thanks, Harrissou ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Long filename/filepath causing git issues
Would it be too crazy to shrink down some offending name to prevent error when switching to more recent branches? I can submit a PR if needed. Seems like a good idea. You'll likely have to change references to that file in the test suite. -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Offering QGIS installers through Windows store
- There is a requirement for every library in the package to be "UWP compatible". At that time, libpq (postgresql) didn't have that, so I stopped. It's hard to say how many other packages will also need special attention. [1] Interesting. I know that people have contributed in the past fixes to make PROJ *compilable* with UWP, but as we don't have a CI target for that, it is unknown how it behaves at runtime. GDAL likely fails to build with UWP according to https://github.com/OSGeo/gdal/issues/1201 Without a CI target that can be hooked into github action (maybe that exists. didn't check), UWP compatibility on the long term is going to be hard to achieve. -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Best way to import gdal, ogr, osr ?
Le 17/10/2022 à 12:36, Raymond Nijssen a écrit : Thank you Even. And that will always be available in QGIS? Without any extra (gdal) installations? yes, I think so. This is needed by the GDAL processing tools, which AFAIK are availalable out-of-the box in the standalone Windows installer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Best way to import gdal, ogr, osr ?
You can just use from osgeo import ogr from osgeo import gdal from osgeo import osr That will work with any GDAL version of the last 15 years or so. Le 17/10/2022 à 10:52, Raymond Nijssen via QGIS-Developer a écrit : Dear developers, What is the best way to import ogr, gdal and osr in my pyqgis script? This used to work in older QGIS versions: import gdal import ogr import osr Now I've changed that to: try: import ogr import gdal import osr except ModuleNotFoundError: from osgeo import ogr from osgeo import gdal from osgeo import osr But it's still unclear to me what would be the way to make sure my script will run in anyone else's installation (Different OS and QGIS version). Maybe there's something like this: from qgis import ogr ??? Kind regards, Raymond ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] gdal2xyz.bat not accepting -b any band number
Le 24/06/2022 à 02:29, Alexandre Neto via QGIS-Developer a écrit : I am testing the GDAL tools on QGIS 3.22.8 on Windows. When running gdal2xyz with a single band raster I am getting the following error: C:\Users\testing_user>gdal2xyz.bat -b 1 C:/Users/testing_user/Desktop/gdal_commands_test/mdt_clipped.tif OUTPUT.csv usage: gdal2xyz.py [-h] [-skip] [-srcwin xoff yoff xsize ysize] [-b band [band ...]] [-allbands] [-csv] [-skipnodata] [-srcnodata [SRC_NODATA ...]] [-dstnodata [DST_NODATA ...]] src_dataset dst_dataset gdal2xyz.py: error: argument -b/-band/--band: invalid int value: 'C:/Users/testing_user/Desktop/gdal_commands_test/mdt_clipped.tif' If I remove the -b option it works. Windows Is now on GDAL 3.5.0. On my linux machine that still uses GDAL 3.0.4 it works fine. Bug report? In here or in GDAL? to GDAL. It looks like the -b option is a bit too greedy in consuming arguments. A workaround is to put the -b 1 option at the end of the command line. Alexandre Neto ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer -- http://www.spatialys.com My software is free, but my time generally not. ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Qgis server: terrible performance with Qgis 3.26
Use sysprof or just run under a debugger and interrupt regularly should
help you spot in which method(s) most time is spent
Le 20/06/2022 à 18:48, David Marteau via QGIS-Developer a écrit :
I used the following test:
readflags = Qgis.ProjectReadFlags() readflags |=
Qgis.ProjectReadFlag.TrustLayerMetadata|Qgis.ProjectReadFlag.DontLoad3DViews|Qgis.ProjectReadFlag.DontLoadProjectStyles
prj = QgsProject()%timeit -n 1
prj.read(f"{datapath}/france_parts3_26.qgs",readflags)
105 ms ± 18.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
This improve the result but we are still much higher that the loading
time of 3.24: 15ms compared to 105ms
Le 20/06/2022 à 18:38, Nyall Dawson a écrit :
On Mon, 20 June 2022, 11:42 pm David Marteau, wrote:
Hi,
> You should set the other new read optimisation flags here --
see
https://github.com/qgis/QGIS/blob/88ebb3f2f513dbce5a775f50e51ea0117f859c65/src/core/qgis.h#L1962
These are already available also in 3.24 and we use the
`TrustLayerMetadata` in the above test. For server, the other
flags are not applicable (Getprint requests need print layouts ).
So that mean we are comparing 3.24 to 3.26 with same level of
read optimisation and that does not change the fact that, for
that very same level of read optimisation, project's loading is
incredibly slower in 3.26.
You'll need to set the new flags relating to skipping the 3d views
and project style loading in order to compare properly. Neither of
those things were occurring in 3.24, and both have an associated
cost. And neither are relevant for server.
Nyall
Le 20/06/2022 à 15:10, Nyall Dawson a écrit :
On Mon, 20 June 2022, 9:37 pm David Marteau via QGIS-Developer,
wrote:
Hi,
I have noticed a huge drop of performance while testing Qgis
server with Qgis 3.26, from 10x to 30x slower as usual.
Trying to reduce the problem I have found a huge increase of
time spend reading the project in 3.26 compared to 3.24.
Here some benchmark with very simple project from the test
data set. Note: the project has been updated with qgis 3.26:
Qgis 3.24:
==
readflags = QgsProject.ReadFlags()readflags |=
QgsProject.FlagTrustLayerMetadata
prj = QgsProject()
%timeit -n 1 prj.read(f"{datapath}/france_parts.qgs",readflags)
> 15.7 ms ± 205 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)
Qgis 3.26:
==
readflags = QgsProject.ReadFlags() readflags |=
QgsProject.FlagTrustLayerMetadata
You should set the other new read optimisation flags here -- see
https://github.com/qgis/QGIS/blob/88ebb3f2f513dbce5a775f50e51ea0117f859c65/src/core/qgis.h#L1962
Nyall
prj = QgsProject()
%timeit -n 1 prj.read(f"{datapath}/france_parts.qgs",readflags)
195 ms ± 44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
There is a factor x10 between Qgis 3.24 et Qgis 3.26 for the
exact same project.
This has serious impact on requests: landing page catalog
request time increased from approximatevely 110ms to more
than 3.0s with our test data set.
David Marteau
www.3liz.com
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info:
https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:
https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info:https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe:https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
http://www.spatialys.com
My software is free, but my time generally not.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Trouble compiling QGIS
The Valgrind output is actually interesting: see ==49441== by 0x150193E6: ??? (in /usr/lib/x86_64-linux-gnu/libproj.so.15.3.1) [...] ==49441== by 0x9C3FDC6: ??? (in /usr/local/lib/libproj.so.22.1.0) So you're linking to 2 PROJ versions. That would have been my first bet, but you didn't mention a custom PROJ version :-) So the custom GDAL links to the custom PROJ. You'll have now to check which of QGIS and/or Spatialite links to the system PROJ instead of the custom one. Le 03/09/2021 à 16:30, Andreas Neumann a écrit : Hi Even, Thank you for your reply. I checked with the ldd about gdal/geos/proj - and they all seem to link to the self-compiled versions in /usr/local/lib gdal: xxx@t480:~/dev/QGIS/build$ ldd /usr/local/lib/libgdal.so.29 linux-vdso.so.1 (0x7fffb930d000) libcrypto.so.1.1 => /lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x7f250960) libjson-c.so.4 => /lib/x86_64-linux-gnu/libjson-c.so.4 (0x7f25095ee000) libfreexl.so.1 => /lib/x86_64-linux-gnu/libfreexl.so.1 (0x7f25095e3000) *libgeos_c.so.1 => /usr/local/lib/libgeos_c.so.1 (0x7f25095a5000)* libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x7f2509577000) libxerces-c-3.2.so => /lib/x86_64-linux-gnu/libxerces-c-3.2.so (0x7f25091e4000) libpq.so.5 => /usr/local/pgsql/lib/libpq.so.5 (0x7f2509193000) libzstd.so.1 => /lib/x86_64-linux-gnu/libzstd.so.1 (0x7f25090ea000) libproj.so.22 => /usr/local/lib/libproj.so.22 (0x7f2508d4) libsqlite3.so.0 => /lib/x86_64-linux-gnu/libsqlite3.so.0 (0x7f2508c17000) libjpeg.so.8 => /lib/x86_64-linux-gnu/libjpeg.so.8 (0x7f2508b92000) libdeflate.so.0 => /lib/x86_64-linux-gnu/libdeflate.so.0 (0x7f2508b74000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f2508b56000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f2508b33000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f2508b2d000) libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x7f2508aba000) libcurl.so.4 => /lib/x86_64-linux-gnu/libcurl.so.4 (0x7f2508a29000) libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x7f250886f000) libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x7f250868b000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7f250853c000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f250834a000) /lib64/ld-linux-x86-64.so.2 (0x7f250afe7000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x7f250832f000) *libgeos-3.9.1.so => /usr/local/lib/libgeos-3.9.1.so (0x7f250811b000)* libcurl-gnutls.so.4 => /lib/x86_64-linux-gnu/libcurl-gnutls.so.4 (0x7f250808c000) libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x7f2507ea4000) libssl.so.1.1 => /lib/x86_64-linux-gnu/libssl.so.1.1 (0x7f2507e11000) libtiff.so.5 => /lib/x86_64-linux-gnu/libtiff.so.5 (0x7f2507d9) libnghttp2.so.14 => /lib/x86_64-linux-gnu/libnghttp2.so.14 (0x7f2507d67000) libidn2.so.0 => /lib/x86_64-linux-gnu/libidn2.so.0 (0x7f2507d46000) librtmp.so.1 => /lib/x86_64-linux-gnu/librtmp.so.1 (0x7f2507d24000) libssh.so.4 => /lib/x86_64-linux-gnu/libssh.so.4 (0x7f2507cb6000) libpsl.so.5 => /lib/x86_64-linux-gnu/libpsl.so.5 (0x7f2507ca3000) libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x7f2507c56000) libldap_r-2.4.so.2 => /lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x7f2507c0) liblber-2.4.so.2 => /lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x7f2507bef000) libbrotlidec.so.1 => /lib/x86_64-linux-gnu/libbrotlidec.so.1 (0x7f2507bdf000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x7f2507bb6000) libnettle.so.7 => /lib/x86_64-linux-gnu/libnettle.so.7 (0x7f2507b7c000) libgnutls.so.30 => /lib/x86_64-linux-gnu/libgnutls.so.30 (0x7f25079a6000) libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x7f2505ee5000) libwebp.so.6 => /lib/x86_64-linux-gnu/libwebp.so.6 (0x7f2505e79000) libjbig.so.0 => /lib/x86_64-linux-gnu/libjbig.so.0 (0x7f2505c6b000) libunistring.so.2 => /lib/x86_64-linux-gnu/libunistring.so.2 (0x7f2505ae9000) libhogweed.so.5 => /lib/x86_64-linux-gnu/libhogweed.so.5 (0x7f2505ab2000) libgmp.so.10 => /lib/x86_64-linux-gnu/libgmp.so.10 (0x7f2505a2e000) libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3 (0x7f250594f000) libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x7f250591e000) libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x7f2505917000) libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0 (0x7f2505908000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
Re: [QGIS-Developer] issue transforming coords from EPSG:4258 'ETRS89' to 4326
Alessandro, the truth (and sometimes issues) is in projinfo: $ projinfo -s EPSG:4258 -t EPSG:4326 Candidate operations found: 1 - Operation No. 1: EPSG:1149, ETRS89 to WGS 84 (1), 1.0 m, Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Moldova; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain; Sweden; Switzerland; United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State. PROJ string: +proj=noop WKT2:2019 string: COORDINATEOPERATION["ETRS89 to WGS 84 (1)", VERSION["EPSG-eur"], SOURCECRS[ GEOGCRS["ETRS89", ENSEMBLE["European Terrestrial Reference System 1989 ensemble", MEMBER["European Terrestrial Reference Frame 1989"], MEMBER["European Terrestrial Reference Frame 1990"], MEMBER["European Terrestrial Reference Frame 1991"], MEMBER["European Terrestrial Reference Frame 1992"], MEMBER["European Terrestrial Reference Frame 1993"], MEMBER["European Terrestrial Reference Frame 1994"], MEMBER["European Terrestrial Reference Frame 1996"], MEMBER["European Terrestrial Reference Frame 1997"], MEMBER["European Terrestrial Reference Frame 2000"], MEMBER["European Terrestrial Reference Frame 2005"], MEMBER["European Terrestrial Reference Frame 2014"], ELLIPSOID["GRS 1980",6378137,298.257222101, LENGTHUNIT["metre",1]], ENSEMBLEACCURACY[0.1]], PRIMEM["Greenwich",0, ANGLEUNIT["degree",0.0174532925199433]], CS[ellipsoidal,2], AXIS["geodetic latitude (Lat)",north, ORDER[1], ANGLEUNIT["degree",0.0174532925199433]], AXIS["geodetic longitude (Lon)",east, ORDER[2], ANGLEUNIT["degree",0.0174532925199433]], ID["EPSG",4258]]], TARGETCRS[ GEOGCRS["WGS 84", ENSEMBLE["World Geodetic System 1984 ensemble", MEMBER["World Geodetic System 1984 (Transit)"], MEMBER["World Geodetic System 1984 (G730)"], MEMBER["World Geodetic System 1984 (G873)"], MEMBER["World Geodetic System 1984 (G1150)"], MEMBER["World Geodetic System 1984 (G1674)"], MEMBER["World Geodetic System 1984 (G1762)"], MEMBER["World Geodetic System 1984 (G2139)"], ELLIPSOID["WGS 84",6378137,298.257223563, LENGTHUNIT["metre",1]], ENSEMBLEACCURACY[2.0]], PRIMEM["Greenwich",0, ANGLEUNIT["degree",0.0174532925199433]], CS[ellipsoidal,2], AXIS["geodetic latitude (Lat)",north, ORDER[1], ANGLEUNIT["degree",0.0174532925199433]], AXIS["geodetic longitude (Lon)",east, ORDER[2], ANGLEUNIT["degree",0.0174532925199433]], ID["EPSG",4326]]], METHOD["Geocentric translations (geog2D domain)", ID["EPSG",9603]], PARAMETER["X-axis translation",0, LENGTHUNIT["metre",1], ID["EPSG",8605]], PARAMETER["Y-axis translation",0, LENGTHUNIT["metre",1], ID["EPSG",8606]], PARAMETER["Z-axis translation",0, LENGTHUNIT["metre",1], ID["EPSG",8607]], OPERATIONACCURACY[1.0], USAGE[ SCOPE["(null/copy) Approximation for medium and low accuracy applications assuming equality between plate-fixed static and earth-fixed dynamic CRSs, ignoring static/dynamic CRS differences."], AREA["Europe - onshore and offshore: Albania; Andorra; Austria; Belgium; Bosnia and Herzegovina; Bulgaria; Croatia; Cyprus; Czechia; Denmark; Estonia; Faroe Islands; Finland; France; Germany; Gibraltar; Greece; Hungary; Ireland; Italy; Kosovo; Latvia; Liechtenstein; Lithuania; Luxembourg; Malta; Moldova; Monaco; Montenegro; Netherlands; North Macedonia; Norway including Svalbard and Jan Mayen; Poland; Portugal; Romania; San Marino; Serbia; Slovakia; Slovenia; Spain; Sweden; Switzerland; United Kingdom (UK) including Channel Islands and Isle of Man; Vatican City State."], BBOX[32.88,-16.1,84.73,40.18]], ID["EPSG",1149], REMARK["ETRS89 and WGS 84 are realizations of ITRS coincident to within 1 metre. This transformation has an accuracy equal to the coincidence figure."]] So basically ETRS89 and WGS 84 are considered the same with a 1 metre
