Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On 3/23/22 21:08, Markus Neteler wrote: Hi, On Thu, Feb 24, 2022 at 9:33 AM Sebastiaan Couwenberg wrote: Now that 7.8.7 has been released, I started looking into updating the Debian package to 8.0.x which requires rebuilding libgdal-grass and qgis. Initial tests with libgdal-grass are not promising. Both gdalinfo and ogrinfo show errors because not all libraries are linked. We saw this too when 7.2 was released: https://trac.osgeo.org/gdal/ticket/6785 The attached patch helps to some extent, but does not resolve all issues. Did you propose the path to GDAL in GitHub yet? No, there was no need after all. I was led astray by the error from the grass sqlite driver which doesn't need changes in gdal-grass. I just merged and backported https://github.com/OSGeo/grass/pull/2269 Perhaps that helps, too? It probably also solves this issue: https://github.com/qgis/QGIS/pull/47576 Perhaps. libgdal-grass and qgis built with grass 8 have only just been available in unstable. The transition to proj 9 is currently ongoing so and qgis hasn't been rebuilt yet, but gdal has, so qgis isn't very usualy right now with dependencies on different versions of proj. Should we perhaps take this opportunity to move the grass libraries to default library search paths as raised on the debian-gis list? https://lists.debian.org/debian-gis/2021/12/msg00023.html Others have answered positively, so the question is when to introduce needed changes. Has any work been done testing and patching gdal-grass and qgis for GRASS 8 yet? - I have successfully locally compiled the gdal-grass driver - also added a COPR for Fedora RPMs: https://copr.fedorainfracloud.org/coprs/neteler/gdal-grass-plugin/ - Nyall and Sandro have patched QGIS to support GRASS GIS 8, some weeks ago More tests are welcome. This seems to be sufficient so far: https://salsa.debian.org/debian-gis-team/gdal-grass/-/commit/0192b0418e04d79678bdd5c4c377b8cece5e0939 We mostly need gdal-grass for gdal 3.5 now. Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
Hi, On Thu, Feb 24, 2022 at 9:33 AM Sebastiaan Couwenberg wrote: > > Now that 7.8.7 has been released, I started looking into updating the > Debian package to 8.0.x which requires rebuilding libgdal-grass and qgis. > > Initial tests with libgdal-grass are not promising. Both gdalinfo and > ogrinfo show errors because not all libraries are linked. We saw this > too when 7.2 was released: https://trac.osgeo.org/gdal/ticket/6785 > > The attached patch helps to some extent, but does not resolve all > issues. Did you propose the path to GDAL in GitHub yet? > ogrinfo still shows errors: > > # ogrinfo -ro -so /tmp/spearfish60_grass7/PERMANENT/vector/roads/head > > Warning 1: GRASS warning: GISBASE environment variable was not set, using: > /usr/lib/grass80 > /usr/lib/grass80/driver/db/sqlite: error while loading shared libraries: > libgrass_dbmidriver.8.0.so: cannot open shared object file: No such file > or directory > dbmi: Protocol error > Warning 1: GRASS warning: Unable to start driver > ERROR 1: Cannot open database sqlite by driver > /tmp/spearfish60_grass7/PERMANENT/sqlite/sqlite.db, check if GISBASE > environment variable is set, the driver is available and the database > is accessible. > INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head' >using driver `OGR_GRASS' successful. > 1: roads (Line String) > > The grass drivers lacks the RPATH that was set for ogr_GRASS: > > # objdump -x /usr/lib/gdalplugins/ogr_GRASS.so | egrep "NEEDED|RUNPATH" >NEEDED libgrass_vector.8.0.so >NEEDED libgrass_dig2.8.0.so >NEEDED libgrass_dgl.8.0.so >NEEDED libgrass_rtree.8.0.so >NEEDED libgrass_linkm.8.0.so >NEEDED libgrass_dbmiclient.8.0.so >NEEDED libgrass_dbmibase.8.0.so >NEEDED libgrass_dbmidriver.8.0.so >NEEDED libgrass_dbstubs.8.0.so >NEEDED libgrass_raster.8.0.so >NEEDED libgrass_imagery.so >NEEDED libgrass_gproj.8.0.so >NEEDED libgrass_gmath.8.0.so >NEEDED libgrass_gis.8.0.so >NEEDED libgrass_datetime.8.0.so >NEEDED libgrass_btree2.8.0.so >NEEDED libgrass_ccmath.8.0.so >NEEDED libgdal.so.30 >NEEDED libstdc++.so.6 >NEEDED libm.so.6 >NEEDED libgcc_s.so.1 >NEEDED libc.so.6 >RUNPATH /usr/lib/grass80/lib > > # objdump -x /usr/lib/grass80/driver/db/sqlite | egrep "NEEDED|RUNPATH" >NEEDED libgrass_dbmidriver.8.0.so >NEEDED libgrass_dbmibase.8.0.so >NEEDED libgrass_gis.8.0.so >NEEDED libsqlite3.so.0 >NEEDED libc.so.6 I just merged and backported https://github.com/OSGeo/grass/pull/2269 Perhaps that helps, too? It probably also solves this issue: https://github.com/qgis/QGIS/pull/47576 > Should we perhaps take this opportunity to move the grass libraries to > default library search paths as raised on the debian-gis list? > > https://lists.debian.org/debian-gis/2021/12/msg00023.html Others have answered positively, so the question is when to introduce needed changes. > Has any work been done testing and patching gdal-grass and qgis for > GRASS 8 yet? - I have successfully locally compiled the gdal-grass driver - also added a COPR for Fedora RPMs: https://copr.fedorainfracloud.org/coprs/neteler/gdal-grass-plugin/ - Nyall and Sandro have patched QGIS to support GRASS GIS 8, some weeks ago More tests are welcome. Markus > Kind Regards, > > Bas > > -- > GPG Key ID: 4096R/6750F10AE88D4AF1 > Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D > 4AF1___ > grass-dev mailing list > grass-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/grass-dev ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
> On 25 Feb 2022, at 15:43, Vaclav Petras wrote: > > > > On Fri, 25 Feb 2022 at 09:07, Sebastiaan Couwenberg > wrote: > > If we want to stop using --prefix=/usr/lib and have FHS complianance > while also having the shared libraries in the default library search > path several changes in GRASS will be required. > > The resulting structure should result in something like: > > /etc/grassconfiguration files > /usr/bin executables > /usr/lib shared libraries > /usr/lib/python3/dist-packages/grass python package > /usr/libexec/grassexecutable helpers > /usr/share/grass architecture independent files > /usr/share/manmanual pages > > Makes sense. Any good examples of how to do this with Autotools, esp. given > that macOS and Windows still need a single directory? I’d warmly support this new suggested structure. And the sooner the better! It would in general work fine on macOS too, only prefix need to be "/usr/local” (no need to put everything in on directory), and the Python package need an alternative path. Best, Nicklas ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On 2/24/22 09:33, Sebastiaan Couwenberg wrote: # ogrinfo -ro -so /tmp/spearfish60_grass7/PERMANENT/vector/roads/head Warning 1: GRASS warning: GISBASE environment variable was not set, using: /usr/lib/grass80 /usr/lib/grass80/driver/db/sqlite: error while loading shared libraries: libgrass_dbmidriver.8.0.so: cannot open shared object file: No such file or directory dbmi: Protocol error Warning 1: GRASS warning: Unable to start driver ERROR 1: Cannot open database sqlite by driver /tmp/spearfish60_grass7/PERMANENT/sqlite/sqlite.db, check if GISBASE environment variable is set, the driver is available and the database is accessible. INFO: Open of `/tmp/spearfish60_grass7/PERMANENT/vector/roads/head' using driver `OGR_GRASS' successful. 1: roads (Line String) FWIW grass (7.8.7-1) in unstable, grass (7.8.6-1) in testing, and grass (7.6.0-1) in buster have the same issue. So the good news it's not a regression in 8.0.x. Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On Fri, 25 Feb 2022 at 09:07, Sebastiaan Couwenberg wrote: > > If we want to stop using --prefix=/usr/lib and have FHS complianance > while also having the shared libraries in the default library search > path several changes in GRASS will be required. > > The resulting structure should result in something like: > > /etc/grassconfiguration files > /usr/bin executables > /usr/lib shared libraries > /usr/lib/python3/dist-packages/grass python package > /usr/libexec/grassexecutable helpers > /usr/share/grass architecture independent files > /usr/share/manmanual pages > Makes sense. Any good examples of how to do this with Autotools, esp. given that macOS and Windows still need a single directory? > This assumes that the grass shared libraries should not be considered > private which does seem to be the case with their use by libgdal-grass > being the exception. > We keep a stable API anyway for custom user executables such as addons, so not private in that sense. Libraries can also be accessed from Python through an API which wraps ctypes calls. I'm not sure what is the current situation in QGIS, but at least historically, that would be another exception, so perhaps not an exception at all, but a rule. The executables like g.region are somewhat internal, so that's perhaps the /usr/libexec/grass group. Alternatively the GRASS executables need to have RPATH set, e.g. with: > > -Wl,-rpath,/usr/lib/grass80/lib > I'm not sure what are the risks involved here. I have seen -rpath-link used in the code, so that's not good enough. > To remove the need for changing the library search path. This is the > road of least resistance. > ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On 2/25/22 14:20, Vaclav Petras wrote: On Fri, 25 Feb 2022 at 01:25, Sebastiaan Couwenberg wrote: On 2/24/22 15:23, Vaclav Petras wrote: On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: Should we perhaps take this opportunity to move the grass libraries to default library search paths as raised on the debian-gis list? https://lists.debian.org/debian-gis/2021/12/msg00023.html I vote yes. I think the reasons to do this are no longer valid. Doing it the standard way seems to me to be the best way forward. Any suggestions/PRs? Not using --prefix=/usr/lib will violate FHS, so not an option for the Debian package. Using --prefix now results in every file from GRASS being installed into --prefix, so I assume the current --prefix behavior is wrong from Debian perspective, yes? Any suggestions on correct behavior? The 8.x prefix behavior is no different from 7.x. If we want to stop using --prefix=/usr/lib and have FHS complianance while also having the shared libraries in the default library search path several changes in GRASS will be required. The resulting structure should result in something like: /etc/grassconfiguration files /usr/bin executables /usr/lib shared libraries /usr/lib/python3/dist-packages/grass python package /usr/libexec/grassexecutable helpers /usr/share/grass architecture independent files /usr/share/manmanual pages This assumes that the grass shared libraries should not be considered private which does seem to be the case with their use by libgdal-grass being the exception. Alternatively the GRASS executables need to have RPATH set, e.g. with: -Wl,-rpath,/usr/lib/grass80/lib To remove the need for changing the library search path. This is the road of least resistance. Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On Fri, 25 Feb 2022 at 01:25, Sebastiaan Couwenberg wrote: > On 2/24/22 15:23, Vaclav Petras wrote: > > On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: > >> Should we perhaps take this opportunity to move the grass libraries to > >> default library search paths as raised on the debian-gis list? > >> > >>https://lists.debian.org/debian-gis/2021/12/msg00023.html > > > > I vote yes. I think the reasons to do this are no longer valid. Doing it > > the standard way seems to me to be the best way forward. Any > > suggestions/PRs? > > Not using --prefix=/usr/lib will violate FHS, so not an option for the > Debian package. > Using --prefix now results in every file from GRASS being installed into --prefix, so I assume the current --prefix behavior is wrong from Debian perspective, yes? Any suggestions on correct behavior? ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On 2/25/22 08:33, Markus Neteler wrote: On Fri, Feb 25, 2022 at 7:24 AM Sebastiaan Couwenberg wrote: On 2/24/22 15:23, Vaclav Petras wrote: On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: Should we perhaps take this opportunity to move the grass libraries to default library search paths as raised on the debian-gis list? https://lists.debian.org/debian-gis/2021/12/msg00023.html I vote yes. I think the reasons to do this are no longer valid. Doing it the standard way seems to me to be the best way forward. Any suggestions/PRs? Not using --prefix=/usr/lib will violate FHS, so not an option for the Debian package. Are users of libgdal-grass expected to set LD_LIBRARY_PATH like the grass executable does to find the grass libraries? In https://grasswiki.osgeo.org/wiki/Compile_and_install_GDAL-GRASS_plugin#Compilation_and_Installation_of_the_GDAL-GRASS_plugin the creation of a LD_LIBRARY_PATH file is suggested: /etc/ld.so.conf.d/grass.conf That violates the Debian policy, and triggers a lintian error: " It is not okay to install libraries in a different directory and then modify the run-time link path. Shared libraries should go into /usr/lib. Alternatively, they can require binaries to set the RPATH to find the library. " https://lintian.debian.org/tags/package-modifies-ld.so-search-path Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On Fri, Feb 25, 2022 at 7:24 AM Sebastiaan Couwenberg wrote: > On 2/24/22 15:23, Vaclav Petras wrote: > > On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: > >> Should we perhaps take this opportunity to move the grass libraries to > >> default library search paths as raised on the debian-gis list? > >> > >>https://lists.debian.org/debian-gis/2021/12/msg00023.html > > > > I vote yes. I think the reasons to do this are no longer valid. Doing it > > the standard way seems to me to be the best way forward. Any > > suggestions/PRs? > > Not using --prefix=/usr/lib will violate FHS, so not an option for the > Debian package. > > Are users of libgdal-grass expected to set LD_LIBRARY_PATH like the > grass executable does to find the grass libraries? In https://grasswiki.osgeo.org/wiki/Compile_and_install_GDAL-GRASS_plugin#Compilation_and_Installation_of_the_GDAL-GRASS_plugin the creation of a LD_LIBRARY_PATH file is suggested: /etc/ld.so.conf.d/grass.conf Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
On 2/24/22 15:23, Vaclav Petras wrote: On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: Should we perhaps take this opportunity to move the grass libraries to default library search paths as raised on the debian-gis list? https://lists.debian.org/debian-gis/2021/12/msg00023.html I vote yes. I think the reasons to do this are no longer valid. Doing it the standard way seems to me to be the best way forward. Any suggestions/PRs? Not using --prefix=/usr/lib will violate FHS, so not an option for the Debian package. Are users of libgdal-grass expected to set LD_LIBRARY_PATH like the grass executable does to find the grass libraries? Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] GRASS 8.0 support in GDAL and QGIS
Hi Bas, On Thu, 24 Feb 2022 at 03:33, Sebastiaan Couwenberg wrote: > > Should we perhaps take this opportunity to move the grass libraries to > default library search paths as raised on the debian-gis list? > > https://lists.debian.org/debian-gis/2021/12/msg00023.html > I vote yes. I think the reasons to do this are no longer valid. Doing it the standard way seems to me to be the best way forward. Any suggestions/PRs? Similarly, the Python package or packages are not on a standard path either. Best, Vaclav ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
