Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependencies
On 01/11/2018 11:19 PM, Jeremy Palmer wrote: > On Fri, Jan 12, 2018 at 8:54 AM, Sebastiaan Couwenberg > wrote: > >> When you need newer versions of the core geospatial libraries like GEOS >> & GDAL than what is provided in the Ubuntu LTS releases, you'll need >> some 3rd party repository because Ubuntu doesn't provide backports. The >> geospatial packages in Ubuntu are actually not actively maintained at >> all, active maintenance only happens in Debian and Ubuntu just copies >> its packages from Debian. > > But I assume Ubuntu universe and multiverse packages could be better > maintained if someone stepped up to do it. Yes, but has not happened yet. It seems that people need to be employed to work on Ubuntu, no one has volunteered to do so. And this makes sense, Ubuntu is a corporate backed distribution unlike Debian. And many corporate users choose Ubuntu over Debian because of that. Why volunteer to do work in that environment when you could be paid to do so? >> The nature of LTS releases is that they don't change the versions of the >> packages they provide, Debian stable, Ubuntu LTS and RedHat/CentOS don't >> differ much in the respect. >> >> When the users of those distributions do have a need for newer packages >> they need to get backports somehow, either through some 3rd party >> repository over which they have no control, or their own in-house >> repository which they do control. The latter is the best option to >> prevent any surprises when the repository gets new versions, it does >> require integration work to rebuild all related packages to work with >> the new versions of the libraries in question. >> >>> I'm not sure if this problem is something that can be done by OSGEO or >>> some >>> other alliance of companies. My feeling is no one company can currently >>> provide this service as it requires expertise across the full stack, or >>> enough scale in business to centralise support for the whole stack. >> >> As long as the geospatial packages in Debian and UbuntuGIS are >> maintained by volunteers the users of those packages can't expect more >> than work on a best effort basis. No one is employed to maintain these >> packages for the wider community, although there seems to be a need at >> least for corporate Ubuntu users. Mapgears sponsored work on UbuntuGIS >> in the past, but that was apparently not successful enough to continue. > > Understood. Maybe Snaps is the solution to all of this, as it provides > binaries for a larger user base. I'm interested on the community view on > this. Snaps and containers just complicate maintenance, you'll have many more images to update to fix security issues in the packages they contain. That's the major downside of the isolation they provide. >> One of the benefits of open source is that companies are not dependent >> on suppliers, and can employ people to the needed work in-house. > > In regards to not being locked into companies, I don't see that providing > more robust community packaging is going to change this situation. > >> This work can be shared with the community, but generally corporate needs >> differ quite a bit from community needs and > > I'm not really clear what the community vs corporate difference are. What > are they? I would say that it's a very grey area between who's community > and who's corporate. Corporate users have schedules, deadlines and financial interests (e.g. selling support/training), community users (e.g. volunteers and hobbyists) generally don't. >> its cheaper to just keep it all in-house. > > IMHO I don't think this is the case, at least for software with a large > user base. I also think this approach means that the QGIS LINUX desktop > uptake for anything other than very large organisations or software > developers will be low. Look at the Windows and MacOSX environment (even in > personal home usage), packaging is done once by the provider of the product > then users will use that packaging without modification. I think many small > to medium size organisation couldn't justify the resources to employ > someone to do packaging. Participating in open source projects for a company generally pays off when they can share the burden. When a single company supports an open source project they don't have the benefit of many shoulders lessening the burden. I don't know any Windows & OSX users that only use the software included in Windows & OSX. They all use additional software from elsewhere, and in the case of OSX generally from open source projects like MacPorts and Homebrew. Most users of any software are just that, users. They don't need to modify it for their needs, they make do with what they have. For the other users there is open source to adapt to their needs. In house packaging is not a full time job, it will be among the tasks for someone doing system engineering or development most likely. >> I don't understand the question. >> >> The version requirement in the dependencies reflect t
Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependencies
Hi Bas, On Fri, Jan 12, 2018 at 8:54 AM, Sebastiaan Couwenberg wrote: > > When you need newer versions of the core geospatial libraries like GEOS > & GDAL than what is provided in the Ubuntu LTS releases, you'll need > some 3rd party repository because Ubuntu doesn't provide backports. The > geospatial packages in Ubuntu are actually not actively maintained at > all, active maintenance only happens in Debian and Ubuntu just copies > its packages from Debian. > But I assume Ubuntu universe and multiverse packages could be better maintained if someone stepped up to do it. > > The nature of LTS releases is that they don't change the versions of the > packages they provide, Debian stable, Ubuntu LTS and RedHat/CentOS don't > differ much in the respect. > > When the users of those distributions do have a need for newer packages > they need to get backports somehow, either through some 3rd party > repository over which they have no control, or their own in-house > repository which they do control. The latter is the best option to > prevent any surprises when the repository gets new versions, it does > require integration work to rebuild all related packages to work with > the new versions of the libraries in question. > > > I'm not sure if this problem is something that can be done by OSGEO or > some > > other alliance of companies. My feeling is no one company can currently > > provide this service as it requires expertise across the full stack, or > > enough scale in business to centralise support for the whole stack. > > As long as the geospatial packages in Debian and UbuntuGIS are > maintained by volunteers the users of those packages can't expect more > than work on a best effort basis. No one is employed to maintain these > packages for the wider community, although there seems to be a need at > least for corporate Ubuntu users. Mapgears sponsored work on UbuntuGIS > in the past, but that was apparently not successful enough to continue. Understood. Maybe Snaps is the solution to all of this, as it provides binaries for a larger user base. I'm interested on the community view on this. One of the benefits of open source is that companies are not dependent on suppliers, and can employ people to the needed work in-house. In regards to not being locked into companies, I don't see that providing more robust community packaging is going to change this siutation. > This work can be shared with the community, but generally corporate needs differ quite a bit from community needs and I'm not really clear what the community vs corporate difference are. What are they? I would say that it's a very grey area between who's community and who's corporate. > its cheaper to just keep it all in-house. IMHO I don't think this is the case, at least for software with a large user base. I also think this approach means that the QGIS LINUX desktop uptake for anything other than very large organisations or software developers will be low. Look at the Windows and MacOSX environment (even in personal home usage), packaging is done once by the provider of the product then users will use that packaging without modification. I think many small to medium size organisation couldn't justify the resources to employ someone to do packaging. > I don't understand the question. > > The version requirement in the dependencies reflect the minimum version > the code in the dependent package requires to work with the library. > Some parts of qgis use features introduced in GDAL 2.x whereas other > parts don't use features introduced after GDAL 1.8.0. Because the > various parts of qgis are split into separate packages you get the > different version requirement for gdal for those parts. > > There is generally only a single gdal package in the repositories like > in Debian and Ubuntu. Only when one adds a 3rd party repository like > UbuntuGIS is another version of the gdal package available, and you get > integration issues like qgis being uninstallable with the UbuntuGIS > dependencies after gdal is updated there but qgis hasn't been rebuilt > with the new version yet. > I think I understand better now. But if the package name includes the version number e.g libgdal20 or abi-2-2-2 and that's package is a QGIS dependency, how can having a lower ABI version requirement below the package version name make any difference? I do except that libgdal20 (>=2.1) does make some sense, but libgdal20 (>=1.8) seems to be silly. Cheers, Jeremy ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependencies
On 01/11/2018 08:23 PM, Jeremy Palmer wrote: >> You should have a look at UbuntuGIS development to see that it is mostly >> the work of one person now. Packages are updated for OSGeo-Live and copied >> to ubuntugis-unstable, after a while (no policy or guidelines) the packages >> from ubuntugis-unstable get copied to ubuntugis-stable. >> This generally happens during an Ubuntu releases lifecycle, not ideal for >> production systems. >> > > Thanks for the information. Good points. > >> For production systems one should not rely on 3rd party repositories, >> other than the one the company maintains itself where it does all the >> integration work. > > This is where I'm a little on the fence. I would much rather do it well for > all of the community, than just for my organisation. At the end of the day > my organisation doesn't need to apply special patches for our desktop > usage. But I agree applications that are developed using these libraries or > require core customisation definitely need a in-house managed repository. > > Our organisation needs to have a QGIS LTS release supported that works on a > Ubuntu LTS release. Correct me if I'm wrong, but QGIS.org seems to provide > this now with the current release/LTS Debian repositories. It's the QGIS > critical dependencies which have varying policies of support that are the > problem. This seems to be a problem of resources/volunteers at the project > and the distribution packaging level. For example in the GDAL project, 1.10 > and 1.11, which are the current GDAL versions in Ubuntu 14.04 and 16.04 > haven't been maintained for many years. Only 2.2.X seems to be getting any > current support currently (thanks Even!). I guess this is part of the wider > problem that was identified by Paul Ramsey in > http://blog.cleverelephant.ca/2017/08/foss4g-keynote.html. Somehow we need > to solve this for the wider geospatial OS community. When you need newer versions of the core geospatial libraries like GEOS & GDAL than what is provided in the Ubuntu LTS releases, you'll need some 3rd party repository because Ubuntu doesn't provide backports. The geospatial packages in Ubuntu are actually not actively maintained at all, active maintenance only happens in Debian and Ubuntu just copies its packages from Debian. The nature of LTS releases is that they don't change the versions of the packages they provide, Debian stable, Ubuntu LTS and RedHat/CentOS don't differ much in the respect. When the users of those distributions do have a need for newer packages they need to get backports somehow, either through some 3rd party repository over which they have no control, or their own in-house repository which they do control. The latter is the best option to prevent any surprises when the repository gets new versions, it does require integration work to rebuild all related packages to work with the new versions of the libraries in question. > I'm not sure if this problem is something that can be done by OSGEO or some > other alliance of companies. My feeling is no one company can currently > provide this service as it requires expertise across the full stack, or > enough scale in business to centralise support for the whole stack. As long as the geospatial packages in Debian and UbuntuGIS are maintained by volunteers the users of those packages can't expect more than work on a best effort basis. No one is employed to maintain these packages for the wider community, although there seems to be a need at least for corporate Ubuntu users. Mapgears sponsored work on UbuntuGIS in the past, but that was apparently not successful enough to continue. One of the benefits of open source is that companies are not dependent on suppliers, and can employ people to the needed work in-house. This work can be shared with the community, but generally corporate needs differ quite a bit from community needs and its cheaper to just keep it all in-house. >>> I've also noted that the 2.18 binaries produced for the 14.04 and >>> 16.04 packaging has inconsistent dependency requirements. e.g 2.18.15 >>> 14.04 64bit : >>> >>> qgis - gdal-abi-2-2-2, libgdal20 (>= 1.8.0) qgis-providers = libgdal20 >>> (>= 2.2.0) qgis-server - libgdal20 (>= 1.8.0) libqgis-dev - >>> libgdal-dev (>= 1.10.1-0~) libqgis-core - libgdal20 (>= 2.2.0) >>> libqgis-app2 libgdal20 (>= 2.0.1) >> >> The version requirement for libgdal20 are determined based on which >> symbols the code in the dependent package uses, qgis & qgis-servers don't >> use any symbols introduced after GDAL 1.8.0, libqgis-dev does but no >> symbols introduced after GDAL 1.10.1, etc. > > Ok thanks. What's the practical use of this approach if it's tied to > gdal-abi-2-2-2 anyway? I don't understand the question. The version requirement in the dependencies reflect the minimum version the code in the dependent package requires to work with the library. Some parts of qgis use features introduced in GDAL 2.x whereas other parts don't use features introduced
Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependencies
> > Hi Bas, > You should have a look at UbuntuGIS development to see that it is mostly > the work of one person now. Packages are updated for OSGeo-Live and copied > to ubuntugis-unstable, after a while (no policy or guidelines) the packages > from ubuntugis-unstable get copied to ubuntugis-stable. > This generally happens during an Ubuntu releases lifecycle, not ideal for > production systems. > Thanks for the information. Good points. > > For production systems one should not rely on 3rd party repositories, > other than the one the company maintains itself where it does all the > integration work. > This is where I'm a little on the fence. I would much rather do it well for all of the community, than just for my organisation. At the end of the day my organisation doesn't need to apply special patches for our desktop usage. But I agree applications that are developed using these libraries or require core customisation definitely need a in-house managed repository. Our organisation needs to have a QGIS LTS release supported that works on a Ubuntu LTS release. Correct me if I'm wrong, but QGIS.org seems to provide this now with the current release/LTS Debian repositories. It's the QGIS critical dependencies which have varying policies of support that are the problem. This seems to be a problem of resources/volunteers at the project and the distribution packaging level. For example in the GDAL project, 1.10 and 1.11, which are the current GDAL versions in Ubuntu 14.04 and 16.04 haven't been maintained for many years. Only 2.2.X seems to be getting any current support currently (thanks Even!). I guess this is part of the wider problem that was identified by Paul Ramsey in http://blog.cleverelephant.ca/2017/08/foss4g-keynote.html. Somehow we need to solve this for the wider geospatial OS community. I'm not sure if this problem is something that can be done by OSGEO or some other alliance of companies. My feeling is no one company can currently provide this service as it requires expertise across the full stack, or enough scale in business to centralise support for the whole stack. > > > I've also noted that the 2.18 binaries produced for the 14.04 and > > 16.04 packaging has inconsistent dependency requirements. e.g 2.18.15 > > 14.04 64bit : > > > > qgis - gdal-abi-2-2-2, libgdal20 (>= 1.8.0) qgis-providers = libgdal20 > > (>= 2.2.0) qgis-server - libgdal20 (>= 1.8.0) libqgis-dev - > > libgdal-dev (>= 1.10.1-0~) libqgis-core - libgdal20 (>= 2.2.0) > > libqgis-app2 libgdal20 (>= 2.0.1) > > The version requirement for libgdal20 are determined based on which > symbols the code in the dependent package uses, qgis & qgis-servers don't > use any symbols introduced after GDAL 1.8.0, libqgis-dev does but no > symbols introduced after GDAL 1.10.1, etc. > Ok thanks. What's the practical use of this approach if it's tied to gdal-abi-2-2-2 anyway? Cheers, Jeremy ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependencies
Stable isn't necessarely more "unstable" than unstable - since "stable" is almost never really well tested. Maybe that depends on your definition of "really well", but anyway, just saying. Also, stable doesn't mean bug-free, it just mean stable, that ist o say, staying as is. If you have issues with default-packages/dependecies, why don't you just containerize it and create a snapcraft-package ? Maybe someone also wants to have 2 different applications that both use the GDAL of a (different) specific version each on the same system ... That could never be stable since one oft he two would probably not work. Snappycraft also has the advantage that you need to create only ONE package for all the major distros, such as Ubuntu&Derivates., Debian, Fedora/CentOS, Arch, Suse, ScientificLinux, Gentoo, etc. Then you can test that one package, and then you know it works or doesn't, and dependencies only get updated when your snapcraft package gets updated. This would be my definition of "stable". Also, snapcraft would make it possible to distribute commercial packages, e.g. with google-maps material, through snapcraft-store https://docs.snapcraft.io/reference/snap-command snap buy https://snapcraft.io https://askubuntu.com/questions/743068/how-to-install-snapcraft-on-14-04 https://developer.ubuntu.com/core https://arstechnica.com/information-technology/2016/06/goodbye-apt-and-yum-ubuntus-snap-apps-are-coming-to-distros-everywhere/ You can also get snapcraft on 14.04: sudo apt install snapcraft snapd to install package: sudo snap install to update package: sudo snap refresh to remove package: sudo snap remove search for package: snap find list installed packages: sudo snap list e.g. sudo snap install nmap sudo snap refresh nmap -Ursprüngliche Nachricht- Von: QGIS-Developer [mailto:qgis-developer-boun...@lists.osgeo.org] Im Auftrag von Sebastiaan Couwenberg Gesendet: Dienstag, 9. Januar 2018 07:37 An: qgis-developer@lists.osgeo.org Betreff: Re: [QGIS-Developer] Ubuntu Package Releases and GDAL Dependancies On 01/09/2018 06:28 AM, Jeremy Palmer wrote: > I've been taking a close look at a way to install QGIS with >= GDAL 2 > on 64bit Ubuntu 14.04 and 16.04 for production desktops. > > It's not possible to install QGIS using the https://qgis.org/debian > package archive as the ubuntu 14.04 archives only supports GDAL > 1.10.1 and 16.04 only supports GDAL 1.11.3. > > With https://qgis.org/ubuntugis it is possible to install QGIS with >= > GDAL > 2.2 as this archive requires ubuntugis-unstable which currently > supports GDAL 2.2.2. > > The issue I have is I'm reluctant to use ubuntugis-unstable for > production use. I would rather use ubuntugis-stable. However this > archive only supports GDAL 2.1 and doesn't provide the minimum > dependency of GDAL 2.2.2 for https://qgis.org/ubuntugis requires. > > I was wondering why ubuntugis stable isn't used as the debian archive > for https://qgis.org/ubuntugis and ubuntugis-unstable is only used for > the development (e.g 2.99) nightly builds? This makes more sense to me. You should have a look at UbuntuGIS development to see that it is mostly the work of one person now. Packages are updated for OSGeo-Live and copied to ubuntugis-unstable, after a while (no policy or guidelines) the packages from ubuntugis-unstable get copied to ubuntugis-stable. This generally happens during an Ubuntu releases lifecycle, not ideal for production systems. For production systems one should not rely on 3rd party repositories, other than the one the company maintains itself where it does all the integration work. > I've also noted that the 2.18 binaries produced for the 14.04 and > 16.04 packaging has inconsistent dependency requirements. e.g 2.18.15 > 14.04 64bit : > > qgis - gdal-abi-2-2-2, libgdal20 (>= 1.8.0) qgis-providers = libgdal20 > (>= 2.2.0) qgis-server - libgdal20 (>= 1.8.0) libqgis-dev - > libgdal-dev (>= 1.10.1-0~) libqgis-core - libgdal20 (>= 2.2.0) > libqgis-app2 libgdal20 (>= 2.0.1) The version requirement for libgdal20 are determined based on which symbols the code in the dependent package uses, qgis & qgis-servers don't use any symbols introduced after GDAL 1.8.0, libqgis-dev does but no symbols introduced after GDAL 1.10.1, etc. Kind Regards, Bas ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer ___ QGIS-Developer mailing list QGIS-Developer@lists.osgeo.org List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
