Re: [gdal-dev] Shortening schedule for CMake adoption ?
FWIW, I'll cast my +0.5 (not a full +1 because I didn't get to test myself, but I'm supportive because the feedback I've read so far sounds great) On 2022-01-17 08:37, Even Rouault wrote: Hi, The new CMake build system (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent progress, and I believe that it should be in a production ready state on time for GDAL 3.5.0 (~ May). It is already very close to it according to a checklist I had created (https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). Consequently we could shorten the rather conservative schedule presented in RFC 84 to : - Formally deprecate GNUmakefile and NMake base file systems. Users and packagers are encouraged to switch to CMake and actively report (and help fixing) issues the find in the process. ==> Target: GDAL 3.5 / May 2022. GDAL 3.5.x point releases will be used to address reported issues. - Completely remove GNUmakefile and NMake base file systems, and make CMake the only build system in GDAL source tree. ==> Target: GDAL 3.6 / November 2022 I can't see real advantages in keeping the 3 build systems longer than strictly needed: - it requires more maintenance effort and makes new contributions more complicated - we won't probably get significant feedback regarding the CMake build system until people have to adopt it because they have no other alternative. We already greatly welcome feedback from people trying with master. To facilitate this, I believe we could cut a GDAL 3.5 alpha in early March so that people who wait for "official" packages have a chance to give it a try too. Thoughts ? Even -- Daniel Morissette Mapgears Inc T: +1 418-696-5056 #201 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
Even Rouault writes: > Greg, > > I will not respond point by point, but the message here is: "CMake > support is available and believed to be in good shape into master > based on our manual tests and initial CI configuration exercising it, > it will replace autotools+nmake soon, be ready for it and help > polishing it". OK, that's great to hear. As someone who is here mainly as a packager, I don't subscribe to github, and I haven't seen anything on gdal-dev@ saying that cmake support is ready for widespread testing (but now I have!). > There will perhaps be areas where it will not do > everything that existing build systems made, but existing build > systems have also their flows that are not easily fixable. So be > it. Having one single build system at the end of the process, and used > in an idiomatic way (our autoconf system without automake is far from > being idiomatic), is the main objective of this whole effort from my > side. Sure, I get it that there will be a few rough edges. > CMake might not be completely ready for 3.5.0 for all imaginable > platforms & configurations (we don't promise to support all platforms > anyway. I don't believe we have a formal list of supported platforms > by the way. I'd say what is tested on our CI is the practical > definition of what is supported), and we won't defer indefinitely > 3.5.0 if it doesn't work on some confs. That's why autoconf will > only be removed in 3.6.0, and we have 3.5.x point releases to help > address issues. Sorry, I misunderstood that the autoconf removal would not be for this release. As long as I can use the existing autoconf support for 3.5.0 if I have to, things are much less tense. And yes, I realize that "supported platforms" is a difficult concept. But here we're talking about a big change that risks regresssions to platforms people have already made work, so I see it more as "we should really avoid causing regressions on platforms known to work". As a random datpoint, I have updated gdal in pkgsrc to 3.4.1 (not yet committed it), tested on NetBSD 9 amd64, and the tests have only a handful of failures that perhaps could be test issues. And qgis with this build works ok as far as I can tell. > The release process is described in HOWTO-RELEASE and it points to > the mkgdaldist.sh script > > You can generate a gdal-master.tar.gz with: > > ./mkgdaldist.sh master -branch master > > No idea if the script works properly on non-Linux systems. You'll need > some prerequisites for the script to run: Sphinx (pip install -r > doc/requirements.txt) to generate man pages, swig I'll try that, but I think you'll get a lot more testing from packagers if you publish an alpha tarball. > Or just clone the git repo and rm -rf autotest .git .github, and that > will be very close to the final tarball, at least for the purpose of > doing a CMake build > > and try packaging this. I'll try that if the above doesn't work. And actually I can try a cmake build from the repo, which is probably a good first step. signature.asc Description: PGP signature ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
Greg, I will not respond point by point, but the message here is: "CMake support is available and believed to be in good shape into master based on our manual tests and initial CI configuration exercising it, it will replace autotools+nmake soon, be ready for it and help polishing it". There will perhaps be areas where it will not do everything that existing build systems made, but existing build systems have also their flows that are not easily fixable. So be it. Having one single build system at the end of the process, and used in an idiomatic way (our autoconf system without automake is far from being idiomatic), is the main objective of this whole effort from my side. CMake might not be completely ready for 3.5.0 for all imaginable platforms & configurations (we don't promise to support all platforms anyway. I don't believe we have a formal list of supported platforms by the way. I'd say what is tested on our CI is the practical definition of what is supported), and we won't defer indefinitely 3.5.0 if it doesn't work on some confs. That's why autoconf will only be removed in 3.6.0, and we have 3.5.x point releases to help address issues. The release process is described in HOWTO-RELEASE and it points to the mkgdaldist.sh script You can generate a gdal-master.tar.gz with: ./mkgdaldist.sh master -branch master No idea if the script works properly on non-Linux systems. You'll need some prerequisites for the script to run: Sphinx (pip install -r doc/requirements.txt) to generate man pages, swig Or just clone the git repo and rm -rf autotest .git .github, and that will be very close to the final tarball, at least for the purpose of doing a CMake build and try packaging this. Even Le 18/01/2022 à 14:06, Greg Troxel a écrit : Even Rouault writes: The new CMake build system (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent progress, and I believe that it should be in a production ready state on time for GDAL 3.5.0 (~ May). It is already very close to it according to a checklist I had created (https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). Do you mean that the master branch already has cmake support that you believe 99% meets the requirements? I think that establishing a date before the code meets requirement risks a later decision to proceed anyway despite meeting the requirements. So I'd like to flip this around to the steps needed, with the autoconf removal decision gated on packager testing. Specifically: - Get master in a shape where the developers believe the requirements are met. This includes build instructions, specifically about how to get the right RPATH behavior. It's going to need testing building to a prefix other than /usr and specifically a prefix not in the default linker search path. I'm unclear on the plan for the test suite. If running py-test in the tests directory against an installed build still works, that's fine -- I don't see a need to couple test improvements with the cmake conversion. - Produce an alpha tarball, following the same (documented) procedure that would be used for a relaase in an autoconf-removed world. This is what packagers package, and users hand build. Tarball creation should be documented and scripted, so this should be a combination of good testing and near-zero effort. - Call for packager and user testing of the alpha tarball. Give them 1 full month, because converting a packaging build from autoconf to cmake is not trivial, and because everything that depends on gdal needs testing too. (In my case, the gdal build control files are much bigger than typical packages.) I expect to find problems, because I usually do on autoconf->cmake transitions, and often around RPATH handling. But I will be happy to report 100% success if that's how it is. And I'll try to do this sooner rather than later. - If there are any failures to meet requirements (including on systems where gdal does not have CI; that's the point of the call for testing), fix and release another alpha. 2 weeks is adequate testing time, if the failures were minor enough to not prevent getting to a working state. - Once the report/fix cycle ends, then the autoconf removal can proceed. The above can be pipelined, if an alpha tarball can be produced that 90-95% meets requirements, enough that it's reasonable to do a draft packaging conversion and end up with an installed gdal that one can build a working qgis against. Will this work for May? I don't know, and I think the big questions are when a believed-requirements-meeting alpha tarball can be produced, and how many residual issues there are. With a alpha in 2 weeks, the odds are good. With an alpha on April 1, I don't see how it can work. Greg -- http://www.spatialys.com My software is
Re: [gdal-dev] Shortening schedule for CMake adoption ?
Even Rouault writes: > The new CMake build system > (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent > progress, and I believe that it should be in a production ready state > on time for GDAL 3.5.0 (~ May). It is already very close to it > according to a checklist I had created > (https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). Do you mean that the master branch already has cmake support that you believe 99% meets the requirements? I think that establishing a date before the code meets requirement risks a later decision to proceed anyway despite meeting the requirements. So I'd like to flip this around to the steps needed, with the autoconf removal decision gated on packager testing. Specifically: - Get master in a shape where the developers believe the requirements are met. This includes build instructions, specifically about how to get the right RPATH behavior. It's going to need testing building to a prefix other than /usr and specifically a prefix not in the default linker search path. I'm unclear on the plan for the test suite. If running py-test in the tests directory against an installed build still works, that's fine -- I don't see a need to couple test improvements with the cmake conversion. - Produce an alpha tarball, following the same (documented) procedure that would be used for a relaase in an autoconf-removed world. This is what packagers package, and users hand build. Tarball creation should be documented and scripted, so this should be a combination of good testing and near-zero effort. - Call for packager and user testing of the alpha tarball. Give them 1 full month, because converting a packaging build from autoconf to cmake is not trivial, and because everything that depends on gdal needs testing too. (In my case, the gdal build control files are much bigger than typical packages.) I expect to find problems, because I usually do on autoconf->cmake transitions, and often around RPATH handling. But I will be happy to report 100% success if that's how it is. And I'll try to do this sooner rather than later. - If there are any failures to meet requirements (including on systems where gdal does not have CI; that's the point of the call for testing), fix and release another alpha. 2 weeks is adequate testing time, if the failures were minor enough to not prevent getting to a working state. - Once the report/fix cycle ends, then the autoconf removal can proceed. The above can be pipelined, if an alpha tarball can be produced that 90-95% meets requirements, enough that it's reasonable to do a draft packaging conversion and end up with an installed gdal that one can build a working qgis against. Will this work for May? I don't know, and I think the big questions are when a believed-requirements-meeting alpha tarball can be produced, and how many residual issues there are. With a alpha in 2 weeks, the odds are good. With an alpha on April 1, I don't see how it can work. Greg signature.asc Description: PGP signature ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
+1 I'll also do some checks in the next few weeks to make sure the things work on Windows as expected. Best regards, Tamas Even Rouault ezt írta (időpont: 2022. jan. 17., H, 14:38): > Hi, > > The new CMake build system > (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent > progress, and I believe that it should be in a production ready state on > time for GDAL 3.5.0 (~ May). It is already very close to it according to > a checklist I had created > ( > https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). > > Consequently we could shorten the rather conservative schedule presented > in RFC 84 to : > > - Formally deprecate GNUmakefile and NMake base file systems. Users and > packagers are encouraged to switch to CMake and actively report (and > help fixing) issues the find in the process. > > ==> Target: GDAL 3.5 / May 2022. GDAL 3.5.x point releases will be used > to address reported issues. > > - Completely remove GNUmakefile and NMake base file systems, and make > CMake the only build system in GDAL source tree. > > ==> Target: GDAL 3.6 / November 2022 > > > I can't see real advantages in keeping the 3 build systems longer than > strictly needed: > > - it requires more maintenance effort and makes new contributions more > complicated > > - we won't probably get significant feedback regarding the CMake build > system until people have to adopt it because they have no other > alternative. > > We already greatly welcome feedback from people trying with master. To > facilitate this, I believe we could cut a GDAL 3.5 alpha in early March > so that people who wait for "official" packages have a chance to give it > a try too. > > Thoughts ? > > Even > > -- > http://www.spatialys.com > My software is free, but my time generally not. > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
+1 I am in favour, and will give it a try. Norman On Mon, Jan 17, 2022 at 12:11 PM Jeff McKenna wrote: > On 2022-01-17 9:37 a.m., Even Rouault wrote: > > > >> Consequently we could shorten the rather conservative schedule > > > > +1 > > thanks! > > -jeff > > > > -- > Jeff McKenna > GatewayGeo: Developers of MS4W, MapServer Consulting and Training > co-founder of FOSS4G > http://gatewaygeo.com/ > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
On 2022-01-17 9:37 a.m., Even Rouault wrote: Consequently we could shorten the rather conservative schedule +1 thanks! -jeff -- Jeff McKenna GatewayGeo: Developers of MS4W, MapServer Consulting and Training co-founder of FOSS4G http://gatewaygeo.com/ ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
> I can't see real advantages in keeping the 3 build systems longer than > strictly needed > fully agree, there's no advantage in delaying shipping if it's ready -- thomas ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
On Mon, 17 Jan 2022 at 14:37, Even Rouault wrote: > The new CMake build system > (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent > progress, and I believe that it should be in a production ready state on > time for GDAL 3.5.0 (~ May). Thanks for those efforts! > Consequently we could shorten the rather conservative schedule +1 Best regards, -- Mateusz Loskot, http://mateusz.loskot.net ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Shortening schedule for CMake adoption ?
+1 (Not that I have a vote). The CMake builds of my github clone seems OK. They do have build failures from time to time but I assume that is always true of bleeding edge code. On Mon, 17 Jan 2022, Even Rouault wrote: Hi, The new CMake build system (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent progress, and I believe that it should be in a production ready state on time for GDAL 3.5.0 (~ May). It is already very close to it according to a checklist I had created (https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). Consequently we could shorten the rather conservative schedule presented in RFC 84 to : - Formally deprecate GNUmakefile and NMake base file systems. Users and packagers are encouraged to switch to CMake and actively report (and help fixing) issues the find in the process. ==> Target: GDAL 3.5 / May 2022. GDAL 3.5.x point releases will be used to address reported issues. - Completely remove GNUmakefile and NMake base file systems, and make CMake the only build system in GDAL source tree. ==> Target: GDAL 3.6 / November 2022 I can't see real advantages in keeping the 3 build systems longer than strictly needed: - it requires more maintenance effort and makes new contributions more complicated - we won't probably get significant feedback regarding the CMake build system until people have to adopt it because they have no other alternative. We already greatly welcome feedback from people trying with master. To facilitate this, I believe we could cut a GDAL 3.5 alpha in early March so that people who wait for "official" packages have a chance to give it a try too. Thoughts ? Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Andrew C. Aitchison Kendal, UK and...@aitchison.me.uk ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Shortening schedule for CMake adoption ?
Hi, The new CMake build system (https://gdal.org/development/rfc/rfc84_cmake.html) has made excellent progress, and I believe that it should be in a production ready state on time for GDAL 3.5.0 (~ May). It is already very close to it according to a checklist I had created (https://docs.google.com/spreadsheets/d/1SsUXiZxKim6jhLjlJFCRs1zwMvNpbJbBMB6yl0ms01c). Consequently we could shorten the rather conservative schedule presented in RFC 84 to : - Formally deprecate GNUmakefile and NMake base file systems. Users and packagers are encouraged to switch to CMake and actively report (and help fixing) issues the find in the process. ==> Target: GDAL 3.5 / May 2022. GDAL 3.5.x point releases will be used to address reported issues. - Completely remove GNUmakefile and NMake base file systems, and make CMake the only build system in GDAL source tree. ==> Target: GDAL 3.6 / November 2022 I can't see real advantages in keeping the 3 build systems longer than strictly needed: - it requires more maintenance effort and makes new contributions more complicated - we won't probably get significant feedback regarding the CMake build system until people have to adopt it because they have no other alternative. We already greatly welcome feedback from people trying with master. To facilitate this, I believe we could cut a GDAL 3.5 alpha in early March so that people who wait for "official" packages have a chance to give it a try too. Thoughts ? Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
