Re: [gdal-dev] GDAL sponsored maintenance report: Aug 2021-March 2023

[Tamas Szekeres]

Hi Even,

Thank you for your hard work to keep GDAL running at a high intensity and
quality level, which also shows well in the openhub stats:

https://openhub.net/p/gdal/contributors?sort=latest_commit&time_span=12+months

Keep up the good work :-)

Best regards,

Tamas




Even Rouault  ezt írta (időpont: 2023. márc.
27., H, 18:35):

> Hi,
>
> After more than one year and a half of benefiting from the GDAL sponsored
> maintenance program, it is time to share with the community what work has
> been accomplished thanks to that sponsorship.  It is quite hard to
> summarize, as the amount of tasks reaches 1800 for a total of 1313 hours.
> So lots of small tasks that make the daily reality of maintaining a project
> like GDAL.
>
> Their global repartition is the following one:
>
>-
>
>Bug fixing: 51% of actions, 55% of time spent (including 177 bug fixes
>related to issues found by oss-fuzz)
>-
>
>Enhancements: 5% of actions, 15% of time spent
>-
>
>CMake related work: 5% of actions, 10% of time spent
>-
>
>Review of contributions: 20% of actions, 5% of time spent
>-
>
>Release management: 2% of actions, 4% of time spent
>-
>
>Bug triaging and analysis (without corresponding bug fix): 8% of
>actions, 3% of time spent
>-
>
>Maintenance of continuous integration setup: 3% of actions and time
>spent
>-
>
>Code enhancements/cleanups: 2% of actions and time spent
>-
>
>Documentation: 1% of actions and time spent
>-
>
>Other activities: mailing list discussions, bug reports to other
>projects, meetings, etc.
>
>
> This work has benefited mostly to GDAL (80%), PROJ (15%), libtiff (2%),
> openjpeg (1%) and more marginally to other projects associated with GDAL
> such as Xerces-C, poppler, libgeotiff, cppcheck, netcdf, curl, libjxl and
> shapelib.
>
> The CMake related work has been a major item of work for the mid 2021 -
> beginning of 2022 period. I cannot overstate the importance of the initial
> contribution of this work made by Hiroshi Miura who brought most of the
> initial material. That said, without the sponsored maintenance program
> which helped polishing and making it production ready for all environments
> supported by GDAL, this would likely have remained a out-of-tree project. I
> believe most stakeholders (contributors and users, at least the ones who
> build from source) are very satisfied with this transition from the
> historic build systems to a unified and more modern one, with consistent
> option naming. Building on Windows is in particular much easier nowadays,
> in particular when leveraging dependencies from distributions like vcpkg or
> Conda Forge. For GDAL developers, the new build system offers integration
> with IDEs and solves long standing annoyances like missing header
> dependency rules for partial rebuilds.
>
> Although it doesn’t show up particularly in the above statistics, making
> sure that the continuous integration configurations remain “green” at all
> times is a constant source of attention. Given the number of environments
> tested and the number of dependencies of GDAL, there is hardly a week where
> some action is not needed in that area to make sure that the code builds
> and compiles cleanly, tests pass, etc. We can now track a few rolling
> distributions to detect as early as possible sources of incompatibilities
> with our dependencies, and act early: report issues to upstream if there
> are bugs, or do changes in our code & build scripts to take them into
> account.
>
> Making sure that code contributions from others are reviewed in a timely
> manner is important for the contributor experience. The average delay for a
> submitted pull request (excluding mine) to be commented by me (or merged if
> no comment needed) is 22 hours, and the median time 3h20 (statistics
> gathered on 601 pull requests since August 2021, source script at
> https://gist.github.com/rouault/0fbd37f59b8e93ae63761468a5600262)
>
> In the category of regular tasks, one can also mention: updating the EPSG
> dataset in PROJ (and coordinating with IOGP when detecting issues),
> refreshing vendored of copies in the GDAL source tree (libtiff typically),
> making sure that new SQLite releases play nicely with PROJ which has some
> stressing SQL queries (we spot recently a performance regression in SQLite
> 3.41.0 and reported it to upstream)
>
> The following RFCs have been implemented (not mentioning a few procedural
> ones) thanks to the sponsorship:
>
>-
>
>RFC 87: Signed int8 data type for raster
>
>-
>
>RFC 88: Use GoogleTest framework for C/C++ unit tests
>
>-
>
>RFC 90: Direct access to compressed raster data
>
>-
>
>RFC 91: GDALDataset::Close() method
>

Re: [gdal-dev] [EXT] GDAL sponsored maintenance report: Aug 2021-March 2023

[Scott]

The significant role GDAL has in GIS software cannot be overstated. 
Virtually all open source GIS would not exist without it. Proprietary 
software would be, well, proprietary.


Thanks to Evan and Frank and the not frequently recognized developers 
for all they do. GDAL is huge, it's the center of the open source universe.


--
www.postholer.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [EXT] GDAL sponsored maintenance report: Aug 2021-March 2023

[Matt . Wilkie]

Thank you Even,

I appreciate this view and corresponding insight into the work of the GDAL 
sponsored maintenance - this is in addition to the actual work you've 
undertaken, and coaching of occasional contributors and question askers like 
myself! I hope that in course of time the bug fixing ratio drops somewhat lower 
than half, and you get to enjoy building more enhancements and new things.

Best,

Matt
Geomatics Developer and Administrator | Environment | T 867-667-8133 | Yukon.ca
Hours: 08:30-16:30, Mon-Wed: Office, Thu: Remote, Fri: Away.

From: gdal-dev  On Behalf Of Even Rouault
Sent: Monday, March 27, 2023 9:35 AM
To: gdal-dev@lists.osgeo.org
Subject: [EXT] [gdal-dev] GDAL sponsored maintenance report: Aug 2021-March 2023


Hi,


After more than one year and a half of benefiting from the GDAL sponsored 
maintenance program, it is time to share with the community what work has been 
accomplished thanks to that sponsorship.  It is quite hard to summarize, as the 
amount of tasks reaches 1800 for a total of 1313 hours. So lots of small tasks 
that make the daily reality of maintaining a project like GDAL.


Their global repartition is the following one:

  *   Bug fixing: 51% of actions, 55% of time spent (including 177 bug fixes 
related to issues found by oss-fuzz)
  *   Enhancements: 5% of actions, 15% of time spent
  *   CMake related work: 5% of actions, 10% of time spent
  *   Review of contributions: 20% of actions, 5% of time spent
  *   Release management: 2% of actions, 4% of time spent
  *   Bug triaging and analysis (without corresponding bug fix): 8% of actions, 
3% of time spent
  *   Maintenance of continuous integration setup: 3% of actions and time spent
  *   Code enhancements/cleanups: 2% of actions and time spent
  *   Documentation: 1% of actions and time spent
  *   Other activities: mailing list discussions, bug reports to other 
projects, meetings, etc.


This work has benefited mostly to GDAL (80%), PROJ (15%), libtiff (2%), 
openjpeg (1%) and more marginally to other projects associated with GDAL such 
as Xerces-C, poppler, libgeotiff, cppcheck, netcdf, curl, libjxl and shapelib.


The CMake related work has been a major item of work for the mid 2021 - 
beginning of 2022 period. I cannot overstate the importance of the initial 
contribution of this work made by Hiroshi Miura who brought most of the initial 
material. That said, without the sponsored maintenance program which helped 
polishing and making it production ready for all environments supported by 
GDAL, this would likely have remained a out-of-tree project. I believe most 
stakeholders (contributors and users, at least the ones who build from source) 
are very satisfied with this transition from the historic build systems to a 
unified and more modern one, with consistent option naming. Building on Windows 
is in particular much easier nowadays, in particular when leveraging 
dependencies from distributions like vcpkg or Conda Forge. For GDAL developers, 
the new build system offers integration with IDEs and solves long standing 
annoyances like missing header dependency rules for partial rebuilds.


Although it doesn't show up particularly in the above statistics, making sure 
that the continuous integration configurations remain "green" at all times is a 
constant source of attention. Given the number of environments tested and the 
number of dependencies of GDAL, there is hardly a week where some action is not 
needed in that area to make sure that the code builds and compiles cleanly, 
tests pass, etc. We can now track a few rolling distributions to detect as 
early as possible sources of incompatibilities with our dependencies, and act 
early: report issues to upstream if there are bugs, or do changes in our code & 
build scripts to take them into account.


Making sure that code contributions from others are reviewed in a timely manner 
is important for the contributor experience. The average delay for a submitted 
pull request (excluding mine) to be commented by me (or merged if no comment 
needed) is 22 hours, and the median time 3h20 (statistics gathered on 601 pull 
requests since August 2021, source script at 
https://gist.github.com/rouault/0fbd37f59b8e93ae63761468a5600262)


In the category of regular tasks, one can also mention: updating the EPSG 
dataset in PROJ (and coordinating with IOGP when detecting issues), refreshing 
vendored of copies in the GDAL source tree (libtiff typically), making sure 
that new SQLite releases play nicely with PROJ which has some stressin

Re: [gdal-dev] GDAL sponsored maintenance report: Aug 2021-March 2023

[Kai Mühlbauer]

Dear Even,

Thanks for this extensive summary. I've followed closely on mailing list and 
GitHub and there have been a lot of good reads and interesting discussions. For 
us users and developers it's good to see that the project and it's surroundings 
are in good shape.

Thanks for your responsiveness and help in all those places. Your and other 
contributors expertise is indeed invaluable.

Thanks to all sponsors making this possible.

Cheers,
Kai 


Am 27. März 2023 16:35:25 UTC schrieb Even Rouault :
>Hi,
>
>
>After more than one year and a half of benefiting from the GDAL sponsored 
>maintenance program, it is time to share with the community what work has been 
>accomplished thanks to that sponsorship.  It is quite hard to summarize, as 
>the amount of tasks reaches 1800 for a total of 1313 hours. So lots of small 
>tasks that make the daily reality of maintaining a project like GDAL.
>
>
>Their global repartition is the following one:
>
> *
>
>   Bug fixing: 51% of actions, 55% of time spent (including 177 bug
>   fixes related to issues found by oss-fuzz)
>
> *
>
>   Enhancements: 5% of actions, 15% of time spent
>
> *
>
>   CMake related work: 5% of actions, 10% of time spent
>
> *
>
>   Review of contributions: 20% of actions, 5% of time spent
>
> *
>
>   Release management: 2% of actions, 4% of time spent
>
> *
>
>   Bug triaging and analysis (without corresponding bug fix): 8% of
>   actions, 3% of time spent
>
> *
>
>   Maintenance of continuous integration setup: 3% of actions and time
>   spent
>
> *
>
>   Code enhancements/cleanups: 2% of actions and time spent
>
> *
>
>   Documentation: 1% of actions and time spent
>
> *
>
>   Other activities: mailing list discussions, bug reports to other
>   projects, meetings, etc.
>
>
>This work has benefited mostly to GDAL (80%), PROJ (15%), libtiff (2%), 
>openjpeg (1%) and more marginally to other projects associated with GDAL such 
>as Xerces-C, poppler, libgeotiff, cppcheck, netcdf, curl, libjxl and shapelib.
>
>
>The CMake related work has been a major item of work for the mid 2021 - 
>beginning of 2022 period. I cannot overstate the importance of the initial 
>contribution of this work made by Hiroshi Miura who brought most of the 
>initial material. That said, without the sponsored maintenance program which 
>helped polishing and making it production ready for all environments supported 
>by GDAL, this would likely have remained a out-of-tree project. I believe most 
>stakeholders (contributors and users, at least the ones who build from source) 
>are very satisfied with this transition from the historic build systems to a 
>unified and more modern one, with consistent option naming. Building on 
>Windows is in particular much easier nowadays, in particular when leveraging 
>dependencies from distributions like vcpkg or Conda Forge. For GDAL 
>developers, the new build system offers integration with IDEs and solves long 
>standing annoyances like missing header dependency rules for partial rebuilds.
>
>
>Although it doesn’t show up particularly in the above statistics, making sure 
>that the continuous integration configurations remain “green” at all times is 
>a constant source of attention. Given the number of environments tested and 
>the number of dependencies of GDAL, there is hardly a week where some action 
>is not needed in that area to make sure that the code builds and compiles 
>cleanly, tests pass, etc. We can now track a few rolling distributions to 
>detect as early as possible sources of incompatibilities with our 
>dependencies, and act early: report issues to upstream if there are bugs, or 
>do changes in our code & build scripts to take them into account.
>
>
>Making sure that code contributions from others are reviewed in a timely 
>manner is important for the contributor experience. The average delay for a 
>submitted pull request (excluding mine) to be commented by me (or merged if no 
>comment needed) is 22 hours, and the median time 3h20 (statistics gathered on 
>601 pull requests since August 2021, source script at 
>https://gist.github.com/rouault/0fbd37f59b8e93ae63761468a5600262)
>
>
>In the category of regular tasks, one can also mention: updating the EPSG 
>dataset in PROJ (and coordinating with IOGP when detecting issues), refreshing 
>vendored of copies in the GDAL source tree (libtiff typically), making sure 
>that new SQLite releases play nicely with PROJ which has some stressing SQL 
>queries (we spot recently a performance regression in SQLite 3.41.0 and 
>reported it to upstream)
>
>
>The following RFCs have been implemented (not mentioning a few procedural 
>ones) thanks to the sponsorship:
>
> *
>
>   RFC 87: Signed int8 data type for raster
>   
>
> *
>
>   RFC 88: Use GoogleTest framework for C/C++ unit tests
>   
>
> *
>
>   RFC 90: Direct access to compressed raster data
>   

[gdal-dev] GDAL sponsored maintenance report: Aug 2021-March 2023

[Even Rouault]

Hi,


After more than one year and a half of benefiting from the GDAL 
sponsored maintenance program, it is time to share with the community 
what work has been accomplished thanks to that sponsorship.  It is quite 
hard to summarize, as the amount of tasks reaches 1800 for a total of 
1313 hours. So lots of small tasks that make the daily reality of 
maintaining a project like GDAL.



Their global repartition is the following one:

 *

   Bug fixing: 51% of actions, 55% of time spent (including 177 bug
   fixes related to issues found by oss-fuzz)

 *

   Enhancements: 5% of actions, 15% of time spent

 *

   CMake related work: 5% of actions, 10% of time spent

 *

   Review of contributions: 20% of actions, 5% of time spent

 *

   Release management: 2% of actions, 4% of time spent

 *

   Bug triaging and analysis (without corresponding bug fix): 8% of
   actions, 3% of time spent

 *

   Maintenance of continuous integration setup: 3% of actions and time
   spent

 *

   Code enhancements/cleanups: 2% of actions and time spent

 *

   Documentation: 1% of actions and time spent

 *

   Other activities: mailing list discussions, bug reports to other
   projects, meetings, etc.


This work has benefited mostly to GDAL (80%), PROJ (15%), libtiff (2%), 
openjpeg (1%) and more marginally to other projects associated with GDAL 
such as Xerces-C, poppler, libgeotiff, cppcheck, netcdf, curl, libjxl 
and shapelib.



The CMake related work has been a major item of work for the mid 2021 - 
beginning of 2022 period. I cannot overstate the importance of the 
initial contribution of this work made by Hiroshi Miura who brought most 
of the initial material. That said, without the sponsored maintenance 
program which helped polishing and making it production ready for all 
environments supported by GDAL, this would likely have remained a 
out-of-tree project. I believe most stakeholders (contributors and 
users, at least the ones who build from source) are very satisfied with 
this transition from the historic build systems to a unified and more 
modern one, with consistent option naming. Building on Windows is in 
particular much easier nowadays, in particular when leveraging 
dependencies from distributions like vcpkg or Conda Forge. For GDAL 
developers, the new build system offers integration with IDEs and solves 
long standing annoyances like missing header dependency rules for 
partial rebuilds.



Although it doesn’t show up particularly in the above statistics, making 
sure that the continuous integration configurations remain “green” at 
all times is a constant source of attention. Given the number of 
environments tested and the number of dependencies of GDAL, there is 
hardly a week where some action is not needed in that area to make sure 
that the code builds and compiles cleanly, tests pass, etc. We can now 
track a few rolling distributions to detect as early as possible sources 
of incompatibilities with our dependencies, and act early: report issues 
to upstream if there are bugs, or do changes in our code & build scripts 
to take them into account.



Making sure that code contributions from others are reviewed in a timely 
manner is important for the contributor experience. The average delay 
for a submitted pull request (excluding mine) to be commented by me (or 
merged if no comment needed) is 22 hours, and the median time 3h20 
(statistics gathered on 601 pull requests since August 2021, source 
script at https://gist.github.com/rouault/0fbd37f59b8e93ae63761468a5600262)



In the category of regular tasks, one can also mention: updating the 
EPSG dataset in PROJ (and coordinating with IOGP when detecting issues), 
refreshing vendored of copies in the GDAL source tree (libtiff 
typically), making sure that new SQLite releases play nicely with PROJ 
which has some stressing SQL queries (we spot recently a performance 
regression in SQLite 3.41.0 and reported it to upstream)



The following RFCs have been implemented (not mentioning a few 
procedural ones) thanks to the sponsorship:


 *

   RFC 87: Signed int8 data type for raster
   

 *

   RFC 88: Use GoogleTest framework for C/C++ unit tests
   

 *

   RFC 90: Direct access to compressed raster data
   

 *

   RFC 91: GDALDataset::Close() method
   

 *

   RFC 93: OGRLayer::UpdateFeature() method
   


Other recent significant work includes enhancements in Python exception 
handling (in preparation for enabling them by default in a future 4.0 
release), and enabling them in the regression test suite and utilities.



The following feature releases have been issued: 3.4.0, 3.5.0 and 3.6.0. 
And the following bug fixes releases: 3.3.2, 3.3.3, 3