[gdal-dev] GDAL Maintainers Meeting Minutes
Howard Butler, Even Rouault, Dan Baston, and Alessandro Pasotti canceled the monthly GDAL Maintainers Meeting that was to be held on 10/26/2023 due to vacation and FOSS4GNA conflicts. Maintainers instead provided text updates which I am relaying as our monthly report. Fundraising update FrontierSI has declined to renew its Bronze sponsorship. More renewal traffic is expected in the next few months as we work to finish this year's cycle. OSGeo Community Sprint -- Howard Butler, Even Rouault, Jukka Rahkonen, and Dan Baston are currently signed up to attend the OSGeo Community Sprint next week in Vienna https://wiki.osgeo.org/wiki/OSGeo_Community_Sprint_2023 GDAL PSC members can obtain travel grants through the sponsorship program if they wish to attend (contact me for info). Maintenance activities update -- Status from Dan: > * Fixed a bug in /vsimem that was causing some tests to not clean up after > themselves. > * Fixed more Python bindings crashes as a result of test execution > independence effort. Status from Alessandro: > This month I reviewed a few PRs (mainly from Even): > https://github.com/OSGeo/gdal/pull/8534 > https://github.com/OSGeo/gdal/pull/8531 > https://github.com/OSGeo/gdal/pull/8529 > https://github.com/OSGeo/gdal/pull/8526 > https://github.com/OSGeo/gdal/pull/8521 > https://github.com/OSGeo/gdal/pull/8519 > https://github.com/OSGeo/gdal/pull/8518 > https://github.com/OSGeo/gdal/pull/8516 > https://github.com/OSGeo/gdal/pull/8495 > > Then I focused on investigating and fixing the following bugs: > https://github.com/OSGeo/gdal/issues/8525 OpenCL always uses first > device per platform on Mac > https://github.com/OSGeo/gdal/issues/8523 Error in > gdal.VectorTranslate for explodecollections=True + some weird python > logging behaviour > https://github.com/OSGeo/gdal/issues/8522 OAPIF: Increase the default page > size > > Lastly, starting from a QGIS upstream issue > https://github.com/qgis/QGIS/issues/54966 I worked on the GDAL side of > the problem and wrote a test https://github.com/OSGeo/gdal/pull/8609, > while investigating the issue I discovered another issue which was > eventually fixed by Even with https://github.com/OSGeo/gdal/pull/8582 Status from Even: > - Regeneration of GDAL 3.7.2 Docker images for WebP and curl recent CVEs > - Add OGRLayer::WriteArrowBatch() functionality which is the write-side of > RFC86, with a generic implementation for all drivers and specialized > implementations for Arrow/Parquet:https://github.com/OSGeo/gdal/pull/8517- > and use it in ogr2ogr when possible, that is when the source layer declares > OLCFastGetArrowStream to give significant speed-ups (including for > GeoPackage->GeoPackage): https://github.com/OSGeo/gdal/pull/8544 > +https://github.com/OSGeo/gdal/pull/8560 . Note that is currently only > enabled with a very restricted set of ogr2ogr options (mostly only -spat, > -where and driver creation options) > - Massive speed-up of GeoPackage spatial index creation. From 26 seconds down > to 6 seconds on a 3.2 million feature table. > https://github.com/OSGeo/gdal/pull/8596 . Also available as a separate > library that can be used by non-GDAL implementations (libspatialite for > example) > - Significant performance improvement of RasterIO() on VRT with > VRTComplexSource that have only a element and data types > Byte/UInt16/Int16: https://github.com/OSGeo/gdal/pull/8562 > - CI: addition of (an embryo of) a performance regression test suite. Note > that due to CI virtual machines being quite unpredictable performance-wise, > it can only detect regressions > 20%. https://github.com/OSGeo/gdal/pull/8538 > - and ~50 other activities: bug fixes, pull request reviews The next GDAL Maintainers Meeting is 11/23/2023 at 9:00 EDT. Any PSC members are welcome to join by reaching out to me for an invite. Howard ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 3.8.0beta1 available for testing
The ghcr.io/osgeo/gdal:ubuntu-small-latest, ghcr.io/osgeo/gdal:ubuntu-full-latest, ghcr.io/osgeo/gdal:alpine-normal-latest Docker images have been refreshed with 3.8.0beta1 (ghcr.io/osgeo/gdal:alpine-small-latest still building at time of writing). All -latest images are now at 3.8.0beta1 (Note: the GDAL master conda builds mentioned at https://gdal.org/download.html#gdal-master-conda-builds have been broken for a couple weeks and are thus not usable to test beta1 currently. I'm investigating) Issue fixed. The procedure at https://gdal.org/download.html#gdal-master-conda-builds will now fetch 3.8.0beta1 -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: OSGeo Community Sprint Financial Support
+1 Daniel On 2023-10-31 12:59, Howard Butler via gdal-dev wrote: Dear PSC, Sorry for the short notice, but I would like to motion that the GDAL Sponsorship Program financially support GDAL PSC members who wish to attend the OSGeo Community Sprint in Vienna [1] next week. Support level is capped at 1000€ for European attendees and 1500€ for others. Howard [1] https://wiki.osgeo.org/wiki/OSGeo_Community_Sprint_2023 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Daniel Morissette Mapgears Inc T: +1 418-696-5056 #201 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: OSGeo Community Sprint Financial Support
+1 Javier On Tue, 31 Oct 2023, 18:04 Even Rouault via gdal-dev, < gdal-dev@lists.osgeo.org> wrote: > +1 Even > > Le 31/10/2023 à 17:59, Howard Butler via gdal-dev a écrit : > > Dear PSC, > > > > Sorry for the short notice, but I would like to motion that the GDAL > Sponsorship Program financially support GDAL PSC members who wish to attend > the OSGeo Community Sprint in Vienna [1] next week. Support level is capped > at 1000€ for European attendees and 1500€ for others. > > > > Howard > > > > [1] https://wiki.osgeo.org/wiki/OSGeo_Community_Sprint_2023 > > ___ > > gdal-dev mailing list > > gdal-dev@lists.osgeo.org > > https://lists.osgeo.org/mailman/listinfo/gdal-dev > > -- > http://www.spatialys.com > My software is free, but my time generally not. > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 3.8.0beta1 available for testing
Le 31/10/2023 à 17:36, Rahkonen Jukka a écrit : Hi, I made a simple test with ogr2ogr and geopackage to geopackage on Windows. With GDAL 3.8.0dev-3e4dc710a2 (no arrow, old R-Tree) the timing was 36 minutes, with GDAL 3.8.0dev-6bbd2c080a the same conversion took 21 minutes. The gpkg file is 75 GB, with 120 layers. Could it be that the new R-Tree implementation does not print anything into debug? Indeed. Debug traces added back in https://github.com/OSGeo/gdal/pull/8640 Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: OSGeo Community Sprint Financial Support
+1 Even Le 31/10/2023 à 17:59, Howard Butler via gdal-dev a écrit : Dear PSC, Sorry for the short notice, but I would like to motion that the GDAL Sponsorship Program financially support GDAL PSC members who wish to attend the OSGeo Community Sprint in Vienna [1] next week. Support level is capped at 1000€ for European attendees and 1500€ for others. Howard [1] https://wiki.osgeo.org/wiki/OSGeo_Community_Sprint_2023 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Motion: OSGeo Community Sprint Financial Support
Dear PSC, Sorry for the short notice, but I would like to motion that the GDAL Sponsorship Program financially support GDAL PSC members who wish to attend the OSGeo Community Sprint in Vienna [1] next week. Support level is capped at 1000€ for European attendees and 1500€ for others. Howard [1] https://wiki.osgeo.org/wiki/OSGeo_Community_Sprint_2023 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 3.8.0beta1 available for testing
Hi, I made a simple test with ogr2ogr and geopackage to geopackage on Windows. With GDAL 3.8.0dev-3e4dc710a2 (no arrow, old R-Tree) the timing was 36 minutes, with GDAL 3.8.0dev-6bbd2c080a the same conversion took 21 minutes. The gpkg file is 75 GB, with 120 layers. Could it be that the new R-Tree implementation does not print anything into debug? With older GDAL I see GPKG: Creating insert/delete feature_count triggers GPKG: 194505 rows inserted into rtree_virtausnuoli_geom but with the brand new just GPKG: Creating insert/delete feature_count triggers GPKG: Creating insert/delete feature_count triggers -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: gdal-dev Puolesta Even Rouault via gdal-dev Lähetetty: tiistai 31. lokakuuta 2023 16.33 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] GDAL 3.8.0beta1 available for testing Hi, I've prepared a beta1 of GDAL 3.8.0 to get feedback from earlier testers. Sorry no updated NEWS.md file yet, but I'd in particular be interesting by testing of ogr2ogr workflows, since they have underwent significant changes in the underlying implementation: - when the source layer is a layer implementing the ArrowStream API (that is GeoPackage, FlatGeoBuf, Arrow or Parquet), and when no ogr2ogr options than -of, -where, -spat, -lco, -dsco, -gt, -append/-overwrite/-update are used (and -sql as well for GeoPackage). When enabling CPL_DEBUG, you'll see a "OGR2OGR: Using WriteArrowBatch()" trace when that new code path is taken. If specifying other options, the feature-iteration-based traditional implementation is used - and/or when the output layer is GeoPackage (new layer), due to the revamped much faster spatial index creation. This enhanced spatial index creation is not ogr2ogr specific and is actually available more generally for CreateLayer() + CreateFeature() or CreateLayer() + WriteArrowBatch() scenarios. Point of attention would be when in situations with large files and/or with low RAM. The ghcr.io/osgeo/gdal:ubuntu-small-latest, ghcr.io/osgeo/gdal:ubuntu-full-latest, ghcr.io/osgeo/gdal:alpine-normal-latest Docker images have been refreshed with 3.8.0beta1 (ghcr.io/osgeo/gdal:alpine-small-latest still building at time of writing). (Note: the GDAL master conda builds mentioned at https://gdal.org/download.html#gdal-master-conda-builds have been broken for a couple weeks and are thus not usable to test beta1 currently. I'm investigating) Source snapshots at: - https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.gz - https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.xz - https://download.osgeo.org/gdal/3.8.0/gdal380beta1.zip Autotest snapshots: - https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.tar.gz - https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.zip Even -- http://www.spatialys.com/ My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 3.8.0beta1 available for testing
Hi, I know it doesn't mean much (with them being unofficial and pretty incomplete), but the Rust bindings appear to work well with both 3.7.3 rc1 and 3.8.0 beta1. Laurentiu On Tue, Oct 31, 2023, at 16:33, Even Rouault via gdal-dev wrote: > Hi, > > I've prepared a beta1 of GDAL 3.8.0 to get feedback from earlier testers. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] GDAL 3.8.0beta1 available for testing
Hi, I've prepared a beta1 of GDAL 3.8.0 to get feedback from earlier testers. Sorry no updated NEWS.md file yet, but I'd in particular be interesting by testing of ogr2ogr workflows, since they have underwent significant changes in the underlying implementation: - when the source layer is a layer implementing the ArrowStream API (that is GeoPackage, FlatGeoBuf, Arrow or Parquet), and when no ogr2ogr options than -of, -where, -spat, -lco, -dsco, -gt, -append/-overwrite/-update are used (and -sql as well for GeoPackage). When enabling CPL_DEBUG, you'll see a "OGR2OGR: Using WriteArrowBatch()" trace when that new code path is taken. If specifying other options, the feature-iteration-based traditional implementation is used - and/or when the output layer is GeoPackage (new layer), due to the revamped much faster spatial index creation. This enhanced spatial index creation is not ogr2ogr specific and is actually available more generally for CreateLayer() + CreateFeature() or CreateLayer() + WriteArrowBatch() scenarios. Point of attention would be when in situations with large files and/or with low RAM. The ghcr.io/osgeo/gdal:ubuntu-small-latest, ghcr.io/osgeo/gdal:ubuntu-full-latest, ghcr.io/osgeo/gdal:alpine-normal-latest Docker images have been refreshed with 3.8.0beta1 (ghcr.io/osgeo/gdal:alpine-small-latest still building at time of writing). (Note: the GDAL master conda builds mentioned at https://gdal.org/download.html#gdal-master-conda-builds have been broken for a couple weeks and are thus not usable to test beta1 currently. I'm investigating) Source snapshots at: - https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.gz - https://download.osgeo.org/gdal/3.8.0/gdal-3.8.0beta1.tar.xz - https://download.osgeo.org/gdal/3.8.0/gdal380beta1.zip Autotest snapshots: - https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.tar.gz - https://download.osgeo.org/gdal/3.8.0/gdalautotest-3.8.0beta1.zip Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] GDAL 3.7.3 RC1 available
3.7.3 RC1 OK for 630 R packages using GDAL through released versions of R packages sf, terra, gdalcubes, gdalraster and/or vapour; no observed test failures related to GDAL. Roger -- Roger Bivand Emeritus Professor Norwegian School of Economics Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway roger.biv...@nhh.no ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] gif gfw and prj
Hi, I made a quick test with some random .gif file from my computer and your world file and those two do find each other without problems. The format of a_datum.prj is not familiar to me. Maybe it is Global Mapper specific. Maybe you can just delete it and use the EPSG:32632 code instead in gdal_translate as -a_srs EPSG:32632. gdalinfo gif.gif Driver: BIGGIF/Graphics Interchange Format (.gif) Files: gif.gif gif.gfw Size is 12000, 12000 Origin = (243987.500,5120012.500) Pixel Size = (25.000,-25.000) Image Structure Metadata: INTERLACED=NO Corner Coordinates: Upper Left ( 243987.500, 5120012.500) Lower Left ( 243987.500, 4820012.500) Upper Right ( 543987.500, 5120012.500) Lower Right ( 543987.500, 4820012.500) Center ( 393987.500, 4970012.500) -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: gdal-dev Puolesta mauge.philippe--- via gdal-dev Lähetetty: tiistai 31. lokakuuta 2023 11.19 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] gif gfw and prj Hi, I have a gif file named img_512_244000.gif with a world file named img_512_244000.gfw that contains : 25.00 0.00 0.00 -25.00 244000.00 512.00 and a prj file named a_datum.prj that contains : Projection UTM Datum WGS84 Zunits NO Units METERS Zone 32 Xshift 0.00 Yshift 0.00 Parameters The 3 files are in the same folder. When I open the gif file with GDAL using GDALOpen, GDAL doesn't read the gfw and the prj files and there is no projection. Is there a way to say to GDAL to read the gfw and the prj files ? I have successfully loaded the file with Global Mapper that read the gfw and the prj files. The gif file can be uploaded at : [link removed] If you have any clue, please let me know. Have a good day Philippe ___ gdal-dev mailing list gdal-dev@lists.osgeo.org ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] gif gfw and prj
Hi, I have a gif file named img_512_244000.gif with a world file named img_512_244000.gfw that contains : 25.00 0.00 0.00 -25.00 244000.00 512.00 and a prj file named a_datum.prj that contains : Projection UTM Datum WGS84 Zunits NO Units METERS Zone 32 Xshift 0.00 Yshift 0.00 Parameters The 3 files are in the same folder. When I open the gif file with GDAL using GDALOpen, GDAL doesn't read the gfw and the prj files and there is no projection. Is there a way to say to GDAL to read the gfw and the prj files ? I have successfully loaded the file with Global Mapper that read the gfw and the prj files. The gif file can be uploaded at : https://pasteboard.co/G5tIWsZ8w7AC.gif If you have any clue, please let me know. Have a good day Philippe ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
