Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
Will do. Thanks! From: "Even Rouault" To: "Rahkonen Jukka (MML)" , "hays" Cc: gdal-dev@lists.osgeo.org Sent: Tuesday, March 30, 2021 11:12:38 AM Subject: Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss. Jukka' suggestion is a good one. Using -tps solver or affine approximation -order 1 fixes the issue. The default behaviour with GCPs when there are 6 or more of them is to use the second order polynomial adjustment. The issue here is when computing output bounds. We actually compute two 2nd order polynomial adjustments: one from source image coordinates to georeferenced ones, and one from georeferenced ones to source image. The former one is used to compute the output bounds, but the second one is used to do warping. The issue with 2nd order polynomial is that those two transformers are generally not the inverse of each other... We should likely compute the source image coordinates -> georeferenced coordinates as an iterative inversion of the other one, so they are invertible. The first order polynomial / affine transformation hasn't that issue due to how affine transformation works. The TPS one isn't likely perfectly invertible but TPS transformation is guaranteed to be exact at GCPs, so that diminishes the effect. But 2nd order polynomial has overshoots particularities that can cause the issue observed. Another workaround is to specify the desired output bounds with -te Anyway, Hays, could you file an issue about that in [ https://github.com/OSGeo/gdal/issues | https://github.com/OSGeo/gdal/issues ] , ideally copying over your web page (or pointing to it if it can remain stable), and pasting my above comment. Even Le 30/03/2021 à 18:51, Rahkonen Jukka (MML) a écrit : Hi, I did not have a close look into your ground control points but perhaps they are not very well distributed and therefore the default warping algorithm eats one corner of the image. Have a try with thin plate splin (-tps) instead. There may be some bug, it does not feel right that some source data are lost with default algorithm. -Jukka Rahkonen- Lähettäjä: Hays Barrett [ mailto:hays.barr...@gmail.com | ] Lähetetty: tiistai 30. maaliskuuta 2021 19.32 Vastaanottaja: Rahkonen Jukka (MML) [ mailto:jukka.rahko...@maanmittauslaitos.fi | ] Kopio: [ mailto:gdal-dev@lists.osgeo.org | gdal-dev@lists.osgeo.org ] Aihe: Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss. I put together a page that better explains the issue and used one of the actual images from our project. [ https://haysbarrett.com/gdal_translate/ | https://haysbarrett.com/gdal_translate/ ] Thanks for looking into this for me. -Hays On Fri, Mar 26, 2021 at 2:59 PM jratike80 < [ mailto:jukka.rahko...@maanmittauslaitos.fi | jukka.rahko...@maanmittauslaitos.fi ] > wrote: BQ_BEGIN Hi, Please give an exact example. Preferably with a raster that can be created from command line with gdal_create followed by other required commands. Or alternatively Python code that re-produces the issue. -Jukka Rahkonen- Hays Barrett wrote > If the gcps stretch the image beyond the dimensions of the original raster > it appears to clip the data that sits outside of the original raster > dimensions rather than increase the raster dimensions to fit the stretched > data . > Is this the expected behavior? > If so is there a way to increase the dimensions of a raster so there is no > data loss? > Thanks, > -Hays > > ___ > gdal-dev mailing list > [ mailto:gdal-dev@.osgeo | gdal-dev@.osgeo ] > [ https://lists.osgeo.org/mailman/listinfo/gdal-dev | > https://lists.osgeo.org/mailman/listinfo/gdal-dev ] -- Sent from: [ http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html | http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html ] ___ gdal-dev mailing list [ mailto:gdal-dev@lists.osgeo.org | gdal-dev@lists.osgeo.org ] [ https://lists.osgeo.org/mailman/listinfo/gdal-dev | https://lists.osgeo.org/mailman/listinfo/gdal-dev ] ___ gdal-dev mailing list [ mailto:gdal-dev@lists.osgeo.org | gdal-dev@lists.osgeo.org ] [ https://lists.osgeo.org/mailman/listinfo/gdal-dev | https://lists.osgeo.org/mailman/listinfo/gdal-dev ] BQ_END -- [ http://www.spatialys.com/ | http://www.spatialys.com ] My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
Jukka' suggestion is a good one. Using -tps solver or affine approximation -order 1 fixes the issue. The default behaviour with GCPs when there are 6 or more of them is to use the second order polynomial adjustment. The issue here is when computing output bounds. We actually compute two 2nd order polynomial adjustments: one from source image coordinates to georeferenced ones, and one from georeferenced ones to source image. The former one is used to compute the output bounds, but the second one is used to do warping. The issue with 2nd order polynomial is that those two transformers are generally not the inverse of each other... We should likely compute the source image coordinates -> georeferenced coordinates as an iterative inversion of the other one, so they are invertible. The first order polynomial / affine transformation hasn't that issue due to how affine transformation works. The TPS one isn't likely perfectly invertible but TPS transformation is guaranteed to be exact at GCPs, so that diminishes the effect. But 2nd order polynomial has overshoots particularities that can cause the issue observed. Another workaround is to specify the desired output bounds with -te Anyway, Hays, could you file an issue about that in https://github.com/OSGeo/gdal/issues, ideally copying over your web page (or pointing to it if it can remain stable), and pasting my above comment. Even Le 30/03/2021 à 18:51, Rahkonen Jukka (MML) a écrit : Hi, I did not have a close look into your ground control points but perhaps they are not very well distributed and therefore the default warping algorithm eats one corner of the image. Have a try with thin plate splin (-tps) instead. There may be some bug, it does not feel right that some source data are lost with default algorithm. -Jukka Rahkonen- *Lähettäjä:*Hays Barrett *Lähetetty:* tiistai 30. maaliskuuta 2021 19.32 *Vastaanottaja:* Rahkonen Jukka (MML) *Kopio:* gdal-dev@lists.osgeo.org *Aihe:* Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss. I put together a page that better explains the issue and used one of the actual images from our project. https://haysbarrett.com/gdal_translate/ <https://haysbarrett.com/gdal_translate/> Thanks for looking into this for me. -Hays On Fri, Mar 26, 2021 at 2:59 PM jratike80 <mailto:jukka.rahko...@maanmittauslaitos.fi>> wrote: Hi, Please give an exact example. Preferably with a raster that can be created from command line with gdal_create followed by other required commands. Or alternatively Python code that re-produces the issue. -Jukka Rahkonen- Hays Barrett wrote > If the gcps stretch the image beyond the dimensions of the original raster > it appears to clip the data that sits outside of the original raster > dimensions rather than increase the raster dimensions to fit the stretched > data . > Is this the expected behavior? > If so is there a way to increase the dimensions of a raster so there is no > data loss? > Thanks, > -Hays > > ___ > gdal-dev mailing list > gdal-dev@.osgeo <mailto:gdal-dev@.osgeo> > https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev> -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html <http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html> ___ gdal-dev mailing list gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org> https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev> ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
Hi, I did not have a close look into your ground control points but perhaps they are not very well distributed and therefore the default warping algorithm eats one corner of the image. Have a try with thin plate splin (-tps) instead. There may be some bug, it does not feel right that some source data are lost with default algorithm. -Jukka Rahkonen- Lähettäjä: Hays Barrett Lähetetty: tiistai 30. maaliskuuta 2021 19.32 Vastaanottaja: Rahkonen Jukka (MML) Kopio: gdal-dev@lists.osgeo.org Aihe: Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss. I put together a page that better explains the issue and used one of the actual images from our project. https://haysbarrett.com/gdal_translate/ Thanks for looking into this for me. -Hays On Fri, Mar 26, 2021 at 2:59 PM jratike80 mailto:jukka.rahko...@maanmittauslaitos.fi>> wrote: Hi, Please give an exact example. Preferably with a raster that can be created from command line with gdal_create followed by other required commands. Or alternatively Python code that re-produces the issue. -Jukka Rahkonen- Hays Barrett wrote > If the gcps stretch the image beyond the dimensions of the original raster > it appears to clip the data that sits outside of the original raster > dimensions rather than increase the raster dimensions to fit the stretched > data . > Is this the expected behavior? > If so is there a way to increase the dimensions of a raster so there is no > data loss? > Thanks, > -Hays > > ___ > gdal-dev mailing list > gdal-dev@.osgeo<mailto:gdal-dev@.osgeo> > https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org> https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
I put together a page that better explains the issue and used one of the actual images from our project. https://haysbarrett.com/gdal_translate/ Thanks for looking into this for me. -Hays - Original Message - From: "jratike80" To: gdal-dev@lists.osgeo.org Sent: Friday, March 26, 2021 2:59:36 PM Subject: Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss. Hi, Please give an exact example. Preferably with a raster that can be created from command line with gdal_create followed by other required commands. Or alternatively Python code that re-produces the issue. -Jukka Rahkonen- Hays Barrett wrote > If the gcps stretch the image beyond the dimensions of the original raster > it appears to clip the data that sits outside of the original raster > dimensions rather than increase the raster dimensions to fit the stretched > data . > Is this the expected behavior? > If so is there a way to increase the dimensions of a raster so there is no > data loss? > Thanks, > -Hays > > ___ > gdal-dev mailing list > gdal-dev@.osgeo > https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
Hi, Please give an exact example. Preferably with a raster that can be created from command line with gdal_create followed by other required commands. Or alternatively Python code that re-produces the issue. -Jukka Rahkonen- Hays Barrett wrote > If the gcps stretch the image beyond the dimensions of the original raster > it appears to clip the data that sits outside of the original raster > dimensions rather than increase the raster dimensions to fit the stretched > data . > Is this the expected behavior? > If so is there a way to increase the dimensions of a raster so there is no > data loss? > Thanks, > -Hays > > ___ > gdal-dev mailing list > gdal-dev@.osgeo > https://lists.osgeo.org/mailman/listinfo/gdal-dev -- Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Adding gcps to a raster using gdal_translate can result in data loss.
If the gcps stretch the image beyond the dimensions of the original raster it appears to clip the data that sits outside of the original raster dimensions rather than increase the raster dimensions to fit the stretched data . Is this the expected behavior? If so is there a way to increase the dimensions of a raster so there is no data loss? Thanks, -Hays ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
