Re: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release
On 11/8/23 09:39, Even Rouault via gdal-dev wrote: Adopt GDAL 3.8.0RC1 as 3.8.0 release Can we get this fixed before the release? https://github.com/OSGeo/gdal/pull/8682 Kind Regards, Bas -- GPG Key ID: 4096R/6750F10AE88D4AF1 Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1 ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use.
I tried using both: gdalwarp -ct "$(projinfo -s EPSG:29902 -t EPSG:4326 -o proj -q)" input.tif output.tif gdalwarp -ct "$(projinfo -s EPSG:29902 -t EPSG:4326 -o proj -q --bbox -10.562,52.08,-10.561,52.1)" input.tif output.tif and they both produce the desired result of a smooth transition across the area of use. Thanks so much for your help. Sam From: Sam Hall Sent: Wednesday, November 8, 2023 14:57 To: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use. I didn't mean to just respond to you Javier, sorry about that. That looks promising. I'll work out which is the one is using which transformation and see if I can use the right one. Sam From: Javier Jimenez Shaw Sent: Wednesday, November 8, 2023 14:39 To: Sam Hall Subject: Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use. projinfo -s EPSG:29902 -t EPSG:4326 -o proj Candidate operations found: 1 - Operation No. 1: unknown id, Inverse of Irish Grid + TM65 to WGS 84 (2), 1 m, Ireland - onshore. United Kingdom (UK) - Northern Ireland (Ulster) - onshore. PROJ string: +proj=pipeline +step +inv +proj=tmerc +lat_0=53.5 +lon_0=-8 +k=1.35 +x_0=20 +y_0=25 +a=6377340.189 +rf=299.3249646 +step +inv +proj=longlat +a=6377340.189 +rf=299.3249646 +step +proj=push +v_3 +step +proj=cart +a=6377340.189 +rf=299.3249646 +step +proj=helmert +x=482.5 +y=-130.6 +z=564.6 +rx=-1.042 +ry=-0.214 +rz=-0.631 +s=8.15 +convention=position_vector +step +inv +proj=cart +ellps=WGS84 +step +proj=pop +v_3 +step +proj=unitconvert +xy_in=rad +xy_out=deg +step +proj=axisswap +order=2,1 projinfo -s EPSG:29902 -t EPSG:4326 -o proj --bbox "-10.562,52.08,-10.561,52.1" Candidate operations found: 1 - Operation No. 1: unknown id, Inverse of Irish Grid + Ballpark geographic offset from TM65 to WGS 84, unknown accuracy, World, has ballpark transformation PROJ string: +proj=pipeline +step +inv +proj=tmerc +lat_0=53.5 +lon_0=-8 +k=1.35 +x_0=20 +y_0=25 +a=6377340.189 +rf=299.3249646 +step +proj=unitconvert +xy_in=rad +xy_out=deg +step +proj=axisswap +order=2,1 So I guess that somehow one is taking the datum transformation and the other doesn't. Which one is doing what you should check. BTW, you are writing only to me, not to the mailing list. Are you aware of that? Best, Javier. On Wed, 8 Nov 2023 at 15:23, Sam Hall wrote: Thanks Javier, It doesn't happen on the Irish mainland. Is it possible to define the transformation to use outside the area of usage of the datum transformation using a proj string with something like this "+proj=hgridshift +grids=first,second" in it? Or are you suggesting that gdalwarp doesn't take the area of usage into account and therefore wouldn't look for the second datum shift and use a ballpark transform. Sam From: Javier Jimenez Shaw Sent: Wednesday, November 8, 2023 13:47 To: Sam Hall Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use. Assuming that it does not happen in the main area of Ireland...I guess it is because it is outside of the area of usage of the datum transformation https://epsg.org/transformation_1641/TM65-to-WGS-84-2.html Probably (this is just a guess) gdalwarp is not taking it into consideration, making a ballpark transformation between TM65 and WGS84, while ogr2ogr doesn't. On Wed, 8 Nov 2023 at 13:35, Sam Hall via gdal-dev wrote: I have some data that covers the whole of the Republic of Ireland, including the Blasket Islands off the West Coast, situated around 24439, 95540 in EPSG:29902. These islands straddle the western boundary of the area of use of the coordinate system -10.56° W. When transforming a raster dataset from EPSG:29902 to EPSG:4326 using gdalwarp at gdal >= 3.5.2, there seems to be a roughly 50 m shift in the data at the area of use boundary, while if I use ogr2ogr on a vector copy of this data, I don't see this shift. I have a reproducible example of this at https://gitlab.com/Sam.Hall1/29902_gdalwarp_vs_ogr2ogr.git. In this example I created a rectangular polygon with interpolated points and saved it to a shapefile, then used gdal_rasterize to create a GeoTiff. gdalwarp gives the desired result using a gdal 3.5.1 docker image. The full commands are in the docker-compose.yml in the repository but the parameter's I'm using are: gdalwarp -t_srs EPSG:4326 -tr 0.45 0.45 -tap -overwrite input.tif output.tif ogr2ogr -t_srs EPSG:4326 output.shp input.shp I've also seen similar results when transforming from EPSG:27700 to EPSG:4326 in Jersey and Guernsey in the English Channel but I'd have to use a much earlier version of gdal to reproduce the desired result. Is there a gridshift file or any additional parameters that I could use to get the
Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use.
I didn't mean to just respond to you Javier, sorry about that. That looks promising. I'll work out which is the one is using which transformation and see if I can use the right one. Sam From: Javier Jimenez Shaw Sent: Wednesday, November 8, 2023 14:39 To: Sam Hall Subject: Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use. projinfo -s EPSG:29902 -t EPSG:4326 -o proj Candidate operations found: 1 - Operation No. 1: unknown id, Inverse of Irish Grid + TM65 to WGS 84 (2), 1 m, Ireland - onshore. United Kingdom (UK) - Northern Ireland (Ulster) - onshore. PROJ string: +proj=pipeline +step +inv +proj=tmerc +lat_0=53.5 +lon_0=-8 +k=1.35 +x_0=20 +y_0=25 +a=6377340.189 +rf=299.3249646 +step +inv +proj=longlat +a=6377340.189 +rf=299.3249646 +step +proj=push +v_3 +step +proj=cart +a=6377340.189 +rf=299.3249646 +step +proj=helmert +x=482.5 +y=-130.6 +z=564.6 +rx=-1.042 +ry=-0.214 +rz=-0.631 +s=8.15 +convention=position_vector +step +inv +proj=cart +ellps=WGS84 +step +proj=pop +v_3 +step +proj=unitconvert +xy_in=rad +xy_out=deg +step +proj=axisswap +order=2,1 projinfo -s EPSG:29902 -t EPSG:4326 -o proj --bbox "-10.562,52.08,-10.561,52.1" Candidate operations found: 1 - Operation No. 1: unknown id, Inverse of Irish Grid + Ballpark geographic offset from TM65 to WGS 84, unknown accuracy, World, has ballpark transformation PROJ string: +proj=pipeline +step +inv +proj=tmerc +lat_0=53.5 +lon_0=-8 +k=1.35 +x_0=20 +y_0=25 +a=6377340.189 +rf=299.3249646 +step +proj=unitconvert +xy_in=rad +xy_out=deg +step +proj=axisswap +order=2,1 So I guess that somehow one is taking the datum transformation and the other doesn't. Which one is doing what you should check. BTW, you are writing only to me, not to the mailing list. Are you aware of that? Best, Javier. On Wed, 8 Nov 2023 at 15:23, Sam Hall wrote: Thanks Javier, It doesn't happen on the Irish mainland. Is it possible to define the transformation to use outside the area of usage of the datum transformation using a proj string with something like this "+proj=hgridshift +grids=first,second" in it? Or are you suggesting that gdalwarp doesn't take the area of usage into account and therefore wouldn't look for the second datum shift and use a ballpark transform. Sam From: Javier Jimenez Shaw Sent: Wednesday, November 8, 2023 13:47 To: Sam Hall Cc: gdal-dev@lists.osgeo.org Subject: Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use. Assuming that it does not happen in the main area of Ireland...I guess it is because it is outside of the area of usage of the datum transformation https://epsg.org/transformation_1641/TM65-to-WGS-84-2.html Probably (this is just a guess) gdalwarp is not taking it into consideration, making a ballpark transformation between TM65 and WGS84, while ogr2ogr doesn't. On Wed, 8 Nov 2023 at 13:35, Sam Hall via gdal-dev wrote: I have some data that covers the whole of the Republic of Ireland, including the Blasket Islands off the West Coast, situated around 24439, 95540 in EPSG:29902. These islands straddle the western boundary of the area of use of the coordinate system -10.56° W. When transforming a raster dataset from EPSG:29902 to EPSG:4326 using gdalwarp at gdal >= 3.5.2, there seems to be a roughly 50 m shift in the data at the area of use boundary, while if I use ogr2ogr on a vector copy of this data, I don't see this shift. I have a reproducible example of this at https://gitlab.com/Sam.Hall1/29902_gdalwarp_vs_ogr2ogr.git. In this example I created a rectangular polygon with interpolated points and saved it to a shapefile, then used gdal_rasterize to create a GeoTiff. gdalwarp gives the desired result using a gdal 3.5.1 docker image. The full commands are in the docker-compose.yml in the repository but the parameter's I'm using are: gdalwarp -t_srs EPSG:4326 -tr 0.45 0.45 -tap -overwrite input.tif output.tif ogr2ogr -t_srs EPSG:4326 output.shp input.shp I've also seen similar results when transforming from EPSG:27700 to EPSG:4326 in Jersey and Guernsey in the English Channel but I'd have to use a much earlier version of gdal to reproduce the desired result. Is there a gridshift file or any additional parameters that I could use to get the output to look more continuous across the area of use line? Sam ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Fwd: oblique cuts on a raster using python GDAL
Hi Naïma, I'm a bit late to the party, but note that the JP2OpenJPEG produces lossy output by default. You need to pass REVERSIBLE=YES and QUALITY=100 as creation options to make it use lossless compression. Laurentiu ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use.
Assuming that it does not happen in the main area of Ireland...I guess it is because it is outside of the area of usage of the datum transformation https://epsg.org/transformation_1641/TM65-to-WGS-84-2.html Probably (this is just a guess) gdalwarp is not taking it into consideration, making a ballpark transformation between TM65 and WGS84, while ogr2ogr doesn't. On Wed, 8 Nov 2023 at 13:35, Sam Hall via gdal-dev wrote: > I have some data that covers the whole of the Republic of Ireland, > including the Blasket Islands off the West Coast, situated around 24439, > 95540 in EPSG:29902. These islands straddle the western boundary of the > area of use of the coordinate system -10.56° W. > > When transforming a raster dataset from EPSG:29902 to EPSG:4326 using > gdalwarp at gdal >= 3.5.2, there seems to be a roughly 50 m shift in the > data at the area of use boundary, while if I use ogr2ogr on a vector copy > of this data, I don't see this shift. > > I have a reproducible example of this at > https://gitlab.com/Sam.Hall1/29902_gdalwarp_vs_ogr2ogr.git. In this > example I created a rectangular polygon with interpolated points and saved > it to a shapefile, then used gdal_rasterize to create a GeoTiff. gdalwarp > gives the desired result using a gdal 3.5.1 docker image. The full commands > are in the docker-compose.yml in the repository but the parameter's I'm > using are: > > gdalwarp -t_srs EPSG:4326 -tr 0.45 0.45 -tap -overwrite input.tif > output.tif > ogr2ogr -t_srs EPSG:4326 output.shp input.shp > > I've also seen similar results when transforming from EPSG:27700 to > EPSG:4326 in Jersey and Guernsey in the English Channel but I'd have to use > a much earlier version of gdal to reproduce the desired result. > > Is there a gridshift file or any additional parameters that I could use to > get the output to look more continuous across the area of use line? > > Sam > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Transforming from EPSG:29902 to EPSG:4326 at the limit of the area of use.
I have some data that covers the whole of the Republic of Ireland, including the Blasket Islands off the West Coast, situated around 24439, 95540 in EPSG:29902. These islands straddle the western boundary of the area of use of the coordinate system -10.56° W. When transforming a raster dataset from EPSG:29902 to EPSG:4326 using gdalwarp at gdal >= 3.5.2, there seems to be a roughly 50 m shift in the data at the area of use boundary, while if I use ogr2ogr on a vector copy of this data, I don't see this shift. I have a reproducible example of this at https://gitlab.com/Sam.Hall1/29902_gdalwarp_vs_ogr2ogr.git. In this example I created a rectangular polygon with interpolated points and saved it to a shapefile, then used gdal_rasterize to create a GeoTiff. gdalwarp gives the desired result using a gdal 3.5.1 docker image. The full commands are in the docker-compose.yml in the repository but the parameter's I'm using are: gdalwarp -t_srs EPSG:4326 -tr 0.45 0.45 -tap -overwrite input.tif output.tif ogr2ogr -t_srs EPSG:4326 output.shp input.shp I've also seen similar results when transforming from EPSG:27700 to EPSG:4326 in Jersey and Guernsey in the English Channel but I'd have to use a much earlier version of gdal to reproduce the desired result. Is there a gridshift file or any additional parameters that I could use to get the output to look more continuous across the area of use line? Sam ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Fwd: oblique cuts on a raster using python GDAL
-- Forwarded message -
De : Naima Dambrine
Date: mar. 7 nov. 2023 à 09:16
Subject: Re: [gdal-dev] oblique cuts on a raster using python GDAL
To: Javier Jimenez Shaw
Hi,
Thank you very much for your help, it let me try and try again, and you are
right when you say that it is possible.
I finally obtain a perfect cut in python, here is the code :
gdal.Warp(
output-raster,
raster-ds,
format= ‘GTiff’,
cutlineDSName= ‘geometry-extraction.shp’,
cropToCutline = True,
xRes = None, yRes=None,
resampleAlg=’nearest’,
multithread= True,
srcNoData=None, dstNodata=255,
options=[‘COMPRESS=DEFLATE’, ‘PREDICTOR=2’, ‘ZLEVEL=9']
)
Have a nice day,
Naïma
Le lun. 6 nov. 2023 à 17:23, Javier Jimenez Shaw a
écrit :
>
>
> On Mon, 6 Nov 2023 at 14:43, Naima Dambrine
> wrote:
>
>> Thank you, can you please paste a zoom image of your result ?
>>
>> because if i translate your gdalwarp in python :
>>
>> gdalwarp 3635_rasters_agreges.jp2 salida.tif -cutline
>> geometry_extraction.shp -crop_to_cutline -dstnodata 0 -cl
>> geometry_extraction -overwrite -of GTiff
>>
>> gdal.Warp('salida.tif', '3635_rasters_agreges.jp2', format='GTiff',
>> cutlineDSName='geometry_extraction.shp', cropToCutline=True, dstNoData=0)
>>
>> it's the same code ... and I do not obtain a clean image at all
>>
>>
>>
>> Le lun. 6 nov. 2023 à 14:05, Javier Jimenez Shaw a
>> écrit :
>>
>>> This is working for me (and also in gdal 3.8.0):
>>>
>>> gdalwarp 3635_rasters_agreges.jp2 salida.tif -cutline
>>> geometry_extraction.shp -crop_to_cutline -dstnodata 0 -cl
>>> geometry_extraction -overwrite -of GTiff
>>>
>>> About the "white" pixels inside the image, it could be that a single
>>> band has a value of 0 (not that strange). Then it is transparent, and you
>>> see the background color. (to avoid those "color misunderstandings" I have
>>> a pink background color, that is not white).
>>>
>>> On Mon, 6 Nov 2023 at 12:35, Naima Dambrine
>>> wrote:
>>>
Hi Javier,
Thank you, good news ...
I'm on Ubuntu 20.04 with gdal 3.6.2. Yes, the original raster format is
JP2, but my output format is GTiff. Here is exactly what I do :
cut_ds = gdal.Warp(outfile, jp2_ds, format='GTiff',
cutlineDCName=shape_file_path,
cropToCutline=True,
copyMetadat=True,
dstNodata=0)
What I see is that there are still black pixels around the image, as
well as white pixels inside the image.
Another point to consider is that, despite the use of compression, the
output file is 75.3 MB, compared with around 14 MB with a JP2 format. Why
is this?
my output :
Le lun. 6 nov. 2023 à 11:33, Javier Jimenez Shaw
a écrit :
> Hi Naima
>
> I have been testing with your dataset. To me, using the GDAL in Ubuntu
> 22.04 (3.4.1) seems to be a problem with the JP2 output format. If you
> output as geotiff it works fine.
>
> On Sun, 5 Nov 2023 at 19:43, Rahkonen Jukka via gdal-dev <
> gdal-dev@lists.osgeo.org> wrote:
>
>> Hi,
>>
>>
>>
>> Please add gdalinfo of the source image. Even better if you can share
>> the image.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* gdal-dev *Puolesta *Naima
>> Dambrine via gdal-dev
>> *Lähetetty:* sunnuntai 5. marraskuuta 2023 17.35
>> *Vastaanottaja:* gdal-dev@lists.osgeo.org
>> *Aihe:* [gdal-dev] oblique cuts on a raster using python GDAL
>>
>>
>>
>> Hi ,
>>
>>
>>
>> I have problems with oblique cuts on a raster using python GDAL
>> (3.6.2)
>>
>>
>>
>> - with this line i obtain black borders around :
>>
>> gdal.warp('raster-dst' , raster-src',
>> cutLineDSName='geometry-extraction.shp', cropToCutline=True)
>>
>>
>>
>> - with this one, the crop is not clean on closer inspection:
>> residual black pixels around image and white pixels appear in the image.
>>
>> gdal. warp( 'raster-dst' , raster-src',
>> cutLineDSName='geometry-extraction.shp', cropToCutline=True,
>> copyMetaData=True, dstNodata=0)
>>
>>
>> I tried, without success, to refine with outputBounds=[minX, maxX,
>> minY, maxY], under QGIS directly ….
>> I've run out of ideas :/
>> A (naive) question comes to mind: Is it possible to make oblique cuts
>> with gdal.warp() & co?
>>
>> Naïma
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release
+1 Javier On Wed, 8 Nov 2023, 10:16 Rahkonen Jukka via gdal-dev, < gdal-dev@lists.osgeo.org> wrote: > +1 > > -Jukka Rahkonen- > > -Alkuperäinen viesti- > Lähettäjä: gdal-dev Puolesta Even > Rouault via gdal-dev > Lähetetty: keskiviikko 8. marraskuuta 2023 10.39 > Vastaanottaja: gdal-dev@lists.osgeo.org > Aihe: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release > > Hi, > > Motion: > > Adopt GDAL 3.8.0RC1 as 3.8.0 release > > Starting with my +1 > > Even > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release
+1 -Jukka Rahkonen- -Alkuperäinen viesti- Lähettäjä: gdal-dev Puolesta Even Rouault via gdal-dev Lähetetty: keskiviikko 8. marraskuuta 2023 10.39 Vastaanottaja: gdal-dev@lists.osgeo.org Aihe: [gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release Hi, Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release Starting with my +1 Even ___ gdal-dev mailing list gdal-dev@lists.osgeo.org ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
[gdal-dev] Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release
Hi, Motion: Adopt GDAL 3.8.0RC1 as 3.8.0 release Starting with my +1 Even -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
