Re: [gdal-dev] Black borders after reprojection GDAL_WARP

[Lorenzo Di Giacomo]

Thanks Even, didn't know about gdalmove. I'll try !!


Il giorno gio 18 nov 2021 alle ore 15:47 Even Rouault <
even.roua...@spatialys.com> ha scritto:

> Black borders are completely expected: reprojection changes the geometry
> of the image.
>
> That said, if the area of interest is sufficiently small and the geometry
> change being approximatively an affine transformation, you can try
> https://gdal.org/programs/gdalmove.html instead of gdalwarp to only alter
> the georeferencing information and not touch pixel values at all.
>
> Even
> Le 18/11/2021 à 15:34, Lorenzo Di Giacomo a écrit :
>
> Hi Carl, thanks for your reply, i noticed that it happens even if i just
> reproject the image, without cutting.
> Of course the dstalpha works, but it increases the size of the image and
> it changes its nature (adding another band) since this operation is an
> intermediate operation the resulting image can't be different from the
> original, just reporojected.
>
>
>
> Il giorno gio 18 nov 2021 alle ore 14:11 Carl Godkin 
> ha scritto:
>
>>  Hi Lorenzo,
>>
>> I have faced this and there are two things to consider.
>>
>> First, you can specify the georeferenced extents _and_ the SRS  of the
>> extents using something like this:
>>
>> gdalwarp -t_srs EPSG:3857 \
>> -te_srs EPSG:4326 -te -109 32 -102 36 \
>> input.tif output.tif
>>
>> Note that I'm warping to one coordinate system but trimming based on
>> another one.  Is it possible that your black boundary is due to trimming in
>> the wrong coordinate system?  For instance, if your input map is projected
>> but the boundary of the map consists of parallels of latitude or
>> meridians of longitude, then you could use something like the above.  (You
>> can actually get even fancier by trimming with polygons too; see this
>> example for inspiration:
>> https://gis.stackexchange.com/questions/45053/gdalwarp-cutline-along-with-shapefile
>> .)
>>
>> Second, recognize that the output will always be a rectangle in the
>> output SRS.  If the black border is due to "no data" areas outside the
>> input map's extent appearing in the output map, then you can use -dstalpha
>> ("Create an output alpha band to identify nodata (unset/transparent)
>> pixels.") to mark the output pixels that shouldn't be part of the map.
>> Basically your black pixels become transparent in this case.
>>
>> I hope that helps,
>> carl
>>
>>
>> On Wed, Nov 17, 2021 at 8:11 AM Lorenzo Di Giacomo 
>> wrote:
>>
>>> Hi all, i have a question, how is it possible to avoid black borders
>>> after reprojection with "gdal_warp" ?
>>> Basically i have an image that is little rotated, when i reprojected
>>> from 32632 to 4326 the resulting image has black borders more or less tight
>>> depending on the rotation.
>>> I saw i can change the colors of those no_data (using -dstnodata) but
>>> how can i do if i dont want it at all? Adding another band result in a size
>>> increase, that i don't want either.
>>>
>>> Thanks !!
>>> ___
>>> gdal-dev mailing list
>>> gdal-dev@lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
> ___
> gdal-dev mailing 
> listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> -- http://www.spatialys.com
> My software is free, but my time generally not.
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer

[Rahkonen Jukka (MML)]

Hi,

Large MaxConnections value leads to more concurrent requests and it may cause 
problems if the WMS server is very busy. Reducing MaxConnections may help if 
you are the only user of that WMS but if there are thousands of other users as 
well the service may still run at 100% and drop request either because they 
reach the server side timeout or because load balancer denies requests. If the 
overload is temporary then retry after a while could help but I do not believe 
that GDAL WMS driver can be configured that way  
https://gdal.org/drivers/raster/wms.html.  Special tile cache programs like 
GeoWebCache are better prepared for failing requests 
https://www.geowebcache.org/docs/current/production/index.html#seed-failure-tolerance

The GDAL WMS configuration options are documented only by example 
https://gdal.org/drivers/raster/wms.html. I wonder where I could find 
information about the meaning of options that I can't guess by the name, like 
"CleanTimeout".

 -Jukka Rahkonen-

Lähettäjä: Roman Breitfuss-Schiffer 
Lähetetty: perjantai 19. marraskuuta 2021 10.18
Vastaanottaja: Rahkonen Jukka (MML) ; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer


Dear Jukka!

Thanks for your answer! I think I'm gonna follow that road you suggested.

We have been thinking a lot and I was just wondering if there could be some 
GDAL configuration which could cause that behaviour? For example the value for 
 in the WMS XML file? Or if more applications are accessing the 
same WMS XML at the same time which is theoretically possible in our case.

Best regards
Roman
Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML):
Hi,

It seems that the WMS server is closing the door and there is nothing else to 
do on the GDAL side except to try again later. Contact the WMS service 
maintainer and report your troubles. They may be able to provide you a more 
reliable service especially if you are ready to pay for it.

-Jukka Rahkonen-

Lähettäjä: gdal-dev 
 
Puolesta Roman Breitfuss-Schiffer
Lähetetty: torstai 18. marraskuuta 2021 11.59
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDALWMS - error: TCP connection reset by peer


Dear all!

We are running a Python application an a linux distribution (RedHat) which 
creates clips in MBTILES format from a WMS. We are using the Python-API and are 
basically doing something like this:

src_ds = gdal.Warp(
tmp_file,
ds_in,
format='GTiff',
outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]],
dstSRS=output_srs,
)
ds_out = gdal.Translate(
out_filepath,
src_ds,
format='MBTILES',
creationOptions=creation_options,
)

The application throughs an error from time to time since a few days. We 
couldn't figure out a pattern yet and therefore we are also not able to 
reproduce the error. The error is the following:

Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 2765, 602.
URL: TCP connection reset by peer
  HTTP status code: 0, error: TCP connection reset by peer.
Add the HTTP status code to  to ignore this error (see 
http://www.gdal.org/frmt_wms.html).
Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: IReadBlock 
failed at X offset 2764, Y offset 601
Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block offset 2764, 
Y block offset 601

The exact same data gets processed successfully in one run and not successfully 
in another run. We didn't change the code nor the Python or GDAL version. 
Hence, we are kind of puzzled as the error comes and goes seemingly at random.

Does anyone of you have any hints?

Thanks & best regards!
Roman

___

Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)

--

___

Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)

Veronikagasse 38/20, 1170 Wien

+43 664 5547678
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDALWMS - error: TCP connection reset by peer

[Roman Breitfuss-Schiffer]

Dear Jukka!

Thanks for your answer! I think I'm gonna follow that road you suggested.

We have been thinking a lot and I was just wondering if there could be 
some GDAL configuration which could cause that behaviour? For example 
the value for  in the WMS XML file? Or if more 
applications are accessing the same WMS XML at the same time which is 
theoretically possible in our case.


Best regards
Roman

Am 18.11.2021 um 12:12 schrieb Rahkonen Jukka (MML):


Hi,

It seems that the WMS server is closing the door and there is nothing 
else to do on the GDAL side except to try again later. Contact the WMS 
service maintainer and report your troubles. They may be able to 
provide you a more reliable service especially if you are ready to pay 
for it.


-Jukka Rahkonen-

*Lähettäjä:* gdal-dev  *Puolesta 
*Roman Breitfuss-Schiffer

*Lähetetty:* torstai 18. marraskuuta 2021 11.59
*Vastaanottaja:* gdal-dev@lists.osgeo.org
*Aihe:* [gdal-dev] GDALWMS - error: TCP connection reset by peer

Dear all!

We are running a Python application an a linux distribution (RedHat) 
which creates clips in MBTILES format from a WMS. We are using the 
Python-API and are basically doing something like this:


src_ds = gdal.Warp(
    tmp_file,
    ds_in,
    format='GTiff',
    outputBounds=[bbox[0], bbox[2], bbox[1], bbox[3]],
    dstSRS=output_srs,
)
ds_out = gdal.Translate(
    out_filepath,
    src_ds,
    format='MBTILES',
    creationOptions=creation_options,
)

The application throughs an error from time to time since a few days. 
We couldn't figure out a pattern yet and therefore we are also not 
able to reproduce the error. The error is the following:


Mon Nov 15 13:48:13 2021: CPLError: GDALWMS: Unable to download block 
2765, 602.

URL: TCP connection reset by peer
  HTTP status code: 0, error: TCP connection reset by peer.
Add the HTTP status code to  to ignore this error 
(see http://www.gdal.org/frmt_wms.html).
Mon Nov 15 13:48:13 2021: CPLError: /wms_xml/wms_config.xml, band 1: 
IReadBlock failed at X offset 2764, Y offset 601
Mon Nov 15 13:48:13 2021: CPLError: GetBlockRef failed at X block 
offset 2764, Y block offset 601


The exact same data gets processed successfully in one run and not 
successfully in another run. We didn't change the code nor the Python 
or GDAL version. Hence, we are kind of puzzled as the error comes and 
goes seemingly at random.


Does anyone of you have any hints?

Thanks & best regards!
Roman

___
Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)


--
___
Dipl.-Ing. Roman Breitfuss-Schiffer, MSc. (GIS)
Veronikagasse 38/20, 1170 Wien
+43 664 5547678
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev