[gdal-dev] RFC 96 adjustment: Re: Call for review and discussion on RFC96: Deferred in-tree C++ plugin loading

2024-05-29 Thread Even Rouault via gdal-dev 

Hi,

I just wanted to point that a shortcoming of RFC 96 
(https://gdal.org/development/rfc/rfc96_deferred_plugin_loading.html) 
was discovered. https://github.com/OSGeo/gdal/pull/10068 will address 
it. Please refer to it for the details.


This PR is aimed at being backported for 3.9.1. Pedantically this will 
be an ABI change (between in-tree drivers built as plugins and core), 
although I suspect that it shouldn't be noticed for most 
use scenarios. Not clear if that's worth a SONAME bump for 
3.9.1, or if that would be more an annoyance.


Even

Le 02/11/2023 à 12:59, Even Rouault via gdal-dev a écrit :

Hi,

I'm seeking for feedback and review on a new RFC (RFC 96: Deferred 
in-tree C++ plugin loading),

detailed in https://github.com/OSGeo/gdal/pull/8648, whose summary is:

This RFC adds a mechanism to defer the loading of in-tree C++ plugin 
drivers to
the point where their executable code is actually needed, and converts 
a number
of relevant drivers to use that mechanism. The aim is to allow for 
more modular

GDAL builds, while improving the performance of plugin loading.

(This is material only for GDAL 3.9 of course)

Even


--
http://www.spatialys.com
My software is free, but my time generally not.

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Even Rouault via gdal-dev 

Hi,

The WMS cache (which is used underneath by the WMTS driver) writes each 
tiles in a separate file. So if you use multi-threaded, as long as you 
read in parallel different GDALDataset* object on the WMTS filename, and 
read disjoint areas, that should be fine. There might be a slight risk 
of things going bad if you happened to read the same tile from different 
GDALDataset* object at the same time, as it could potentially read a 
partially written cached tile, or concurrently write the tile in the 
cache and corrupt it. The logic is in frmts/wms/gdalwmscache.cpp


Even

Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
Because this is not clear in documentation, especially in WMTS driver 
docs, my question is strictly about this issue to programmers who have 
already explored this topic.


Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka 
 napisał(a):


Hi,

No, I am not sure because I do not program myself. Reading this
document https://gdal.org/user/multithreading.html makes me feel
that it is possible to do some things in parallel, but the
programmer must know how to do it right.

-Jukka Rahkonen-

*Lähettäjä:*Javier Jimenez Shaw 
*Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
*Vastaanottaja:* Rahkonen Jukka 
*Kopio:* Michał Kowalczuk ;
gdal-dev@lists.osgeo.org
*Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation

On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev
mailto:gdal-dev@lists.osgeo.org>> wrote:

Hi,

When you have a RasterBand  from the WMTS data source, it is
abstracted and you can read the raster data just like from any
other data source and raster band
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data

“There
are a few ways to read raster data, but the most common is via
the GDALRasterBand::RasterIO() method. This method will
automatically take care of data type conversion, up/down
sampling and windowing.” GDAL knows which tiles to read.

I do not know if the WMTS driver can do parallel tile
downloads. If not, it is possible to run many RasterIO() at
the same time, each reading data from a different window like
in this rasterio document
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html
.

Jukka, are you sure you can run several RasterIO in parallel (on
the same dataset)? in GDAL GeoTIFF you cannot: the cache may be
corrupted. In that case I open several datasets over the same
file. I do not know about WMTS.

-Jukka Rahkonen-

*Lähettäjä:*gdal-dev mailto:gdal-dev-boun...@lists.osgeo.org>> *Puolesta *Michal
Kowalczuk via gdal-dev
*Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
*Vastaanottaja:* gdal-dev@lists.osgeo.org

*Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows

This is my first post on this mailing list, so I'm asking for
understanding.

As all we know the purpose of using WMTS over WMS, I'd like to
implement parallel downloading tiles from service using C API.

In my opinion GDAL documentation
(https://gdal.org/drivers/raster/wmts.html
) says nothing on
this topic.

I can get capabilities from WMTS, I can open the selected
subdataset but how to get tiles for given extent? I could not
find any information how to do it, even in the GDAL tests on
github.

I am kindly asking for tips. It also can be in python. How
using pure GDAL API fetch tiles to dynamically complete the
displayed map. This is my goal.

Thank you!

Michal

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Michał Kowalczuk via gdal-dev 
Because this is not clear in documentation, especially in WMTS driver docs,
my question is strictly about this issue to programmers who have already
explored this topic.

Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
jukka.rahko...@maanmittauslaitos.fi> napisał(a):

> Hi,
>
>
>
> No, I am not sure because I do not program myself. Reading this document
> https://gdal.org/user/multithreading.html makes me feel that it is
> possible to do some things in parallel, but the programmer must know how to
> do it right.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Javier Jimenez Shaw 
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
> *Vastaanottaja:* Rahkonen Jukka 
> *Kopio:* Michał Kowalczuk ;
> gdal-dev@lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
>
>
>
>
> On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
> gdal-dev@lists.osgeo.org> wrote:
>
> Hi,
>
>
>
> When you have a RasterBand  from the WMTS data source, it is abstracted
> and you can read the raster data just like from any other data source and
> raster band
> https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There
> are a few ways to read raster data, but the most common is via the
> GDALRasterBand::RasterIO() method. This method will automatically take care
> of data type conversion, up/down sampling and windowing.” GDAL knows which
> tiles to read.
>
>
>
> I do not know if the WMTS driver can do parallel tile downloads. If not,
> it is possible to run many RasterIO() at the same time, each reading data
> from a different window like in this rasterio document
> https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.
>
>
>
> Jukka, are you sure you can run several RasterIO in parallel (on the same
> dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that
> case I open several datasets over the same file. I do not know about WMTS.
>
>
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* gdal-dev  *Puolesta *Michal
> Kowalczuk via gdal-dev
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
> *Vastaanottaja:* gdal-dev@lists.osgeo.org
> *Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi GDAL fellows
>
> This is my first post on this mailing list, so I'm asking for
> understanding.
>
>
>
> As all we know the purpose of using WMTS over WMS, I'd like to implement
> parallel downloading tiles from service using C API.
>
> In my opinion GDAL documentation (
> https://gdal.org/drivers/raster/wmts.html) says nothing on this topic.
>
> I can get capabilities from WMTS, I can open the selected subdataset but
> how to get tiles for given extent? I could not find any information how to
> do it, even in the GDAL tests on github.
>
>
>
> I am kindly asking for tips. It also can be in python. How using pure GDAL
> API fetch tiles to dynamically complete the displayed map. This is my goal.
>
>
>
> Thank you!
>
> Michal
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Rahkonen Jukka via gdal-dev 
Hi,

No, I am not sure because I do not program myself. Reading this document 
https://gdal.org/user/multithreading.html makes me feel that it is possible to 
do some things in parallel, but the programmer must know how to do it right.

-Jukka Rahkonen-

Lähettäjä: Javier Jimenez Shaw 
Lähetetty: keskiviikko 29. toukokuuta 2024 10.41
Vastaanottaja: Rahkonen Jukka 
Kopio: Michał Kowalczuk ; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation



On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data "There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing." GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.

Jukka, are you sure you can run several RasterIO in parallel (on the same 
dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that case 
I open several datasets over the same file. I do not know about WMTS.


-Jukka Rahkonen-

Lähettäjä: gdal-dev 
mailto:gdal-dev-boun...@lists.osgeo.org>> 
Puolesta Michal Kowalczuk via gdal-dev
Lähetetty: keskiviikko 29. toukokuuta 2024 9.08
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement 
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html) 
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but how to 
get tiles for given extent? I could not find any information how to do it, even 
in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL API 
fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Javier Jimenez Shaw via gdal-dev 
On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
gdal-dev@lists.osgeo.org> wrote:

> Hi,
>
>
>
> When you have a RasterBand  from the WMTS data source, it is abstracted
> and you can read the raster data just like from any other data source and
> raster band
> https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There
> are a few ways to read raster data, but the most common is via the
> GDALRasterBand::RasterIO() method. This method will automatically take care
> of data type conversion, up/down sampling and windowing.” GDAL knows which
> tiles to read.
>
>
>
> I do not know if the WMTS driver can do parallel tile downloads. If not,
> it is possible to run many RasterIO() at the same time, each reading data
> from a different window like in this rasterio document
> https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.
>

Jukka, are you sure you can run several RasterIO in parallel (on the same
dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that
case I open several datasets over the same file. I do not know about WMTS.


>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* gdal-dev  *Puolesta *Michal
> Kowalczuk via gdal-dev
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
> *Vastaanottaja:* gdal-dev@lists.osgeo.org
> *Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi GDAL fellows
>
> This is my first post on this mailing list, so I'm asking for
> understanding.
>
>
>
> As all we know the purpose of using WMTS over WMS, I'd like to implement
> parallel downloading tiles from service using C API.
>
> In my opinion GDAL documentation (
> https://gdal.org/drivers/raster/wmts.html) says nothing on this topic.
>
> I can get capabilities from WMTS, I can open the selected subdataset but
> how to get tiles for given extent? I could not find any information how to
> do it, even in the GDAL tests on github.
>
>
>
> I am kindly asking for tips. It also can be in python. How using pure GDAL
> API fetch tiles to dynamically complete the displayed map. This is my goal.
>
>
>
> Thank you!
>
> Michal
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Rahkonen Jukka via gdal-dev 
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data "There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing." GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 
https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.

-Jukka Rahkonen-

Lähettäjä: gdal-dev  Puolesta Michal 
Kowalczuk via gdal-dev
Lähetetty: keskiviikko 29. toukokuuta 2024 9.08
Vastaanottaja: gdal-dev@lists.osgeo.org
Aihe: [gdal-dev] GDAL WMTS get tiles no documentation

Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement 
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html) 
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but how to 
get tiles for given extent? I could not find any information how to do it, even 
in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL API 
fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL WMTS get tiles no documentation

2024-05-29 Thread Michał Kowalczuk via gdal-dev 
Hi GDAL fellows
This is my first post on this mailing list, so I'm asking for understanding.

As all we know the purpose of using WMTS over WMS, I'd like to implement
parallel downloading tiles from service using C API.
In my opinion GDAL documentation (https://gdal.org/drivers/raster/wmts.html)
says nothing on this topic.
I can get capabilities from WMTS, I can open the selected subdataset but
how to get tiles for given extent? I could not find any information how to
do it, even in the GDAL tests on github.

I am kindly asking for tips. It also can be in python. How using pure GDAL
API fetch tiles to dynamically complete the displayed map. This is my goal.

Thank you!
Michal
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev