Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Michał Kowalczuk via gdal-dev]

Thank you again.
Now I know which direction should I go.This is my goal to have a control
over this process using the low-level RasterIO.
I browse GDAL github repository a lot and it's very helpful, especially the
tests.
This time I was confused with WMTS and I needed a support at the begining.

Have a great weekend!
Michal


pt., 31 maj 2024 o 16:14 Rahkonen Jukka 
napisał(a):

> Hi,
>
>
>
> You do not need to use GDALTranslate even if may be convenient. In the
> RasterIO() function GDALRasterBand C++ API — GDAL documentation
>  the region is selected by
> using XOff, YOff, XSize and YSize which map directly with -srcwin. But I
> was thinking that very soon you’ll ask about how to convert a georeferenced
> coordinates into pixel space, and gdal_translate supports that. So maybe
> you want to read the source code
> https://github.com/OSGeo/gdal/blob/master/apps/gdal_translate_lib.cpp.
> And meanwhile you can read and convert data from WMTS with gdal_translate,
> enjoy in seeing that it works, and be confident that you will be able to do
> the same and more with your own code. If programmers think that way, I do
> not know.
>
>
>
> -Jukka-
>
>
>
> *Lähettäjä:* Michał Kowalczuk 
> *Lähetetty:* perjantai 31. toukokuuta 2024 16.32
> *Vastaanottaja:* Rahkonen Jukka 
> *Kopio:* Even Rouault ;
> gdal-dev@lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Thank you Rukka,
>
> Since *-srcwin* and *-projwin* are options only for *gdal_translate*
> utility, your suggestion leads me to use *GDALTranslate()* function with
> these parameters.
>
>
>
> This means that I cannot use directly created WMTS dataset to retrieve
> data (using GDALDatasetRasterIO), but create a new (in-memory or
> file-based) for each map extent change.
>
>
>
> And this is the standard way of doing it?
>
> Do I understand it correctly?
>
>
>
> Michal
>
>
>
> pt., 31 maj 2024 o 15:13 Rahkonen Jukka <
> jukka.rahko...@maanmittauslaitos.fi> napisał(a):
>
> Hi,
>
>
>
> Before starting to program you can familiar yourself with the topic by
> using gdal_translate gdal_translate — GDAL documentation
> .
>
> Play with the -srcwin and -projwin / -projwin_srs options and gather some
> trust that is is possible to select and download any part of the large WMTS
> layer. Add “-- debug on” and you will see that GDAL downloads tiles
> separately just like QGIS. Actually, with WMTS it is the only way to
> download data. Maybe you have read the standard already
> https://portal.ogc.org/files/?artifact_id=35326. It is not necessary
> because GDAL knows what to do, but a review every now and then does not
> harm. Especially the TileMatrix and TileMatrixSet things are not
> self-evident, but they provide the answer to the question about how a WMTS
> client knows which row, col, and zoom to download.
>
>
>
> Do not touch the DataWindow. It defines the extent of the whole WMTS layer
> and it is used for example if gdal_translate is run without spatial
> selection (-srcwin or -projwin). Touch in your future code things which are
> equivalent to -srcwin or -projwin of gdal_translate.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Michał Kowalczuk 
> *Lähetetty:* perjantai 31. toukokuuta 2024 15.45
> *Vastaanottaja:* Even Rouault 
> *Kopio:* Rahkonen Jukka ;
> gdal-dev@lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi, thank you for your replies!
> As I'm still a newbie especially with WMTS. I need to take a step back to
> the basics. Let's forget about multithreading for now.
>
>- I'm in the stage that I can open a dataset using WMTS service path
>from data provider
>- I can list subdatasets and open the selected layer as a new dataset
>using path from *SUBDATASET_id_NAME*.
>- This dataset reports large height and width values, which is obvious.
>- The first question is how to read any fragment from this dataset
>using *GDALDatasetRasterIO*.
>Is there a similar way to *SpatialFIlter *from vector layers?
>I see in the docs there is a ** tag for a local service
>descriptions which defines extents of the data. Does it mean that for each
>map extent change I need to create a new local XML file and open a new
>dataset to read demanded bbox?
>Is this the only way for doing it?
>- From other site I noticed that QGIS downloads tiles separately? But
>how it knows which row/col and zoom should be used in GET request? Does
>it use GDAL for WMTS reading at all?
>
> I will be grateful for any tip and trucks for a newbie ;-)
>
>
>
> Regards,
>
> Michal
>
>
>
> śr., 29 maj 2024 o 12:41 Even Rouault 
> napisał(a):
>
> Hi,
>
> The WMS cache (which is used underneath by the WMTS driver) writes each
> tiles in a separate file. So if you use multi-threaded, as long as you read
> in parallel different GDALDataset* object on the WMTS filename, and read
> disjoint 

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

Hi,

You do not need to use GDALTranslate even if may be convenient. In the 
RasterIO() function GDALRasterBand C++ API — GDAL 
documentation the region is 
selected by using XOff, YOff, XSize and YSize which map directly with -srcwin. 
But I was thinking that very soon you’ll ask about how to convert a 
georeferenced coordinates into pixel space, and gdal_translate supports that. 
So maybe you want to read the source code 
https://github.com/OSGeo/gdal/blob/master/apps/gdal_translate_lib.cpp. And 
meanwhile you can read and convert data from WMTS with gdal_translate, enjoy in 
seeing that it works, and be confident that you will be able to do the same and 
more with your own code. If programmers think that way, I do not know.

-Jukka-

Lähettäjä: Michał Kowalczuk 
Lähetetty: perjantai 31. toukokuuta 2024 16.32
Vastaanottaja: Rahkonen Jukka 
Kopio: Even Rouault ; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

Thank you Rukka,
Since -srcwin and -projwin are options only for gdal_translate utility, your 
suggestion leads me to use GDALTranslate() function with these parameters.

This means that I cannot use directly created WMTS dataset to retrieve data 
(using GDALDatasetRasterIO), but create a new (in-memory or file-based) for 
each map extent change.

And this is the standard way of doing it?
Do I understand it correctly?

Michal

pt., 31 maj 2024 o 15:13 Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 napisał(a):
Hi,

Before starting to program you can familiar yourself with the topic by using 
gdal_translate gdal_translate — GDAL 
documentation.
Play with the -srcwin and -projwin / -projwin_srs options and gather some trust 
that is is possible to select and download any part of the large WMTS layer. 
Add “-- debug on” and you will see that GDAL downloads tiles separately just 
like QGIS. Actually, with WMTS it is the only way to download data. Maybe you 
have read the standard already https://portal.ogc.org/files/?artifact_id=35326. 
It is not necessary because GDAL knows what to do, but a review every now and 
then does not harm. Especially the TileMatrix and TileMatrixSet things are not 
self-evident, but they provide the answer to the question about how a WMTS 
client knows which row, col, and zoom to download.

Do not touch the DataWindow. It defines the extent of the whole WMTS layer and 
it is used for example if gdal_translate is run without spatial selection 
(-srcwin or -projwin). Touch in your future code things which are equivalent to 
-srcwin or -projwin of gdal_translate.

-Jukka Rahkonen-

Lähettäjä: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>
Lähetetty: perjantai 31. toukokuuta 2024 15.45
Vastaanottaja: Even Rouault 
mailto:even.roua...@spatialys.com>>
Kopio: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>;
 gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

Hi, thank you for your replies!
As I'm still a newbie especially with WMTS. I need to take a step back to the 
basics. Let's forget about multithreading for now.

  *   I'm in the stage that I can open a dataset using WMTS service path from 
data provider
  *   I can list subdatasets and open the selected layer as a new dataset using 
path from SUBDATASET_id_NAME.
  *   This dataset reports large height and width values, which is obvious.
  *   The first question is how to read any fragment from this dataset using 
GDALDatasetRasterIO.
Is there a similar way to SpatialFIlter from vector layers?
I see in the docs there is a  tag for a local service descriptions 
which defines extents of the data. Does it mean that for each map extent change 
I need to create a new local XML file and open a new dataset to read demanded 
bbox?
Is this the only way for doing it?
  *   From other site I noticed that QGIS downloads tiles separately? But how 
it knows which row/col and zoom should be used in GET request? Does it use GDAL 
for WMTS reading at all?
[cid:image001.png@01DAB379.1058A2F0]
I will be grateful for any tip and trucks for a newbie ;-)

Regards,
Michal

śr., 29 maj 2024 o 12:41 Even Rouault 
mailto:even.roua...@spatialys.com>> napisał(a):

Hi,

The WMS cache (which is used underneath by the WMTS driver) writes each tiles 
in a separate file. So if you use multi-threaded, as long as you read in 
parallel different GDALDataset* object on the WMTS filename, and read disjoint 
areas, that should be fine. There might be a slight risk of things going bad if 
you happened to read the same tile from different GDALDataset* object at the 
same time, as it could potentially read a partially written cached tile, or 
concurrently write the tile in the cache and corrupt it. The logic is in 
frmts/wms/gdalwmscache.cpp

Even
Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
Because this is not clear in 

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

(resending, need to cut old discussion due to too many bytes in the body)

Hi,

gdalinfo 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
Driver: WMTS/OGC Web Map Tile Service
Files: none associated
Size is 305719, 257379

The unit of size is pixels. You used -srcwin 1820157 6144777 512 512, with 
offsets far outsize the layer. This returns me something greenish.

gdal_translate 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
 -srcwin 10 10 512 512 tile.jpg

-Jukka Rahkonen-


Lähettäjä: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>
Lähetetty: perjantai 31. toukokuuta 2024 16.58
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: Even Rouault 
mailto:even.roua...@spatialys.com>>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

As you proposed I tested gdal_translate utility but for the following sample 
command I get an empty image (tested with jpg and png)

gdal_translate 
"WMTS:http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
 -srcwin 1820157 6144777 512 512 tile.jpg
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Michał Kowalczuk via gdal-dev]

As you proposed I tested *gdal_translate *utility but for the following
sample command I get an empty image (tested with jpg and png)

gdal_translate "WMTS:
http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer;
-srcwin 1820157 6144777 512 512 tile.jpg

[image: image.png]

pt., 31 maj 2024 o 15:31 Michał Kowalczuk 
napisał(a):

> Thank you Rukka,
> Since *-srcwin* and *-projwin* are options only for *gdal_translate*
> utility, your suggestion leads me to use *GDALTranslate()* function with
> these parameters.
>
> This means that I cannot use directly created WMTS dataset to retrieve
> data (using GDALDatasetRasterIO), but create a new (in-memory or
> file-based) for each map extent change.
>
> And this is the standard way of doing it?
> Do I understand it correctly?
>
> Michal
>
> pt., 31 maj 2024 o 15:13 Rahkonen Jukka <
> jukka.rahko...@maanmittauslaitos.fi> napisał(a):
>
>> Hi,
>>
>>
>>
>> Before starting to program you can familiar yourself with the topic by
>> using gdal_translate gdal_translate — GDAL documentation
>> .
>>
>> Play with the -srcwin and -projwin / -projwin_srs options and gather some
>> trust that is is possible to select and download any part of the large WMTS
>> layer. Add “-- debug on” and you will see that GDAL downloads tiles
>> separately just like QGIS. Actually, with WMTS it is the only way to
>> download data. Maybe you have read the standard already
>> https://portal.ogc.org/files/?artifact_id=35326. It is not necessary
>> because GDAL knows what to do, but a review every now and then does not
>> harm. Especially the TileMatrix and TileMatrixSet things are not
>> self-evident, but they provide the answer to the question about how a WMTS
>> client knows which row, col, and zoom to download.
>>
>>
>>
>> Do not touch the DataWindow. It defines the extent of the whole WMTS
>> layer and it is used for example if gdal_translate is run without spatial
>> selection (-srcwin or -projwin). Touch in your future code things which are
>> equivalent to -srcwin or -projwin of gdal_translate.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* Michał Kowalczuk 
>> *Lähetetty:* perjantai 31. toukokuuta 2024 15.45
>> *Vastaanottaja:* Even Rouault 
>> *Kopio:* Rahkonen Jukka ;
>> gdal-dev@lists.osgeo.org
>> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>> Hi, thank you for your replies!
>> As I'm still a newbie especially with WMTS. I need to take a step back to
>> the basics. Let's forget about multithreading for now.
>>
>>- I'm in the stage that I can open a dataset using WMTS service path
>>from data provider
>>- I can list subdatasets and open the selected layer as a new dataset
>>using path from *SUBDATASET_id_NAME*.
>>- This dataset reports large height and width values, which is
>>obvious.
>>- The first question is how to read any fragment from this dataset
>>using *GDALDatasetRasterIO*.
>>Is there a similar way to *SpatialFIlter *from vector layers?
>>I see in the docs there is a ** tag for a local service
>>descriptions which defines extents of the data. Does it mean that for each
>>map extent change I need to create a new local XML file and open a new
>>dataset to read demanded bbox?
>>Is this the only way for doing it?
>>- From other site I noticed that QGIS downloads tiles separately? But
>>how it knows which row/col and zoom should be used in GET request? Does
>>it use GDAL for WMTS reading at all?
>>
>> I will be grateful for any tip and trucks for a newbie ;-)
>>
>>
>>
>> Regards,
>>
>> Michal
>>
>>
>>
>> śr., 29 maj 2024 o 12:41 Even Rouault 
>> napisał(a):
>>
>> Hi,
>>
>> The WMS cache (which is used underneath by the WMTS driver) writes each
>> tiles in a separate file. So if you use multi-threaded, as long as you read
>> in parallel different GDALDataset* object on the WMTS filename, and read
>> disjoint areas, that should be fine. There might be a slight risk of things
>> going bad if you happened to read the same tile from different GDALDataset*
>> object at the same time, as it could potentially read a partially written
>> cached tile, or concurrently write the tile in the cache and corrupt it.
>> The logic is in frmts/wms/gdalwmscache.cpp
>>
>> Even
>>
>> Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
>>
>> Because this is not clear in documentation, especially in WMTS driver
>> docs, my question is strictly about this issue to programmers who have
>> already explored this topic.
>>
>>
>>
>> Regards
>>
>> Michal
>>
>>
>>
>> W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
>> jukka.rahko...@maanmittauslaitos.fi> napisał(a):
>>
>> Hi,
>>
>>
>>
>> No, I am not sure because I do not program myself. Reading this document
>> https://gdal.org/user/multithreading.html makes me feel that it is
>> possible to do some things in parallel, but the programmer must know how to
>> do it right.
>>
>>
>>
>> -Jukka 

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Michał Kowalczuk via gdal-dev]

Thank you Rukka,
Since *-srcwin* and *-projwin* are options only for *gdal_translate*
utility, your suggestion leads me to use *GDALTranslate()* function with
these parameters.

This means that I cannot use directly created WMTS dataset to retrieve data
(using GDALDatasetRasterIO), but create a new (in-memory or file-based) for
each map extent change.

And this is the standard way of doing it?
Do I understand it correctly?

Michal

pt., 31 maj 2024 o 15:13 Rahkonen Jukka 
napisał(a):

> Hi,
>
>
>
> Before starting to program you can familiar yourself with the topic by
> using gdal_translate gdal_translate — GDAL documentation
> .
>
> Play with the -srcwin and -projwin / -projwin_srs options and gather some
> trust that is is possible to select and download any part of the large WMTS
> layer. Add “-- debug on” and you will see that GDAL downloads tiles
> separately just like QGIS. Actually, with WMTS it is the only way to
> download data. Maybe you have read the standard already
> https://portal.ogc.org/files/?artifact_id=35326. It is not necessary
> because GDAL knows what to do, but a review every now and then does not
> harm. Especially the TileMatrix and TileMatrixSet things are not
> self-evident, but they provide the answer to the question about how a WMTS
> client knows which row, col, and zoom to download.
>
>
>
> Do not touch the DataWindow. It defines the extent of the whole WMTS layer
> and it is used for example if gdal_translate is run without spatial
> selection (-srcwin or -projwin). Touch in your future code things which are
> equivalent to -srcwin or -projwin of gdal_translate.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Michał Kowalczuk 
> *Lähetetty:* perjantai 31. toukokuuta 2024 15.45
> *Vastaanottaja:* Even Rouault 
> *Kopio:* Rahkonen Jukka ;
> gdal-dev@lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi, thank you for your replies!
> As I'm still a newbie especially with WMTS. I need to take a step back to
> the basics. Let's forget about multithreading for now.
>
>- I'm in the stage that I can open a dataset using WMTS service path
>from data provider
>- I can list subdatasets and open the selected layer as a new dataset
>using path from *SUBDATASET_id_NAME*.
>- This dataset reports large height and width values, which is obvious.
>- The first question is how to read any fragment from this dataset
>using *GDALDatasetRasterIO*.
>Is there a similar way to *SpatialFIlter *from vector layers?
>I see in the docs there is a ** tag for a local service
>descriptions which defines extents of the data. Does it mean that for each
>map extent change I need to create a new local XML file and open a new
>dataset to read demanded bbox?
>Is this the only way for doing it?
>- From other site I noticed that QGIS downloads tiles separately? But
>how it knows which row/col and zoom should be used in GET request? Does
>it use GDAL for WMTS reading at all?
>
> I will be grateful for any tip and trucks for a newbie ;-)
>
>
>
> Regards,
>
> Michal
>
>
>
> śr., 29 maj 2024 o 12:41 Even Rouault 
> napisał(a):
>
> Hi,
>
> The WMS cache (which is used underneath by the WMTS driver) writes each
> tiles in a separate file. So if you use multi-threaded, as long as you read
> in parallel different GDALDataset* object on the WMTS filename, and read
> disjoint areas, that should be fine. There might be a slight risk of things
> going bad if you happened to read the same tile from different GDALDataset*
> object at the same time, as it could potentially read a partially written
> cached tile, or concurrently write the tile in the cache and corrupt it.
> The logic is in frmts/wms/gdalwmscache.cpp
>
> Even
>
> Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
>
> Because this is not clear in documentation, especially in WMTS driver
> docs, my question is strictly about this issue to programmers who have
> already explored this topic.
>
>
>
> Regards
>
> Michal
>
>
>
> W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
> jukka.rahko...@maanmittauslaitos.fi> napisał(a):
>
> Hi,
>
>
>
> No, I am not sure because I do not program myself. Reading this document
> https://gdal.org/user/multithreading.html makes me feel that it is
> possible to do some things in parallel, but the programmer must know how to
> do it right.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Javier Jimenez Shaw 
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
> *Vastaanottaja:* Rahkonen Jukka 
> *Kopio:* Michał Kowalczuk ;
> gdal-dev@lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
>
>
>
>
> On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
> gdal-dev@lists.osgeo.org> wrote:
>
> Hi,
>
>
>
> When you have a RasterBand  from the WMTS data source, it is abstracted
> and you can read the raster data just like from any other data source and
> raster 

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Rahkonen Jukka via gdal-dev]

Hi,

Before starting to program you can familiar yourself with the topic by using 
gdal_translate gdal_translate — GDAL 
documentation.
Play with the -srcwin and -projwin / -projwin_srs options and gather some trust 
that is is possible to select and download any part of the large WMTS layer. 
Add “-- debug on” and you will see that GDAL downloads tiles separately just 
like QGIS. Actually, with WMTS it is the only way to download data. Maybe you 
have read the standard already https://portal.ogc.org/files/?artifact_id=35326. 
It is not necessary because GDAL knows what to do, but a review every now and 
then does not harm. Especially the TileMatrix and TileMatrixSet things are not 
self-evident, but they provide the answer to the question about how a WMTS 
client knows which row, col, and zoom to download.

Do not touch the DataWindow. It defines the extent of the whole WMTS layer and 
it is used for example if gdal_translate is run without spatial selection 
(-srcwin or -projwin). Touch in your future code things which are equivalent to 
-srcwin or -projwin of gdal_translate.

-Jukka Rahkonen-

Lähettäjä: Michał Kowalczuk 
Lähetetty: perjantai 31. toukokuuta 2024 15.45
Vastaanottaja: Even Rouault 
Kopio: Rahkonen Jukka ; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation

Hi, thank you for your replies!
As I'm still a newbie especially with WMTS. I need to take a step back to the 
basics. Let's forget about multithreading for now.

  *   I'm in the stage that I can open a dataset using WMTS service path from 
data provider
  *   I can list subdatasets and open the selected layer as a new dataset using 
path from SUBDATASET_id_NAME.
  *   This dataset reports large height and width values, which is obvious.
  *   The first question is how to read any fragment from this dataset using 
GDALDatasetRasterIO.
Is there a similar way to SpatialFIlter from vector layers?
I see in the docs there is a  tag for a local service descriptions 
which defines extents of the data. Does it mean that for each map extent change 
I need to create a new local XML file and open a new dataset to read demanded 
bbox?
Is this the only way for doing it?
  *   From other site I noticed that QGIS downloads tiles separately? But how 
it knows which row/col and zoom should be used in GET request? Does it use GDAL 
for WMTS reading at all?
[cid:image001.png@01DAB373.3E20F850]
I will be grateful for any tip and trucks for a newbie ;-)

Regards,
Michal

śr., 29 maj 2024 o 12:41 Even Rouault 
mailto:even.roua...@spatialys.com>> napisał(a):

Hi,

The WMS cache (which is used underneath by the WMTS driver) writes each tiles 
in a separate file. So if you use multi-threaded, as long as you read in 
parallel different GDALDataset* object on the WMTS filename, and read disjoint 
areas, that should be fine. There might be a slight risk of things going bad if 
you happened to read the same tile from different GDALDataset* object at the 
same time, as it could potentially read a partially written cached tile, or 
concurrently write the tile in the cache and corrupt it. The logic is in 
frmts/wms/gdalwmscache.cpp

Even
Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
Because this is not clear in documentation, especially in WMTS driver docs, my 
question is strictly about this issue to programmers who have already explored 
this topic.

Regards
Michal

W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
 napisał(a):
Hi,

No, I am not sure because I do not program myself. Reading this document 
https://gdal.org/user/multithreading.html makes me feel that it is possible to 
do some things in parallel, but the programmer must know how to do it right.

-Jukka Rahkonen-

Lähettäjä: Javier Jimenez Shaw 
mailto:j...@jimenezshaw.com>>
Lähetetty: keskiviikko 29. toukokuuta 2024 10.41
Vastaanottaja: Rahkonen Jukka 
mailto:jukka.rahko...@maanmittauslaitos.fi>>
Kopio: Michał Kowalczuk 
mailto:michkowalc...@gmail.com>>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] GDAL WMTS get tiles no documentation



On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev 
mailto:gdal-dev@lists.osgeo.org>> wrote:
Hi,

When you have a RasterBand  from the WMTS data source, it is abstracted and you 
can read the raster data just like from any other data source and raster band 
https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There are a 
few ways to read raster data, but the most common is via the 
GDALRasterBand::RasterIO() method. This method will automatically take care of 
data type conversion, up/down sampling and windowing.” GDAL knows which tiles 
to read.

I do not know if the WMTS driver can do parallel tile downloads. If not, it is 
possible to run many RasterIO() at the same time, each reading data from a 
different window like in this rasterio document 

Re: [gdal-dev] GDAL WMTS get tiles no documentation

[Michał Kowalczuk via gdal-dev]

Hi, thank you for your replies!
As I'm still a newbie especially with WMTS. I need to take a step back to
the basics. Let's forget about multithreading for now.

   - I'm in the stage that I can open a dataset using WMTS service path
   from data provider
   - I can list subdatasets and open the selected layer as a new dataset
   using path from *SUBDATASET_id_NAME*.
   - This dataset reports large height and width values, which is obvious.
   - The first question is how to read any fragment from this dataset using
   *GDALDatasetRasterIO*.
   Is there a similar way to *SpatialFIlter *from vector layers?
   I see in the docs there is a ** tag for a local service
   descriptions which defines extents of the data. Does it mean that for each
   map extent change I need to create a new local XML file and open a new
   dataset to read demanded bbox?
   Is this the only way for doing it?
   - From other site I noticed that QGIS downloads tiles separately? But
   how it knows which row/col and zoom should be used in GET request? Does it
   use GDAL for WMTS reading at all?
   [image: image.png]

I will be grateful for any tip and trucks for a newbie ;-)

Regards,
Michal

śr., 29 maj 2024 o 12:41 Even Rouault 
napisał(a):

> Hi,
>
> The WMS cache (which is used underneath by the WMTS driver) writes each
> tiles in a separate file. So if you use multi-threaded, as long as you read
> in parallel different GDALDataset* object on the WMTS filename, and read
> disjoint areas, that should be fine. There might be a slight risk of things
> going bad if you happened to read the same tile from different GDALDataset*
> object at the same time, as it could potentially read a partially written
> cached tile, or concurrently write the tile in the cache and corrupt it.
> The logic is in frmts/wms/gdalwmscache.cpp
>
> Even
> Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
>
> Because this is not clear in documentation, especially in WMTS driver
> docs, my question is strictly about this issue to programmers who have
> already explored this topic.
>
> Regards
> Michal
>
> W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
> jukka.rahko...@maanmittauslaitos.fi> napisał(a):
>
>> Hi,
>>
>>
>>
>> No, I am not sure because I do not program myself. Reading this document
>> https://gdal.org/user/multithreading.html makes me feel that it is
>> possible to do some things in parallel, but the programmer must know how to
>> do it right.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* Javier Jimenez Shaw 
>> *Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
>> *Vastaanottaja:* Rahkonen Jukka 
>> *Kopio:* Michał Kowalczuk ;
>> gdal-dev@lists.osgeo.org
>> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>>
>>
>>
>>
>> On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
>> gdal-dev@lists.osgeo.org> wrote:
>>
>> Hi,
>>
>>
>>
>> When you have a RasterBand  from the WMTS data source, it is abstracted
>> and you can read the raster data just like from any other data source and
>> raster band
>> https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data
>> “There are a few ways to read raster data, but the most common is via the
>> GDALRasterBand::RasterIO() method. This method will automatically take care
>> of data type conversion, up/down sampling and windowing.” GDAL knows which
>> tiles to read.
>>
>>
>>
>> I do not know if the WMTS driver can do parallel tile downloads. If not,
>> it is possible to run many RasterIO() at the same time, each reading data
>> from a different window like in this rasterio document
>> https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.
>>
>>
>>
>> Jukka, are you sure you can run several RasterIO in parallel (on the same
>> dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that
>> case I open several datasets over the same file. I do not know about WMTS.
>>
>>
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* gdal-dev  *Puolesta *Michal
>> Kowalczuk via gdal-dev
>> *Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
>> *Vastaanottaja:* gdal-dev@lists.osgeo.org
>> *Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>> Hi GDAL fellows
>>
>> This is my first post on this mailing list, so I'm asking for
>> understanding.
>>
>>
>>
>> As all we know the purpose of using WMTS over WMS, I'd like to implement
>> parallel downloading tiles from service using C API.
>>
>> In my opinion GDAL documentation (
>> https://gdal.org/drivers/raster/wmts.html) says nothing on this topic.
>>
>> I can get capabilities from WMTS, I can open the selected subdataset but
>> how to get tiles for given extent? I could not find any information how to
>> do it, even in the GDAL tests on github.
>>
>>
>>
>> I am kindly asking for tips. It also can be in python. How using pure
>> GDAL API fetch tiles to dynamically complete the displayed map. This is my
>> goal.
>>
>>
>>
>> Thank you!
>>
>> Michal
>>
>> 

Re: [gdal-dev] OGR SQL CAST geometry example

[Rahkonen Jukka via gdal-dev]

Hi,

As far I know the the Spatialite functions 
https://www.gaia-gis.it/gaia-sins/spatialite-sql-4.2.0.html by design work row 
by row and one input row returns one output row. Therefore DissolvePoints 
creates one row with a multipoint geometry, not several rows like ST_Dump does. 
You can discuss about that in the spatialite-users forum 
https://groups.google.com/g/spatialite-users.
It is possible to extract the points from linestrings with SpatiaLite SQL by 
using NumPoints and then PointN in a loop, but such SQL is not possible to 
execute with ogrinfo/ogr2ogr. It is possible to do amazing things with ogr2ogr 
and SQL but it does make it the right tool for demanding ETL processes. 
Typically people turn to use Python at that point.

If you have a real need to extract points from a linestring by using only GDAL 
command line utilities, it should be possible to do in two steps. Create an 
interim file by using the Spatialite function DissolvePoints, and then run 
ogr2ogr with the -explodecollections option.

-Jukka Rahkonen-

-Alkuperäinen viesti-
Lähettäjä: gdal-dev  Puolesta Dan Jacobson 
via gdal-dev
Lähetetty: perjantai 31. toukokuuta 2024 8.36
Vastaanottaja: Daniel Baston 
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] OGR SQL CAST geometry example

Alas, even for MULTIPOINT, as there are no facilities for extracting from 
aggregates, not even ST_Dump, in the end all one can do is $ ogrinfo 
0.lines.kml -al | perl -nwle \
   'if(/LINESTRING.*\((.*)\)/){print for split /,/, $1}'
-100.79005679 36.38022 1000
-100.79008013 36.38531 1000
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev