Re: [gdal-dev] Ogr2ogr, SQLite and overwriting existing database

[Rahkonen Jukka (MML)]

Hi Even,

Thanks for the clarification. By the way, the behavior with QuietDelete may be 
different with  SQLite driver but the messages are the same:

GDAL: GDALOpen(foo.sqlite, this=01DD6450) succeeds as SQLite.
RASTERLITE: SQLite DB Open
OGR: Unloading VirtualOGR module
GDAL: GDALClose(foo.sqlite, this=01DD6450)
SQLITE: OGR style SQLite DB found !
OGR: Unloading VirtualOGR module
GDAL: QuietDelete(foo.sqlite) invoking Delete()
GDAL: GDALDriver::Create(SQLite,foo.sqlite,0,0,0,Unknown,)
ERROR 1: It seems a file system object called 'foo.sqlite' already exists.
ERROR 1: sqlite driver failed to create foo.sqlite

-Jukka-


Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 9. toukokuuta 2018 19:43
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Ogr2ogr, SQLite and overwriting existing database


Hi,



> What is the expected behavior if user asks ogr2ogr to create a SQLite

> database that already exists? The driver page

> http://www.gdal.org/drv_sqlite.html tells that "The SQLite driver supports

> creating new SQLite database files, or adding tables to existing ones. Note

> that a new database file cannot be created over an existing file."



Outdated statement from 1.X era (just removed it). Now that the SQLite driver 
is a GDAL driver, it follows the general mechanism of deleting the existing 
file.



>

> But if I lock the existing database by creating a connection into it I get

> these messages: ERROR 1: A file system object called 'foo.gpkg' already

> exists.

> ERROR 1: gpkg driver failed to create foo.gpkg

>

> Does "ERROR 1: A file system object called 'foo.gpkg' already exists." mean

> that the purpose to prevent overwriting but the check does not work? I have

> been using quit overwriting as a feature but is it actually a bug?



This is a particularity of the GPKG driver. The file should have normally 
deleted but due t thhe lock QuietDelete() failed. And the GPKG driver has a 
specific code to prevent overwriting of an exiting file in its Create() method. 
In fact it errors out if APPEND_SUBDATASET=YES is not specified, and if it is 
specified it happens a new raster dataset to the existing file



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Ogr2ogr, SQLite and overwriting existing database

[Rahkonen Jukka (MML)]

Hi,

What is the expected behavior if user asks ogr2ogr to create a SQLite database 
that already exists? The driver page http://www.gdal.org/drv_sqlite.html tells 
that
"The SQLite driver supports creating new SQLite database files, or adding 
tables to existing ones. Note that a new database file cannot be created over 
an existing file."

However, all these commands do overwrite the existing SQlite database

ogr2ogr -f sqlite -dsco spatialite=yes foo.sqlite foo.jml
ogr2ogr -f sqlite foo.sqlite foo.jml
ogr2ogr -f gpkg foo.gpkg foo.jml

But if I lock the existing database by creating a connection into it I get 
these messages:
ERROR 1: A file system object called 'foo.gpkg' already exists.
ERROR 1: gpkg driver failed to create foo.gpkg

Does "ERROR 1: A file system object called 'foo.gpkg' already exists." mean 
that the purpose to prevent overwriting but the check does not work? I have 
been using quit overwriting as a feature but is it actually a bug?

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] OGR_STYLE BRUSH(fc:#0000FF) converts into MapInfo Brush (1, 255)

[Rahkonen Jukka (MML)]

Hi MITAB-folks,

See this: 
https://gis.stackexchange.com/questions/281584/export-to-tab-with-styling-using-ogr2ogr
Is it right or wrong to convert blue fill into Brush (1,255)?

-Jukka Rahkonen-




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Quete_string_fields creation option for CSV

[Rahkonen Jukka (MML)]

Even Rouault wrote:
Re: [gdal-dev] Quete_string_fields creation option for CSV
 
> Actually I've reworked this to be
 
> STRING_QUOTING=IF_NEEDED/IF_AMBIGUOUS/ALWAYS (Starting with GDAL 2.3): 
> whether to double-quote strings. IF_AMBIGUOUS means that string values that 
> look like numbers will be quoted (it also implies IF_NEEDED). Defaults to 
> IF_AMBIGUOUS (behaviour in older versions was IF_NEEDED)
 
Feels like a good set of options and I will probably enjoy IF_AMBIGUOUS myself 
sooner or later because we have IDs with leading zeroes here and there.

-Jukka-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Quete_string_fields creation option for CSV

[Rahkonen Jukka (MML)]

Hi,

See this 
https://gis.stackexchange.com/questions/279960/ogr2ogr-create-csv-with-strings-wrapped-in-double-quotes

The GDAL CSV driver supports quoted string fields in reading but I suppose that 
data can't be written like that.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GDAL can't open MBTiles sample from the MapBox site

[Rahkonen Jukka (MML)]

Hi Even,

Thanks, after updating my GDAL to a more recent 2.3.0dev version ogrinfo indeed 
shows that information.

-Jukka-

Lähettäjä: Even Rouault <even.roua...@spatialys.com>
Lähetetty: 28. maaliskuuta 2018 17:47
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML)
Aihe: Re: [gdal-dev] GDAL can't open MBTiles sample from the MapBox site

On mercredi 28 mars 2018 14:28:14 CEST Rahkonen Jukka (MML) wrote:
> Hi,
> On page https://www.mapbox.com/help/define-mbtiles/ there is a link into
> https://www.mapbox.com/help/data/trails.mbtiles
>
> My GDAL 2.3.0dev does list MBTiles as a supported format but it can't still
> open this database
>
> gdalinfo trails.mbtiles
> ERROR 4: `trails.mbtiles' not recognized as a supported file format.
> gdalinfo failed - unable to open 'trails.mbtiles'.

Jukka,

Yep, there's no raster in that file, but vector instead.
So:

$ ogrinfo trails.mbtiles -al -so
INFO: Open of `trails.mbtiles'
  using driver `MBTiles' successful.
Metadata:
  ZOOM_LEVEL=14
  center=-113.8348,48.6402,9
  format=pbf
  maxzoom=14
  minzoom=0

Layer name: multilinestring-glacier_trails
Geometry: Unknown (any)
Feature Count: 1409
Extent: (-12743581.355705, 6149206.051486) - (-12604160.216112,
6276397.266552)
Layer SRS WKT:
PROJCS["WGS 84 / Pseudo-Mercator",
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0,
AUTHORITY["EPSG","8901"]],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AUTHORITY["EPSG","4326"]],
PROJECTION["Mercator_1SP"],
PARAMETER["central_meridian",0],
PARAMETER["scale_factor",1],
PARAMETER["false_easting",0],
PARAMETER["false_northing",0],
UNIT["metre",1,
AUTHORITY["EPSG","9001"]],
AXIS["X",EAST],
AXIS["Y",NORTH],
EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0
+x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs"],
AUTHORITY["EPSG","3857"]]
mvt_id: Integer64 (0.0)
Avg_Slope: Real (0.0)
CLASS: Real (0.0)
DESC_SEG: String (0.0)
LENGTH: Real (0.0)
MILES: Real (0.0)
NAME: String (0.0)
ROUTE_NO: Real (0.0)
SEG_NO: Real (0.0)
evaluating: String (0.0)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL can't open MBTiles sample from the MapBox site

[Rahkonen Jukka (MML)]

Hi,
On page https://www.mapbox.com/help/define-mbtiles/ there is a link into 
https://www.mapbox.com/help/data/trails.mbtiles

My GDAL 2.3.0dev does list MBTiles as a supported format but it can't still 
open this database

gdalinfo trails.mbtiles
ERROR 4: `trails.mbtiles' not recognized as a supported file format.
gdalinfo failed - unable to open 'trails.mbtiles'.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Adding WRITE_ID creation option for GeoJSON

[Rahkonen Jukka (MML)]

Hi,

Am I right that there is no option to force the writing of "id" member to 
GeoJSON features? I have noticed that if I run ogr2ogr with -preserve_fid 
option then the "id" member is added
{ "type": "Feature", "id": 0, "properties":

However, if I don't use -preserve_fid I have not discovered any way to get "id" 
into GeoJSON. If I am right, would it make sense to add the WRITE_ID option 
like the existing WRITE_NAME http://www.gdal.org/drv_geojson.html?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: adopt RFC 71: Migration to GitHub

[Rahkonen Jukka (MML)]

+1

-Jukka Rahkonen-


> Hi,

> I move to adopt RFC 71: Migration to GitHub

 https://trac.osgeo.org/gdal/wiki/rfc71_github_migration

> Starting with my +1,

> Best regards,

> Even
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motion: promote GDAL 2.2.4 RC1 for release

[Rahkonen Jukka (MML)]

+1

-Jukka Rahkonen-

>> Motion: promote GDAL 2.2.4 RC1 for release

> Friendly nudge to PSC members to please cast their votes.

> Even
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Any good alternative for the Timeline of Trac available in GitHub?

[Rahkonen Jukka (MML)]

Hi,

I see that GDAL is going to move into GitHub. What I will certainly miss it the 
Timeline of Trac. For a curious reader like me it gives an excellent view to 
anything that happens in the project. Is it so that the closest to Timeline 
that GitHub can offer is the Pulse view (for example 
https://github.com/mapserver/mapserver/pulse/monthly) and that does not offer a 
way to look any farther in the history than the last month?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Validate_gpkg and TEXT with length

[Rahkonen Jukka (MML)]

Even Rouault wrote:

>Jukka,
 
>> 
>> I checked some .gpkg database that is created by FME with the
>> validate_gpkg.py script. It gives this error:
>> 
>> __main__.GPKGCheckException: Req 5: table stand has column
>> standnumberextension of unexpected type TEXT ( 2 )
>> 
>> Excerpt from the standard:
>> TEXT{(maxchar_count)}
>> Variable length string encoded in either UTF-8 or UTF-16, determined by
>> PRAGMA encoding; see http://www.sqlite.org/pragma.html#pragma_encoding The
>> optional maxchar_count defines the maximum number of characters in the
>> string. If not specified, the length is unbounded. The count is provided
>> for informational purposes, and applications MAY choose to truncate longer
>> strings if encountered. When present, it is best practice for applications
>> to adhere to the character count. Stored as SQLite TEXT
>> 
>> Unfortunately there is no example in the standard about how the text field
>> with maxchar count should be created correctly and how it should appear in
>> the PRAGMA. I wonder if the error is caused by the spaces around number 2
>> in TEXT ( 2 ).
 
> Yes this is due to spaces.
 
> My understanding of the specification text, confirmed by the official test 
> suite in
> https://github.com/opengeospatial/ets-gpkg12/blob/02ba0439a4d060c6cb036caa5d8225e18241e8e6/src/main/java/org/opengis/cite/gpkg12/core/DataContentsTests.java#L266
> is that spaces are not allowed, so this should be TEXT(2)
 
> The OGR driver will just recognize TEXT (2) as TEXT.

Hi,

It appeared that this suspicious GeoPackage database was initially created with 
some generic SQLite database design tool. FME and GDAL were only used for 
inserting data into already existing tables and they were not involved with the 
non-GeoPackage standard conformant  contents in sqlite_master. Be aware when 
using such tools because what is OK for SQLite is not necessarily OK for the 
gpkg standard.

Good news is that the validate_gpkg.py script was able to find this issue. 

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Validate_gpkg and TEXT with length

[Rahkonen Jukka (MML)]

Hi,

I checked some .gpkg database that is created by FME with the validate_gpkg.py 
script. It gives this error:

__main__.GPKGCheckException: Req 5: table stand has column standnumberextension
of unexpected type TEXT ( 2 )

Excerpt from the standard:
TEXT{(maxchar_count)}
Variable length string encoded in either UTF-8 or UTF-16, determined by PRAGMA 
encoding; see http://www.sqlite.org/pragma.html#pragma_encoding The optional 
maxchar_count defines the maximum number of characters in the string. If not 
specified, the length is unbounded. The count is provided for informational 
purposes, and applications MAY choose to truncate longer strings if 
encountered. When present, it is best practice for applications to adhere to 
the character count. Stored as SQLite TEXT

Unfortunately there is no example in the standard about how the text field with 
maxchar count should be created correctly and how it should appear in the 
PRAGMA. I wonder if the error is caused by the spaces around number 2 in
TEXT ( 2 ).

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] QGIS3 - Save Map as Image dialog bug?

[Rahkonen Jukka (MML)]

Hi,

Do not cross post. If you are rather sure that you have found a bug, as I feel 
in your case, I suggest to write to the developers list. If there is a chance 
that you perhaps did something wrong it is good to write to the users list. 
There are more people there than in the developers list, some user may know the 
answer for  you, your problem may teach something for other users, and in the 
ideal world the developers are also reading the users mails.

-Jukka Rahkonen-
Lähettäjä: Idan Miara [mailto:i...@miara.com]
Lähetetty: 1. maaliskuuta 2018 12:34
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Kopio: gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] QGIS3 - Save Map as Image dialog bug?

Yes it did sent it intentionally to this list, but I was wandering if it's more 
a developer-list or a user-list question.
Did you mean that I should have better post it on both lists or that there is a 
combined list?


On 1 March 2018 at 12:27, jratike80 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Idan Miara wrote
> Hi,
>
> i think there is a bug in the Save Map as Image dialog in QGIS3:

Did you intentially write to GDAL-devel list? I believe that
QGIS-users/developers would be a better choice.

-Jukka Rahkonen-



--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Check validity of geometries before writing them into vector tiles?

[Rahkonen Jukka (MML)]

Even Rouault wrote:

> Perhaps you could play with the SIMPLIFICATION and SIMPLIFICATION_MAX_ZOOM 
> options ?
Sure, as I wrote I admit that my test did not make much sense, but trying 
things before reading the manual sometimes reveals something interesting.  

> Perhaps you should also use an already simplified layer for the lowest zoom 
> level (see the CONF option)
For sure yes, coordinate space of 4096x4096 is far too small for this dataset.

> Are you sure you get polygons with less than 4 points ? Normally they should 
> be discarded.
Quite sure yes by looking at what the ST_IsValidReason from the SQLite dialect 
prints 
ST_IsValidReason(geometry) (String) = Invalid: Toxic Geometry ... too few points

There are other variants of invalid geometries and actually a few valid as 
well. Here is one example with an invalid component

  MULTIPOLYGON (((516384 6815744,516384 6815744,532768 6815744,516384 6815744)),
((516384 6815744,516384 6815744)),((516384 6815744,516384 6815744)))

My zero tile is here http://www.latuviitta.org/downloads/0.pbf

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Check validity of geometries before writing them into vector tiles?

[Rahkonen Jukka (MML)]

Hi,

I admit that my test was somewhat lunatic but I took some random dataset from 
Finland with 68101 polygons and converted data into MVT with default settings 
which means that minzoom was 0.  As a result 12196 of the source polygons were 
written into the 0-level protobuf tile (in EPSG:3067 gridset) and none of the 
polygons is valid. Most polygons have too few points and those which have 
enough points have self-intersections.

Perhaps there should be some sort of geometry validator in the writer chain?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Vector Tiles in OGR

[Rahkonen Jukka (MML)]

Even Rouault wrote:

>On mardi 13 février 2018 02:06:47 CET jratike80 wrote:
>> Even Rouault-2 wrote
>> 
>> > On lundi 22 janvier 2018 23:40:55 CET Even Rouault wrote:
>> >> Hi,
>> >> 
>> >> > P.S. @Even: Any news on a GDAL/OGR MVT reader?
>> >> 
>> >> Yes, a new OGR MVT reader has just been committed in GDAL trunk:
>> >> http://gdal.org/drv_mvt.html
>> > 
>> > And the writer part (for MVT and MBTILES) driver is now in trunk
>> > 
>> > http://gdal.org/drv_mvt.html
>> > http://gdal.org/frmt_mbtiles.html
>> > 
>> > Even
>> 
>> Hi,
>> 
>> At the moment we create MVT tiles from PostGIS with ST_AsMVT to suit with
>> the national gridset that is based on ETRS-TM35FIN (EPSG:3067) coordinate
>> system. How difficult would you consider to enhance the driver to split and
>> write vector tiles according to other gridsets than the Web Mercator one?
 
>Not that much. But is there a standard way of deriving the tiling scheme for 
>that CRS ? I guess not. You'd probably need to manually define the >coordinate 
>of the left-corner of the tile (0,0) in the CRS and the resolution of the tile 
>at zoom level 0.
 
There is actually a Finnish national recommendation about the EPSG:3067 tiling 
schema. The top-left corner of the zero tile is at
-548576 E, 8388608 N, with resolution of 8192 meters.

http://docs.jhs-suositukset.fi/jhs-suositukset/JHS180_liite1/JHS180_liite1.html#H7

-Jukka-

>Even
 
-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Does ogr2ogr -overwrite truncate also the spatial index in GeoPackage?

[Rahkonen Jukka (MML)]

Hi,

That was the case, once I updated into recent trunk the error message 
disappeared.

-Jukka-

Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 6. helmikuuta 2018 14:54
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Does ogr2ogr -overwrite truncate also the spatial index in 
GeoPackage?


Jukka,



Which version do you use ? If it is a trunk version before Jan 23, you may run 
into the bug fixed by https://trac.osgeo.org/gdal/changeset/41325



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Does ogr2ogr -overwrite truncate also the spatial index in GeoPackage?

[Rahkonen Jukka (MML)]

Hi,

Title says most of it. Running "ogr2ogr -f gpkg -update -overwrite" for a 
vector table in GeoPackage gives an error:

ERROR 1: Spatial index already existing

I wonder if the overwrite mode still does the right thing. Is the R-Tree table 
truncated automatically by the triggers together with the main table? If it is 
then  the result is a valid spatial index and the only issue is the misleading 
error message.

-Jukka Rahkonen

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Cloud Optimized Vectors?

[Rahkonen Jukka (MML)]

Hi,

Cloud optimised tiffs hosted on a HTTP file server work pretty well without a 
need to set up a special tile/WMS/WCS server
https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF

That made me think that how would a Cloud Optimized vector format look like? 
Obviously it should offer fast access to spatial index which delivers offsets 
to data records of perhaps spatially organized chunks of data. Or is there 
already some format that suits relatively well for BBOX based queries over HTTP?

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Why gdal_rasterize fails with MBTiles output?

[Rahkonen Jukka (MML)]

Hi Even,

I agree that the first error message says it all and I guessed that the issue 
was in initializing the output to suit with the tile boundaries. I would also 
say that the documentation of MBTiles format is not so clear for users who have 
as little experience as I have, or less.

"The driver implements the Create() and IWriteBlock() methods, so that 
arbitrary writing of raster blocks is possible, enabling the direct use of 
MBTiles as the output dataset of utilities such as gdalwarp.

On creation, raster blocks can be written only if the geotransformation matrix 
has been set with SetGeoTransform() This is effectively needed to determine the 
zoom level of the full resolution dataset based on the pixel resolution, 
dataset and tile dimensions."

In this case also message ERROR 6: IWriteBlock() not supported if 
georeferencing not set
can make user to believe that there is something wrong with srs of input 
vectors.
-Jukka-

Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 18. joulukuuta 2017 13:12
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Why gdal_rasterize fails with MBTiles output?


Jukka,



>

> gdal_rasterize -ot Byte -burn 255 -burn 0 -burn 0 -ts 4950 4950

> -of MBTiles points_3857.json points_3857.mbtiles

> ERROR 6: Could not find an appropriate zoom level that matches raster pixel

> size



The error message says it all, doesn't it ? MBTiles only accepts a single 
tiling scheme, with a pre-defined set of zoom levels / resolutions. By default, 
when gdal_rasterize will initialize the output raster, the chance that it fits 
to those constraints is extremely unlikely.



Now you may wonder why you can gdal_translate an arbitrary raster as MBTiles. 
This is because MBTiles (similarly to GeoPackage with a predefined tiling 
scheme) sheat in its CreateCopy() implementation to invoke the warping engine 
to reproject/resample to one of the compatible zoom level.



But gdal_rasterize use the Create() interface, so this isn't possible.



Obvious workaround: rasterize to TIF and translate to MBTiles.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Why gdal_rasterize fails with MBTiles output?

[Rahkonen Jukka (MML)]

Hi,

Captured from gis.stackexchange 
https://gis.stackexchange.com/questions/264899/create-mbtiles-from-geojson-in-raster-format

I made some test data in EPSG:3857 and while GeoTIFF output works fine with 
GDAL 2.3dev

gdal_rasterize -ot Byte  -burn 255 -burn 0 -burn 0 -ts 4950 4950
-of GTiff points_3857.json points_3857.tif -co tiled=yes
0...10...20...30...40...50...60...70...80...90...100 - done.

the same command with MBTiles outpur gives me the same error than in the 
question

gdal_rasterize -ot Byte  -burn 255 -burn 0 -burn 0 -ts 4950 4950
-of MBTiles points_3857.json points_3857.mbtiles
ERROR 6: Could not find an appropriate zoom level that matches raster pixel size

0...10...20...30...40...50...60...70...80...90...100 - done.
ERROR 6: IWriteBlock() not supported if georeferencing not set
ERROR 6: IWriteBlock() not supported if georeferencing not set
ERROR 6: IWriteBlock() not supported if georeferencing not set

So it seems that having geojson as input is not the issue, but having MBTiles 
as output.

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Sub-optimal access to cloud optimized GeoTIFF

[Rahkonen Jukka (MML)]

Hi,

This image http://193.166.24.122/gtk/mp_50_750.tif should be prepared 
as adviced in https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF and the 
validation script agrees on that.

python c:\temp\validate_cloud_optimized_geotiff.py mp_50_750.tif
Result:
mp_50_750.tif is a valid cloud optimized GeoTIFF

However, it seems to me that gdal_translate is reading the whole GeoTIFF (from 
byte 0 to 3430936 ) once and then a little bit more (2850413 to 2984219). My 
command and debug info looks like this:

gdal_translate --config GDAL_DISABLE_READDIR_ON_OPEN YES --config 
CPL_VSIL_CURL_ALLOWED_EXTENSIONS .tif  -of GTiff -outsize 10% 10% 
/vsicurl/http://193.166.24.122/gtk/mp_50_750.tif one_tif.tif --debug on
...
HTTP: libcurl/7.37.1 OpenSSL/1.0.1h zlib/1.2.3
VSICURL: GetFileSize(http://193.166.24.122/gtk/mp_50_750.tif)=3430937  
response_code=200
VSICURL: Downloading 0-16383 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Got response_code=206
VSICURL: Downloading 16384-409599 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/http://193.166.24.122/gtk/mp_50_750.tif, 
this=0329B420) succ
eeds as GTiff.
Input file size is 1, 1
GTiff: ScanDirectories()
VSICURL: Downloading 409600-1196031 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Got response_code=206
GTiff: Opened 5000x5000 overview.
VSICURL: Downloading 1196032-2768895 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Got response_code=206
GTiff: Opened 2000x2000 overview.
GTiff: Opened 1000x1000 overview.
GTiff: Opened 500x500 overview.
GTiff: Opened 200x200 overview.
VSICURL: Downloading 2768896-3430936 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Got response_code=206
GTiff: Opened 100x100 overview.
GDAL: QuietDelete(one_tif.tif) invoking Delete()
GDAL: GDALOpen(one_tif.tif, this=0329D090) succeeds as GTiff.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(one_tif.tif, this=0329D090)
0GDAL: GDAL_CACHEMAX = 802 MB
GDAL: GDALDatasetCopyWholeRaster(): 1000*896 swaths, bInterleave=0
VSICURL: Downloading 2850413-2969284 
(http://193.166.24.122/gtk/mp_50_750.tif)...
VSICURL: Download completed
...10...20...30...40...50VSICURL: Downloading 2969285-2984219 
(http://193.166.24.122/gtk/mp_50_7
50.tif)...
VSICURL: Download completed
...60...70...80...90...100 - done.
GDAL: GDALClose(one_tif.tif, this=0329D090)
GDAL: GDALClose(/vsicurl/http://193.166.24.122/gtk/mp_50_750.tif, 
this=0329B420)

I wonder what is going on.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Wiki page for WCS discussion and best practices created

[Rahkonen Jukka (MML)]

Hi,

As an initial step I created a new page about WCS into the developers section 
of the GDAL wiki 
https://trac.osgeo.org/gdal/wiki/WCS%2Binteroperability

I suppose that all who have OSGeo user account can edit the page.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Question about GeoPackage and views

[Rahkonen Jukka (MML)]

Hi,

I was reading the GeoPackage standard and only now read carefully the 
requirement 29

"A GeoPackage MAY contain tables or updateable views containing vector 
features. Every such
feature table or view in a GeoPackage SHALL have a column with column type 
INTEGER and
'PRIMARY KEY AUTOINCREMENT' column constraints"

Should I really read this so that GeoPackage MUST NOT contain non-updateable 
views, or at least they cannot be included in the standard  gpkg_contents and 
gpkg_geometry_columns tables but used only through some extension?

Writable views in SQLite are not too simple to create, see for example 
https://www.gaia-gis.it/fossil/libspatialite/wiki?name=writable-view
However, even read-only views could be very handy from the users and data 
providers point of view :).

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] VRT KernelFilteredSource and overviews

[Rahkonen Jukka (MML)]

Hi,

I was playing with kernel filters in VRT and using this really applies sharpen 
filter on-the-fly when looking the image at native resolution with QGIS:


Red

  image.tif
  1
  
  
  

  3
   0 -1 0 -1 5 -1 0 -1 0
  

  

However, filter is not applied to overviews which means that image looks 
different when I zoom out. Is this the intended behavior?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Gdal_edit and TIFFTAG_EXTRASAMPLES

[Rahkonen Jukka (MML)]

Thanks Even.


I would guess that also other users who have both RGB+alpha and RGB+NIR images 
will appreciate these changes. At least for me it has been plain mystery when 
the fourth band appears as alpha and when as undefined, and how to change the 
interpretation despite using VRT.


-Jukka-


Lähettäjä: Even Rouault <even.roua...@spatialys.com>
Lähetetty: 1. lokakuuta 2017 22:54
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML)
Aihe: Re: [gdal-dev] Gdal_edit and TIFFTAG_EXTRASAMPLES


Updates:



1) I've improved the GTiff driver to support updating color interpretation in 
udpate mode

2) I've improved gdal_edit to support a

-colorinterp_X red|green|blue|alpha|undefined option where X is the band number.

3) I've fixed "tiffset -s ExtraSamples 1 val" to work (the issue was specific 
to specifying just one value indeed)



Note that with libtiff when you change an existing tag, the file size will 
slightly increase each time you do a change.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Gdal_edit and TIFFTAG_EXTRASAMPLES

[Rahkonen Jukka (MML)]

Hi,

I have been searching an easy way to change the interpretion of 4-channel 
images between RGBA and RGB+undefined. Wouldn't if be useful to make gdal_edit 
to support also changing the TIFFTAG_EXTRASAMPLES metadata? Or do we have some 
other easy way for performing that task?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] KML recognized as GML by ogr2ogr

[Rahkonen Jukka (MML)]

Hi,

See 
https://gis.stackexchange.com/questions/256861/kml-to-shapefile-problems-ogr2ogr

I have verified with todays GDAL 2.3.0dev  from gisinternals that ogrinfos open 
the KML file form that question with LIBKML driver while ogr2ogr opens it with 
GML driver and fails.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] ESRI json REST service and first page of data

[Rahkonen Jukka (MML)]

Hi,

Based on 
https://gis.stackexchange.com/questions/256668/ogr2ogr-get-geojson-from-map-server-misses-features-and-creates-duplicates
 it is possible that ESRI REST service may return different first features when 
query is sent without "resultRecordCount" and "resultOffset" than if the query 
includes resultRecordCount==0.  I know that the same happens 
with Geoserver WFS so I would not be very surprised.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Case sensitive open option in GMLAS?

[Rahkonen Jukka (MML)]

Hi,

I get this error with GMLAS driver and "-oo remove_unused_layers=yes"

Warning 6: driver GMLAS does not support open option remove_usused_layers

Warning disappears with -oo REMOVE_UNUSED_LAYERS=yes. 
The very similar open option "-oo remove_unused_fields" does not give an error.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] How to read CSV with GeoJSON correctly?

[Rahkonen Jukka (MML)]

Hi,

I was thinking about this question on gis.stackexchange 
https://gis.stackexchange.com/questions/254536/how-to-convert-txt-containing-two-columns-id-and-geojson-geometry

I copy-pasted the CSV example and edited it to use semicolon as field 
separator. Then I read http://www.gdal.org/drv_csv.html and had a try with 
ogrinfo 

ogrinfo geojson_csv.csv -al -oo GEOM_POSSIBLE_NAMES=geoJSON
INFO: Open of `geojson_csv.csv'
  using driver `CSV' successful.

Layer name: geojson_csv
Geometry: Unknown (any)
Feature Count: 1
Layer SRS WKT:
(unknown)
Geometry Column = geoJSON
Id: String (0.0)
geoJSON: String (0.0)
OGRFeature(geojson_csv):1
  Id (String) = 49518
  geoJSON (String) = {type:Polygon,coordinates:[[[29.9596,-2.3272],[29.9189,-2.7
9],[29.3706,-2.8401],[29.1497,-2.5917],[29.0405,-2.7447],[28.8619,-2.5309],[29.1
1,-1.8602],[29.3616,-1.5092],[29.8237,-1.3089],[29.9149,-1.4825],[30.0524,-1.431
,[30.8394,-1.6514],[30.8567,-2.3151],[30.1356,-2.4382],[29.9596,-2.3272]]]}

The GeoJSON geometry is not recognized but ogrinfo converts it into a string, 
with double quetes removed. Is there something wrong in the CSV or in my syntax?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Tilesize in JPEG2000 and gdalinfo

[Rahkonen Jukka (MML)]

Hi,

If I run gdalinfo against the JPEG2000 image in 
http://kartat.kapsi.fi/files/orto/etrs-tm35fin/mara_v_25000_50/2013/N53/02m/1/N5144H.jp2
the JP2ECW driver reports the blocksize as (256x256) and JPEG2000OpenJPEG 
driver reports that the block size is (1024x1024).

What interests me would be to know the fact that this image has only one 
JPEG2000 tile with a tilesize of (11754x11754). Is there any driver-independent 
way for getting this information?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Geopackage raster and COPY_SRC_OVERVIEWS

[Rahkonen Jukka (MML)]

Hi Even,

Thank you for the advice. I thought a bit further and mixed-source overviews 
are also common in tile cache storages (MapCache, MapProxy) or when GeoPackage 
is used as an outputformat for WMS (GeoServer). There seems to be some need for 
having an easy general method for copying overviews. Let's see if someone 
appears with funding or just implements it.

-Jukka-

Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 28. elokuuta 2017 15:03
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Geopackage raster and COPY_SRC_OVERVIEWS


On lundi 28 août 2017 11:30:00 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

>

> I wonder if it could be feasible to support similar COPY_SRC_OVERVIEWS

> option than GeoTIFF driver does for existing overview levels which match

> with the power of two series. Some people have constructed GeoTIFFs or

> virtual VRT rasters which have overviews from different raster sources

> (raster maps with different rendering according to the scale) and gdaladdo

> can't generate such overviews from the best resolution level.



That's certainly doable (pending someone actually implementing it) and probably 
something that could be generalized at the gdal_translate level



In the meantime, you can achieve a similar result with for example gdalwarp (in 
update mode) and using the GPKG ZOOM_LEVEL open option.



Example:



- gdal_translate small_world.tif small_world.gpkg -of gpkg

(the full resolution data is at zoom level 1)



- gdal_translate small_world.tif ovr.tif -outsize 50% 50% -scale 0 255 127 255

(create a brighter overview)



- gdalwarp ovr.tif small_world.gpkg -doo ZOOM_LEVEL=0

(populate zoom_level=0, ie first overview level, with the content of this file)



The above can be generalized too as many overviews as needed. You may need to 
use the -ovr switch of gdalwarp to select the appropriate source overview.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Geopackage raster and COPY_SRC_OVERVIEWS

[Rahkonen Jukka (MML)]

Hi,


I wonder if it could be feasible to support similar COPY_SRC_OVERVIEWS option 
than GeoTIFF driver does for existing overview levels which match with the 
power of two series. Some people have constructed GeoTIFFs or virtual VRT 
rasters which have overviews from different raster sources (raster maps with 
different rendering according to the scale) and gdaladdo can't generate such 
overviews from the best resolution level.



-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Schema of gpkg_metadata in Geopackage

[Rahkonen Jukka (MML)]

Hi,


Our document

http://www.gdal.org/drv_geopackage.html says that gpkg driver targets standard 
version 1.0/1.0.1. For me it seems that starting from GDAL 2.2 the target is 
version 1.2, it that right?


-Jukka Rahkonen-


Lähettäjä: Even Rouault <even.roua...@spatialys.com>
Lähetetty: 23. elokuuta 2017 16:55
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML)
Aihe: Re: [gdal-dev] Schema of gpkg_metadata in Geopackage


On mercredi 23 août 2017 13:33:07 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

> GDAL 2.3-dev creates gpkg_metadata table as

>

> CREATE TABLE gpkg_metadata (id INTEGER CONSTRAINT m_pk PRIMARY KEY ASC NOT

> NULL UNIQUE,md_scope TEXT NOT NULL DEFAULT 'dataset',md_standard_uri TEXT

> NOT NULL,mime_type TEXT NOT NULL DEFAULT 'text/xml',metadata TEXT NOT NULL

> DEFAULT '');

>

> In GPKG standard v. 1.0.1 and 1.2 the SQL is

>

>

> CREATE TABLE gpkg_metadata (

>

> id INTEGER CONSTRAINT m_pk PRIMARY KEY ASC NOT NULL,

>

> md_scope TEXT NOT NULL DEFAULT 'dataset',

>

> md_standard_uri TEXT NOT NULL,

>

> mime_type TEXT NOT NULL DEFAULT 'text/xml',

>

> metadata TEXT NOT NULL

> )

>

> It seems that UNIQUE was removed in 1.0.1 corrigendum.



You've good eyes ;-) I've just removed UNIQUE in trunk to align with latest 
specs, but I guess that doesn't make a difference in practice (and validation 
checks are fine with or without UNIQUE). A primary key is necessarily unique.



> I am not sure about

> what to think about having "metadata TEXT NOT NULL DEFAULT ''". In table 14

> in standard version 1.0.1 and also in table 32 in standard v. 1.2 there is

> an empty string as a default value but it does not reflect to the CREATE

> TABLE example.



In the live spec at http://www.geopackage.org/spec/#gpkg_metadata_sql , the 
example is metadata TEXT NOT NULL DEFAULT '' . Spec was fixed per

https://github.com/opengeospatial/geopackage/issues/243





But I see the http://www.geopackage.org/spec/#metadata_example_appendix is 
wrong...

Filed as https://github.com/opengeospatial/geopackage/issues/382





Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Schema of gpkg_metadata in Geopackage

[Rahkonen Jukka (MML)]

Hi,

GDAL 2.3-dev creates gpkg_metadata table as

CREATE TABLE gpkg_metadata (id INTEGER CONSTRAINT m_pk PRIMARY KEY ASC NOT NULL 
UNIQUE,md_scope TEXT NOT NULL DEFAULT 'dataset',md_standard_uri TEXT NOT 
NULL,mime_type TEXT NOT NULL DEFAULT 'text/xml',metadata TEXT NOT NULL DEFAULT 
'');

In GPKG standard v. 1.0.1 and 1.2 the SQL is


CREATE TABLE gpkg_metadata (

id INTEGER CONSTRAINT m_pk PRIMARY KEY ASC NOT NULL,

md_scope TEXT NOT NULL DEFAULT 'dataset',

md_standard_uri TEXT NOT NULL,

mime_type TEXT NOT NULL DEFAULT 'text/xml',

metadata TEXT NOT NULL
)

It seems that UNIQUE was removed in 1.0.1 corrigendum. I am not sure about what 
to think about having "metadata TEXT NOT NULL DEFAULT ''". In table 14 in 
standard version 1.0.1 and also in table 32 in standard v. 1.2 there is an 
empty string as a default value but it does not reflect to the CREATE TABLE 
example.

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] How to drop a raster layer from GeoPackage?

[Rahkonen Jukka (MML)]

Hi,

There is an interesting question in gis.stackexchange about how to remove a 
raster layer from GeoPackage 
https://gis.stackexchange.com/questions/252782/remove-a-raster-from-geopackage.

I had a try and gdalmanage did not have any effect on the GeoPackage database 
but the workflow with ogrinfo works for me. Are there any other alternatives?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Autodetect "0012" as string in CSV

[Rahkonen Jukka (MML)]

Hi Even,


Yeah, should have been reading the documents more closely. This keeps the 
leading zeroes.


ogrinfo csvtext.csv -oo autodetect_type=yes -oo quoted_fields_as_string=yes -al
INFO: Open of `csvtext.csv'
  using driver `CSV' successful.

Layer name: csvtext
Geometry: None
Feature Count: 1
Layer SRS WKT:
(unknown)
field_1: String (0.0)
field_2: Integer (0.0)
field_3: Integer (0.0)
field_4: Real (0.0)
field_5: String (0.0)
OGRFeature(csvtext):1
  field_1 (String) = 00012
  field_2 (Integer) = 12345
  field_3 (Integer) = 1234
  field_4 (Real) = 234.65
  field_5 (String) = text


-Jukka-



Lähettäjä: Even Rouault <even.roua...@spatialys.com>
Lähetetty: 18. elokuuta 2017 14:30
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML)
Aihe: Re: [gdal-dev] Autodetect "0012" as string in CSV


On vendredi 18 août 2017 10:59:47 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

> The schema of a CSV file like this

> "00012",012345,1234,234.65,"text"

> is autodetected this way:

>

> ogrinfo csvtext.csv -oo autodetect_type=yes -al

> INFO: Open of `csvtext.csv'

> using driver `CSV' successful.

>

> Layer name: csvtext

> Geometry: None

> Feature Count: 1

> Layer SRS WKT:

> (unknown)

> field_1: Integer (0.0)

> field_2: Integer (0.0)

> field_3: Integer (0.0)

> field_4: Real (0.0)

> field_5: String (0.0)

> OGRFeature(csvtext):1

> field_1 (Integer) = 12

> field_2 (Integer) = 12345

> field_3 (Integer) = 1234

> field_4 (Real) = 234.65

> field_5 (String) = text

>

> It would be good to autodetect also "00012" as a string. Double quotes are

> correctly stripped of from "text" so I suppose that GDAL already knows that

> they are used for enclosing strings.



Jukka,



Add -oo QUOTED_FIELDS_AS_STRING=YES



I don't remember the rationale for the default of this option to be NO. 
Presumably the use case while developping that was with a file generated by a 
CSV exporter always surrounding field content by double quotes.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Autodetect "0012" as string in CSV

[Rahkonen Jukka (MML)]

Hi,

The schema of a CSV file like this
"00012",012345,1234,234.65,"text"
is autodetected this way:

ogrinfo csvtext.csv -oo autodetect_type=yes -al
INFO: Open of `csvtext.csv'
  using driver `CSV' successful.

Layer name: csvtext
Geometry: None
Feature Count: 1
Layer SRS WKT:
(unknown)
field_1: Integer (0.0)
field_2: Integer (0.0)
field_3: Integer (0.0)
field_4: Real (0.0)
field_5: String (0.0)
OGRFeature(csvtext):1
  field_1 (Integer) = 12
  field_2 (Integer) = 12345
  field_3 (Integer) = 1234
  field_4 (Real) = 234.65
  field_5 (String) = text

It would be good to autodetect also "00012" as a string. Double quotes are 
correctly stripped of from "text" so I suppose that GDAL already knows that 
they are used for enclosing strings.

See 
https://gis.stackexchange.com/questions/252482/keeping-leading-zeroes-in-qgis-in-big-files

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Interesting issue with ogr2ogr and -tps

[Rahkonen Jukka (MML)]

Hi,

There is a well written question in gis.stackexchange about how the -tps option 
in ogr2ogr behaves 
https://gis.stackexchange.com/questions/245391/ogr2org-tps-spline-method-creates-progressive-error

Does anybody have a clue about what happens?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a shapefile

[Rahkonen Jukka (MML)]

Hmm, but how about other formats? Is there some general workaround? For example 
conversion into shapefiles fails because of this error while geopackage seems 
to be OK.

-Jukka-


Lähettäjä: Rahkonen Jukka (MML)
Lähetetty: 19. toukokuuta 2017 18:32
Vastaanottaja: 'Even Rouault' <even.roua...@spatialys.com>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a 
shapefile

Hi,

That's it. I think I should have known to read carefully the PG driver manual 
http://www.gdal.org/drv_pg.html.

-Jukka-

Even Rouault wrote:

Re: [gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a shapefile


On vendredi 19 mai 2017 15:15:02 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

> See this gis.stackexchange question

> https://gis.stackexchange.com/questions/240900/shp2pgsql-segmentation-fault

> .

>

> I have confirmed that Europe_coastline.shp from this zip

> https://www.eea.europa.eu/data-and-maps/data/eea-coastline-for-analysis-1/g

> is-data/europe-coastline-shapefile/at_download/file can't be saved into

> PostGIS with ogr2ogr. The error message is exactly this

>

> ERROR 1: COPY statement failed. ERROR: numeric field overflow

> DETAIL: A field with precision 19, scale 11 must round to an absolute value

> less than 10^8. CONTEXT: COPY coastline, line 1, column shape_leng:

> "275736334.1323000"

>

> The error seems to happen in PostGIS which creates a field for "shape_leng"

> as NUMERIC(19,11) and then insertion of value 275736334.1323000 fails.

> If GDAL user gets this kind of (invalid?) shapefile, what can they do? I

> had a try with -oo ADJUST_TYPE=YES but with no luck. Opening the shapefile

> and saving a copy with OpenJUMP indeed works because OpenJUMP is changing

> the field definition from Shape_Leng: Real (19.11) into Shape_Leng: Real

> (33.16).



Use -lco PRECISION=NO (PostGIS driver option) to create the field as FLOAT8 
instead.



>

> -Jukka Rahkonen-





--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a shapefile

[Rahkonen Jukka (MML)]

Hi,

That's it. I think I should have known to read carefully the PG driver manual 
http://www.gdal.org/drv_pg.html.

-Jukka-

Even Rouault wrote:

Re: [gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a shapefile


On vendredi 19 mai 2017 15:15:02 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

> See this gis.stackexchange question

> https://gis.stackexchange.com/questions/240900/shp2pgsql-segmentation-fault

> .

>

> I have confirmed that Europe_coastline.shp from this zip

> https://www.eea.europa.eu/data-and-maps/data/eea-coastline-for-analysis-1/g

> is-data/europe-coastline-shapefile/at_download/file can't be saved into

> PostGIS with ogr2ogr. The error message is exactly this

>

> ERROR 1: COPY statement failed. ERROR: numeric field overflow

> DETAIL: A field with precision 19, scale 11 must round to an absolute value

> less than 10^8. CONTEXT: COPY coastline, line 1, column shape_leng:

> "275736334.1323000"

>

> The error seems to happen in PostGIS which creates a field for "shape_leng"

> as NUMERIC(19,11) and then insertion of value 275736334.1323000 fails.

> If GDAL user gets this kind of (invalid?) shapefile, what can they do? I

> had a try with -oo ADJUST_TYPE=YES but with no luck. Opening the shapefile

> and saving a copy with OpenJUMP indeed works because OpenJUMP is changing

> the field definition from Shape_Leng: Real (19.11) into Shape_Leng: Real

> (33.16).



Use -lco PRECISION=NO (PostGIS driver option) to create the field as FLOAT8 
instead.



>

> -Jukka Rahkonen-





--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Ogr2ogr, PostGIS and numeric field overflow with a shapefile

[Rahkonen Jukka (MML)]

Hi,

See this gis.stackexchange question 
https://gis.stackexchange.com/questions/240900/shp2pgsql-segmentation-fault.

I have confirmed that Europe_coastline.shp from this zip 
https://www.eea.europa.eu/data-and-maps/data/eea-coastline-for-analysis-1/gis-data/europe-coastline-shapefile/at_download/file
 can't be saved into PostGIS with ogr2ogr.  The error message is exactly this

ERROR 1: COPY statement failed. ERROR:  numeric field overflow
DETAIL:  A field with precision 19, scale 11 must round to an absolute value 
less than 10^8.
CONTEXT:  COPY coastline, line 1, column shape_leng: "275736334.1323000"

The error seems to happen in PostGIS which creates a field for "shape_leng" as 
NUMERIC(19,11) and then insertion of value 275736334.1323000 fails. If GDAL 
user gets this kind of (invalid?) shapefile, what can they do? I had a try with 
-oo ADJUST_TYPE=YES but with no luck. Opening the shapefile and saving a copy 
with OpenJUMP indeed works because OpenJUMP is changing the field definition 
from Shape_Leng: Real (19.11) into Shape_Leng: Real (33.16).

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Wrong namespaces in GML 3.2 written by GDAL

[Rahkonen Jukka (MML)]

Hi Evan,

Oh yeah, a few lines later I can read
http://www.opengis.net/gml/3.2; 
schemaLocation="http://schemas.opengis.net/gml/3.2.1/gml.xsd"/>
http://www.opengis.net/gmlsf/2.0; 
schemaLocation="http://schemas.opengis.net/gmlsfProfile/2.0/gmlsfLevels.xsd"/>

Sorry for the noise.

-Jukka-

Even Rouault  wrote:
Re: [gdal-dev] Wrong namespaces in GML 3.2 written by GDAL


On jeudi 27 avril 2017 17:01:27 CEST Rahkonen Jukka (MML) wrote:

> Hi,

>

> With GML 3.2 output GDAL 2.2-dev writes into the schema

>

> xmlns:gml="http://www.opengis.net/gml/3.2;

> xmlns:gmlsf="http://www.opengis.net/gmlsf/2.0;

>

> The first one redirects to http://schemas.opengis.net/gml/3.2.1/gml.xsd.

> The latter one gives "Not found". I guess that the schema is nowadays in

> http://schemas.opengis.net/gmlsfProfile/2.0/



Jukka,



I think what GDAL writes currently is correct. There's no requirement for a 
namespace URI to be HTTP resolvable and point to a schema. This is just a 
conventional name.



Actually if you look at

http://schemas.opengis.net/gmlsfProfile/2.0/gmlsfLevels.xsd , you can see

xmlns:gmlsf="http://www.opengis.net/gmlsf/2.0;



Even




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Wrong namespaces in GML 3.2 written by GDAL

[Rahkonen Jukka (MML)]

Hi,

With GML 3.2 output GDAL 2.2-dev writes into the schema

xmlns:gml="http://www.opengis.net/gml/3.2;
xmlns:gmlsf="http://www.opengis.net/gmlsf/2.0;

The first one redirects to http://schemas.opengis.net/gml/3.2.1/gml.xsd.
The latter one gives "Not found". I guess that the schema is nowadays in 
http://schemas.opengis.net/gmlsfProfile/2.0/

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] SQLite-SQL problem with some layer names

[Rahkonen Jukka (MML)]

Hi,

Hi,


I had a try with editing the .dox file that is used for generating 
http://www.gdal.org/ogr_sql_sqlite.html
 with changeset #37971. Could you check if the text makes sense and appears 
right?


-Jukka-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] SQLite-SQL problem with some layer names

[Rahkonen Jukka (MML)]

Hi,

I have problems in using the SQLite dialect if layer names contains numbers or 
underscores. I am on Windows. 

ogrinfo -dialect sqlite 3d.shp -sql "select * from 3d"
INFO: Open of `3d.shp'
  using driver `ESRI Shapefile' successful.
ERROR 1: In ExecuteSQL(): sqlite3_prepare(select * from 3d):
  unrecognized token: "3d"

Same happens with a proper SQL with double quotes around the identifier
ogrinfo -dialect sqlite 3d.shp -sql "select * from "3d""

However, with single quotes it works
ogrinfo -dialect sqlite 3d.shp -sql "select * from '3d'"
Even this one, yields a correct result
ogrinfo -dialect sqlite 3d.shp -sql "select 'fixed_string'  from '3d'"

Layers without numbers or underscores in names behave right 
ogrinfo -dialect sqlite test.shp -sql "select * from "test"'"

Is there something that could be fixed in the code, or should I add a line to 
two into the documentation http://www.gdal.org/ogr_sql_sqlite.html?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] [Geoserver-devel] GML lat-lon coverages and interoperability

[Rahkonen Jukka (MML)]

Hi,

Even wrote:
However I'm not sure in which way GDAL is involved in this discussion :-)


I am preparing for the future when GDAL will have a support for WCS 2.0.1 and 
it will try to read EPSG:4326 coverage data from GeoServers witch describe 
coverage like this: 
http://demo.geo-solutions.it/geoserver/wcs?service=WCS=2.0.1=DescribeCoverage=nurc__Arc_Sample

I have a feeling that without a) making GDAL to interpret gridFunction or b) 
Geoserver changing the way to define rectified grid (if it is possible) GDAL 
will fail. If not before then at latest when it tries to read a subset of 
coverage or to put the server to do server side scaling to size.

Andrea wrote:

If there is no 1:1 correspondence between geographic and raster space axis, 
then I see no other way but to use Function to explicitly correlate the two.
How would you propose to proceed instead?

Perhaps with offset vectors? An example taken from “OGC GML in JPEG 2000 
(GMLJP2) Encoding StandardPart 1: Core”



http://www.opengis.net/def/crs/EPSG/0/4326;>


1 1
3 10


Lat Long


9.9 9.9


0 3.71e-005
-3.71e-005 0


I hope I have understood right when I believe that this advice means “Read data 
from the JPEG 2000 file in sequence. Place the first data value to the origin 
and the second one where the first offsetVector points. It is: keep the 
latitude and increase the longitude by 3.71e-005 . Continue till the “high” 
limit of Long axis (10). Now the first row  is full. Place the 11th value where 
the second offsetVector points – so decrease Lat by -3.71e-005 and keep Long at 
the initial value. Fill the second row. Then one more step down and fill the 
3rd row. You have finished”.

So even the grid axes are “column first – rows next” it is possible to fill the 
grid by streaming from the data source that is organized as “rows first” like 
JPEG 2000 or GeoTIFF. I can imagine I could demonstrate that if I had a chess 
board and 64 beans, even if someone rotates the board by 90 degrees.

I must admit that I am not at all sure that I have understood it right. 
Actually it is not even important for me. I just would like to see WCS to work 
fluently so that all the servers and clients would understand each other. 
Somehow I have hoped the OGC would support this goal by giving understandable 
examples and best practice documents.


By the way, Mr Baumann told that  
coverages.wg-ow...@lists.opengeospatial.org
 is an open list but it does not seem to be open for everybody:


"This is an OGC member's only list. More information can be found at:

http://www.opengeospatial.org/ogc/join . If your organization is a member, 
please have your organization's representative create you an OGC portal 
account. Once you have an account, you may then request access to the desired 
lists. If you have questions, you may send you and your company's contact 
information along with an explanation to ListMaster @OpenGeospatial.org . Thank 
you. OGC ListMaster"



-Jukka Rahkonen-







Lähettäjä: Andrea Aime [mailto:andrea.a...@geo-solutions.it]
Lähetetty: 10. huhtikuuta 2017 16:57
Vastaanottaja: Even Rouault 
Kopio: Geoserver-devel ; 
'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
; Peter Baumann 
Aihe: Re: [Geoserver-devel] [gdal-dev] GML lat-lon coverages and 
interoperability

Hi Even,
in CIS 1.1 they state:

Coverages are assumed to have a 1:1 correlation between the axis names given in 
axis-Labels and gridLabels, i.e.: they shall relate pairwise, given by their 
sequence position. For example, axisLabels=“Lat Long h date” and gridLabels={i 
j k l} implies a correspondence of Lat with i, Long with j, h with k, and date 
with l.

If there is no 1:1 correspondence between geographic and raster space axis, 
then I see no other way but to use Function to explicitly correlate the two.
How would you propose to proceed instead?

Cheers
Andrea


On Mon, Apr 10, 2017 at 3:31 PM, Even Rouault 
> wrote:

On lundi 10 avril 2017 15:06:55 CEST Andrea Aime wrote:

> Hi,

> sorry I lost track of the subject. So, to close up, and approach where the

> envelope is reported as "lat lon" and the "i j" raster space

> axis would map pairwise, so i pointing north-wards and j east-wards, would

> be considered valid?

> E.g., if one rescaling says "i is going to be 200 pixels" it really means

> the output image will be 200 pixels tall?



IMHO, importing EPSG axis order mess in raster space would seem to be foolish 
to me and cause major annoyances. Performance-wise, processing flipped rasters 
with classic algorithms that process line per line would be terrible as soon as 
they are large enough, since reading each line would in practice require to 
read the whole raster each time.



Even



--

Spatialys - Geospatial professional 

Re: [gdal-dev] GML lat-lon coverages and interoperability

[Rahkonen Jukka (MML)]

Hi,


Adding mapserver-dev to recipients in hope that Stephan Meissl notices and 
could share his opinion.  Mapserver WCS 2.0.1 does not generate axisOrder.


-Jukka Rahkonen-


Lähettäjä: Even Rouault <even.roua...@spatialys.com>
Lähetetty: 3. huhtikuuta 2017 23:03
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML); Andrea Aime; geoserver-de...@lists.sourceforge.net
Aihe: Re: [gdal-dev] GML lat-lon coverages and interoperability


On lundi 3 avril 2017 13:19:18 CEST Rahkonen Jukka (MML) wrote:

> Great, first hit from Andrea. I am awaiting if Even strikes back.



I don't really have an opinion on this. I haven't read the relevant part of the 
GML or GMLCOV spec and I'm afraid I wouldn't understand it well. The GDAL 
GMLJP2 code just follows what the GMLJP2 spec and examples suggest, as well as 
data productions. Perhaps it is not "correct"...



> His

> creation (GDAL JP2OpenJPEG driver) writes out EPSG:4326 coverage into GML

> JP2 this way, without having gridFunction.

>  xmlns:gml="http://www.opengis.net/gml;

> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;

> xsi:schemaLocation="http://www.opengis.net/gml

> http://schemas.opengis.net/gml/3.1.1/profiles/gmlJP2Profile/1.0.0/gmlJP2Pro

> file.xsd">



> 

> 61.7282236395245 24.7248340978191

> 61.7838827490233 24.8422668729657

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 

> 0 0

> 15517 7354

> 

> 

> x

> y

> 

>  srsName="urn:ogc:def:crs:EPSG::4326">

61.7838789652633

> 24.7248378815791 

> 

> 0

> 7.56751998624388e-006

 srsName="urn:ogc:def:crs:EPSG::4326">-7.56751998624388e-006

> 0 

> 

> 

> 

> 

> gmljp2://codestream/0

> Record Interleaved

> 

> 

> 

> 

> 

> 

> 

>

> -Jukka-

>

> Lähettäjä: andrea.a...@gmail.com [mailto:andrea.a...@gmail.com] Puolesta

> Andrea Aime

Lähetetty: 3. huhtikuuta 2017 15:58

> Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>

> Kopio: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org)

> <gdal-dev@lists.osgeo.org>; geoserver-de...@lists.sourceforge.net

Aihe:

> Re: [gdal-dev] GML lat-lon coverages and interoperability

>

> On Mon, Apr 3, 2017 at 2:32 PM, Rahkonen Jukka (MML)

> <jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahkonen@maanmittauslaito

> s.fi>> wrote:

With orthophoto coverages I believe that all cases could be

> handled with the default gridFunctions: startPoint at gml:Low, axisOrder +1

> +2, sequence rule “Linear”. As far as I have understood GDAL trusts that

> this is also what the other partners do. However, it seems that GeoServer

> developers have decided to handle the latitude-longitude axis flip in WCS

> 2.0.1 by changing the order of grid axis with a function instead of writing

> the offsetVectors to suit with the default +1 +2 axis order.

> 

> 

> Linear

> 0 0

> 

> 

>

> Hi Jukka,

> the decision was not arbitrary, and imho, necessary.

>

> Rasters pixels are normally represented as east/north, but OGC forces us to

> describe geographic data in lat/lon order.

So how do you tell the client,

> "look, I'm telling you that the axis order is north/east, but really, I'm

> giving you back data in east/north order instead"?

> Cheers

> Andrea

>

> --

> ==

> GeoServer Professional Services from the experts! Visit

> http://goo.gl/it488V for more information.

> ==

>

> Ing. Andrea Aime

> @geowolf

> Technical Lead

>

> GeoSolutions S.A.S.

> Via di Montramito 3/A

> 55054 Massarosa (LU)

> phone: +39 0584 962313

> fax: +39 0584 1660272

> mob: +39 339 8844549

>

> http://www.geo-solutions.it

> http://twitter.com/geosolutions_it

>

>

> AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

>

> Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i

> file/s allegato/i sono da considerarsi strettamente riservate. Il loro

> utilizzo è consentito esclusivamente al destinatario del messaggio, per le

> finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio

> senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia

> via e-mail e di procedere alla distruzione del messaggio stesso,

> cancellandolo dal Vostro sistema. Conservare il messaggio stesso,

> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od

> utilizzarlo per finalità diverse, costituisce comportamento contrario ai

> principi dettati dal D.Lgs. 196/2003.

>

>

> The information in this message and/or attachments, is intended solely for

> the attention and use of the named addressee(s) and 

Re: [gdal-dev] GML lat-lon coverages and interoperability

[Rahkonen Jukka (MML)]

Hi,

Gdalinfo about my test image “4326.jp2”:

Size is 15518, 7355
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (24.724834097819130,61.783882749023277)
Pixel Size = (0.07567519986,-0.07567519986)
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (  24.7248341,  61.7838827) ( 24d43'29.40"E, 61d47' 1.98"N)
Lower Left  (  24.7248341,  61.7282236) ( 24d43'29.40"E, 61d43'41.61"N)
Upper Right (  24.8422669,  61.7838827) ( 24d50'32.16"E, 61d47' 1.98"N)
Lower Right (  24.8422669,  61.7282236) ( 24d50'32.16"E, 61d43'41.61"N)

-Jukka-

Lähettäjä: andrea.a...@gmail.com [mailto:andrea.a...@gmail.com] Puolesta Andrea 
Aime
Lähetetty: 3. huhtikuuta 2017 16:31
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Kopio: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
<gdal-dev@lists.osgeo.org>; geoserver-de...@lists.sourceforge.net
Aihe: Re: [gdal-dev] GML lat-lon coverages and interoperability

On Mon, Apr 3, 2017 at 3:19 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:

  61.7282236395245 24.7248340978191
  61.7838827490233 24.8422668729657

Can you clarify which one is longitude and which one is latitude? :-)

It's one thing that really gets me about CITE tests too, if they wanted to make 
order obvious without further specs, the
coordinates of test data could have had some values greater than 90 ;-)

Cheers
Andrea

--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i 
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo 
è consentito esclusivamente al destinatario del messaggio, per le finalità 
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne 
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di 
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro 
sistema. Conservare il messaggio stesso, divulgarlo anche in parte, 
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, 
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for the 
attention and use of the named addressee(s) and may be confidential or 
proprietary in nature or covered by the provisions of privacy act (Legislative 
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in 
accord with its purpose, any disclosure, reproduction, copying, distribution, 
or either dissemination, either whole or partial, is strictly forbidden except 
previous formal approval of the named addressee(s). If you are not the intended 
recipient, please contact immediately the sender by telephone, fax or e-mail 
and delete the information in this message that has been received in error. The 
sender does not give any warranty or accept liability as the content, accuracy 
or completeness of sent messages and accepts no responsibility  for changes 
made after they were sent or for other risks which arise as a result of e-mail 
transmission, viruses, etc.

---
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GML lat-lon coverages and interoperability

[Rahkonen Jukka (MML)]

Great, first hit from Andrea. I am awaiting if Even strikes back. His creation 
(GDAL JP2OpenJPEG driver) writes out EPSG:4326 coverage into GML JP2 this way, 
without having gridFunction.

http://www.opengis.net/gml;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="http://www.opengis.net/gml 
http://schemas.opengis.net/gml/3.1.1/profiles/gmlJP2Profile/1.0.0/gmlJP2Profile.xsd;>
  

  61.7282236395245 24.7248340978191
  61.7838827490233 24.8422668729657

  
  

  

  

  

  0 0
  15517 7354

  
  x
  y
  

  61.7838789652633 24.7248378815791

  
  0 
7.56751998624388e-006
  -7.56751998624388e-006 0

  
  

  
  gmljp2://codestream/0
  Record Interleaved

  

  

  


-Jukka-

Lähettäjä: andrea.a...@gmail.com [mailto:andrea.a...@gmail.com] Puolesta Andrea 
Aime
Lähetetty: 3. huhtikuuta 2017 15:58
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Kopio: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
<gdal-dev@lists.osgeo.org>; geoserver-de...@lists.sourceforge.net
Aihe: Re: [gdal-dev] GML lat-lon coverages and interoperability

On Mon, Apr 3, 2017 at 2:32 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
With orthophoto coverages I believe that all cases could be handled with the 
default gridFunctions: startPoint at gml:Low, axisOrder +1 +2, sequence rule 
“Linear”. As far as I have understood GDAL trusts that this is also what the 
other partners do. However, it seems that GeoServer developers have decided to 
handle the latitude-longitude axis flip in WCS 2.0.1 by changing the order of 
grid axis with a function instead of writing the offsetVectors to suit with the 
default +1 +2 axis order.



Linear
0 0



Hi Jukka,
the decision was not arbitrary, and imho, necessary.

Rasters pixels are normally represented as east/north, but OGC forces us to 
describe geographic data in lat/lon order.
So how do you tell the client, "look, I'm telling you that the axis order is 
north/east, but really, I'm giving you back data in east/north order instead"?

Cheers
Andrea

--
==
GeoServer Professional Services from the experts! Visit
http://goo.gl/it488V for more information.
==

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A
55054  Massarosa (LU)
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39  339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


AVVERTENZE AI SENSI DEL D.Lgs. 196/2003

Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i 
file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo 
è consentito esclusivamente al destinatario del messaggio, per le finalità 
indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne 
il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di 
procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro 
sistema. Conservare il messaggio stesso, divulgarlo anche in parte, 
distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, 
costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.



The information in this message and/or attachments, is intended solely for the 
attention and use of the named addressee(s) and may be confidential or 
proprietary in nature or covered by the provisions of privacy act (Legislative 
Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in 
accord with its purpose, any disclosure, reproduction, copying, distribution, 
or either dissemination, either whole or partial, is strictly forbidden except 
previous formal approval of the named addressee(s). If you are not the intended 
recipient, please contact immediately the sender by telephone, fax or e-mail 
and delete the information in this message that has been received in error. The 
sender does not give any warranty or accept liability as the content, accuracy 
or completeness of sent messages and accepts no responsibility  for changes 
made after they were sent or for other risks which arise as a result of e-mail 
transmission, viruses, etc.

---
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GML lat-lon coverages and interoperability

[Rahkonen Jukka (MML)]

Cross posting intentionally.

TL;DR

I wonder if all projects under the OSGeo umbrella, starting from GDAL and 
Geoserver, could make a common agreement on not to use GridFunctions in GML 
coverages if they are not strictly needed. The other alternative would be to 
make all projects to support GML gridFunctions, and do it right.

Long story:

I have been trying to understand how GML coverages are used in JPEG 2000 and in 
WCS services.  So far I have understood that coverages can be very complex and 
GMLCOV and next generation CIS specifications are complex so that even the most 
nasty coverage types can be modelled. What seems to be missing from the 
specifications is a simple and unified way to model simple coverages, like 
orthophotos.
Not surprisingly my pain started from latitude-longitude or Northing-Easting 
axis order of some coordinate systems. The GDAL source code in 
https://trac.osgeo.org/gdal/browser/trunk/gdal/gcore/gdaljp2metadata.cpp is 
prepared to handle georeferencign by looking at

-  grid origin

-  offset vectors

-  EPSG database for handling the coordinate axis order in GML

At least in this place GDAL does not seem to be prepared in seeing 
gridFunctions which are defined in GML 3.2.1. With GridFunction it is possible 
to slide the starting point of the coverage with "startPoint", change the order 
and direction of grid axes with "axisOrder" and define weird sequences for data 
points "Linear"/"Boustrophedonic"/"Cantor-diagonal"/"Spiral"/"Morton"/"Hilbert"

With orthophoto coverages I believe that all cases could be handled with the 
default gridFunctions: startPoint at gml:Low, axisOrder +1 +2, sequence rule 
"Linear". As far as I have understood GDAL trusts that this is also what the 
other partners do. However, it seems that GeoServer developers have decided to 
handle the latitude-longitude axis flip in WCS 2.0.1 by changing the order of 
grid axis with a function instead of writing the offsetVectors to suit with the 
default +1 +2 axis order.



Linear
0 0



-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogrinfo does not open any more raster-only geopackage

[Rahkonen Jukka (MML)]

Even Rouault wrote:
Re: [gdal-dev] Ogrinfo does not open any more raster-only geopackage


> Hi Jukka,

> I could only reproduce with a .gpkg that didn't have a gpkg_geometry_columns 
> table. Which is valid, but not what GDAL creates, so I guess this comes from 
> another software ?


That GeoPackage was generated by MapProxy 
https://mapproxy.org/docs/nightly/caches.html#geopackage

Should I encourage MapProxy developers to create also an empty 
gpkg_geometry_columns table in the future for enhanced interoperability?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Ogrinfo does not open any more raster-only geopackage

[Rahkonen Jukka (MML)]

Hi,

Recent GDAL 2.2-dev versions can't open geopackage that contains only raster 
data.
ogrinfo -ro -sql "PRAGMA journal_mode" raster.gpkg

I found a 2.2-dev version from 2016-12-09 and that does run my PRAGMA query.  
Could it be possible to get this possibility back?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GeoPackege, views and spatial index

[Rahkonen Jukka (MML)]

Hi,

I can create a spatial view into GeoPackage database by following the document 
http://www.geopackage.org/modeling_guidelines.html and I can open such a view 
with QGIS. But have I understood right that there is no way in GeoPackage to 
link the R-Tree spatial index of the main table with the view? In Spatialite it 
is possible via the views_geometry_columns table but I can't find anything like 
that from the GeoPackage specification. I wonder if GDAL and QGIS could have 
support for same kind of linking metadata table than Spatialite.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] sqlite -> spatialite (rowid)

[Rahkonen Jukka (MML)]

Hi Sandro,

I have only one detail to add: the MAX+1 policy that is used when AUTOINCREMENT 
is not declared can lead to re-use of previously used ROWIDs long before 
reaching the max possible value for an INTEGER 64bit. User just needs to delete 
the record with the highest ROWID. I trust that triggers in SpatiaLite behave 
right but I just want to emphasize for other readers that re-using ROWIDs is 
not really "an highly unrealistic condition" and it must be considered if ROWID 
is used as a foreign key.  Re-use happens always and inevidently for example if 
table is truncated. Test with this SQL and table foo2 will have ROWID=1 
(calculated as 0+1=1) while table foo3 has  ROWID=2 because that value is taken 
from a sequence by autoincrement.

create table foo2 (id INTEGER PRIMARY KEY,bar INTEGER);
create table foo3 (id INTEGER PRIMARY KEY AUTOINCREMENT,bar INTEGER);
insert into foo2 (bar) values (1);
insert into foo3 (bar) values (1);
delete from foo2;
delete from foo3;
insert into foo2 (bar) values (2);
insert into foo3 (bar) values (2);



-Jukka-



Lähettäjä: a.furi...@lqt.it <a.furi...@lqt.it>
Lähetetty: 7. maaliskuuta 2017 18:01
Vastaanottaja: Rahkonen Jukka (MML)
Kopio: Even Rouault; gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] sqlite -> spatialite (rowid)

On Tue, 7 Mar 2017 15:07:04 +0000, Rahkonen Jukka (MML) wrote:
> Hi,
>
> I have a slight feeling that there may be some trouble if attribute
> "ROWID" is INTEGER PRIMARY KEY but it does not AUTOINCREMENT and
> users
> do lot of deletes and inserts. Maybe not if the triggers in r-tree
> tables are clever enough. Just a slight feeling but perhaps you
> should
> test also cases where ROWID differs from OID and __ROWID__?
>

Hi Jukka,

we can safely exclude any possible dangerous side-effect.

in SQLite any INTEGER PRIMARY KEY definition will always
ensure that an appropriate value will be automatically
set if not explicitly provided.

what exactly does the AUTOINCREMENT keyword is just
slightly changing the mechanism adopted to automatically
assign PK values.

1. not declaring AUTOINCREMENT
SQLite will search for the biggest PK value, and
will adopt this value increased by 1 as the new
PK value.
if the max possible value for an INTEGER 64bit
has already been used (an highly unrealistic
condition) then SQLite will search if there
is some free "hole" in the PK sequence, and
will reuse the first unused value it can find
note: in this exceptional case it's not
exactly true that ROWIDs will be generated
in a regularly ascending order; sometimes
lower values could be possibly returned.

2. declaring AUTOINCREMENT
in this case the automatically generated PK
values can never return back; they can simply
growth until the maximum value supported
by an INTEGER 64bit is reached.
and at this point the auto-sequence will
definitely stop working.

bye Sandro


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] sqlite -> spatialite (rowid)

[Rahkonen Jukka (MML)]

Hi,

I have a slight feeling that there may be some trouble if attribute "ROWID" is 
INTEGER PRIMARY KEY but it does not AUTOINCREMENT and users do lot of deletes 
and inserts. Maybe not if the triggers in r-tree tables are clever enough. Just 
a slight feeling but perhaps you should test also cases where ROWID differs 
from OID and _ROWID_?

-Jukka-

Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 7. maaliskuuta 2017 16:51
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>; 
a.furi...@lqt.it
Aihe: Re: [gdal-dev] sqlite -> spatialite (rowid)


Arg, it seems my email has been truncated in text version. Retrying...



> Now I delete feature with rowid=3:

>



Let's see the state:



$ ogrinfo test.db -al

INFO: Open of `test.db'

using driver `SQLite' successful.



Layer name: test

Geometry: Unknown (any)

Feature Count: 2

Extent: (1.00, 2.00) - (5.00, 6.00)

Layer SRS WKT:

(unknown)

FID Column = rowid

Geometry Column = GEOMETRY

OGRFeature(test):1

POINT (1 2)



OGRFeature(test):5

POINT (5 6)



$ ogrinfo test.db -sql "select * from idx_test_GEOMETRY"

INFO: Open of `test.db'

using driver `SQLite' successful.



Layer name: SELECT

Geometry: None

Feature Count: 2

Layer SRS WKT:

(unknown)

pkid: Integer (0.0)

xmin: Real (0.0)

xmax: Real (0.0)

ymin: Real (0.0)

ymax: Real (0.0)

OGRFeature(SELECT):0

pkid (Integer) = 1

xmin (Real) = 1

xmax (Real) = 1

ymin (Real) = 2

ymax (Real) = 2



OGRFeature(SELECT):1

pkid (Integer) = 5

xmin (Real) = 5

xmax (Real) = 5

ymin (Real) = 6

ymax (Real) = 6





-> The spatial index has been properly maintained: the entry corresponding to 
the deleted feature has been deleted as well.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] WFS driver and GML application schema

[Rahkonen Jukka (MML)]

Hi,

What if a WFS server is returning GML that has complex schema, does GDAL know 
to open the response as GML application schema data automatically or with some 
open option?  I  ask because I can see "succeeds as GML" in the debug log:

HTTP: 
Fetch(https://opaskartta.turku.fi/TeklaOGCWeb/WFS.ashx?SERVICE=WFS=1.0.0=DescribeFeatureType=akaava:Kaavayksikko)
GML: Generating /vsimem/tempwfs_034A8DC0/file.gfs file, ignoring 
/vsimem/tmp_gml_xsd_03B73E80.xsd
GDAL: GDALOpen(/vsimem/tempwfs_034A8DC0/file.gml, 
this=03B73E80) succeeds as GML.


-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Reading SQLite in WAL mode as read-only

[Rahkonen Jukka (MML)]

Hi,

I saw this changeset https://trac.osgeo.org/gdal/changeset/37594 and started to 
think that how  GDAL is reading SQLite db that is in WAL journal mode from zip, 
tar, vsicurl or from non-writeble directory if it is documented to be generally 
not possible http://www.sqlite.org/draft/wal.html?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Testing of OGR generated GeoPackage datasets with other implementations

[Rahkonen Jukka (MML)]

Hi

Right, ne_10m_admin_0_countries_france_no_ogr_feature_count.gpkg opens with 
MapInfo 15.2.

-Jukka-

Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 2. maaliskuuta 2017 15:49
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] Testing of OGR generated GeoPackage datasets with other 
implementations


On jeudi 2 mars 2017 06:36:06 CET jratike80 wrote:

> Even Rouault-2 wrote

>

> > So I'd be interested if people could test

> > http://even.rouault.free.fr/geopackage/ne_10m_admin_0_countries_france.gpk

> > g

> > (or any other gpkg you may generate with today's ogr2ogr) with other

> > software package that

> > don't use OGR underneath (ArcGIS for example).

>

> Hi,

>

> MapInfo Pro 15.2 does not open your geopackage. It does open the geopackage

> which I cleaned with a bit older GDAL build as

>

> ogr2ogr -f gpkg clean.gpkg ne_10m_admin_0_countries_france.gpkg

>

> MapInfo did not give any error message, it just does not find any layers and

> quits. I could not find anything like error log from MapInfo directory. I

> will send a message to MapInfo support as well with a link to your

> geopackage.



Thanks for your feedback



Interesting, would you mind testing

http://even.rouault.free.fr/geopackage/ne_10m_admin_0_countries_france_no_ogr_feature_count.gpkg



which is the same with today's trunk but with -dsco 
ADD_OGR_FEATURE_COUNT_COLUMN=NO



If that works (and it should probably given your above testing), that means I 
should revise my design as

we want to be compatible with existing implementations.



I've though of an alternative that would use an auxiliary table instead of 
extending gpkg_contents, like a

gpkg_ogr_contents table. I've simulated it manually in

http://even.rouault.free.fr/geopackage/ne_10m_admin_0_countries_france_alternative.gpkg



Generated with :

cp ne_10m_admin_0_countries_france_no_ogr_feature_count.gpkg 
ne_10m_admin_0_countries_france_alternative.gpkg

ogrinfo ne_10m_admin_0_countries_france_alternative.gpkg -sql "CREATE TABLE 
gpkg_ogr_contents (table_name TEXT NOT NULL, feature_count INTEGER DEFAULT 
NULL)"

ogrinfo ne_10m_admin_0_countries_france_alternative.gpkg -sql "INSERT INTO 
gpkg_ogr_contents VALUES ('ne_10m_admin_0_countries', 1)"





Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] FW: Don't we have any ideas for GSoC 2017?

[Rahkonen Jukka (MML)]

Forwarding to the list because Jordan's trial  was not successful.
Jordan Bess wrote:
Re: [gdal-dev] Don't we have any ideas for GSoC 2017?


Vector tiles! Mbtiles or folder of pbf

It would be great if OGR supported creation of vector tiles in protocol buff 
format

Get Outlook for Android<https://aka.ms/ghei36>


From: gdal-dev 
<gdal-dev-boun...@lists.osgeo.org<mailto:gdal-dev-boun...@lists.osgeo.org>> on 
behalf of Casper Børgesen (CABO) <c...@niras.dk<mailto:c...@niras.dk>>
Sent: Tuesday, February 28, 2017 5:16:19 AM
To: Rahkonen Jukka (MML); 'gdal-dev@lists.osgeo.org' 
(gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>)
Subject: Re: [gdal-dev] Don't we have any ideas for GSoC 2017?

Hi,

I don't know if the idea is too simple, but implementing indexing on VRT files 
would be great to have. I thinks it's currently a feature request.


Regards, Casper


-Original Message-
From: gdal-dev [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Rahkonen 
Jukka (MML)
Sent: 28. februar 2017 09:07
To: 'gdal-dev@lists.osgeo.org' 
(gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>)
Subject: [gdal-dev] Don't we have any ideas for GSoC 2017?

Hi,

OSGeo is accepted as an organization for the Google Summer of Code 2017 but 
there are no GDAL related ideas on 
https://wiki.osgeo.org/wiki/Google_Summer_of_Code_2017_Ideas. Could we raise 
some of the old ideas or discover some brand new ones?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Don't we have any ideas for GSoC 2017?

[Rahkonen Jukka (MML)]

Hi,

OSGeo is accepted as an organization for the Google Summer of Code 2017 but 
there are no GDAL related ideas on 
https://wiki.osgeo.org/wiki/Google_Summer_of_Code_2017_Ideas. Could we raise 
some of the old ideas or discover some brand new ones?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Precision of classification in gdal_sieve.py

[Rahkonen Jukka (MML)]

Hi,

See this gis.stackexchange question about gdal_sieve 
http://gis.stackexchange.com/questions/227709/why-is-all-0-after-using-gdal-sieve-on-a-raster-layer.
By the documentation http://www.gdal.org/gdal_sieve.html it is not clear with 
which precision the pixels are classified to have the same value. Is it by 
integers or does sieve handle also floating point numbers?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] GMLAS does not resolve schema from WFS data

[Rahkonen Jukka (MML)]

Thanks,

I added CPl_CURL_VERBOSE but I do not quite understand what it tries to tell 
me. GDAL seems to reach the deegree WFS server but then "* no chunk, no close, 
no size. Assume close to signal end"
* Couldn't find host inspire-wfs.maanmittauslaitos.fi in the _netrc file; using 
defaults
* timeout on name lookup is not supported
* Hostname was NOT found in DNS cache
*   Trying 10.37.213.18...
* Connected to inspire-wfs.maanmittauslaitos.fi (10.37.213.18) port 443 (#0)
* libcurl is now using a weak random seed!
* SSL connection using TLSv1.2 / DHE-RSA-AES256-GCM-SHA384
* Server certificate:
*   subject: C=FI; ST=Finland; L=HELSINKI; 
O=Maanmittauslaitos; OU=Mitpa; CN=*.maanmittauslaitos.fi
*   start date: 2016-10-11 00:00:00 GMT
*   expire date: 2019-10-11 23:59:59 GMT
*   issuer: C=US; O=thawte, Inc.; CN=thawte SSL CA - G2
*   SSL certificate verify result: unable to get local 
issuer certificate (20), continuing anyway.
> GET 
> /inspire-wfs/cp?SERVICE=WFS=2.0.0=DescribeFeatureType=application%2Fgml%2Bxml%3B+version%3D3.2=cp:CadastralParcel=xmlns(cp,http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fcp%2F4.0)
>  HTTP/1.1

Host: inspire-wfs.maanmittauslaitos.fi
Accept: */*
Accept-Encoding: gzip
< HTTP/1.1 200 OK
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< Cache-Control: no-cache, no-store
< deegree-version: 3.3.18
< Content-Type: text/xml;charset=UTF-8
< Date: Wed, 18 Jan 2017 11:11:18 GMT

* no chunk, no close, no size. Assume close to signal end
<

* Closing connection 0
ERROR 3: Cannot resolve 
https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?SERVICE=WFS=2.0.0=DescribeFeatureType=application%2Fgml%2Bxml%3B+version%3D3.2=cp:CadastralParcel=xmlns(cp,http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fcp%2F4.0)

-Jukka-

Even Rouault wrote:
Re: [gdal-dev] GMLAS does not resolve schema from WFS data


On mercredi 18 janvier 2017 10:52:28 CET Rahkonen Jukka (MML) wrote:

> Hi,

>

> I downloaded one CadastralParcel from an INSPIRE service with curl

>

> curl -k -o inspire-cp.gml

> "https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?service=WFS

> n=2.0.0=GetFeature=cp:CadastralParcel=1"

>

> Then I tried to check what I have with ogrinfo but it fails

>

> ogrinfo GMLAS:inspire-cp.gml

>

> ERROR 3: Cannot resolve

> https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?

> SERVICE=WFS=2.0.0=DescribeFeatureType=applicat

> ion%2

> Fgml%2Bxml%3B+version%3D3.2=cp:CadastralParcel=xmlns(cp

> ,http %3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fcp%2F4.0)

>

> However, the DescribeFeatureType is successful when I copy-paste it into

> browser. I wonder what goes wrong. I had to use "-k" with curl to bypass a

> problem with certificate and therefore I tried also to set unsafe SSL for

> GDAL with "set GDAL_HTTP_UNSAFESSL=YES" but that did not change anything.



I don't reproduce this issue on Ubuntu 16.04. You can try adding --config 
CPL_CURL_VERBOSE YES to get more low level details about what goes wrong



But I get other errors :



$ ogrinfo gmlas:inspire-cp.gml

ERROR 1: 
/vsicurl_streaming/https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?SERVICE=WFS=2.0.0=DescribeFeatureType=application%2Fgml%2Bxml%3B+version%3D3.2=xmlns%28http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fau%2F4.0%29:11:73
 namespace 'http://inspire.ec.europa.eu/schemas/base/3.3' is referenced without 
import declaration

ERROR 1: 
/vsicurl_streaming/https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?SERVICE=WFS=2.0.0=DescribeFeatureType=application%2Fgml%2Bxml%3B+version%3D3.2=xmlns%28http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fau%2F4.0%29:114:73
 namespace 'http://inspire.ec.europa.eu/schemas/base/3.3' is referenced without 
import declaration

[...]



The schema at 
https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?SERVICE=WFS=2.0.0=DescribeFeatureType=application%2Fgml%2Bxml%3B+version%3D3.2=xmlns(http%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fau%2F4.0)
 indeed references namespaces without importing them (except GML). It indeed 
looks invalid to me.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GMLAS does not resolve schema from WFS data

[Rahkonen Jukka (MML)]

Hi,

I downloaded one CadastralParcel from an INSPIRE service with curl

curl -k -o inspire-cp.gml 
"https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?service=WFS=2.0.0=GetFeature=cp:CadastralParcel=1;

Then I tried to check what I have with ogrinfo but it fails

ogrinfo GMLAS:inspire-cp.gml

ERROR 3: Cannot resolve https://inspire-wfs.maanmittauslaitos.fi/inspire-wfs/cp?
SERVICE=WFS=2.0.0=DescribeFeatureType=application%2
Fgml%2Bxml%3B+version%3D3.2=cp:CadastralParcel=xmlns(cp,http
%3A%2F%2Finspire.ec.europa.eu%2Fschemas%2Fcp%2F4.0)

However, the DescribeFeatureType is successful when I copy-paste it into 
browser. I wonder what goes wrong. I had to use "-k" with curl to bypass a 
problem with certificate and therefore I tried also to set unsafe SSL for GDAL 
with "set GDAL_HTTP_UNSAFESSL=YES" but that did not change anything.

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Wrong interpretation of DIMENSION features in GDAL DXF driver

[Rahkonen Jukka (MML)]

Sorry, Nabble did not include my ogr2ogr command to the mailing list mail, 
reposting.

You can do the join as suggested in 
http://www.geopackage.org/modeling_guidelines.html by converting the first 
geometry with all the attributes into GeoPackage as you have done, and then 
adding  another layer with a command like

ogr2ogr -append -update -f gpkg -sql "SELECT second_geometry AS geometry, id AS 
id from table_with_two_geometries"
my_geopackage.gpkg -nln table_with_second_geometry_to_be_joined PG:.

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Creating JPEG compressed external overviews fails

[Rahkonen Jukka (MML)]

Hi,

This has worked for me and still works with GDAL version GDAL 2.0dev, released 
2011/12/29
gdaladdo -r average -ro --config COMPRESS_OVERVIEW JPEG --config 
PHOTOMETRIC_OVERVIEW YCBCR --config INTERLEAVE_OVERVIEW PIXEL input.tif  2 4 8 
16 32 64

However, GDAL 2.2.0dev, released 2016/99/99 fails:

0...10...20...30...40...50...60...70...80...90.ERROR 1: JPEGLib:Huffman table 
0x00 was not defined
ERROR 1: TIFFReadEncodedTile() failed.
ERROR 1: IReadBlock failed at X offset 0, Y offset 0
ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 0
..100 - done.
Overview building failed.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Call for discussion on RFC 64: Triangle, Polyhedral surface and TIN

[Rahkonen Jukka (MML)]

Hi,

Perhaps some problem with markup, the WKT dows show in Nabble 
http://osgeo-org.1560.x6.nabble.com/gdal-dev-Call-for-discussion-on-RFC-64-Triangle-Polyhedral-surface-and-TIN-tc5298840.html

MULTIPOLYGON ((( 553 352, 327 363, 442 486, 553 352 )), (( 327 363, 553 352, 
422 186, 327 363 )))

-Jukka-



Even Rouault wrote:
Re: [gdal-dev] Call for discussion on RFC 64: Triangle, Polyhedral surface and 
TIN


On mercredi 7 décembre 2016 02:01:54 CET jratike80 wrote:

> Even Rouault-2 wrote

>

> > Hi,

> >

> > This is a call to discuss the proposed RFC 64: Triangle, Polyhedral

> > surface and TIN

> >

> > https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin

> >

> > ~

>

> I am wondering this part:

>

> "GEOS methods are still used in some cases, but with the following

> limitations - a Triangle is converted to a Polygon with one exterior ring;

> Polyhedral Surfaces and Triangulated Surfaces are converted to

> MultiPolygons?. (each Triangle in a Triangulated Surface is converted to a

> Polygon as described previously)"

>

> Does it mean presenting a TIN with two triangles as something like

>



Something missing here, but I guess you meant as a multipolygon made of 2 
polygons ?



>

>

> As far as I know such multipolygon is not valid accorfing to the Simple

> feature specification where "boundaries and may touch at only a finite

> number of Points". Valid presentation would probably be to wrap polygons

> into GeometryCollection. Is GEOS used in a relaxed way so that shared parts

> of boundaries in MultiPolygons do not throw an error?



That's a very good remark. Indeed for GEOS purposes, a MultiPolygon with 
Polygons that have shared boundaries is invalid. I've changed that to a 
GeometryCollection. But anyway, TIN & PoyhedralSurface in GEOS world are 
degraded cases where behaviour will always be suboptimal.



Thanks for the feedback.



Even



--

Spatialys - Geospatial professional services

http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How does GDAL OSM driver deal with closed ways?

[Rahkonen Jukka (MML)]

Djordje Spasic wrote:



I will definitively read through the links you gave me Jukka. Thank you.

Until then, do you mind if I ask just for a tiny clarification?


> The default rendering style file has such a list
https://github.com/openstreetmap/osm2pgsql/blob/master/default.style
Is this the part of that file that I need to take a look at:
https://github.com/openstreetmap/osm2pgsql/blob/master/default.style#L83-L152

Yes.

> and you can find it also from the osmconf.ini file
But osmconf.ini file only defines the polygons keys, not linear ones?
This is the line in osmconf.ini which defines polygon keys:
closed_ways_are_polygons=


Those closed ways which are not defined to be polygons are considered to be 
linear rings and they go to the lines layer.

-Jukka-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] osm_id vs osm_way_id?

[Rahkonen Jukka (MML)]

Hi,

ogr2ogr creates these ids:


-  for point data node_id becomes ”osm_id”

-  for linestrings way_id becomes “osm_id”

-  for multilinestrings relation_id becomes “osm_id”

-  for other relations relation_id becomes “osm_id”

-  for polygons which come from relations relation_id becomes “osm_id” 
and osm_way_id is null

-  for polygons which come from ways the way_id becomes osm_way_id and 
osm_id is null

GDAL cannot write ids of the way polygons and relation polygons into the same 
ID field because there would be duplicate ids (same numbers coming from way_id 
and relation_id).

Historically GDAL wrote way polygons into layer named ”polygons” and relation 
polygons into ”multipolygons” and both had just “osm_id”. It was perhaps more 
clear when it comes to ids but more fuzzy for users to find polygons from two 
different tables. Remember that in the OSM model a way polygon must be 
converted into relation polygon if user wants to make a hole into the polygon.

If I understand right you consider that instead of having now in the 
multipolygon layer relation_id as osm_id and way_id as osm_way_id it would be 
better to have way_id as osm_id and relation_id as osm_relation_id.  Perhaps 
yes, but for my mind the difference is not huge.

I guess that when the developer of the OSM driver stopped using the “polygons” 
layer they made the driver to write also the way polygons into the existing 
“multipolygons” layer. That layer already had osm_id so a new field osm_way_id 
was simply added into that layer.

If you can prove that there are some general use cases where renaming the id 
fields of the multipolygons layer would be a big improvement, please file a 
feature request.

-Jukka Rahkonen-


Djordje Spasic wrote:
Re: [gdal-dev] osm_id vs osm_way_id?

Hi Jukka,


As always thank you for the useful share of knowledge!!

> IDs are unique only for nodes/ways/relations but not across the elements. I
do not think that renaming osm_id into relation_id (and probably osm_id of
points into node_id, or?) and osm_way_id into way_id to follow the native
model would make things very much more clear.

I was only questioning why can't "osm_way_id" be renamed to "osm_relation_id". 
The "osm_id" would remain the same.

Of course I am not a developer, I was just thinking out loud, and hopping that 
somebody might explain to me, why is there the "osm_way_id" key.
I could totally be wrong.



All three examples you've sent me:

http://www.openstreetmap.org/api/0.6/node/389960161
http://www.openstreetmap.org/api/0.6/way/34028339 
http://www.openstreetmap.org/api/0.6/relation/125519

Show the id of three different elements: An id of a node, of a way, and of a 
relation.

What makes it confusing (at least to me) is why did developers of the GDAL OSM 
driver decided to use a separate key(field) to depict the closed ways?

I understand that and ESRI shapefile can not differentiate between a closed way 
and a relation (for example: a relation consisted of two closed ways). So they 
had to "group" both closed ways and relation into a single shapefile geometry 
type (multi)polygons/polylines. But still, if they wanted to differentiate 
between these two elements, it's more logical (at least to me) that a relation 
should have gotten a separate key(field), instead of a close way. A relation is 
higher in the hierarchy of OSM elements, and can contain different elements (a 
node and a way, three ways, fours nodes...).

I am just thinking out loud, nothing more.
I can definitively be wrong, and I am eager to learn more about the OSM in 
general.





On Thursday, December 1, 2016 11:03 PM, jratike80 
>
 wrote:

georges wrote
> Hello,
> I am converting an .osm file to shapefile, and there is one thing that I
> do not understand:
>
> What is the difference between: osm_id and osm_way_id fields (keys)?
>
> They are defined in osmconf.ini file, but only for [multipolygons].
> Here is an explanation from the osmconf.ini file:
>
> # note: for multipolygons, osm_id=yes instanciates a osm_id field for the
> id of relations
> # and a osm_way_id field for the id of closed ways. Both fields are
> exclusively set.
> Does that mean that osm_way_id will be used to represent the osm id number
> of a closed way, while osm_id will only be used for to represent the osm
> id number of relations?
>
> It's a bit confusing concept.
>
> What makes the closed ways so special so that they need to have their own
> special field (osm_way_id)?
> Why wasn't there a separate field for relations instead: like:
> osm_relation_id, while closed ways would use the osm_id.
>
> Thank you for the reply.

Indeed, OSM data model is a bit confusing concept. You can read about it
from https://wiki.openstreetmap.org/wiki/Elements. Simple, small polygons
are usually modeled as a one continuous, closed ring. That makes one 

[gdal-dev] Append new data to GeoPackege raster table as overview

[Rahkonen Jukka (MML)]

Hi,

Let's say that I have these raster maps with 2048 m and 1024 m pixel sizes
http://kartat.kapsi.fi/files/taustakarttasarja_jhs180/taustakartta_8m/2048m/etrs89/png/Taustakartta_8milj.png
http://kartat.kapsi.fi/files/taustakarttasarja_jhs180/taustakartta_8m/2048m/etrs89/png/Taustakartta_8milj.pgw

http://kartat.kapsi.fi/files/taustakarttasarja_jhs180/taustakartta_4m/1024m/etrs89/png/Taustakartta_4milj.png
http://kartat.kapsi.fi/files/taustakarttasarja_jhs180/taustakartta_4m/1024m/etrs89/png/Taustakartta_4milj.pgw

Converting the 8milj.png into GeoPackage raster table is easy

gdal_translate -of gpkg -a_srs epsg:3067 Taustakartta_8milj.png overview.gpkg

Image goes to zoom level 2 as reported by gdalinfo:
Metadata:
  IDENTIFIER=overview
  ZOOM_LEVEL=2

But now the hard part: I would like to append the second image 4milj.png to the 
same layer "overview" as ZOOM_LEVEL=3 because that  map is designed for that 
resolution. Is it possible to do that by using some parameters of the driver 
http://www.gdal.org/drv_geopackage_raster.html or with some clever trick?

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How does WMTS driver detect DataWindow?

[Rahkonen Jukka (MML)]

Hi,


I got to understand why WMTS service has so large WGS84 bounding box in 
GetCapabilities.  It is because the native EPSG:3067 bounding box is first 
converted into WGS84 by using a densified polygon geometry. The 
WGS84BoundingBox is then the envelope of this warped polygon. Because of 
warping the rectangle gets wider towards the poles. This is logical and the 
result does not look so bad in this image

https://gist.github.com/anonymous/2445ea91a8ac7b15bf39

However, if the bounding box goes closer to -90°S or 90°N the distortion gets 
bigger.  In our case the maximum latitude is at 83°N and the result it quite 
funny

http://latuviitta.org/downloads/WGS84BoundingBox.png


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How does WMTS driver detect DataWindow?

[Rahkonen Jukka (MML)]

With a little help GDAL picks up correct extent. Removing the erroneous 
WGS84BoundingBox hint from GetCapabilities made it.

I consider that there is a bug in the WMTS server which is advertising invalid 
bounding box but I don't really know GDAL should do.
I am not good at reading the code from 
https://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/wmts/wmtsdataset.cpp but I 
do see that there are comments about making a clip with the implied bounding 
box of the most precise tile matrix, taking into account the TileMatrix limits 
if they appear in GetCapabilities

The logic seems to be now


1)  Use the BoundingBox of the layer expressed in the SRS of the tile 
matrix set if that exists (row 1394)

2)  Otherwise use the BoundingBox of the layer expressed in some other SRS 
(for example WGS84) if that exists (row 1506)

3)  Otherwise use the BoundingBox of the tile matrix set if that exists 
(row 1550)

4)  Otherwise compute the bounding box from the definition of the most 
precise tile matrix (1558)

The comments about clipping are on rows 1572 and 1587.

So if the meaning was to take the advertised bounding box and clip it with the 
box that is computed from the tile matrix set and its limits, it seems to fail 
with our data

In our case the layer has this faulty

-6.38153862706 55.6179644952
60.3815386271 75.5825702342


and our most precise TM is

13
3571.42857143
-548576.0 8388608.0
256
256
8192
8192


With WGS84BoundingBox GDAL finds

-1553364.00402
8388608
2553364.01241
6681097.99317
  

and without WGS84BoundingBox the correct

-548576
8388608
1548576
6291456
  

-Jukka Rahkonen-


Lähettäjä: Even Rouault [mailto:even.roua...@spatialys.com]
Lähetetty: 29. marraskuuta 2016 0:07
Vastaanottaja: gdal-dev@lists.osgeo.org
Kopio: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Aihe: Re: [gdal-dev] How does WMTS driver detect DataWindow?


On lundi 28 novembre 2016 15:33:31 CET Rahkonen Jukka (MML) wrote:

> Hi,

>

> While playing with gdal_translate and WMST my colleague found that GDAL was

> picking a wrong DataWindow with

>

>

> gdal_translate

> "WMTS:http://xx.yy.fi/foo/service?REQUEST=GetCapabilities=WMTS,laye

> r=bar" bar.xml -of WMTS

>

>

>

> After some thinking we started to believe that the reason may be in the

> wrong WGS84BoundingBox that this WMTS service puts into GetCapabilities

>

> 

>

> -6.38153862706 55.6179644952

>

> 60.3815386271 75.5825702342

>

> 

>

>

>

> I checked with gdaltransform that longitude -6.38153862706 converts into the

> value that was used as the DataWindow UpperLeftX value:

>

> -1553364.00402

>

>

>

> However, the TopLeftCorner of the TileMatrix as it is defined in the

> GetCapabilities has a much smaller X value: -548576.00

>

>

>

> 

>

> 0

>

> 29257142.85714285820722579956

>

> -548576.00 8388608.00

>

> 256

>

> 256

>

> 1

>

> 1

>

> 

>

>

>

> Probably because of the wrong UpperLeftX value was gdal_translate seems to

> think that the leftmost tile in this layer has a negative TileX value

>

> -3924

>

>

>

> By reading the WMTS standard the negative index values are not valid in

> WMTS:

>

> TileCol: Column index of tile matrix

>

> Non negative integer type value between 0 and MatrixWidth-1 of this tile

> matrix defined in the ServiceMetadata document

>

>

>

> BoundingBoxes does not seem to be mandatory in WMTS because they can be

> computed from other, mandatory parameters as it is written in the notes of

> Table 14 - Parts of TileMatrix data structure

>

>

>

> "NOTE 7 The bounding box of a tile matrix is not supplied explicitly because

> it can be calculated from topLeftCorner, tileWidth, tileHeight and

> scaleDenominator."

>

>

>

> It feels that gdal_translate should never write a DataWindow that is bigger

> than the extents of the TileMatrix. Perhaps BoundingBoxes, if they exist,

> could be used for limiting the maximum extent, like the TileMatrixSetLimits

> if they are found from the GetCapabilities. Or is there something special

> in WMTS that makes it much more comfortable for GDAL to trust in

> BoundingBoxes? Like the fact that different tile levels in WMTS may have

> different bounding boxes?



Hi Jukka,



I don't recall exactly the details but there are several strategies in place in 
the WMTS driver to determine the bounding box of interest based on what you 
quotes: WGS84BoundingBox, TileMatrix, TileMatrixSetLimits. The 
 is likely used as you noticed as the most prioritary to 
indicate the real zone of interest in case of using global tiling schemes. But 
there's probably a lack of checks to va

[gdal-dev] ElasticSearch and native geometry

[Rahkonen Jukka (MML)]

Hi,

What if somebody would like to use ElasticSearch for queries but still somehow 
get the native geometries included in the output?  The geometry must be 
re-projected into EPSG:4326 for creating either geo_point or geo_shape, but can 
anybody suggest a clever way for saving the native geometry? It is of course 
possible to save it into a text field for example as WKT of JSON but could it 
be possible to keep the original geometry in "_source" and just create the 
geo_point or geo_shape on-the-fly or the index?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Question about how GDAL makes an ElasticSearch index

[Rahkonen Jukka (MML)]

Hi,

It seems that all examples about how to query ElasticSearch with GDAL in 
http://www.gdal.org/drv_elasticsearch.html are failing with ElasticSearch 
versions 2.x.
Reason is the breaking change "A field can no longer be referenced using its 
short name. Instead, the full path to the field is required" as documented in
https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking_20_mapping_changes.html

I am planning to update the documents but I would like to be sure that I have 
understood it right that:

-  GDAL creates an ElasticSearch index with the name = OGR layer name

-  That ES index has always _type named FeatureCollection

-  attributes of the source data are always within "properties"

Therefore this
ogrinfo -ro ES: my_type -where '{ "filter": { "term": { "EAS_ID": 168 } } }'

must be used in this way with ES 2.x
ogrinfo -ro ES: my_type -where '{ "filter": { "term": { "properties.EAS_ID": 
168 } } }'

similarly the Windows escaping example
C:\GDAL_on_Windows>ogrinfo ES: my_type -where "{\"query\": { \"match\": { 
\"properties.NAME\": \"Helsinki\" } } }"

Do you agree?

-Jukka Rahkonen-



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GDAL and ElasticSearch 5.0

[Rahkonen Jukka (MML)]

Hi,

I tried to test new ElasticSearch version with no success and I wonder what is 
wrong. My command and debug data follows.


ogr2ogr -f "ElasticSearch" http://localhost:9200
-s_srs epsg:3067 -t_srs epsg:4326 test_es50.shp --debug on

GDAL: GDALOpen(test_es50.shp, this=02454920) succeeds as ESRI Shapefile.
HTTP: Fetch(http://localhost:9200)
HTTP: These HTTP headers were set: Accept: text/plain, application/json
GeoJSON: No valid GeoJSON data found in source 'http://localhost:9200'
GDAL: GDALDriver::Create(ElasticSearch,http://localhost:9200,0,0,0,Unknown,0
000)
HTTP: Fetch(http://localhost:9200/_stats)
HTTP: Fetch(http://localhost:9200/test_es50)
HTTP: Fetch(http://localhost:9200/test_es50)
HTTP: These POSTFIELDS were sent:
HTTP: These HTTP headers were set: Content-Type: application/x-javascript;
charset=UTF-8
ERROR 1: HTTP error code : 400
ERROR 1: No handler found for uri [/test_es50] and method [POST]
ERROR 1: Terminating translation prematurely after failed
translation of layer test_es50 (use -skipfailures to skip errors)
GDAL: GDALClose(http://localhost:9200, this=024545E0)
GDAL: GDALClose(test_es50.shp, this=02454920)

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Why are fields required in osmconf.ini file?

[Rahkonen Jukka (MML)]

You need to have both: include all_tags in osmconf.ini and define with the 
ogr2ogr command that field “all_tags” in PostGIS should be initialized as 
hstore. If you do not define column type all_tags will probably become text 
type column. Make a test and you will see it.

-Jukka Rahkonen-

Djordje Spasic wrote:

Re: [gdal-dev] Why are fields required in osmconf.ini file?

Thank you for the useful reply Jukka!

The command you gave me:

ogr2ogr -f PostgreSQL "PG:dbname=osm" test.pbf -lco
COLUMN_TYPES=all_tags=hstore
does the same thing as "all_tags=yes" in the osmconf.ini file:

# uncomment to create "all_tags" field. "all_tags" and "other_tags" are 
exclusive
all_tags=yes
?

If not, then what is the difference between these two?

On Friday, November 18, 2016 8:55 AM, jratike80 
>
 wrote:

georges wrote

> Thank you for the quick reply Even!
>
> So this required predefined set of fields is a necessary step for any .osm
> file conversion with ogr2ogr.exe?
> Not only conversion from .osm to .shp?
> But also to .pbf, geojson etc. ?


Only exception and highly recommended if you want to get all the tags is to
use PostGIS and "all_tags" which puts everything into hstore:

ogr2ogr -f PostgreSQL "PG:dbname=osm" test.pbf -lco
COLUMN_TYPES=all_tags=hstore

-Jukka Rahkonen-



--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/gdal-dev-Why-are-fields-required-in-osmconf-ini-file-tp5296342p5296397.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr -dim and Oracle

[Rahkonen Jukka (MML)]

Hi,

Probably SDO_COORD_REF_SYS is what to use nowadays even I guess that it points 
to MDSYS tables. Looks like something has changed with Oracle 10.2 and I 
believe that it is very hard to use so old Oracle version without paying much 
extra.
https://docs.oracle.com/database/121/SPATL/GUID-7DEF5C91-777C-4023-AE2E-E2E29F5213D2.htm#SPATL652
 

-Jukka Rahkonen- 


Even Rouault wrote:

Le jeudi 17 novembre 2016 14:19:10, Ivan Lucena a écrit :
> Hi Even,
> 
>> Did you try your change?

>Yes

>> 
> There has been a lot of changes in both sides over the years that are 
> not reflected on the OCI driver code.
> 
> That hard coded SRID conversion should be deleted since EPSG is fully 
> supported now.

Do you know which since which version of OCI this "now" apply ?

With 11.2.0, I see things like :

OGRFeature(select * from MDSYS.CS_SRS):6
  CS_NAME (String) = NAD83(CSRS98) / UTM zone 17N
  SRID (Integer64) = 2150
  AUTH_SRID (Integer64) = 2150
  AUTH_NAME (String) = Geomatics Canada.
  WKTEXT (String) = PROJCS["NAD83(CSRS98) / UTM zone 17N", GEOGCS [ 
"NAD83(CSRS98)", DATUM ["NAD83 Canadian Spatial Reference System (EPSG ID 
6140)", SPHEROID ["GRS 1980 (EPSG ID 7019)", 6378137.0, 298.257222101]], PRIMEM 
[ "Greenwich", 0.00 ], UNIT ["degree minute second hemisphere", 
0.0174532925199433]], PROJECTION ["Transverse Mercator"], PARAMETER 
["Latitude_Of_Origin", 0.0], PARAMETER ["Central_Meridian", -81.0], PARAMETER 
["Scale_Factor", 0.9996], PARAMETER ["False_Easting", 50.0], PARAMETER 
["False_Northing", 0.0], UNIT ["Meter", 1.0]]
  WKTEXT3D (String) = PROJCS[
  "NAD83(CSRS98) / UTM zone 17N",
  GEOGCS["NAD83(CSRS98)",
DATUM["NAD83 Canadian Spatial Reference System",
  SPHEROID[
"GRS 1980",
6378137.0,
298.257222101,
AUTHORITY["EPSG", "7019"]],
  TOWGS84[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
  AUTHORITY["EPSG", "6140"]],
PRIMEM["Greenwich", 0.00, AUTHORITY["EPSG","8901"]],
UNIT["degree minute second hemisphere", 0.0174532925199433, 
AUTHORITY["EPSG", "9108"]],
AXIS["Lat", NORTH],
AXIS["Long", EAST],
AUTHORITY["EPSG", "4140"]],
  PROJECTION ["Transverse Mercator"],
  PARAMETER ["Latitude_Of_Origin", 0.0],
  PARAMETER ["Central_Meridian", -81.0],
  PARAMETER ["Scale_Factor", 0.9996],
  PARAMETER ["False_Easting", 50.0],
  PARAMETER ["False_Northing", 0.0],
  UNIT["metre", 1.0, AUTHORITY["EPSG", "9001"]],
  AXIS["E", EAST],
  AXIS["N", NORTH],
  AUTHORITY["EPSG", "2150"]]

What surprises me is the AUTH_NAME which is not EPSG (for other codes it is 
null). So it is not entirely clear how you would find the SRID if you can't 
search by AUTH_NAME = 'EPSG' AND AUTH_SRID = epsg_code. Skimming through the 
list, it seems that SRID = AUTH_SRID for EPSG codes however.

> 
> He georaster driver uses some PLSQL functions to try to match WKT to
> possible SRIDs. That could be added to the OCI. I think.

OK. Good to know

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Ogr2ogr -dim and Oracle

[Rahkonen Jukka (MML)]

Hi,

I would also check how ogr2ogr options –a_srs and –t_srs  behave or if the OCI 
driver option –lco SRID is still needed.

-Jukka-

Lähettäjä: Michael Smith [mailto:michael.smith.e...@gmail.com]
Lähetetty: 17. marraskuuta 2016 13:49
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>
Kopio: 'gdal-dev@lists.osgeo.org' (gdal-dev@lists.osgeo.org) 
<gdal-dev@lists.osgeo.org>
Aihe: Re: [gdal-dev] Ogr2ogr -dim and Oracle

Yes even the current version needs the layer creation option for dim. I would 
be +1 for this change (never really understood why it was this way). I think we 
are probably one of the biggest users of the oracle driver and I think this 
would be a good improvement.

Sent from my iPhone

On Nov 17, 2016, at 2:06 AM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:
Hi,

I noticed that Even was editing the ogr2ogr help about the –dim parameter and  
remembered that I think that it does not have an effect on the OCI driver. For 
forcing a new Oracle table into 2D you must use the OCI drivers own layer 
creation option “DIM” instead http://www.gdal.org/drv_oci.html. I am not sure 
if that happens also with the current GDAL version.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Ogr2ogr -dim and Oracle

[Rahkonen Jukka (MML)]

Hi,

I noticed that Even was editing the ogr2ogr help about the -dim parameter and  
remembered that I think that it does not have an effect on the OCI driver. For 
forcing a new Oracle table into 2D you must use the OCI drivers own layer 
creation option "DIM" instead http://www.gdal.org/drv_oci.html. I am not sure 
if that happens also with the current GDAL version.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

If your version and environment requires escaping, then you need to do

"select geometry, \"building\" as bldg,\"building:levels\" as 
bldg_levels,\”height\” as height from multipolygons ... "
It may work also without escaping as
"select geometry, "building" as bldg,"building:levels" as bldg_levels,height as 
height from multipolygons ... "

If you plan to follow the SQL route I really recommend you to read some SQL 
tutorials and exercise at home. I am sure that you will learn the basics soon 
and tutorials are always in the web while I am not. SQL is so close to English 
and that you can think that you are making a (comma separated) shopping list
“SELECT milk,bread,juice FROM Wal-Mart”.

-Jukka-




Lähettäjä: Djordje Spasic [mailto:issworld2...@yahoo.com]
Lähetetty: 14. marraskuuta 2016 17:21
Vastaanottaja: Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>;
 gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: VS: [gdal-dev] How to replace an osm key?

Thank you Jukka,

How should I edit the command you gave me:
"C:\gdal\ogr2ogr.exe" --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f 
"ESRI Shapefile" -dialect sqlite -sql "select geometry, \"building\" as bldg 
from multipolygons where \"building\" is not null limit 1" 
"C:\output_shp_files" "C:\input.osm"
so that "building", "building:levels", and "height" fields are included, not 
only the "building" field?

On Monday, November 14, 2016 2:10 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:

Hi,

If you enclose something between single quotes in SQL it becomes a string.
SELECT ’building’  -> selects literally the given *string* –building—
However,

SELECT "building" -> selects the *value* of *field* –building-- (or 
“identifier” as is it called in SQL)

See for example https://www.sqlite.org/lang_keywords.html

I tested on Windows with trunk version of GDAL that this syntax works:

ogrinfo -dialect sqlite -sql "select geometry, "building" as bldg
from multipolygons where "building" is not null limit 1" hamburg-latest.osm.pbf

By reading this old thread 
http://osgeo-org.1560.x6.nabble.com/gdal-dev-OGR-SQL-Excecuting-select-where-column-name-contains-quot-quot-td5006377.html
 some older GDAL versions may need some more tweaking and inner double quotes 
must be escaped by back slash as \”

ogrinfo -dialect sqlite -sql "select geometry, \"building\" as bldg
from multipolygons where \"building\" is not null limit 1" 
hamburg-latest.osm.pbf


-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

If you enclose something between single quotes in SQL it becomes a string.
SELECT ’building’  -> selects literally the given *string* –building—
However,

SELECT "building" -> selects the *value* of *field* –building-- (or 
“identifier” as is it called in SQL)

See for example https://www.sqlite.org/lang_keywords.html

I tested on Windows with trunk version of GDAL that this syntax works:

ogrinfo -dialect sqlite -sql "select geometry, "building" as bldg
from multipolygons where "building" is not null limit 1" hamburg-latest.osm.pbf

By reading this old thread 
http://osgeo-org.1560.x6.nabble.com/gdal-dev-OGR-SQL-Excecuting-select-where-column-name-contains-quot-quot-td5006377.html
 some older GDAL versions may need some more tweaking and inner double quotes 
must be escaped by back slash as \”

ogrinfo -dialect sqlite -sql "select geometry, \"building\" as bldg
from multipolygons where \"building\" is not null limit 1" 
hamburg-latest.osm.pbf


-Jukka Rahkonen-

Djordje Spasic wrote:

Thank you for the reply Jukka.

Neither "building:levels" nor "height" fields were located in osmconf.ini under 
the "[multipolygons]".
I have just added them as you suggested, and now the previously mentioned 
command creates the .shp file without raising an error.
I tried the initial:
-dialect sqlite -sql "select geometry, building as bldg, 'building:levels' as 
bldg_level, height as height from multipolygons" 
"C:\ladybug\osm_files\barcelona_shp_files"
and
-dialect sqlite -sql "select geometry, building as bldg, \"building:levels\" as 
bldg_level, height as height from multipolygons" 
"C:\ladybug\osm_files\barcelona_shp_files"
And they both work.

The problem is that: once the .shp file is created, and I open it, all shapes 
have the "building:level" value for the "bldg_level" field. Here is a 
screenshot:
https://www.dropbox.com/s/nmlkkzr85w225qb/sql_choosing_keys.jpg?dl=0

Why is this happening?

Kind Regards,
Djordje

---

On Saturday, November 12, 2016 10:29 AM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:

Hi,

Related to your previous question, you can use SQL select only for fields which 
are included in osmconf.ini. Perhaps "height" is not. 'building:levels' may 
make a surprise for you, if ogr2ogr follows SQL specification it will select a 
fixed string. You should use double quotes instead and they may require 
escaping but I do not remember with which character. Start with easy test and 
more attributes with odd names one by one.

-Jukka-

Lähettäjä: Djordje Spasic<mailto:issworld2...@yahoo.com>
Lähetetty: ‎12.‎11.‎2016 3:37
Vastaanottaja: Rahkonen Jukka 
(MML)<mailto:jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: VS: [gdal-dev] How to replace an osm key?
Hi Jukka,
I have just tried to rename the fields with SQL. Here is the syntax:
"C:\gdal\ogr2ogr.exe" --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f 
"ESRI Shapefile" -dialect sqlite -sql "select geometry, building as bldg, 
'building:levels' as bldg_level, height as height from multipolygons" 
output_shps input.osm
It raises the following error message:
ERROR 1: In ExecutableSQL<>: sqlite3_prepare:
no such column: height
Interestingly when I convert an .osm file to .shp files without using the SQL 
syntax, then the multipolygons.shp file does contain the "height" field. Not 
all shapes inside it contain it, but some do.

So why is the upper "no such column: height" error message raised?

Is something wrong with the SQL syntax?

Thank you for the reply.

Kind regards,
Djordje
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

Related to your previous question, you can use SQL select only for fields which 
are included in osmconf.ini. Perhaps "height" is not. 'building:levels' may 
make a surprise for you, if ogr2ogr follows SQL specification it will select a 
fixed string. You should use double quotes instead and they may require 
escaping but I do not remember with which character. Start with easy test and 
more attributes with odd names one by one.

-Jukka-

Lähettäjä: Djordje Spasic<mailto:issworld2...@yahoo.com>
Lähetetty: ‎12.‎11.‎2016 3:37
Vastaanottaja: Rahkonen Jukka 
(MML)<mailto:jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: VS: [gdal-dev] How to replace an osm key?

Hi Jukka,

I have just tried to rename the fields with SQL. Here is the syntax:
"C:\gdal\ogr2ogr.exe" --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f 
"ESRI Shapefile" -dialect sqlite -sql "select geometry, building as bldg, 
'building:levels' as bldg_level, height as height from multipolygons" 
output_shps input.osm
It raises the following error message:
ERROR 1: In ExecutableSQL<>: sqlite3_prepare:
no such column: height
Interestingly when I convert an .osm file to .shp files without using the SQL 
syntax, then the multipolygons.shp file does contain the "height" field. Not 
all shapes inside it contain it, but some do.

So why is the upper "no such column: height" error message raised?

Is something wrong with the SQL syntax?

Thank you for the reply.

Kind regards,
Djordje

----


On Friday, November 11, 2016 6:46 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi> wrote:


Hi,

If you select geometry and attribute "foo" that's all you get. If you want 
attribute "bar" as well you must do "select geometry,foo,bar from your_table". 
That is plain SQL, you will find tutorials from the web.

-Jukka-
____
Lähettäjä: Djordje Spasic<mailto:issworld2...@yahoo.com>
Lähetetty: ‎11.‎11.‎2016 19:10
Vastaanottaja: Rahkonen Jukka 
(MML)<mailto:jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: [gdal-dev] How to replace an osm key?

Thank you Jukka.

I understand now why I need to repeat the same syntax 4 times (by only changing 
multipolygon/multilinestrings/lines/points).



Can you explain this part as well please:
> You must edit the SQL parameter to include all the fields that you want to 
> get into the result  "select geometry, building as bldg, another_attribute as 
> attr_2 …”
?


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

If you select geometry and attribute "foo" that's all you get. If you want 
attribute "bar" as well you must do "select geometry,foo,bar from your_table". 
That is plain SQL, you will find tutorials from the web.

-Jukka-

Lähettäjä: Djordje Spasic<mailto:issworld2...@yahoo.com>
Lähetetty: ‎11.‎11.‎2016 19:10
Vastaanottaja: Rahkonen Jukka 
(MML)<mailto:jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org<mailto:gdal-dev@lists.osgeo.org>
Aihe: Re: [gdal-dev] How to replace an osm key?

Thank you Jukka.

I understand now why I need to repeat the same syntax 4 times (by only changing 
multipolygon/multilinestrings/lines/points).



Can you explain this part as well please:
> You must edit the SQL parameter to include all the fields that you want to 
> get into the result  "select geometry, building as bldg, another_attribute as 
> attr_2 …”
?


--------
On Friday, November 11, 2016 4:04 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi> wrote:

Hi,

“Building” tag can exist in any layer. It is controlled with the osmconf.ini 
file https://svn.osgeo.org/gdal/trunk/gdal/data/osmconf.ini

It is true that by default it is selected only for multipolygons. However, I 
suppose that in the native OSM data the building tag is sometimes used also on 
points and relations (other_relations for GDAL). I also aimed at a general 
answer that could be useful for all tags and layers in OSM data.

-Jukka Rahkonen-

Lähettäjä: Djordje Spasic [mailto:issworld2...@yahoo.com]
Lähetetty: 11. marraskuuta 2016 16:38
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] How to replace an osm key?

Thank you for the reply Jukka.

> and you must run the query separately for all the layers ( points, lines, 
> multilinestrings, multipolygons).
What does this part mean?
That I need to call ogr2ogr.exe four times:
ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from 
multipolygons” output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from points” 
output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from lines” 
output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from 
multilinestrings” output.shp input.osm
?
Doesn't "building:..." key (field) only exist among the multipolygons?



> You must edit the SQL parameter to include all the fields that you want to 
> get into the result  "select geometry, building as bldg, another_attribute as 
> attr_2 …”
I didn't understand you this part either.

The "select geometry, building as bldg, another_attribute as attr_2 …” part 
needs to include only those keys (fields) which I would like to get changed? 
Not all the others which I would like to have them remain the same as in 
original .osm file?

If I misunderstood you, can you please provide an example syntax?

Thank you, and my apology for further disturbance.

Djordje
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

“Building” tag can exist in any layer. It is controlled with the osmconf.ini 
file https://svn.osgeo.org/gdal/trunk/gdal/data/osmconf.ini

It is true that by default it is selected only for multipolygons. However, I 
suppose that in the native OSM data the building tag is sometimes used also on 
points and relations (other_relations for GDAL). I also aimed at a general 
answer that could be useful for all tags and layers in OSM data.

-Jukka Rahkonen-

Lähettäjä: Djordje Spasic [mailto:issworld2...@yahoo.com]
Lähetetty: 11. marraskuuta 2016 16:38
Vastaanottaja: Rahkonen Jukka (MML) <jukka.rahko...@maanmittauslaitos.fi>; 
gdal-dev@lists.osgeo.org
Aihe: Re: [gdal-dev] How to replace an osm key?

Thank you for the reply Jukka.

> and you must run the query separately for all the layers ( points, lines, 
> multilinestrings, multipolygons).
What does this part mean?
That I need to call ogr2ogr.exe four times:
ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from 
multipolygons” output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from points” 
output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from lines” 
output.shp input.osm

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" -dialect sqlite –sql "select geometry, building as bldg from 
multilinestrings” output.shp input.osm
?
Doesn't "building:..." key (field) only exist among the multipolygons?



> You must edit the SQL parameter to include all the fields that you want to 
> get into the result  "select geometry, building as bldg, another_attribute as 
> attr_2 …”
I didn't understand you this part either.

The "select geometry, building as bldg, another_attribute as attr_2 …” part 
needs to include only those keys (fields) which I would like to get changed? 
Not all the others which I would like to have them remain the same as in 
original .osm file?

If I misunderstood you, can you please provide an example syntax?

Thank you, and my apology for further disturbance.

Djordje



On Friday, November 11, 2016 3:14 PM, Rahkonen Jukka (MML) 
<jukka.rahko...@maanmittauslaitos.fi<mailto:jukka.rahko...@maanmittauslaitos.fi>>
 wrote:

Hi,

Ogrinfo and ogr2ogr usually accept the same syntax and therefore I gave the 
ogrinfo example. I apologize, it can’t be self-evident for all GDAL users.

You can do the whole thing with one command like this:

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile"
-dialect sqlite –sql "select geometry, building as bldg from multipolygons” 
output.shp input.osm

You must edit the SQL parameter to include all the fields that you want to get 
into the result  "select geometry, building as bldg, another_attribute as 
attr_2 …” and you must run the query separately for all the layers ( points, 
lines, multilinestrings, multipolygons).

-Jukka Rahkonen-

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to replace an osm key?

[Rahkonen Jukka (MML)]

Hi,

Ogrinfo and ogr2ogr usually accept the same syntax and therefore I gave the 
ogrinfo example. I apologize, it can’t be self-evident for all GDAL users.

You can do the whole thing with one command like this:

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile"
-dialect sqlite –sql "select geometry, building as bldg from multipolygons” 
output.shp input.osm

You must edit the SQL parameter to include all the fields that you want to get 
into the result  "select geometry, building as bldg, another_attribute as 
attr_2 …” and you must run the query separately for all the layers ( points, 
lines, multilinestrings, multipolygons).

-Jukka Rahkonen-



Djordje Spasic wrote:

Hi Jukka,

Thank you for the quick and informative reply!

But I am not sure I understood you.

So first I need to convert the .osm file to .shp files with ogr2ogr.exe
And then I am calling the ogrinfo.exe on specific .shp file (multipolygons.dbf 
?)

ogr2ogr.exe --config OSM_USE_CUSTOM_INDEXING NO -skipfailures -f "ESRI 
Shapefile" output.shp input.osm

ogrinfo.exe -dialect sqlite -sql "select geometry, building as bldg
from output.shp/multipolygons.dbf" input.osm.pbf

?

On Friday, November 11, 2016 2:36 PM, jratike80 
>
 wrote:

Hi,

You can rename the fields with SQL

ogrinfo -dialect sqlite -sql "select geometry, building as bldg
from multipolygons" hamburg-latest.osm.pbf


Geometry Column = GEOMETRY
bldg: String (0.0)


Slight drawback is that you must include all the attributes in SELECT but
you can save the command into text file for reuse.

-Jukka Rahkonen-



--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/gdal-dev-How-to-replace-an-osm-key-tp5295339p5295342.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Make gdal_polygonize to support the mask band

[Rahkonen Jukka (MML)]

Hi,

Once again someone wants to get the footprints of valid data in images 
http://gis.stackexchange.com/questions/216624/creating-a-shapefile-for-footprint-of-tasters-valid-data-areas-with-gdal.

A year ago, in here 
https://lists.osgeo.org/pipermail/gdal-dev/2015-October/042859.html
Even suggested to create a mask-only image with 
gdal_translate in.tif mask.tif -b mask
and then polygonize that into vectors with gdal_polygonize.

I wonder how hard it would be to make gdal_polygonize to find the mask band 
directly and recognize it with "-b mask" just like gdal_translate does.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Attach database and GeoPackage

[Rahkonen Jukka (MML)]

Hi,

I have been thinking that it could be nice to be able to open GeoPackage with 
attached database https://www.sqlite.org/lang_attach.html. Attaching database 
that contains spatial tables would probably give too much troubles because 
reading the spatial metadata of the attached db through the master db so that 
GeoPackage clients understands what happens feels difficult.  However, 
attaching tables with pure attribute data (aspatial tables) might make some 
sense by allowing to keep the heavy but stable geometries in one db and more 
frequently updated attributes in another. Wondering if there could be some open 
option like -oo attach_database=path_to_another_sqlite_db. I do not have any 
clear use case in my mind yet, just thinking.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Should GDAL handle this KML superoverlay?

[Rahkonen Jukka (MML)]

Hi,

I have some images which are georeferenced with a KML file and they show fine 
with Google Earth. However, GDAL does not know what to do with them.
I put some test data here:

http://latuviitta.org/downloads/shaded.kmz
http://latuviitta.org/downloads/shaded.kml
http://latuviitta.org/downloads/shaded.png

Test with gdalinfo (GDAL 2.2-dev, has KMLSUPEROVERLAY driver)

gdalinfo shaded.kmz
ERROR 4: `shaded.kmz' not recognized as a supported file format.
gdalinfo failed - unable to open 'shaded.kmz'.

By looking at the only documentation I found from KMLSUPEROVERLAY (i.e. running 
"gdalinfo --format kmlsuperoverlay") it looks like KMLSUPEROVERVIEW likes to 
write tiles instead of referring one whole png file as in the dataset that I 
have.  This mail thread may be related 
http://osgeo-org.1560.x6.nabble.com/kmz-KMLSUPEROVERLAY-back-to-a-georeferenced-raster-td5286480.html.


-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Ogr and long list of errors due to missing SQLite module

[Rahkonen Jukka (MML)]

Hi,

If a GeoPackage file is touched with recent Spatialite-gui it creates 
automatically a virtual GPKG table for each native spatial GPKG table. If I now 
use ogrinfo or ogr2ogr with such database it will print an error for each 
virtual layer like

ERROR 1: no such module: VirtualGPKG
Warning 1: unable to read table definition for 'vgpkg_Virtausnuoli'

Ogr2ogr works despite the messages but if there are 130 layers in the 
GeoPackage the error list is rather long. For most users it would be enough, or 
still too much, to get a list of the missing modules just one time for the 
whole datastore.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] RFC 64 wiki page is empty

[Rahkonen Jukka (MML)]

Hi,

There is no contents in 
https://trac.osgeo.org/gdal/wiki/rfc64_triangle_polyhedralsurface_tin but it 
would be nice to have something. I suppose that RFC was defined and implemented 
in
https://github.com/avyavkumar/gdal/wiki

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Gdal_grid with a mask layer?

[Rahkonen Jukka (MML)]

Hi,

I have depth data from a lake as scattered points and I would like to convert 
them into raster DEM.  Unfortunately this lake is not a rectangular, north 
oriented one but it has a curvy shoreline and lots of islands. In my first 
trials the resulting DEM leaks away from the water to the dry ground. I guess 
that some other algorithms than the invdist would give better results near the 
shoreline. Generating fake zero depth points on the ground area would probably 
help with all the algorithms. However, what I would like to do is to define a 
mask layer for telling the valid area for the DEM that will be generated.  Is 
that possible somehow? If it is not then other good tricks are welcome. The 
good result should have zero depth on the ground and sharp shoreline at correct 
place.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] GPKG: Drop table but how to drop the rest?

[Rahkonen Jukka (MML)]

Hi,

I wonder if it could be possible to do something like "DROP CASCADING" in 
GeoPackage and delete the spatial index by the same. It will for sure be 
unusable once the table is dropped. I feel that SQLite does not support that 
because cascading drop seems to require that parent and child tables have 
foreign key relation, but RTree index which is created through the Virtual 
RTRee does not offer any way for defining the foreign key.

I know I can delete the index with another SQL request with ogrinfo

ogrinfo -sql "drop table bar" foo.gpkg
ogrinfo -sql "drop table rtree_bar_geom" foo.gpkg

Metadata tables are not cleaned automatically and two more commands would be 
needed:

ogrinfo -sql "delete from gpkg_contents where table_name='bar' and 
identifier='bar'" foo.gpkg
ogrinfo -sql "delete from gpkg_geometry_columns where table_name='bar' and 
column_name='geom'" foo.gpkg

However, it is impossible to run the latter one because GPKG database is at 
this moment in a bad shape


ogrinfo -sql "delete from gpkg_geometry_columns where table_name=bar' and 
column_name='geom'" foo.gpkg
ERROR 1: bad result for PRAGMA foreign_key_check, got 1 rows, expected 0
ERROR 1: pragma foreign_key_check on 'foo.gpkg' failed
ERROR 1: bad result for PRAGMA foreign_key_check, got 1 rows, expected 0
ERROR 1: pragma foreign_key_check on 'foo.gpkg' failed
FAILURE:
Unable to open datasource `foo.gpkg' with the following drivers.
...
I can delete that row from gpkg_geometry_columns with spatialite_gui.

So what is the right procedure and order for dropping a table from GeoPackage 
with ogrinfo?

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Preserve Source Raster Resolution When Merging

[Rahkonen Jukka (MML)]

Sam Wilson wrote:

> Hi,

> I'm attempting to merge (or maybe the proper term is mosaic) overlapping 
> raster datasets of varying resolutions. I'd like to have the higher  
> resolution datasets override the lower resolution datasets where available.

> I've been able to get close using various methods i.e. gdal_merge.py or by 
> constructing a VRT using gdalbuildvrt and providing the input file list.

> However, both of those methods don't seem to be ideal, unless I'm mistaken, 
> because I'd prefer to preserve the native resolution of all datasets being 
> merged.

> For example, when using gdal_merge, you can provide a single pixel size, 
> which I think makes the entire merged raster at that resolution. Thus, I end 
> up gaining resolution where I don't want it and losing resolution where I do 
> want it. The same goes using the gdalbuildvrt method using the -resolution 
> argument.

> The end goal is to have a single raster containing all the input rasters at 
> their native resolutions.  I've done a lot of searching around online and it 
> appears that this can be achieved using ArcGIS tools.  However, I'm wondering 
> if there are any workarounds to achieve that other than using ArcGIS.  For 
> example, is their a modified version of gdal_merge.py that keeps the source 
> rasters at their native resolutions when merging?

> Thanks for any help!

Hi Sam,

Unfortunately you can't do that. From 
https://en.wikipedia.org/wiki/Raster_graphics
"In computer graphics, a raster graphics image is a dot matrix data structure 
representing a generally rectangular grid of pixels, or points of color, 
viewable via a monitor, paper, or other display medium. Raster images are 
stored in image files with varying formats."

It is hard to believe that ArcGIS could write raster files with variable pixel 
size either. I rather believe that it keeps the native resolution images 
somewhere in files or in the database and mosaics, resamples, and renders them 
on-the-fly. Something that you can simply demonstrate by opening your native 
images into the same QGIS project. But because of your online search work you 
must have some links to share with us.

BTW. if you select the best resolution of all the native images for the mosaic 
you will not lose anything, except disk space.

-Jukka Rahkonen-

> Cheers,
> Sam

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Issues with MapInfo Conversion to GeoJson using ogr2ogr

[Rahkonen Jukka (MML)]

Suman Humagain wrote:

> Hi folks,

> I am converting the Mapinfo files to GeoJson format using the ogr2ogr tool.
> The things are working great and its converting fine but i am getting
> different polylines than i have drawn in the mapinfo. Some extra polylines
> are created in the intersections. If there is of the two polylines are
> intersected in the MapInfo, during ogr2ogr conversion, we can find 4
> polylines in the GeoJson file. Am i missing something here or it the way
> ogr2ogr is designed.

> Thanks in advance.

Please attach a small MapInfo file that can be used for reproducing the issue.

Thanks in advance.

-Jukka Rahkonen-

> -- 
> *Suman Humagain*
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Motions: Promote 1.11.5 RC 1, 2.0.3 RC2 and and 2.1.1 RC2 for release

[Rahkonen Jukka (MML)]

+1 for all three.

-Jukka Rahkonen-


Lähettäjä: Even Rouault<mailto:even.roua...@spatialys.com>
Lähetetty: ‎12.‎7.‎2016 12:15
Vastaanottaja: Rahkonen Jukka (MML)<mailto:jukka.rahko...@maanmittauslaitos.fi>
Aihe: Fwd: [gdal-dev] Motions: Promote 1.11.5 RC 1, 2.0.3 RC2 and and 2.1.1 RC2 
for release


--  Message transmis  --

Sujet : Fwd: [gdal-dev] Motions: Promote 1.11.5 RC 1, 2.0.3 RC2 and and 2.1.1
RC2 for release
Date : mardi 12 juillet 2016, 11:18:07
De : Even Rouault <even.roua...@spatialys.com>
À : Frank Warmerdam <warmer...@pobox.com>, Daniel Morissette
<dmorisse...@mapgears.com>, Tamas Szekeres <szeker...@gmail.com>, Howard
Butler <how...@hobu.co>, Andrey Kiselev <d...@ak4719.spb.edu>, Jukka Rahkonen
<jukka.rahko...@mmmtike.fi>

Dear PSC members,

I'd appreciate a few more votes on those motions, especially regarding
2.0.3RC2 that hasn't yet reached the minimum +2 threshold.

Best regards,

Even

--  Message transmis  --

Sujet : [gdal-dev] Motions: Promote 1.11.5 RC 1, 2.0.3 RC2 and and 2.1.1 RC2
for release
Date : vendredi 08 juillet 2016, 12:02:10
De : Even Rouault <even.roua...@spatialys.com>
À : gdal-dev@lists.osgeo.org

Hi,

Motions under vote:
Motion 1: GDAL/OGR 1.11.5 RC1 is promoted to be the official 1.11.5 final
release.

Motion 2: GDAL/OGR 2.0.3 RC2 is promoted to be the official 2.0.3 final release.

To replace the retracted motion 3:
Motion 3bis: GDAL/OGR 2.1.1 RC2 is promoted to be the official 2.1.1 final
release.

--

My votes :
Motion 1: + 1
Motion 2: + 1
Motion 3bis: + 1

Best regards,

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev
---
--
Spatialys - Geospatial professional services
http://www.spatialys.com

---
--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to add more than one column to a shapefile using ogrinfo

[Rahkonen Jukka (MML)]

Andrea wrote:


>Hi Jukka,





>Rahkonen Jukka (MML) wrote

>> Do you know some database that accepts SQL "ALTER TABLE foo ADD COLUMN

>> bar1, bar2"? At least SQLite and PostgreSQL don't. That kind of usage is

>> not mentioned in a bunch of documents about SQL language which I browsed

>> through, for example http://www.w3schools.com/sql/sql_alter.asp.



>it's certainly a PostgreSQL syntax, and my (wrong) question derives from it.

>And it's just like you wrote:

>ALTER TABLE public.table ADD COLUMN col1 int, ADD COLUMN col2 int;



>And in MySQL should be something like:

>ALTER TABLE table

>ADD COLUMN colA VARCHAR(10) NOT NULL,

>ADD COLUMN colB INT(10) UNSIGNED NOT NULL,

>AFTER thiscolumn



>I think it works also in Microsoft SQL server and Oracle.



Hi,



Using the same SQL syntax that works with pgAdmin works also with ogrinfo:



ogrinfo PG:"dbname='mydb' host='localhost' port='5432' user='myself'" -sql 
"alter table foo add column bar7 text, add column bar8 text"



The syntax with PostgreSQL seems really require two ADD COLUMNs

ALTER TABLE foo ADD COLUMN bar4 text, ADD COLUMNS bar5 text;



In Oracle this is supported with syntax



alter table
   table_name
add
   (
   column1_name column1_datatype column1_constraint,
   column2_name column2_datatype column2_constraint,
   column3_name column3_datatype column3_constraint
   );



That syntax is not accepted by PostgreSQL, tested with



alter table foo add column (bar3 text, bar4 text);



ERROR:  syntax error at or near "("

LINE 1: alter table foo add column (bar3 text, bar4 text)



SQLite accepts only one column or perhaps I did not find the right syntax.



It seems to me that adding a general support for adding multiple columns with a 
single request into OGR SQL dialect would require to go through every driver: 
With PostgreSQL, build SQL statement with two ADD COLUMNs, with Oracle use one 
ADD and list of column definition, with SQLite make as many separate ALTER 
TABLEs as needed, with file based formats do whatever is needed. That is for 
sure doable but feels like a big work. Users should also be somehow aware that 
certain drivers use by default some other SQL dialect than the OGR SQL and 
adding multiple columns would require different syntax with native dialect and 
with OGR SQL.



-Jukka Rahkonen-









>Best regards,



Andrea

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to add more than one column to a shapefile using ogrinfo

[Rahkonen Jukka (MML)]

Andrea Borruso wrote:


>thank you very much. I had read that page, but stupidly I did not think
>that "ALTER TABLE tablename ADD [COLUMN] columnname columntype" meant
>strictly that I could only use it in this way.
>My wrong intepretion was that I could use "alter table add" SQL command
>and its variations. But it's only on my my mind.


Hi,

Do you know some database that accepts SQL "ALTER TABLE foo ADD COLUMN bar1, 
bar2"? At least SQLite and PostgreSQL don't. That kind of usage is not 
mentioned in a bunch of documents about SQL language which I browsed through, 
for example http://www.w3schools.com/sql/sql_alter.asp.

-Jukka Rahkonen-
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] How to append to all layers with ogr2ogr?

[Rahkonen Jukka (MML)]

Hi,

Did you try to add another GML file into existing GeoPackege?

First run OK:
ogr2ogr -f gpkg -update -append mtk.gpkg /vsizip/K2344R_mtk.zip

Secong run gives an error:
ogr2ogr -f gpkg -update -append mtk.gpkg /vsizip/K2433L_mtk.zip
ERROR 1: Layer Kunta already exists, and -append not specified.
Consider using -append, or -overwrite.
ERROR 1: Terminating translation prematurely after failed
translation of layer Kunta (use -skipfailures to skip errors)

-Jukka-

Even Rouault wrote:


> Jukka,

>> 
>> I would like to collect data from a bunch of GML files into one 
>> GeoPackage but GML files may contain more than 20 layers and ogr2ogr 
>> does not want to append data for me because the -append switch 
>> probably has an effect only to one explicitly given layer.

> -append is supposed to work with several layers too.

> I've just downloaded
> http://kartat.kapsi.fi/files/maastotietokanta/kaikki/etrs89/gml/K2/K24/K2433L_mtk.zip
> and tried your below command, both a non inexisting mtk.gpkg, and one created 
> with it, and it works as expected for me.

> Perhaps the bug only triggers with some initial content of mtk.gpkg ?

> 
> >ogr2ogr -f gpkg -update -append mtk.gpkg /vsizip/K2433L_mtk.zip

Even

--
Spatialys - Geospatial professional services http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev