Re: [gdal-dev] Native library load failed while using gdal maven dependency - UnsatisfiedLinkError

[ssaniev]

So the solution for using gdal bindings in java was to install the proper
version which came with all the dll's and simply using the jar from the
maven repo.

I installed core gdal installer in windows, and set my environment
variables, and gdal jar started to work correctly in my maven project.

Installer used:
http://www.gisinternals.com/release.php
gdal-111-1800-x64-core.msi  Generic installer for the GDAL core components
http://download.gisinternals.com/sdk/downloads/release-1800-x64-gdal-1-11-3-mapserver-6-4-2/gdal-111-1800-x64-core.msi

maven dependency:

org.gdal
gdal
1.11.2



Environment variables:
GDAL_BIN=C:\Program Files\GDAL
GDAL_DATA=C:\Program Files\GDAL\gdal-data
GDAL_DRIVER=C:\Program Files\GDAL\gdalplugins

Also in eclipse, passing in as a VM argument to find needed *.dll-s :
-Djava.library.path="C:\Program Files\GDAL;${env_var:PATH}"

---

So code works at this point. 

import org.gdal.gdal.gdal;
import org.gdal.ogr.ogr;

public class ShapeFileWorker {
ogr.RegisterAll();
gdal.AllRegister();
gdal.VersionInfo();
}



Pulled a lot of hairs until now, but finally works.



--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/Native-library-load-failed-while-using-gdal-maven-dependency-UnsatisfiedLinkError-tp5249728p5249930.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Ari Jolma]

10.02.2016, 15:21, Kurt Schwehr kirjoitti:


One request:

#define OGR_G_NOT_EMPTY_POINT 0x1
#define OGR_G_3D 0x2
#define OGR_G_MEASURED 0x4
#define OGR_G_IGNORE_MEASURED 0x8
Can you make these a static const int or static const unsigned int 
member of the OGRGeometry class?  #defines make life more difficult.


These constants are used also outside the classes. How does one access 
those for example in OGRWktReadPointsM, which receives the flags as a 
parameter?


Ari

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Ari Jolma]

I've started a wiki page for the status of the drivers regarding 
measured geometries:


https://trac.osgeo.org/gdal/wiki/MeasuredGeometriesInDrivers

Ari

10.02.2016, 15:05, Even Rouault kirjoitti:

Le mercredi 10 février 2016 13:53:28, Peter Halls a écrit :

Types on page 2).
PointZ
A PointZ consists of a triplet of double-precision coordinates in the

order


X, Y, Z plus a measure"

I had misinterpreted their meaning of "optional" here and submitted a
documentation query to ESRI on being told I was wrong.  The response was
that M being optional means that it can be valueless, but is always
present.

Woo, really  That's a genuine scoop.

I reported it to  Frank at the time ... but several email systems later, I
no longer have my copies of the correspondence with ESRI nor the OGR bug
report, in which these were copied.  Somewhere I may still have a backup of
my work area, to see what I did in my code.  However, as you say, this is
20+ years ago   It was something which, at the time, was causing us
some problems 

I think you to be correct in your Shapelib summary below:

Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
without M values for more than 20 years. I'm surprised we wouldn't have
heard about that if such shapefiles couldn't be read. Or perhaps ESRI
software is robust to missing M too

Would someone looking at this discussion and having access to ESRI software be
willing to generate tiny (meaning just one single shape, with the smallest
number of vertices) shapefiles of type PointZ, ArcZ, PolygonZ and MultiPointZ,
and *without* M values ? So as to enhance the test suite and see in particular
which value is set exactly as the nodata value in the M component

I was wondering if we should change the behaviour of the shape driver to add
that nodata M component for XYZ shapes. This wouldn't cause compat problems
with previous versions of GDAL, but it would cause .shp sizes to grow by ~
1/3. So perhaps go on with current behaviour of stripping M component if most
implementations seem to be OK with that ?


Best wishes,

Peter




On 10 February 2016 at 12:31, Even Rouault 

wrote:

Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :

Ari, et al,

   ESRI handle this in a non-intuitive way:  XYM is supported, but Z

always has a Measure, so is XYZM!   The formal definition is here:
https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf
(1998)

   Shape Types having Z are defined on pp19ff where it states:
"Shape Types inX,Y,Z Space
Shapes of this type have an optional coordinate
M. Note that "no data" value can be specified as a value for M (see

Numeric


Types on page 2).
PointZ
A PointZ consists of a triplet of double-precision coordinates in the

order


X, Y, Z plus a measure"

I had misinterpreted their meaning of "optional" here and submitted a
documentation query to ESRI on being told I was wrong.  The response
was that M being optional means that it can be valueless, but is
always present.

Woo, really  That's a genuine scoop.

Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
without M values for more than 20 years. I'm surprised we wouldn't have
heard
about that if such shapefiles couldn't be read. Or perhaps ESRI software
is robust to missing M too


Best wishes,

Peter

--
Spatialys - Geospatial professional services
http://www.spatialys.com


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Kurt Schwehr]

OGRGeometry::OGR_G_NOT_EMPTY_POINT

On Wed, Feb 10, 2016 at 7:34 AM, Ari Jolma  wrote:

>
> 10.02.2016, 15:21, Kurt Schwehr kirjoitti:
>
>>
>> One request:
>>
>> #define OGR_G_NOT_EMPTY_POINT 0x1
>> #define OGR_G_3D 0x2
>> #define OGR_G_MEASURED 0x4
>> #define OGR_G_IGNORE_MEASURED 0x8
>> Can you make these a static const int or static const unsigned int member
>> of the OGRGeometry class?  #defines make life more difficult.
>>
>
> These constants are used also outside the classes. How does one access
> those for example in OGRWktReadPointsM, which receives the flags as a
> parameter?
>
> Ari
>
>


-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Mike Toews]

On 11 February 2016 at 01:31, Even Rouault  wrote:
> Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :
>>   ESRI handle this in a non-intuitive way:  XYM is supported, but Z
>> always has a Measure, so is XYZM!   The formal definition is here:
>> https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf   (1998)
>>
>>   Shape Types having Z are defined on pp19ff where it states:
>>
>> "Shape Types inX,Y,Z Space
>> Shapes of this type have an optional coordinate
>
> Woo, really  That's a genuine scoop.
>
> Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
> without M values for more than 20 years. I'm surprised we wouldn't have heard
> about that if such shapefiles couldn't be read. Or perhaps ESRI software is
> robust to missing M too

I've looked into this a few years ago, because I found differences in
XYZ files written by GDAL and Esri software. There are two styles of
(e.g.) PointZ files, where the coordinates are written with either
three or four doubles (the last with "no data" if M is not used). The
style of PointZ is determined by the content length provided in the
record header as 16-bit words (either 14 or 18). Therefore, the specs
are correct when they describe M as an "optional coordinate" (p.15).

For XYZ shapefiles, GDAL always write three doubles, whereas Esri
write four doubles with "no data" as the last coordinate. As a result,
Esri's XYZ shp files are slightly larger than GDAL's.

Occasionally I use software that requires PointZ shapefiles to have
four doubles, so my existing workaround is to use Esri's
CopyFeatures_management tool to rewrite the shapefile.

Could "ogr2ogr -dim 4" be added to this RFC to write XYZ[M] Shapefiles
that always use four floats for coordinates, as Esri software does?


On 11 February 2016 at 02:05, Even Rouault  wrote:
> Would someone looking at this discussion and having access to ESRI software be
> willing to generate tiny (meaning just one single shape, with the smallest
> number of vertices) shapefiles of type PointZ, ArcZ, PolygonZ and MultiPointZ,
> and *without* M values ? So as to enhance the test suite and see in particular
> which value is set exactly as the nodata value in the M component

I can generate these today, and will attach a zip file to
https://trac.osgeo.org/gdal/wiki/MeasuredGeometriesInDrivers
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] NetCDF+HDF5 and threadsafety

[William Kyngesburye]

Finally got a chance to test this.

With HDF5 built with thread safety, without HL (of course, no NetCDF4), the 
multireadtest has no problems on a HDF5 file.

The opposite - no thread safety + HL and NetCDF4, I get a bunch of "pointer 
being freed was not allocated" errors.

If I force HL + threadsafety (new config option for HDF5), multireadtest again 
succeeds, and also on a NetCDF4 file.  Austotests also succeed on both.

> On Feb 5, 2016, at 9:23 AM, William Kyngesburye  
> wrote:
> 
> On Feb 5, 2016, at 2:38 AM, Even Rouault  wrote:
>> 
 There's an optionnaly compiled app you can use to test threadsafety :
 
 cd apps
 make multireadtest
 ./multireadtest -nlo [-oi number_of_iterations] file_to_test
 
 Increase number_of_iterations so that it runs for a few seconds.
>>> 
>>> How do I trigger building that? 
>> 
>> Just like the above mentionned lines :-) After you've initially built GDAL 
>> from the top directory
> 
> Ah, I'm not reading the details, my brain just saw the run line ;)
> 

-
William Kyngesburye 
http://www.kyngchaos.com/

[Trillian]  What are you supposed to do WITH a maniacally depressed robot?

[Marvin]  You think you have problems?  What are you supposed to do if you ARE 
a maniacally depressed robot?  No, don't try and answer, I'm 50,000 times more 
intelligent than you and even I don't know the answer...

- HitchHiker's Guide to the Galaxy


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Ari Jolma]

One more issue came up.

Shapefiles, which have Z will also have M, at least a placeholder for it.

Having shapefiles as XYZM always in that case may be problematic.

It is possible to use the open options SHPT=type to override the 
default. We could have Z => XYZ as default for backwards compatibility 
and require open option for Z  => XYZM.


Ari


09.02.2016, 17:07, Ari Jolma kirjoitti:
I declare this motion passed with +1 from Even, Tamas, Jukka and 
Daniel and no -1.


I'll commit soon the changes to the core and then later the drivers: 
memory, shape and pg. The goal is to keep the travis test build passing.


Ari

05.02.2016, 10:04, Ari Jolma kirjoitti:
I'd like to ask the PSC and others to vote on adopting RFC 61: 
Support for measured geometries.


The draft RFC is at

https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries

and a draft implementation is at

https://github.com/ajolma/GDAL-XYZM

which is tested at

https://travis-ci.org/ajolma/GDAL-XYZM

(the test #34, which passed, was the so far last one after core 
changes with unchanged autotests and a rather comprehensive set of 
WKT tests in the Perl tests directory)


This is seemingly a small change but it is at the heart of OGR so it 
has some important implications. The only backwards incompatibilities 
that have appeared so far are with some drivers, for example 
shapefile, which can be lessened with, e.g., open options.


Best,

Ari





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Even Rouault]

Le mercredi 10 février 2016 12:26:59, Ari Jolma a écrit :
> One more issue came up.
> 
> Shapefiles, which have Z will also have M, at least a placeholder for it.
> 
> Having shapefiles as XYZM always in that case may be problematic.

Small precision for those not necessarily familiar with the internals of the 
shapefile format. The .shp and .shx header have a declared shape type (used to 
set the OGR layer geometry type), that can be (restricting to the case of 
polygon for the sake of clarity, but true for other geometry types) :
SHPT_POLYGON : the shapefile contains only Polygon 2D shapes
SHPT_POLYGONZ:  the shapefile may contain a mix of PolygonZ or PolygonZM shapes
SHPT_POLYGONM: the shapefile may contain a mix of Polygon or PolygonM shapes

Declaring a SHPT_POLYGONM as a PolygonM layer is OK (if the shapefile doesn't 
contain PolygonM shapes, it should declare only SHPT_POLYGON)

But declaring SHPT_POLYGONZ as a PolygonZM layer, when there are in fact no 
PolygonZM shapes in it but just PolygonZ, is more problematic for backward 
compatibility issues. Ideally when dealing with shapefiles without M values we 
shouldn't advertize M neither in the layer geometry type, nor in the feature 
geometry.

> 
> It is possible to use the open options SHPT=type to override the
> default. We could have Z => XYZ as default for backwards compatibility
> and require open option for Z  => XYZM.

That's one possibility. As open options should be a last resort, if SHPT isn't 
specified perhaps we could do a probing of the first shape. If it has a M 
component then advertize ZM geometry type, otherwise just xZ. That's 
of course not completely bullet proof as there can be in theory a mix of Z and 
ZM shapes, but my feeling is that this should work in > 90% of the cases.

That's an improvement I could have a look at during the Code Sprint if you 
don't feel confident enough in the shape driver.


~

Somehow related issue I've noticed but that should be more easily fixable. Now 
creating a Polygon25D shapefile creates PolygonZM and not PolygonZ shapes (and 
perhaps true for other shape types).

trunk:
$ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
$ ll poly25d.*
-rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
-rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
-rw-r--r-- 1 even even 8820 2016-02-10 12:36 poly25d.shp
-rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx

2.0:
$ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
$ ll poly25d.*
-rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
-rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
-rw-r--r-- 1 even even 6700 2016-02-10 12:36 poly25d.shp
-rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx

On the read size, I've commited https://trac.osgeo.org/gdal/changeset/33408
"Shape: fix crash when reading a MultiPointZ without M values, and when reading 
a PointZ without M do not create a PointZM geometry (trunk only)"

Quickly checking at the code, I think other geometry types are OK, but perhaps 
we should have more extensive cases :
- adding Z shapefiles of other geometry types generated with GDAL 2.0 and 
checking that the ExportToIsoWKT() read with trunk is the one expected
- and/or generating Z shapes with trunk and checking they are read as Z only 
with ExportToIsoWKT()

> 
> Ari
> 
> 09.02.2016, 17:07, Ari Jolma kirjoitti:
> > I declare this motion passed with +1 from Even, Tamas, Jukka and
> > Daniel and no -1.
> > 
> > I'll commit soon the changes to the core and then later the drivers:
> > memory, shape and pg. The goal is to keep the travis test build passing.
> > 
> > Ari
> > 
> > 05.02.2016, 10:04, Ari Jolma kirjoitti:
> >> I'd like to ask the PSC and others to vote on adopting RFC 61:
> >> Support for measured geometries.
> >> 
> >> The draft RFC is at
> >> 
> >> https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries
> >> 
> >> and a draft implementation is at
> >> 
> >> https://github.com/ajolma/GDAL-XYZM
> >> 
> >> which is tested at
> >> 
> >> https://travis-ci.org/ajolma/GDAL-XYZM
> >> 
> >> (the test #34, which passed, was the so far last one after core
> >> changes with unchanged autotests and a rather comprehensive set of
> >> WKT tests in the Perl tests directory)
> >> 
> >> This is seemingly a small change but it is at the heart of OGR so it
> >> has some important implications. The only backwards incompatibilities
> >> that have appeared so far are with some drivers, for example
> >> shapefile, which can be lessened with, e.g., open options.
> >> 
> >> Best,
> >> 
> >> Ari

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Even Rouault]

Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :
> Ari, et al,
> 
>   ESRI handle this in a non-intuitive way:  XYM is supported, but Z
> always has a Measure, so is XYZM!   The formal definition is here:
> https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf   (1998)
> 
>   Shape Types having Z are defined on pp19ff where it states:
> 
> "Shape Types inX,Y,Z Space
> Shapes of this type have an optional coordinate
> M. Note that "no data" value can be specified as a value for M (see Numeric
> Types on page 2).
> PointZ
> A PointZ consists of a triplet of double-precision coordinates in the order
> X, Y, Z plus a measure"
> 
> I had misinterpreted their meaning of "optional" here and submitted a
> documentation query to ESRI on being told I was wrong.  The response was
> that M being optional means that it can be valueless, but is always
> present.

Woo, really  That's a genuine scoop.

Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles 
without M values for more than 20 years. I'm surprised we wouldn't have heard 
about that if such shapefiles couldn't be read. Or perhaps ESRI software is 
robust to missing M too

> 
> Best wishes,
> 
> Peter

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Even Rouault]

Le mercredi 10 février 2016 13:22:49, Ari Jolma a écrit :
> 10.02.2016, 13:55, Even Rouault kirjoitti:
> > Le mercredi 10 février 2016 12:26:59, Ari Jolma a écrit :
> >> One more issue came up.
> >> 
> >> Shapefiles, which have Z will also have M, at least a placeholder for
> >> it.
> >> 
> >> Having shapefiles as XYZM always in that case may be problematic.
> > 
> > Small precision for those not necessarily familiar with the internals of
> > the shapefile format. The .shp and .shx header have a declared shape
> > type (used to set the OGR layer geometry type), that can be (restricting
> > to the case of polygon for the sake of clarity, but true for other
> > geometry types) : SHPT_POLYGON : the shapefile contains only Polygon 2D
> > shapes
> > SHPT_POLYGONZ:  the shapefile may contain a mix of PolygonZ or PolygonZM
> > shapes
> 
> Would you differentiate between *Z or *ZM and * and *M shapes using the
> ESRI "no data":
> 
> "Nevertheless, shapefiles support the concept of "no data" values, but
> they are currently used only for measures. Any floating point number
> smaller than –1038 is considered by a shapefile reader to
> represent a "no data" value."

Given latest Peter's message, yes likely. And also completely missing M as 
produced by shapelib up to now.

> 
> > SHPT_POLYGONM: the shapefile may contain a mix of Polygon or PolygonM
> > shapes
> > 
> > Declaring a SHPT_POLYGONM as a PolygonM layer is OK (if the shapefile
> > doesn't contain PolygonM shapes, it should declare only SHPT_POLYGON)
> > 
> > But declaring SHPT_POLYGONZ as a PolygonZM layer, when there are in fact
> > no PolygonZM shapes in it but just PolygonZ, is more problematic for
> > backward compatibility issues. Ideally when dealing with shapefiles
> > without M values we shouldn't advertize M neither in the layer geometry
> > type, nor in the feature geometry.
> > 
> >> It is possible to use the open options SHPT=type to override the
> >> default. We could have Z => XYZ as default for backwards compatibility
> >> and require open option for Z  => XYZM.
> > 
> > That's one possibility. As open options should be a last resort, if SHPT
> > isn't specified perhaps we could do a probing of the first shape. If it
> > has a M component then advertize ZM geometry type, otherwise just
> > xZ. That's of course not completely bullet proof as there can be in
> > theory a mix of Z and ZM shapes, but my feeling is that this should work
> > in > 90% of the cases.
> > 
> > That's an improvement I could have a look at during the Code Sprint if
> > you don't feel confident enough in the shape driver.
> 
> Feel free to have a look. I'm now stuck with the pg driver :)

Hopefully it is less tricky than shapefiles. I see there are other test 
failures related to shapefiles as well

> 
> Maybe, if what I wrote above is correct, read the whole shapefile in and
> then, if all M values are "no data" downgrade all geometries to XY or XYZ?

For performance reason, I'd perhaps not read all M values of all shapes, but 
only the ones of the first shape.

An extra enhancement would be to have an option to do a full scan of the 
shapefile. We could also use that to avoid advertizing Polygon layer geometry 
type when there's a mix of Polygon/MultiPolygon. 

> 
> Ari
> 
> > ~
> > 
> > Somehow related issue I've noticed but that should be more easily
> > fixable. Now creating a Polygon25D shapefile creates PolygonZM and not
> > PolygonZ shapes (and perhaps true for other shape types).
> > 
> > trunk:
> > $ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
> > $ ll poly25d.*
> > -rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
> > -rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
> > -rw-r--r-- 1 even even 8820 2016-02-10 12:36 poly25d.shp
> > -rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx
> > 
> > 2.0:
> > $ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
> > $ ll poly25d.*
> > -rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
> > -rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
> > -rw-r--r-- 1 even even 6700 2016-02-10 12:36 poly25d.shp
> > -rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx
> > 
> > On the read size, I've commited
> > https://trac.osgeo.org/gdal/changeset/33408 "Shape: fix crash when
> > reading a MultiPointZ without M values, and when reading a PointZ
> > without M do not create a PointZM geometry (trunk only)"
> > 
> > Quickly checking at the code, I think other geometry types are OK, but
> > perhaps we should have more extensive cases :
> > - adding Z shapefiles of other geometry types generated with GDAL 2.0 and
> > checking that the ExportToIsoWKT() read with trunk is the one expected
> > - and/or generating Z shapes with trunk and checking they are read as Z
> > only with ExportToIsoWKT()
> > 
> >> Ari
> >> 
> >> 09.02.2016, 17:07, Ari Jolma kirjoitti:
> >>> I declare this motion passed with +1 from Even, Tamas, Jukka and
> >>> Daniel and no -1.
> >>> 
> >>> 

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Jukka Rahkonen]

Ari Jolma  gmail.com> writes:

> 
> 
> One more issue came up.
> Shapefiles, which have Z will also have M, at least a placeholder
> for it.
> Having shapefiles as XYZM always in that case may be problematic.
> It is possible to use the open options SHPT=type to override the
> default. We could have Z => XYZ as default for backwards
> compatibility and require open option for Z  => XYZM.
> Ari09.02.2016, 17:07, Ari Jolma kirjoitti:

I noticed your changeset and I was just reading the shapefile specification.
I never knew that there are no separate XYZ and XYZM shapetypes. I also
believe I have never met a shapefile that really contains XYZM data.

I agree that the default should be to convert such shapefiles to XYZ instead
of XYZM. That would affect shapetype values

11 PointZ
13 PolyLineZ
15 PolygonZ
18 MultiPointZ

-Jukka Rahkonen-


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Peter Halls]

Ari, et al,

  ESRI handle this in a non-intuitive way:  XYM is supported, but Z
always has a Measure, so is XYZM!   The formal definition is here:
https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf   (1998)

  Shape Types having Z are defined on pp19ff where it states:

"Shape Types inX,Y,Z Space
Shapes of this type have an optional coordinate
M. Note that "no data" value can be specified as a value for M (see Numeric
Types on page 2).
PointZ
A PointZ consists of a triplet of double-precision coordinates in the order
X, Y, Z plus a measure"

I had misinterpreted their meaning of "optional" here and submitted a
documentation query to ESRI on being told I was wrong.  The response was
that M being optional means that it can be valueless, but is always present.

Best wishes,

Peter
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Ari Jolma]

10.02.2016, 13:55, Even Rouault kirjoitti:

Le mercredi 10 février 2016 12:26:59, Ari Jolma a écrit :

One more issue came up.

Shapefiles, which have Z will also have M, at least a placeholder for it.

Having shapefiles as XYZM always in that case may be problematic.

Small precision for those not necessarily familiar with the internals of the
shapefile format. The .shp and .shx header have a declared shape type (used to
set the OGR layer geometry type), that can be (restricting to the case of
polygon for the sake of clarity, but true for other geometry types) :
SHPT_POLYGON : the shapefile contains only Polygon 2D shapes
SHPT_POLYGONZ:  the shapefile may contain a mix of PolygonZ or PolygonZM shapes


Would you differentiate between *Z or *ZM and * and *M shapes using the 
ESRI "no data":


"Nevertheless, shapefiles support the concept of "no data" values, but 
they are currently used only for measures. Any floating point number 
smaller than –1038 is considered by a shapefile reader to

represent a "no data" value."




SHPT_POLYGONM: the shapefile may contain a mix of Polygon or PolygonM shapes

Declaring a SHPT_POLYGONM as a PolygonM layer is OK (if the shapefile doesn't
contain PolygonM shapes, it should declare only SHPT_POLYGON)

But declaring SHPT_POLYGONZ as a PolygonZM layer, when there are in fact no
PolygonZM shapes in it but just PolygonZ, is more problematic for backward
compatibility issues. Ideally when dealing with shapefiles without M values we
shouldn't advertize M neither in the layer geometry type, nor in the feature
geometry.


It is possible to use the open options SHPT=type to override the
default. We could have Z => XYZ as default for backwards compatibility
and require open option for Z  => XYZM.

That's one possibility. As open options should be a last resort, if SHPT isn't
specified perhaps we could do a probing of the first shape. If it has a M
component then advertize ZM geometry type, otherwise just xZ. That's
of course not completely bullet proof as there can be in theory a mix of Z and
ZM shapes, but my feeling is that this should work in > 90% of the cases.

That's an improvement I could have a look at during the Code Sprint if you
don't feel confident enough in the shape driver.


Feel free to have a look. I'm now stuck with the pg driver :)

Maybe, if what I wrote above is correct, read the whole shapefile in and 
then, if all M values are "no data" downgrade all geometries to XY or XYZ?


Ari





~

Somehow related issue I've noticed but that should be more easily fixable. Now
creating a Polygon25D shapefile creates PolygonZM and not PolygonZ shapes (and
perhaps true for other shape types).

trunk:
$ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
$ ll poly25d.*
-rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
-rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
-rw-r--r-- 1 even even 8820 2016-02-10 12:36 poly25d.shp
-rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx

2.0:
$ ogr2ogr poly25d.shp poly.shp -nlt multipolygon25d -overwrite
$ ll poly25d.*
-rw-r--r-- 1 even even  529 2016-02-10 12:36 poly25d.dbf
-rw-r--r-- 1 even even  557 2016-02-10 12:36 poly25d.prj
-rw-r--r-- 1 even even 6700 2016-02-10 12:36 poly25d.shp
-rw-r--r-- 1 even even  180 2016-02-10 12:36 poly25d.shx

On the read size, I've commited https://trac.osgeo.org/gdal/changeset/33408
"Shape: fix crash when reading a MultiPointZ without M values, and when reading
a PointZ without M do not create a PointZM geometry (trunk only)"

Quickly checking at the code, I think other geometry types are OK, but perhaps
we should have more extensive cases :
- adding Z shapefiles of other geometry types generated with GDAL 2.0 and
checking that the ExportToIsoWKT() read with trunk is the one expected
- and/or generating Z shapes with trunk and checking they are read as Z only
with ExportToIsoWKT()


Ari

09.02.2016, 17:07, Ari Jolma kirjoitti:

I declare this motion passed with +1 from Even, Tamas, Jukka and
Daniel and no -1.

I'll commit soon the changes to the core and then later the drivers:
memory, shape and pg. The goal is to keep the travis test build passing.

Ari

05.02.2016, 10:04, Ari Jolma kirjoitti:

I'd like to ask the PSC and others to vote on adopting RFC 61:
Support for measured geometries.

The draft RFC is at

https://trac.osgeo.org/gdal/wiki/rfc61_support_for_measured_geometries

and a draft implementation is at

https://github.com/ajolma/GDAL-XYZM

which is tested at

https://travis-ci.org/ajolma/GDAL-XYZM

(the test #34, which passed, was the so far last one after core
changes with unchanged autotests and a rather comprehensive set of
WKT tests in the Perl tests directory)

This is seemingly a small change but it is at the heart of OGR so it
has some important implications. The only backwards incompatibilities
that have appeared so far are with some drivers, for example
shapefile, which can be lessened with, e.g., open options.

Best,

Ari

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Peter Halls]

> Types on page 2).
> PointZ
> A PointZ consists of a triplet of double-precision coordinates in the
order
> X, Y, Z plus a measure"
>
> I had misinterpreted their meaning of "optional" here and submitted a
> documentation query to ESRI on being told I was wrong.  The response was
> that M being optional means that it can be valueless, but is always
> present.

Woo, really  That's a genuine scoop.

I reported it to  Frank at the time ... but several email systems later, I
no longer have my copies of the correspondence with ESRI nor the OGR bug
report, in which these were copied.  Somewhere I may still have a backup of
my work area, to see what I did in my code.  However, as you say, this is
20+ years ago   It was something which, at the time, was causing us
some problems 

I think you to be correct in your Shapelib summary below:

Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
without M values for more than 20 years. I'm surprised we wouldn't have
heard about that if such shapefiles couldn't be read. Or perhaps ESRI
software is robust to missing M too

Best wishes,

Peter




On 10 February 2016 at 12:31, Even Rouault 
wrote:

> Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :
> > Ari, et al,
> >
> >   ESRI handle this in a non-intuitive way:  XYM is supported, but Z
> > always has a Measure, so is XYZM!   The formal definition is here:
> > https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf   (1998)
> >
> >   Shape Types having Z are defined on pp19ff where it states:
> >
> > "Shape Types inX,Y,Z Space
> > Shapes of this type have an optional coordinate
> > M. Note that "no data" value can be specified as a value for M (see
> Numeric
> > Types on page 2).
> > PointZ
> > A PointZ consists of a triplet of double-precision coordinates in the
> order
> > X, Y, Z plus a measure"
> >
> > I had misinterpreted their meaning of "optional" here and submitted a
> > documentation query to ESRI on being told I was wrong.  The response was
> > that M being optional means that it can be valueless, but is always
> > present.
>
> Woo, really  That's a genuine scoop.
>
> Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
> without M values for more than 20 years. I'm surprised we wouldn't have
> heard
> about that if such shapefiles couldn't be read. Or perhaps ESRI software is
> robust to missing M too
>
> >
> > Best wishes,
> >
> > Peter
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>



-- 

Peter J Halls, PhD Student, Post-war Reconstruction and Development Unit
(PRDU),
  University of York

Snail mail: PRDU, Derwent College, University of York,
Heslington, York YO10 5DD
This message has the status of a private and personal communication

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Even Rouault]

Le mercredi 10 février 2016 13:53:28, Peter Halls a écrit :
> > Types on page 2).
> > PointZ
> > A PointZ consists of a triplet of double-precision coordinates in the
> 
> order
> 
> > X, Y, Z plus a measure"
> > 
> > I had misinterpreted their meaning of "optional" here and submitted a
> > documentation query to ESRI on being told I was wrong.  The response was
> > that M being optional means that it can be valueless, but is always
> > present.
> 
> Woo, really  That's a genuine scoop.
> 
> I reported it to  Frank at the time ... but several email systems later, I
> no longer have my copies of the correspondence with ESRI nor the OGR bug
> report, in which these were copied.  Somewhere I may still have a backup of
> my work area, to see what I did in my code.  However, as you say, this is
> 20+ years ago   It was something which, at the time, was causing us
> some problems 
> 
> I think you to be correct in your Shapelib summary below:
> 
> Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
> without M values for more than 20 years. I'm surprised we wouldn't have
> heard about that if such shapefiles couldn't be read. Or perhaps ESRI
> software is robust to missing M too

Would someone looking at this discussion and having access to ESRI software be 
willing to generate tiny (meaning just one single shape, with the smallest 
number of vertices) shapefiles of type PointZ, ArcZ, PolygonZ and MultiPointZ, 
and *without* M values ? So as to enhance the test suite and see in particular 
which value is set exactly as the nodata value in the M component

I was wondering if we should change the behaviour of the shape driver to add 
that nodata M component for XYZ shapes. This wouldn't cause compat problems 
with previous versions of GDAL, but it would cause .shp sizes to grow by ~ 
1/3. So perhaps go on with current behaviour of stripping M component if most 
implementations seem to be OK with that ?

> 
> Best wishes,
> 
> Peter
> 
> 
> 
> 
> On 10 February 2016 at 12:31, Even Rouault 
> 
> wrote:
> > Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :
> > > Ari, et al,
> > > 
> > >   ESRI handle this in a non-intuitive way:  XYM is supported, but Z
> > > 
> > > always has a Measure, so is XYZM!   The formal definition is here:
> > > https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf  
> > > (1998)
> > > 
> > >   Shape Types having Z are defined on pp19ff where it states:
> > > "Shape Types inX,Y,Z Space
> > > Shapes of this type have an optional coordinate
> > > M. Note that "no data" value can be specified as a value for M (see
> > 
> > Numeric
> > 
> > > Types on page 2).
> > > PointZ
> > > A PointZ consists of a triplet of double-precision coordinates in the
> > 
> > order
> > 
> > > X, Y, Z plus a measure"
> > > 
> > > I had misinterpreted their meaning of "optional" here and submitted a
> > > documentation query to ESRI on being told I was wrong.  The response
> > > was that M being optional means that it can be valueless, but is
> > > always present.
> > 
> > Woo, really  That's a genuine scoop.
> > 
> > Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
> > without M values for more than 20 years. I'm surprised we wouldn't have
> > heard
> > about that if such shapefiles couldn't be read. Or perhaps ESRI software
> > is robust to missing M too
> > 
> > > Best wishes,
> > > 
> > > Peter
> > 
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Kurt Schwehr]

I'm +0.  Neither for nor against.

One request:

#define OGR_G_NOT_EMPTY_POINT 0x1
#define OGR_G_3D 0x2
#define OGR_G_MEASURED 0x4
#define OGR_G_IGNORE_MEASURED 0x8

Can you make these a static const int or static const unsigned int member
of the OGRGeometry class?  #defines make life more difficult.

find . -name \*.h -o -name \*.cpp | xargs grep '#define' | egrep '[0-9"]' |
egrep -v '[(]|\[' | wc -l
4517

A random example of doing this:

https://trac.osgeo.org/gdal/browser/trunk/gdal/alg/gdal_simplesurf.h?rev=33127#L93

// Descriptor length
static const int DESC_SIZE = 64;

-kurt


On Wed, Feb 10, 2016 at 4:53 AM, Peter Halls  wrote:

> > Types on page 2).
> > PointZ
> > A PointZ consists of a triplet of double-precision coordinates in the
> order
> > X, Y, Z plus a measure"
> >
> > I had misinterpreted their meaning of "optional" here and submitted a
> > documentation query to ESRI on being told I was wrong.  The response was
> > that M being optional means that it can be valueless, but is always
> > present.
>
> Woo, really  That's a genuine scoop.
>
> I reported it to  Frank at the time ... but several email systems later, I
> no longer have my copies of the correspondence with ESRI nor the OGR bug
> report, in which these were copied.  Somewhere I may still have a backup of
> my work area, to see what I did in my code.  However, as you say, this is
> 20+ years ago   It was something which, at the time, was causing us
> some problems 
>
> I think you to be correct in your Shapelib summary below:
>
> Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
> without M values for more than 20 years. I'm surprised we wouldn't have
> heard about that if such shapefiles couldn't be read. Or perhaps ESRI
> software is robust to missing M too
>
> Best wishes,
>
> Peter
>
>
>
>
> On 10 February 2016 at 12:31, Even Rouault 
> wrote:
>
>> Le mercredi 10 février 2016 13:05:20, Peter Halls a écrit :
>> > Ari, et al,
>> >
>> >   ESRI handle this in a non-intuitive way:  XYM is supported, but Z
>> > always has a Measure, so is XYZM!   The formal definition is here:
>> > https://www.*esri*.com/library/whitepapers/pdfs/*shapefile*.pdf
>>  (1998)
>> >
>> >   Shape Types having Z are defined on pp19ff where it states:
>> >
>> > "Shape Types inX,Y,Z Space
>> > Shapes of this type have an optional coordinate
>> > M. Note that "no data" value can be specified as a value for M (see
>> Numeric
>> > Types on page 2).
>> > PointZ
>> > A PointZ consists of a triplet of double-precision coordinates in the
>> order
>> > X, Y, Z plus a measure"
>> >
>> > I had misinterpreted their meaning of "optional" here and submitted a
>> > documentation query to ESRI on being told I was wrong.  The response was
>> > that M being optional means that it can be valueless, but is always
>> > present.
>>
>> Woo, really  That's a genuine scoop.
>>
>> Unless I'm seriously mistaken, shapelib / OGR has produced XYZ shapefiles
>> without M values for more than 20 years. I'm surprised we wouldn't have
>> heard
>> about that if such shapefiles couldn't be read. Or perhaps ESRI software
>> is
>> robust to missing M too
>>
>> >
>> > Best wishes,
>> >
>> > Peter
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>>
>
>
>
> --
>
> 
> Peter J Halls, PhD Student, Post-war Reconstruction and Development Unit
> (PRDU),
>   University of York
>
> Snail mail: PRDU, Derwent College, University of York,
> Heslington, York YO10 5DD
> This message has the status of a private and personal communication
>
> 
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] RFC 61: Call for vote on adoption

[Mike Toews]

On 11 February 2016 at 02:05, Even Rouault  wrote:
> Would someone looking at this discussion and having access to ESRI software be
> willing to generate tiny (meaning just one single shape, with the smallest
> number of vertices) shapefiles of type PointZ, ArcZ, PolygonZ and MultiPointZ,
> and *without* M values ? So as to enhance the test suite and see in particular
> which value is set exactly as the nodata value in the M component

Done, see 
https://trac.osgeo.org/gdal/attachment/wiki/MeasuredGeometriesInDrivers/

I created two geometry examples per shapefile, and added an "Expected"
attribute to describe the WKT of the expected geometry. While the M
values were editable (despite not requesting this creation option in
ArcCatalog), I left these as "NaN".
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev