Re: [gdal-dev] FFI Interface (to Perl)

[Joaquim Manuel Freire Luís]

You have also the Julia wrapper that needs no ffi lib to do the bindings

https://github.com/visr/GDAL.jl

From: gdal-dev [mailto:gdal-dev-boun...@lists.osgeo.org] On Behalf Of Ari Jolma
Sent: Thursday, December 21, 2017 1:03 AM
To: Sean Gillies ; Kurt Schwehr 
Cc: gdal dev 
Subject: Re: [gdal-dev] FFI Interface (to Perl)

Sean Gillies kirjoitti 21.12.2017 klo 11:48:

Ari,

This sounds great. I don't use Perl as much as I used to – unless you count GNU 
Parallel, which I use about 320 hours a day – but I'll follow your progress.

It seems to me that the idea of a common API across different scripting 
languages isn't attractive to programmers like it once was. R and Node.js 
interfaces for GDAL are being developed elsewhere, as are my Rasterio and Fiona 
projects. As long as we're all attentive to the maintenance of the core library 
the diversity is a good thing.

I'm more interested in mixing Perl with R for example (sending some things from 
Perl to be done in R) than having a common API. Every language has its 
traditions for API.



Shapely uses libffi to bind GEOS functions. The only reason I don't use libffi 
(and Python's cffi module) for Rasterio and Fiona is that Cython makes 
integration with Numpy so painless.

So far I've not tried integration with PDL, which is for similar purposes than 
Numpy. We'll see how that goes eventually, it is anyway something I use for 
raster processing.

Ari




On Wed, Dec 20, 2017 at 5:14 PM, Kurt Schwehr 
> wrote:
Cool!  Should be useful for other languages to see what is possible with libffi

On Dec 20, 2017 3:23 PM, "Ari Jolma" 
> wrote:
Folks,

I've been experimenting with the Perl module FFI::Platypus, to create a Perl 
interface to GDAL with its help(1) and it's progressing well. FFI::Platypus 
itself uses libffi, which is a portable interface library. That, and 
Alien::gdal, which is a Perl module to use GDAL in a system that has Perl, 
allow some significant simplifications and streamlining for the Perl bindings.

I can imagine those leading to the retirement of the Swig based Perl bindings 
at some point (like GDAL 3.0) and having only a single distribution of Perl 
GDAL bindings that does not live in GDAL source tree and is distributed through 
CPAN. In the meantime the Swig based bindings will live in GDAL source tree and 
a copy of that with a modified configuration system (that uses the Alien::gdal) 
in github(2) + CPAN.

I don't think I'm aiming at 100% similar API with the FFI interface than 
Geo::GDAL since in the latter has accumulated some undesirable things over the 
years. Anybody interested in the new API is welcome to collaborate.

Best regards,

Ari

(1) https://github.com/ajolma/Geo-GDAL-FFI

(2) https://github.com/ajolma/Geo-GDAL-2.02


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev



--
Sean Gillies

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FFI Interface (to Perl)

[Ari Jolma]

Sean Gillies kirjoitti 21.12.2017 klo 11:48:

Ari,

This sounds great. I don't use Perl as much as I used to – unless you 
count GNU Parallel, which I use about 320 hours a day – but I'll 
follow your progress.


It seems to me that the idea of a common API across different 
scripting languages isn't attractive to programmers like it once was. 
R and Node.js interfaces for GDAL are being developed elsewhere, as 
are my Rasterio and Fiona projects. As long as we're all attentive to 
the maintenance of the core library the diversity is a good thing.


I'm more interested in mixing Perl with R for example (sending some 
things from Perl to be done in R) than having a common API. Every 
language has its traditions for API.




Shapely uses libffi to bind GEOS functions. The only reason I don't 
use libffi (and Python's cffi module) for Rasterio and Fiona is that 
Cython makes integration with Numpy so painless.


So far I've not tried integration with PDL, which is for similar 
purposes than Numpy. We'll see how that goes eventually, it is anyway 
something I use for raster processing.


Ari




On Wed, Dec 20, 2017 at 5:14 PM, Kurt Schwehr > wrote:


Cool!  Should be useful for other languages to see what is
possible with libffi

On Dec 20, 2017 3:23 PM, "Ari Jolma" > wrote:

Folks,

I've been experimenting with the Perl module FFI::Platypus, to
create a Perl interface to GDAL with its help(1) and it's
progressing well. FFI::Platypus itself uses libffi, which is a
portable interface library. That, and Alien::gdal, which is a
Perl module to use GDAL in a system that has Perl, allow some
significant simplifications and streamlining for the Perl
bindings.

I can imagine those leading to the retirement of the Swig
based Perl bindings at some point (like GDAL 3.0) and having
only a single distribution of Perl GDAL bindings that does not
live in GDAL source tree and is distributed through CPAN. In
the meantime the Swig based bindings will live in GDAL source
tree and a copy of that with a modified configuration system
(that uses the Alien::gdal) in github(2) + CPAN.

I don't think I'm aiming at 100% similar API with the FFI
interface than Geo::GDAL since in the latter has accumulated
some undesirable things over the years. Anybody interested in
the new API is welcome to collaborate.

Best regards,

Ari

(1) https://github.com/ajolma/Geo-GDAL-FFI


(2) https://github.com/ajolma/Geo-GDAL-2.02



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev





--
Sean Gillies


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FFI Interface (to Perl)

[Sean Gillies]

Ari,

This sounds great. I don't use Perl as much as I used to – unless you count
GNU Parallel, which I use about 320 hours a day – but I'll follow your
progress.

It seems to me that the idea of a common API across different scripting
languages isn't attractive to programmers like it once was. R and Node.js
interfaces for GDAL are being developed elsewhere, as are my Rasterio and
Fiona projects. As long as we're all attentive to the maintenance of the
core library the diversity is a good thing.

Shapely uses libffi to bind GEOS functions. The only reason I don't use
libffi (and Python's cffi module) for Rasterio and Fiona is that Cython
makes integration with Numpy so painless.


On Wed, Dec 20, 2017 at 5:14 PM, Kurt Schwehr  wrote:

> Cool!  Should be useful for other languages to see what is possible with
> libffi
>
> On Dec 20, 2017 3:23 PM, "Ari Jolma"  wrote:
>
> Folks,
>
> I've been experimenting with the Perl module FFI::Platypus, to create a
> Perl interface to GDAL with its help(1) and it's progressing well.
> FFI::Platypus itself uses libffi, which is a portable interface library.
> That, and Alien::gdal, which is a Perl module to use GDAL in a system that
> has Perl, allow some significant simplifications and streamlining for the
> Perl bindings.
>
> I can imagine those leading to the retirement of the Swig based Perl
> bindings at some point (like GDAL 3.0) and having only a single
> distribution of Perl GDAL bindings that does not live in GDAL source tree
> and is distributed through CPAN. In the meantime the Swig based bindings
> will live in GDAL source tree and a copy of that with a modified
> configuration system (that uses the Alien::gdal) in github(2) + CPAN.
>
> I don't think I'm aiming at 100% similar API with the FFI interface than
> Geo::GDAL since in the latter has accumulated some undesirable things over
> the years. Anybody interested in the new API is welcome to collaborate.
>
> Best regards,
>
> Ari
>
> (1) https://github.com/ajolma/Geo-GDAL-FFI
>
> (2) https://github.com/ajolma/Geo-GDAL-2.02
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] FFI Interface (to Perl)

[Kurt Schwehr]

Cool!  Should be useful for other languages to see what is possible with
libffi

On Dec 20, 2017 3:23 PM, "Ari Jolma"  wrote:

Folks,

I've been experimenting with the Perl module FFI::Platypus, to create a
Perl interface to GDAL with its help(1) and it's progressing well.
FFI::Platypus itself uses libffi, which is a portable interface library.
That, and Alien::gdal, which is a Perl module to use GDAL in a system that
has Perl, allow some significant simplifications and streamlining for the
Perl bindings.

I can imagine those leading to the retirement of the Swig based Perl
bindings at some point (like GDAL 3.0) and having only a single
distribution of Perl GDAL bindings that does not live in GDAL source tree
and is distributed through CPAN. In the meantime the Swig based bindings
will live in GDAL source tree and a copy of that with a modified
configuration system (that uses the Alien::gdal) in github(2) + CPAN.

I don't think I'm aiming at 100% similar API with the FFI interface than
Geo::GDAL since in the latter has accumulated some undesirable things over
the years. Anybody interested in the new API is welcome to collaborate.

Best regards,

Ari

(1) https://github.com/ajolma/Geo-GDAL-FFI

(2) https://github.com/ajolma/Geo-GDAL-2.02


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] FFI Interface (to Perl)

[Ari Jolma]

Folks,

I've been experimenting with the Perl module FFI::Platypus, to create a 
Perl interface to GDAL with its help(1) and it's progressing well. 
FFI::Platypus itself uses libffi, which is a portable interface library. 
That, and Alien::gdal, which is a Perl module to use GDAL in a system 
that has Perl, allow some significant simplifications and streamlining 
for the Perl bindings.


I can imagine those leading to the retirement of the Swig based Perl 
bindings at some point (like GDAL 3.0) and having only a single 
distribution of Perl GDAL bindings that does not live in GDAL source 
tree and is distributed through CPAN. In the meantime the Swig based 
bindings will live in GDAL source tree and a copy of that with a 
modified configuration system (that uses the Alien::gdal) in github(2) + 
CPAN.


I don't think I'm aiming at 100% similar API with the FFI interface than 
Geo::GDAL since in the latter has accumulated some undesirable things 
over the years. Anybody interested in the new API is welcome to collaborate.


Best regards,

Ari

(1) https://github.com/ajolma/Geo-GDAL-FFI

(2) https://github.com/ajolma/Geo-GDAL-2.02


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Ari Jolma]

Dmitry Baryshnikov kirjoitti 21.12.2017 klo 08:58:


Hi Ari,

The pull request and discussion can be found here: 
https://github.com/OSGeo/gdal/pull/280




Yes. Sorry. I didn't see that. Maybe Kurt also didn't see that - or the 
aspect of it that he brought up here. Maybe a place for some 
institutional improvements.


I cannot imagine that this improvements will affect something else. 
MD5 used for cache paths initially, I only added some improvements for 
WMS cache size limits, expire time and split cache per dataset base 
(i.e. to delete cached files when dataset deletes).


What about your idea about common caching classes/functions - this is 
topic to discuss.




I needed a simple URL -> a few files cache for the WCS. Not much code 
but probably other drivers need such functionality too. I didn't do good 
research although I knew WMS has similar needs. The code is not much but 
issues such as unique filenames and simultaneous access from 
threads/processes add complication.


Ari



Best regards,
 Dmitry
21.12.2017 0:32, Ari Jolma пишет:

My comments on this:

1) The MD5 function seems to be needed only(?) for the WMS cache and 
probably also WFS. To me that highlights the need for some 
integration work regarding caching in GDAL. I also implemented yet 
another caching code for the WCS (without MD5). Looking at my 
$HOME/.gdal I see also gmlas_xsd_cache and srs_cache.


2) This discussion thread also highlights the need to go towards more 
pull request style development at least for bigger changes.


Ari


Kurt Schwehr kirjoitti 21.12.2017 klo 02:56:

Dmitry,

Statements like this indicate a world of trouble:

> I'm not sure that Python produce the same hash than cvs_MD5*.
> Also I'm not sure what in future they will be the same.

If you remove the swig and python stuff, move the CPLMD5String to 
cpl_md5.{cpp,h}, and follow Even's suggestions, I will be much 
happier with the change.  A C++ test would be good too. I'll be 
happy to do a quick fix up of a few things that I see in the md5 
code once it is to that point.  I do think it is a good thing that 
there be md5 support in port along with the existing sha{1,256} code.


I see what you are saying about 
the OGRGeometry::exportTo{json,kml,gml}, but that doesn't (to me) 
support your argument.  My personal opinion is that GDAL would have 
been stronger if those had separate and not been a part of OGRGeometry.


If there is a critical difference between CPLMD5String or a language 
and use case where md5 support in that language from GDAL is really 
required, you need to first document that issue.


On Wed, Dec 20, 2017 at 7:46 AM, Dmitry Baryshnikov 
> wrote:


    Just the note that CPLMD5String not only for Python, but any other
    API clients: C/C++, Java, Perl, etc.

    But, I agree, it can be easily removed.

    Best regards,
        Dmitry

    20.12.2017 18 :35, Even Rouault пишет:

    And why are you exposing this to python? Python has
    md5 hashing

    already

    built in.

    I'm not sure that Python produce the same hash than 
cvs_MD5*.


    I do hope they return the same thing. MD5 is a standardized
    algorithm:
https://tools.ietf.org/html/rfc1321


    I'm not sure we really need to expose it our CPL MD5 through
    Python.


    ___
    gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev





--
--
http://schwehr.org


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Even Rouault]

On jeudi 21 décembre 2017 00:58:53 CET Dmitry Baryshnikov wrote:
> Hi Ari,
> 
> The pull request and discussion can be found here:
> https://github.com/OSGeo/gdal/pull/280
> 
> I cannot imagine that this improvements will affect something else. MD5
> used for cache paths initially, I only added some improvements for WMS
> cache size limits, expire time and split cache per dataset base (i.e. to
> delete cached files when dataset deletes).
> 
> What about your idea about common caching classes/functions - this is
> topic to discuss.

Ah, and in a driver I'm going to commit in a few hours, I also have a 
(primitive) on-disk tile 
caching.

For in-memory caching, I found this useful class that implements a LRU cache:
https://github.com/rouault/gdal2/blob/idaho_driver/gdal/port/cpl_mem_cache.h

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Dmitry Baryshnikov]

Hi Ari,

The pull request and discussion can be found here: 
https://github.com/OSGeo/gdal/pull/280


I cannot imagine that this improvements will affect something else. MD5 
used for cache paths initially, I only added some improvements for WMS 
cache size limits, expire time and split cache per dataset base (i.e. to 
delete cached files when dataset deletes).


What about your idea about common caching classes/functions - this is 
topic to discuss.


Best regards,
Dmitry

21.12.2017 0:32, Ari Jolma пишет:

My comments on this:

1) The MD5 function seems to be needed only(?) for the WMS cache and 
probably also WFS. To me that highlights the need for some integration 
work regarding caching in GDAL. I also implemented yet another caching 
code for the WCS (without MD5). Looking at my $HOME/.gdal I see also 
gmlas_xsd_cache and srs_cache.


2) This discussion thread also highlights the need to go towards more 
pull request style development at least for bigger changes.


Ari


Kurt Schwehr kirjoitti 21.12.2017 klo 02:56:

Dmitry,

Statements like this indicate a world of trouble:

> I'm not sure that Python produce the same hash than cvs_MD5*.
> Also I'm not sure what in future they will be the same.

If you remove the swig and python stuff, move the CPLMD5String to 
cpl_md5.{cpp,h}, and follow Even's suggestions, I will be much 
happier with the change.  A C++ test would be good too. I'll be happy 
to do a quick fix up of a few things that I see in the md5 code once 
it is to that point.  I do think it is a good thing that there be md5 
support in port along with the existing sha{1,256} code.


I see what you are saying about 
the OGRGeometry::exportTo{json,kml,gml}, but that doesn't (to me) 
support your argument.  My personal opinion is that GDAL would have 
been stronger if those had separate and not been a part of OGRGeometry.


If there is a critical difference between CPLMD5String or a language 
and use case where md5 support in that language from GDAL is really 
required, you need to first document that issue.


On Wed, Dec 20, 2017 at 7:46 AM, Dmitry Baryshnikov 
> wrote:


    Just the note that CPLMD5String not only for Python, but any other
    API clients: C/C++, Java, Perl, etc.

    But, I agree, it can be easily removed.

    Best regards,
        Dmitry

    20.12.2017 18 :35, Even Rouault пишет:

    And why are you exposing this to python?  Python has
    md5 hashing

    already

    built in.

    I'm not sure that Python produce the same hash than 
cvs_MD5*.


    I do hope they return the same thing. MD5 is a standardized
    algorithm:
    https://tools.ietf.org/html/rfc1321
    

    I'm not sure we really need to expose it our CPL MD5 through
    Python.


    ___
    gdal-dev mailing list
    gdal-dev@lists.osgeo.org 
    https://lists.osgeo.org/mailman/listinfo/gdal-dev
    




--
--
http://schwehr.org


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Ari Jolma]

My comments on this:

1) The MD5 function seems to be needed only(?) for the WMS cache and 
probably also WFS. To me that highlights the need for some integration 
work regarding caching in GDAL. I also implemented yet another caching 
code for the WCS (without MD5). Looking at my $HOME/.gdal I see also 
gmlas_xsd_cache and srs_cache.


2) This discussion thread also highlights the need to go towards more 
pull request style development at least for bigger changes.


Ari


Kurt Schwehr kirjoitti 21.12.2017 klo 02:56:

Dmitry,

Statements like this indicate a world of trouble:

> I'm not sure that Python produce the same hash than cvs_MD5*.
> Also I'm not sure what in future they will be the same.

If you remove the swig and python stuff, move the CPLMD5String to 
cpl_md5.{cpp,h}, and follow Even's suggestions, I will be much happier 
with the change.  A C++ test would be good too.  I'll be happy to do a 
quick fix up of a few things that I see in the md5 code once it is to 
that point.  I do think it is a good thing that there be md5 support 
in port along with the existing sha{1,256} code.


I see what you are saying about 
the OGRGeometry::exportTo{json,kml,gml}, but that doesn't (to me) 
support your argument.  My personal opinion is that GDAL would have 
been stronger if those had separate and not been a part of OGRGeometry.


If there is a critical difference between CPLMD5String or a language 
and use case where md5 support in that language from GDAL is really 
required, you need to first document that issue.


On Wed, Dec 20, 2017 at 7:46 AM, Dmitry Baryshnikov 
> wrote:


Just the note that CPLMD5String not only for Python, but any other
API clients: C/C++, Java, Perl, etc.

But, I agree, it can be easily removed.

Best regards,
    Dmitry

20.12.2017 18 :35, Even Rouault пишет:

And why are you exposing this to python?  Python has
md5 hashing

already

built in.

I'm not sure that Python produce the same hash than cvs_MD5*.

I do hope they return the same thing. MD5 is a standardized
algorithm:
https://tools.ietf.org/html/rfc1321


I'm not sure we really need to expose it our CPL MD5 through
Python.


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org 
https://lists.osgeo.org/mailman/listinfo/gdal-dev





--
--
http://schwehr.org


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Segfault when accessing vpf dqyarea

[Even Rouault]

Mike,

I could replicate. The issue is not in GDAL, but in the underlying ogdi 
library. 
Ubuntu 16.04 ships with ogdi 3.2.0-beta2. With final ogdi 3.2.0 (available in 
Ubuntu zesty), the crash no longer occurs.

Compiling ogdi from source for modern distributions is a bit of a pain. You 
could probably try building from the source  package of ogdi in ubuntu 16.04 
and substituting the ogdi-3.2.0.tar.gz toogdi-dfsg_3.2.0~beta2.orig.tar.gz

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Kurt Schwehr]

Dmitry,

Statements like this indicate a world of trouble:

> I'm not sure that Python produce the same hash than cvs_MD5*.
> Also I'm not sure what in future they will be the same.

If you remove the swig and python stuff, move the CPLMD5String to
cpl_md5.{cpp,h}, and follow Even's suggestions, I will be much happier with
the change.  A C++ test would be good too.  I'll be happy to do a quick fix
up of a few things that I see in the md5 code once it is to that point.  I
do think it is a good thing that there be md5 support in port along with
the existing sha{1,256} code.

I see what you are saying about the OGRGeometry::exportTo{json,kml,gml},
but that doesn't (to me) support your argument.  My personal opinion is
that GDAL would have been stronger if those had separate and not been a
part of OGRGeometry.

If there is a critical difference between CPLMD5String or a language and
use case where md5 support in that language from GDAL is really required,
you need to first document that issue.

On Wed, Dec 20, 2017 at 7:46 AM, Dmitry Baryshnikov 
wrote:

> Just the note that CPLMD5String not only for Python, but any other API
> clients: C/C++, Java, Perl, etc.
>
> But, I agree, it can be easily removed.
>
> Best regards,
> Dmitry
>
> 20.12.2017 18:35, Even Rouault пишет:
>
>> And why are you exposing this to python?  Python has md5 hashing

>>> already
>>
>>> built in.

>>> I'm not sure that Python produce the same hash than cvs_MD5*.
>>>
>> I do hope they return the same thing. MD5 is a standardized algorithm:
>> https://tools.ietf.org/html/rfc1321
>>
>> I'm not sure we really need to expose it our CPL MD5 through Python.
>>
>>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Segfault when accessing vpf dqyarea

[Mike]

So using Ubuntu 16.04, and Ubuntu gis-unstable ppa, which has gdal 2.2.2.

Sample data

https://dnc.nga.mil/DNC/library.php?c=C12

Direct link

https://dnc.nga.mil/DNCSiteContent/StaticFiles/data12/vpf/coa12f.zip

I can do an ogrinfo gltp:/vrf/path/DNC12/COA12F

and get all the layers. I can pick any layer to get further info on, except
dqyarea@dqy(*)_area, where it then segfaults. I'm unsure if this is a bug
or on how it was packaged.

Of note, gdal 2.2.0 (from qgis 2.18.9) on Windows reads dqyarea fine.

Any tips on how to read the layer?

-Mike
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Dmitry Baryshnikov]

Just the note that CPLMD5String not only for Python, but any other API 
clients: C/C++, Java, Perl, etc.


But, I agree, it can be easily removed.

Best regards,
Dmitry

20.12.2017 18:35, Even Rouault пишет:

And why are you exposing this to python?  Python has md5 hashing

already

built in.

I'm not sure that Python produce the same hash than cvs_MD5*.

I do hope they return the same thing. MD5 is a standardized algorithm:
https://tools.ietf.org/html/rfc1321

I'm not sure we really need to expose it our CPL MD5 through Python.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Dmitry Baryshnikov]

Just the note that CPLMD5String not only for Python, but any other API 
clients: C/C++, Java, Perl, etc.


But, I agree, it can be easily removed.

Best regards,
Dmitry

20.12.2017 18:35, Even Rouault пишет:

And why are you exposing this to python?  Python has md5 hashing

already

built in.

I'm not sure that Python produce the same hash than cvs_MD5*.

I do hope they return the same thing. MD5 is a standardized algorithm:
https://tools.ietf.org/html/rfc1321

I'm not sure we really need to expose it our CPL MD5 through Python.



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Even Rouault]

> > And why are you exposing this to python?  Python has md5 hashing 
already
> > built in.
> 
> I'm not sure that Python produce the same hash than cvs_MD5*.

I do hope they return the same thing. MD5 is a standardized algorithm:
https://tools.ietf.org/html/rfc1321

I'm not sure we really need to expose it our CPL MD5 through Python.

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Dmitry Baryshnikov]

Will answer in letter body

20.12.2017 17:39, Kurt Schwehr пишет:

My personal opinion...

It was not in fact in the core of GDAL for all users.  My primary
environment does not include the WMS *driver*.  My take is that only the
memory and gtiff raster drivers are really required for GDAL (vector, I'm
not so sure).

What do you mean by the same or similar for generic and geojson?  And
because something isn't great elsewhere, that does not mean we want to
introduce more similar code that is hard to impossible to opt out of.

For example we have such method - char * OGRGeometry::exportToJson() const

In this method executed OGR_G_ExportToJson from 
ogr/ogrsf_frmts/geojson/ogrgeojsonwriter.cpp


So the core OGR using some code from GeoJSON driver.

The same way I can move md5 back to WMS and use it from port function 
CPLMD5String.


But I think ogr and gdal must use port (and port shouldn't depends on 
org or gcore, etc.),
also drivers can use ogr or gdal, but ogr and gdal shouldn't depends on 
some driver.


This is only my opinion.


Dmitry's question about a reasonable solution is a good one (if you buy my
being seriously uncomfortable with cvs_MD5*).  Possibilities include:

- Make everything from md5.cpp and md5.h be internal to md5.cpp (aka static
or inline) and move CPLMD5String to md5.{cpp,h} at least keeping the
trouble localize
- Refactor md5 to be cleaner
- Reimplement md5 hashing for gdal
- Find some other code that is license compatible that is cleaner

And why are you exposing this to python?  Python has md5 hashing already
built in.

I'm not sure that Python produce the same hash than cvs_MD5*.
Also I'm not sure what in future they will be the same.
Finally not only Python use CPLMD5String - any program utilizing API may 
need the hash formed the same way as WMS/WFS does.


Can others please weigh in?

On Wed, Dec 20, 2017 at 6:25 AM, Dmitry Baryshnikov 
wrote:


In fact it was part of core of GDAL but hidden in WMS driver folder. The
same or similar is in:

1. ogr/ogrsf_frmts/generic

2. ogr/ogrsf_frmts/geojson

What solution can be suitable here (I mean md5 case)?


Best regards,
 Dmitry

20.12.2017 17:19, Kurt Schwehr пишет:


I have not looked in the wms code much, but my comment was not about
adding
an external dependency. It is about code quality in the core of GDAL.

On Dec 20, 2017 6:07 AM, "Dmitry Baryshnikov" 
wrote:

Hi Kurt.

The md5.cpp and md5.h were in frmst/wms driver folder. As I see from
header this files are not initially from GDAL project.

I moved this files to port with minimum changes - only macros for
suppress
Doxygen parsing.

The motivation of this moving was:

1. md5 functions used in several drivers (WMS and WFS) and may be others
in future.

2. need API function to get the same md5 hash as used in WMS for
autotesting functionality.

3. downloading area of interest into the WMS file based cache by external
programs. May be some other utilization.

I think ideally the OpenSSL MD5 functions must be using. But this will be
one more dependency. Do we need it?


Best regards,
  Dmitry

20.12.2017 16:43, Kurt Schwehr пишет:

Hi all,

Can we discuss cvs_MD5* from https://trac.osgeo.org/gdal/changeset/41086
?

I very much appreciate the work that bishop is doing and I hate to slow
down a contributor

I am really worried about code like this going into GDAL, especially into
port/

But my worries are:

- this in no way conforms to other code in GDAL
- has the potential to collide with other code that imports this code
- it has a very awkward C style
- the commit message did not point to where it came from (at least it's
not
hard to guess)
- the file naming is different than (most) of the rest of the directory
- and...

Yes, we have other libraries included, but it seems like a road we don't
want to go down

-kurt




___
gdal-dev mailing listgdal-dev@lists.osgeo.orghttps://
lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev






Best regards,
Dmitry

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Even Rouault]

On mercredi 20 décembre 2017 06:39:37 CET Kurt Schwehr wrote:
> My personal opinion...
> 
> It was not in fact in the core of GDAL for all users.  My primary
> environment does not include the WMS *driver*.  My take is that only the
> memory and gtiff raster drivers are really required for GDAL (vector, I'm
> not so sure).
> 
> What do you mean by the same or similar for generic and geojson?  And
> because something isn't great elsewhere, that does not mean we want to
> introduce more similar code that is hard to impossible to opt out of.
> 
> Dmitry's question about a reasonable solution is a good one (if you buy my
> being seriously uncomfortable with cvs_MD5*).  Possibilities include:
> 
> - Make everything from md5.cpp and md5.h be internal to md5.cpp (aka static
> or inline) and move CPLMD5String to md5.{cpp,h} at least keeping the
> trouble localize
> - Refactor md5 to be cleaner
> - Reimplement md5 hashing for gdal
> - Find some other code that is license compatible that is cleaner
> 
> And why are you exposing this to python?  Python has md5 hashing already
> built in.
> 
> Can others please weigh in?

For consistency with the rest, I'd suggest:
- rename md5.* to cpl_md5.*
- rename cvs_MD5* to CPLMD5*
- include cpl_port.h in cpl_md5.h and remove the #if 
defined(CPL_BASE_H_INCLUDED) stuff. 
Currently if someone includes md5.h as the first header, cvs_uint32 will expand 
to a unsigned 
long, whereas the md5.cpp file has been compiled with GUInt32. Not good

Other follow-up cleanups can then be done by interested parties.

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Poll: is Visual Studio 2013 support needed for GDAL 2.3 ?

[Jeff McKenna]

On 2017-12-20 9:41 AM, Jeff McKenna wrote:

https://trac.osgeo.org/gdal/wiki/SupportedCompilers



Thanks for this page Kurt.  Can you also make sure that it is linked to 
from the main wiki opening page?  (so people can find it) 
https://trac.osgeo.org/gdal/wiki   Thanks,





ah I found it on BuildHints page.  Great!  Sorry for this noise.   -jeff


https://trac.osgeo.org/gdal/wiki/BuildHints





___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Poll: is Visual Studio 2013 support needed for GDAL 2.3 ?

[Kurt Schwehr]

I'm hesitant to link that page in the main page.  It's not particularly
complete and this was the first update in years.

On Wed, Dec 20, 2017 at 6:41 AM, Jeff McKenna  wrote:

> On 2017-12-19 7:01 PM, Kurt Schwehr wrote:
>
>>
>> I just took a pass at updating the supported compiler list.  Please add
>> more tested configs.  I mostly just wanted to get statements about
>> supporting old platforms out of there.
>>
>> https://trac.osgeo.org/gdal/wiki/SupportedCompilers
>>
>>
> Thanks for this page Kurt.  Can you also make sure that it is linked to
> from the main wiki opening page?  (so people can find it)
> https://trac.osgeo.org/gdal/wiki   Thanks,
>
> -jeff
>
>
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Poll: is Visual Studio 2013 support needed for GDAL 2.3 ?

[Jeff McKenna]

On 2017-12-19 7:01 PM, Kurt Schwehr wrote:


I just took a pass at updating the supported compiler list.  Please add 
more tested configs.  I mostly just wanted to get statements about 
supporting old platforms out of there.


https://trac.osgeo.org/gdal/wiki/SupportedCompilers



Thanks for this page Kurt.  Can you also make sure that it is linked to 
from the main wiki opening page?  (so people can find it) 
https://trac.osgeo.org/gdal/wiki   Thanks,


-jeff



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Kurt Schwehr]

My personal opinion...

It was not in fact in the core of GDAL for all users.  My primary
environment does not include the WMS *driver*.  My take is that only the
memory and gtiff raster drivers are really required for GDAL (vector, I'm
not so sure).

What do you mean by the same or similar for generic and geojson?  And
because something isn't great elsewhere, that does not mean we want to
introduce more similar code that is hard to impossible to opt out of.

Dmitry's question about a reasonable solution is a good one (if you buy my
being seriously uncomfortable with cvs_MD5*).  Possibilities include:

- Make everything from md5.cpp and md5.h be internal to md5.cpp (aka static
or inline) and move CPLMD5String to md5.{cpp,h} at least keeping the
trouble localize
- Refactor md5 to be cleaner
- Reimplement md5 hashing for gdal
- Find some other code that is license compatible that is cleaner

And why are you exposing this to python?  Python has md5 hashing already
built in.

Can others please weigh in?

On Wed, Dec 20, 2017 at 6:25 AM, Dmitry Baryshnikov 
wrote:

> In fact it was part of core of GDAL but hidden in WMS driver folder. The
> same or similar is in:
>
> 1. ogr/ogrsf_frmts/generic
>
> 2. ogr/ogrsf_frmts/geojson
>
> What solution can be suitable here (I mean md5 case)?
>
>
> Best regards,
> Dmitry
>
> 20.12.2017 17:19, Kurt Schwehr пишет:
>
>> I have not looked in the wms code much, but my comment was not about
>> adding
>> an external dependency. It is about code quality in the core of GDAL.
>>
>> On Dec 20, 2017 6:07 AM, "Dmitry Baryshnikov" 
>> wrote:
>>
>> Hi Kurt.
>>>
>>> The md5.cpp and md5.h were in frmst/wms driver folder. As I see from
>>> header this files are not initially from GDAL project.
>>>
>>> I moved this files to port with minimum changes - only macros for
>>> suppress
>>> Doxygen parsing.
>>>
>>> The motivation of this moving was:
>>>
>>> 1. md5 functions used in several drivers (WMS and WFS) and may be others
>>> in future.
>>>
>>> 2. need API function to get the same md5 hash as used in WMS for
>>> autotesting functionality.
>>>
>>> 3. downloading area of interest into the WMS file based cache by external
>>> programs. May be some other utilization.
>>>
>>> I think ideally the OpenSSL MD5 functions must be using. But this will be
>>> one more dependency. Do we need it?
>>>
>>>
>>> Best regards,
>>>  Dmitry
>>>
>>> 20.12.2017 16:43, Kurt Schwehr пишет:
>>>
>>> Hi all,
>>>
>>> Can we discuss cvs_MD5* from https://trac.osgeo.org/gdal/changeset/41086
>>> ?
>>>
>>> I very much appreciate the work that bishop is doing and I hate to slow
>>> down a contributor
>>>
>>> I am really worried about code like this going into GDAL, especially into
>>> port/
>>>
>>> But my worries are:
>>>
>>> - this in no way conforms to other code in GDAL
>>> - has the potential to collide with other code that imports this code
>>> - it has a very awkward C style
>>> - the commit message did not point to where it came from (at least it's
>>> not
>>> hard to guess)
>>> - the file naming is different than (most) of the rest of the directory
>>> - and...
>>>
>>> Yes, we have other libraries included, but it seems like a road we don't
>>> want to go down
>>>
>>> -kurt
>>>
>>>
>>>
>>>
>>> ___
>>> gdal-dev mailing listgdal-dev@lists.osgeo.orghttps://
>>> lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>>>
>>>
>>> ___
>>> gdal-dev mailing list
>>> gdal-dev@lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>>>
>


-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] port/md5 cvs_MD5*

[Dmitry Baryshnikov]

In fact it was part of core of GDAL but hidden in WMS driver folder. The 
same or similar is in:


1. ogr/ogrsf_frmts/generic

2. ogr/ogrsf_frmts/geojson

What solution can be suitable here (I mean md5 case)?


Best regards,
Dmitry

20.12.2017 17:19, Kurt Schwehr пишет:

I have not looked in the wms code much, but my comment was not about adding
an external dependency. It is about code quality in the core of GDAL.

On Dec 20, 2017 6:07 AM, "Dmitry Baryshnikov"  wrote:


Hi Kurt.

The md5.cpp and md5.h were in frmst/wms driver folder. As I see from
header this files are not initially from GDAL project.

I moved this files to port with minimum changes - only macros for suppress
Doxygen parsing.

The motivation of this moving was:

1. md5 functions used in several drivers (WMS and WFS) and may be others
in future.

2. need API function to get the same md5 hash as used in WMS for
autotesting functionality.

3. downloading area of interest into the WMS file based cache by external
programs. May be some other utilization.

I think ideally the OpenSSL MD5 functions must be using. But this will be
one more dependency. Do we need it?


Best regards,
 Dmitry

20.12.2017 16:43, Kurt Schwehr пишет:

Hi all,

Can we discuss cvs_MD5* from https://trac.osgeo.org/gdal/changeset/41086 ?

I very much appreciate the work that bishop is doing and I hate to slow
down a contributor

I am really worried about code like this going into GDAL, especially into
port/

But my worries are:

- this in no way conforms to other code in GDAL
- has the potential to collide with other code that imports this code
- it has a very awkward C style
- the commit message did not point to where it came from (at least it's not
hard to guess)
- the file naming is different than (most) of the rest of the directory
- and...

Yes, we have other libraries included, but it seems like a road we don't
want to go down

-kurt




___
gdal-dev mailing 
listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] port/md5 cvs_MD5*

[Kurt Schwehr]

Hi all,

Can we discuss cvs_MD5* from https://trac.osgeo.org/gdal/changeset/41086 ?

I very much appreciate the work that bishop is doing and I hate to slow
down a contributor

I am really worried about code like this going into GDAL, especially into
port/

But my worries are:

- this in no way conforms to other code in GDAL
- has the potential to collide with other code that imports this code
- it has a very awkward C style
- the commit message did not point to where it came from (at least it's not
hard to guess)
- the file naming is different than (most) of the rest of the directory
- and...

Yes, we have other libraries included, but it seems like a road we don't
want to go down

-kurt
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev