Re: [gdal-dev] python 2.7 bindings with trunk (2.3)

[Even Rouault]

> That worked for me (although I would have never found it, so I appreciate
> your help).  I've put these two lines in gdal/swig/makefile.vc, e.g.
> 
> --- a/gdal/swig/makefile.vc
> +++ b/gdal/swig/makefile.vc
> @@ -20,6 +20,8 @@ python: gdalvars
>  $(SWIG) -c++ -python -modern -new_repr -I../include/python
> -I../include/python/docs -o extensions/ogr_wrap.cpp -outdir osgeo
> ..\include\ogr.i $(SWIG) -c++ -python -modern -new_repr -I../include/python
> -I../include/python/docs -o extensions/gnm_wrap.cpp -outdir osgeo
> ..\include\gnm.i $(SWIG) -c++ -python -modern -new_repr -I../include/python
> -I../include/python/docs -o extensions/gdal_array_wrap.cpp -outdir osgeo
> ..\include\gdal_array.i
> +set DISTUTILS_USE_SDK=1
> +set MSSdk=1
>  $(PYDIR)\python.exe setup.py build
>   cd ..
> 
> Is this a solution that would work across all branches and compilers?

Would indeed be a better solution that a documentation hint. CC'ing Tamas 
Szekeres if he 
sees any issue with that approach (since I stole the trick from his 
gisinternals build recipees)

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] gdal_translate precision error

[Marco De Nadai]

Dear gdal community,
I'm a new user of gdal and I recently had a problem with precision. I tried
to translate a big GeoTIFF to a smaller one, specifying the extent of the
window in this way:

gdal_translate -projwin_srs EPSG:4326 -epo -strict -a_nodata 0 -projwin
9.50 46.00 10.00 45.50 -of GTiff -co COMPRESS=LZW
2015_e009_n46_e010_n45.tif try.tif

Now try.tif has extent:

8.999648999638,44.9992232790750890 :
10.0006417207349152,46.0002160001700062


Is this normal? Because it's a pretty big error for me.


Then, you can see this error in the map https://imgur.com/a/CNx7U. The
white color is the original (source) GeoTIFF in background. The black color
is try.tif and the vertical line is the extent I specified in the command.


Can you help me?

Thanks



-- 
*Marco De Nadai*
*Ph.D. student at Fondazione Bruno Kessler (FBK) - *
*MobS Unit*
*University of Trento*
Via Sommarive, 18 - Povo
38123 Trento (TN) - Italy

E-mail: dena...@fbk.eu
LinkedIn: https://it.linkedin.com/in/marcodenadai

Latest research paper: The Death and Life of Great Italian Cities: A Mobile
Phone Data Perspective

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] python 2.7 bindings with trunk (2.3)

[Gregory, Matthew]

Thanks Even,

Even Rouault wrote:
> Define
> SET DISTUTILS_USE_SDK=1
> SET MSSdk=1
>  
> so that the active MSVC version is used when building the Python
> bindings. This is what is used in gisinternals builds & appveyor builds

That worked for me (although I would have never found it, so I appreciate your 
help).  I've put these two lines in gdal/swig/makefile.vc, e.g.

--- a/gdal/swig/makefile.vc
+++ b/gdal/swig/makefile.vc
@@ -20,6 +20,8 @@ python: gdalvars
 $(SWIG) -c++ -python -modern -new_repr -I../include/python 
-I../include/python/docs -o extensions/ogr_wrap.cpp -outdir osgeo 
..\include\ogr.i
 $(SWIG) -c++ -python -modern -new_repr -I../include/python 
-I../include/python/docs -o extensions/gnm_wrap.cpp -outdir osgeo 
..\include\gnm.i
 $(SWIG) -c++ -python -modern -new_repr -I../include/python 
-I../include/python/docs -o extensions/gdal_array_wrap.cpp -outdir osgeo 
..\include\gdal_array.i
+set DISTUTILS_USE_SDK=1
+set MSSdk=1
 $(PYDIR)\python.exe setup.py build
cd ..

Is this a solution that would work across all branches and compilers?

thanks, matt
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Compiling ECW 3.3 on MacOSX

[Even Rouault]

> Also do you think
> https://trac.osgeo.org/gdal/browser/branches/2.2/autotest/gdrivers/ecw.py
> is setup to test the 3.3 SDK nicely? If so I can verify the patch on LINUX
> and MacOSX with that.

The tests pass fine (*) for me on Linux with ECW SDK 3.3 + libecwj2-3.3.patch. 
No idea on Mac

(*) some are skipped since testing features / fixes of later SDK versions

Here's the output I get:

$ python ecw.py
  TEST: ecw_init ... success
  TEST: ecw_1 ... success
  TEST: ecw_2 ... success
  TEST: ecw_3 ... success
  TEST: ecw_4 ... success
  TEST: ecw_5 ... success
  TEST: ecw_6 ... success
  TEST: ecw_7 ... Warning 6: NITF only supports WGS84 geographic and UTM 
projections.

success
  TEST: ecw_8 ... success
  TEST: ecw_9 ... success
  TEST: ecw_10 ... success
  TEST: ecw_11 ... success
  TEST: ecw_12 ... success
  TEST: ecw_13 ... success
  TEST: ecw_14 ... success
  TEST: ecw_15 ... success
  TEST: ecw_16 ... success
  TEST: ecw_17 ... Diff at pixel (0, 0) : -5.00
Diff at pixel (1, 0) : 1.00
Diff at pixel (2, 0) : -2.00
Diff at pixel (3, 0) : -1.00
Diff at pixel (4, 0) : -3.00
Diff at pixel (5, 0) : -3.00
Diff at pixel (6, 0) : -1.00
Diff at pixel (9, 0) : 2.00
Diff at pixel (10, 0) : -3.00
Diff at pixel (13, 7) : -6.00
Max diff : 6
Number of diffs : 302
success
  TEST: ecw_18 ... success
  TEST: ecw_19 ... success
  TEST: ecw_20 ... success
  TEST: ecw_21 ... success
  TEST: ecw_22 ... success
  TEST: ecw_23 ... success
  TEST: ecw_24 ... success
  TEST: ecw_25 ... success
  TEST: ecw_26 ... success
  TEST: ecw_27 ... success
  TEST: ecw_28 ... success
  TEST: ecw_29 ... success
  TEST: ecw_30 ... success
  TEST: ecw_31 ... skip
  TEST: ecw_32 ... success
  TEST: ecw_33 ... success
  TEST: ecw_33_bis ... success
  TEST: ecw_34 ... skip
  TEST: ecw_35 ... success
  TEST: ecw_36 ... skip
  TEST: ecw_37 ... skip
  TEST: ecw_38 ... skip
  TEST: ecw_39 ... skip
  TEST: ecw_40 ... ERROR 1: An error has occurred: Error 86 "File is invalid or 
corrupt"  file "" 
line 0
ERROR 1: Cannot open data/stefan_full_rgba_ecwv3_meta.ecw which looks like a 
ECW 
format v3 file, that requires ECW SDK 5.0 or later
skip
  TEST: ecw_41 ... skip
  TEST: ecw_42 ... skip
  TEST: ecw_43 ... success
  TEST: ecw_44 ... skip
  TEST: ecw_45 ... success
  TEST: ecw_46 ... success
  TEST: ecw_47 ... skip
  TEST: ecw_48 ... success
  TEST: ecw_49 ... success
  TEST: ecw_online_1 ... success
  TEST: ecw_online_2 ... success
  TEST: ecw_online_4 ... As GDAL_DOWNLOAD_TEST_DATA environment variable is not 
defined, some tests relying on data to downloaded from the Web will be skipped
skip
  TEST: ecw_online_5 ... success
  TEST: ecw_online_6 ... success
  TEST: ecw_online_7 ... success
  TEST: ecw_cleanup ... success

Test Script: ecw
Succeeded: 46
Failed:0 (0 blew exceptions)
Skipped:   12
Expected fail:0
Duration:  13.19s
As GDAL_DOWNLOAD_TEST_DATA environment variable is not defined, 1 tests relying 
on 
data to downloaded from the Web have been skipped



-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Mateusz Loskot]

On 5 January 2018 at 21:43, Kurt Schwehr  wrote:
> My preference (and not speaking for the gdal community) for C++ classes
> would be:
>
> 1. replace const char * -> const std::string &
> 2. replace CPLString -> std::string

+1

> Then in about 2022, we can see about std::string_view :(

+1

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Kurt Schwehr]

My preference (and not speaking for the gdal community) for C++ classes
would be:

1. replace const char * -> const std::string &
2. replace CPLString -> std::string

std::string GetString(const std::string &soName, const std::string
&soDefault = "") const;

This is where it would be good to get input from others.

I base the above on maximizing safety while trying to let the compiler do
its best job at optimizing.  Then in about 2022, we can see about
std::string_view :(

On Fri, Jan 5, 2018 at 12:26 PM, Dmitry Baryshnikov 
wrote:

> Hi Kurt,
>
> Can you explain what should be done in PR?
>
> Do you mean to replace all const char* to?
>
> 1. const char* -> const CPLString &
>
> const char *GetString(const char *pszName, const char *pszDefault = "")
> const; ->
>
> CPLString GetString(const CPLString &soName, const CPLString &soDefault =
> "") const;
>
> or
>
> 2. const char* -> const std::string &
>
> const char *GetString(const char *pszName, const char *pszDefault = "")
> const; ->
>
> std::string GetString(const std::string &soName, const std::string
> &soDefault = "") const;
>
> or?
>
> Best regards,
> Dmitry
>
> 05.01.2018 18:54, Kurt Schwehr пишет:
>
> +1 for wrapping the old C code in some cleaner abstractions!
>
> But +10 for switching to a from the ground up C++ JSON library unless there
> are clear reasons for a core C library (I don't think there are)
>
> If we are talking about this kind of code, there are several things that
> have bugged me in general about GDAL for a long time.
>
> * Passing char *psz yada all over the place in pure C++ code.  A const
> std::string is usually not a noticeable expense and is a lot safer
> * CPLString when std::string will do just fine.  And we can write free
> functions to operate on strings.  I'm generally bothered by subclassing of
> std::string as CPLString.  After reading large amounts of C++ code, I think
> it adds more confusion than it ever helps over having clean free
> functions.  Interop and analysis with CPLString's is no fun.
> https://stackoverflow.com/questions/6006860/why-should-one-not-derive-from-c-std-string-class
>
> -kurt
>
> On Fri, Jan 5, 2018 at 7:44 AM, Sean Gillies  
>  wrote:
>
>
> Hi Dmitry,
>
> I scanned the PR and it seems reasonable to me. I'm barely a C++
> programmer at all and it's clear to me, more clear than before. That said,
> I'm not a fan of wrapping things that could be replaced. Have you looked
> into whether a more performant C++ JSON library could be used? I haven't
> run the benchmarks, but json-c compares pretty poorly to others 
> inhttps://github.com/miloyip/nativejson-benchmark.
>
>
> On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov  
> 
> wrote:
>
>
> Hi everybody,
>
> Happy new year and lot of success in 2018!
>
> I would like to discuss my pull request https://github.com/OSGeo/gdal/
> pull/282
>
> I created a thin wrapper around the json-c library which wide using in
> GDAL.
>
> This is C++ interface which hides C memory management and provides nice
> API. The web or disk json documents reading chunk by chunk with progress
> indication also added.
>
> In future, the json-c can be easily switch to something other without
> breaking the existing code.
>
> The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
> found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.
>
> Is this ready to merge into the trunk? Any objections?
>
> Best regards,
> Dmitry
>
>
> ___
> gdal-dev mailing 
> listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> --
> Sean Gillies
>
> ___
> gdal-dev mailing 
> listgdal-dev@lists.osgeo.orghttps://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>


-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Dmitry Baryshnikov]

Hi Kurt,

Can you explain what should be done in PR?

Do you mean to replace all const char* to?

1. const char* -> const CPLString &

   const char *GetString(const char *pszName, const char *pszDefault =
   "") const; ->

   CPLString GetString(const CPLString &soName, const CPLString
   &soDefault = "") const;

or

2. const char* -> const std::string &

   const char *GetString(const char *pszName, const char *pszDefault =
   "") const; ->

   std::string GetString(const std::string &soName, const std::string
   &soDefault = "") const;

or?

Best regards,
Dmitry

05.01.2018 18:54, Kurt Schwehr пишет:

+1 for wrapping the old C code in some cleaner abstractions!

But +10 for switching to a from the ground up C++ JSON library unless there
are clear reasons for a core C library (I don't think there are)

If we are talking about this kind of code, there are several things that
have bugged me in general about GDAL for a long time.

* Passing char *psz yada all over the place in pure C++ code.  A const
std::string is usually not a noticeable expense and is a lot safer
* CPLString when std::string will do just fine.  And we can write free
functions to operate on strings.  I'm generally bothered by subclassing of
std::string as CPLString.  After reading large amounts of C++ code, I think
it adds more confusion than it ever helps over having clean free
functions.  Interop and analysis with CPLString's is no fun.

https://stackoverflow.com/questions/6006860/why-should-one-not-derive-from-c-std-string-class

-kurt

On Fri, Jan 5, 2018 at 7:44 AM, Sean Gillies  wrote:


Hi Dmitry,

I scanned the PR and it seems reasonable to me. I'm barely a C++
programmer at all and it's clear to me, more clear than before. That said,
I'm not a fan of wrapping things that could be replaced. Have you looked
into whether a more performant C++ JSON library could be used? I haven't
run the benchmarks, but json-c compares pretty poorly to others in
https://github.com/miloyip/nativejson-benchmark.


On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov 
wrote:


Hi everybody,

Happy new year and lot of success in 2018!

I would like to discuss my pull request https://github.com/OSGeo/gdal/
pull/282

I created a thin wrapper around the json-c library which wide using in
GDAL.

This is C++ interface which hides C memory management and provides nice
API. The web or disk json documents reading chunk by chunk with progress
indication also added.

In future, the json-c can be easily switch to something other without
breaking the existing code.

The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.

Is this ready to merge into the trunk? Any objections?

Best regards,
 Dmitry


___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev




--
Sean Gillies

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev






___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Dmitry Baryshnikov]

Hi Sean,

First of all I agreed with Kurt. Json-c is GDAL internal library and had 
inspected by fuzzer.


Also json-c widely used in GDAL:

gdal/gcore
gdal/ogr/ogrsf_frmts/carto
gdal/ogr/ogrsf_frmts/amigocloud
gdal/ogr/ogrsf_frmts/cloudant
gdal/ogr/ogrsf_frmts/plscenes
gdal/ogr/ogrsf_frmts/geojson
gdal/ogr/ogrsf_frmts/gmlas
gdal/ogr/ogrsf_frmts/elastic
gdal/ogr/ogrsf_frmts/couchdb
gdal/ogr
gdal/frmts/rda
gdal/frmts/arg
gdal/frmts/mbtiles
gdal/frmts/plmosaic
gdal/frmts/pds

It's big work to replace json-c in all code smoothly.

Best regards,
Dmitry

05.01.2018 19:05, Kurt Schwehr пишет:

I think the more important factors than speed for a C++ lib are: security,
stability, maintainability, and memory usage (low stack usage and
constrainable heap).

I really don't want to have us go through yet more piles of fuzzer bugs for
a library we depend on.  It would be nice to have that already done well by
the lib.

On Fri, Jan 5, 2018 at 7:54 AM, Kurt Schwehr  wrote:


+1 for wrapping the old C code in some cleaner abstractions!

But +10 for switching to a from the ground up C++ JSON library unless
there are clear reasons for a core C library (I don't think there are)

If we are talking about this kind of code, there are several things that
have bugged me in general about GDAL for a long time.

* Passing char *psz yada all over the place in pure C++ code.  A const
std::string is usually not a noticeable expense and is a lot safer
* CPLString when std::string will do just fine.  And we can write free
functions to operate on strings.  I'm generally bothered by subclassing of
std::string as CPLString.  After reading large amounts of C++ code, I think
it adds more confusion than it ever helps over having clean free
functions.  Interop and analysis with CPLString's is no fun.
   https://stackoverflow.com/questions/6006860/why-should-
one-not-derive-from-c-std-string-class

-kurt

On Fri, Jan 5, 2018 at 7:44 AM, Sean Gillies  wrote:


Hi Dmitry,

I scanned the PR and it seems reasonable to me. I'm barely a C++
programmer at all and it's clear to me, more clear than before. That said,
I'm not a fan of wrapping things that could be replaced. Have you looked
into whether a more performant C++ JSON library could be used? I haven't
run the benchmarks, but json-c compares pretty poorly to others in
https://github.com/miloyip/nativejson-benchmark.


On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov 
wrote:
Hi everybody,

Happy new year and lot of success in 2018!

I would like to discuss my pull request https://github.com/OSGeo/gdal/
pull/282

I created a thin wrapper around the json-c library which wide using in
GDAL.

This is C++ interface which hides C memory management and provides nice
API. The web or disk json documents reading chunk by chunk with progress
indication also added.

In future, the json-c can be easily switch to something other without
breaking the existing code.

The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.

Is this ready to merge into the trunk? Any objections?

Best regards,
 Dmitry




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Compiling ECW 3.3 on MacOSX

[Jeremy Palmer]

Hi Even,

Happy New Year :)

On Sat, Jan 6, 2018 at 2:12 AM, Even Rouault 
wrote:

> Hi Jeremy,
>
>
>
> I see that your gist has some of the fixes of libecwj2-3.3.patch, and a
> lot more (but some are not so obvious without digging more in the code)
>

I now have an internal git repo with both the libecwj2-3.3.patch and a
selection of the MacOSX fixes provided by John. Once I have done further
real world testing I can provide a new accumulative patch.

Also do you think
https://trac.osgeo.org/gdal/browser/branches/2.2/autotest/gdrivers/ecw.py
is setup to test the 3.3 SDK nicely? If so I can verify the patch on LINUX
and MacOSX with that.

Cheers
Jeremy
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Compiling ECW 3.3 on MacOSX

[John Daniel]

As I explained to Jeremy in an e-mail, I made those changes years ago. I don’t 
remember the exact details. I do know that those are all specific to OS X, and 
specifically to OS X and the Xcode toolchain circa 2011 (maybe?). Things are 
much different today. I must have needed those changes to make it work on the 
Mac. I did use this code to process a fair amount of JPEG2000-encoded NITF 
data. It seemed to work fine at the time.

However, I never applied these changes to any production Linux builds. They 
were only for the Mac and I only used the Mac for development. I would not 
recommend using this patch for any real data.

As soon as OpenJPEG became viable, I switched to that. Plus that ECW code has 
questionable licensing - real mess. Unfortunately, Jeremy has actual ECW files. 
I never had any of those.

However, you could use this patch to get a functional GDAL build with ECW and 
run tests. Then try to back out some of the more mysterious changes. That is 
always easier to do when you have it running and testable.

John Daniel
Etresoft, Inc.

On Jan 5, 2018, at 8:12 AM, Even Rouault 
mailto:even.roua...@spatialys.com>> wrote:

Hi Jeremy,



> https://gist.github.com/palmerj/54fa389c4e60fa7b9f1d5ceea7ac3359



Note that https://trac.osgeo.org/gdal/wiki/ECW references
https://trac.osgeo.org/gdal/attachment/wiki/ECW/libecwj2-3.3.patch which is the 
cumlated patch of a number of patches that have been collected over the years 
(for Linux builds)



I see that your gist has some of the fixes of libecwj2-3.3.patch, and a lot 
more (but some are not so obvious without digging more in the code)



This ECW situation is such a mess. Sigh



Even



--
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] python 2.7 bindings with trunk (2.3)

[Even Rouault]

On vendredi 5 janvier 2018 17:51:26 CET Gregory, Matthew wrote:
> Hi all,
> 
> I'm sure to show my ignorance here, but I've been periodically building both
> the 2.2 and trunk branches on Windows using MSVC for Python 2.7 (as I think
> the compiler version needs to match how Python itself was built [MSC v.1500
> 32 bit (Intel)]).  With the recent update in trunk requiring C++11, MSVC
> for Python 2.7 now errors out:
> 
>   c:\foss4g\gdal_trunk\gdal\port\cpl_port.h(187) : fatal error C1189:
> #error :  Must have C++11 or newer.
> 
> Is there a way to compile the GDAL python bindings with still using MSVC for
> Python 2.7 -or- if I update to MSVC 14, can I still build bindings that
> will work with 2.7?

Define
 SET DISTUTILS_USE_SDK=1
 SET MSSdk=1

so that the active MSVC version is used when building the Python bindings. This 
is what is 
used in gisinternals builds & appveyor builds

We should likely document that somewhere appropriate. Perhaps in
https://trac.osgeo.org/gdal/wiki/BuildingOnWindows

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] python 2.7 bindings with trunk (2.3)

[Gregory, Matthew]

Hi all,

I'm sure to show my ignorance here, but I've been periodically building both 
the 2.2 and trunk branches on Windows using MSVC for Python 2.7 (as I think the 
compiler version needs to match how Python itself was built [MSC v.1500 32 bit 
(Intel)]).  With the recent update in trunk requiring C++11, MSVC for Python 
2.7 now errors out:

  c:\foss4g\gdal_trunk\gdal\port\cpl_port.h(187) : fatal error C1189:
#error :  Must have C++11 or newer.

Is there a way to compile the GDAL python bindings with still using MSVC for 
Python 2.7 -or- if I update to MSVC 14, can I still build bindings that will 
work with 2.7?

Sorry if I've missed discussion around this topic.  I reviewed RFC 68, but 
didn't see any reference to this issue.

thanks, matt
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Kurt Schwehr]

I think the more important factors than speed for a C++ lib are: security,
stability, maintainability, and memory usage (low stack usage and
constrainable heap).

I really don't want to have us go through yet more piles of fuzzer bugs for
a library we depend on.  It would be nice to have that already done well by
the lib.

On Fri, Jan 5, 2018 at 7:54 AM, Kurt Schwehr  wrote:

> +1 for wrapping the old C code in some cleaner abstractions!
>
> But +10 for switching to a from the ground up C++ JSON library unless
> there are clear reasons for a core C library (I don't think there are)
>
> If we are talking about this kind of code, there are several things that
> have bugged me in general about GDAL for a long time.
>
> * Passing char *psz yada all over the place in pure C++ code.  A const
> std::string is usually not a noticeable expense and is a lot safer
> * CPLString when std::string will do just fine.  And we can write free
> functions to operate on strings.  I'm generally bothered by subclassing of
> std::string as CPLString.  After reading large amounts of C++ code, I think
> it adds more confusion than it ever helps over having clean free
> functions.  Interop and analysis with CPLString's is no fun.
>   https://stackoverflow.com/questions/6006860/why-should-
> one-not-derive-from-c-std-string-class
>
> -kurt
>
> On Fri, Jan 5, 2018 at 7:44 AM, Sean Gillies  wrote:
>
>> Hi Dmitry,
>>
>> I scanned the PR and it seems reasonable to me. I'm barely a C++
>> programmer at all and it's clear to me, more clear than before. That said,
>> I'm not a fan of wrapping things that could be replaced. Have you looked
>> into whether a more performant C++ JSON library could be used? I haven't
>> run the benchmarks, but json-c compares pretty poorly to others in
>> https://github.com/miloyip/nativejson-benchmark.
>>
>>
>> On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov > > wrote:
>>
>>> Hi everybody,
>>>
>>> Happy new year and lot of success in 2018!
>>>
>>> I would like to discuss my pull request https://github.com/OSGeo/gdal/
>>> pull/282
>>>
>>> I created a thin wrapper around the json-c library which wide using in
>>> GDAL.
>>>
>>> This is C++ interface which hides C memory management and provides nice
>>> API. The web or disk json documents reading chunk by chunk with progress
>>> indication also added.
>>>
>>> In future, the json-c can be easily switch to something other without
>>> breaking the existing code.
>>>
>>> The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
>>> found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.
>>>
>>> Is this ready to merge into the trunk? Any objections?
>>>
>>> Best regards,
>>> Dmitry
>>>
>>>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Kurt Schwehr]

+1 for wrapping the old C code in some cleaner abstractions!

But +10 for switching to a from the ground up C++ JSON library unless there
are clear reasons for a core C library (I don't think there are)

If we are talking about this kind of code, there are several things that
have bugged me in general about GDAL for a long time.

* Passing char *psz yada all over the place in pure C++ code.  A const
std::string is usually not a noticeable expense and is a lot safer
* CPLString when std::string will do just fine.  And we can write free
functions to operate on strings.  I'm generally bothered by subclassing of
std::string as CPLString.  After reading large amounts of C++ code, I think
it adds more confusion than it ever helps over having clean free
functions.  Interop and analysis with CPLString's is no fun.

https://stackoverflow.com/questions/6006860/why-should-one-not-derive-from-c-std-string-class

-kurt

On Fri, Jan 5, 2018 at 7:44 AM, Sean Gillies  wrote:

> Hi Dmitry,
>
> I scanned the PR and it seems reasonable to me. I'm barely a C++
> programmer at all and it's clear to me, more clear than before. That said,
> I'm not a fan of wrapping things that could be replaced. Have you looked
> into whether a more performant C++ JSON library could be used? I haven't
> run the benchmarks, but json-c compares pretty poorly to others in
> https://github.com/miloyip/nativejson-benchmark.
>
>
> On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov 
> wrote:
>
>> Hi everybody,
>>
>> Happy new year and lot of success in 2018!
>>
>> I would like to discuss my pull request https://github.com/OSGeo/gdal/
>> pull/282
>>
>> I created a thin wrapper around the json-c library which wide using in
>> GDAL.
>>
>> This is C++ interface which hides C memory management and provides nice
>> API. The web or disk json documents reading chunk by chunk with progress
>> indication also added.
>>
>> In future, the json-c can be easily switch to something other without
>> breaking the existing code.
>>
>> The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
>> found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.
>>
>> Is this ready to merge into the trunk? Any objections?
>>
>> Best regards,
>> Dmitry
>>
>>
>> ___
>> gdal-dev mailing list
>> gdal-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
>
> --
> Sean Gillies
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Sean Gillies]

Hi Dmitry,

I scanned the PR and it seems reasonable to me. I'm barely a C++ programmer
at all and it's clear to me, more clear than before. That said, I'm not a
fan of wrapping things that could be replaced. Have you looked into whether
a more performant C++ JSON library could be used? I haven't run the
benchmarks, but json-c compares pretty poorly to others in
https://github.com/miloyip/nativejson-benchmark.


On Wed, Jan 3, 2018 at 12:04 PM, Dmitry Baryshnikov 
wrote:

> Hi everybody,
>
> Happy new year and lot of success in 2018!
>
> I would like to discuss my pull request https://github.com/OSGeo/gdal/
> pull/282
>
> I created a thin wrapper around the json-c library which wide using in
> GDAL.
>
> This is C++ interface which hides C memory management and provides nice
> API. The web or disk json documents reading chunk by chunk with progress
> indication also added.
>
> In future, the json-c can be easily switch to something other without
> breaking the existing code.
>
> The CPLJSONDocument/CPLJSONObject/CPLJSONArray usage examples can be
> found in frmts/pds driver and c++ unit test in autotest/cpp/test_cpl.cpp.
>
> Is this ready to merge into the trunk? Any objections?
>
> Best regards,
> Dmitry
>
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev




-- 
Sean Gillies
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] CPLJSONDocument

[Even Rouault]

Dmitry,

> Is this ready to merge into the trunk? Any objections?

I did some review of your code. If nobody has extra remarks in the 
next days, just merge it.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Prevent extrapolating with ReadAsArray when resampling

[Even Rouault]

Rutger,

ReadAsArray() / IRasterIO() with non-nearest resampling use the interpolation 
code of 
overview building (except for Lanczos where it uses the warping code), which 
was of course 
targetted at downsampling rather than upsampling. Nodata is taken into account 
when 
mixing together source pixels for a target pixel (those pixels are assigned a 
zero weight in 
convolution computations), but it seems that we should likely skip entirely a 
target pixel if 
the center source pixel corresponding to it is nodata, so that nodata areas do 
not fade 
away (which is done in the warping code). What surprises me is that you get 
different 
results depending on the nodata value (I could also see that)

Regarding Lanczos, might be a bug specific to it indeed. Didn't look.

Perhaps file this as a ticket about this.

Even

> Dear list,
> 
> I regularly use (and enjoy!) the resampling option when using the
> ReadAsArray method of a Dataset. However, i noticed that in the case of
> nodata values in the raster, the method extrapolates "into" those nodata
> values. This is not always desirable in my case, especially when using any
> resampling algorithms other than Bilinear or Nearest, since these can
> overshoot significantly.
> 
> This made me wondering if there is a way to prevent this extrapolation? One
> thing i noticed it that when the nodata value is non-numerical, like NaN,
> there is a lot less extrapolation (eg more nodata values returned). That
> could be used a workaround, but it not convenient to map numerical nodata to
> NaN values before reading the input.
> 
> In the process of exploring the behavior, i also encountered some issues
> (perhaps a bug?) when using Lanczos in combination with certain datasets and
> certain output grids. This happened especially when the output size is an
> uneven factor of the input size, like (buf_xsize=xsize*3,
> buf_ysize=ysize*3) or (buf_xsize=xs*5, buf_ysize=ys*5) etc.
> 
> Here is a notebook showing what i did with a linearly increasing grid:
> https://gist.github.com/RutgerK/30fbd03d9b2ab79861eedfbb95064732
> 
> And a random grid:
> https://gist.github.com/RutgerK/bdbd6c124224cf49961082e1620ce604
> 
> Regards,
> Rutger
> 
> 
> 
> 
> 
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Compiling ECW 3.3 on MacOSX

[Even Rouault]

Hi Jeremy,

> https://gist.github.com/palmerj/54fa389c4e60fa7b9f1d5ceea7ac3359

Note that https://trac.osgeo.org/gdal/wiki/ECW references
https://trac.osgeo.org/gdal/attachment/wiki/ECW/libecwj2-3.3.patch which is the 
cumlated patch of a number of patches that have been collected over the years 
(for 
Linux builds)

I see that your gist has some of the fixes of libecwj2-3.3.patch, and a lot 
more (but 
some are not so obvious without digging more in the code)

This ECW situation is such a mess. Sigh

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] /vsicurl caching behavior

[Even Rouault]

> OK, I understand now the VSICachedFile lifecycle and it makes sense. But
> then the disk cache should not be used at all for files in
> the CPL_VSIL_CURL_NON_CACHED variable.

Indeed. The cache disk code hasn't been touched/tested in years, and 
CPL_VSIL_CURL_NON_CACHED was added afterwards.

> 
> I'll try to find some happy hacking time (or client funding, ideally) to
> improve the disk caching. What is the official GDAL procedure for PRs? I
> see a few in the github repo, but the doc says the source is located in SVN.

The master is still in SVN (we will likely switch to full github workflow at 
some point) but 
contributions being sent as pull request against the github mirror make it 
easier for the 
person reviewing & merging them.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] /vsicurl caching behavior

[Patrick Valsecchi]

Hi Even,

Long time no see!

On Fri, Jan 5, 2018 at 12:21 PM, Even Rouault 
wrote:

> > 8) In the case discussed in 7), CPL_VSIL_CURL_NON_CACHED will just purge
>
> > the data from 1 the 3 caches: papsRegions. The vsil_cache and the disk
> will
>
> > still cache the content.
>
>
>
> CPL_VSIL_CURL_NON_CACHED avoids the content of a file to be preserved in
> the papsRegion cache when a file handle is closed and re-opened. And
> VSICachedFile is only valid during the lifetime of the file handle. So I
> don't think there's an issue there. Perhaps the naming
> CPL_VSIL_CURL_NON_CACHED is a bit misleading: there's always some cache
> (otherwise /vsicurl performance would be just too horrible), it is just
> that it doesn't survive file closing.
>
>
OK, I understand now the VSICachedFile lifecycle and it makes sense. But
then the disk cache should not be used at all for files in
the CPL_VSIL_CURL_NON_CACHED variable.

I'll try to find some happy hacking time (or client funding, ideally) to
improve the disk caching. What is the official GDAL procedure for PRs? I
see a few in the github repo, but the doc says the source is located in SVN.

Thanks.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] /vsicurl caching behavior

[Even Rouault]

Hi,

> 1) There are a lot of knobs that can be used to tune the thing that are not
> documented. For example CPL_VSIL_CURL_USE_CACHE. Is it on purpose?

Yes, the disk cache is an experiment that isn't used anywhere (from what I 
know) and likely 
not in a finished state as you noticed in your below points which are all valid 
and should be 
addressed if someone wanted to make it production ready.

> 
> 6) If VSI_CACHE is enabled the data is cached twice in memory (papsRegions
> and VSICachedFile). Is it wanted?

The scope of the caches are not the same. papsRegions is a global cache shared 
by all /
vsicurl/ handles, and persistant (in memory) on their closing (so that if the 
same filename is 
closed and re-opened in sequence, already read parts can be reused), whereas 
VSICachedFile 
is associated with a single file handle.
I guess there could be some optimizations to avoid those duplications, but that 
could 
complicate substantially the code which is already non trivial.

> 
> 7) If the file's content is modified, it's the total mess. We'll end up
> having portions of the file having the old data while the rest has the new
> data. I'm quite sure the GeoTiff we end up with won't be very valid.

Indeed. But the mess would also happen with no caching mechanism if a file is 
modified while 
being read. Even for a local file, GDAL using glibc FILE buffering API, so even 
if you modify 
some portions of a GeoTIFF that haven't been read yet, but you already read 
closing regions, 
there's a chance, you'll read old data in part.

> 
> 8) In the case discussed in 7), CPL_VSIL_CURL_NON_CACHED will just purge
> the data from 1 the 3 caches: papsRegions. The vsil_cache and the disk will
> still cache the content.

CPL_VSIL_CURL_NON_CACHED avoids the content of a file to be preserved in the 
papsRegion cache when a file handle is closed and re-opened. And VSICachedFile 
is only valid 
during the lifetime of the file handle. So I don't think there's an issue 
there. Perhaps the 
naming CPL_VSIL_CURL_NON_CACHED is a bit misleading: there's always some cache 
(otherwise /vsicurl performance would be just too horrible), it is just that it 
doesn't survive 
file closing.

Even


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev