Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Even Rouault]

> I misinterpreted this thread because of the title.  It seems this isn't
> really about code style or C++ 11.  

I agree this thread mixes different topics, which causes some confusion. The 
word C++11 appearing in a doc has created some passion, whereas it is probably 
not the heart of the subject.

Kurt has contributed to a lot of cleanup in the code base over the last 
months, mostly in applying stricter formatting rules, bool'ifying, 
const'ifying when possible, moving variable declarations close to their 
initialization, etc.. (Kurt, correct me if I'm wrong.), also addressing a 
bunch of Covertiy Scan warnings, etc.

I've actually encouraged him to share his thoughts regarding cleanups with the 
rest of the community so that we can decide which are worth to be rules that 
apply to the project as a whole, and if some tooling is needed to 
automate/enforce them.

> It's really about Google wanting to use
> heap over stack because of its particular use of the library.  There is no
> need for C++ 11 for that and it would seem that doing a compile-time
> policy-based array isn't too hard (proof of concept below).  Perhaps Google
> could flush something like this out to its liking and replace current stack
> allocations that cause it issues. 

This use case is probably shared by other actors that use GDAL for massive 
data processing, and if the solution to reduce stack usage doesn't hurt more 
mundane use cases, we probably don't need the #ifdef complication.

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Andrew Bell]

On Thu, May 5, 2016 at 10:19 AM, Kurt Schwehr  wrote:

> Thanks all for your comments!  I will try to write up a similar proposal
> for C++11 based on what people have written.
>
> This local stack storage change is for a real need, so it's probably good
> that I add a use case that explains why this change is important.  And I
> >>believe<< (but am not 100% sure) that using std::vector is the strongest
> solution for the majority of instances of large buffers on the stack in
> gdal.
>
> Use case:  (note that I can only share generals)
>
> At Google, we have jobs that use GDAL running with thousands of cores and
> >20 threads per process that run for days.  The overhead of having to allow
> for bumping the stack size adds substantial overhead to all those jobs.
> GDAL is the only thing driving the requirement for the larger stack.  For
> people using GDAL on a desktop machine, the larger stack is unlikely to be
> a real issue.  For anyone running large multithreaded jobs in the cloud,
> the large stack is a real cost issue.  The overhead of vector and heap
> allocation in all these cases I've looked at in GDAL so far is far smaller
> than the cost from the stack size overhead.
>

I misinterpreted this thread because of the title.  It seems this isn't
really about code style or C++ 11.  It's really about Google wanting to use
heap over stack because of its particular use of the library.  There is no
need for C++ 11 for that and it would seem that doing a compile-time
policy-based array isn't too hard (proof of concept below).  Perhaps Google
could flush something like this out to its liking and replace current stack
allocations that cause it issues.  C++ 11 and code style seem another
conversation.

==

#include 

struct Stack
{};

struct Heap
{};

template 
struct Arr
{
};

template 
struct Arr
{
Arr() : m_instance(new TYPE[SIZE ? SIZE : 1])
{}

~Arr()
{
delete [] m_instance;
}

TYPE *m_instance;

TYPE& operator[](std::size_t idx)
{ return *(m_instance + idx); }
};

template 
struct Arr
{
TYPE m_instance[SIZE ? SIZE : 1];

TYPE& operator[](std::size_t idx)
{ return m_instance[idx]; }
};


#ifdef HEAPALLOC
template 
struct Array : public Arr
{};
#else
template 
struct Array : public Arr
{};
#endif

int main()
{
Array a;

a[10] = 15;
std::cout << "A[10] = " << a[10] << "!\n";

return 0;
}

-- 
Andrew Bell
andrew.bell...@gmail.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Ray Gardener]

How about C++11 threads?

Ray


On 5/5/2016, Thursday 12:43 PM, Mark Coletti wrote:
On Thu, May 5, 2016 at 11:15 AM, Even Rouault 
> wrote:


[...]
That would be interesting if you (or anyone) could list which
C++11 features would be killer features
to justify the upgrade to C++11 vs the potential pains that such a
move might cause (especially
as I raised in other emails, given the fact that GDAL uses a
number of other libs and is used by a number
of other libs) [...]


This is an off the cuff list of C++-11 niceties, and by no means 
exhaustive:


- std::move()
- *lambda expressions*
- auto keyword
- native smart pointer support, such as std::unique_ptr and 
std::shared_ptr (though there's always the Boost fallback)

- concurrency API
- nullptr

Also, it's been many, many years since I last looked at GDAL/OGR 
source.  When I did I noted a lot of C-style I/O calls instead of 
using C++ equivalents.  Is that still the case?  If so, that's one 
area ripe for refactoring.


Cheers,

Mark
--
mcole...@gmail.com 



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Mark Coletti]

On Thu, May 5, 2016 at 11:15 AM, Even Rouault 
wrote:

> [...]
> That would be interesting if you (or anyone) could list which C++11
> features would be killer features
> to justify the upgrade to C++11 vs the potential pains that such a move
> might cause (especially
> as I raised in other emails, given the fact that GDAL uses a number of
> other libs and is used by a number
> of other libs) [...]


This is an off the cuff list of C++-11 niceties, and by no means exhaustive:

- std::move()
- *lambda expressions*
- auto keyword
- native smart pointer support, such as std::unique_ptr and std::shared_ptr
(though there's always the Boost fallback)
- concurrency API
- nullptr


Also, it's been many, many years since I last looked at GDAL/OGR source.
When I did I noted a lot of C-style I/O calls instead of using C++
equivalents.  Is that still the case?  If so, that's one area ripe for
refactoring.

Cheers,

Mark
-- 
mcole...@gmail.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[David Strip]

  
  
On 5/5/2016 9:00 AM, Kurt Schwehr
  wrote:

Thanks!  I've integrated your derived class in the
  alternates section and Even's response about commenting on resize
  into the drawbacks line
  
  
  Can you provide (on the list would be best) a bit more on why
the vector suggestion makes the it less tempting to do pointer
tricks?


First I want to make clear that use of std::vector doesn't prevent
using pointer tricks. At least for me (and many others I've worked
with) the fact that something has been defined as a std::vector
creates a mental barrier to using an interface other than that
declared in std::vector. To a large degree the strength of this
barrier will reflect the programmer's feelings towards
object-orientation and all that goes with it. If you're an
old-school C programmer who thinks all this new C++ "O-O crap" is
for sissies, then the first thing you'll do is grab the address of
element zero and use all the nasty, obscure coding style you've
always used. (Are my prejudices showing? And for what it's worth, I
certainly qualify as old school by virtue of age, but like to think
I've learned something along the way.) If, on the other hand, you
actually program in C++ (as opposed to using the C++ compiler on C
code), you'll look at the object as a vector and use the interface.

Here's what I mean by "pointer tricks" - 

Imagine you have to do some operations on the diagonal of a square
matrix. Consider the readability of the two snippets:

// I'm assuming we defined our array as a vector of
  vectors.
  for (int i = 0; i < matrix_size;  ++i)
  {
      do something(the_matrix[i][i]); 
  }


vs.

//Since I would never do this myself, a "real C
  programmer" would almost certainly write this differently, esp.
  the test for the end of the loop, but you get the idea
  for (int * i = the_matrix; i < sizeof(the_matrix) +
  the_matrix;  i += matrix_size + 1))
  {
      do_something(i);
  }


naming the loop var as diag_element would help, but in the first
example it's clear that do_something() is operating on a diagonal
element. In the second example it's not clear what you're operating
on at all. 

  

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Damian Dixon]

I'm currently stuck on the following compilers:

   - Visual Studio 2010 SP1
   - GCC 4.7

C++11 is only partially supported with these compilers.

Regards
Damian


On 4 May 2016 at 23:51, Mateusz Loskot  wrote:

> On 4 May 2016 at 23:30, Kurt Schwehr  wrote:
>
>>
>> To start off the conversation, I wrote up a doc on changing large C
>> arrays on the stack to std::vectors to get this data off of the stack and
>> to simplify initialization.
>>
>
>
> Since many, if not most, of the ideas rely on availability of the C++11
> features,
> it might be a good idea to first agree on C++11 as the lowest
> required C++ compilation mode.
>
> Let's poll if there are any users who require C++03 at all.
>
> Best regards,
> --
> Mateusz Loskot, http://mateusz.loskot.net
>
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Damian Dixon]

In the VS World...

If you delete C/C++ memory/objects in a DLL that has been compiled with the
same VS compiler you are ok.

If you attempt to delete a memory/object in DLL/application compiled with a
different version of the VS compiler all bets are off.

For example:

Allocate an object in VS2010 compiled code delete in VS2015 compiled code,
you will likely crash.


You will have to distribute the runtime distribution DLLs for the versions
of VS that you used or your 3rd party DLLs used.

Regards
Damian


On 5 May 2016 at 10:28, Even Rouault  wrote:

> Le jeudi 05 mai 2016 11:09:48, Mateusz Loskot a écrit :
> > On 5 May 2016 at 01:45, Even Rouault  wrote:
> > > Of the potential issues with requiring C++11, I can think of OSGeo4W.
> It
> > > is mostly(completely?) built with Visual Studio 2010. And from
> > > https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of
> C++11
> > > is only partial in VS 2010
> >
> > VS2013 is the lowest version sensible to consider regarding C++11
> support.
> >
> > > On the other hand regarding dependencies of GDAL, the binary propritary
> > > SDKs with a C++ API could be a problem, although they will likely move
> > > on too. - FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
> > > - ECW SDK 5.2.1: available for VS2010, VS2012, VS2013
> >
> > AFAICT, GDAL built using VS2015 links against the 5.2.1 version fine.
>
> Do you link against the VS2013 .lib ?
> >
> > > - MrSID SDK: I didn't check. Perhaps Kirk can tell us ?
> >
> > Similarly to the ECW above, version 9.1.0 links fine too.
> >
> > > But I'm not sure about the compatibility of C++11 build against
> non-C++11
> > > builds in the VS realm : can a GDAL C++11 build link against a library
> > > built without C++11 enabled ? Will not there be ABI problems ?
> >
> > Although mixing C run-time libraries is a bad idea generally,
> > kosher C APIs should be fine.
>
> I'm not too worried about C ABI. My mention of C++03 ABI vs C++11 ABI was
> for
> the dependencies with those SDK that we use through their C++ API. In GCC
> world, with same compiler but with different -std=... flags, there have
> been
> troubles ( https://gcc.gnu.org/wiki/Cxx11AbiCompatibility ,
> https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html ), so
> I
> was wondering how the situation is in the VS world.
> Actually since I mention this, the issue does apply to Linux too. Looking
> at
> the C++11 Linux build (
>
> https://github.com/rouault/gdal_coverage/blob/trunk_gcc4.8_stdc11/.travis.yml
> ) , I see I couldn't build against FileGDB with -std=c++11. This was a
> compilation issue with its headers, rather than a link/runtime issue.
>
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Andrew Bell]

On Thu, May 5, 2016 at 10:15 AM, Even Rouault 
wrote:

>
> > As for the notion of replacing some allocations with vectors, well,
> great.
> > But the GDAL codebase is creaky in lots of places and to get too
> particular
> > about the advantage/disadvantage of vectors seems a little silly.  At
> some
> > point it's just reasonable to say that you're only going to support
> > compilers/systems made after some date/version.  People who don't want to
> > upgrade a system aren't harmed.  They always have access to an older
> > version of the library.
>
> (Sometimes they want the latest version of the lib for a particular
> feature on a ancient system.)
>

Yes, but I'm unsympathetic.  This is free software.  To make the system
less maintainable for developers often working for free and more difficult
for many other users for the sake of a few outliers is to me unappealing.
If people want a feature integrated into some old system, they can
certainly hire someone to do just that.

-- 
Andrew Bell
andrew.bell...@gmail.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Kurt Schwehr]

Thanks all for your comments!  I will try to write up a similar proposal
for C++11 based on what people have written.

I've integrated (and still working on) a number of the points from the
discussion so far on the stack storage issue into the doc and added a
section titled "Impact of the proposed change"

This local stack storage change is for a real need, so it's probably good
that I add a use case that explains why this change is important.  And I
>>believe<< (but am not 100% sure) that using std::vector is the strongest
solution for the majority of instances of large buffers on the stack in
gdal.

Use case:  (note that I can only share generals)

At Google, we have jobs that use GDAL running with thousands of cores and
>20 threads per process that run for days.  The overhead of having to allow
for bumping the stack size adds substantial overhead to all those jobs.
GDAL is the only thing driving the requirement for the larger stack.  For
people using GDAL on a desktop machine, the larger stack is unlikely to be
a real issue.  For anyone running large multithreaded jobs in the cloud,
the large stack is a real cost issue.  The overhead of vector and heap
allocation in all these cases I've looked at in GDAL so far is far smaller
than the cost from the stack size overhead.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Even Rouault]

Le jeudi 05 mai 2016 16:49:37, Andrew Bell a écrit :
> On Thu, May 5, 2016 at 8:26 AM, Even Rouault 
> 
> wrote:
> > Might be a little trickier with GCC, particularly with the new GCC 5
> > C++11 *ABI* and the dual ABI thing
> > (https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html),
> > but if
> > I understand well, if people use the same GCC version and do not play
> > with _GLIBCXX_USE_CXX11_ABI, linking code compiled with different
> > -std=C++?? setting
> > should work.
> > 
> > I've confirmed it with the following experiment:
> > 
> > $ cat a.cpp
> > #include 
> > 
> > std::string foo()
> > {
> > 
> >   return std::string("foo");
> > 
> > }
> > 
> > With gcc 5.2, both
> > g++ -std=c++11 -fPIC -shared a.cpp -o liba.so
> > g++ -std=c++03 -fPIC -shared a.cpp -o liba.so
> > result in the following symbol to be exported:
> > _Z3fooB5cxx11v
> > 
> > whereas if you add -D_GLIBCXX_USE_CXX11_ABI=0, the symbol name is
> > _Z3foov, which is the same as if you compile with GCC 4.X
> > 
> > So the potential problems would be when linking a GCC 5 compiled GDAL
> > (whether
> > it be C++03 or C++11) against a GCC 4.X compiled lib (think of one of the
> > SDK), so this might already occur currently. Or reversely, if an
> > application/library compiled with GCC 4.X would link against GDAL
> > compiled with GCC 5.X
> 
> ABI compatibility is a mess: https://isocpp.org/files/papers/n4028.pdf
> 
> With that in mind, what you do inside your own codebase isn't much of an
> issue and I'm guessing that even if GDAL wants to be as accommodating as
> possible, maintainers of dependent libraries probably won't take such
> care.  In the end, if you're doing development and building things
> yourself, you need to make sure you use the same compiler and switches for
> everything if you're providing a C++ interface.  Is this a pain?  Yes.  Is
> it necessary?  Yes.
> 
> As for the notion of replacing some allocations with vectors, well, great.
> But the GDAL codebase is creaky in lots of places and to get too particular
> about the advantage/disadvantage of vectors seems a little silly.  At some
> point it's just reasonable to say that you're only going to support
> compilers/systems made after some date/version.  People who don't want to
> upgrade a system aren't harmed.  They always have access to an older
> version of the library. 

(Sometimes they want the latest version of the lib for a particular feature on 
a ancient system.)

> There is just so much cruft in GDAL that could
> benefit from use of the standard library and C++ 11 that I don't know where
> to start.

That would be interesting if you (or anyone) could list which C++11 features 
would be killer features
to justify the upgrade to C++11 vs the potential pains that such a move might 
cause (especially
as I raised in other emails, given the fact that GDAL uses a number of other 
libs and is used by a number
of other libs)


> That said, changing code that works just because you like this or that
> language feature always presents an opportunity for bugs to creep in.

I agree completely with that :-) Especially for parts of the code base that are
not or poorly tested :
http://rawgit.com/rouault/gdalautotest-coverage-results/master/coverage_html/index.html
 


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Mateusz Loskot]

On 5 May 2016 at 16:31, Even Rouault  wrote:
> Le jeudi 05 mai 2016 16:14:34, David Strip a écrit :
>> On 5/4/2016 4:30 PM, Kurt Schwehr wrote:
>>
>> Drawbacks:
>>
>>
>> It is possible to change the size of the vector later on in the code
>> Vector has some storage overhead and bookkeeping that has to be done (but
>> often the compiler can probably optimize away most of that). TODO:
>> References that explain this? Resizing the array could break anything that
>> was using a C style array pointer to the vector’s data
>>
>>  Drawbacks one and three can be eliminated by deriving a class from vector
>> that hides resize, so there really is only the single drawback of storage
>> overhead and bookkeeping, which are often minor.
>
> I'd not find it very attractive to create our own class just to prevent
> resizing for such a simple thing as an array of integers... "std::vector
> oVals(256, 0)" is very readable. If it must not be resized for some reason, a
> comment should do.

An assertion, you mean :)

> One thing to keep in mind however is the potential impact of heap allocation
> in performance sensitive code paths.

Indeed.
I would divide Kurt's proposal to several phases, roughly:
1. Switch to C++11
2. Refactor: clean-up, clarify, simplify, make it less error prone
using C++11 idioms and features (i.e. int[256] => std::array)
3. Update coding guidelines
4. Measure and optimise using C++11 features with potential
functional/behavioural changes (yes, switch from stack to heap
allocation drags a behavioural change, e.g. exceptions).

Although it looks like a long and boring process, points 2 and 4 are
optional, IMO.
However, mixing plain refactoring with functional and behaviour
changes is a bad idea.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Andrew Bell]

On Thu, May 5, 2016 at 8:26 AM, Even Rouault 
wrote:

>
> Might be a little trickier with GCC, particularly with the new GCC 5 C++11
> *ABI* and the dual ABI thing
> (https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html),
> but if
> I understand well, if people use the same GCC version and do not play with
> _GLIBCXX_USE_CXX11_ABI, linking code compiled with different -std=C++??
> setting
> should work.
>
> I've confirmed it with the following experiment:
>
> $ cat a.cpp
> #include 
>
> std::string foo()
> {
>   return std::string("foo");
> }
>
> With gcc 5.2, both
> g++ -std=c++11 -fPIC -shared a.cpp -o liba.so
> g++ -std=c++03 -fPIC -shared a.cpp -o liba.so
> result in the following symbol to be exported:
> _Z3fooB5cxx11v
>
> whereas if you add -D_GLIBCXX_USE_CXX11_ABI=0, the symbol name is _Z3foov,
> which is the same as if you compile with GCC 4.X
>
> So the potential problems would be when linking a GCC 5 compiled GDAL
> (whether
> it be C++03 or C++11) against a GCC 4.X compiled lib (think of one of the
> SDK), so this might already occur currently. Or reversely, if an
> application/library compiled with GCC 4.X would link against GDAL compiled
> with GCC 5.X
>

ABI compatibility is a mess: https://isocpp.org/files/papers/n4028.pdf

With that in mind, what you do inside your own codebase isn't much of an
issue and I'm guessing that even if GDAL wants to be as accommodating as
possible, maintainers of dependent libraries probably won't take such
care.  In the end, if you're doing development and building things
yourself, you need to make sure you use the same compiler and switches for
everything if you're providing a C++ interface.  Is this a pain?  Yes.  Is
it necessary?  Yes.

As for the notion of replacing some allocations with vectors, well, great.
But the GDAL codebase is creaky in lots of places and to get too particular
about the advantage/disadvantage of vectors seems a little silly.  At some
point it's just reasonable to say that you're only going to support
compilers/systems made after some date/version.  People who don't want to
upgrade a system aren't harmed.  They always have access to an older
version of the library.  There is just so much cruft in GDAL that could
benefit from use of the standard library and C++ 11 that I don't know where
to start.

That said, changing code that works just because you like this or that
language feature always presents an opportunity for bugs to creep in.

-- 
Andrew Bell
andrew.bell...@gmail.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Even Rouault]

Le jeudi 05 mai 2016 16:14:34, David Strip a écrit :
> On 5/4/2016 4:30 PM, Kurt Schwehr wrote:
> 
> 
> 
> 
> 
> 
> 
> Drawbacks:
> 
> 
> It is possible to change the size of the vector later on in the code
> Vector has some storage overhead and bookkeeping that has to be done (but
> often the compiler can probably optimize away most of that). TODO:
> References that explain this? Resizing the array could break anything that
> was using a C style array pointer to the vector’s data
> 
>  Drawbacks one and three can be eliminated by deriving a class from vector
> that hides resize, so there really is only the single drawback of storage
> overhead and bookkeeping, which are often minor.

I'd not find it very attractive to create our own class just to prevent 
resizing for such a simple thing as an array of integers... "std::vector 
oVals(256, 0)" is very readable. If it must not be resized for some reason, a 
comment should do.

One thing to keep in mind however is the potential impact of heap allocation 
in performance sensitive code paths.


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Segfault in GDALWriteBlock

[Kurt Schwehr]

What version of GDAL are you using?  Can you provide the command or calling
code that you are using?   Can you use a debugger and give a stack trace?

On Thu, May 5, 2016 at 12:42 AM, jramm  wrote:

> Hi I am getting a segfault in a call to GDALWriteBlock with this code:
>
>
>
> Stepping through seems to suggest that the segfault is arising from line
> 171
> of "geo_new.c" in libgeotiff:
>
>
>
> I cant find previsely what it is I have done wrong to cause this in my
> codeany ideas?
>
>
>
> --
> View this message in context:
> http://osgeo-org.1560.x6.nabble.com/Segfault-in-GDALWriteBlock-tp5264660.html
> Sent from the GDAL - Dev mailing list archive at Nabble.com.
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev




-- 
--
http://schwehr.org
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Even Rouault]

Le jeudi 05 mai 2016 14:22:03, Jeff McKenna a écrit :
> On 2016-05-05 9:10 AM, Jeff McKenna wrote:
> > On 2016-05-05 5:14 AM, Dmitry Baryshnikov wrote:
> >> Hi,
> >> 
> >> I think it's time to go forward and shift to C++11 in i.e. GDAL 2.2 and
> >> drop old staff as it was with Windows mobile support, VB, etc.
> >> Yes, it may be some regression in ABI, but this is less evil than
> >> support lot of ancient compilers.
> >> 
> >> During our work on cmake build system for GDAL I faced that some
> >> dependency library needs VC2013SP2 on Windows - i.e liblzma, as it needs
> >> C99.
> >> We worked around it by including some logic to cmake, that if VC
> >> compiler is less than specific version the support of dependent features
> >> will be disabled.
> >> 
> >> By the way, the repo with cmake for GDAL is here:
> >> https://github.com/nextgis-extra/lib_gdal
> >> The ubuntu ppa of libgdal2 build using cmake:
> >> https://launchpad.net/~nextgis/+archive/ubuntu/ppa/+packages
> >> Windows builds also using cmake: http://nextgis.ru/en/borsch/
> > 
> > About a month ago I made the hard decision to upgrade my entire MS4W
> > build environment from VC 2008 to 2015.  Driving this is that PHP 7 only
> > supports 2015.
> > 
> > The only GDAL dependent library not supported for VC 2015 is FileGDB
> > (I've seen an Esri developer state that 2015 SDK will be released with
> > their next FileGDB API upgrade).
> > 
> > MrSID 9.5.1 SDK works well with 2015.
> > 
> > I didn't try an ECW 5.x SDK, but the old 3.3 SDK works well with 2015.
> > 
> > Short story is I support future GDAL builds requiring a more recent
> > Visual Studio compiler (2013+), as other projects are already doing so.
> > 
> > For those packagers cringing in the thought of upgrading their whole
> > stack: join the club :)
> 
> I should also note that for compiling Python 3.5.x VC 2015 is the
> supported environment.  Yet another GDAL dependent library that favors
> an upgraded compiler on Windows.

Thanks for your feedback. Although I'd note that upgrading compiler version is 
one thing, and enabling C++11 mode *might* be another, particularly if there's 
a mix of C++ libs linking against each other. But from what I can read, it 
seems that with Visual Studio you don't have the equivalent of -std=c++?? of 
GCC, so if things are compiled with the same VS version, that should be OK.

Might be a little trickier with GCC, particularly with the new GCC 5 C++11 
*ABI* and the dual ABI thing 
(https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html), but if 
I understand well, if people use the same GCC version and do not play with 
_GLIBCXX_USE_CXX11_ABI, linking code compiled with different -std=C++?? setting 
should work.

I've confirmed it with the following experiment:

$ cat a.cpp 
#include 

std::string foo()
{
  return std::string("foo");
}

With gcc 5.2, both
g++ -std=c++11 -fPIC -shared a.cpp -o liba.so
g++ -std=c++03 -fPIC -shared a.cpp -o liba.so
result in the following symbol to be exported:
_Z3fooB5cxx11v

whereas if you add -D_GLIBCXX_USE_CXX11_ABI=0, the symbol name is _Z3foov, 
which is the same as if you compile with GCC 4.X

So the potential problems would be when linking a GCC 5 compiled GDAL (whether 
it be C++03 or C++11) against a GCC 4.X compiled lib (think of one of the 
SDK), so this might already occur currently. Or reversely, if an 
application/library compiled with GCC 4.X would link against GDAL compiled 
with GCC 5.X

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Joaquim Luis]

I'm also updating my builds to support 2015 for a future inclusion in GMT.
There is, however, a bit storm ahead. Files created with HDF5.10 break  
compatibility with older versions and can't be read. Put netCDF in this  
bag as well. See GMT's #3495 for a recent case.


http://gmt.soest.hawaii.edu/boards/1/topics/3495



On 2016-05-05 5:14 AM, Dmitry Baryshnikov wrote:

Hi,

I think it's time to go forward and shift to C++11 in i.e. GDAL 2.2 and
drop old staff as it was with Windows mobile support, VB, etc.
Yes, it may be some regression in ABI, but this is less evil than
support lot of ancient compilers.

During our work on cmake build system for GDAL I faced that some
dependency library needs VC2013SP2 on Windows - i.e liblzma, as it needs
C99.
We worked around it by including some logic to cmake, that if VC
compiler is less than specific version the support of dependent features
will be disabled.

By the way, the repo with cmake for GDAL is here:
https://github.com/nextgis-extra/lib_gdal
The ubuntu ppa of libgdal2 build using cmake:
https://launchpad.net/~nextgis/+archive/ubuntu/ppa/+packages
Windows builds also using cmake: http://nextgis.ru/en/borsch/


About a month ago I made the hard decision to upgrade my entire MS4W  
build environment from VC 2008 to 2015.  Driving this is that PHP 7 only  
supports 2015.


The only GDAL dependent library not supported for VC 2015 is FileGDB  
(I've seen an Esri developer state that 2015 SDK will be released with  
their next FileGDB API upgrade).


MrSID 9.5.1 SDK works well with 2015.

I didn't try an ECW 5.x SDK, but the old 3.3 SDK works well with 2015.

Short story is I support future GDAL builds requiring a more recent  
Visual Studio compiler (2013+), as other projects are already doing so.


For those packagers cringing in the thought of upgrading their whole  
stack: join the club :)


-jeff




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Jeff McKenna]

On 2016-05-05 9:10 AM, Jeff McKenna wrote:

On 2016-05-05 5:14 AM, Dmitry Baryshnikov wrote:

Hi,

I think it's time to go forward and shift to C++11 in i.e. GDAL 2.2 and
drop old staff as it was with Windows mobile support, VB, etc.
Yes, it may be some regression in ABI, but this is less evil than
support lot of ancient compilers.

During our work on cmake build system for GDAL I faced that some
dependency library needs VC2013SP2 on Windows - i.e liblzma, as it needs
C99.
We worked around it by including some logic to cmake, that if VC
compiler is less than specific version the support of dependent features
will be disabled.

By the way, the repo with cmake for GDAL is here:
https://github.com/nextgis-extra/lib_gdal
The ubuntu ppa of libgdal2 build using cmake:
https://launchpad.net/~nextgis/+archive/ubuntu/ppa/+packages
Windows builds also using cmake: http://nextgis.ru/en/borsch/


About a month ago I made the hard decision to upgrade my entire MS4W
build environment from VC 2008 to 2015.  Driving this is that PHP 7 only
supports 2015.

The only GDAL dependent library not supported for VC 2015 is FileGDB
(I've seen an Esri developer state that 2015 SDK will be released with
their next FileGDB API upgrade).

MrSID 9.5.1 SDK works well with 2015.

I didn't try an ECW 5.x SDK, but the old 3.3 SDK works well with 2015.

Short story is I support future GDAL builds requiring a more recent
Visual Studio compiler (2013+), as other projects are already doing so.

For those packagers cringing in the thought of upgrading their whole
stack: join the club :)



I should also note that for compiling Python 3.5.x VC 2015 is the 
supported environment.  Yet another GDAL dependent library that favors 
an upgraded compiler on Windows.


-jeff




--
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Jeff McKenna]

On 2016-05-05 5:14 AM, Dmitry Baryshnikov wrote:

Hi,

I think it's time to go forward and shift to C++11 in i.e. GDAL 2.2 and
drop old staff as it was with Windows mobile support, VB, etc.
Yes, it may be some regression in ABI, but this is less evil than
support lot of ancient compilers.

During our work on cmake build system for GDAL I faced that some
dependency library needs VC2013SP2 on Windows - i.e liblzma, as it needs
C99.
We worked around it by including some logic to cmake, that if VC
compiler is less than specific version the support of dependent features
will be disabled.

By the way, the repo with cmake for GDAL is here:
https://github.com/nextgis-extra/lib_gdal
The ubuntu ppa of libgdal2 build using cmake:
https://launchpad.net/~nextgis/+archive/ubuntu/ppa/+packages
Windows builds also using cmake: http://nextgis.ru/en/borsch/


About a month ago I made the hard decision to upgrade my entire MS4W 
build environment from VC 2008 to 2015.  Driving this is that PHP 7 only 
supports 2015.


The only GDAL dependent library not supported for VC 2015 is FileGDB 
(I've seen an Esri developer state that 2015 SDK will be released with 
their next FileGDB API upgrade).


MrSID 9.5.1 SDK works well with 2015.

I didn't try an ECW 5.x SDK, but the old 3.3 SDK works well with 2015.

Short story is I support future GDAL builds requiring a more recent 
Visual Studio compiler (2013+), as other projects are already doing so.


For those packagers cringing in the thought of upgrading their whole 
stack: join the club :)


-jeff




--
Jeff McKenna
MapServer Consulting and Training Services
http://www.gatewaygeomatics.com/




___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Mateusz Loskot]

On 5 May 2016 at 11:28, Even Rouault  wrote:
> Le jeudi 05 mai 2016 11:09:48, Mateusz Loskot a écrit :
>> On 5 May 2016 at 01:45, Even Rouault  wrote:
>> > Of the potential issues with requiring C++11, I can think of OSGeo4W. It
>> > is mostly(completely?) built with Visual Studio 2010. And from
>> > https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of C++11
>> > is only partial in VS 2010
>>
>> VS2013 is the lowest version sensible to consider regarding C++11 support.
>>
>> > On the other hand regarding dependencies of GDAL, the binary propritary
>> > SDKs with a C++ API could be a problem, although they will likely move
>> > on too. - FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
>> > - ECW SDK 5.2.1: available for VS2010, VS2012, VS2013
>>
>> AFAICT, GDAL built using VS2015 links against the 5.2.1 version fine.
>
> Do you link against the VS2013 .lib ?

Yes, namely these:

ls -1 ERDAS_ECW_JPEG_2000\lib\vc120\Win32
NCSEcw.lib
NCSEcwd.lib
NCSEcwS.lib
NCSEcwSd.lib

>> > - MrSID SDK: I didn't check. Perhaps Kirk can tell us ?
>>
>> Similarly to the ECW above, version 9.1.0 links fine too.
>>
>> > But I'm not sure about the compatibility of C++11 build against non-C++11
>> > builds in the VS realm : can a GDAL C++11 build link against a library
>> > built without C++11 enabled ? Will not there be ABI problems ?
>>
>> Although mixing C run-time libraries is a bad idea generally,
>> kosher C APIs should be fine.
>
> I'm not too worried about C ABI. My mention of C++03 ABI vs C++11 ABI was for
> the dependencies with those SDK that we use through their C++ API

Visual C++ does not promise ABI compatibility for anything but a
"portable type" [1]
- a C built-in type or a struct that contains only C built-in types

Mixing binaries built with different compiler versions or even
compiler/linker settings
is not recommended [2]. It might or might not work :)

[1] https://msdn.microsoft.com/en-us/library/hh438475.aspx
[2] https://msdn.microsoft.com/en-us/library/bb531344.aspx

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Even Rouault]

Le jeudi 05 mai 2016 11:09:48, Mateusz Loskot a écrit :
> On 5 May 2016 at 01:45, Even Rouault  wrote:
> > Of the potential issues with requiring C++11, I can think of OSGeo4W. It
> > is mostly(completely?) built with Visual Studio 2010. And from
> > https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of C++11
> > is only partial in VS 2010
> 
> VS2013 is the lowest version sensible to consider regarding C++11 support.
> 
> > On the other hand regarding dependencies of GDAL, the binary propritary
> > SDKs with a C++ API could be a problem, although they will likely move
> > on too. - FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
> > - ECW SDK 5.2.1: available for VS2010, VS2012, VS2013
> 
> AFAICT, GDAL built using VS2015 links against the 5.2.1 version fine.

Do you link against the VS2013 .lib ?
> 
> > - MrSID SDK: I didn't check. Perhaps Kirk can tell us ?
> 
> Similarly to the ECW above, version 9.1.0 links fine too.
> 
> > But I'm not sure about the compatibility of C++11 build against non-C++11
> > builds in the VS realm : can a GDAL C++11 build link against a library
> > built without C++11 enabled ? Will not there be ABI problems ?
> 
> Although mixing C run-time libraries is a bad idea generally,
> kosher C APIs should be fine.

I'm not too worried about C ABI. My mention of C++03 ABI vs C++11 ABI was for 
the dependencies with those SDK that we use through their C++ API. In GCC 
world, with same compiler but with different -std=... flags, there have been 
troubles ( https://gcc.gnu.org/wiki/Cxx11AbiCompatibility , 
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_dual_abi.html ), so I 
was wondering how the situation is in the VS world.
Actually since I mention this, the issue does apply to Linux too. Looking at 
the C++11 Linux build ( 
https://github.com/rouault/gdal_coverage/blob/trunk_gcc4.8_stdc11/.travis.yml 
) , I see I couldn't build against FileGDB with -std=c++11. This was a 
compilation issue with its headers, rather than a link/runtime issue.


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Mateusz Loskot]

On 5 May 2016 at 01:45, Even Rouault  wrote:
> Of the potential issues with requiring C++11, I can think of OSGeo4W. It is
> mostly(completely?) built with Visual Studio 2010. And from
> https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of C++11 is
> only partial in VS 2010

VS2013 is the lowest version sensible to consider regarding C++11 support.

> On the other hand regarding dependencies of GDAL, the binary propritary SDKs
> with a C++ API could be a problem, although they will likely move on too.
> - FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
> - ECW SDK 5.2.1: available for VS2010, VS2012, VS2013

AFAICT, GDAL built using VS2015 links against the 5.2.1 version fine.

> - MrSID SDK: I didn't check. Perhaps Kirk can tell us ?

Similarly to the ECW above, version 9.1.0 links fine too.

> But I'm not sure about the compatibility of C++11 build against non-C++11
> builds in the VS realm : can a GDAL C++11 build link against a library built
> without C++11 enabled ? Will not there be ABI problems ?

Although mixing C run-time libraries is a bad idea generally,
kosher C APIs should be fine.

Best regards,
-- 
Mateusz Loskot, http://mateusz.loskot.net
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Peter Halls]

I am a humble geologist, rather than a Computer Scientist, and do not
pretend to understand all the ins-and-outs of this type of discussion - and
hence read, in the hopes of learning, but otherwise keep quiet!

I suspect that I am also something of a 'throw-back' in that I continue to
use Simula as my primary programming language.  Today's Simula compilers
are 'cross-compilers', in that they generate code in c, which is then
compiled and linked with an appropriate c compiler (it actually makes
providing software for multiple platforms a lot easier ...).

Having said that, my only concern with the use of (C++) Vector structures,
rather than c Arrays, would be the mappability required between application
language and implementation language.  If the C++ structures are universal,
fine.  If not, then either interface code (needing maintenance) or c
alternatives would be needed.  My example is based on Simula: it would be
interesting to know how Java or Python might interact.

Best wishes,

Peter


On 5 May 2016 at 00:45, Even Rouault  wrote:

> Le jeudi 05 mai 2016 00:51:28, Mateusz Loskot a écrit :
> > On 4 May 2016 at 23:30, Kurt Schwehr  wrote:
> > > To start off the conversation, I wrote up a doc on changing large C
> > > arrays on the stack to std::vectors to get this data off of the stack
> > > and to simplify initialization.
> >
> > Since many, if not most, of the ideas rely on availability of the C++11
> > features,
> > it might be a good idea to first agree on C++11 as the lowest
> > required C++ compilation mode.
> >
> > Let's poll if there are any users who require C++03 at all.
>
> I think there are different topics in what Kurt proposes :
> - style and other changes that are not bound to a particular compiler
> version
> - changes potentially dependant of the C++ version
>
> Of the potential issues with requiring C++11, I can think of OSGeo4W. It is
> mostly(completely?) built with Visual Studio 2010. And from
> https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of C++11
> is
> only partial in VS 2010
>
> Regarding the current use of VS2010 in OSGeo4W, this is discussed here :
> https://lists.osgeo.org/pipermail/discuss/2016-February/015658.html
>
> Potentially we could imagine having GDAL compiled with a later VS version
> and
> have the rest using VS2010, since most (all?) other software in OSGeo4W
> use it
> probably through its C API. Hum, actually that must not be true for OTB
> that
> uses the C++ API I think. Perhaps Jürgen has something to add regarding
> this
> compiler issue ?
>
> On the other hand regarding dependencies of GDAL, the binary propritary
> SDKs
> with a C++ API could be a problem, although they will likely move on too.
> - FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
> - ECW SDK 5.2.1: available for VS2010, VS2012, VS2013
> - MrSID SDK: I didn't check. Perhaps Kirk can tell us ?
> But I'm not sure about the compatibility of C++11 build against non-C++11
> builds in the VS realm : can a GDAL C++11 build link against a library
> built
> without C++11 enabled ? Will not there be ABI problems ?
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-dev@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev




-- 

Peter J Halls, PhD Student, Post-war Reconstruction and Development Unit
(PRDU),
  University of York

Snail mail: PRDU, Derwent College, University of York,
Heslington, York YO10 5DD
This message has the status of a private and personal communication

___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

Re: [gdal-dev] Starting a discussion on style and coding guidelines

[Dmitry Baryshnikov]

Hi,

I think it's time to go forward and shift to C++11 in i.e. GDAL 2.2 and 
drop old staff as it was with Windows mobile support, VB, etc.
Yes, it may be some regression in ABI, but this is less evil than 
support lot of ancient compilers.


During our work on cmake build system for GDAL I faced that some 
dependency library needs VC2013SP2 on Windows - i.e liblzma, as it needs 
C99.
We worked around it by including some logic to cmake, that if VC 
compiler is less than specific version the support of dependent features 
will be disabled.


By the way, the repo with cmake for GDAL is here: 
https://github.com/nextgis-extra/lib_gdal
The ubuntu ppa of libgdal2 build using cmake: 
https://launchpad.net/~nextgis/+archive/ubuntu/ppa/+packages

Windows builds also using cmake: http://nextgis.ru/en/borsch/

Please note, that most but not all drivers are available now (for 
details see readme in repo) and only python bindings.


Best regards,
Dmitry

05.05.2016 02:45, Even Rouault пишет:

Le jeudi 05 mai 2016 00:51:28, Mateusz Loskot a écrit :

On 4 May 2016 at 23:30, Kurt Schwehr  wrote:

To start off the conversation, I wrote up a doc on changing large C
arrays on the stack to std::vectors to get this data off of the stack
and to simplify initialization.

Since many, if not most, of the ideas rely on availability of the C++11
features,
it might be a good idea to first agree on C++11 as the lowest
required C++ compilation mode.

Let's poll if there are any users who require C++03 at all.

I think there are different topics in what Kurt proposes :
- style and other changes that are not bound to a particular compiler version
- changes potentially dependant of the C++ version

Of the potential issues with requiring C++11, I can think of OSGeo4W. It is
mostly(completely?) built with Visual Studio 2010. And from
https://msdn.microsoft.com/en-us/library/hh567368.aspx , support of C++11 is
only partial in VS 2010

Regarding the current use of VS2010 in OSGeo4W, this is discussed here :
https://lists.osgeo.org/pipermail/discuss/2016-February/015658.html

Potentially we could imagine having GDAL compiled with a later VS version and
have the rest using VS2010, since most (all?) other software in OSGeo4W use it
probably through its C API. Hum, actually that must not be true for OTB that
uses the C++ API I think. Perhaps Jürgen has something to add regarding this
compiler issue ?

On the other hand regarding dependencies of GDAL, the binary propritary SDKs
with a C++ API could be a problem, although they will likely move on too.
- FileGDB SDK 1.4: available for VS2010, VS2012, VS2013
- ECW SDK 5.2.1: available for VS2010, VS2012, VS2013
- MrSID SDK: I didn't check. Perhaps Kirk can tell us ?
But I'm not sure about the compatibility of C++11 build against non-C++11
builds in the VS realm : can a GDAL C++11 build link against a library built
without C++11 enabled ? Will not there be ABI problems ?

Even



___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev

[gdal-dev] Segfault in GDALWriteBlock

[jramm]

Hi I am getting a segfault in a call to GDALWriteBlock with this code:



Stepping through seems to suggest that the segfault is arising from line 171
of "geo_new.c" in libgeotiff:



I cant find previsely what it is I have done wrong to cause this in my
codeany ideas?



--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/Segfault-in-GDALWriteBlock-tp5264660.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
___
gdal-dev mailing list
gdal-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev