Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-27 Thread Ray at Daylon via gdal-dev
This may already be known, but the use of std::string in CPLString can also cause problems e.g. building a program with a version of a C++ stdlib that differs from the one that a prebuilt GDAL DLL used. Visual Studio e.g. has afaik a 48-byte std::string in MSVC 2010 vs. 40 bytes in MSVC 2022. It's

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Simon Eves via gdal-dev
As we're not ready to upgrade Arrow just yet, I also did the namespace change as a pre-build code patch on the regular 3.7.3 bundle. Apologies to Bjorn and anyone else on that team for any inference that this was a flatbuffers bug. Not my intention. Glad we worked it out. Simon On Sun, Feb 25, 2

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Even Rouault via gdal-dev
OK, so I guess we might be able to avoid it by upgrading Arrow, as long as that doesn't break something else. I guess you need to do the custom namespace thing too, though. (ab)use of preprocessor functionality makes it easy: https://github.com/OSGeo/gdal/pull/9313 ... I hate computers.

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Simon Eves via gdal-dev
Arrow moved to using a custom namespace in v10.0.0 On Sun, Feb 25, 2024 at 5:26 PM Simon Eves wrote: > Ooh, good call! > > That also corresponds with what I just tried, which was to leave the > change in, but have the *size()* method return a value derived the old > way instead of just returning

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Simon Eves via gdal-dev
Ooh, good call! That also corresponds with what I just tried, which was to leave the change in, but have the *size()* method return a value derived the old way instead of just returning *size_*, and also compare the two and log any mismatch. This also fails, which would seem to discount my thought

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Even Rouault via gdal-dev
Not obvious why that change would have broken anything, and certainly still absolutely no idea why it only happens in a full static build. At that point, I would slightly bet on the fact that your whole application would have another component using flatbuffers at a different version, whic

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-25 Thread Simon Eves via gdal-dev
I went ahead and did a bisect on the *flatbuffers* code itself. Turns out there were only 63 commits that touched *include/flatbuffers* between the versions that GDAL 3.5.x and 3.6.x use. This is the commit that breaks it, specifically the change to *vector_downward.h* https://github.com/google/f

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-24 Thread Simon Eves via gdal-dev
OK, so I tried a custom build of 3.7.3 with the latest *flatbuffers* (23.5.26), which was a drop-in replacement for 2.0.6 other than the version asserts. This does not exhibit the original problem either. However, while it produces files which the stock static build, the static build with the old

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-23 Thread Simon Eves via gdal-dev
Our emails crossed. I am indeed testing with the latest flatbuffers now too. Agreed on the rest. On Fri, Feb 23, 2024 at 3:42 PM Even Rouault wrote: > Simon, > > did you try to update to the latest > https://github.com/google/flatbuffers/releases to see if that would solve > the issue ? If that

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-23 Thread Even Rouault via gdal-dev
Simon, did you try to update to the latest https://github.com/google/flatbuffers/releases to see if that would solve the issue ? If that worked, that would be the best way forward... Otherwise if the issue persists with the latest flatbuffers release, a (admitedly rather tedious) option woul

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-23 Thread Simon Eves via gdal-dev
I'm also testing a build that uses the latest *flatbuffers* 23.5.26, which is a drop-in replacement and does not require the *VerifyField* change, just in case the issue has been fixed since 2.0.6. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https:

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-23 Thread Simon Eves via gdal-dev
I was able to create a fork of 3.7.3 with just the *flatbuffers* replaced with the pre-3.6.x version (2.0.0). This seemed to only require changes to the version asserts and adding an *align* parameter to *Table::VerifyField()* to match the newer API. https://github.com/heavyai/gdal/tree/simon.eve

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-22 Thread Simon Eves via gdal-dev
Thank you, Robert, for the RR tip. I shall try it. I have new findings to report, however. First of all, I confirmed that a build against GDAL 3.4.1 (the version we were on before) still works. I also confirmed that builds against 3.7.3 and 3.8.4 still failed even with no additional library depen

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-20 Thread Robert Coup via gdal-dev
Hi, On Tue, 20 Feb 2024 at 21:44, Robert Coup wrote: > Hi Simon, > > On Tue, 20 Feb 2024 at 21:11, Simon Eves wrote: > >> Here's the stack trace for the original assert. Something is stepping on >> scratch_ to make it 0x10 instead of null, which it starts out as >> when the flatbuffer o

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-20 Thread Simon Eves via gdal-dev
gdb) p m_fbb $5 = (flatbuffers::FlatBufferBuilder &) @0x7fff9070: {static kFileIdentifierLength = 4, buf_ = {allocator_ = 0x0, own_allocator_ = false, initial_size_ = 1024, buffer_minalign_ = 8, reserved_ = 0, size_ = 0, buf_ = 0x0, cur_ = 0x0, scratch_ = 0x1 }, num_field_loc =

Re: [gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-20 Thread Simon Eves via gdal-dev
Here's the stack trace for the original assert. Something is stepping on scratch_ to make it 0x10 instead of null, which it starts out as when the flatbuffer object is created, but by the time it gets to allocating memory, it's broken. On Tue, Feb 20, 2024 at 1:05 PM Simon Eves wrote: >

[gdal-dev] Assert due to stack corruption in FlatGeoBuf export

2024-02-20 Thread Simon Eves via gdal-dev
(starting a new thread to avoid derailing the static-build one any further) Totally agreed on the mismatch idea, but the code in question is all self-contained down in *ogr/ogrsf_frmts/flatgeobuf* and the *flatbuffers* sub-project (which is a snapshot of a Google OSS project) so I'm struggling to