Thanks Even, I was not aware of the risk of symbol clash. I will try and apply the #define trickery.
Den ons 18 maj 2022 kl 13:31 skrev Even Rouault <even.roua...@spatialys.com >: > I'm 0 on this. While this is always nice to see a perf improvement, I'm > somewhat concerned by the duplication of code between MapServer and GDAL. > > And from a purely technical point of view, one of my worry is that > symbol clashes might occur between GDAL and MapServer on the common code > they share in the FlatGeobuf and flatbuffers C++ namespaces. If the > implementations are not kept in sync (and that will necessarily happen > at some point because we cannot control which GDAL version is used by > MapServer), weird things (crashes) could possibly happen at runtime (or > at build time if doing static builds). That worry could be solved by > possibly having a mapserver:: prefix in front of those namespace names > (can be done with some #define trickery. See > > https://github.com/OSGeo/PROJ/blob/9f89c7288e44c64ba234a299b1d5528a54d9d61e/include/proj/util.hpp#L104 > for potential inspiration) > > The interesting take-away from my perspective is that the OGR overhead > for FlatGeobuf is only 5 ms, but 13 ms for Shapefile. So one > interpretation would be that there's some inefficiency in the OGR > Shapefile driver vs the native MapServer provider. > > Even > > Le 17/05/2022 à 15:19, Jeff McKenna a écrit : > > Update: Björn has completed the effort (which now includes several > > msautotests) and we've created an RFC (137) for the native FlatGeobuf > > support: https://mapserver.org/development/rfc/ms-rfc-137.html > > > > I therefore motion to include FlatGeobuf as a native MapServer driver, > > per RFC 137, and maintained by Björn Harrtell. > > > > starting with my +1 > > > > Some other interesting updates: > > > > - great to see changes here made through testing benefiting so many > > other projects > > > > - here is an updated benchmark result (that is mentioned in the RFC) : > > > > > > FlatGeobuf (native) 0.008s > > Shapefile (native) 0.010s > > FlatGeobuf (OGR) 0.013s > > Shapefile (OGR) 0.023s > > GeoPackage (OGR) 0.042s > > SpatiaLite (OGR) 0.045s > > PostGIS (native) 0.053s > > GeoJSON (OGR) 0.089s > > > > (that was with yesterday's main, GDAL 3.4.3, with MS4W on Windows) > > > > - with all this testing done, and the existing FlatGeobuf > > documentation effort, it will be very easy to add this to the main > > documentation now... > > > > Thanks, > > > > > > -Björn & Jeff > > > > > > > > > > > > On 2022-04-27 3:07 p.m., Jeff McKenna wrote: > >> Great point Even, here is an updated result: > >> > >> Shapefile 0.011s > >> FlatGeobuf 0.014s > >> Shapefile (OGR) 0.024s > >> GeoPackage 0.042s > >> SpatiaLite 0.045s > >> PostGIS 0.053s > >> GeoJSON 0.089s > >> > >> > >> -jeff > >> > >> > >> > >> > >> On 2022-04-27 2:09 p.m., Even Rouault wrote: > >>> Jeff, > >>> > >>> as a data point, perhaps you could enhance your bench with > >>> shapefiles through OGR ? It would be interesting to have a sense of > >>> the OGR overhead (that will be the same for any OGR supported > >>> datasource) to have an idea if it is really worth the effort to have > >>> a native FlatGeobuf provider for MapServer w.r.t going through OGR > >>> (it is hard to beat shapefile, and the current difference between > >>> shapefile and flatgeobuf is small) > >>> > >>> Even > >>> > >>> Le 26/04/2022 à 12:57, Jeff McKenna a écrit : > >>>> (to followup from our chat on IRC yesterday) > >>>> > >>>> To clarify: this would be a new native format in MapServer, such as > >>>> Shapefile or PostGIS, and the goal would be to not connect to > >>>> FlatGeobuf through OGR, but instead directly, such as: > >>>> > >>>> DATA countries.fgb > >>>> > >>>> instead of: > >>>> > >>>> CONNECTIONTYPE OGR > >>>> CONNECTION "countries.fgb" > >>>> DATA "countries" > >>>> > >>>> This should improve the performance even more. And it will be easy > >>>> to update the documentation, as the existing files just need to be > >>>> updated (see the new page at > >>>> https://mapserver.org/input/vector/flatgeobuf.html ) > >>>> > >>>> Great plan Björn! I look forward to your Pull Request, ha, and > >>>> will tackle the documentation changes for you. > >>>> > >>>> This is also great news for MapServer, and will give us some nice > >>>> publicity. > >>>> > >>>> Thanks so much! > >>>> > >>>> -jeff > >>>> > >>>> > >>>> > >>>> On 2022-04-25 5:57 p.m., Björn Harrtell wrote: > >>>>> Hi mapserver devs! > >>>>> > >>>>> I got interested in the subject because of the tests made recently > >>>>> by Jeff that shows the potential of the format. I believe it > >>>>> should be possible to get significant additional performance out > >>>>> of FlatGeobuf in MapServer if it was built in just like Shapefile > >>>>> support is. > >>>>> > >>>>> FlatGeobuf C++ implementation is rather small, doesn't require any > >>>>> dependencies, and I don't expect it to require any invasive > >>>>> changes to existing code. > >>>>> > >>>>> Would love to get started on this ASAP, if there are no > >>>>> objections. :) > >>>>> > >>>>> Regards, > >>>>> Björn Harrtell > >>>>> > >> ______ > > _______________________________________________ > > MapServer-dev mailing list > > MapServer-dev@lists.osgeo.org > > https://lists.osgeo.org/mailman/listinfo/mapserver-dev > > -- > http://www.spatialys.com > My software is free, but my time generally not. > >
_______________________________________________ MapServer-dev mailing list MapServer-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-dev
