For the record I can't see the #define trick as reasonable/acceptable if i understand it correctly. I also have no other ideas, so I'm (very) sorry to say this is now dead in the water if nothing new comes to light.
Den tors 19 maj 2022 09:13Björn Harrtell <bjorn.harrt...@gmail.com> skrev: > Thanks Even, I was not aware of the risk of symbol clash. I will try and > apply the #define trickery. > > Den ons 18 maj 2022 kl 13:31 skrev Even Rouault < > even.roua...@spatialys.com>: > >> I'm 0 on this. While this is always nice to see a perf improvement, I'm >> somewhat concerned by the duplication of code between MapServer and GDAL. >> >> And from a purely technical point of view, one of my worry is that >> symbol clashes might occur between GDAL and MapServer on the common code >> they share in the FlatGeobuf and flatbuffers C++ namespaces. If the >> implementations are not kept in sync (and that will necessarily happen >> at some point because we cannot control which GDAL version is used by >> MapServer), weird things (crashes) could possibly happen at runtime (or >> at build time if doing static builds). That worry could be solved by >> possibly having a mapserver:: prefix in front of those namespace names >> (can be done with some #define trickery. See >> >> https://github.com/OSGeo/PROJ/blob/9f89c7288e44c64ba234a299b1d5528a54d9d61e/include/proj/util.hpp#L104 >> for potential inspiration) >> >> The interesting take-away from my perspective is that the OGR overhead >> for FlatGeobuf is only 5 ms, but 13 ms for Shapefile. So one >> interpretation would be that there's some inefficiency in the OGR >> Shapefile driver vs the native MapServer provider. >> >> Even >> >> Le 17/05/2022 à 15:19, Jeff McKenna a écrit : >> > Update: Björn has completed the effort (which now includes several >> > msautotests) and we've created an RFC (137) for the native FlatGeobuf >> > support: https://mapserver.org/development/rfc/ms-rfc-137.html >> > >> > I therefore motion to include FlatGeobuf as a native MapServer driver, >> > per RFC 137, and maintained by Björn Harrtell. >> > >> > starting with my +1 >> > >> > Some other interesting updates: >> > >> > - great to see changes here made through testing benefiting so many >> > other projects >> > >> > - here is an updated benchmark result (that is mentioned in the RFC) : >> > >> > >> > FlatGeobuf (native) 0.008s >> > Shapefile (native) 0.010s >> > FlatGeobuf (OGR) 0.013s >> > Shapefile (OGR) 0.023s >> > GeoPackage (OGR) 0.042s >> > SpatiaLite (OGR) 0.045s >> > PostGIS (native) 0.053s >> > GeoJSON (OGR) 0.089s >> > >> > (that was with yesterday's main, GDAL 3.4.3, with MS4W on Windows) >> > >> > - with all this testing done, and the existing FlatGeobuf >> > documentation effort, it will be very easy to add this to the main >> > documentation now... >> > >> > Thanks, >> > >> > >> > -Björn & Jeff >> > >> > >> > >> > >> > >> > On 2022-04-27 3:07 p.m., Jeff McKenna wrote: >> >> Great point Even, here is an updated result: >> >> >> >> Shapefile 0.011s >> >> FlatGeobuf 0.014s >> >> Shapefile (OGR) 0.024s >> >> GeoPackage 0.042s >> >> SpatiaLite 0.045s >> >> PostGIS 0.053s >> >> GeoJSON 0.089s >> >> >> >> >> >> -jeff >> >> >> >> >> >> >> >> >> >> On 2022-04-27 2:09 p.m., Even Rouault wrote: >> >>> Jeff, >> >>> >> >>> as a data point, perhaps you could enhance your bench with >> >>> shapefiles through OGR ? It would be interesting to have a sense of >> >>> the OGR overhead (that will be the same for any OGR supported >> >>> datasource) to have an idea if it is really worth the effort to have >> >>> a native FlatGeobuf provider for MapServer w.r.t going through OGR >> >>> (it is hard to beat shapefile, and the current difference between >> >>> shapefile and flatgeobuf is small) >> >>> >> >>> Even >> >>> >> >>> Le 26/04/2022 à 12:57, Jeff McKenna a écrit : >> >>>> (to followup from our chat on IRC yesterday) >> >>>> >> >>>> To clarify: this would be a new native format in MapServer, such as >> >>>> Shapefile or PostGIS, and the goal would be to not connect to >> >>>> FlatGeobuf through OGR, but instead directly, such as: >> >>>> >> >>>> DATA countries.fgb >> >>>> >> >>>> instead of: >> >>>> >> >>>> CONNECTIONTYPE OGR >> >>>> CONNECTION "countries.fgb" >> >>>> DATA "countries" >> >>>> >> >>>> This should improve the performance even more. And it will be easy >> >>>> to update the documentation, as the existing files just need to be >> >>>> updated (see the new page at >> >>>> https://mapserver.org/input/vector/flatgeobuf.html ) >> >>>> >> >>>> Great plan Björn! I look forward to your Pull Request, ha, and >> >>>> will tackle the documentation changes for you. >> >>>> >> >>>> This is also great news for MapServer, and will give us some nice >> >>>> publicity. >> >>>> >> >>>> Thanks so much! >> >>>> >> >>>> -jeff >> >>>> >> >>>> >> >>>> >> >>>> On 2022-04-25 5:57 p.m., Björn Harrtell wrote: >> >>>>> Hi mapserver devs! >> >>>>> >> >>>>> I got interested in the subject because of the tests made recently >> >>>>> by Jeff that shows the potential of the format. I believe it >> >>>>> should be possible to get significant additional performance out >> >>>>> of FlatGeobuf in MapServer if it was built in just like Shapefile >> >>>>> support is. >> >>>>> >> >>>>> FlatGeobuf C++ implementation is rather small, doesn't require any >> >>>>> dependencies, and I don't expect it to require any invasive >> >>>>> changes to existing code. >> >>>>> >> >>>>> Would love to get started on this ASAP, if there are no >> >>>>> objections. :) >> >>>>> >> >>>>> Regards, >> >>>>> Björn Harrtell >> >>>>> >> >> ______ >> > _______________________________________________ >> > MapServer-dev mailing list >> > MapServer-dev@lists.osgeo.org >> > https://lists.osgeo.org/mailman/listinfo/mapserver-dev >> >> -- >> http://www.spatialys.com >> My software is free, but my time generally not. >> >>
_______________________________________________ MapServer-dev mailing list MapServer-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-dev
