Agreed, it should start as a community module. Happy to review changes to make subclassing the existing GeoJSON producer easier. Or you can just copy and modify, if you like, to get prototyping moving on faster.
If you are interested in a ready made compact and efficient format instead, I suggest looking into flatgeobuf <https://github.com/flatgeobuf/flatgeobuf>, it already has a Javascript reader library and GeoServer has a community module to generate it as a WFS output format. Cheers Andrea On Tue, Sep 13, 2022 at 1:54 PM Ian Turton <ijtur...@gmail.com> wrote: > Sorry hit send too soon! > > I would also prefer it if it was an extension rather than part of the core > GeoServer. > > Ian > > > On Tue, 13 Sept 2022 at 12:52, Ian Turton <ijtur...@gmail.com> wrote: > >> I would suggest calling it something other than GeoJSON since if it >> strays too far from the spec clients will become confused. >> >> Ian >> >> On Tue, 13 Sept 2022 at 12:07, Carsten Klein <c.kl...@datagis.com> wrote: >> >>> Hello everyone, >>> >>> I'd like to enhance WFS responses in JSON/JSONP format. In some of our >>> Web GIS Applications, one may download up to several thousand Simple >>> Features through WFS. Some Layers may have between 400 and 500 columns. >>> >>> Although JSON produces much smaller responses than XML/GML, such >>> responses quickly have sizes between 60 and 120 MB (uncompressed). >>> >>> The idea is to remove some redundancy from the GeoJSON format, which >>> provides a feature's attributes in a key/value map (JSON Object): >>> >>> { >>> "type":"FeatureCollection", >>> "features":[ >>> { >>> "type":"Feature", >>> "id":"bugsites.3", >>> "geometry":{ >>> "type":"Point", >>> "coordinates":[ >>> 590529, >>> 4914625 >>> ] >>> }, >>> "geometry_name":"the_geom", >>> "properties":{ >>> "cat":3, >>> "str1":"Beetle site" >>> } >>> }, >>> { >>> "type":"Feature", >>> "id":"bugsites.4", >>> "geometry":{ >>> "type":"Point", >>> "coordinates":[ >>> 590546, >>> 4915353 >>> ] >>> }, >>> "geometry_name":"the_geom", >>> "properties":{ >>> "cat":4, >>> "str1":"Beetle site" >>> } >>> } >>> ], >>> "totalFeatures":2, >>> "numberMatched":2, >>> "numberReturned":2, >>> "timeStamp":"2022-09-13T08:44:45.118Z", >>> "crs":{ >>> "type":"name", >>> "properties":{ >>> "name":"urn:ogc:def:crs:EPSG::26713" >>> } >>> } >>> } >>> >>> Also, the "geometry_name" property is repeated for every feature >>> returned. >>> >>> With lots of features and columns (the latter do not necessarily have >>> short names), this repeated schema information can quickly become the >>> dominating factor regarding the size of the response. (Of course, that >>> also depends on the type and complexity of the geometry.) >>> >>> Likely the repeated "type":"Feature" could be omitted as well. It's just >>> there in order to satisfy the GeoJSON specs. Maybe the "level of >>> compaction" could be specified for a request. >>> >>> By including schema information only once in the FeatureCollection >>> object, a more compact form of the JSON response may look like this: >>> >>> { >>> "type":"FeatureCollection", >>> "features":[ >>> { >>> "type":"Feature", >>> "id":"bugsites.3", >>> "geometry":{ >>> "type":"Point", >>> "coordinates":[ >>> 590529, >>> 4914625 >>> ] >>> }, >>> "geometry_name":"the_geom", >>> "properties":[ >>> 3, >>> "Beetle site" >>> ] >>> }, >>> { >>> "type":"Feature", >>> "id":"bugsites.4", >>> "geometry":{ >>> "type":"Point", >>> "coordinates":[ >>> 590546, >>> 4915353 >>> ] >>> }, >>> "properties":[ >>> 4, >>> "Beetle site" >>> ] >>> } >>> ], >>> "totalFeatures":2, >>> "numberMatched":2, >>> "numberReturned":2, >>> "timeStamp":"2022-09-13T08:44:45.118Z", >>> "crs":{ >>> "type":"name", >>> "properties":{ >>> "name":"urn:ogc:def:crs:EPSG::26713" >>> } >>> }, >>> "schema":{ >>> "geometry_name":"the_geom", >>> "properties":[ >>> "cat", >>> "str1" >>> ] >>> } >>> } >>> >>> Here, a new "schema" object in the root "FeatureCollection" object >>> contains the name of the geometry field as well as the names of the >>> other properties as an array. The "properties" object in the "Feature" >>> objects have become arrays as well, containing the property values only. >>> >>> Both arrays are "parallel", that is: >>> >>> Key: "schema"."properties"[0] => cat >>> Val: features[N]."properties"[0] => 3 or 4 (depending on N) >>> >>> Key: "schema"."properties"[1] => str1 >>> Val: features[N]."properties"[1] => Beetle site >>> >>> In the above example with only two short-named fields savings are almost >>> zero. However, with requests getting some thousand features, each having >>> 300+ fields, savings may be quite significant. >>> >>> The new compact format is not GeoJSON, of course. However, what >>> GeoServer currently returns is already not really compatible with >>> GeoJSON specs, which, for example, only permit EPGS:4326 coordinates. In >>> any case, that format is likely much smaller for requests described >>> above. >>> >>> On the wire, these responses are typically compressed (deflate, brotli >>> etc.). However, compressing smaller amounts of data typically results in >>> smaller compressed junks. Also, it is not only about transferring the >>> data; the data must be created and compressed by GeoServer and must be >>> decompressed and parsed in the client. After all, smaller junks of data >>> seem to be less resource consuming than larger ones. >>> >>> I'd like to add this new compact format directly into GeoServer's core >>> WFS JSON code: >>> >>> gs-wfs: org.geoserver.wfs.json.GeoJSONGetFeatureResponse >>> >>> More or less, only method encodeSimpleFeatures must be modified in order >>> to implement this new compact JSON format. >>> >>> However, that method iterates over a list of FeatureCollection objects >>> (argument List<FeatureCollection> resultsList). Could there be more than >>> one FeatureCollection is the request returns simple features only? >>> Shouldn't all simple features requested through WFS be of the same type? >>> In other words, must I expect to deal with several distinct "schema" >>> objects when requesting simple features? (don't think so) >>> >>> Since using this format, of course, is optional, I need a mechanism to >>> tell the server whether to return compact JSON or not (and maybe the >>> level of compaction it should use). >>> >>> One quite obvious option is to use vendor parameter "format_options". >>> Here, a parameter like "json:compact" could trigger responding with >>> compact JSON. This seems quite simple to implement. >>> >>> Another idea is to use different outputFormat MIME types or MIME types >>> with additional parameters: >>> >>> application/json; type=compact; omitTypes=true >>> text/javascript; type=compact >>> >>> Although distinguishing the format via the MIME type may be some more >>> work to do, I do prefer this approach over the "format_options" way. >>> >>> What else do I have to consider? >>> >>> Many thanks in advance for your ideas on this :) >>> >>> Regards, Carsten >>> >>> >>> _______________________________________________ >>> Geoserver-devel mailing list >>> Geoserver-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>> >> >> >> -- >> Ian Turton >> > > > -- > Ian Turton > _______________________________________________ > Geoserver-devel mailing list > Geoserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-devel > -- Regards, Andrea Aime == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
