paleolimbot opened a new issue, #6317: URL: https://github.com/apache/datafusion-comet/issues/6317
### What is the problem the feature request solves? To my knowledge, as of Spark 4.2 the GEOMETRY type will be turned on and the limited set of spatial functions will be present: ST_AsBinary, ST_GeogFromWKB, ST_GeomFromWKB, ST_SRID and ST_SetSRID. As per https://github.com/apache/datafusion-comet/issues/4455 I don't think more advanced functions are in scope for Comet, but implementing those few I think force a few things with respect to the ability to capture Spark's type metadata (the SRID can be either type level or per-row, thus requires some FieldRefs where there are currently DataTypes). ### Describe the potential solution The LakeSail implementation of geometry and geography types is possibly the closest to the scope of Comet's support here: https://github.com/lakehq/sail/pull/1325 , which I believe uses GeoArrow as the Arrow storage but I haven't followed how it handles SRID mappings (I think in Spark there's a mapper from integer to authority:code / string to handle Spark's SRID-based system with Parquet / Icebergs string-based system. ### Additional context Happy to help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
