Gabriel39 commented on PR #69032: URL: https://github.com/apache/doris/pull/69032#issuecomment-6596060355
Could we make the Iceberg schema carrier preserve the complete spatial type here by adding optional `spatial_crs` / `spatial_algorithm` fields to `TColumnType`, emitting the actual `GEOMETRY` / `GEOGRAPHY` tags in `IcebergSchemaUtils.buildPrimitiveColumnType()`, and reconstructing `DataTypeSpatial` from that metadata? The new guard in [`restore_current_primitive_type()`](https://github.com/apache/doris/blob/bd368a0ba53e6c10fec53da606329fa42341379a/be/src/format_v2/table_reader.cpp#L339) only preserves spatial identity when the caller has already resolved it. The fallback has different sources: - Projected-column annotation passes the slot type from the full `TTypeDesc`. - [Equality-delete field resolution](https://github.com/apache/doris/blob/bd368a0ba53e6c10fec53da606329fa42341379a/be/src/format_v2/table/iceberg_reader.cpp#L2395) initially passes `delete_file_type`, inferred from the delete file. An unannotated WKB column can therefore supply `VARBINARY`, which does not enter this guard; the auxiliary schema still supplies `STRING`. The current workaround consequently leaves schema type reconstruction dependent on the caller's projection or file annotations. Carrying the kind, CRS, and algorithm explicitly would let field-ID lookup reconstruct the declared table type independently, while retaining any necessary legacy-carrier fallback. Please update the FE serialization and BE reconstruction together, including `ScalarType.toColumnTypeThrift()` where applicable. A focused test should assert that schema lookup retains custom CRS/algorithm for an unprojected spatial equality key even when its delete-file fallback is unannotated `VARBINARY`; the existing raw-WKB tests currently accept a `STRING` schema key in that case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
