Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24549 )
Change subject: IMPALA-15160: Add WKB_EXPERIMENTAL geospatial serialization mode ...................................................................... IMPALA-15160: Add WKB_EXPERIMENTAL geospatial serialization mode Add WKB_EXPERIMENTAL as a new value for the --geospatial_library flag. When enabled, the copied ESRI UDFs serialize geometries as OGC Well-Known Binary instead of the ESRI Shape format. The format is chosen at startup via a static member in GeometryUtils. The flag must be consistently set up in coordinators and executors. Key behavioral differences in WKB mode: 1. Native C++ functions are disabled, all functions run in Java. The plan is to implement most functions in c++ in the future with WKB serialiation. 2. WKB doesn't store SRID, a serialization roundtrip sets it to 0. 3. Higher dimension geometry support is dropped. WKB could store them, but not supporting these rare cases for now makes implementing and testing c++ functions easier. 4. Relational functions (ST_Intersects, ST_Contains, ...) accept only (BINARY, BINARY) arguments; the STRING overloads are removed Other changes: - Deserialized geometry caching logic is removed - this was never working in Impala environment which always uses the same ByteBuffer. Change-Id: Ifbb1b91171bf9a8834a55443d4a19518b38b759e Assisted-by: Claude Opus 4.8 Reviewed-on: http://gerrit.cloudera.org:8080/24549 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/common/global-flags.cc M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M fe/src/compat-hive-3/java/org/apache/impala/compat/HiveEsriGeospatialBuiltins.java M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java M java/hive-geospatial-functions/src/main/java/org/apache/impala/hive/geospatial/esri/GeometryUtils.java M java/hive-geospatial-functions/src/main/java/org/apache/impala/hive/geospatial/esri/HiveGeometryOIHelper.java M java/hive-geospatial-functions/src/main/java/org/apache/impala/hive/geospatial/esri/README.md M testdata/workloads/functional-query/queries/QueryTest/geospatial-esri-srid.test M testdata/workloads/functional-query/queries/QueryTest/geospatial-esri.test M tests/custom_cluster/test_geospatial_library.py 12 files changed, 182 insertions(+), 69 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifbb1b91171bf9a8834a55443d4a19518b38b759e Gerrit-Change-Number: 24549 Gerrit-PatchSet: 9 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Balazs Hevele <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]>
