ayushtkn commented on code in PR #6581:
URL: https://github.com/apache/hive/pull/6581#discussion_r3637898160
##########
ql/src/java/org/apache/hadoop/hive/ql/udf/esri/GeometryUtils.java:
##########
@@ -17,29 +17,236 @@
*/
package org.apache.hadoop.hive.ql.udf.esri;
-import com.esri.core.geometry.Geometry;
-import com.esri.core.geometry.GeometryEngine;
-import com.esri.core.geometry.MapGeometry;
-import com.esri.core.geometry.OperatorImportFromESRIShape;
-import com.esri.core.geometry.Polygon;
-import com.esri.core.geometry.SpatialReference;
-import com.esri.core.geometry.ogc.OGCGeometry;
import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorFactory;
import
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableBinaryObjectInspector;
import org.apache.hadoop.io.BytesWritable;
+import org.locationtech.jts.algorithm.Orientation;
+import org.locationtech.jts.geom.CoordinateSequence;
+import org.locationtech.jts.geom.Geometry;
+import org.locationtech.jts.geom.GeometryCollection;
+import org.locationtech.jts.geom.GeometryFactory;
+import org.locationtech.jts.geom.LineString;
+import org.locationtech.jts.geom.LinearRing;
+import org.locationtech.jts.geom.MultiPolygon;
+import org.locationtech.jts.geom.Point;
+import org.locationtech.jts.geom.Polygon;
+import org.locationtech.jts.io.Ordinate;
+import org.locationtech.jts.io.ParseException;
+import org.locationtech.jts.io.WKBReader;
+import org.locationtech.jts.io.WKBWriter;
+import org.locationtech.jts.io.WKTReader;
+import org.locationtech.jts.io.WKTWriter;
+import org.locationtech.jts.io.geojson.GeoJsonReader;
+import org.locationtech.jts.io.geojson.GeoJsonWriter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
+import java.util.Arrays;
+import java.util.Set;
public class GeometryUtils {
+ private static final Logger LOG =
LoggerFactory.getLogger(GeometryUtils.class);
+
private static final int SIZE_WKID = 4;
private static final int SIZE_TYPE = 1;
public static final int WKID_UNKNOWN = 0;
+ // Magic byte at offset 4 to distinguish new WKB format from old ESRI format.
Review Comment:
On mixed tables: Hive reads both formats (auto-detected), and since the old
format was already Hive-specific, mixing doesn't regress non-Hive interop — the
new format is actually more recoverable (strip 5 bytes → WKB).
To normalize a table to the new format there's no dedicated function, but a
WKB round-trip does it: INSERT OVERWRITE TABLE t SELECT ...,
ST_GeomFromWKB(ST_AsBinary(geom), ST_SRID(geom)) FROM t (a plain projection
copies the BINARY bytes as-is, so the round-trip is needed).
Yes — ST_GeomFromShape reads the ESRI shape body only; I've added a comment
to that effect. On the bigger picture: this format is transitional and reading
the old format will always be supported; the long-term direction is a
native/Iceberg GEOMETRY type (which this JTS migration lays the groundwork
for), where functions return geometry instead of BINARY. I'd hold off on an
Impala-style write-format flag for now since read-both already covers
compatibility — happy to revisit if a specific non-Hive consumer needs it.
TBH I haven't started digging into too much on Iceberg Geom right now, My
first plan would be to make it work then see how it operates with these and
tweak accordingly, For me currently Geo types aren't working for writes with
the current Iceberg lib, so I will hold on making claims around it right now :-(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
