This is an automated email from the ASF dual-hosted git repository.
mchades pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new c2c1b8a672 [#11955] feat(iceberg): native geometry and geography type
support for Iceberg V3 (#11958)
c2c1b8a672 is described below
commit c2c1b8a6727bcd54742e248fb6e5f16e7ca497bb
Author: Nevin Zheng <[email protected]>
AuthorDate: Mon Jul 13 01:57:05 2026 -0700
[#11955] feat(iceberg): native geometry and geography type support for
Iceberg V3 (#11958)
### What changes were proposed in this pull request?
Adds native `Types.GeometryType(crs)` and `Types.GeographyType(crs,
algorithm)` to Gravitino's unified type system, so Iceberg V3 geospatial
columns load through the native metadata API as first-class types
instead of the read-only `ExternalType("GEOMETRY")` /
`ExternalType("GEOGRAPHY")` stopgap, and become creatable through
Gravitino.
- **api**: `Type.Name.GEOMETRY`/`GEOGRAPHY` + two parameterized
`PrimitiveType`s (modeled on `FixedType`/`DecimalType`). CRS is an
opaque string preserved verbatim and compared case-insensitively; the
geography edge algorithm is validated against the five spec values
(`spherical`, `vincenty`, `thomas`, `andoyer`, `karney`) and normalized
to lowercase. Defaults (`OGC:CRS84`, `spherical`) elide to the bare
`geometry`/`geography` token.
- **common (`JsonUtils`)**: registers the default tokens and parses the
parameterized form; the matcher runs against the original
(non-lowercased) type string so a case-sensitive CRS survives the
round-trip.
- **catalog-lakehouse-iceberg**: `FromIcebergType`/`ToIcebergType`
round-trip CRS (and algorithm) via the existing `atomic()` path.
- **clients/client-python, docs, OpenAPI**: mirror both types + serde.
### Why are the changes needed?
Iceberg V3 adds `geometry` and `geography` (planar vs. spheroidal
geospatial shapes, WKB-encoded, parameterized by CRS ± edge algorithm).
Today they load as `external(GEOMETRY/GEOGRAPHY)`, which drops the
CRS/algorithm to a bare name and is opaque to consumers (external types
have caused downstream issues — #10957, #11805), and the write path
throws. Native types preserve the parameters and make the columns
writable. As a catalog, Gravitino only describes the column and
preserves metadata; it does not interpret the WKB.
Fix: #11955
### Does this PR introduce _any_ user-facing change?
Yes. New unified types `Types.GeometryType` / `Types.GeographyType`
(Java + Python), with JSON tokens `geometry` / `geometry(<crs>)` and
`geography` / `geography(<crs>,<algorithm>)`. Iceberg
`geometry`/`geography` columns now load natively and can be created via
Gravitino at `format-version = 3`. No property-key changes.
### How was this patch tested?
- Unit: `TestTypes` (contract + validation + case-insensitivity),
`TestJsonUtils` (JSON serde incl. mixed-case CRS), `TestConvertUtil`
(converter both directions, non-default CRS/algorithm); Python
`test_types.py` + `test_type_serdes.py` (serialize/deserialize
round-trip for `geometry(srid:3857)` and `geography(EPSG:4326,karney)`).
- Integration (REST/IRC backend): `CatalogIcebergRestIT` —
`testV3TypeConversionViaIcebergClient` (IRC→Gravitino native load),
`testCreateGeometryColumnWriteRoundTrip` and
`testCreateGeographyColumnWriteRoundTrip` (Gravitino→IRC write
round-trip). Ran green: 3 tests, 0 skipped, 0 failures.
---------
Co-authored-by: Claude Opus 4.8 <[email protected]>
---
.../java/org/apache/gravitino/rel/types/Type.java | 10 ++
.../java/org/apache/gravitino/rel/types/Types.java | 165 +++++++++++++++++++++
.../java/org/apache/gravitino/rel/TestTypes.java | 26 ++++
.../iceberg/converter/FromIcebergType.java | 8 +
.../lakehouse/iceberg/converter/ToIcebergType.java | 9 ++
.../iceberg/converter/TestConvertUtil.java | 42 ++++++
.../integration/test/CatalogIcebergBaseIT.java | 119 ++++++++++++++-
.../rel/types/json_serdes/_helper/serdes_utils.py | 24 ++-
.../client-python/gravitino/api/rel/types/type.py | 8 +
.../client-python/gravitino/api/rel/types/types.py | 129 ++++++++++++++++
clients/client-python/gravitino/utils/serdes.py | 10 ++
.../tests/unittests/api/rel/test_types.py | 26 ++++
.../unittests/json_serdes/test_type_serdes.py | 7 +
.../java/org/apache/gravitino/json/JsonUtils.java | 19 +++
.../org/apache/gravitino/json/TestJsonUtils.java | 33 +++++
docs/lakehouse-iceberg-catalog.md | 2 +
docs/manage-relational-metadata-using-gravitino.md | 2 +
docs/open-api/datatype.yaml | 4 +
18 files changed, 634 insertions(+), 9 deletions(-)
diff --git a/api/src/main/java/org/apache/gravitino/rel/types/Type.java
b/api/src/main/java/org/apache/gravitino/rel/types/Type.java
index 223762259c..8884a04dc6 100644
--- a/api/src/main/java/org/apache/gravitino/rel/types/Type.java
+++ b/api/src/main/java/org/apache/gravitino/rel/types/Type.java
@@ -78,6 +78,16 @@ public interface Type {
* schema.
*/
VARIANT,
+ /**
+ * The geometry type. A geometry holds a geospatial shape (WKB-encoded) on
a planar coordinate
+ * reference system.
+ */
+ GEOMETRY,
+ /**
+ * The geography type. A geography holds a geospatial shape (WKB-encoded)
on a spheroidal
+ * coordinate reference system, with an edge-interpolation algorithm.
+ */
+ GEOGRAPHY,
/**
* The struct type. A struct type is a complex type that contains a set of
named fields, each
* with a type, and optionally a comment.
diff --git a/api/src/main/java/org/apache/gravitino/rel/types/Types.java
b/api/src/main/java/org/apache/gravitino/rel/types/Types.java
index 11484db648..222d2ba24e 100644
--- a/api/src/main/java/org/apache/gravitino/rel/types/Types.java
+++ b/api/src/main/java/org/apache/gravitino/rel/types/Types.java
@@ -19,8 +19,11 @@
package org.apache.gravitino.rel.types;
import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableSet;
import java.util.Arrays;
+import java.util.Locale;
import java.util.Objects;
+import java.util.Set;
import java.util.StringJoiner;
/** The helper class for {@link Type}. */
@@ -679,6 +682,168 @@ public class Types {
}
}
+ /**
+ * The geometry type in Gravitino. A geometry column holds a geospatial
shape (point, line, or
+ * polygon) encoded as WKB on a planar coordinate reference system (CRS).
The CRS is carried as
+ * type metadata; Gravitino, as a catalog, does not interpret the shape
values.
+ */
+ public static class GeometryType extends Type.PrimitiveType {
+
+ /** The default coordinate reference system, {@code OGC:CRS84} (WGS84
longitude/latitude). */
+ public static final String DEFAULT_CRS = "OGC:CRS84";
+
+ /**
+ * @return A {@link GeometryType} with the default CRS ("OGC:CRS84").
+ */
+ public static GeometryType crs84() {
+ return new GeometryType(DEFAULT_CRS);
+ }
+
+ /**
+ * @param crs The coordinate reference system, e.g. "OGC:CRS84",
"EPSG:4326" or "srid:3857".
+ * @return A {@link GeometryType} with the given CRS.
+ */
+ public static GeometryType of(String crs) {
+ return new GeometryType(crs);
+ }
+
+ private final String crs;
+
+ private GeometryType(String crs) {
+ Preconditions.checkArgument(crs != null && !crs.isEmpty(), "crs cannot
be null or empty");
+ this.crs = crs;
+ }
+
+ /**
+ * @return The coordinate reference system of this geometry type.
+ */
+ public String crs() {
+ return crs;
+ }
+
+ @Override
+ public Name name() {
+ return Name.GEOMETRY;
+ }
+
+ @Override
+ public String simpleString() {
+ return DEFAULT_CRS.equalsIgnoreCase(crs) ? "geometry" :
String.format("geometry(%s)", crs);
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof GeometryType)) {
+ return false;
+ }
+ GeometryType that = (GeometryType) o;
+ return crs.equalsIgnoreCase(that.crs);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(crs.toLowerCase(Locale.ROOT));
+ }
+ }
+
+ /**
+ * The geography type in Gravitino. A geography column holds a geospatial
shape (point, line, or
+ * polygon) encoded as WKB on a spheroidal coordinate reference system
(CRS), with an
+ * edge-interpolation algorithm describing how edges between points are
computed on the surface.
+ * Both are carried as type metadata; Gravitino, as a catalog, does not
interpret the shape
+ * values.
+ */
+ public static class GeographyType extends Type.PrimitiveType {
+
+ /** The default coordinate reference system, {@code OGC:CRS84} (WGS84
longitude/latitude). */
+ public static final String DEFAULT_CRS = "OGC:CRS84";
+
+ /** The default edge-interpolation algorithm, {@code spherical}. */
+ public static final String DEFAULT_ALGORITHM = "spherical";
+
+ /** The edge-interpolation algorithms defined by the geography
specification. */
+ private static final Set<String> VALID_ALGORITHMS =
+ ImmutableSet.of("spherical", "vincenty", "thomas", "andoyer",
"karney");
+
+ /**
+ * @return A {@link GeographyType} with the default CRS ("OGC:CRS84") and
algorithm
+ * ("spherical").
+ */
+ public static GeographyType crs84() {
+ return new GeographyType(DEFAULT_CRS, DEFAULT_ALGORITHM);
+ }
+
+ /**
+ * @param crs The coordinate reference system, e.g. "OGC:CRS84" or
"EPSG:4326".
+ * @param algorithm The edge-interpolation algorithm: one of "spherical",
"vincenty", "thomas",
+ * "andoyer" or "karney".
+ * @return A {@link GeographyType} with the given CRS and algorithm.
+ */
+ public static GeographyType of(String crs, String algorithm) {
+ return new GeographyType(crs, algorithm);
+ }
+
+ private final String crs;
+ private final String algorithm;
+
+ private GeographyType(String crs, String algorithm) {
+ Preconditions.checkArgument(crs != null && !crs.isEmpty(), "crs cannot
be null or empty");
+ Preconditions.checkArgument(
+ algorithm != null &&
VALID_ALGORITHMS.contains(algorithm.toLowerCase(Locale.ROOT)),
+ "algorithm must be one of %s, but got: %s",
+ VALID_ALGORITHMS,
+ algorithm);
+ this.crs = crs;
+ this.algorithm = algorithm.toLowerCase(Locale.ROOT);
+ }
+
+ /**
+ * @return The coordinate reference system of this geography type.
+ */
+ public String crs() {
+ return crs;
+ }
+
+ /**
+ * @return The edge-interpolation algorithm of this geography type.
+ */
+ public String algorithm() {
+ return algorithm;
+ }
+
+ @Override
+ public Name name() {
+ return Name.GEOGRAPHY;
+ }
+
+ @Override
+ public String simpleString() {
+ return DEFAULT_CRS.equalsIgnoreCase(crs) &&
DEFAULT_ALGORITHM.equals(algorithm)
+ ? "geography"
+ : String.format("geography(%s,%s)", crs, algorithm);
+ }
+
+ @Override
+ public boolean equals(Object o) {
+ if (this == o) {
+ return true;
+ }
+ if (!(o instanceof GeographyType)) {
+ return false;
+ }
+ GeographyType that = (GeographyType) o;
+ return crs.equalsIgnoreCase(that.crs) &&
algorithm.equals(that.algorithm);
+ }
+
+ @Override
+ public int hashCode() {
+ return Objects.hash(crs.toLowerCase(Locale.ROOT), algorithm);
+ }
+ }
+
/**
* Fixed-length byte array type, if you want to use variable-length byte
array, use {@link
* BinaryType} instead.
diff --git a/api/src/test/java/org/apache/gravitino/rel/TestTypes.java
b/api/src/test/java/org/apache/gravitino/rel/TestTypes.java
index 8b58751721..4d2b66d3ae 100644
--- a/api/src/test/java/org/apache/gravitino/rel/TestTypes.java
+++ b/api/src/test/java/org/apache/gravitino/rel/TestTypes.java
@@ -159,6 +159,32 @@ public class TestTypes {
Assertions.assertSame(variantType, Types.VariantType.get());
Assertions.assertEquals("variant", variantType.simpleString());
+ Types.GeometryType geometryType = Types.GeometryType.crs84();
+ Assertions.assertEquals(Type.Name.GEOMETRY, geometryType.name());
+ Assertions.assertEquals("OGC:CRS84", geometryType.crs());
+ Assertions.assertEquals("geometry", geometryType.simpleString());
+ Assertions.assertEquals(geometryType, Types.GeometryType.of("ogc:crs84"));
+ Assertions.assertEquals("geometry",
Types.GeometryType.of("ogc:crs84").simpleString());
+ Types.GeometryType customCrs = Types.GeometryType.of("srid:3857");
+ Assertions.assertEquals("geometry(srid:3857)", customCrs.simpleString());
+ Assertions.assertNotEquals(geometryType, customCrs);
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
Types.GeometryType.of(""));
+
+ Types.GeographyType geographyType = Types.GeographyType.crs84();
+ Assertions.assertEquals(Type.Name.GEOGRAPHY, geographyType.name());
+ Assertions.assertEquals("OGC:CRS84", geographyType.crs());
+ Assertions.assertEquals("spherical", geographyType.algorithm());
+ Assertions.assertEquals("geography", geographyType.simpleString());
+ Assertions.assertEquals(geographyType, Types.GeographyType.of("ogc:crs84",
"SPHERICAL"));
+ Types.GeographyType karney = Types.GeographyType.of("EPSG:4326", "Karney");
+ Assertions.assertEquals("karney", karney.algorithm());
+ Assertions.assertEquals("geography(EPSG:4326,karney)",
karney.simpleString());
+ Assertions.assertNotEquals(geographyType, karney);
+ Assertions.assertThrows(
+ IllegalArgumentException.class, () ->
Types.GeographyType.of("OGC:CRS84", "bogus"));
+ Assertions.assertThrows(
+ IllegalArgumentException.class, () -> Types.GeographyType.of("",
"spherical"));
+
Types.FixedType fixedType = Types.FixedType.of(10);
Assertions.assertEquals(Type.Name.FIXED, fixedType.name());
Assertions.assertEquals(10, fixedType.length());
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/FromIcebergType.java
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/FromIcebergType.java
index 02b8be1dbd..12bc00ed36 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/FromIcebergType.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/FromIcebergType.java
@@ -120,6 +120,14 @@ public class FromIcebergType extends
TypeUtil.SchemaVisitor<Type> {
Types.DecimalType decimal = (Types.DecimalType) primitive;
return org.apache.gravitino.rel.types.Types.DecimalType.of(
decimal.precision(), decimal.scale());
+ case GEOMETRY:
+ return org.apache.gravitino.rel.types.Types.GeometryType.of(
+ ((Types.GeometryType) primitive).crs());
+ case GEOGRAPHY:
+ Types.GeographyType geography = (Types.GeographyType) primitive;
+ // EdgeAlgorithm.toString() is the lowercase spec name, which
GeographyType requires.
+ return org.apache.gravitino.rel.types.Types.GeographyType.of(
+ geography.crs(), geography.algorithm().toString());
default:
return
org.apache.gravitino.rel.types.Types.ExternalType.of(primitive.typeId().name());
}
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ToIcebergType.java
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ToIcebergType.java
index b41e29cf12..2b0ec5d4d0 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ToIcebergType.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/ToIcebergType.java
@@ -20,6 +20,7 @@ package
org.apache.gravitino.catalog.lakehouse.iceberg.converter;
import com.google.common.collect.Lists;
import java.util.List;
+import org.apache.iceberg.types.EdgeAlgorithm;
import org.apache.iceberg.types.Type;
import org.apache.iceberg.types.Types;
@@ -164,6 +165,14 @@ public class ToIcebergType extends
ToIcebergTypeVisitor<Type> {
return Types.UUIDType.get();
} else if (primitive instanceof
org.apache.gravitino.rel.types.Types.VariantType) {
return Types.VariantType.get();
+ } else if (primitive instanceof
org.apache.gravitino.rel.types.Types.GeometryType) {
+ return Types.GeometryType.of(
+ ((org.apache.gravitino.rel.types.Types.GeometryType)
primitive).crs());
+ } else if (primitive instanceof
org.apache.gravitino.rel.types.Types.GeographyType) {
+ org.apache.gravitino.rel.types.Types.GeographyType geography =
+ (org.apache.gravitino.rel.types.Types.GeographyType) primitive;
+ // EdgeAlgorithm.fromName is case-insensitive; the Gravitino algorithm
is a validated name.
+ return Types.GeographyType.of(geography.crs(),
EdgeAlgorithm.fromName(geography.algorithm()));
}
throw new UnsupportedOperationException("Not a supported type: " +
primitive.toString());
}
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/TestConvertUtil.java
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/TestConvertUtil.java
index 659f180a2c..99beaa52ce 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/TestConvertUtil.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/converter/TestConvertUtil.java
@@ -36,6 +36,7 @@ import org.apache.gravitino.rel.expressions.sorts.SortOrder;
import org.apache.gravitino.rel.types.Types.ByteType;
import org.apache.gravitino.rel.types.Types.ShortType;
import org.apache.iceberg.Schema;
+import org.apache.iceberg.types.EdgeAlgorithm;
import org.apache.iceberg.types.Type;
import org.apache.iceberg.types.Types;
import org.junit.jupiter.api.Assertions;
@@ -54,6 +55,47 @@ public class TestConvertUtil extends TestBaseConvert {
instanceof Types.VariantType);
}
+ @Test
+ public void testGeometryType() {
+ // Iceberg V3 geometry <-> Gravitino GeometryType, preserving the CRS in
both directions.
+ org.apache.gravitino.rel.types.Type defaultFromIceberg =
+ CONVERTER.toGravitino(Types.GeometryType.crs84());
+ Assertions.assertInstanceOf(
+ org.apache.gravitino.rel.types.Types.GeometryType.class,
defaultFromIceberg);
+ Assertions.assertEquals(
+ "OGC:CRS84",
+ ((org.apache.gravitino.rel.types.Types.GeometryType)
defaultFromIceberg).crs());
+
+ org.apache.gravitino.rel.types.Types.GeometryType customCrs =
+ org.apache.gravitino.rel.types.Types.GeometryType.of("srid:3857");
+ org.apache.iceberg.types.Type icebergCustom =
CONVERTER.fromGravitino(customCrs);
+ Assertions.assertInstanceOf(Types.GeometryType.class, icebergCustom);
+ Assertions.assertEquals("srid:3857", ((Types.GeometryType)
icebergCustom).crs());
+ Assertions.assertEquals(customCrs, CONVERTER.toGravitino(icebergCustom));
+ }
+
+ @Test
+ public void testGeographyType() {
+ // Iceberg V3 geography <-> Gravitino GeographyType, preserving CRS and
algorithm both ways.
+ org.apache.gravitino.rel.types.Type defaultFromIceberg =
+ CONVERTER.toGravitino(Types.GeographyType.crs84());
+ Assertions.assertInstanceOf(
+ org.apache.gravitino.rel.types.Types.GeographyType.class,
defaultFromIceberg);
+ org.apache.gravitino.rel.types.Types.GeographyType defaultGeography =
+ (org.apache.gravitino.rel.types.Types.GeographyType)
defaultFromIceberg;
+ Assertions.assertEquals("OGC:CRS84", defaultGeography.crs());
+ Assertions.assertEquals("spherical", defaultGeography.algorithm());
+
+ org.apache.gravitino.rel.types.Types.GeographyType custom =
+ org.apache.gravitino.rel.types.Types.GeographyType.of("EPSG:4326",
"karney");
+ org.apache.iceberg.types.Type icebergCustom =
CONVERTER.fromGravitino(custom);
+ Assertions.assertInstanceOf(Types.GeographyType.class, icebergCustom);
+ Types.GeographyType icebergGeography = (Types.GeographyType) icebergCustom;
+ Assertions.assertEquals("EPSG:4326", icebergGeography.crs());
+ Assertions.assertEquals(EdgeAlgorithm.KARNEY,
icebergGeography.algorithm());
+ Assertions.assertEquals(custom, CONVERTER.toGravitino(icebergCustom));
+ }
+
@Test
public void testToIcebergSchema() {
Column[] columns = createColumns("col_1", "col_2", "col_3", "col_4");
diff --git
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
index eb568ae126..53badb768e 100644
---
a/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
+++
b/catalogs/catalog-lakehouse-iceberg/src/test/java/org/apache/gravitino/catalog/lakehouse/iceberg/integration/test/CatalogIcebergBaseIT.java
@@ -646,10 +646,18 @@ public abstract class CatalogIcebergBaseIT extends BaseIT
{
// like Spark 4 would, directly through the Iceberg catalog, then load
through the native
// metadata interface.
//
- // variant has native support and loads as VariantType. The other V3
net-new types are not
- // modeled in Gravitino's unified type system yet and load as
ExternalType; native support for
- // them is pending and tracked in apache/gravitino#11929.
+ // variant loads as VariantType; geometry and geography load as their
native types. The
+ // remaining V3 net-new types are not modeled in Gravitino's unified type
system yet and load
+ // as ExternalType; native support for them is pending and tracked in
apache/gravitino#11929.
assertV3LoadsAsVariant("v3_variant");
+ assertV3LoadsAsGeometry("v3_geometry",
org.apache.iceberg.types.Types.GeometryType.crs84());
+ assertV3LoadsAsGeometry(
+ "v3_geometry_srid",
org.apache.iceberg.types.Types.GeometryType.of("srid:3857"));
+ assertV3LoadsAsGeography("v3_geography",
org.apache.iceberg.types.Types.GeographyType.crs84());
+ assertV3LoadsAsGeography(
+ "v3_geography_karney",
+ org.apache.iceberg.types.Types.GeographyType.of(
+ "EPSG:4326", org.apache.iceberg.types.EdgeAlgorithm.KARNEY));
// TODO(apache/gravitino#11929): expect native types once these gain
unified-model support.
assertV3LoadsAsExternal(
@@ -660,10 +668,6 @@ public abstract class CatalogIcebergBaseIT extends BaseIT {
"v3_timestamptz_ns",
org.apache.iceberg.types.Types.TimestampNanoType.withZone(),
"TIMESTAMP_NANO");
- assertV3LoadsAsExternal(
- "v3_geometry", org.apache.iceberg.types.Types.GeometryType.crs84(),
"GEOMETRY");
- assertV3LoadsAsExternal(
- "v3_geography", org.apache.iceberg.types.Types.GeographyType.crs84(),
"GEOGRAPHY");
assertV3LoadsAsExternal(
"v3_unknown", org.apache.iceberg.types.Types.UnknownType.get(),
"UNKNOWN");
}
@@ -730,6 +734,89 @@ public abstract class CatalogIcebergBaseIT extends BaseIT {
"3",
loaded.properties().get(IcebergTablePropertiesMetadata.FORMAT_VERSION));
}
+ @Test
+ void testCreateGeometryColumnWriteRoundTrip() {
+ // Write path: create a geometry column with a non-default CRS *through
the Gravitino relational
+ // API*, write it to the REST (IRC) backend, then load it back and confirm
both the type and the
+ // CRS round-trip. REST-backend only, for the same reason as
+ // testV3TypeConversionViaIcebergClient: the CI Hive metastore cannot
store V3 column types.
+ Assumptions.assumeTrue(
+ "rest".equalsIgnoreCase(TYPE),
+ "Geometry columns require a backend that does not validate against the
Hive metastore");
+
+ NameIdentifier ident = NameIdentifier.of(schemaName, "t_geometry_write");
+ Column[] columns =
+ new Column[] {
+ Column.of("id", Types.IntegerType.get(), "id"),
+ Column.of("shape", Types.GeometryType.of("srid:3857"), "geometry
col")
+ };
+ Map<String, String> properties = Maps.newHashMap();
+ // Geometry is a V3 type, so the table must be created at format-version 3.
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, "3");
+
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ Table created =
+ tableCatalog.createTable(
+ ident,
+ columns,
+ "geometry write",
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ Distributions.NONE,
+ new SortOrder[0]);
+ Assertions.assertInstanceOf(Types.GeometryType.class,
created.columns()[1].dataType());
+
+ // Load it back through the native metadata API to confirm the round-trip
against the backend.
+ Table loaded = tableCatalog.loadTable(ident);
+ Assertions.assertInstanceOf(Types.GeometryType.class,
loaded.columns()[1].dataType());
+ Assertions.assertEquals(
+ "srid:3857", ((Types.GeometryType)
loaded.columns()[1].dataType()).crs());
+ Assertions.assertEquals(
+ "3",
loaded.properties().get(IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
+ @Test
+ void testCreateGeographyColumnWriteRoundTrip() {
+ // Write path: create a geography column with a non-default CRS and edge
algorithm *through the
+ // Gravitino relational API*, write it to the REST (IRC) backend, then
load it back and confirm
+ // the type, CRS and algorithm round-trip. REST-backend only, for the same
reason as
+ // testV3TypeConversionViaIcebergClient: the CI Hive metastore cannot
store V3 column types.
+ Assumptions.assumeTrue(
+ "rest".equalsIgnoreCase(TYPE),
+ "Geography columns require a backend that does not validate against
the Hive metastore");
+
+ NameIdentifier ident = NameIdentifier.of(schemaName, "t_geography_write");
+ Column[] columns =
+ new Column[] {
+ Column.of("id", Types.IntegerType.get(), "id"),
+ Column.of("shape", Types.GeographyType.of("EPSG:4326", "karney"),
"geography col")
+ };
+ Map<String, String> properties = Maps.newHashMap();
+ // Geography is a V3 type, so the table must be created at format-version
3.
+ properties.put(IcebergTablePropertiesMetadata.FORMAT_VERSION, "3");
+
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ Table created =
+ tableCatalog.createTable(
+ ident,
+ columns,
+ "geography write",
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ Distributions.NONE,
+ new SortOrder[0]);
+ Assertions.assertInstanceOf(Types.GeographyType.class,
created.columns()[1].dataType());
+
+ // Load it back through the native metadata API to confirm the round-trip
against the backend.
+ Table loaded = tableCatalog.loadTable(ident);
+ Assertions.assertInstanceOf(Types.GeographyType.class,
loaded.columns()[1].dataType());
+ Types.GeographyType geography = (Types.GeographyType)
loaded.columns()[1].dataType();
+ Assertions.assertEquals("EPSG:4326", geography.crs());
+ Assertions.assertEquals("karney", geography.algorithm());
+ Assertions.assertEquals(
+ "3",
loaded.properties().get(IcebergTablePropertiesMetadata.FORMAT_VERSION));
+ }
+
@Test
void testCreateVariantColumnRequiresFormatVersion3() {
Assumptions.assumeTrue(
@@ -825,6 +912,24 @@ public abstract class CatalogIcebergBaseIT extends BaseIT {
Assertions.assertInstanceOf(Types.VariantType.class, loaded.dataType());
}
+ private void assertV3LoadsAsGeometry(
+ String tableName, org.apache.iceberg.types.Types.GeometryType
icebergType) {
+ NameIdentifier ident = createV3Table(tableName, icebergType);
+ Column loaded = catalog.asTableCatalog().loadTable(ident).columns()[0];
+ Assertions.assertInstanceOf(Types.GeometryType.class, loaded.dataType());
+ Assertions.assertEquals(icebergType.crs(), ((Types.GeometryType)
loaded.dataType()).crs());
+ }
+
+ private void assertV3LoadsAsGeography(
+ String tableName, org.apache.iceberg.types.Types.GeographyType
icebergType) {
+ NameIdentifier ident = createV3Table(tableName, icebergType);
+ Column loaded = catalog.asTableCatalog().loadTable(ident).columns()[0];
+ Assertions.assertInstanceOf(Types.GeographyType.class, loaded.dataType());
+ Types.GeographyType geography = (Types.GeographyType) loaded.dataType();
+ Assertions.assertEquals(icebergType.crs(), geography.crs());
+ Assertions.assertEquals(icebergType.algorithm().toString(),
geography.algorithm());
+ }
+
private void assertV3LoadsAsExternal(
String tableName, org.apache.iceberg.types.Type icebergType, String
expectedCatalogString) {
NameIdentifier ident = createV3Table(tableName, icebergType);
diff --git
a/clients/client-python/gravitino/api/rel/types/json_serdes/_helper/serdes_utils.py
b/clients/client-python/gravitino/api/rel/types/json_serdes/_helper/serdes_utils.py
index 25656fa0e4..cdc60abb59 100644
---
a/clients/client-python/gravitino/api/rel/types/json_serdes/_helper/serdes_utils.py
+++
b/clients/client-python/gravitino/api/rel/types/json_serdes/_helper/serdes_utils.py
@@ -157,7 +157,7 @@ class SerdesUtils(SerdesUtilsBase):
)
if isinstance(type_data, str):
- return cls.from_primitive_type_string(type_data.lower())
+ return cls.from_primitive_type_string(type_data.lower(), type_data)
if isinstance(type_data, dict) and type_data.get(cls.TYPE):
type_str = type_data[cls.TYPE]
@@ -177,11 +177,31 @@ class SerdesUtils(SerdesUtilsBase):
return Types.UnparsedType.of(unparsed_type=json.dumps(type_data))
@classmethod
- def from_primitive_type_string(cls, type_string: str) -> Type:
+ def from_primitive_type_string( # pylint:
disable=too-many-return-statements
+ cls, type_string: str, original_type_string: str = None
+ ) -> Type:
+ # Flat type dispatch: one return per supported type, no branching
logic — the return
+ # count reflects the number of types, not complexity. A simple
converter function.
type_instance = cls.TYPES.get(type_string)
if type_instance is not None:
return type_instance
+ # Match against the original (non-lowercased) string so the CRS keeps
its case.
+ # fullmatch (not match) so a trailing suffix can't partially parse,
matching the
+ # Java side's Matcher.matches().
+ original = (
+ original_type_string if original_type_string is not None else
type_string
+ )
+ geometry_matched = cls.GEOMETRY_PATTERN.fullmatch(original)
+ if geometry_matched:
+ return Types.GeometryType.of(geometry_matched.group(1))
+
+ geography_matched = cls.GEOGRAPHY_PATTERN.fullmatch(original)
+ if geography_matched:
+ return Types.GeographyType.of(
+ geography_matched.group(1), geography_matched.group(2)
+ )
+
decimal_matched = cls.DECIMAL_PATTERN.match(type_string)
if decimal_matched:
return Types.DecimalType.of(
diff --git a/clients/client-python/gravitino/api/rel/types/type.py
b/clients/client-python/gravitino/api/rel/types/type.py
index 7cf9f921d8..949bf5378d 100644
--- a/clients/client-python/gravitino/api/rel/types/type.py
+++ b/clients/client-python/gravitino/api/rel/types/type.py
@@ -85,6 +85,14 @@ class Name(Enum):
""" The variant type. A variant holds semi-structured data whose shape is
not fixed by the
schema. """
+ GEOMETRY = "GEOMETRY"
+ """ The geometry type. A geometry holds a geospatial shape (WKB-encoded)
on a planar coordinate
+ reference system. """
+
+ GEOGRAPHY = "GEOGRAPHY"
+ """ The geography type. A geography holds a geospatial shape (WKB-encoded)
on a spheroidal
+ coordinate reference system, with an edge-interpolation algorithm. """
+
STRUCT = "STRUCT"
"""
The struct type.
diff --git a/clients/client-python/gravitino/api/rel/types/types.py
b/clients/client-python/gravitino/api/rel/types/types.py
index 622c4ac8cc..35165daedc 100644
--- a/clients/client-python/gravitino/api/rel/types/types.py
+++ b/clients/client-python/gravitino/api/rel/types/types.py
@@ -728,6 +728,135 @@ class Types:
def simple_string(self) -> str:
return "variant"
+ class GeometryType(PrimitiveType):
+ """The geometry type in Gravitino. A geometry column holds a
geospatial shape
+ (WKB-encoded) on a planar coordinate reference system (CRS). The CRS
is carried as type
+ metadata."""
+
+ DEFAULT_CRS: str = "OGC:CRS84"
+
+ _crs: str
+
+ def __init__(self, crs: str):
+ if not crs:
+ raise ValueError("crs cannot be null or empty")
+ self._crs = crs
+
+ @classmethod
+ def crs84(cls) -> Types.GeometryType:
+ """
+ Returns:
+ A GeometryType instance with the default CRS ("OGC:CRS84").
+ """
+ return cls(cls.DEFAULT_CRS)
+
+ @classmethod
+ def of(cls, crs: str) -> Types.GeometryType:
+ """
+ Args:
+ crs: The coordinate reference system, e.g. "OGC:CRS84",
"EPSG:4326" or "srid:3857".
+
+ Returns:
+ A GeometryType instance with the given CRS.
+ """
+ return cls(crs)
+
+ def name(self) -> Name:
+ return Name.GEOMETRY
+
+ def crs(self) -> str:
+ return self._crs
+
+ def simple_string(self) -> str:
+ # The default CRS elides to a bare "geometry"; a custom CRS is
carried verbatim.
+ if self._crs.lower() == self.DEFAULT_CRS.lower():
+ return "geometry"
+ return f"geometry({self._crs})"
+
+ def __eq__(self, other):
+ if not isinstance(other, Types.GeometryType):
+ return False
+ return self._crs.lower() == other._crs.lower()
+
+ def __hash__(self):
+ return hash(self._crs.lower())
+
+ class GeographyType(PrimitiveType):
+ """The geography type in Gravitino. A geography column holds a
geospatial shape
+ (WKB-encoded) on a spheroidal coordinate reference system (CRS), with
an edge-interpolation
+ algorithm. Both are carried as type metadata."""
+
+ DEFAULT_CRS: str = "OGC:CRS84"
+ DEFAULT_ALGORITHM: str = "spherical"
+ _VALID_ALGORITHMS = frozenset(
+ {"spherical", "vincenty", "thomas", "andoyer", "karney"}
+ )
+
+ _crs: str
+ _algorithm: str
+
+ def __init__(self, crs: str, algorithm: str):
+ if not crs:
+ raise ValueError("crs cannot be null or empty")
+ if algorithm is None or algorithm.lower() not in
self._VALID_ALGORITHMS:
+ raise ValueError(
+ f"algorithm must be one of
{sorted(self._VALID_ALGORITHMS)}, "
+ f"but got: {algorithm}"
+ )
+ self._crs = crs
+ self._algorithm = algorithm.lower()
+
+ @classmethod
+ def crs84(cls) -> Types.GeographyType:
+ """
+ Returns:
+ A GeographyType instance with the default CRS ("OGC:CRS84")
and algorithm
+ ("spherical").
+ """
+ return cls(cls.DEFAULT_CRS, cls.DEFAULT_ALGORITHM)
+
+ @classmethod
+ def of(cls, crs: str, algorithm: str) -> Types.GeographyType:
+ """
+ Args:
+ crs: The coordinate reference system, e.g. "OGC:CRS84" or
"EPSG:4326".
+ algorithm: The edge-interpolation algorithm: one of
"spherical", "vincenty",
+ "thomas", "andoyer" or "karney".
+
+ Returns:
+ A GeographyType instance with the given CRS and algorithm.
+ """
+ return cls(crs, algorithm)
+
+ def name(self) -> Name:
+ return Name.GEOGRAPHY
+
+ def crs(self) -> str:
+ return self._crs
+
+ def algorithm(self) -> str:
+ return self._algorithm
+
+ def simple_string(self) -> str:
+ # The default CRS and algorithm elide to a bare "geography";
otherwise both are carried.
+ if (
+ self._crs.lower() == self.DEFAULT_CRS.lower()
+ and self._algorithm == self.DEFAULT_ALGORITHM
+ ):
+ return "geography"
+ return f"geography({self._crs},{self._algorithm})"
+
+ def __eq__(self, other):
+ if not isinstance(other, Types.GeographyType):
+ return False
+ return (
+ self._crs.lower() == other._crs.lower()
+ and self._algorithm == other._algorithm
+ )
+
+ def __hash__(self):
+ return hash((self._crs.lower(), self._algorithm))
+
class StructType(ComplexType):
"""The struct type in Gravitino."""
diff --git a/clients/client-python/gravitino/utils/serdes.py
b/clients/client-python/gravitino/utils/serdes.py
index fa5fbe3c80..3130712dfd 100644
--- a/clients/client-python/gravitino/utils/serdes.py
+++ b/clients/client-python/gravitino/utils/serdes.py
@@ -92,6 +92,14 @@ class SerdesUtilsBase:
TIMESTAMP_TZ_PATTERN: Final[Pattern[str]] = re.compile(
r"timestamp_tz\(\s*(\d+)\s*\)"
)
+ # The keyword folds case; group 1 is the CRS, an opaque identifier kept
as-is.
+ GEOMETRY_PATTERN: Final[Pattern[str]] = re.compile(
+ r"geometry\(\s*(.+?)\s*\)", re.IGNORECASE
+ )
+ # Group 1 is the CRS (kept as-is); group 2 is the edge-interpolation
algorithm.
+ GEOGRAPHY_PATTERN: Final[Pattern[str]] = re.compile(
+ r"geography\(\s*(.+?)\s*,\s*(.+?)\s*\)", re.IGNORECASE
+ )
TYPES: Final[Mapping] = MappingProxyType(
{
type_instance.simple_string(): type_instance
@@ -117,6 +125,8 @@ class SerdesUtilsBase:
Types.StringType.get(),
Types.UUIDType.get(),
Types.VariantType.get(),
+ Types.GeometryType.crs84(),
+ Types.GeographyType.crs84(),
)
}
)
diff --git a/clients/client-python/tests/unittests/api/rel/test_types.py
b/clients/client-python/tests/unittests/api/rel/test_types.py
index 4cf0d41cc9..cfa78390f9 100644
--- a/clients/client-python/tests/unittests/api/rel/test_types.py
+++ b/clients/client-python/tests/unittests/api/rel/test_types.py
@@ -188,6 +188,32 @@ class TestTypes(unittest.TestCase):
self.assertEqual(instance.simple_string(), "variant")
self.assertIs(instance, Types.VariantType.get())
+ def test_geometry_type(self):
+ instance: Types.GeometryType = Types.GeometryType.crs84()
+ self.assertEqual(instance.name(), Name.GEOMETRY)
+ self.assertEqual(instance.crs(), "OGC:CRS84")
+ self.assertEqual(instance.simple_string(), "geometry")
+ self.assertEqual(instance, Types.GeometryType.of("ogc:crs84"))
+ self.assertEqual(Types.GeometryType.of("ogc:crs84").simple_string(),
"geometry")
+ custom = Types.GeometryType.of("srid:3857")
+ self.assertEqual(custom.simple_string(), "geometry(srid:3857)")
+ self.assertNotEqual(instance, custom)
+ self.assertRaises(ValueError, Types.GeometryType.of, "")
+
+ def test_geography_type(self):
+ instance: Types.GeographyType = Types.GeographyType.crs84()
+ self.assertEqual(instance.name(), Name.GEOGRAPHY)
+ self.assertEqual(instance.crs(), "OGC:CRS84")
+ self.assertEqual(instance.algorithm(), "spherical")
+ self.assertEqual(instance.simple_string(), "geography")
+ self.assertEqual(instance, Types.GeographyType.of("ogc:crs84",
"SPHERICAL"))
+ karney = Types.GeographyType.of("EPSG:4326", "Karney")
+ self.assertEqual(karney.algorithm(), "karney")
+ self.assertEqual(karney.simple_string(), "geography(EPSG:4326,karney)")
+ self.assertNotEqual(instance, karney)
+ self.assertRaises(ValueError, Types.GeographyType.of, "OGC:CRS84",
"bogus")
+ self.assertRaises(ValueError, Types.GeographyType.of, "", "spherical")
+
def test_fixed_type(self):
instance: Types.FixedType = Types.FixedType.of(5)
self.assertEqual(instance.name(), Name.FIXED)
diff --git
a/clients/client-python/tests/unittests/json_serdes/test_type_serdes.py
b/clients/client-python/tests/unittests/json_serdes/test_type_serdes.py
index f20e55cb9d..4be159e04f 100644
--- a/clients/client-python/tests/unittests/json_serdes/test_type_serdes.py
+++ b/clients/client-python/tests/unittests/json_serdes/test_type_serdes.py
@@ -65,6 +65,10 @@ class TestTypeSerdes(unittest.TestCase):
"timestamp(0)": Types.TimestampType.without_time_zone(0),
"timestamp_tz(6)": Types.TimestampType.with_time_zone(6),
"timestamp_tz(3)": Types.TimestampType.with_time_zone(3),
+ "geometry(srid:3857)": Types.GeometryType.of("srid:3857"),
+ "geography(EPSG:4326,karney)": Types.GeographyType.of(
+ "EPSG:4326", "karney"
+ ),
},
}
@@ -191,6 +195,9 @@ class TestTypeSerdes(unittest.TestCase):
{"invalid_key": "value"},
list(range(10)),
True,
+ # A valid-looking prefix with trailing characters must not
partially parse.
+ "geometry(EPSG:4326) trailing",
+ "geography(OGC:CRS84,spherical) trailing",
]
for data in unparsed_data:
result = TypeSerdes.deserialize(data=data)
diff --git a/common/src/main/java/org/apache/gravitino/json/JsonUtils.java
b/common/src/main/java/org/apache/gravitino/json/JsonUtils.java
index 5c87d28566..ad71feb557 100644
--- a/common/src/main/java/org/apache/gravitino/json/JsonUtils.java
+++ b/common/src/main/java/org/apache/gravitino/json/JsonUtils.java
@@ -171,6 +171,8 @@ public class JsonUtils {
Types.StringType.get(),
Types.UUIDType.get(),
Types.VariantType.get(),
+ Types.GeometryType.crs84(),
+ Types.GeographyType.crs84(),
Types.BinaryType.get(),
Types.IntervalYearType.get(),
Types.IntervalDayType.get()),
@@ -180,6 +182,10 @@ public class JsonUtils {
private static final Pattern VARCHAR =
Pattern.compile("varchar\\(\\s*(\\d+)\\s*\\)");
private static final Pattern DECIMAL =
Pattern.compile("decimal\\(\\s*(\\d+)\\s*,\\s*(\\d+)\\s*\\)");
+ private static final Pattern GEOMETRY =
+ Pattern.compile("geometry\\(\\s*(.+?)\\s*\\)", Pattern.CASE_INSENSITIVE);
+ private static final Pattern GEOGRAPHY =
+ Pattern.compile("geography\\(\\s*(.+?)\\s*,\\s*(.+?)\\s*\\)",
Pattern.CASE_INSENSITIVE);
private static final Pattern TIME = Pattern.compile("time\\((\\d+)\\)");
private static final Pattern TIMESTAMP_TZ =
Pattern.compile("timestamp_tz\\((\\d+)\\)");
private static final Pattern TIMESTAMP =
Pattern.compile("timestamp\\((\\d+)\\)");
@@ -682,6 +688,8 @@ public class JsonUtils {
case INTERVAL_YEAR:
case UUID:
case VARIANT:
+ case GEOMETRY:
+ case GEOGRAPHY:
case FIXED:
case BINARY:
case NULL:
@@ -871,6 +879,17 @@ public class JsonUtils {
Integer.parseInt(decimal.group(1)),
Integer.parseInt(decimal.group(2)));
}
+ // Match against the original string, not the lowercased one, so the CRS
keeps its case.
+ Matcher geometry = GEOMETRY.matcher(orignalTypeString);
+ if (geometry.matches()) {
+ return Types.GeometryType.of(geometry.group(1));
+ }
+
+ Matcher geography = GEOGRAPHY.matcher(orignalTypeString);
+ if (geography.matches()) {
+ return Types.GeographyType.of(geography.group(1), geography.group(2));
+ }
+
Matcher time = TIME.matcher(lowerTypeString);
if (time.matches()) {
return Types.TimeType.of(Integer.parseInt(time.group(1)));
diff --git a/common/src/test/java/org/apache/gravitino/json/TestJsonUtils.java
b/common/src/test/java/org/apache/gravitino/json/TestJsonUtils.java
index da92570e1d..84497a0505 100644
--- a/common/src/test/java/org/apache/gravitino/json/TestJsonUtils.java
+++ b/common/src/test/java/org/apache/gravitino/json/TestJsonUtils.java
@@ -126,6 +126,39 @@ public class TestJsonUtils {
Assertions.assertEquals(
Types.VariantType.get(), JsonUtils.objectMapper().readValue(jsonValue,
Type.class));
+ type = Types.GeometryType.crs84();
+ jsonValue = JsonUtils.objectMapper().writeValueAsString(type);
+ expected = "\"geometry\"";
+ Assertions.assertEquals(objectMapper.readTree(expected),
objectMapper.readTree(jsonValue));
+ Assertions.assertEquals(
+ Types.GeometryType.crs84(),
JsonUtils.objectMapper().readValue(jsonValue, Type.class));
+
+ // A custom, case-sensitive CRS round-trips verbatim (it is not lowercased
on parse).
+ type = Types.GeometryType.of("EPSG:4326");
+ jsonValue = JsonUtils.objectMapper().writeValueAsString(type);
+ expected = "\"geometry(EPSG:4326)\"";
+ Assertions.assertEquals(objectMapper.readTree(expected),
objectMapper.readTree(jsonValue));
+ Types.GeometryType parsedGeometry =
+ (Types.GeometryType) JsonUtils.objectMapper().readValue(jsonValue,
Type.class);
+ Assertions.assertEquals("EPSG:4326", parsedGeometry.crs());
+
+ type = Types.GeographyType.crs84();
+ jsonValue = JsonUtils.objectMapper().writeValueAsString(type);
+ expected = "\"geography\"";
+ Assertions.assertEquals(objectMapper.readTree(expected),
objectMapper.readTree(jsonValue));
+ Assertions.assertEquals(
+ Types.GeographyType.crs84(),
JsonUtils.objectMapper().readValue(jsonValue, Type.class));
+
+ // A custom CRS + algorithm round-trips, with the case-sensitive CRS
preserved verbatim.
+ type = Types.GeographyType.of("EPSG:4326", "karney");
+ jsonValue = JsonUtils.objectMapper().writeValueAsString(type);
+ expected = "\"geography(EPSG:4326,karney)\"";
+ Assertions.assertEquals(objectMapper.readTree(expected),
objectMapper.readTree(jsonValue));
+ Types.GeographyType parsedGeography =
+ (Types.GeographyType) JsonUtils.objectMapper().readValue(jsonValue,
Type.class);
+ Assertions.assertEquals("EPSG:4326", parsedGeography.crs());
+ Assertions.assertEquals("karney", parsedGeography.algorithm());
+
type =
Types.StructType.of(
Types.StructType.Field.nullableField("name",
Types.StringType.get(), "name field"),
diff --git a/docs/lakehouse-iceberg-catalog.md
b/docs/lakehouse-iceberg-catalog.md
index df9a44aaf3..b71fb65784 100644
--- a/docs/lakehouse-iceberg-catalog.md
+++ b/docs/lakehouse-iceberg-catalog.md
@@ -435,6 +435,8 @@ If you doesn't specify distribution expressions, the table
distribution will be
| `Binary` | `Binary` |
| `UUID` | `UUID` |
| `Variant` | `Variant` |
+| `Geometry` | `Geometry` |
+| `Geography` | `Geography` |
:::info
Apache Iceberg doesn't support Gravitino `Varchar` `Fixedchar` `Byte` `Short`
`Union` type.
diff --git a/docs/manage-relational-metadata-using-gravitino.md
b/docs/manage-relational-metadata-using-gravitino.md
index cfda9dc231..ebdace5c19 100644
--- a/docs/manage-relational-metadata-using-gravitino.md
+++ b/docs/manage-relational-metadata-using-gravitino.md
@@ -931,6 +931,8 @@ The following types that Gravitino supports:
| Union | `Types.UnionType.of([type1, type2, ...])`
| `{"type": "union", "types": [type JSON, ...]}`
| Union type, indicates a union of types
|
| UUID | `Types.UUIDType.get()`
| `uuid`
| UUID type, indicates a universally unique identifier
|
| Variant | `Types.VariantType.get()`
| `variant`
| Variant type, indicates semi-structured data whose shape is not fixed by the
schema
|
+| Geometry | `Types.GeometryType.of(crs)`
| `geometry` or `geometry(<crs>)`
| Geometry type, indicates a geospatial shape (WKB-encoded) on a planar
coordinate reference system (CRS), default `OGC:CRS84`
|
+| Geography | `Types.GeographyType.of(crs, algorithm)`
| `geography` or `geography(<crs>,<algorithm>)`
| Geography type, indicates a geospatial shape (WKB-encoded) on a spheroidal
CRS (default `OGC:CRS84`) with an edge-interpolation algorithm (default
`spherical`) |
The related java doc is
[here](pathname:///docs/2.0.0-SNAPSHOT/api/java/org/apache/gravitino/rel/types/Type.html).
diff --git a/docs/open-api/datatype.yaml b/docs/open-api/datatype.yaml
index a244aee188..0cc75b5e19 100644
--- a/docs/open-api/datatype.yaml
+++ b/docs/open-api/datatype.yaml
@@ -56,6 +56,10 @@ components:
- "fixed(16)"
- "binary"
- "variant"
+ - "geometry"
+ - "geometry(srid:3857)"
+ - "geography"
+ - "geography(EPSG:4326,karney)"
UnionType:
type: object
