This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new 1249341afd [#12102] feat(jdbc-doris): Support frequently-used table
properties (#12103)
1249341afd is described below
commit 1249341afdd47b5748a144eafd0f16621145df71
Author: StormSpirit <[email protected]>
AuthorDate: Thu Jul 23 14:35:50 2026 +0800
[#12102] feat(jdbc-doris): Support frequently-used table properties (#12103)
### What changes were proposed in this pull request?
Register 5 frequently-used table properties in
`DorisTablePropertiesMetadata`:
- 3 writable: `compression`, `bloom_filter_columns`, `storage_policy`
- 2 reserved (read-only): `light_schema_change`,
`enable_unique_key_merge_on_write`
The existing read and write paths are already generic, so the
new properties pass through without additional logic changes. A
defense-in-depth filter in `appendNecessaryProperties` strips reserved
properties from the DDL before SQL generation.
### Why are the changes needed?
The jdbc-doris catalog currently only exposes `replication_num`. Users
cannot configure bloom filter columns, compression, or storage policy
through Gravitino, requiring manual SQL intervention after table
creation.
Fix: #12102
### Does this PR introduce _any_ user-facing change?
Yes — 3 new writable table properties and 2 new read-only properties
are exposed in `table.properties()`. Writable properties can be set
when creating or altering a table. Reserved properties are rejected
with a clear error if passed to `createTable`.
### How was this patch tested?
- `./gradlew :catalogs:catalog-jdbc-doris:test -PskipITs`
(unit tests: metadata validation + reserved property rejection)
- `./gradlew :catalogs:catalog-jdbc-doris:test -PskipDockerTests=false
-PdorisMultiVersionTest` (Docker IT: 1.2.x / 3.0.x / 4.0.x, all
passing)
- `./gradlew :catalogs:catalog-jdbc-doris:spotlessCheck`
- `./gradlew rat`
---------
Signed-off-by: jiangxt2 <[email protected]>
---
.../doris/DorisTablePropertiesMetadata.java | 44 ++++++-
.../gravitino/catalog/doris/TestDorisCatalog.java | 124 +++++++++++++++++++-
.../doris/integration/test/CatalogDoris3xIT.java | 128 ++++++++++++++++++++
.../doris/integration/test/CatalogDoris4xIT.java | 130 +++++++++++++++++++++
.../doris/integration/test/CatalogDorisIT.java | 40 +++++++
docs/jdbc-doris-catalog.md | 19 ++-
6 files changed, 474 insertions(+), 11 deletions(-)
diff --git
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/DorisTablePropertiesMetadata.java
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/DorisTablePropertiesMetadata.java
index 4cad4c0351..62beaa17e4 100644
---
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/DorisTablePropertiesMetadata.java
+++
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/DorisTablePropertiesMetadata.java
@@ -27,9 +27,15 @@ import org.apache.gravitino.connector.PropertyEntry;
public class DorisTablePropertiesMetadata extends JdbcTablePropertiesMetadata {
+ // ---- writable properties ----
public static final String REPLICATION_FACTOR = "replication_num";
public static final int DEFAULT_REPLICATION_FACTOR = 1;
public static final int DEFAULT_REPLICATION_FACTOR_IN_SERVER_SIDE = 3;
+ public static final String COMPRESSION = "compression";
+ public static final String BLOOM_FILTER_COLUMNS = "bloom_filter_columns";
+ public static final String STORAGE_POLICY = "storage_policy";
+ public static final String LIGHT_SCHEMA_CHANGE = "light_schema_change";
+ public static final String ENABLE_UNIQUE_KEY_MERGE_ON_WRITE =
"enable_unique_key_merge_on_write";
private static final Map<String, PropertyEntry<?>> PROPERTIES_METADATA;
@@ -38,10 +44,44 @@ public class DorisTablePropertiesMetadata extends
JdbcTablePropertiesMetadata {
ImmutableList.of(
PropertyEntry.integerOptionalPropertyEntry(
REPLICATION_FACTOR,
- "The number of replications for the table. If not specified
and the number of backend server less than 3,"
- + " the default value will be used",
+ "The number of replications for the table. If not specified
and the number"
+ + " of backend server less than 3, the default value will
be used",
false /* immutable */,
DEFAULT_REPLICATION_FACTOR, /* default value */
+ false /* hidden */),
+ PropertyEntry.stringOptionalPropertyEntry(
+ COMPRESSION,
+ "The compression type for the table (ZSTD, LZ4, LZ4F, ZLIB)."
+ + " Deprecated as a table-level property in Doris 4.0+."
+ + " Cannot be changed via ALTER TABLE.",
+ true /* immutable */,
+ null /* default value */,
+ false /* hidden */),
+ PropertyEntry.stringOptionalPropertyEntry(
+ BLOOM_FILTER_COLUMNS,
+ "Comma-separated list of columns for which bloom filter
indexes are created",
+ false /* immutable */,
+ null /* default value */,
+ false /* hidden */),
+ PropertyEntry.stringOptionalPropertyEntry(
+ STORAGE_POLICY,
+ "The name of the storage policy for cold-hot separation",
+ false /* immutable */,
+ null /* default value */,
+ false /* hidden */),
+ PropertyEntry.stringOptionalPropertyEntry(
+ LIGHT_SCHEMA_CHANGE,
+ "Whether light schema change is enabled for the table. "
+ + "Can be modified via ALTER TABLE SET",
+ false /* immutable */,
+ null /* default value */,
+ false /* hidden */),
+ PropertyEntry.stringOptionalPropertyEntry(
+ ENABLE_UNIQUE_KEY_MERGE_ON_WRITE,
+ "Whether merge-on-write is enabled for Unique Key tables. "
+ + "Must be set at CREATE TABLE time; Doris rejects ALTER
TABLE SET",
+ true /* immutable */,
+ null /* default value */,
false /* hidden */));
PROPERTIES_METADATA = Maps.uniqueIndex(propertyEntries,
PropertyEntry::getName);
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/TestDorisCatalog.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/TestDorisCatalog.java
index 039a15b0dc..c014d85080 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/TestDorisCatalog.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/TestDorisCatalog.java
@@ -19,9 +19,19 @@
package org.apache.gravitino.catalog.doris;
+import static java.util.Collections.emptyMap;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.BLOOM_FILTER_COLUMNS;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.COMPRESSION;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.ENABLE_UNIQUE_KEY_MERGE_ON_WRITE;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.LIGHT_SCHEMA_CHANGE;
import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.REPLICATION_FACTOR;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.STORAGE_POLICY;
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import java.util.HashMap;
import java.util.Map;
+import org.apache.gravitino.catalog.PropertiesMetadataHelpers;
import org.apache.gravitino.connector.PropertyEntry;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
@@ -33,13 +43,115 @@ public class TestDorisCatalog {
DorisTablePropertiesMetadata dorisTablePropertiesMetadata = new
DorisTablePropertiesMetadata();
Map<String, PropertyEntry<?>> propertyEntryMap =
dorisTablePropertiesMetadata.specificPropertyEntries();
- Assertions.assertTrue(propertyEntryMap.containsKey(REPLICATION_FACTOR));
- PropertyEntry<?> propertyEntry = propertyEntryMap.get(REPLICATION_FACTOR);
- Assertions.assertEquals(REPLICATION_FACTOR, propertyEntry.getName());
- Assertions.assertFalse(propertyEntry.isImmutable());
+ // Verify the total number of registered properties.
+ // 6 = 1 existing (replication_num) + 5 new (compression,
bloom_filter_columns,
+ // storage_policy, light_schema_change, enable_unique_key_merge_on_write).
+ Assertions.assertEquals(6, propertyEntryMap.size());
+
+ // ---- replication_num (integerOptional) ----
+ Assertions.assertTrue(propertyEntryMap.containsKey(REPLICATION_FACTOR));
+ PropertyEntry<?> replication = propertyEntryMap.get(REPLICATION_FACTOR);
+ Assertions.assertEquals(REPLICATION_FACTOR, replication.getName());
+ Assertions.assertFalse(replication.isRequired());
+ Assertions.assertFalse(replication.isImmutable());
+ Assertions.assertFalse(replication.isReserved());
+ Assertions.assertFalse(replication.isHidden());
+ Assertions.assertEquals(Integer.class, replication.getJavaType());
Assertions.assertEquals(
- DorisTablePropertiesMetadata.DEFAULT_REPLICATION_FACTOR,
propertyEntry.getDefaultValue());
- Assertions.assertFalse(propertyEntry.isHidden());
+ DorisTablePropertiesMetadata.DEFAULT_REPLICATION_FACTOR,
replication.getDefaultValue());
+
+ // ---- compression (stringOptional) ----
+ Assertions.assertTrue(propertyEntryMap.containsKey(COMPRESSION));
+ PropertyEntry<?> compression = propertyEntryMap.get(COMPRESSION);
+ Assertions.assertEquals(COMPRESSION, compression.getName());
+ Assertions.assertFalse(compression.isRequired());
+ Assertions.assertTrue(compression.isImmutable());
+ Assertions.assertFalse(compression.isReserved());
+ Assertions.assertFalse(compression.isHidden());
+ Assertions.assertEquals(String.class, compression.getJavaType());
+ Assertions.assertNull(compression.getDefaultValue());
+
+ // ---- bloom_filter_columns (stringOptional) ----
+ Assertions.assertTrue(propertyEntryMap.containsKey(BLOOM_FILTER_COLUMNS));
+ PropertyEntry<?> bloomFilter = propertyEntryMap.get(BLOOM_FILTER_COLUMNS);
+ Assertions.assertEquals(BLOOM_FILTER_COLUMNS, bloomFilter.getName());
+ Assertions.assertFalse(bloomFilter.isRequired());
+ Assertions.assertFalse(bloomFilter.isImmutable());
+ Assertions.assertFalse(bloomFilter.isReserved());
+ Assertions.assertFalse(bloomFilter.isHidden());
+ Assertions.assertEquals(String.class, bloomFilter.getJavaType());
+ Assertions.assertNull(bloomFilter.getDefaultValue());
+
+ // ---- storage_policy (stringOptional) ----
+ Assertions.assertTrue(propertyEntryMap.containsKey(STORAGE_POLICY));
+ PropertyEntry<?> storagePolicy = propertyEntryMap.get(STORAGE_POLICY);
+ Assertions.assertEquals(STORAGE_POLICY, storagePolicy.getName());
+ Assertions.assertFalse(storagePolicy.isRequired());
+ Assertions.assertFalse(storagePolicy.isImmutable());
+ Assertions.assertFalse(storagePolicy.isReserved());
+ Assertions.assertFalse(storagePolicy.isHidden());
+ Assertions.assertEquals(String.class, storagePolicy.getJavaType());
+ Assertions.assertNull(storagePolicy.getDefaultValue());
+
+ // ---- light_schema_change (stringOptional, mutable) ----
+ Assertions.assertTrue(propertyEntryMap.containsKey(LIGHT_SCHEMA_CHANGE));
+ PropertyEntry<?> lightSchemaChange =
propertyEntryMap.get(LIGHT_SCHEMA_CHANGE);
+ Assertions.assertEquals(LIGHT_SCHEMA_CHANGE, lightSchemaChange.getName());
+ Assertions.assertFalse(lightSchemaChange.isRequired());
+ Assertions.assertFalse(lightSchemaChange.isImmutable());
+ Assertions.assertFalse(lightSchemaChange.isReserved());
+ Assertions.assertFalse(lightSchemaChange.isHidden());
+ Assertions.assertEquals(String.class, lightSchemaChange.getJavaType());
+ Assertions.assertNull(lightSchemaChange.getDefaultValue());
+
+ // ---- enable_unique_key_merge_on_write (stringOptional, immutable) ----
+
Assertions.assertTrue(propertyEntryMap.containsKey(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE));
+ PropertyEntry<?> mergeOnWrite =
propertyEntryMap.get(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE);
+ Assertions.assertEquals(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE,
mergeOnWrite.getName());
+ Assertions.assertFalse(mergeOnWrite.isRequired());
+ Assertions.assertTrue(mergeOnWrite.isImmutable());
+ Assertions.assertFalse(mergeOnWrite.isReserved());
+ Assertions.assertFalse(mergeOnWrite.isHidden());
+ Assertions.assertEquals(String.class, mergeOnWrite.getJavaType());
+ Assertions.assertNull(mergeOnWrite.getDefaultValue());
+ }
+
+ @Test
+ void testWritablePropertiesAcceptedOnCreate() {
+ // Verify that all registered properties (including the formerly-reserved
light_schema_change
+ // and enable_unique_key_merge_on_write) pass create-table validation.
+ DorisTablePropertiesMetadata metadata = new DorisTablePropertiesMetadata();
+
+ // All writable properties should pass validation — no exception.
+ Map<String, String> props = new HashMap<>();
+ props.put(BLOOM_FILTER_COLUMNS, "col1,col2");
+ props.put(COMPRESSION, "ZSTD");
+ props.put(LIGHT_SCHEMA_CHANGE, "true");
+ props.put(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE, "false");
+ assertDoesNotThrow(() ->
PropertiesMetadataHelpers.validatePropertyForCreate(metadata, props));
+
+ // Verify that immutable properties are rejected by ALTER validation.
+ // compression is immutable (Doris rejects ALTER TABLE SET compression).
+ Map<String, String> alterProps = new HashMap<>();
+ alterProps.put(COMPRESSION, "ZLIB");
+ assertThrows(
+ IllegalArgumentException.class,
+ () -> PropertiesMetadataHelpers.validatePropertyForAlter(metadata,
alterProps, emptyMap()));
+
+ // enable_unique_key_merge_on_write is immutable (Doris rejects ALTER
TABLE SET).
+ Map<String, String> alterProps2 = new HashMap<>();
+ alterProps2.put(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE, "true");
+ assertThrows(
+ IllegalArgumentException.class,
+ () ->
+ PropertiesMetadataHelpers.validatePropertyForAlter(metadata,
alterProps2, emptyMap()));
+
+ // light_schema_change is mutable — should pass ALTER validation.
+ Map<String, String> alterProps3 = new HashMap<>();
+ alterProps3.put(LIGHT_SCHEMA_CHANGE, "false");
+ assertDoesNotThrow(
+ () ->
+ PropertiesMetadataHelpers.validatePropertyForAlter(metadata,
alterProps3, emptyMap()));
}
}
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
index 712b5201d8..ce51c4005b 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
@@ -18,14 +18,21 @@
*/
package org.apache.gravitino.catalog.doris.integration.test;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.BLOOM_FILTER_COLUMNS;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.COMPRESSION;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.ENABLE_UNIQUE_KEY_MERGE_ON_WRITE;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.LIGHT_SCHEMA_CHANGE;
import static
org.apache.gravitino.integration.test.util.ITUtils.assertPartition;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.common.collect.Maps;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
+import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
@@ -427,6 +434,127 @@ public class CatalogDoris3xIT extends BaseIT {
partitions.get("p3"));
}
+ @Test
+ void testTablePropertiesRoundTrip() {
+ // Verify writable properties survive the create → Doris 3.0 → load
round-trip.
+ // Covers: bloom_filter_columns, compression, light_schema_change (all
writable).
+ // NOTE: storage_policy requires a storage policy resource in Doris
(cold-hot separation
+ // infrastructure) and cannot be tested end-to-end in a Docker
environment. Its metadata
+ // is verified in TestDorisCatalog.testDorisTablePropertiesMetadata().
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_props_roundtrip"));
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(BLOOM_FILTER_COLUMNS, colName1 + "," + colName2);
+ properties.put(COMPRESSION, "ZSTD");
+ properties.put(LIGHT_SCHEMA_CHANGE, "true");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ Table loaded = tc.loadTable(tid);
+ Map<String, String> loadedProps = loaded.properties();
+
+ // bloom_filter_columns: verify round-trip with Doris normalization
+ assertTrue(loadedProps.containsKey(BLOOM_FILTER_COLUMNS));
+ assertEquals(colName1 + ", " + colName2,
loadedProps.get(BLOOM_FILTER_COLUMNS));
+
+ // compression: verify it survives round-trip on 3.0.x
+ assertTrue(
+ loadedProps.containsKey(COMPRESSION),
+ "compression should appear in SHOW CREATE TABLE properties on Doris
3.0.x");
+
+ // light_schema_change: writable property, should survive round-trip on
3.0.x
+ assertTrue(
+ loadedProps.containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change should appear in SHOW CREATE TABLE properties on
Doris 3.0.x");
+ }
+
+ @Test
+ void testUniqueKeyTableProperties() {
+ // Verify that enable_unique_key_merge_on_write (writable, immutable) and
+ // light_schema_change (writable, mutable) can be set and read back
+ // for UNIQUE KEY tables on Doris 3.0.x.
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_uk_props"));
+ Index[] indexes =
+ new Index[] {Indexes.of(Index.IndexType.UNIQUE_KEY, "uk_pk", new
String[][] {{colName1}})};
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE, "true");
+ properties.put(LIGHT_SCHEMA_CHANGE, "true");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ indexes);
+
+ Table loaded = tc.loadTable(tid);
+ Map<String, String> loadedProps = loaded.properties();
+
+ assertTrue(
+ loadedProps.containsKey(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE),
+ "enable_unique_key_merge_on_write should appear for UNIQUE KEY tables
on Doris 3.0.x");
+ assertTrue(
+ loadedProps.containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change should appear in SHOW CREATE TABLE properties on
Doris 3.0.x");
+ }
+
+ @Test
+ void testAlterTableProperty() {
+ // Verify that light_schema_change (mutable) can be altered via ALTER
TABLE SET
+ // on a real Doris 3.0.x instance. Only false→true is tested because Doris
3.0
+ // rejects true→false ("Can not alter light_schema_change from true to
false").
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_alter_props"));
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(LIGHT_SCHEMA_CHANGE, "false");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ // When light_schema_change is "false" (the effective default), Doris does
not
+ // emit it in SHOW CREATE TABLE PROPERTIES.
+ Table afterCreate = tc.loadTable(tid);
+ assertFalse(
+ afterCreate.properties().containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change=false should NOT appear in PROPERTIES (default)");
+
+ tc.alterTable(tid, TableChange.setProperty(LIGHT_SCHEMA_CHANGE, "true"));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () ->
+ assertTrue(
+
tc.loadTable(tid).properties().containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change=true should appear after ALTER TABLE
SET"));
+ }
+
private Column findColumn(Table table, String columnName) {
return Arrays.stream(table.columns())
.filter(c -> c.name().equals(columnName))
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
index b6d1f67f7c..39af05faf7 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
@@ -18,14 +18,21 @@
*/
package org.apache.gravitino.catalog.doris.integration.test;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.BLOOM_FILTER_COLUMNS;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.COMPRESSION;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.ENABLE_UNIQUE_KEY_MERGE_ON_WRITE;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.LIGHT_SCHEMA_CHANGE;
import static
org.apache.gravitino.integration.test.util.ITUtils.assertPartition;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
import com.google.common.collect.Maps;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
+import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
@@ -426,6 +433,129 @@ public class CatalogDoris4xIT extends BaseIT {
partitions.get("p3"));
}
+ @Test
+ void testTablePropertiesRoundTrip() {
+ // Verify writable properties survive the create → Doris 4.0 → load
round-trip.
+ // Covers: bloom_filter_columns (writable), compression (writable,
deprecated in 4.0),
+ // light_schema_change (writable, auto-populated by Doris),
+ // storage_policy (writable, requires infrastructure; metadata
verified in unit test).
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_props_roundtrip"));
+
+ // NOTE: compression is deprecated as a table-level property in Doris 4.0.6
+ // (it is silently ignored in SHOW CREATE TABLE). We include it to verify
+ // that table creation does not fail — the property key is still accepted.
+ Map<String, String> properties = new HashMap<>();
+ properties.put(BLOOM_FILTER_COLUMNS, colName1 + "," + colName2);
+ properties.put(COMPRESSION, "ZSTD");
+ properties.put(LIGHT_SCHEMA_CHANGE, "true");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ Table loaded = tc.loadTable(tid);
+ Map<String, String> loadedProps = loaded.properties();
+
+ // bloom_filter_columns: Doris normalizes format (adding spaces after
commas)
+ assertEquals(colName1 + ", " + colName2,
loadedProps.get(BLOOM_FILTER_COLUMNS));
+
+ // compression: silently ignored at table level in 4.0.6 (deprecated).
+ // Verify it is NOT present in SHOW CREATE TABLE output.
+ assertFalse(
+ loadedProps.containsKey(COMPRESSION),
+ "compression should NOT appear in SHOW CREATE TABLE on Doris 4.0.x
(deprecated)");
+
+ // light_schema_change: writable property, should survive round-trip on
4.0.x
+ assertTrue(
+ loadedProps.containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change should appear in SHOW CREATE TABLE properties on
Doris 4.0.x");
+ }
+
+ @Test
+ void testUniqueKeyTableProperties() {
+ // Verify that enable_unique_key_merge_on_write (writable, immutable) and
+ // light_schema_change (writable, mutable) can be set and read back
+ // for UNIQUE KEY tables on Doris 4.0.x.
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_uk_props"));
+ Index[] indexes =
+ new Index[] {Indexes.of(Index.IndexType.UNIQUE_KEY, "uk_pk", new
String[][] {{colName1}})};
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE, "true");
+ properties.put(LIGHT_SCHEMA_CHANGE, "true");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ indexes);
+
+ Table loaded = tc.loadTable(tid);
+ Map<String, String> loadedProps = loaded.properties();
+
+ assertTrue(
+ loadedProps.containsKey(ENABLE_UNIQUE_KEY_MERGE_ON_WRITE),
+ "enable_unique_key_merge_on_write should appear for UNIQUE KEY tables
on Doris 4.0.x");
+ assertTrue(
+ loadedProps.containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change should appear in SHOW CREATE TABLE properties on
Doris 4.0.x");
+ }
+
+ @Test
+ void testAlterTableProperty() {
+ // Verify that light_schema_change (mutable) can be altered via ALTER
TABLE SET
+ // on a real Doris 4.0.x instance. Only false→true is tested because Doris
+ // rejects true→false ("Can not alter light_schema_change from true to
false").
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(schemaName,
GravitinoITUtils.genRandomName("t_alter_props"));
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(LIGHT_SCHEMA_CHANGE, "false");
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ properties,
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ // When light_schema_change is "false" (the effective default), Doris does
not
+ // emit it in SHOW CREATE TABLE PROPERTIES.
+ Table afterCreate = tc.loadTable(tid);
+ assertFalse(
+ afterCreate.properties().containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change=false should NOT appear in PROPERTIES (default)");
+
+ tc.alterTable(tid, TableChange.setProperty(LIGHT_SCHEMA_CHANGE, "true"));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () ->
+ assertTrue(
+
tc.loadTable(tid).properties().containsKey(LIGHT_SCHEMA_CHANGE),
+ "light_schema_change=true should appear after ALTER TABLE
SET"));
+ }
+
private Column findColumn(Table table, String columnName) {
return Arrays.stream(table.columns())
.filter(c -> c.name().equals(columnName))
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
index e474c10fa7..792f52dda1 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
@@ -18,6 +18,8 @@
*/
package org.apache.gravitino.catalog.doris.integration.test;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.BLOOM_FILTER_COLUMNS;
+import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.COMPRESSION;
import static
org.apache.gravitino.integration.test.util.ITUtils.assertPartition;
import static
org.apache.gravitino.rel.Column.DEFAULT_VALUE_OF_CURRENT_TIMESTAMP;
import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -202,6 +204,44 @@ public class CatalogDorisIT extends BaseIT {
assertEquals(createdSchema.properties().get(propKey), propValue);
}
+ @Test
+ void testTablePropertiesRoundTrip() {
+ // Verify writable table properties survive the create → Doris 1.2 → load
round-trip.
+ // Covers: bloom_filter_columns, compression (both writable).
+ // NOTE: light_schema_change and enable_unique_key_merge_on_write are
newer Doris
+ // features (introduced in 2.1+) and do not appear in 1.2.x SHOW CREATE
TABLE output.
+ // They are registered as writable properties and their metadata is
verified in
+ // TestDorisCatalog.testDorisTablePropertiesMetadata().
+ // storage_policy requires cold-hot separation infrastructure and is
covered by
+ // the unit test metadata verification.
+ TableCatalog tc = catalog.asTableCatalog();
+ String tableName = GravitinoITUtils.genRandomName("t_props_roundtrip");
+ NameIdentifier tid = NameIdentifier.of(schemaName, tableName);
+ Column[] columns = createColumns();
+ Distribution distribution = createDistribution();
+
+ Map<String, String> properties = new HashMap<>();
+ properties.put(BLOOM_FILTER_COLUMNS, DORIS_COL_NAME1 + "," +
DORIS_COL_NAME2);
+ properties.put(COMPRESSION, "ZSTD");
+
+ tc.createTable(
+ tid, columns, table_comment, properties, Transforms.EMPTY_TRANSFORM,
distribution, null);
+
+ Table loaded = tc.loadTable(tid);
+ Map<String, String> loadedProps = loaded.properties();
+
+ // bloom_filter_columns: verify round-trip with Doris normalization
+ assertTrue(
+ loadedProps.containsKey(BLOOM_FILTER_COLUMNS),
+ "bloom_filter_columns should appear in SHOW CREATE TABLE properties");
+ assertEquals(DORIS_COL_NAME1 + ", " + DORIS_COL_NAME2,
loadedProps.get(BLOOM_FILTER_COLUMNS));
+
+ // compression: verify round-trip on Doris 1.2.x
+ assertTrue(
+ loadedProps.containsKey(COMPRESSION),
+ "compression should appear in SHOW CREATE TABLE properties on Doris
1.2.x");
+ }
+
private Column[] createColumns() {
Column col1 =
Column.of(DORIS_COL_NAME1, Types.IntegerType.get(), "col_1_comment",
false, false, null);
diff --git a/docs/jdbc-doris-catalog.md b/docs/jdbc-doris-catalog.md
index 474ceee62f..91ddc01de8 100644
--- a/docs/jdbc-doris-catalog.md
+++ b/docs/jdbc-doris-catalog.md
@@ -49,7 +49,6 @@ Besides the [common catalog
properties](./gravitino-server-config.md#catalog-pro
| `jdbc-password` | The JDBC password.
| (none) | Yes | 0.5.0 |
| `jdbc.pool.min-size` | The minimum number of connections in the pool. `2`
by default.
| `2` | No | 0.5.0 |
| `jdbc.pool.max-size` | The maximum number of connections in the pool.
`10` by default.
| `10` | No | 0.5.0
|
-| `replication_num` | The number of replications for the table. If not
specified and the number of backend servers less than 3, then the default value
is 1; If not specified and the number of backend servers greater or equals to
3, the default value (3) in Doris server will be used. For more, see the
[doc](https://doris.apache.org/docs/1.2/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-TABLE/)
| `1` or `3` | No | 0.6.0-incubating |
| `jdbc.pool.max-wait-ms` | The maximum Duration that the pool will wait for a
connection to be returned. `30000` by default.
| `30000` | No | 1.1.0 |
Before using the Doris Catalog, you must download the corresponding JDBC
driver to the `catalogs/jdbc-doris/libs` directory.
@@ -207,8 +206,22 @@ Index[] indexes = new Index[] {
### Table Properties
-- Doris supports table properties, and you can set them in the table
properties.
-- Only supports Doris table properties and doesn't support user-defined
properties.
+Doris table properties can be set when creating a table.
+Only Doris built-in table properties are supported; user-defined properties
are not supported.
+
+| Property Name | Description
| Default
Value | Required | Reserved | Immutable | Since Version |
+|--------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|----------|----------|-----------|------------------|
+| `replication_num` | The number of replications for the
table. If not specified and the number of backend servers less than 3, then the
default value is 1; If BE ≥ 3, the server-side default (3) will be used. | `1`
or `3` | No | No | No | 0.6.0-incubating |
+| `compression` | The compression type for the table.
Supported values: `ZSTD`, `LZ4`, `LZ4F`, `ZLIB`. Deprecated as a table-level
property in Doris 4.0+. Cannot be changed after table creation. |
(none) | No | No | Yes | 2.0.0 |
+| `bloom_filter_columns` | Comma-separated list of columns for
which bloom filter indexes are created.
|
(none) | No | No | No | 2.0.0 |
+| `storage_policy` | The name of the storage policy for
cold-hot separation.
|
(none) | No | No | No | 2.0.0 |
+| `light_schema_change` | Whether light schema change is
enabled for the table. Can be modified via ALTER TABLE SET.
|
`true` | No | No | No | 2.0.0 |
+| `enable_unique_key_merge_on_write` | Whether merge-on-write is enabled for
Unique Key tables. Must be set at CREATE TABLE time; cannot be changed after
creation. |
`true` | No | No | Yes | 2.0.0 |
+
+:::note
+**Immutable** properties can be set at CREATE TABLE time but cannot be changed
via ALTER TABLE.
+**Reserved** properties (none currently) are read-only and cannot be set by
users.
+:::
### Table Indexes
