This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 271de7deac [docs] Add document for nested column evolution (#4619)
271de7deac is described below
commit 271de7deac24b5ecc6306d345dcb5607d9f4a35a
Author: tsreaper <[email protected]>
AuthorDate: Mon Dec 2 15:59:39 2024 +0800
[docs] Add document for nested column evolution (#4619)
---
docs/content/flink/sql-alter.md | 16 ++++++++
docs/content/spark/sql-alter.md | 84 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 100 insertions(+)
diff --git a/docs/content/flink/sql-alter.md b/docs/content/flink/sql-alter.md
index bee616f047..877995cc63 100644
--- a/docs/content/flink/sql-alter.md
+++ b/docs/content/flink/sql-alter.md
@@ -78,6 +78,10 @@ If you use object storage, such as S3 or OSS, please use
this syntax carefully,
The following SQL adds two columns `c1` and `c2` to table `my_table`.
+{{< hint info >}}
+To add a column in a row type, see [Changing Column
Type](#changing-column-type).
+{{< /hint >}}
+
```sql
ALTER TABLE my_table ADD (c1 INT, c2 STRING);
```
@@ -99,6 +103,10 @@ otherwise this operation may fail, throws an exception like
`The following colum
ALTER TABLE my_table DROP (c1, c2);
```
+{{< hint info >}}
+To drop a column in a row type, see [Changing Column
Type](#changing-column-type).
+{{< /hint >}}
+
## Dropping Partitions
The following SQL drops the partitions of the paimon table.
@@ -185,6 +193,14 @@ The following SQL changes type of column `col_a` to
`DOUBLE`.
ALTER TABLE my_table MODIFY col_a DOUBLE;
```
+Paimon also supports changing columns of row type, array type, and map type.
+
+```sql
+-- col_a previously has type ARRAY<MAP<INT, ROW(f1 INT, f2 STRING)>>
+-- the following SQL changes f1 to BIGINT, drops f2, and adds f3
+ALTER TABLE my_table MODIFY col_a ARRAY<MAP<INT, ROW(f1 BIGINT, f3 DOUBLE)>>;
+```
+
## Adding watermark
The following SQL adds a computed column `ts` from existing column `log_ts`,
and a watermark with strategy `ts - INTERVAL '1' HOUR` on column `ts` which is
marked as event time attribute of table `my_table`.
diff --git a/docs/content/spark/sql-alter.md b/docs/content/spark/sql-alter.md
index 11af186e62..3ad7204802 100644
--- a/docs/content/spark/sql-alter.md
+++ b/docs/content/spark/sql-alter.md
@@ -95,6 +95,27 @@ ALTER TABLE my_table ADD COLUMNS (
);
```
+The following SQL adds a nested column `f3` to a struct type.
+
+```sql
+-- column v previously has type STRUCT<f1: STRING, f2: INT>
+ALTER TABLE my_table ADD COLUMN v.f3 STRING;
+```
+
+The following SQL adds a nested column `f3` to a struct type, which is the
element type of an array type.
+
+```sql
+-- column v previously has type ARRAY<STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table ADD COLUMN v.element.f3 STRING;
+```
+
+The following SQL adds a nested column `f3` to a struct type, which is the
value type of a map type.
+
+```sql
+-- column v previously has type MAP<INT, STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table ADD COLUMN v.value.f3 STRING;
+```
+
## Renaming Column Name
The following SQL renames column `c0` in table `my_table` to `c1`.
@@ -103,6 +124,27 @@ The following SQL renames column `c0` in table `my_table`
to `c1`.
ALTER TABLE my_table RENAME COLUMN c0 TO c1;
```
+The following SQL renames a nested column `f1` to `f100` in a struct type.
+
+```sql
+-- column v previously has type STRUCT<f1: STRING, f2: INT>
+ALTER TABLE my_table RENAME COLUMN v.f1 to f100;
+```
+
+The following SQL renames a nested column `f1` to `f100` in a struct type,
which is the element type of an array type.
+
+```sql
+-- column v previously has type ARRAY<STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table RENAME COLUMN v.element.f1 to f100;
+```
+
+The following SQL renames a nested column `f1` to `f100` in a struct type,
which is the value type of a map type.
+
+```sql
+-- column v previously has type MAP<INT, STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table RENAME COLUMN v.value.f1 to f100;
+```
+
## Dropping Columns
The following SQL drops two columns `c1` and `c2` from table `my_table`.
@@ -111,6 +153,27 @@ The following SQL drops two columns `c1` and `c2` from
table `my_table`.
ALTER TABLE my_table DROP COLUMNS (c1, c2);
```
+The following SQL drops a nested column `f2` from a struct type.
+
+```sql
+-- column v previously has type STRUCT<f1: STRING, f2: INT>
+ALTER TABLE my_table DROP COLUMN v.f2;
+```
+
+The following SQL drops a nested column `f2` from a struct type, which is the
element type of an array type.
+
+```sql
+-- column v previously has type ARRAY<STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table DROP COLUMN v.element.f2;
+```
+
+The following SQL drops a nested column `f2` from a struct type, which is the
value type of a map type.
+
+```sql
+-- column v previously has type MAP<INT, STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table DROP COLUMN v.value.f2;
+```
+
## Dropping Partitions
The following SQL drops the partitions of the paimon table. For spark sql, you
need to specify all the partition columns.
@@ -156,3 +219,24 @@ ALTER TABLE my_table ALTER COLUMN col_a AFTER col_b;
```sql
ALTER TABLE my_table ALTER COLUMN col_a TYPE DOUBLE;
```
+
+The following SQL changes the type of a nested column `f2` to `BIGINT` in a
struct type.
+
+```sql
+-- column v previously has type STRUCT<f1: STRING, f2: INT>
+ALTER TABLE my_table ALTER COLUMN v.f2 TYPE BIGINT;
+```
+
+The following SQL changes the type of a nested column `f2` to `BIGINT` in a
struct type, which is the element type of an array type.
+
+```sql
+-- column v previously has type ARRAY<STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table ALTER COLUMN v.element.f2 TYPE BIGINT;
+```
+
+The following SQL changes the type of a nested column `f2` to `BIGINT` in a
struct type, which is the value type of a map type.
+
+```sql
+-- column v previously has type MAP<INT, STRUCT<f1: STRING, f2: INT>>
+ALTER TABLE my_table ALTER COLUMN v.value.f2 TYPE BIGINT;
+```
