This is an automated email from the ASF dual-hosted git repository.
ipolyzos pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fluss.git
The following commit(s) were added to refs/heads/main by this push:
new db2af5cc [Docs] consistency & syntax fixes (#1243)
db2af5cc is described below
commit db2af5ccd44e051fc1d0b7ba001060eedaec880a
Author: Giannis Polyzos <[email protected]>
AuthorDate: Thu Jul 17 10:04:39 2025 +0300
[Docs] consistency & syntax fixes (#1243)
* change PrimaryKey table to Primary Key Table across pages
* syntactic fixes
* make some more minor fixes
* fix broken link
* address yuxia's comments
---
website/docs/engine-flink/ddl.md | 36 ++++++++++------------
website/docs/intro.md | 4 +--
website/docs/table-design/overview.md | 16 +++++-----
website/docs/table-design/table-types/log-table.md | 6 ++--
.../table-types/pk-table/_category_.json | 2 +-
.../table-design/table-types/pk-table/index.md | 34 ++++++++++----------
.../pk-table/merge-engines/first-row.md | 4 +--
.../table-types/pk-table/merge-engines/index.md | 6 ++--
8 files changed, 52 insertions(+), 56 deletions(-)
diff --git a/website/docs/engine-flink/ddl.md b/website/docs/engine-flink/ddl.md
index 6b75e029..0e248925 100644
--- a/website/docs/engine-flink/ddl.md
+++ b/website/docs/engine-flink/ddl.md
@@ -41,17 +41,17 @@ The following properties can be set if using the Fluss
catalog:
| Option | Required | Default | Description
|
|--------------------------------|----------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| type | required | (none) | Catalog type, must
to be 'fluss' here.
|
+| type | required | (none) | Catalog type, must
be 'fluss' here.
|
| bootstrap.servers | required | (none) | Comma separated list
of Fluss servers.
|
| default-database | optional | fluss | The default database
to use when switching to this catalog.
|
| client.security.protocol | optional | PLAINTEXT | The security
protocol used to communicate with brokers. Currently, only `PLAINTEXT` and
`SASL` are supported, the configuration value is case insensitive.
|
-| `client.security.{protocol}.*` | optional | (none) | Client-side
configuration properties for a specific authentication protocol. E.g.,
client.security.sasl.jaas.config. More Details in
[authentication](../security/authentication.md) | (none) |
+| `client.security.{protocol}.*` | optional | (none) | Client-side
configuration properties for a specific authentication protocol. E.g.,
client.security.sasl.jaas.config. More Details in
[authentication](../security/authentication.md) |
-The following introduced statements assuming the current catalog is switched
to the Fluss catalog using `USE CATALOG <catalog_name>` statement.
+The following statements assume that the current catalog has been switched to
the Fluss catalog using the `USE CATALOG <catalog_name>` statement.
## Create Database
-By default, FlussCatalog will use the `fluss` database in Flink. Using the
following example to create a separate database in order to avoid creating
tables under the default `fluss` database:
+By default, FlussCatalog will use the `fluss` database in Flink. You can use
the following example to create a separate database to avoid creating tables
under the default `fluss` database:
```sql title="Flink SQL"
CREATE DATABASE my_db;
@@ -77,9 +77,9 @@ DROP DATABASE my_db;
## Create Table
-### PrimaryKey Table
+### Primary Key Table
-The following SQL statement will create a [PrimaryKey
Table](table-design/table-types/pk-table/index.md) with a primary key
consisting of shop_id and user_id.
+The following SQL statement will create a [Primary Key
Table](table-design/table-types/pk-table/index.md) with a primary key
consisting of shop_id and user_id.
```sql title="Flink SQL"
CREATE TABLE my_pk_table (
shop_id BIGINT,
@@ -107,14 +107,14 @@ CREATE TABLE my_log_table (
);
```
-### Partitioned (PrimaryKey/Log) Table
+### Partitioned (Primary Key/Log) Table
:::note
1. Currently, Fluss only supports partitioned field with `STRING` type
-2. For the Partitioned PrimaryKey Table, the partitioned field (`dt` in this
case) must be a subset of the primary key (`dt, shop_id, user_id` in this case)
+2. For the Partitioned Primary Key Table, the partitioned field (`dt` in this
case) must be a subset of the primary key (`dt, shop_id, user_id` in this case)
:::
-The following SQL statement creates a Partitioned PrimaryKey Table in Fluss.
+The following SQL statement creates a Partitioned Primary Key Table in Fluss.
```sql title="Flink SQL"
CREATE TABLE my_part_pk_table (
@@ -147,7 +147,7 @@ But you can still use the [Add
Partition](engine-flink/ddl.md#add-partition) sta
#### Multi-Fields Partitioned Table
-Fluss also support [Multi-Fields
Partitioning](table-design/data-distribution/partitioning.md#multi-field-partitioned-tables),
the following SQL statement creates a Multi-Fields Partitioned Log Table in
Fluss:
+Fluss also supports [Multi-Fields
Partitioning](table-design/data-distribution/partitioning.md#multi-field-partitioned-tables),
the following SQL statement creates a Multi-Fields Partitioned Log Table in
Fluss:
```sql title="Flink SQL"
CREATE TABLE my_multi_fields_part_log_table (
@@ -160,9 +160,9 @@ CREATE TABLE my_multi_fields_part_log_table (
) PARTITIONED BY (dt, nation);
```
-#### Auto partitioned (PrimaryKey/Log) table
+#### Auto Partitioned (Primary Key/Log) Table
-Fluss also support creat Auto Partitioned (PrimaryKey/Log) Table. The
following SQL statement creates an Auto Partitioned PrimaryKey Table in Fluss.
+Fluss also supports creating Auto Partitioned (Primary Key/Log) Table. The
following SQL statement creates an Auto Partitioned Primary Key Table in Fluss.
```sql title="Flink SQL"
CREATE TABLE my_auto_part_pk_table (
@@ -195,7 +195,7 @@ CREATE TABLE my_auto_part_log_table (
);
```
-For more details about Auto Partitioned (PrimaryKey/Log) Table, refer to [Auto
Partitioning](table-design/data-distribution/partitioning.md#auto-partitioning).
+For more details about Auto Partitioned (Primary Key/Log) Table, refer to
[Auto
Partitioning](table-design/data-distribution/partitioning.md#auto-partitioning).
### Options
@@ -240,8 +240,8 @@ This will entirely remove all the data of the table in the
Fluss cluster.
## Add Partition
-Fluss support manually add partitions to an exists partitioned table by Fluss
Catalog. If the specified partition
-not exists, Fluss will create the partition. If the specified partition
already exists, Fluss will ignore the request
+Fluss supports manually adding partitions to an existing partitioned table
through the Fluss Catalog. If the specified partition
+does not exist, Fluss will create the partition. If the specified partition
already exists, Fluss will ignore the request
or throw an exception.
To add partitions, run:
@@ -277,8 +277,8 @@ For more details, refer to the [Flink SHOW
PARTITIONS](https://nightlies.apache.
## Drop Partition
-Fluss also support manually drop partitions from an exists partitioned table
by Fluss Catalog. If the specified partition
-not exists, Fluss will ignore the request or throw an exception.
+Fluss also supports manually dropping partitions from an existing partitioned
table through the Fluss Catalog. If the specified partition
+does not exist, Fluss will ignore the request or throw an exception.
To drop partitions, run:
@@ -291,5 +291,3 @@ ALTER TABLE my_multi_fields_part_log_table DROP PARTITION
(dt = '2025-03-05', na
```
For more details, refer to the [Flink ALTER
TABLE(DROP)](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/alter/#drop)
documentation.
-
-
diff --git a/website/docs/intro.md b/website/docs/intro.md
index a7402ca6..5be20329 100644
--- a/website/docs/intro.md
+++ b/website/docs/intro.md
@@ -28,7 +28,7 @@ Fluss is a streaming storage built for real-time analytics
which can serve as th
-It bridges the gap between **streaming data** and the data **Lakehouse** by
enabling low-latency, high-throughput data ingestion and processing while
seamlessly integrating with popular compute engines like **Apache Flink**,
while **Apache Spark**, and **StarRocks** are coming soon.
+It bridges the gap between **streaming data** and the data **Lakehouse** by
enabling low-latency, high-throughput data ingestion and processing while
seamlessly integrating with popular compute engines like **Apache Flink**, with
**Apache Spark** and **StarRocks** coming soon.
Fluss supports `streaming reads` and `writes` with sub-second latency and
stores data in a columnar format, enhancing query performance and reducing
storage costs.
It offers flexible table types, including append-only **Log Tables** and
updatable **PrimaryKey Tables**, to accommodate diverse real-time analytics and
processing needs.
@@ -46,7 +46,7 @@ The following is a list of (but not limited to) use-cases
that Fluss shines ✨:
* **📡 Real-time IoT Pipelines**
* **🚓 Real-time Fraud Detection**
* **🚨 Real-time Alerting Systems**
-* **💫 Real-tim ETL/Data Warehouses**
+* **💫 Real-time ETL/Data Warehouses**
* **🌐 Real-time Geolocation Services**
* **🚚 Real-time Shipment Update Tracking**
diff --git a/website/docs/table-design/overview.md
b/website/docs/table-design/overview.md
index 6711bb44..c2fe9875 100644
--- a/website/docs/table-design/overview.md
+++ b/website/docs/table-design/overview.md
@@ -34,13 +34,13 @@ Tables are classified into two types based on the presence
of a primary key:
- **Log Tables:**
- Designed for append-only scenarios.
- Support only INSERT operations.
-- **PrimaryKey Tables:**
+- **Primary Key Tables:**
- Used for updating and managing data in business databases.
- Support INSERT, UPDATE, and DELETE operations based on the defined primary
key.
-A Table becomes a [Partitioned
Table](table-design/data-distribution/partitioning.md) when a partition column
is defined. Data with the same partition value is stored in the same partition.
Partition columns can be applied to both Log Tables and PrimaryKey Tables, but
with specific considerations:
+A Table becomes a [Partitioned Table](data-distribution/partitioning.md) when
a partition column is defined. Data with the same partition value is stored in
the same partition. Partition columns can be applied to both Log Tables and
Primary Key Tables, but with specific considerations:
- **For Log Tables**, partitioning is commonly used for log data, typically
based on date columns, to facilitate data separation and cleaning.
-- **For PrimaryKey Tables**, the partition column must be a subset of the
primary key to ensure uniqueness.
+- **For Primary Key Tables**, the partition column must be a subset of the
primary key to ensure uniqueness.
This design ensures efficient data organization, flexibility in handling
different use cases, and adherence to data integrity constraints.
@@ -60,14 +60,12 @@ The number of buckets `N` can be configured per table. A
bucket is the smallest
The data of a bucket consists of a LogTablet and a (optional) KvTablet.
### LogTablet
-A **LogTablet** needs to be generated for each bucket of Log and PrimaryKey
tables.
-For Log Tables, the LogTablet is both the primary table data and the log data.
For PrimaryKey tables, the LogTablet acts
+A **LogTablet** needs to be generated for each bucket of Log and Primary Key
Tables.
+For Log Tables, the LogTablet is both the primary table data and the log data.
For Primary Key Tables, the LogTablet acts
as the log data for the primary table data.
- **Segment:** The smallest unit of log storage in the **LogTablet**. A
segment consists of an **.index** file and a **.log** data file.
-- **.index:** An `offset sparse index` that stores the mappings between the
physical byte address in the message relative offset -> .log file.
+- **.index:** An `offset sparse index` that maps message relative offsets to
their corresponding physical byte addresses in the .log file.
- **.log:** Compact arrangement of log data.
### KvTablet
-Each bucket of the PrimaryKey table needs to generate a KvTablet. Underlying,
each KvTablet corresponds to an embedded RocksDB instance. RocksDB is an LSM
(log structured merge) engine which helps KvTablet supports high-performance
updates and lookup query.
-
-
+Each bucket of the Primary Key Table needs to generate a KvTablet. Underlying,
each KvTablet corresponds to an embedded RocksDB instance. RocksDB is an LSM
(log structured merge) engine which helps KvTablet support high-performance
updates and lookup queries.
diff --git a/website/docs/table-design/table-types/log-table.md
b/website/docs/table-design/table-types/log-table.md
index 4aea0881..db8dbaeb 100644
--- a/website/docs/table-design/table-types/log-table.md
+++ b/website/docs/table-design/table-types/log-table.md
@@ -62,7 +62,7 @@ Log Tables in Fluss allow real-time data consumption,
preserving the order of da
## Column Pruning
Column pruning is a technique used to reduce the amount of data that needs to
be read from storage by eliminating unnecessary columns from the query.
-Fluss supports column pruning for Log Tables and the changelog of PrimaryKey
Tables, which can significantly improve query performance by reducing the
amount of data that needs to be read from storage and lowering networking costs.
+Fluss supports column pruning for Log Tables and the changelog of Primary Key
Tables, which can significantly improve query performance by reducing the
amount of data that needs to be read from storage and lowering networking costs.
What sets Fluss apart is its ability to apply **column pruning during
streaming reads**, a capability that is both unique and industry-leading. This
ensures that even in real-time streaming scenarios, only the required columns
are processed, minimizing resource usage and maximizing efficiency.
@@ -90,7 +90,7 @@ Additionally, compression is applied to each column
independently, preserving th
When compression is enabled:
- For **Log Tables**, data is compressed by the writer on the client side,
written in a compressed format, and decompressed by the log scanner on the
client side.
-- For **PrimaryKey Table changelogs**, compression is performed server-side
since the changelog is generated on the server.
+- For **Primary Key Table changelogs**, compression is performed server-side
since the changelog is generated on the server.
Log compression significantly reduces networking and storage costs. Benchmark
results demonstrate that using the ZSTD compression with level 3 achieves a
compression ratio of approximately **5x** (e.g., reducing 5GB of data to 1GB).
Furthermore, read/write throughput improves substantially due to reduced
networking overhead.
@@ -133,4 +133,4 @@ In the above example, we set the compression codec to
`LZ4_FRAME` and the compre
:::
## Log Tiering
-Log Table supports tiering data to different storage tiers. See more details
about [Remote Log](maintenance/tiered-storage/remote-storage.md).
\ No newline at end of file
+Log Table supports tiering data to different storage tiers. See more details
about [Remote Log](maintenance/tiered-storage/remote-storage.md).
diff --git a/website/docs/table-design/table-types/pk-table/_category_.json
b/website/docs/table-design/table-types/pk-table/_category_.json
index 2aade25d..7374558c 100644
--- a/website/docs/table-design/table-types/pk-table/_category_.json
+++ b/website/docs/table-design/table-types/pk-table/_category_.json
@@ -1,4 +1,4 @@
{
- "label": "PrimaryKey Table",
+ "label": "Primary Key Table",
"position": 1
}
diff --git a/website/docs/table-design/table-types/pk-table/index.md
b/website/docs/table-design/table-types/pk-table/index.md
index d3b29e8f..5efa5dd4 100644
--- a/website/docs/table-design/table-types/pk-table/index.md
+++ b/website/docs/table-design/table-types/pk-table/index.md
@@ -1,5 +1,5 @@
---
-title: PrimaryKey Table
+title: Primary Key Table
sidebar_position: 1
---
@@ -21,15 +21,15 @@ sidebar_position: 1
limitations under the License.
-->
-# PrimaryKey Table
+# Primary Key Table
## Basic Concept
-PrimaryKey Table in Fluss ensure the uniqueness of the specified primary key
and supports `INSERT`, `UPDATE`,
+Primary Key Table in Fluss ensures the uniqueness of the specified primary key
and supports `INSERT`, `UPDATE`,
and `DELETE` operations.
-A PrimaryKey Table is created by specifying a `PRIMARY KEY` clause in the
`CREATE TABLE` statement. For example, the
-following Flink SQL statement creates a PrimaryKey Table with `shop_id` and
`user_id` as the primary key and distributes
+A Primary Key Table is created by specifying a `PRIMARY KEY` clause in the
`CREATE TABLE` statement. For example, the
+following Flink SQL statement creates a Primary Key Table with `shop_id` and
`user_id` as the primary key and distributes
the data into 4 buckets:
```sql title="Flink SQL"
@@ -49,13 +49,13 @@ In Fluss primary key table, each row of data has a unique
primary key.
If multiple entries with the same primary key are written to the Fluss primary
key table, only the last entry will be
retained.
-For [Partitioned PrimaryKey
Table](table-design/data-distribution/partitioning.md), the primary key must
contain the
+For [Partitioned Primary Key
Table](table-design/data-distribution/partitioning.md), the primary key must
contain the
partition key.
## Bucket Assigning
For primary key tables, Fluss always determines which bucket the data belongs
to based on the hash value of the bucket
-key (It must be a subset of the primary keys excluding partition keys of the
primary key table) for each record. If the bucket key is not specified, the
bucket key will used as the primary key (excluding the partition key).
+key (It must be a subset of the primary keys excluding partition keys of the
primary key table) for each record. If the bucket key is not specified, the
bucket key will be used as the primary key (excluding the partition key).
Data with the same hash value will be distributed to the same bucket.
## Partial Update
@@ -94,20 +94,20 @@ follows:
## Merge Engines
-The **Merge Engine** in Fluss is a core component designed to efficiently
handle and consolidate data updates for PrimaryKey Tables.
+The **Merge Engine** in Fluss is a core component designed to efficiently
handle and consolidate data updates for Primary Key Tables.
It offers users the flexibility to define how incoming data records are merged
with existing records sharing the same primary key.
-However, users can specify a different merge engine to customize the merging
behavior according to their specific use cases
+However, users can specify a different merge engine to customize the merging
behavior according to their specific use cases.
The following merge engines are supported:
-1. [Default Merge Engine
(LastRow)](table-design/table-types/pk-table/merge-engines/default.md)
-2. [FirstRow Merge
Engine](table-design/table-types/pk-table/merge-engines/first-row.md)
-3. [Versioned Merge
Engine](table-design/table-types/pk-table/merge-engines/versioned.md)
+1. [Default Merge Engine (LastRow)](merge-engines/default.md)
+2. [FirstRow Merge Engine](merge-engines/first-row.md)
+3. [Versioned Merge Engine](merge-engines/versioned.md)
## Changelog Generation
-Fluss will capture the changes when inserting, updating, deleting records on
the primary-key table, which is known as
+Fluss will capture the changes when inserting, updating, deleting records on
the Primary Key Table, which is known as
the changelog. Downstream consumers can directly consume the changelog to
obtain the changes in the table. For example,
consider the following primary key table in Fluss:
@@ -121,7 +121,7 @@ CREATE TABLE T
);
```
-If the data written to the primary-key table is
+If the data written to the Primary Key Table is
sequentially `+I(1, 2.0, 'apple')`, `+I(1, 4.0, 'banana')`, `-D(1, 4.0,
'banana')`, then the following change data will
be generated. For example, the following Flink SQL statements illustrate this
behavior:
@@ -164,13 +164,13 @@ For primary key tables, Fluss supports various kinds of
querying abilities.
For a primary key table, the default read method is a full snapshot followed
by incremental data. First, the
snapshot data of the table is consumed, followed by the changelog data of the
table.
-It is also possible to only consume the changelog data of the table. For more
details, please refer to the [Flink Reads](engine-flink/reads.md)
+It is also possible to only consume the changelog data of the table. For more
details, please refer to the [Flink Reads](../../../engine-flink/reads.md)
### Lookup
-Fluss primary key table can lookup data by the primary keys. If the key exists
in Fluss, lookup will return a unique row. it always used in [Flink Lookup
Join](engine-flink/lookups.md#lookup).
+Fluss primary key table can lookup data by the primary keys. If the key exists
in Fluss, lookup will return a unique row. It is always used in [Flink Lookup
Join](../../../engine-flink/lookups.md#lookup).
### Prefix Lookup
Fluss primary key table can also do prefix lookup by the prefix subset primary
keys. Unlike lookup, prefix lookup
-will scan data based on the prefix of primary keys and may return multiple
rows. It always used in [Flink Prefix Lookup
Join](engine-flink/lookups.md#prefix-lookup).
\ No newline at end of file
+will scan data based on the prefix of primary keys and may return multiple
rows. It is always used in [Flink Prefix Lookup
Join](../../../engine-flink/lookups.md#prefix-lookup).
diff --git
a/website/docs/table-design/table-types/pk-table/merge-engines/first-row.md
b/website/docs/table-design/table-types/pk-table/merge-engines/first-row.md
index 72aff5f5..c7dfc141 100644
--- a/website/docs/table-design/table-types/pk-table/merge-engines/first-row.md
+++ b/website/docs/table-design/table-types/pk-table/merge-engines/first-row.md
@@ -25,7 +25,7 @@ sidebar_position: 3
# FirstRow Merge Engine
By setting `'table.merge-engine' = 'first_row'` in the table properties, users
can retain the first record for each primary key.
-This configuration generates an insert-only changelog, allowing downstream
Flink jobs to treat the PrimaryKey Table as an append-only Log Table.
+This configuration generates an insert-only changelog, allowing downstream
Flink jobs to treat the Primary Key Table as an append-only Log Table.
As a result, downstream transformations that do not support
retractions/changelogs, such as [Window
Aggregations](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/queries/window-agg/)
and [Interval
Joins](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/queries/joins/#interval-joins),
can be applied seamlessly.
@@ -62,4 +62,4 @@ SELECT * FROM T WHERE k = 1;
-- +---+-----+------+
-- | 1 | 2.0 | t1 |
-- +---+-----+------+
-```
\ No newline at end of file
+```
diff --git
a/website/docs/table-design/table-types/pk-table/merge-engines/index.md
b/website/docs/table-design/table-types/pk-table/merge-engines/index.md
index c98c9f95..6e1c5d26 100644
--- a/website/docs/table-design/table-types/pk-table/merge-engines/index.md
+++ b/website/docs/table-design/table-types/pk-table/merge-engines/index.md
@@ -23,12 +23,12 @@ sidebar_position: 1
# Merge Engines
-The **Merge Engine** in Fluss is a core component designed to efficiently
handle and consolidate data updates for PrimaryKey Tables.
+The **Merge Engine** in Fluss is a core component designed to efficiently
handle and consolidate data updates for Primary Key Tables.
It offers users the flexibility to define how incoming data records are merged
with existing records sharing the same primary key.
-However, users can specify a different merge engine to customize the merging
behavior according to their specific use cases
+However, users can specify a different merge engine to customize the merging
behavior according to their specific use cases.
The following merge engines are supported:
1. [Default Merge Engine
(LastRow)](table-design/table-types/pk-table/merge-engines/default.md)
2. [FirstRow Merge
Engine](table-design/table-types/pk-table/merge-engines/first-row.md)
-3. [Versioned Merge
Engine](table-design/table-types/pk-table/merge-engines/versioned.md)
\ No newline at end of file
+3. [Versioned Merge
Engine](table-design/table-types/pk-table/merge-engines/versioned.md)
