This is an automated email from the ASF dual-hosted git repository. diwu pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new 51e5c8419f9 [Doc](flink) add arrow flight sql read doc (#1241) 51e5c8419f9 is described below commit 51e5c8419f9fdc3d9c87f5567e59e333378b18c9 Author: wudi <676366...@qq.com> AuthorDate: Thu Oct 31 10:48:49 2024 +0800 [Doc](flink) add arrow flight sql read doc (#1241) --- common_docs_zh/ecosystem/flink-doris-connector.md | 21 +++++++++++++++++++ ecosystem/flink-doris-connector.md | 25 +++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/common_docs_zh/ecosystem/flink-doris-connector.md b/common_docs_zh/ecosystem/flink-doris-connector.md index 2fe2f9eb1a6..c16a821c6d5 100644 --- a/common_docs_zh/ecosystem/flink-doris-connector.md +++ b/common_docs_zh/ecosystem/flink-doris-connector.md @@ -100,6 +100,25 @@ CREATE TABLE flink_doris_source ( 'password' = 'password' ); ``` +:::info 备注 +Flink Connector 24.0.0 版本之后支持使用[Arrow Flight SQL](https://doris.apache.org/zh-CN/docs/dev/db-connect/arrow-flight-sql-connect/) 读取数据 +::: +```sql +CREATE TABLE doris_source ( +name STRING, +age int +) +WITH ( + 'connector' = 'doris', + 'fenodes' = 'FE_IP:HTTP_PORT', + 'table.identifier' = 'database.table', + 'source.use-flight-sql' = 'true', + 'source.flight-sql-port' = '{fe.conf:arrow_flight_sql_port}', + 'username' = 'root', + 'password' = '' +) +``` + #### DataStream @@ -342,6 +361,8 @@ ON a.city = c.city | doris.exec.mem.limit | 8192mb | N | 单个查询的内存限制。默认为 8GB,单位为字节 | | doris.deserialize.arrow.async | FALSE | N | 是否支持异步转换 Arrow 格式到 flink-doris-connector 迭代所需的 RowBatch | | doris.deserialize.queue.size | 64 | N | 异步转换 Arrow 格式的内部处理队列,当 doris.deserialize.arrow.async 为 true 时生效 | +| source.use-flight-sql | FALSE | N | 是否使用 Arrow Flight SQL 读取 | +| source.flight-sql-port | - | N | 使用 Arrow Flight SQL 读取时,FE 的 arrow_flight_sql_port | #### DataStream 专有配置项 | Key | Default Value | Required | Comment | diff --git a/ecosystem/flink-doris-connector.md b/ecosystem/flink-doris-connector.md index 9e7f0aa6239..5e0fa9b5c2f 100644 --- a/ecosystem/flink-doris-connector.md +++ b/ecosystem/flink-doris-connector.md @@ -98,6 +98,24 @@ CREATE TABLE flink_doris_source ( 'password' = 'password' ); ``` +:::info Note +Flink Connector 24.0.0 and later versions support using [Arrow Flight SQL](https://doris.apache.org/docs/dev/db-connect/arrow-flight-sql-connect/) to read data +::: +```sql +CREATE TABLE doris_source ( +name STRING, +age int +) +WITH ( + 'connector' = 'doris', + 'fenodes' = 'FE_IP:HTTP_PORT', + 'table.identifier' = 'database.table', + 'source.use-flight-sql' = 'true', + 'source.flight-sql-port' = '{fe.conf:arrow_flight_sql_port}', + 'username' = 'root', + 'password' = '' +) +``` **DataStream** @@ -335,9 +353,16 @@ ON a.city = c.city | doris.exec.mem.limit | 2147483648 | N | Memory limit for a single query. The default is 2GB, in bytes | | doris.deserialize.arrow.async | FALSE | N | Whether to support asynchronous conversion of Arrow format to RowBatch needed for flink-doris-connector iterations | | doris.deserialize.queue.size | 64 | N | Asynchronous conversion of internal processing queue in Arrow format, effective when doris.deserialize.arrow.async is true | +| source.use-flight-sql | FALSE | N | Whether to use Arrow Flight SQL to read | +| source.flight-sql-port | - | N | When using ArrowFlightSQL to read, FE's arrow_flight_sql_port | + +#### Datastream-specific configuration items +| Key | Default Value | Required | Comment | +| ----------------------------- | ------------------ | -------- | ------------------------------------------------------------ | | doris.read.field | -- | N | Read the list of column names of the Doris table, separated by commas | | doris.filter.query | -- | N | The expression to filter the read data, this expression is transparently passed to Doris. Doris uses this expression to complete source-side data filtering. For example age=18. | + ### Sink configuration items | Key | Default Value | Required | Comment | --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org