This is an automated email from the ASF dual-hosted git repository. shengkai pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push: new 00e1ef38f98 [FLINK-28658][docs] Add docs for job statements 00e1ef38f98 is described below commit 00e1ef38f98d176b2ed00aed3176b66d9b75ec9c Author: Paul Lin <paullin3...@gmail.com> AuthorDate: Sat Feb 11 17:17:08 2023 +0800 [FLINK-28658][docs] Add docs for job statements This closes #21912 --- docs/content.zh/docs/dev/table/sql/jar.md | 2 +- docs/content.zh/docs/dev/table/sql/job.md | 96 ++++++++++++ docs/content.zh/docs/dev/table/sql/show.md | 12 +- docs/content.zh/docs/dev/table/sqlClient.md | 161 ++++++++++++++++++++- .../content/docs/dev/table/sql-gateway/overview.md | 2 +- docs/content/docs/dev/table/sql/jar.md | 4 +- docs/content/docs/dev/table/sql/job.md | 97 +++++++++++++ docs/content/docs/dev/table/sql/show.md | 13 +- docs/content/docs/dev/table/sqlClient.md | 49 ++++++- 9 files changed, 416 insertions(+), 20 deletions(-) diff --git a/docs/content.zh/docs/dev/table/sql/jar.md b/docs/content.zh/docs/dev/table/sql/jar.md index c51698e20d5..f6857ee753e 100644 --- a/docs/content.zh/docs/dev/table/sql/jar.md +++ b/docs/content.zh/docs/dev/table/sql/jar.md @@ -1,5 +1,5 @@ --- -title: "JAR Statements" +title: "JAR 语句" weight: 16 type: docs aliases: diff --git a/docs/content.zh/docs/dev/table/sql/job.md b/docs/content.zh/docs/dev/table/sql/job.md new file mode 100644 index 00000000000..e3ce4783ebb --- /dev/null +++ b/docs/content.zh/docs/dev/table/sql/job.md @@ -0,0 +1,96 @@ +--- +title: "JOB 语句" +weight: 16 +type: docs +aliases: +- /dev/table/sql/job.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# JOB Statements + +Job 语句用于管理作业的生命周期。 + +目前 Flink SQL 支持以下 JOB 语句: +- SHOW JOBS +- STOP JOB + +## 执行 JOB 语句 + +{{< tabs "show jobs statement" >}} +{{< tab "SQL CLI" >}} + +以下示例展示如何在 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 中执行 JOB 语句. + +{{< /tab >}} +{{< /tabs >}} + +{{< tabs "show jobs" >}} +{{< tab "SQL CLI" >}} +```sql +Flink SQL> SHOW JOBS; ++----------------------------------+----------+---------+-------------------------+ +| job id | job name | status | start time | ++----------------------------------+----------+---------+-------------------------+ +| 228d70913eab60dda85c5e7f78b5782c | myjob | RUNNING | 2023-02-11T05:03:51.523 | ++----------------------------------+----------+---------+-------------------------+ + +Flink SQL> SET 'state.savepoints.dir'='file:/tmp/'; +[INFO] Execute statement succeed. + +Flink SQL> STOP JOB '228d70913eab60dda85c5e7f78b5782c' WITH SAVEPOINT; ++-----------------------------------------+ +| savepoint path | ++-----------------------------------------+ +| file:/tmp/savepoint-3addd4-0b224d9311e6 | ++-----------------------------------------+ +``` +{{< /tab >}} +{{< /tabs >}} + +## SHOW JOBS + +```sql +SHOW JOBS +``` + +展示 Flink 集群上的作业。 + +<span class="label label-danger">Attention</span> SHOW JOBS 语句仅适用于 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 或者 [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). + +## STOP JOB + +```sql +STOP JOB '<job_id>' [WITH SAVEPOINT] [WITH DRAIN] +``` + +停止指定作业。 + +**WITH SAVEPOINT** +在作业停止之前执行 Savepoin。 Savepoint 的路径可以通过集群配置的 +[state.savepoints.dir]({{< ref "docs/deployment/config" >}}#state-savepoints-dir) 指定, +或者通过 `SET` 语句指定(后者有更高优先级)。 + +**WITH DRAIN** +在触发 savepoint 之前将 Watermark 提升至最大。该操作会可能会触发窗口的计算。请您注意该操作可能导致您之后从该创建的 savepoint 恢复的作业结果不正确。 + +<span class="label label-danger">Attention</span> STOP JOB 语句仅适用于 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 或者 [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). + +{{< top >}} diff --git a/docs/content.zh/docs/dev/table/sql/show.md b/docs/content.zh/docs/dev/table/sql/show.md index 76210678b9d..3bbc5b3d7c7 100644 --- a/docs/content.zh/docs/dev/table/sql/show.md +++ b/docs/content.zh/docs/dev/table/sql/show.md @@ -746,6 +746,16 @@ SHOW JARS 展示所有通过 [`ADD JAR`]({{< ref "docs/dev/table/sql/jar" >}}#add-jar) 语句加入到 session classloader 中的 jar。 -<span class="label label-danger">Attention</span> 当前 SHOW JARS 命令只能在 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 中使用。 +<span class="label label-danger">Attention</span> 当前 SHOW JARS 命令只能在 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 或者 [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}) 中使用. + +## SHOW JOBS + +```sql +SHOW JOBS +``` + +展示集群中所有作业。 + +<span class="label label-danger">Attention</span> 当前 SHOW JOBS 命令只能在 [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) 或者 [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}) 中使用. {{< top >}} diff --git a/docs/content.zh/docs/dev/table/sqlClient.md b/docs/content.zh/docs/dev/table/sqlClient.md index 9d89ef29233..130a08be6a8 100644 --- a/docs/content.zh/docs/dev/table/sqlClient.md +++ b/docs/content.zh/docs/dev/table/sqlClient.md @@ -167,7 +167,7 @@ Received a total of 5 rows ### Key-strokes -There is a list of available key-strokes in SQL client +There is a list of available key-strokes in SQL Client | Key-Stroke (Linux, Windows(WSL)) | Key-Stroke (Mac) | Description | |:---------------------------------|------------------|:---------------------------------------------------------------------------------------| @@ -199,7 +199,13 @@ There is a list of available key-strokes in SQL client | `ctrl-r` | `⌘-r` | History incremental search backward | | `ctrl-s` | `⌘-s` | History incremental search forward | -<a name="configuration"></a> +### Getting help + +The documentation of the SQL Client commands can be accessed by typing the `HELP` command. + +See also the general [SQL]({{< ref "docs/dev/table/sql/overview" >}}) documentation. + +{{< top >}} Configuration ------------- @@ -387,8 +393,112 @@ Mode "gateway" mode connects to the SQL gateway for submission. ### SQL Client Configuration +You can configure the SQL Client by setting the options below, or any valid [Flink configuration]({{< ref "docs/dev/table/config" >}}) entry: + +```sql +SET 'key' = 'value'; +``` + {{< generated/sql_client_configuration >}} +### SQL Client result modes + +The CLI supports **three modes** for maintaining and visualizing results. + +The **table mode** materializes results in memory and visualizes them in a regular, paginated table representation. +It can be enabled by executing the following command in the CLI: + +```text +SET 'sql-client.execution.result-mode' = 'table'; +``` + +The result of a query would then look like this, you can use the keys indicated at the bottom of the screen as well +as the arrows keys to navigate and open the various records: + +```text + + name age isHappy dob height + user1 20 true 1995-12-03 1.7 + user2 30 true 1972-08-02 1.89 + user3 40 false 1983-12-23 1.63 + user4 41 true 1977-11-13 1.72 + user5 22 false 1998-02-20 1.61 + user6 12 true 1969-04-08 1.58 + user7 38 false 1987-12-15 1.6 + user8 62 true 1996-08-05 1.82 + + + + +Q Quit + Inc Refresh G Goto Page N Next Page O Open Row +R Refresh - Dec Refresh L Last Page P Prev Page +``` + +The **changelog mode** does not materialize results and visualizes the result stream that is produced +by a [continuous query]({{< ref "docs/dev/table/concepts/dynamic_tables" >}}#continuous-queries) consisting of insertions (`+`) and retractions (`-`). + +```text +SET 'sql-client.execution.result-mode' = 'changelog'; +``` + +The result of a query would then look like this: + +```text + op name age isHappy dob height + +I user1 20 true 1995-12-03 1.7 + +I user2 30 true 1972-08-02 1.89 + +I user3 40 false 1983-12-23 1.63 + +I user4 41 true 1977-11-13 1.72 + +I user5 22 false 1998-02-20 1.61 + +I user6 12 true 1969-04-08 1.58 + +I user7 38 false 1987-12-15 1.6 + +I user8 62 true 1996-08-05 1.82 + + + + +Q Quit + Inc Refresh O Open Row +R Refresh - Dec Refresh + +``` + +The **tableau mode** is more like a traditional way which will display the results in the screen directly with a tableau format. +The displaying content will be influenced by the query execution type (`execution.type`). + +```text +SET 'sql-client.execution.result-mode' = 'tableau'; +``` + +The result of a query would then look like this: + +```text ++----+--------------------------------+-------------+---------+------------+--------------------------------+ +| op | name | age | isHappy | dob | height | ++----+--------------------------------+-------------+---------+------------+--------------------------------+ +| +I | user1 | 20 | true | 1995-12-03 | 1.7 | +| +I | user2 | 30 | true | 1972-08-02 | 1.89 | +| +I | user3 | 40 | false | 1983-12-23 | 1.63 | +| +I | user4 | 41 | true | 1977-11-13 | 1.72 | +| +I | user5 | 22 | false | 1998-02-20 | 1.61 | +| +I | user6 | 12 | true | 1969-04-08 | 1.58 | +| +I | user7 | 38 | false | 1987-12-15 | 1.6 | +| +I | user8 | 62 | true | 1996-08-05 | 1.82 | ++----+--------------------------------+-------------+---------+------------+--------------------------------+ +Received a total of 8 rows +``` + +Note that when you use this mode with streaming query, the result will be continuously printed on the console. If the input data of +this query is bounded, the job will terminate after Flink processed all input data, and the printing will also be stopped automatically. +Otherwise, if you want to terminate a running query, just type `CTRL-C` in this case, the job and the printing will be stopped. + +All these result modes can be useful during the prototyping of SQL queries. In all these modes, +results are stored in the Java heap memory of the SQL Client. In order to keep the CLI interface responsive, +the changelog mode only shows the latest 1000 changes. The table mode allows for navigating through +bigger results that are only limited by the available main memory and the configured +[maximum number of rows](#sql-client-execution-max-table-result-rows) (`sql-client.execution.max-table-result.rows`). + +<span class="label label-danger">Attention</span> Queries that are executed in a batch environment, can only be retrieved using the `table` or `tableau` result mode. + ### Initialize Session Using SQL Files A SQL query needs a configuration environment in which it is executed. SQL Client supports the `-i` @@ -486,7 +596,9 @@ The full list of offered SQL JARs can be found on the [connection to external sy You can refer to the [configuration]({{< ref "docs/dev/configuration/connector" >}}) section for information on how to configure connector and format dependencies. -Use SQL Client to submit job +{{< top >}} + +Usage ---------------------------- SQL Client allows users to submit jobs either within the interactive command line or using `-f` option to execute sql file. @@ -495,7 +607,7 @@ In both modes, SQL Client supports to parse and execute all types of the Flink s ### Interactive Command Line -In interactive Command Line, the SQL Client reads user inputs and executes the statement terminated by semicolon (`;`). +In interactive Command Line, the SQL Client reads user inputs and executes the statement terminated by a semicolon (`;`). SQL Client will print success message if the statement is executed successfully. When getting errors, SQL Client will also print error messages. By default, the error message only contains the error cause. In order to print the full exception stack for debugging, please set the @@ -552,7 +664,7 @@ This configuration: - set the savepoint path, - submit a sql job that load the savepoint from the specified savepoint path. -<span class="label label-danger">Attention</span> Compared to the interactive mode, SQL Client will stop execution and exit when there are errors. +<span class="label label-danger">Attention</span> Compared to the interactive mode, SQL Client will stop execution and exits when there are errors. ### Execute a set of SQL statements @@ -699,11 +811,15 @@ Cluster ID: StandaloneClusterId Job ID: 6f922fe5cba87406ff23ae4a7bb79044 ``` -<span class="label label-danger">Attention</span> The SQL Client does not track the status of the running Flink job after submission. The CLI process can be shutdown after the submission without affecting the detached query. Flink's `restart strategy` takes care of the fault-tolerance. A query can be cancelled using Flink's web interface, command-line, or REST API. +<span class="label label-danger">Attention</span> The SQL Client does not track the status of the +running Flink job after submission. The CLI process can be shutdown after the submission without +affecting the detached query. Flink's `restart strategy` takes care of the fault-tolerance. Please +use the job statements to [monitor the detached query status]({{< ref "docs/dev/table/sqlClient" >}}#monitoring-job-status) +or [stop the detached query]({{< ref "docs/dev/table/sqlClient" >}}#terminating-a-job). However, for batch users, it's more common that the next DML statement requires waiting until the previous DML statement finishes. In order to execute DML statements synchronously, you can set -`table.dml-sync` option `true` in SQL Client. +`table.dml-sync` option to `true` in SQL Client. ```sql Flink SQL> SET 'table.dml-sync' = 'true'; @@ -761,4 +877,35 @@ Flink SQL> RESET pipeline.name; If the option `pipeline.name` is not specified, SQL Client will generate a default name for the submitted job, e.g. `insert-into_<sink_table_name>` for `INSERT INTO` statements. +### Monitoring Job Status + +SQL Client supports to list jobs status in the cluster through `SHOW JOBS` statements. + +```sql +Flink SQL> SHOW JOBS; ++----------------------------------+---------------+----------+-------------------------+ +| job id | job name | status | start time | ++----------------------------------+---------------+----------+-------------------------+ +| 228d70913eab60dda85c5e7f78b5782c | kafka-to-hive | RUNNING | 2023-02-11T05:03:51.523 | ++----------------------------------+---------------+----------+-------------------------+ +``` + +### Terminating a Job + +SQL Client supports to stop jobs with or without savepoints through `STOP JOB` statements. + +```sql +Flink SQL> STOP JOB '228d70913eab60dda85c5e7f78b5782c' WITH SAVEPOINT; ++-----------------------------------------+ +| savepoint path | ++-----------------------------------------+ +| file:/tmp/savepoint-3addd4-0b224d9311e6 | ++-----------------------------------------+ +``` + +The savepoint path could be specified with [state.savepoints.dir]({{< ref "docs/deployment/config" >}}#state-savepoints-dir) +either in the cluster configuration or session configuration (the latter would take precedence). + +For more details about stopping jobs, please refer to [Job Statements]({{< ref "docs/dev/table/sql/job" >}}#stop-job). + {{< top >}} diff --git a/docs/content/docs/dev/table/sql-gateway/overview.md b/docs/content/docs/dev/table/sql-gateway/overview.md index 625fef2d851..b4c75ae088c 100644 --- a/docs/content/docs/dev/table/sql-gateway/overview.md +++ b/docs/content/docs/dev/table/sql-gateway/overview.md @@ -214,7 +214,7 @@ $ ./sql-gateway -Dkey=value Supported Endpoints ---------------- -Flink natively support [REST Endpoint]({{< ref "docs/dev/table/sql-gateway/rest" >}}) and [HiveServer2 Endpoint]({{< ref "docs/dev/table/hive-compatibility/hiveserver2" >}}). +Flink natively supports [REST Endpoint]({{< ref "docs/dev/table/sql-gateway/rest" >}}) and [HiveServer2 Endpoint]({{< ref "docs/dev/table/hive-compatibility/hiveserver2" >}}). The SQL Gateway is bundled with the REST Endpoint by default. With the flexible architecture, users are able to start the SQL Gateway with the specified endpoints by calling ```bash diff --git a/docs/content/docs/dev/table/sql/jar.md b/docs/content/docs/dev/table/sql/jar.md index 28008bb7e6e..cbce49b030b 100644 --- a/docs/content/docs/dev/table/sql/jar.md +++ b/docs/content/docs/dev/table/sql/jar.md @@ -76,7 +76,7 @@ ADD JAR '<path_to_filename>.jar' Add a JAR file to the list of resources, it supports adding the jar locates in a local or remote [file system]({{< ref "docs/deployment/filesystems/overview" >}}). The added JAR file can be listed using [`SHOW JARS`](#show-jars) statements. ### Limitation -Please don't use `ADD JAR` statement to load Hive source/sink/function/catalog. This is a known limitation of Hive connector and will be fixed in the future version. Currently, it's recommended to follow this [instruction]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies) to setup Hive integration. +Please don't use `ADD JAR` statements to load Hive source/sink/function/catalog. This is a known limitation of Hive connector and will be fixed in the future version. Currently, it's recommended to follow this [instruction]({{< ref "docs/connectors/table/hive/overview" >}}#dependencies) to setup Hive integration. ## SHOW JARS @@ -94,6 +94,6 @@ REMOVE JAR '<path_to_filename>.jar' Remove the specified jar that is added by the [`ADD JAR`](#add-jar) statements. -<span class="label label-danger">Attention</span> REMOVE JAR statement only work in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}). +<span class="label label-danger">Attention</span> REMOVE JAR statements only work in [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}). {{< top >}} diff --git a/docs/content/docs/dev/table/sql/job.md b/docs/content/docs/dev/table/sql/job.md new file mode 100644 index 00000000000..3cb45ff3efd --- /dev/null +++ b/docs/content/docs/dev/table/sql/job.md @@ -0,0 +1,97 @@ +--- +title: "JOB Statements" +weight: 16 +type: docs +aliases: +- /dev/table/sql/job.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# JOB Statements + +Job statements are used for management of Flink jobs. + +Flink SQL supports the following JOB statements for now: +- SHOW JOBS +- STOP JOB + +## Run a JOB statement + +{{< tabs "show jobs statement" >}} +{{< tab "SQL CLI" >}} + +The following examples show how to run `JOB` statements in [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}). + +{{< /tab >}} +{{< /tabs >}} + +{{< tabs "show jobs" >}} +{{< tab "SQL CLI" >}} +```sql +Flink SQL> SHOW JOBS; ++----------------------------------+----------+---------+-------------------------+ +| job id | job name | status | start time | ++----------------------------------+----------+---------+-------------------------+ +| 228d70913eab60dda85c5e7f78b5782c | myjob | RUNNING | 2023-02-11T05:03:51.523 | ++----------------------------------+----------+---------+-------------------------+ + +Flink SQL> SET 'state.savepoints.dir'='file:/tmp/'; +[INFO] Execute statement succeed. + +Flink SQL> STOP JOB '228d70913eab60dda85c5e7f78b5782c' WITH SAVEPOINT; ++-----------------------------------------+ +| savepoint path | ++-----------------------------------------+ +| file:/tmp/savepoint-3addd4-0b224d9311e6 | ++-----------------------------------------+ +``` +{{< /tab >}} +{{< /tabs >}} + +## SHOW JOBS + +```sql +SHOW JOBS +``` + +Show the jobs in the Flink cluster. + +<span class="label label-danger">Attention</span> SHOW JOBS statements only work in [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) or [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). + +## STOP JOB + +```sql +STOP JOB '<job_id>' [WITH SAVEPOINT] [WITH DRAIN] +``` + +Stop the specified job. + +**WITH SAVEPOINT** +Perform a savepoint right before stopping the job. The savepoint path could be specified with +[state.savepoints.dir]({{< ref "docs/deployment/config" >}}#state-savepoints-dir) either in +the cluster configuration or via `SET` statements (the latter would take precedence). + +**WITH DRAIN** +Increase the watermark to the maximum value before the last checkpoint barrier. Use it when you +want to terminate the job permanently. + +<span class="label label-danger">Attention</span> STOP JOB statements only work in [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) or [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). + +{{< top >}} diff --git a/docs/content/docs/dev/table/sql/show.md b/docs/content/docs/dev/table/sql/show.md index 33d3c9c71a0..71b5317e606 100644 --- a/docs/content/docs/dev/table/sql/show.md +++ b/docs/content/docs/dev/table/sql/show.md @@ -43,6 +43,7 @@ Flink SQL supports the following SHOW statements for now: - SHOW FUNCTIONS - SHOW MODULES - SHOW JARS +- SHOW JOBS ## Run a SHOW statement @@ -745,6 +746,16 @@ SHOW JARS Show all added jars in the session classloader which are added by [`ADD JAR`]({{< ref "docs/dev/table/sql/jar" >}}#add-jar) statements. -<span class="label label-danger">Attention</span> Currently `SHOW JARS` only works in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}). +<span class="label label-danger">Attention</span> Currently `SHOW JARS` statements only work in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) or [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). + +## SHOW JOBS + +```sql +SHOW JOBS +``` + +Show the jobs in the Flink cluster. + +<span class="label label-danger">Attention</span> Currently `SHOW JOBS` statements only work in the [SQL CLI]({{< ref "docs/dev/table/sqlClient" >}}) or [SQL Gateway]({{< ref "docs/dev/table/sql-gateway/overview" >}}). {{< top >}} diff --git a/docs/content/docs/dev/table/sqlClient.md b/docs/content/docs/dev/table/sqlClient.md index ee585f4238f..25ac9cf0e9b 100644 --- a/docs/content/docs/dev/table/sqlClient.md +++ b/docs/content/docs/dev/table/sqlClient.md @@ -87,7 +87,7 @@ FROM GROUP BY name; ``` -The SQL client will retrieve the results from the cluster and visualize them (you can close the result view by pressing the `Q` key): +The SQL Client will retrieve the results from the cluster and visualize them (you can close the result view by pressing the `Q` key): ```text +-------+-----+ @@ -106,7 +106,7 @@ The [configuration section](#configuration) explains how to declare table source ### Key-strokes -There is a list of available key-strokes in SQL client +There is a list of available key-strokes in SQL Client | Key-Stroke (Linux, Windows(WSL)) | Key-Stroke (Mac) | Description | |:---------------------------------|------------------|:---------------------------------------------------------------------------------------| @@ -140,7 +140,7 @@ There is a list of available key-strokes in SQL client ### Getting help -The documentation of the SQL client commands can be accessed by typing the `HELP` command. +The documentation of the SQL Client commands can be accessed by typing the `HELP` command. See also the general [SQL]({{< ref "docs/dev/table/sql/overview" >}}) documentation. @@ -332,7 +332,7 @@ Mode "gateway" mode connects to the SQL gateway for submission. ### SQL Client Configuration -You can configure the SQL client by setting the options below, or any valid [Flink configuration]({{< ref "docs/dev/table/config" >}}) entry: +You can configure the SQL Client by setting the options below, or any valid [Flink configuration]({{< ref "docs/dev/table/config" >}}) entry: ```sql SET 'key' = 'value'; @@ -340,7 +340,7 @@ SET 'key' = 'value'; {{< generated/sql_client_configuration >}} -### SQL client result modes +### SQL Client result modes The CLI supports **three modes** for maintaining and visualizing results. @@ -537,7 +537,7 @@ information on how to configure connector and format dependencies. {{< top >}} -Use SQL Client to submit job +Usage ---------------------------- SQL Client allows users to submit jobs either within the interactive command line or using `-f` option to execute sql file. @@ -750,7 +750,11 @@ Cluster ID: StandaloneClusterId Job ID: 6f922fe5cba87406ff23ae4a7bb79044 ``` -<span class="label label-danger">Attention</span> The SQL Client does not track the status of the running Flink job after submission. The CLI process can be shutdown after the submission without affecting the detached query. Flink's `restart strategy` takes care of the fault-tolerance. A query can be cancelled using Flink's web interface, command-line, or REST API. +<span class="label label-danger">Attention</span> The SQL Client does not track the status of the +running Flink job after submission. The CLI process can be shutdown after the submission without +affecting the detached query. Flink's `restart strategy` takes care of the fault-tolerance. Please +use the job statements to [monitor the detached query status]({{< ref "docs/dev/table/sqlClient" >}}#monitoring-job-status) +or [stop the detached query]({{< ref "docs/dev/table/sqlClient" >}}#terminating-a-job). However, for batch users, it's more common that the next DML statement requires waiting until the previous DML statement finishes. In order to execute DML statements synchronously, you can set @@ -812,4 +816,35 @@ Flink SQL> RESET pipeline.name; If the option `pipeline.name` is not specified, SQL Client will generate a default name for the submitted job, e.g. `insert-into_<sink_table_name>` for `INSERT INTO` statements. +### Monitoring Job Status + +SQL Client supports to list jobs status in the cluster through `SHOW JOBS` statements. + +```sql +Flink SQL> SHOW JOBS; ++----------------------------------+---------------+----------+-------------------------+ +| job id | job name | status | start time | ++----------------------------------+---------------+----------+-------------------------+ +| 228d70913eab60dda85c5e7f78b5782c | kafka-to-hive | RUNNING | 2023-02-11T05:03:51.523 | ++----------------------------------+---------------+----------+-------------------------+ +``` + +### Terminating a Job + +SQL Client supports to stop jobs with or without savepoints through `STOP JOB` statements. + +```sql +Flink SQL> STOP JOB '228d70913eab60dda85c5e7f78b5782c' WITH SAVEPOINT; ++-----------------------------------------+ +| savepoint path | ++-----------------------------------------+ +| file:/tmp/savepoint-3addd4-0b224d9311e6 | ++-----------------------------------------+ +``` + +The savepoint path could be specified with [state.savepoints.dir]({{< ref "docs/deployment/config" >}}#state-savepoints-dir) +either in the cluster configuration or session configuration (the latter would take precedence). + +For more details about stopping jobs, please refer to [Job Statements]({{< ref "docs/dev/table/sql/job" >}}#stop-job). + {{< top >}}