This is an automated email from the ASF dual-hosted git repository.
twalthr pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/master by this push:
new 5b7a64f [FLINK-10071] [docs] Document usage of INSERT INTO in SQL
Client
5b7a64f is described below
commit 5b7a64fcef3ad09df1ccc0f91386766ee553bf45
Author: Timo Walther <twal...@apache.org>
AuthorDate: Mon Aug 6 15:26:51 2018 +0200
[FLINK-10071] [docs] Document usage of INSERT INTO in SQL Client
This closes #6505.
---
docs/dev/table/sqlClient.md | 68 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 63 insertions(+), 5 deletions(-)
diff --git a/docs/dev/table/sqlClient.md b/docs/dev/table/sqlClient.md
index b735705..d35aa59 100644
--- a/docs/dev/table/sqlClient.md
+++ b/docs/dev/table/sqlClient.md
@@ -106,7 +106,9 @@ Alice, 1
Greg, 1
{% endhighlight %}
-The [configuration section](sqlClient.html#configuration) explains how to read
from table sources and configure other table program properties.
+Both result modes can be useful during the prototyping of SQL queries.
+
+After a query is defined, it can be submitted to the cluster as a
long-running, detached Flink job. For this, a target system that stores the
results needs to be specified using the [INSERT INTO
statement](sqlClient.html#detached-sql-queries). The [configuration
section](sqlClient.html#configuration) explains how to declare table sources
for reading data, how to declare table sinks for writing data, and how to
configure other table program properties.
{% top %}
@@ -161,7 +163,7 @@ Every environment file is a regular [YAML
file](http://yaml.org/). An example of
# Define table sources here.
tables:
- - name: MyTableName
+ - name: MyTableSource
type: source
connector:
type: filesystem
@@ -286,8 +288,8 @@ Both `connector` and `format` allow to define a property
version (which is curre
{% top %}
-User-defined Functions
---------------------
+### User-defined Functions
+
The SQL Client allows users to create custom, user-defined functions to be
used in SQL queries. Currently, these functions are restricted to be defined
programmatically in Java/Scala classes.
In order to provide a user-defined function, you need to first implement and
compile a function class that extends `ScalarFunction`, `AggregateFunction` or
`TableFunction` (see [User-defined Functions]({{ site.baseurl
}}/dev/table/udfs.html)). One or more functions can then be packaged into a
dependency JAR for the SQL Client.
@@ -313,7 +315,7 @@ functions:
Make sure that the order and types of the specified parameters strictly match
one of the constructors of your function class.
-### Constructor Parameters
+#### Constructor Parameters
Depending on the user-defined function, it might be necessary to parameterize
the implementation before using it in SQL statements.
@@ -369,6 +371,62 @@ This process can be recursively performed until all the
constructor parameters a
{% top %}
+Detached SQL Queries
+------------------------
+
+In order to define end-to-end SQL pipelines, SQL's `INSERT INTO` statement can
be used for submitting long-running, detached queries to a Flink cluster. These
queries produce their results into an external system instead of the SQL
Client. This allows for dealing with higher parallelism and larger amounts of
data. The CLI itself does not have any control over a detached query after
submission.
+
+{% highlight sql %}
+INSERT INTO MyTableSink SELECT * FROM MyTableSource
+{% endhighlight %}
+
+The table sink `MyTableSink` has to be declared in the environment file. See
the [connection page](connect.html) for more information about supported
external systems and their configuration. An example for an Apache Kafka table
sink is shown below.
+
+{% highlight yaml %}
+tables:
+ - name: MyTableSink
+ type: sink
+ update-mode: append
+ connector:
+ property-version: 1
+ type: kafka
+ version: 0.11
+ topic: OutputTopic
+ properties:
+ - key: zookeeper.connect
+ value: localhost:2181
+ - key: bootstrap.servers
+ value: localhost:9092
+ - key: group.id
+ value: testGroup
+ format:
+ property-version: 1
+ type: json
+ derive-schema: true
+ schema:
+ - name: rideId
+ type: LONG
+ - name: lon
+ type: FLOAT
+ - name: lat
+ type: FLOAT
+ - name: rideTime
+ type: TIMESTAMP
+{% endhighlight %}
+
+The SQL Client makes sure that a statement is successfully submitted to the
cluster. Once the query is submitted, the CLI will show information about the
Flink job.
+
+{% highlight text %}
+[INFO] Table update statement has been successfully submitted to the cluster:
+Cluster ID: StandaloneClusterId
+Job ID: 6f922fe5cba87406ff23ae4a7bb79044
+Web interface: http://localhost:8081
+{% endhighlight %}
+
+<span class="label label-danger">Attention</span> The SQL Client does not
track the status of the running Flink job after submission. The CLI process can
be shutdown after the submission without affecting the detached query. Flink's
[restart strategy]({{ site.baseurl }}/dev/restart_strategies.html) takes care
of the fault-tolerance. A query can be cancelled using Flink's web interface,
command-line, or REST API.
+
+{% top %}
+
Limitations & Future
--------------------
