This is an automated email from the ASF dual-hosted git repository. lzljs3620320 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/flink-table-store.git
The following commit(s) were added to refs/heads/master by this push: new e2612c0c [FLINK-28381] Add Spark Reader documentation e2612c0c is described below commit e2612c0cfb8e14e5d1c33df280e827f3f67c0426 Author: Jingsong Lee <jingsongl...@gmail.com> AuthorDate: Tue Jul 5 15:18:51 2022 +0800 [FLINK-28381] Add Spark Reader documentation This closes #194 --- docs/content/docs/development/overview.md | 3 +- docs/content/docs/engines/_index.md | 26 ++++++++ docs/content/docs/engines/overview.md | 31 +++++++++ docs/content/docs/engines/spark.md | 80 ++++++++++++++++++++++++ docs/content/docs/try-table-store/quick-start.md | 5 +- 5 files changed, 140 insertions(+), 5 deletions(-) diff --git a/docs/content/docs/development/overview.md b/docs/content/docs/development/overview.md index cf9bf08a..56809e48 100644 --- a/docs/content/docs/development/overview.md +++ b/docs/content/docs/development/overview.md @@ -56,8 +56,7 @@ __Note:__ Table Store is only supported since Flink 1.15. {{< /hint >}} You can get the bundle jar for the Table Store in one of the following ways: -- [Download the latest bundle jar](https://flink.apache.org/downloads.html) of - Flink Table Store. +- Download [flink-table-store-dist-{{< version >}}.jar](https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-store-dist/{{< version >}}/flink-table-store-dist-{{< version >}}.jar). - Build bundle jar under submodule `flink-table-store-dist` from source code. Flink Table Store has shaded all the dependencies in the package, so you don't have diff --git a/docs/content/docs/engines/_index.md b/docs/content/docs/engines/_index.md new file mode 100644 index 00000000..acf3f226 --- /dev/null +++ b/docs/content/docs/engines/_index.md @@ -0,0 +1,26 @@ +--- +title: Engines +icon: <i class="fa fa-random title maindish" aria-hidden="true"></i> +bold: true +sectionBreak: true +bookCollapseSection: true +weight: 3 +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> diff --git a/docs/content/docs/engines/overview.md b/docs/content/docs/engines/overview.md new file mode 100644 index 00000000..c35d0b35 --- /dev/null +++ b/docs/content/docs/engines/overview.md @@ -0,0 +1,31 @@ +--- +title: "Overview" +weight: 1 +type: docs +aliases: +- /engines/overview.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Overview + +Table Store not only supports Flink SQL writes and queries natively, +but also provides queries from other popular engines, such as +Apache Hive and Apache Spark. diff --git a/docs/content/docs/engines/spark.md b/docs/content/docs/engines/spark.md new file mode 100644 index 00000000..5883cd2c --- /dev/null +++ b/docs/content/docs/engines/spark.md @@ -0,0 +1,80 @@ +--- +title: "Spark" +weight: 3 +type: docs +aliases: +- /engines/spark.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Spark + +Table Store supports reading table store tables through Spark. + +## Version + +Table Store supports Spark 3+. It is highly recommended to use Spark 3+ version with many improvements. + +## Install + +{{< stable >}} +Download [flink-table-store-spark-{{< version >}}.jar](https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-store-spark/{{< version >}}/flink-table-store-spark-{{< version >}}.jar). +{{< /stable >}} +{{< unstable >}} + +You are using an unreleased version of Table Store. The simplest way of building Table Store is by running: + +```bash +mvn clean install -DskipTests +``` + +You can find the bundle jar in `"./flink-table-store-spark/target/flink-table-store-spark-{{< version >}}.jar"`. + +{{< /unstable >}} + +Copy Table Store Spark bundle jar to `spark/jars`. + +## Table Store Catalog + +The following command registers the Table Store's Spark catalog with the name `table_store`: + +```bash +spark-sql --conf spark.sql.catalog.table_store=org.apache.flink.table.store.spark.SparkCatalog \ + --conf spark.sql.catalog.table_store.warehouse=file:/tmp/warehouse +``` + +## Create Temporary View + +Use the `CREATE TEMPORARY VIEW` command to create a Spark mapping table on top of +an existing Table Store table if you don't want to use Table Store Catalog. + +```sql +CREATE TEMPORARY VIEW myTable +USING tablestore +OPTIONS ( + path "file:/tmpe/warehouse/default.db/myTable" +) +``` + +## Query Table + +```sql +SELECT * FROM table_store.default.myTable; +``` diff --git a/docs/content/docs/try-table-store/quick-start.md b/docs/content/docs/try-table-store/quick-start.md index 59a4f4e7..5c5ae941 100644 --- a/docs/content/docs/try-table-store/quick-start.md +++ b/docs/content/docs/try-table-store/quick-start.md @@ -45,8 +45,7 @@ tar -xzf flink-*.tgz ## Step 2: Copy Table Store Bundle Jar {{< stable >}} -[Download the latest bundle jar](https://flink.apache.org/downloads.html) of -Flink Table Store. +Download [flink-table-store-dist-{{< version >}}.jar](https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-store-dist/{{< version >}}/flink-table-store-dist-{{< version >}}.jar). {{< /stable >}} {{< unstable >}} @@ -65,7 +64,7 @@ The simplest way of building Table Store is by running: mvn clean install -DskipTests ``` -You can find the bundle jar in `"./flink-table-store-dist/target/flink-table-store-dist-*.jar"`. +You can find the bundle jar in `"./flink-table-store-dist/target/flink-table-store-dist-{{< version >}}.jar"`. {{< /unstable >}}