This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 486a058 [community] modify the doc of verifying apache release (#8084)
486a058 is described below
commit 486a0586ac545643bd40807727ba18d65cd861be
Author: Mingyu Chen <morningman....@gmail.com>
AuthorDate: Thu Feb 17 10:53:31 2022 +0800
[community] modify the doc of verifying apache release (#8084)
---
docs/en/community/verify-apache-release.md | 30 ++++++++++++++--------
docs/en/extending-doris/flink-doris-connector.md | 21 ++++++---------
docs/en/extending-doris/spark-doris-connector.md | 13 +++-------
docs/zh-CN/community/verify-apache-release.md | 30 ++++++++++++++--------
.../zh-CN/extending-doris/flink-doris-connector.md | 13 +++-------
.../zh-CN/extending-doris/spark-doris-connector.md | 12 +++------
6 files changed, 57 insertions(+), 62 deletions(-)
diff --git a/docs/en/community/verify-apache-release.md
b/docs/en/community/verify-apache-release.md
index 34bc4eb..e83fb08 100644
--- a/docs/en/community/verify-apache-release.md
+++ b/docs/en/community/verify-apache-release.md
@@ -30,7 +30,7 @@ To verify the release, following checklist can used to
reference:
1. [ ] Download links are valid.
2. [ ] Checksums and PGP signatures are valid.
-3. [ ] DISCLAIMER-WIP is included.
+3. [ ] DISCLAIMER or DISCLAIMER-WIP is included.
4. [ ] Source code artifacts have correct names matching the current release.
5. [ ] LICENSE and NOTICE files are correct for the repository.
6. [ ] All files have license headers if necessary.
@@ -55,6 +55,8 @@ wget https://www.apache.org/dist/incubator/doris/KEYS
GnuPG is recommended, which can install by yum install gnupg or apt-get
install gnupg.
+Here we use Doris main code release as an example. Other releases are similar.
+
``` shell
gpg --import KEYS
gpg --verify apache-doris-a.b.c-incubating-src.tar.gz.asc
apache-doris-a.b.c-incubating-src.tar.gz
@@ -63,20 +65,28 @@ sha512sum --check
apache-doris-a.b.c-incubating-src.tar.gz.sha512
## 3. Verify license header
-Apache RAT is recommended to verify license header, which can download as
following command.
+Here we use
[apache/skywalking-eyes](https://github.com/apache/skywalking-eyes) for source
license header validation.
-``` shell
-wget
http://mirrors.tuna.tsinghua.edu.cn/apache/creadur/apache-rat-0.13/apache-rat-0.13-bin.tar.gz
-tar zxvf apache-rat-0.13-bin.tar.gz
+Go to the source directory and execute the following command (requires a
Docker environment).
+
+```
+docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header
check
```
-Given your source dir is apache-doris-a.b.c-incubating-src, you can check with
following command.
-It will output a file list which don't include ASF license header, and these
files used other licenses.
+The output is similar to the following:
-``` shell
-/usr/java/jdk/bin/java -jar apache-rat-0.13/apache-rat-0.13.jar -a -d
apache-doris-a.b.c-incubating-src -E
apache-doris-a.b.c-incubating-src/.rat-excludes
```
+INFO GITHUB_TOKEN is not set, license-eye won't comment on the pull request
+INFO Loading configuration from file: .licenserc.yaml
+INFO Totally checked 5611 files, valid: 3926, invalid: 0, ignored: 1685,
fixed: 0
+```
+
+where an invalid of 0 means the check passed.
+
+> Some non-Apache License header files are documented in `.licenserc.yaml`.
## 4. Verify building
-To compile the Doris, please read [Compilation](../installing/compilation.html)
+* For Doris main code compilation, see [compilation
documentation](../installing/compilation.html)
+* Flink Doris Connector compilation, see [compilation
documentation](../extending-doris/flink-doris-connector.md)
+* Spark Doris Connector compilation, see [compilation
documentation](../extending-doris/spark-doris-connector.md)
diff --git a/docs/en/extending-doris/flink-doris-connector.md
b/docs/en/extending-doris/flink-doris-connector.md
index 3538f52..9b2ec5e 100644
--- a/docs/en/extending-doris/flink-doris-connector.md
+++ b/docs/en/extending-doris/flink-doris-connector.md
@@ -28,14 +28,14 @@ under the License.
- The Flink Doris Connector can support operations (read, insert, modify,
delete) data stored in Doris through Flink.
-Github: https://github.com/apache/incubator-doris-connectors
+Github: https://github.com/apache/incubator-doris-flink-connector
- * `Doris` table can be mapped to `DataStream` or `Table`.
+* `Doris` table can be mapped to `DataStream` or `Table`.
- >**Note:**
- >
- >1. Modification and deletion are only supported on the Unique Key model
- >2. The current deletion is to support Flink CDC to access data to achieve
automatic deletion. If it is to delete other data access methods, you need to
implement it yourself. For the data deletion usage of Flink CDC, please refer
to the last section of this document
+>**Note:**
+>
+>1. Modification and deletion are only supported on the Unique Key model
+>2. The current deletion is to support Flink CDC to access data to achieve
automatic deletion. If it is to delete other data access methods, you need to
implement it yourself. For the data deletion usage of Flink CDC, please refer
to the last section of this document
## Version Compatibility
@@ -63,18 +63,13 @@ Just change the `flink.version` here to be the same as your
Flink cluster versio
## Build and Install
-Execute following command in dir `extension/flink-doris-connector/`:
-
-**Notice:**
-
-1. If you have not compiled the doris source code as a whole, you need to
compile the Doris source code first, otherwise the thrift command will not be
found, and you need to execute `sh build.sh` in the `incubator-doris` directory.
-2. It is recommended to compile under the docker compile environment
`apache/incubator-doris:build-env-1.2` of doris, because the JDK version below
1.3 is 11, there will be compilation problems.
+Execute following command in source dir:
```bash
sh build.sh 1.11.6 2.12 # flink 1.11.6 scala 2.12
```
-After successful compilation, the file `doris-flink-1.0.0-SNAPSHOT.jar` will
be generated in the `output/` directory. Copy this file to `ClassPath` in
`Flink` to use `Flink-Doris-Connector`. For example, `Flink` running in `Local`
mode, put this file in the `jars/` folder. `Flink` running in `Yarn` cluster
mode, put this file in the pre-deployment package.
+After successful compilation, the file
`doris-flink-1.11.6-2.12-1.0.0-SNAPSHOT.jar` will be generated in the `output/`
directory. Copy this file to `ClassPath` in `Flink` to use
`Flink-Doris-Connector`. For example, `Flink` running in `Local` mode, put this
file in the `jars/` folder. `Flink` running in `Yarn` cluster mode, put this
file in the pre-deployment package.
**Remarks:**
diff --git a/docs/en/extending-doris/spark-doris-connector.md
b/docs/en/extending-doris/spark-doris-connector.md
index c7d71db..04c9bcc 100644
--- a/docs/en/extending-doris/spark-doris-connector.md
+++ b/docs/en/extending-doris/spark-doris-connector.md
@@ -28,7 +28,7 @@ under the License.
Spark Doris Connector can support reading data stored in Doris and writing
data to Doris through Spark.
-Github: https://github.com/apache/incubator-doris-connectors
+Github: https://github.com/apache/incubator-doris-spark-connector
- Support reading data from `Doris`.
- Support `Spark DataFrame` batch/stream writing data to `Doris`
@@ -42,23 +42,16 @@ Github: https://github.com/apache/incubator-doris-connectors
| 2.3.4-2.11.xx | 2.x | 0.12+ | 8 | 2.11 |
| 3.1.2-2.12.xx | 3.x | 0.12.+ | 8 | 2.12 |
-
-
## Build and Install
-Execute following command in dir `extension/spark-doris-connector/`:
-
-**Notice:**
-
-1. If you have not compiled the doris source code as a whole, you need to
compile the Doris source code first, otherwise the thrift command will not be
found, and you need to execute `sh build.sh` in the `incubator-doris` directory.
-2. It is recommended to compile under the docker compile environment
`apache/incubator-doris:build-env-1.2` of doris, because the JDK version below
1.3 is 11, there will be compilation problems.
+Execute following command in source dir
```bash
sh build.sh 2.3.4 2.11 ## spark 2.3.4 version, and scala 2.11
sh build.sh 3.1.2 2.12 ## spark 3.1.2 version, and scala 2.12
```
-After successful compilation, the file `doris-spark-1.0.0-SNAPSHOT.jar` will
be generated in the `output/` directory. Copy this file to `ClassPath` in
`Spark` to use `Spark-Doris-Connector`. For example, `Spark` running in `Local`
mode, put this file in the `jars/` folder. `Spark` running in `Yarn` cluster
mode, put this file in the pre-deployment package.
+After successful compilation, the file
`doris-spark-2.3.4-2.11-1.0.0-SNAPSHOT.jar` will be generated in the `output/`
directory. Copy this file to `ClassPath` in `Spark` to use
`Spark-Doris-Connector`. For example, `Spark` running in `Local` mode, put this
file in the `jars/` folder. `Spark` running in `Yarn` cluster mode, put this
file in the pre-deployment package.
## Using Maven
diff --git a/docs/zh-CN/community/verify-apache-release.md
b/docs/zh-CN/community/verify-apache-release.md
index a50c176..d342b74 100644
--- a/docs/zh-CN/community/verify-apache-release.md
+++ b/docs/zh-CN/community/verify-apache-release.md
@@ -30,7 +30,7 @@ under the License.
1. [ ] 下载链接是否合法。
2. [ ] 校验值和 PGP 签名是否合法。
-3. [ ] 是否包含 DISCLAIMER-WIP。
+3. [ ] 是否包含 DISCLAIMER 或 DISCLAIMER-WIP 文件。
4. [ ] 代码是否和当前发布版本相匹配。
5. [ ] LICENSE 和 NOTICE 文件是否正确。
6. [ ] 所有文件都携带必要的协议说明。
@@ -60,6 +60,8 @@ CentOS: yum install gnupg
Ubuntu: apt-get install gnupg
```
+这里以 Doris 主代码 release 为例。其他 release 类似。
+
``` shell
gpg --import KEYS
gpg --verify apache-doris-a.b.c-incubating-src.tar.gz.asc
apache-doris-a.b.c-incubating-src.tar.gz
@@ -68,20 +70,28 @@ sha512sum --check
apache-doris-a.b.c-incubating-src.tar.gz.sha512
## 3. 验证源码协议头
-推荐使用 Apache RAT 验证源码协议，可以从以下链接下载：
+这里我们使用 [apache/skywalking-eyes](https://github.com/apache/skywalking-eyes)
进行源码协议头验证。
-``` shell
-wget
http://mirrors.tuna.tsinghua.edu.cn/apache/creadur/apache-rat-0.13/apache-rat-0.13-bin.tar.gz
-tar zxvf apache-rat-0.13-bin.tar.gz
+进入源码目录，执行以下命令（需要 Docker 环境）：
+
+```
+docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header
check
```
-假设源码目录名称为 apache-doris-a.b.c-incubating-src，可以使用以下命令进行验证。
-这个命令会产生一个文件，其中列举了所有非 ASF 协议的文件。
+输出类似如下结果:
-``` shell
-/usr/java/jdk/bin/java -jar apache-rat-0.13/apache-rat-0.13.jar -a -d
apache-doris-a.b.c-incubating-src -E
apache-doris-a.b.c-incubating-src/.rat-excludes
```
+INFO GITHUB_TOKEN is not set, license-eye won't comment on the pull request
+INFO Loading configuration from file: .licenserc.yaml
+INFO Totally checked 5611 files, valid: 3926, invalid: 0, ignored: 1685,
fixed: 0
+```
+
+其中 invalid 为 0 表示检查通过。
+
+> 部分非 Apache License 协议头的文件记录在 `.licenserc.yaml` 中。
## 4. 验证编译
-详细的编译步骤，请参阅 [编译文档](../installing/compilation.html)
+* Doris 主代码编译，请参阅 [编译文档](../installing/compilation.html)
+* Flink Doris Connector 编译，请参阅
[编译文档](../extending-doris/flink-doris-connector.md)
+* Spark Doris Connector 编译，请参阅
[编译文档](../extending-doris/spark-doris-connector.md)
diff --git a/docs/zh-CN/extending-doris/flink-doris-connector.md
b/docs/zh-CN/extending-doris/flink-doris-connector.md
index 756c99e..d34e648 100644
--- a/docs/zh-CN/extending-doris/flink-doris-connector.md
+++ b/docs/zh-CN/extending-doris/flink-doris-connector.md
@@ -30,7 +30,7 @@ under the License.
Flink Doris Connector 可以支持通过 Flink 操作（读取、插入、修改、删除） Doris 中存储的数据。
-代码库地址：https://github.com/apache/incubator-doris-connectors
+代码库地址：https://github.com/apache/incubator-doris-flink-connector
* 可以将`Doris`表映射为`DataStream`或者`Table`。
@@ -65,18 +65,13 @@ Flink Doris Connector 可以支持通过 Flink 操作（读取、插入、修改
## 编译与安装
-在 `extension/flink-doris-connector/` 源码目录下执行：
-
-**注意**
-
-1. 这里如果你没有整体编译过 doris 源码，需要首先编译一次 Doris 源码，不然会出现 thrift 命令找不到的情况，需要到
`incubator-doris` 目录下执行 `sh build.sh`
-2. 建议在 doris 的 docker 编译环境 `apache/incubator-doris:build-env-1.2` 下进行编译，因为 1.3
下面的JDK 版本是 11，会存在编译问题。
+在源码目录下执行：
```bash
sh build.sh 1.11.6 2.12 # flink 1.11.6 scala 2.12
```
-编译成功后，会在 `output/` 目录下生成文件 `doris-flink-1.0.0-SNAPSHOT.jar` 。将此文件复制到 `Flink` 的
`ClassPath` 中即可使用 `Flink-Doris-Connector` 。例如， `Local` 模式运行的 `Flink` ，将此文件放入
`jars/` 文件夹下。 `Yarn` 集群模式运行的 `Flink` ，则将此文件放入预部署包中。
+编译成功后，会在 `output/` 目录下生成文件 `doris-flink-1.11.6-2.12-1.0.0-SNAPSHOT.jar`
。将此文件复制到 `Flink` 的 `ClassPath` 中即可使用 `Flink-Doris-Connector` 。例如， `Local` 模式运行的
`Flink` ，将此文件放入 `jars/` 文件夹下。 `Yarn` 集群模式运行的 `Flink` ，则将此文件放入预部署包中。
**备注**
@@ -105,8 +100,6 @@ enable_http_server_v2 = true
`1.11.6 ` 可以根据flink 版本替换成替换成 `1.12.7` 或者 `1.13.5`
-
-
## 使用方法
Flink读写Doris数据主要有三种方式
diff --git a/docs/zh-CN/extending-doris/spark-doris-connector.md
b/docs/zh-CN/extending-doris/spark-doris-connector.md
index 81e7f91..1b4391d 100644
--- a/docs/zh-CN/extending-doris/spark-doris-connector.md
+++ b/docs/zh-CN/extending-doris/spark-doris-connector.md
@@ -28,7 +28,7 @@ under the License.
Spark Doris Connector 可以支持通过 Spark 读取 Doris 中存储的数据，也支持通过Spark写入数据到Doris。
-代码库地址：https://github.com/apache/incubator-doris-connectors
+代码库地址：https://github.com/apache/incubator-doris-spark-connector
- 支持从`Doris`中读取数据
- 支持`Spark DataFrame`批量/流式 写入`Doris`
@@ -42,15 +42,9 @@ Spark Doris Connector 可以支持通过 Spark 读取 Doris 中存储的数据
| 2.3.4-2.11.xx | 2.x | 0.12+ | 8 | 2.11 |
| 3.1.2-2.12.xx | 3.x | 0.12.+ | 8 | 2.12 |
-
## 编译与安装
-在 `extension/spark-doris-connector/` 源码目录下执行：
-
-**注意：**
-
-1. 这里如果你没有整体编译过 doris 源码，需要首先编译一次 Doris 源码，不然会出现 thrift 命令找不到的情况，需要到
`incubator-doris` 目录下执行 `sh build.sh`
-2. 建议在 doris 的 docker 编译环境 `apache/incubator-doris:build-env-1.2` 下进行编译，因为 1.3
下面的JDK 版本是 11，会存在编译问题。
+在源码目录下执行：
```bash
sh build.sh 2.3.4 2.11 ## spark 2.3.4, scala 2.11
@@ -58,7 +52,7 @@ sh build.sh 3.1.2 2.12 ## spark 3.1.2, scala 2.12
```
-编译成功后，会在 `output/` 目录下生成文件 `doris-spark-1.0.0-SNAPSHOT.jar`。将此文件复制到 `Spark` 的
`ClassPath` 中即可使用 `Spark-Doris-Connector`。例如，`Local` 模式运行的 `Spark`，将此文件放入
`jars/` 文件夹下。`Yarn`集群模式运行的`Spark`，则将此文件放入预部署包中。
+编译成功后，会在 `output/` 目录下生成文件 `doris-spark-2.3.4-2.11-1.0.0-SNAPSHOT.jar`。将此文件复制到
`Spark` 的 `ClassPath` 中即可使用 `Spark-Doris-Connector`。例如，`Local` 模式运行的
`Spark`，将此文件放入 `jars/` 文件夹下。`Yarn`集群模式运行的`Spark`，则将此文件放入预部署包中。
## 使用Maven管理
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
