This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 500540b683 remove broker from backup (#878)
500540b683 is described below
commit 500540b6830923b28dd904b22449ad9185c6c9c9
Author: Yongqiang YANG <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Tue Jul 23 13:02:11 2024 +0800
remove broker from backup (#878)
---
docs/admin-manual/data-admin/backup.md | 81 +++-----------
.../Backup-and-Restore/BACKUP.md | 2 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 72 ++-----------
.../Backup-and-Restore/DROP-REPOSITORY.md | 6 +-
.../current/admin-manual/data-admin/backup.md | 75 ++-----------
.../{reparing-data.md => repairing-data.md} | 0
.../Backup-and-Restore/BACKUP.md | 2 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 58 ++--------
.../Backup-and-Restore/DROP-REPOSITORY.md | 2 +-
.../version-2.0/admin-manual/data-admin/backup.md | 119 +++++----------------
.../Backup-and-Restore/BACKUP.md | 9 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 60 ++---------
.../Backup-and-Restore/DROP-REPOSITORY.md | 2 +-
.../version-2.1/admin-manual/data-admin/backup.md | 75 ++-----------
.../Backup-and-Restore/BACKUP.md | 2 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 60 ++---------
.../Backup-and-Restore/DROP-REPOSITORY.md | 2 +-
.../version-2.0/admin-manual/data-admin/backup.md | 74 ++++---------
.../Backup-and-Restore/BACKUP.md | 9 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 57 ++--------
.../Backup-and-Restore/DROP-REPOSITORY.md | 6 +-
.../version-2.1/admin-manual/data-admin/backup.md | 81 +++-----------
.../Backup-and-Restore/BACKUP.md | 2 +-
.../Backup-and-Restore/CREATE-REPOSITORY.md | 57 ++--------
.../Backup-and-Restore/DROP-REPOSITORY.md | 6 +-
25 files changed, 171 insertions(+), 748 deletions(-)
diff --git a/docs/admin-manual/data-admin/backup.md
b/docs/admin-manual/data-admin/backup.md
index 881b23c1c8..c6c5698eef 100644
--- a/docs/admin-manual/data-admin/backup.md
+++ b/docs/admin-manual/data-admin/backup.md
@@ -26,12 +26,10 @@ under the License.
# Data Backup
-Doris supports backing up the current data in the form of files to the remote
storage system through the broker. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
+Doris supports backing up the current data in the form of files to the remote
storage system like S3 and HDFS. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
This feature requires Doris version 0.8.2+
-To use this function, you need to deploy the broker corresponding to the
remote storage. Such as BOS, HDFS, etc. You can view the currently deployed
broker through `SHOW BROKER;`.
-
## A brief explanation of the principle
The backup operation is to upload the data of the specified table or partition
directly to the remote warehouse for storage in the form of files stored by
Doris. When a user submits a Backup request, the system will perform the
following operations:
@@ -56,9 +54,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
## Start Backup
-1. Create a hdfs remote warehouse example_repo:
-
- **WITH HDFS (Recommended)**
+1. Create a hdfs remote warehouse example_repo (S3 skips step 1):
```sql
CREATE REPOSITORY `example_repo`
@@ -71,22 +67,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
);
```
- **WITH BROKER**
-
- This requires starting a broker process first.
-
- ```sql
- CREATE REPOSITORY `example_repo`
- WITH BROKER `broker_name`
- ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
- PROPERTIES
- (
- "username" = "user",
- "password" = "password"
- );
- ```
-
-2. Create a remote repository for s3 : s3_repo
+2. Create a remote repository for S3 : s3_repo (HDFS skips step 2)
```
CREATE REPOSITORY `s3_repo`
@@ -105,7 +86,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
>
>ON LOCATION is followed by Bucket Name here
-1. Full backup of table example_tbl under example_db to warehouse example_repo:
+3. Full backup of table example_tbl under example_db to warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label1
@@ -114,7 +95,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
PROPERTIES ("type" = "full");
```
-2. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
+4. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label2
@@ -126,7 +107,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
);
```
-4. View the execution of the most recent backup job:
+5. View the execution of the most recent backup job:
```sql
mysql> show BACKUP\G;
@@ -148,7 +129,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
1 row in set (0.01 sec)
```
-5. View existing backups in remote repositories:
+6. View existing backups in remote repositories:
```sql
mysql> SHOW SNAPSHOT ON example_repo WHERE SNAPSHOT = "snapshot_label1";
@@ -181,10 +162,7 @@ It is recommended to import the new and old clusters in
parallel for a period of
3. Both backup and recovery support operations at the minimum partition
(Partition) level. When the amount of data in the table is large, it is
recommended to perform operations by partition to reduce the cost of failed
retry.
4. Because of the backup and restore operations, the operations are the actual
data files. Therefore, when a table has too many shards, or a shard has too
many small versions, it may take a long time to backup or restore even if the
total amount of data is small. Users can use `SHOW PARTITIONS FROM table_name;`
and `SHOW TABLETS FROM table_name;` to view the number of shards in each
partition and the number of file versions in each shard to estimate job
execution time. The number of file [...]
5. When checking job status via `SHOW BACKUP` or `SHOW RESTORE` command. It is
possible to see error messages in the `TaskErrMsg` column. But as long as the
`State` column is not `CANCELLED`, the job is still continuing. These tasks may
retry successfully. Of course, some Task errors will also directly cause the
job to fail.
- Common `TaskErrMsg` errors are as follows:
- Q1: Backup to HDFS, the status shows UPLOADING, TaskErrMsg error
message: [13333: Close broker writer failed,
broker:TNetworkAddress(hostname=10.10.0.0, port=8000) msg:errors while close
file output stream, cause by: DataStreamer Exception : ]
- This is generally a network communication problem. Check the broker log
to see if a certain ip or port is blocked. If it is a cloud service, you need
to check whether is accessed the intranet. If so, you can add hdfs-site.xml in
the broker/conf folder, you need to add dfs.client.use.datanode.hostname=true
under the hdfs-site.xml configuration file, and configure the hostname mapping
of the HADOOP cluster on the broker node.
-7. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
+6. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
## Related Commands
@@ -192,58 +170,27 @@ It is recommended to import the new and old clusters in
parallel for a period of
1. CREATE REPOSITORY
- Create a remote repository path for backup or restore. This command
needs to use the Broker process to access the remote storage. Different brokers
need to provide different parameters. For details, please refer to [Broker
documentation](../../data-operate/import/broker-load-manual), or you can
directly back up to support through the S3 protocol For the remote storage of
AWS S3 protocol, or directly back up to HDFS, please refer to [Create Remote
Warehouse Documentation](../../sql- [...]
+ Create a remote repository path for backup or restore. Please refer to
[Create Repository
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md).
2. BACKUP
- Perform a backup operation.
+ Perform a backup operation. Please refer to [Backup
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md).
3. SHOW BACKUP
- View the execution of the most recent backup job, including:
-
- - JobId: The id of this backup job.
- - SnapshotName: The name (Label) of this backup job specified by the
user.
- - DbName: Database corresponding to the backup job.
- - State: The current stage of the backup job:
- - PENDING: The initial status of the job.
- - SNAPSHOTING: A snapshot operation is in progress.
- - UPLOAD_SNAPSHOT: The snapshot is over, ready to upload.
- - UPLOADING: Uploading snapshot.
- - SAVE_META: The metadata file is being generated locally.
- - UPLOAD_INFO: Upload metadata files and information about this backup
job.
- - FINISHED: The backup is complete.
- - CANCELLED: Backup failed or was canceled.
- - BackupObjs: List of tables and partitions involved in this backup.
- - CreateTime: Job creation time.
- - SnapshotFinishedTime: Snapshot completion time.
- - UploadFinishedTime: Snapshot upload completion time.
- - FinishedTime: The completion time of this job.
- - UnfinishedTasks: During `SNAPSHOTTING`, `UPLOADING` and other stages,
there will be multiple subtasks going on at the same time. The current stage
shown here is the task id of the unfinished subtasks.
- - TaskErrMsg: If there is an error in the execution of a subtask, the
error message of the corresponding subtask will be displayed here.
- - Status: Used to record some status information that may appear during
the entire job process.
- - Timeout: The timeout period of the job, in seconds.
+ View the execution of the most recent backup job. Please refer to [Show
Backup
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
4. SHOW SNAPSHOT
- View existing backups in the remote repository.
-
- - Snapshot: The name (Label) of the backup specified during backup.
- - Timestamp: Timestamp of the backup.
- - Status: Whether the backup is normal.
-
- More detailed backup information can be displayed if a where clause is
specified after `SHOW SNAPSHOT`.
-
- - Database: The database corresponding to the backup.
- - Details: Shows the complete data directory structure of the backup.
+ View existing backups in the remote repository. Please refer to [Show
Snapshot
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md).
5. CANCEL BACKUP
- Cancel the currently executing backup job.
+ Cancel the currently executing backup job. Please refer to [Cancel
Backup Reference]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md).
6. DROP REPOSITORY
- Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data.
+ Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data. Please refer to [Drop Repository Reference]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md).
## More Help
diff --git
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index ef5f7a286c..f6ea2a1a26 100644
---
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -103,7 +103,7 @@ BACKUP
1. Only one backup operation can be performed under the same database.
-2. The backup operation will back up the underlying table and [materialized
view](../../../../query-acceleration/materialized-view.md) of the specified
table or partition, and only one copy will be backed up.
+2. The backup operation will back up the underlying table and [Synchronous
materialized
view](../../../../query/view-materialized-view/materialized-view.md) of the
specified table or partition, and only one replica will be backed up.
[Asynchronous materialized
view](../../../../query/view-materialized-view/async-materialized-view.md) is
not supported.
3. Efficiency of backup operations
diff --git
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index 64398dcb61..72cf65c1e6 100644
---
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,21 @@ grammar:
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
illustrate:
-- Creation of repositories, relying on existing brokers or accessing cloud
storage directly through AWS s3 protocol, or accessing HDFS directly.
+- Creation of repositories, accessing cloud storage directly through AWS S3
protocol, or accessing HDFS directly.
- If it is a read-only repository, restores can only be done on the
repository. If not, backup and restore operations are available.
-- PROPERTIES are different according to different types of broker or S3 or
hdfs, see the example for details.
+- PROPERTIES are different according to different types of S3 or hdfs, see the
example for details.
- ON LOCATION : if it is S3 , here followed by the Bucket Name.
### Example
-1. Create a warehouse named bos_repo, rely on BOS broker "bos_broker", and the
data root directory is: bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. Create the same repository as Example 1, but with read-only properties:
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. Create a warehouse named hdfs_repo, rely on Baidu hdfs broker
"hdfs_broker", the data root directory is:
hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. Create a repository named s3_repo to link cloud storage directly without
going through the broker.
+1. Create a repository named s3_repo.
```sql
CREATE REPOSITORY `s3_repo`
@@ -109,7 +68,7 @@ PROPERTIES
);
```
-5. Create a repository named hdfs_repo to link HDFS directly without going
through the broker.
+2. Create a repository named hdfs_repo.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -124,7 +83,7 @@ PROPERTIES
### Keywords
```
-6. Create a repository named minio_repo to link minio storage directly through
the s3 protocol.
+6. Create a repository named minio_repo to link minio storage directly through
the S3 protocol.
```
CREATE REPOSITORY `minio_repo`
@@ -172,25 +131,6 @@ PROPERTIES
);
```
-9. Create repository and delete snapshots if exists.
-
-```sql
-CREATE REPOSITORY `s3_repo`
-WITH S3
-ON LOCATION "s3://s3-repo"
-PROPERTIES
-(
- "s3.endpoint" = "http://s3-REGION.amazonaws.com";,
- "s3.region" = "s3-REGION",
- "s3.access_key" = "AWS_ACCESS_KEY",
- "s3.secret_key"="AWS_SECRET_KEY",
- "s3.region" = "REGION",
- "delete_if_exists" = "true"
-);
-```
-
-Note: only the s3 service supports the "delete_if_exists" property.
-
### Keywords
CREATE, REPOSITORY
diff --git
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 6a7e6e5a66..7a0b937cdc 100644
---
a/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/docs/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,14 +42,14 @@ DROP REPOSITORY `repo_name`;
illustrate:
-- Deleting a warehouse just deletes the warehouse's mapping in Palo, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same broker and LOCATION.
+- Deleting a warehouse just deletes the warehouse's mapping in Doris, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same LOCATION.
### Example
-1. Delete the repository named bos_repo:
+1. Delete the repository named example_repo:
```sql
-DROP REPOSITORY `bos_repo`;
+DROP REPOSITORY `example_repo`;
```
### Keywords
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/backup.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/backup.md
index 819ca68924..bc9e63b5f0 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/backup.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/backup.md
@@ -26,13 +26,11 @@ under the License.
# 数据备份
-Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存储系统中。之后可以通过 恢复 命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期的进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移。
+Doris 支持将当前数据以文件的形式备份到 HDFS 和对象存储。之后可以通过恢复命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期地进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移，集群间无损迁移可以使用 CCR
(ccr.md)。
该功能需要 Doris 版本 0.8.2+
-使用该功能，需要部署对应远端存储的 broker。如 BOS、HDFS 等。可以通过 `SHOW BROKER;` 查看当前部署的 broker。
-
-## 简要原理说明
+## 原理说明
备份操作是将指定表或分区的数据，直接以 Doris 存储的文件的形式，上传到远端仓库中进行存储。当用户提交 Backup 请求后，系统内部会做如下操作：
@@ -56,9 +54,8 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
## 开始备份
-1. 创建一个 hdfs 的远程仓库 example_repo：
+1. 创建一个 HDFS 的远程仓库 example_repo（S3 存储请参考 2）：
- **WITH HDFS(推荐使用)**
```sql
CREATE REPOSITORY `example_repo`
WITH HDFS
@@ -70,22 +67,7 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
);
```
- **WITH BROKER**
-
- 需要先启动一个 BROKER 进程。
-
- ```sql
- CREATE REPOSITORY `example_repo`
- WITH BROKER `broker_name`
- ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
- PROPERTIES
- (
- "username" = "user",
- "password" = "password"
- );
- ```
-
-2. 创建一个 s3 的远程仓库 : s3_repo
+2. 创建一个 s3 的远程仓库 : s3_repo（HDFS 存储请参考 1）
```
CREATE REPOSITORY `s3_repo`
@@ -173,7 +155,7 @@ BACKUP 的更多用法可参考 [这里](../../sql-manual/sql-statements/Data-De
建议在迁移完成后，对新旧两个集群并行导入一段时间。完成数据和业务正确性校验后，再将业务迁移到新的集群。
-## 重点说明
+## 说明
1. 备份恢复相关的操作目前只允许拥有 ADMIN 权限的用户执行。
2. 一个 Database 内，只允许有一个正在执行的备份或恢复作业。
@@ -191,59 +173,24 @@ BACKUP 的更多用法可参考
[这里](../../sql-manual/sql-statements/Data-De
1. CREATE REPOSITORY
- 创建一个远端仓库路径，用于备份或恢复。该命令需要借助 Broker 进程访问远端存储，不同的 Broker 需要提供不同的参数，具体请参阅
[Broker 文档](../../data-operate/import/broker-load-manual#其他-broker-导入)，也可以直接通过
S3 协议备份到支持 AWS S3 协议的远程存储上去，也可以直接备份到 HDFS，具体参考
[创建远程仓库文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md)
+ 创建一个远端仓库路径，用于备份或恢复。具体参考
[创建远程仓库文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md)。
2. BACKUP
- 执行一次备份操作。
+ 执行一次备份操作。具体参考 [备份文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)。
3. SHOW BACKUP
- 查看最近一次 backup 作业的执行情况，包括：
-
- - JobId：本次备份作业的 id。
- - SnapshotName：用户指定的本次备份作业的名称（Label）。
- - DbName：备份作业对应的 Database。
- - State：备份作业当前所在阶段：
- - PENDING：作业初始状态。
- - SNAPSHOTING：正在进行快照操作。
- - UPLOAD_SNAPSHOT：快照结束，准备上传。
- - UPLOADING：正在上传快照。
- - SAVE_META：正在本地生成元数据文件。
- - UPLOAD_INFO：上传元数据文件和本次备份作业的信息。
- - FINISHED：备份完成。
- - CANCELLED：备份失败或被取消。
- - BackupObjs：本次备份涉及的表和分区的清单。
- - CreateTime：作业创建时间。
- - SnapshotFinishedTime：快照完成时间。
- - UploadFinishedTime：快照上传完成时间。
- - FinishedTime：本次作业完成时间。
- - UnfinishedTasks：在 `SNAPSHOTTING`，`UPLOADING`
等阶段，会有多个子任务在同时进行，这里展示的当前阶段，未完成的子任务的 task id。
- - TaskErrMsg：如果有子任务执行出错，这里会显示对应子任务的错误信息。
- - Status：用于记录在整个作业过程中，可能出现的一些状态信息。
- - Timeout：作业的超时时间，单位是秒。
+ 查看最近一次 backup 作业的执行情况。具体参考
[查看备份作业文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
4. SHOW SNAPSHOT
- 查看远端仓库中已存在的备份。
-
- - Snapshot：备份时指定的该备份的名称（Label）。
- - Timestamp：备份的时间戳。
- - Status：该备份是否正常。
-
- 如果在 `SHOW SNAPSHOT` 后指定了 where 子句，则可以显示更详细的备份信息。
-
- - Database：备份时对应的 Database。
- - Details：展示了该备份完整的数据目录结构。
+ 查看远端仓库中已存在的备份。具体参考
[查看备份文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md)。
5. CANCEL BACKUP
- 取消当前正在执行的备份作业。
+ 取消当前正在执行的备份作业。具体参考 [取消备份作业文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md)。
6. DROP REPOSITORY
- 删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。
-
-## 更多帮助
-
- 关于 BACKUP 使用的更多详细语法及最佳实践，请参阅
[BACKUP](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)
命令手册，你也可以在 MySql 客户端命令行下输入 `HELP BACKUP` 获取更多帮助信息。
+ 删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。具体参考 [删除远程仓库文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md)。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/reparing-data.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/repairing-data.md
similarity index 100%
rename from
i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/reparing-data.md
rename to
i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/data-admin/repairing-data.md
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index 82081e5537..4840f01757 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -107,7 +107,7 @@ BACKUP
1. 同一个数据库下只能进行一个备份操作。
-2. 备份操作会备份指定表或分区的基础表及
[物化视图](../../../../query-acceleration/materialized-view.md)，并且仅备份一副本。
+2.
备份操作会备份指定表或分区的基础表及同步物化视图[物化视图](../../../../query/view-materialized-view/materialized-view.md)，并且仅备份一副本，异步物化视图(../../../../query/view-materialized-view/async-materialized-view.md)当前未支持。
3. 备份操作的效率
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index b6b4538475..49a97bc15f 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,20 @@ CREATE REPOSITORY
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
说明：
-- 仓库的创建，依赖于已存在的 broker 或者直接通过 AWS s3 协议访问云存储，或者直接访问 HDFS
- 如果是只读仓库，则只能在仓库上进行恢复。如果不是，则可以进行备份和恢复操作。
-- 根据 broker 或者 S3、hdfs 的不同类型，PROPERTIES 有所不同，具体见示例。
+- 根据 S3、HDFS 的不同类型，PROPERTIES 有所不同，具体见示例。
- ON LOCATION ,如果是 S3 , 这里后面跟的是 Bucket Name。
### Example
-1. 创建名为 bos_repo 的仓库，依赖 BOS broker "bos_broker"，数据根目录为：bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. 创建和示例 1 相同的仓库，但属性为只读：
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. 创建名为 hdfs_repo 的仓库，依赖 Baidu hdfs broker
"hdfs_broker"，数据根目录为：hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. 创建名为 s3_repo 的仓库，直接链接云存储，而不通过 broker.
+1. 创建名为 s3_repo 的仓库.
```sql
CREATE REPOSITORY `s3_repo`
@@ -108,7 +66,7 @@ PROPERTIES
);
```
-5. 创建名为 hdfs_repo 的仓库，直接链接 HDFS，而不通过 broker.
+2. 创建名为 hdfs_repo 的仓库.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -121,7 +79,7 @@ PROPERTIES
);
```
-6. 创建名为 minio_repo 的仓库，直接通过 s3 协议链接 minio.
+3. 创建名为 minio_repo 的仓库.
```sql
CREATE REPOSITORY `minio_repo`
@@ -137,7 +95,7 @@ PROPERTIES
);
```
-7. 使用临时秘钥创建名为 minio_repo 的仓库
+4. 使用临时秘钥创建名为 minio_repo 的仓库
```
CREATE REPOSITORY `minio_repo`
@@ -153,7 +111,7 @@ PROPERTIES
)
```
-1. 使用腾讯云 COS 创建仓库
+5. 使用腾讯云 COS 创建仓库
```
CREATE REPOSITORY `cos_repo`
@@ -168,7 +126,7 @@ PROPERTIES
);
```
-9. 创建仓库并删除已经存在的 snapshot
+6. 创建仓库并删除已经存在的 snapshot
```sql
CREATE REPOSITORY `s3_repo`
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 81854d0ef1..01ea970885 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,7 +42,7 @@ DROP REPOSITORY `repo_name`;
说明：
-- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 broker 和 LOCATION
映射到该仓库。
+- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 LOCATION 映射到该仓库。
### Example
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/data-admin/backup.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/data-admin/backup.md
index ae6c965fdd..f2aa5efd80 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/data-admin/backup.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/data-admin/backup.md
@@ -24,15 +24,13 @@ specific language governing permissions and limitations
under the License.
-->
+# 数据备份
-
-Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存储系统中。之后可以通过 恢复 命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期的进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移。
+Doris 支持将当前数据以文件的形式备份到 HDFS 和对象存储。之后可以通过恢复命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期地进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移，集群间无损迁移可以使用 CCR
(ccr.md)。
该功能需要 Doris 版本 0.8.2+
-使用该功能，需要部署对应远端存储的 broker。如 BOS、HDFS 等。可以通过 `SHOW BROKER;` 查看当前部署的 broker。
-
-## 简要原理说明
+## 原理说明
备份操作是将指定表或分区的数据，直接以 Doris 存储的文件的形式，上传到远端仓库中进行存储。当用户提交 Backup 请求后，系统内部会做如下操作：
@@ -56,20 +54,20 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
## 开始备份
-1. 创建一个 hdfs 的远程仓库 example_repo：
+1. 创建一个 HDFS 的远程仓库 example_repo（S3 存储请参考 2）：
```sql
CREATE REPOSITORY `example_repo`
- WITH BROKER `hdfs_broker`
+ WITH HDFS
ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
PROPERTIES
(
- "username" = "user",
- "password" = "password"
+ "fs.defaultFS"="hdfs://hdfs_host:port",
+ "hadoop.username" = "hadoop"
);
```
-2. 创建一个 s3 的远程仓库 : s3_repo
+2. 创建一个 s3 的远程仓库 : s3_repo（HDFS 存储请参考 1）
```
CREATE REPOSITORY `s3_repo`
@@ -83,11 +81,10 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
"AWS_REGION" = "xxx"
);
```
- :::caution
- 注意：
-
- ON LOCATION 这里后面跟的是 Bucket Name
- :::
+
+ >注意：
+ >
+ >ON LOCATION 这里后面跟的是 Bucket Name
2. 全量备份 example_db 下的表 example_tbl 到仓库 example_repo 中：
@@ -144,7 +141,7 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
1 row in set (0.15 sec)
```
-BACKUP 的更多用法可参考
[这里](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP)。
+BACKUP 的更多用法可参考
[这里](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)。
## 最佳实践
@@ -158,106 +155,42 @@ BACKUP 的更多用法可参考
[这里](../../sql-manual/sql-reference/Data-Def
建议在迁移完成后，对新旧两个集群并行导入一段时间。完成数据和业务正确性校验后，再将业务迁移到新的集群。
-## 重点说明
+## 说明
1. 备份恢复相关的操作目前只允许拥有 ADMIN 权限的用户执行。
-
2. 一个 Database 内，只允许有一个正在执行的备份或恢复作业。
-
3. 备份和恢复都支持最小分区（Partition）级别的操作，当表的数据量很大时，建议按分区分别执行，以降低失败重试的代价。
-
4.
因为备份恢复操作，操作的都是实际的数据文件。所以当一个表的分片过多，或者一个分片有过多的小版本时，可能即使总数据量很小，依然需要备份或恢复很长时间。用户可以通过
`SHOW PARTITIONS FROM table_name;` 和 `SHOW TABLETS FROM table_name;`
来查看各个分区的分片数量，以及各个分片的文件版本数量，来预估作业执行时间。文件数量对作业执行的时间影响非常大，所以建议在建表时，合理规划分区分桶，以避免过多的分片。
-
5. 当通过 `SHOW BACKUP` 或者 `SHOW RESTORE` 命令查看作业状态时。有可能会在 `TaskErrMsg`
一列中看到错误信息。但只要 `State` 列不为 `CANCELLED`，则说明作业依然在继续。这些 Task 有可能会重试成功。当然，有些 Task
错误，也会直接导致作业失败。
-
常见的`TaskErrMsg`错误如下：
Q1：备份到 HDFS，状态显示 UPLOADING，TaskErrMsg 错误信息：[13333: Close broker writer
failed, broker:TNetworkAddress(hostname=10.10.0.0,port=8000) msg:errors while
close file output stream, cause by: DataStreamer Exception: ]
这个一般是网络通信问题，查看broker日志，看某个ip
或者端口不通，如果是云服务，则需要查看是否访问了内网，如果是，则可以在borker/conf文件夹下添加hdfs-site.xml，还需在hdfs-site.xml配置文件下添加dfs.client.use.datanode.hostname=true，并在broker节点上配置HADOOP集群的主机名映射。
-
6. 如果恢复作业是一次覆盖操作（指定恢复数据到已经存在的表或分区中），那么从恢复作业的 `COMMIT`
阶段开始，当前集群上被覆盖的数据有可能不能再被还原。此时如果恢复作业失败或被取消，有可能造成之前的数据已损坏且无法访问。这种情况下，只能通过再次执行恢复操作，并等待作业完成。因此，我们建议，如无必要，尽量不要使用覆盖的方式恢复数据，除非确认当前数据已不再使用。
## 相关命令
和备份恢复功能相关的命令如下。以下命令，都可以通过 mysql-client 连接 Doris 后，使用 `help cmd;` 的方式查看详细帮助。
-**1. CREATE REPOSITORY**
-
-创建一个远端仓库路径，用于备份或恢复。该命令需要借助 Broker 进程访问远端存储，不同的 Broker 需要提供不同的参数，具体请参阅 [Broker
文档](../../data-operate/import/broker-load-manual#其他-broker-导入)，也可以直接通过 S3
协议备份到支持 AWS S3 协议的远程存储上去，也可以直接备份到 HDFS，具体参考
[创建远程仓库文档](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY)
-
-**2. BACKUP**
-
-执行一次备份操作。
-
-**3. SHOW BACKUP**
-
-查看最近一次 backup 作业的执行情况，包括：
-
-- JobId：本次备份作业的 id。
-
-- SnapshotName：用户指定的本次备份作业的名称（Label）。
-
-- DbName：备份作业对应的 Database。
-
-- State：备份作业当前所在阶段：
-
- - PENDING：作业初始状态。
-
- - SNAPSHOTING：正在进行快照操作。
-
- - UPLOAD_SNAPSHOT：快照结束，准备上传。
-
- - UPLOADING：正在上传快照。
-
- - SAVE_META：正在本地生成元数据文件。
-
- - UPLOAD_INFO：上传元数据文件和本次备份作业的信息。
-
- - FINISHED：备份完成。
-
- - CANCELLED：备份失败或被取消。
-
-- BackupObjs：本次备份涉及的表和分区的清单。
-
-- CreateTime：作业创建时间。
-
-- SnapshotFinishedTime：快照完成时间。
-
-- UploadFinishedTime：快照上传完成时间。
-
-- FinishedTime：本次作业完成时间。
-
-- UnfinishedTasks：在 `SNAPSHOTTING`，`UPLOADING`
等阶段，会有多个子任务在同时进行，这里展示的当前阶段，未完成的子任务的 task id。
-
-- TaskErrMsg：如果有子任务执行出错，这里会显示对应子任务的错误信息。
-
-- Status：用于记录在整个作业过程中，可能出现的一些状态信息。
-
-- Timeout：作业的超时时间，单位是秒。
-
-**4. SHOW SNAPSHOT**
-
-查看远端仓库中已存在的备份。
-
-- Snapshot：备份时指定的该备份的名称（Label）。
+1. CREATE REPOSITORY
-- Timestamp：备份的时间戳。
+ 创建一个远端仓库路径，用于备份或恢复。具体参考
[创建远程仓库文档](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md)。
-- Status：该备份是否正常。
+2. BACKUP
-如果在 `SHOW SNAPSHOT` 后指定了 where 子句，则可以显示更详细的备份信息。
+ 执行一次备份操作。具体参考 [备份文档]
(../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)。
-- Database：备份时对应的 Database。
+3. SHOW BACKUP
-- Details：展示了该备份完整的数据目录结构。
+ 查看最近一次 backup 作业的执行情况。具体参考
[查看备份作业文档](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
-**5. CANCEL BACKUP**
+4. SHOW SNAPSHOT
-取消当前正在执行的备份作业。
+ 查看远端仓库中已存在的备份。具体参考
[查看备份文档](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md)。
-**6. DROP REPOSITORY**
+5. CANCEL BACKUP
-删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。
+ 取消当前正在执行的备份作业。具体参考 [取消备份作业文档]
(../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md)。
-## 更多帮助
+6. DROP REPOSITORY
- 关于 BACKUP 使用的更多详细语法及最佳实践，请参阅
[BACKUP](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP)
命令手册，你也可以在 MySql 客户端命令行下输入 `HELP BACKUP` 获取更多帮助信息。
+ 删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。具体参考 [删除远程仓库文档]
(../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md)。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index 97368d347b..4840f01757 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -90,6 +90,13 @@ TO example_repo
EXCLUDE (example_tbl);
```
+4. 全量备份 example_db 下的表到仓库 example_repo 中：
+
+```sql
+BACKUP SNAPSHOT example_db.snapshot_label3
+TO example_repo;
+```
+
### Keywords
```text
@@ -100,7 +107,7 @@ BACKUP
1. 同一个数据库下只能进行一个备份操作。
-2. 备份操作会备份指定表或分区的基础表及
[物化视图](../../../../query/view-materialized-view/materialized-view)，并且仅备份一副本。
+2.
备份操作会备份指定表或分区的基础表及同步物化视图[物化视图](../../../../query/view-materialized-view/materialized-view.md)，并且仅备份一副本，异步物化视图(../../../../query/view-materialized-view/async-materialized-view.md)当前未支持。
3. 备份操作的效率
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index 972f0b7f6d..49a97bc15f 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,20 @@ CREATE REPOSITORY
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
说明：
-- 仓库的创建，依赖于已存在的 broker 或者直接通过AWS s3 协议访问云存储，或者直接访问HDFS
- 如果是只读仓库，则只能在仓库上进行恢复。如果不是，则可以进行备份和恢复操作。
-- 根据 broker 或者S3、hdfs的不同类型，PROPERTIES 有所不同，具体见示例。
+- 根据 S3、HDFS 的不同类型，PROPERTIES 有所不同，具体见示例。
- ON LOCATION ,如果是 S3 , 这里后面跟的是 Bucket Name。
### Example
-1. 创建名为 bos_repo 的仓库，依赖 BOS broker "bos_broker"，数据根目录为：bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. 创建和示例 1 相同的仓库，但属性为只读：
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. 创建名为 hdfs_repo 的仓库，依赖 Baidu hdfs broker
"hdfs_broker"，数据根目录为：hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. 创建名为 s3_repo 的仓库，直接链接云存储，而不通过broker.
+1. 创建名为 s3_repo 的仓库.
```sql
CREATE REPOSITORY `s3_repo`
@@ -108,7 +66,7 @@ PROPERTIES
);
```
-5. 创建名为 hdfs_repo 的仓库，直接链接HDFS，而不通过broker.
+2. 创建名为 hdfs_repo 的仓库.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -121,7 +79,7 @@ PROPERTIES
);
```
-6. 创建名为 minio_repo 的仓库，直接通过 s3 协议链接 minio.
+3. 创建名为 minio_repo 的仓库.
```sql
CREATE REPOSITORY `minio_repo`
@@ -137,9 +95,7 @@ PROPERTIES
);
```
-7. 使用临时秘钥创建名为 minio_repo 的仓库
-
-<version since="1.2"></version>
+4. 使用临时秘钥创建名为 minio_repo 的仓库
```
CREATE REPOSITORY `minio_repo`
@@ -155,7 +111,7 @@ PROPERTIES
)
```
-8. 使用腾讯云 COS 创建仓库
+5. 使用腾讯云 COS 创建仓库
```
CREATE REPOSITORY `cos_repo`
@@ -170,7 +126,7 @@ PROPERTIES
);
```
-9. 创建仓库并删除已经存在的 snapshot
+6. 创建仓库并删除已经存在的 snapshot
```sql
CREATE REPOSITORY `s3_repo`
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 81854d0ef1..01ea970885 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,7 +42,7 @@ DROP REPOSITORY `repo_name`;
说明：
-- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 broker 和 LOCATION
映射到该仓库。
+- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 LOCATION 映射到该仓库。
### Example
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/data-admin/backup.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/data-admin/backup.md
index 819ca68924..fef61e9763 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/data-admin/backup.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/data-admin/backup.md
@@ -26,13 +26,11 @@ under the License.
# 数据备份
-Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存储系统中。之后可以通过 恢复 命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期的进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移。
+Doris 支持将当前数据以文件的形式备份到 HDFS 和对象存储。之后可以通过恢复命令，从远端存储系统中将数据恢复到任意 Doris
集群。通过这个功能，Doris 可以支持将数据定期地进行快照备份。也可以通过这个功能，在不同集群间进行数据迁移，集群间无损迁移可以使用 CCR
(../ccr.md)。
该功能需要 Doris 版本 0.8.2+
-使用该功能，需要部署对应远端存储的 broker。如 BOS、HDFS 等。可以通过 `SHOW BROKER;` 查看当前部署的 broker。
-
-## 简要原理说明
+## 原理说明
备份操作是将指定表或分区的数据，直接以 Doris 存储的文件的形式，上传到远端仓库中进行存储。当用户提交 Backup 请求后，系统内部会做如下操作：
@@ -56,9 +54,8 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
## 开始备份
-1. 创建一个 hdfs 的远程仓库 example_repo：
+1. 创建一个 HDFS 的远程仓库 example_repo（S3 存储请参考 2）：
- **WITH HDFS(推荐使用)**
```sql
CREATE REPOSITORY `example_repo`
WITH HDFS
@@ -70,22 +67,7 @@ Doris 支持将当前数据以文件的形式，通过 broker 备份到远端存
);
```
- **WITH BROKER**
-
- 需要先启动一个 BROKER 进程。
-
- ```sql
- CREATE REPOSITORY `example_repo`
- WITH BROKER `broker_name`
- ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
- PROPERTIES
- (
- "username" = "user",
- "password" = "password"
- );
- ```
-
-2. 创建一个 s3 的远程仓库 : s3_repo
+2. 创建一个 S3 的远程仓库 : s3_repo（HDFS 存储请参考 1）
```
CREATE REPOSITORY `s3_repo`
@@ -173,7 +155,7 @@ BACKUP 的更多用法可参考 [这里](../../sql-manual/sql-statements/Data-De
建议在迁移完成后，对新旧两个集群并行导入一段时间。完成数据和业务正确性校验后，再将业务迁移到新的集群。
-## 重点说明
+## 说明
1. 备份恢复相关的操作目前只允许拥有 ADMIN 权限的用户执行。
2. 一个 Database 内，只允许有一个正在执行的备份或恢复作业。
@@ -191,59 +173,24 @@ BACKUP 的更多用法可参考
[这里](../../sql-manual/sql-statements/Data-De
1. CREATE REPOSITORY
- 创建一个远端仓库路径，用于备份或恢复。该命令需要借助 Broker 进程访问远端存储，不同的 Broker 需要提供不同的参数，具体请参阅
[Broker 文档](../../data-operate/import/broker-load-manual#其他-broker-导入)，也可以直接通过
S3 协议备份到支持 AWS S3 协议的远程存储上去，也可以直接备份到 HDFS，具体参考
[创建远程仓库文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md)
+ 创建一个远端仓库路径，用于备份或恢复。具体参考
[创建远程仓库文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md)。
2. BACKUP
- 执行一次备份操作。
+ 执行一次备份操作。具体参考 [备份文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)。
3. SHOW BACKUP
- 查看最近一次 backup 作业的执行情况，包括：
-
- - JobId：本次备份作业的 id。
- - SnapshotName：用户指定的本次备份作业的名称（Label）。
- - DbName：备份作业对应的 Database。
- - State：备份作业当前所在阶段：
- - PENDING：作业初始状态。
- - SNAPSHOTING：正在进行快照操作。
- - UPLOAD_SNAPSHOT：快照结束，准备上传。
- - UPLOADING：正在上传快照。
- - SAVE_META：正在本地生成元数据文件。
- - UPLOAD_INFO：上传元数据文件和本次备份作业的信息。
- - FINISHED：备份完成。
- - CANCELLED：备份失败或被取消。
- - BackupObjs：本次备份涉及的表和分区的清单。
- - CreateTime：作业创建时间。
- - SnapshotFinishedTime：快照完成时间。
- - UploadFinishedTime：快照上传完成时间。
- - FinishedTime：本次作业完成时间。
- - UnfinishedTasks：在 `SNAPSHOTTING`，`UPLOADING`
等阶段，会有多个子任务在同时进行，这里展示的当前阶段，未完成的子任务的 task id。
- - TaskErrMsg：如果有子任务执行出错，这里会显示对应子任务的错误信息。
- - Status：用于记录在整个作业过程中，可能出现的一些状态信息。
- - Timeout：作业的超时时间，单位是秒。
+ 查看最近一次 backup 作业的执行情况。具体参考
[查看备份作业文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
4. SHOW SNAPSHOT
- 查看远端仓库中已存在的备份。
-
- - Snapshot：备份时指定的该备份的名称（Label）。
- - Timestamp：备份的时间戳。
- - Status：该备份是否正常。
-
- 如果在 `SHOW SNAPSHOT` 后指定了 where 子句，则可以显示更详细的备份信息。
-
- - Database：备份时对应的 Database。
- - Details：展示了该备份完整的数据目录结构。
+ 查看远端仓库中已存在的备份。具体参考
[查看备份文档](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md)。
5. CANCEL BACKUP
- 取消当前正在执行的备份作业。
+ 取消当前正在执行的备份作业。具体参考 [取消备份作业文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md)。
6. DROP REPOSITORY
- 删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。
-
-## 更多帮助
-
- 关于 BACKUP 使用的更多详细语法及最佳实践，请参阅
[BACKUP](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)
命令手册，你也可以在 MySql 客户端命令行下输入 `HELP BACKUP` 获取更多帮助信息。
+ 删除已创建的远端仓库。删除仓库，仅仅是删除该仓库在 Doris 中的映射，不会删除实际的仓库数据。具体参考 [删除远程仓库文档]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md)。
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index 82081e5537..4840f01757 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -107,7 +107,7 @@ BACKUP
1. 同一个数据库下只能进行一个备份操作。
-2. 备份操作会备份指定表或分区的基础表及
[物化视图](../../../../query-acceleration/materialized-view.md)，并且仅备份一副本。
+2.
备份操作会备份指定表或分区的基础表及同步物化视图[物化视图](../../../../query/view-materialized-view/materialized-view.md)，并且仅备份一副本，异步物化视图(../../../../query/view-materialized-view/async-materialized-view.md)当前未支持。
3. 备份操作的效率
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index 972f0b7f6d..49a97bc15f 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,20 @@ CREATE REPOSITORY
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
说明：
-- 仓库的创建，依赖于已存在的 broker 或者直接通过AWS s3 协议访问云存储，或者直接访问HDFS
- 如果是只读仓库，则只能在仓库上进行恢复。如果不是，则可以进行备份和恢复操作。
-- 根据 broker 或者S3、hdfs的不同类型，PROPERTIES 有所不同，具体见示例。
+- 根据 S3、HDFS 的不同类型，PROPERTIES 有所不同，具体见示例。
- ON LOCATION ,如果是 S3 , 这里后面跟的是 Bucket Name。
### Example
-1. 创建名为 bos_repo 的仓库，依赖 BOS broker "bos_broker"，数据根目录为：bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. 创建和示例 1 相同的仓库，但属性为只读：
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. 创建名为 hdfs_repo 的仓库，依赖 Baidu hdfs broker
"hdfs_broker"，数据根目录为：hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. 创建名为 s3_repo 的仓库，直接链接云存储，而不通过broker.
+1. 创建名为 s3_repo 的仓库.
```sql
CREATE REPOSITORY `s3_repo`
@@ -108,7 +66,7 @@ PROPERTIES
);
```
-5. 创建名为 hdfs_repo 的仓库，直接链接HDFS，而不通过broker.
+2. 创建名为 hdfs_repo 的仓库.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -121,7 +79,7 @@ PROPERTIES
);
```
-6. 创建名为 minio_repo 的仓库，直接通过 s3 协议链接 minio.
+3. 创建名为 minio_repo 的仓库.
```sql
CREATE REPOSITORY `minio_repo`
@@ -137,9 +95,7 @@ PROPERTIES
);
```
-7. 使用临时秘钥创建名为 minio_repo 的仓库
-
-<version since="1.2"></version>
+4. 使用临时秘钥创建名为 minio_repo 的仓库
```
CREATE REPOSITORY `minio_repo`
@@ -155,7 +111,7 @@ PROPERTIES
)
```
-8. 使用腾讯云 COS 创建仓库
+5. 使用腾讯云 COS 创建仓库
```
CREATE REPOSITORY `cos_repo`
@@ -170,7 +126,7 @@ PROPERTIES
);
```
-9. 创建仓库并删除已经存在的 snapshot
+6. 创建仓库并删除已经存在的 snapshot
```sql
CREATE REPOSITORY `s3_repo`
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 81854d0ef1..01ea970885 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,7 +42,7 @@ DROP REPOSITORY `repo_name`;
说明：
-- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 broker 和 LOCATION
映射到该仓库。
+- 删除仓库，仅仅是删除该仓库在 Palo 中的映射，不会删除实际的仓库数据。删除后，可以再次通过指定相同的 LOCATION 映射到该仓库。
### Example
diff --git a/versioned_docs/version-2.0/admin-manual/data-admin/backup.md
b/versioned_docs/version-2.0/admin-manual/data-admin/backup.md
index aba6785fea..820e0831f5 100644
--- a/versioned_docs/version-2.0/admin-manual/data-admin/backup.md
+++ b/versioned_docs/version-2.0/admin-manual/data-admin/backup.md
@@ -26,12 +26,10 @@ under the License.
# Data Backup
-Doris supports backing up the current data in the form of files to the remote
storage system through the broker. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
+Doris supports backing up the current data in the form of files to the remote
storage system like S3 and HDFS. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
This feature requires Doris version 0.8.2+
-To use this function, you need to deploy the broker corresponding to the
remote storage. Such as BOS, HDFS, etc. You can view the currently deployed
broker through `SHOW BROKER;`.
-
## A brief explanation of the principle
The backup operation is to upload the data of the specified table or partition
directly to the remote warehouse for storage in the form of files stored by
Doris. When a user submits a Backup request, the system will perform the
following operations:
@@ -56,20 +54,20 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
## Start Backup
-1. Create a hdfs remote warehouse example_repo:
+1. Create a hdfs remote warehouse example_repo (S3 skips step 1):
```sql
CREATE REPOSITORY `example_repo`
- WITH BROKER `hdfs_broker`
+ WITH HDFS
ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
PROPERTIES
(
- "username" = "user",
- "password" = "password"
+ "fs.defaultFS"="hdfs://hdfs_host:port",
+ "hadoop.username" = "hadoop"
);
```
-2. Create a remote repository for s3 : s3_repo
+2. Create a remote repository for s3 : s3_repo (HDFS skips step 2)
```
CREATE REPOSITORY `s3_repo`
@@ -88,7 +86,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
>
>ON LOCATION is followed by Bucket Name here
-1. Full backup of table example_tbl under example_db to warehouse example_repo:
+3. Full backup of table example_tbl under example_db to warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label1
@@ -97,7 +95,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
PROPERTIES ("type" = "full");
```
-2. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
+4. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label2
@@ -109,7 +107,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
);
```
-4. View the execution of the most recent backup job:
+5. View the execution of the most recent backup job:
```sql
mysql> show BACKUP\G;
@@ -131,7 +129,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
1 row in set (0.01 sec)
```
-5. View existing backups in remote repositories:
+6. View existing backups in remote repositories:
```sql
mysql> SHOW SNAPSHOT ON example_repo WHERE SNAPSHOT = "snapshot_label1";
@@ -143,7 +141,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
1 row in set (0.15 sec)
```
-For the detailed usage of BACKUP, please refer to
[here](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP).
+For the detailed usage of BACKUP, please refer to
[here](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md).
## Best Practices
@@ -164,10 +162,7 @@ It is recommended to import the new and old clusters in
parallel for a period of
3. Both backup and recovery support operations at the minimum partition
(Partition) level. When the amount of data in the table is large, it is
recommended to perform operations by partition to reduce the cost of failed
retry.
4. Because of the backup and restore operations, the operations are the actual
data files. Therefore, when a table has too many shards, or a shard has too
many small versions, it may take a long time to backup or restore even if the
total amount of data is small. Users can use `SHOW PARTITIONS FROM table_name;`
and `SHOW TABLETS FROM table_name;` to view the number of shards in each
partition and the number of file versions in each shard to estimate job
execution time. The number of file [...]
5. When checking job status via `SHOW BACKUP` or `SHOW RESTORE` command. It is
possible to see error messages in the `TaskErrMsg` column. But as long as the
`State` column is not `CANCELLED`, the job is still continuing. These tasks may
retry successfully. Of course, some Task errors will also directly cause the
job to fail.
- Common `TaskErrMsg` errors are as follows:
- Q1: Backup to HDFS, the status shows UPLOADING, TaskErrMsg error
message: [13333: Close broker writer failed,
broker:TNetworkAddress(hostname=10.10.0.0, port=8000) msg:errors while close
file output stream, cause by: DataStreamer Exception : ]
- This is generally a network communication problem. Check the broker log
to see if a certain ip or port is blocked. If it is a cloud service, you need
to check whether is accessed the intranet. If so, you can add hdfs-site.xml in
the broker/conf folder, you need to add dfs.client.use.datanode.hostname=true
under the hdfs-site.xml configuration file, and configure the hostname mapping
of the HADOOP cluster on the broker node.
-7. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
+6. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
## Related Commands
@@ -175,59 +170,28 @@ It is recommended to import the new and old clusters in
parallel for a period of
1. CREATE REPOSITORY
- Create a remote repository path for backup or restore. This command
needs to use the Broker process to access the remote storage. Different brokers
need to provide different parameters. For details, please refer to [Broker
documentation](../../data-operate/import/broker-load-manual), or you can
directly back up to support through the S3 protocol For the remote storage of
AWS S3 protocol, or directly back up to HDFS, please refer to [Create Remote
Warehouse Documentation](../../sql- [...]
+ Create a remote repository path for backup or restore. Please refer to
[Create Repository
Reference](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md).
2. BACKUP
- Perform a backup operation.
+ Perform a backup operation. Please refer to [Backup
Reference](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md).
3. SHOW BACKUP
- View the execution of the most recent backup job, including:
-
- - JobId: The id of this backup job.
- - SnapshotName: The name (Label) of this backup job specified by the
user.
- - DbName: Database corresponding to the backup job.
- - State: The current stage of the backup job:
- - PENDING: The initial status of the job.
- - SNAPSHOTING: A snapshot operation is in progress.
- - UPLOAD_SNAPSHOT: The snapshot is over, ready to upload.
- - UPLOADING: Uploading snapshot.
- - SAVE_META: The metadata file is being generated locally.
- - UPLOAD_INFO: Upload metadata files and information about this backup
job.
- - FINISHED: The backup is complete.
- - CANCELLED: Backup failed or was canceled.
- - BackupObjs: List of tables and partitions involved in this backup.
- - CreateTime: Job creation time.
- - SnapshotFinishedTime: Snapshot completion time.
- - UploadFinishedTime: Snapshot upload completion time.
- - FinishedTime: The completion time of this job.
- - UnfinishedTasks: During `SNAPSHOTTING`, `UPLOADING` and other stages,
there will be multiple subtasks going on at the same time. The current stage
shown here is the task id of the unfinished subtasks.
- - TaskErrMsg: If there is an error in the execution of a subtask, the
error message of the corresponding subtask will be displayed here.
- - Status: Used to record some status information that may appear during
the entire job process.
- - Timeout: The timeout period of the job, in seconds.
+ View the execution of the most recent backup job. Please refer to [Show
Backup
Reference](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
4. SHOW SNAPSHOT
- View existing backups in the remote repository.
-
- - Snapshot: The name (Label) of the backup specified during backup.
- - Timestamp: Timestamp of the backup.
- - Status: Whether the backup is normal.
-
- More detailed backup information can be displayed if a where clause is
specified after `SHOW SNAPSHOT`.
-
- - Database: The database corresponding to the backup.
- - Details: Shows the complete data directory structure of the backup.
+ View existing backups in the remote repository. Please refer to [Show
Snapshot
Reference](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md).
5. CANCEL BACKUP
- Cancel the currently executing backup job.
+ Cancel the currently executing backup job. Please refer to [Cancel
Backup Reference]
(../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md).
6. DROP REPOSITORY
- Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data.
+ Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data. Please refer to [Drop Repository Reference]
(../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md).
## More Help
- For more detailed syntax and best practices used by BACKUP, please refer to
the
[BACKUP](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP)
command manual, You can also type `HELP BACKUP` on the MySql client command
line for more help.
+ For more detailed syntax and best practices used by BACKUP, please refer to
the
[BACKUP](../../sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md)
command manual, You can also type `HELP BACKUP` on the MySql client command
line for more help.
diff --git
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index 39984d8e59..f6ea2a1a26 100644
---
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -88,6 +88,13 @@ TO example_repo
EXCLUDE (example_tbl);
```
+4. Fully back up tables under example_db to the repository example_repo:
+
+```sql
+BACKUP SNAPSHOT example_db.snapshot_label3
+TO example_repo;
+```
+
### Keywords
BACKUP
@@ -96,7 +103,7 @@ BACKUP
1. Only one backup operation can be performed under the same database.
-2. The backup operation will back up the underlying table and [materialized
view](../../../../query/view-materialized-view/materialized-view) of the
specified table or partition, and only one copy will be backed up.
+2. The backup operation will back up the underlying table and [Synchronous
materialized
view](../../../../query/view-materialized-view/materialized-view.md) of the
specified table or partition, and only one replica will be backed up.
[Asynchronous materialized
view](../../../../query/view-materialized-view/async-materialized-view.md) is
not supported.
3. Efficiency of backup operations
diff --git
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index f5571c4c44..144fab4839 100644
---
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,21 @@ grammar:
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
illustrate:
-- Creation of repositories, relying on existing brokers or accessing cloud
storage directly through AWS s3 protocol, or accessing HDFS directly.
+- Creation of repositories, accessing cloud storage directly through AWS s3
protocol, or accessing HDFS directly.
- If it is a read-only repository, restores can only be done on the
repository. If not, backup and restore operations are available.
-- PROPERTIES are different according to different types of broker or S3 or
hdfs, see the example for details.
-- ON LOCATION ： if it is S3 , here followed by the Bucket Name.
+- PROPERTIES are different according to different types of S3 or hdfs, see the
example for details.
+- ON LOCATION : if it is S3 , here followed by the Bucket Name.
### Example
-1. Create a warehouse named bos_repo, rely on BOS broker "bos_broker", and the
data root directory is: bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. Create the same repository as Example 1, but with read-only properties:
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. Create a warehouse named hdfs_repo, rely on Baidu hdfs broker
"hdfs_broker", the data root directory is:
hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. Create a repository named s3_repo to link cloud storage directly without
going through the broker.
+1. Create a repository named s3_repo.
```sql
CREATE REPOSITORY `s3_repo`
@@ -109,7 +68,7 @@ PROPERTIES
);
```
-5. Create a repository named hdfs_repo to link HDFS directly without going
through the broker.
+2. Create a repository named hdfs_repo.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -143,8 +102,6 @@ PROPERTIES
7. Create a repository named minio_repo via temporary security credentials.
-<version since="1.2"></version>
-
```
CREATE REPOSITORY `minio_repo`
WITH S3
@@ -159,7 +116,7 @@ PROPERTIES
)
```
-8. Create repository using Tencent COS
+1. Create repository using Tencent COS
```
CREATE REPOSITORY `cos_repo`
diff --git
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 6a7e6e5a66..7a0b937cdc 100644
---
a/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/versioned_docs/version-2.0/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,14 +42,14 @@ DROP REPOSITORY `repo_name`;
illustrate:
-- Deleting a warehouse just deletes the warehouse's mapping in Palo, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same broker and LOCATION.
+- Deleting a warehouse just deletes the warehouse's mapping in Doris, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same LOCATION.
### Example
-1. Delete the repository named bos_repo:
+1. Delete the repository named example_repo:
```sql
-DROP REPOSITORY `bos_repo`;
+DROP REPOSITORY `example_repo`;
```
### Keywords
diff --git a/versioned_docs/version-2.1/admin-manual/data-admin/backup.md
b/versioned_docs/version-2.1/admin-manual/data-admin/backup.md
index 881b23c1c8..3021d0cab0 100644
--- a/versioned_docs/version-2.1/admin-manual/data-admin/backup.md
+++ b/versioned_docs/version-2.1/admin-manual/data-admin/backup.md
@@ -26,12 +26,10 @@ under the License.
# Data Backup
-Doris supports backing up the current data in the form of files to the remote
storage system through the broker. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
+Doris supports backing up the current data in the form of files to the remote
storage system like S3 and HDFS. Afterwards, you can restore data from the
remote storage system to any Doris cluster through the restore command. Through
this function, Doris can support periodic snapshot backup of data. You can also
use this function to migrate data between different clusters.
This feature requires Doris version 0.8.2+
-To use this function, you need to deploy the broker corresponding to the
remote storage. Such as BOS, HDFS, etc. You can view the currently deployed
broker through `SHOW BROKER;`.
-
## A brief explanation of the principle
The backup operation is to upload the data of the specified table or partition
directly to the remote warehouse for storage in the form of files stored by
Doris. When a user submits a Backup request, the system will perform the
following operations:
@@ -56,9 +54,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
## Start Backup
-1. Create a hdfs remote warehouse example_repo:
-
- **WITH HDFS (Recommended)**
+1. Create a hdfs remote warehouse example_repo (S3 skips step 1):
```sql
CREATE REPOSITORY `example_repo`
@@ -71,22 +67,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
);
```
- **WITH BROKER**
-
- This requires starting a broker process first.
-
- ```sql
- CREATE REPOSITORY `example_repo`
- WITH BROKER `broker_name`
- ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
- PROPERTIES
- (
- "username" = "user",
- "password" = "password"
- );
- ```
-
-2. Create a remote repository for s3 : s3_repo
+2. Create a remote repository for s3 : s3_repo (HDFS skips step 2)
```
CREATE REPOSITORY `s3_repo`
@@ -105,7 +86,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
>
>ON LOCATION is followed by Bucket Name here
-1. Full backup of table example_tbl under example_db to warehouse example_repo:
+3. Full backup of table example_tbl under example_db to warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label1
@@ -114,7 +95,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
PROPERTIES ("type" = "full");
```
-2. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
+4. Under the full backup example_db, the p1, p2 partitions of the table
example_tbl, and the table example_tbl2 to the warehouse example_repo:
```sql
BACKUP SNAPSHOT example_db.snapshot_label2
@@ -126,7 +107,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
);
```
-4. View the execution of the most recent backup job:
+5. View the execution of the most recent backup job:
```sql
mysql> show BACKUP\G;
@@ -148,7 +129,7 @@ ALTER TABLE tbl1 SET ("dynamic_partition.enable"="true")
1 row in set (0.01 sec)
```
-5. View existing backups in remote repositories:
+6. View existing backups in remote repositories:
```sql
mysql> SHOW SNAPSHOT ON example_repo WHERE SNAPSHOT = "snapshot_label1";
@@ -181,10 +162,7 @@ It is recommended to import the new and old clusters in
parallel for a period of
3. Both backup and recovery support operations at the minimum partition
(Partition) level. When the amount of data in the table is large, it is
recommended to perform operations by partition to reduce the cost of failed
retry.
4. Because of the backup and restore operations, the operations are the actual
data files. Therefore, when a table has too many shards, or a shard has too
many small versions, it may take a long time to backup or restore even if the
total amount of data is small. Users can use `SHOW PARTITIONS FROM table_name;`
and `SHOW TABLETS FROM table_name;` to view the number of shards in each
partition and the number of file versions in each shard to estimate job
execution time. The number of file [...]
5. When checking job status via `SHOW BACKUP` or `SHOW RESTORE` command. It is
possible to see error messages in the `TaskErrMsg` column. But as long as the
`State` column is not `CANCELLED`, the job is still continuing. These tasks may
retry successfully. Of course, some Task errors will also directly cause the
job to fail.
- Common `TaskErrMsg` errors are as follows:
- Q1: Backup to HDFS, the status shows UPLOADING, TaskErrMsg error
message: [13333: Close broker writer failed,
broker:TNetworkAddress(hostname=10.10.0.0, port=8000) msg:errors while close
file output stream, cause by: DataStreamer Exception : ]
- This is generally a network communication problem. Check the broker log
to see if a certain ip or port is blocked. If it is a cloud service, you need
to check whether is accessed the intranet. If so, you can add hdfs-site.xml in
the broker/conf folder, you need to add dfs.client.use.datanode.hostname=true
under the hdfs-site.xml configuration file, and configure the hostname mapping
of the HADOOP cluster on the broker node.
-7. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
+6. If the recovery job is an overwrite operation (specifying the recovery data
to an existing table or partition), then from the `COMMIT` phase of the
recovery job, the overwritten data on the current cluster may no longer be
restored. If the restore job fails or is canceled at this time, the previous
data may be damaged and inaccessible. In this case, the only way to do it is to
perform the recovery operation again and wait for the job to complete.
Therefore, we recommend that if unnece [...]
## Related Commands
@@ -192,58 +170,27 @@ It is recommended to import the new and old clusters in
parallel for a period of
1. CREATE REPOSITORY
- Create a remote repository path for backup or restore. This command
needs to use the Broker process to access the remote storage. Different brokers
need to provide different parameters. For details, please refer to [Broker
documentation](../../data-operate/import/broker-load-manual), or you can
directly back up to support through the S3 protocol For the remote storage of
AWS S3 protocol, or directly back up to HDFS, please refer to [Create Remote
Warehouse Documentation](../../sql- [...]
+ Create a remote repository path for backup or restore. Please refer to
[Create Repository
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md).
2. BACKUP
- Perform a backup operation.
+ Perform a backup operation. Please refer to [Backup
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md).
3. SHOW BACKUP
- View the execution of the most recent backup job, including:
-
- - JobId: The id of this backup job.
- - SnapshotName: The name (Label) of this backup job specified by the
user.
- - DbName: Database corresponding to the backup job.
- - State: The current stage of the backup job:
- - PENDING: The initial status of the job.
- - SNAPSHOTING: A snapshot operation is in progress.
- - UPLOAD_SNAPSHOT: The snapshot is over, ready to upload.
- - UPLOADING: Uploading snapshot.
- - SAVE_META: The metadata file is being generated locally.
- - UPLOAD_INFO: Upload metadata files and information about this backup
job.
- - FINISHED: The backup is complete.
- - CANCELLED: Backup failed or was canceled.
- - BackupObjs: List of tables and partitions involved in this backup.
- - CreateTime: Job creation time.
- - SnapshotFinishedTime: Snapshot completion time.
- - UploadFinishedTime: Snapshot upload completion time.
- - FinishedTime: The completion time of this job.
- - UnfinishedTasks: During `SNAPSHOTTING`, `UPLOADING` and other stages,
there will be multiple subtasks going on at the same time. The current stage
shown here is the task id of the unfinished subtasks.
- - TaskErrMsg: If there is an error in the execution of a subtask, the
error message of the corresponding subtask will be displayed here.
- - Status: Used to record some status information that may appear during
the entire job process.
- - Timeout: The timeout period of the job, in seconds.
+ View the execution of the most recent backup job. Please refer to [Show
Backup
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-BACKUP.md)。
4. SHOW SNAPSHOT
- View existing backups in the remote repository.
-
- - Snapshot: The name (Label) of the backup specified during backup.
- - Timestamp: Timestamp of the backup.
- - Status: Whether the backup is normal.
-
- More detailed backup information can be displayed if a where clause is
specified after `SHOW SNAPSHOT`.
-
- - Database: The database corresponding to the backup.
- - Details: Shows the complete data directory structure of the backup.
+ View existing backups in the remote repository. Please refer to [Show
Snapshot
Reference](../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/SHOW-SNAPSHOT.md).
5. CANCEL BACKUP
- Cancel the currently executing backup job.
+ Cancel the currently executing backup job. Please refer to [Cancel
Backup Reference]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CANCEL-BACKUP.md).
6. DROP REPOSITORY
- Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data.
+ Delete the created remote repository. Deleting a warehouse only deletes
the mapping of the warehouse in Doris, and does not delete the actual warehouse
data. Please refer to [Drop Repository Reference]
(../../sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md).
## More Help
diff --git
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
index ef5f7a286c..f6ea2a1a26 100644
---
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
+++
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/BACKUP.md
@@ -103,7 +103,7 @@ BACKUP
1. Only one backup operation can be performed under the same database.
-2. The backup operation will back up the underlying table and [materialized
view](../../../../query-acceleration/materialized-view.md) of the specified
table or partition, and only one copy will be backed up.
+2. The backup operation will back up the underlying table and [Synchronous
materialized
view](../../../../query/view-materialized-view/materialized-view.md) of the
specified table or partition, and only one replica will be backed up.
[Asynchronous materialized
view](../../../../query/view-materialized-view/async-materialized-view.md) is
not supported.
3. Efficiency of backup operations
diff --git
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
index f5571c4c44..144fab4839 100644
---
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
+++
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/CREATE-REPOSITORY.md
@@ -38,62 +38,21 @@ grammar:
```sql
CREATE [READ ONLY] REPOSITORY `repo_name`
-WITH [BROKER `broker_name`|S3|hdfs]
+WITH [S3|hdfs]
ON LOCATION `repo_location`
PROPERTIES ("key"="value", ...);
```
illustrate:
-- Creation of repositories, relying on existing brokers or accessing cloud
storage directly through AWS s3 protocol, or accessing HDFS directly.
+- Creation of repositories, accessing cloud storage directly through AWS s3
protocol, or accessing HDFS directly.
- If it is a read-only repository, restores can only be done on the
repository. If not, backup and restore operations are available.
-- PROPERTIES are different according to different types of broker or S3 or
hdfs, see the example for details.
-- ON LOCATION ： if it is S3 , here followed by the Bucket Name.
+- PROPERTIES are different according to different types of S3 or hdfs, see the
example for details.
+- ON LOCATION : if it is S3 , here followed by the Bucket Name.
### Example
-1. Create a warehouse named bos_repo, rely on BOS broker "bos_broker", and the
data root directory is: bos://palo_backup
-
-```sql
-CREATE REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_secret_accesskey"
-);
-```
-
-2. Create the same repository as Example 1, but with read-only properties:
-
-```sql
-CREATE READ ONLY REPOSITORY `bos_repo`
-WITH BROKER `bos_broker`
-ON LOCATION "bos://palo_backup"
-PROPERTIES
-(
- "bos_endpoint" = "http://gz.bcebos.com";,
- "bos_accesskey" = "bos_accesskey",
- "bos_secret_accesskey"="bos_accesskey"
-);
-```
-
-3. Create a warehouse named hdfs_repo, rely on Baidu hdfs broker
"hdfs_broker", the data root directory is:
hdfs://hadoop-name-node:54310/path/to/repo/
-
-```sql
-CREATE REPOSITORY `hdfs_repo`
-WITH BROKER `hdfs_broker`
-ON LOCATION "hdfs://hadoop-name-node:54310/path/to/repo/"
-PROPERTIES
-(
- "username" = "user",
- "password" = "password"
-);
-```
-
-4. Create a repository named s3_repo to link cloud storage directly without
going through the broker.
+1. Create a repository named s3_repo.
```sql
CREATE REPOSITORY `s3_repo`
@@ -109,7 +68,7 @@ PROPERTIES
);
```
-5. Create a repository named hdfs_repo to link HDFS directly without going
through the broker.
+2. Create a repository named hdfs_repo.
```sql
CREATE REPOSITORY `hdfs_repo`
@@ -143,8 +102,6 @@ PROPERTIES
7. Create a repository named minio_repo via temporary security credentials.
-<version since="1.2"></version>
-
```
CREATE REPOSITORY `minio_repo`
WITH S3
@@ -159,7 +116,7 @@ PROPERTIES
)
```
-8. Create repository using Tencent COS
+1. Create repository using Tencent COS
```
CREATE REPOSITORY `cos_repo`
diff --git
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
index 6a7e6e5a66..7a0b937cdc 100644
---
a/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
+++
b/versioned_docs/version-2.1/sql-manual/sql-statements/Data-Definition-Statements/Backup-and-Restore/DROP-REPOSITORY.md
@@ -42,14 +42,14 @@ DROP REPOSITORY `repo_name`;
illustrate:
-- Deleting a warehouse just deletes the warehouse's mapping in Palo, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same broker and LOCATION.
+- Deleting a warehouse just deletes the warehouse's mapping in Doris, not the
actual warehouse data. Once deleted, it can be mapped to the repository again
by specifying the same LOCATION.
### Example
-1. Delete the repository named bos_repo:
+1. Delete the repository named example_repo:
```sql
-DROP REPOSITORY `bos_repo`;
+DROP REPOSITORY `example_repo`;
```
### Keywords
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
