This is an automated email from the ASF dual-hosted git repository.
wangdan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pegasus-website.git
The following commit(s) were added to refs/heads/master by this push:
new 92093d23 Update deployment docs (#64)
92093d23 is described below
commit 92093d23836981545a2e0ece8a8ef7f0fb221cf8
Author: Yingchun Lai <[email protected]>
AuthorDate: Wed Jan 24 16:11:14 2024 +0800
Update deployment docs (#64)
---
_docs/en/administration/deployment.md | 115 +++++++++++++++++++++++++++++++++-
_docs/zh/administration/deployment.md | 55 ++++++++--------
2 files changed, 141 insertions(+), 29 deletions(-)
diff --git a/_docs/en/administration/deployment.md
b/_docs/en/administration/deployment.md
index c1a059cd..76929bf3 100755
--- a/_docs/en/administration/deployment.md
+++ b/_docs/en/administration/deployment.md
@@ -2,4 +2,117 @@
permalink: administration/deployment
---
-TRANSLATING
+# Prepare servers
+
+The Pegasus distributed cluster requires at least these servers to be prepared:
+* MetaServer: 2 or 3 servers, no SSD required.
+* ReplicaServer: At least 3 servers, it is recommended to mount SSD disks.
Multiple disks can improve the throughput capacity of a single node, and each
server should mount the same number and performance (e.g., IOPS, bandwidth and
R/W latency) of disks to ensure load balancing.
+* Collector: Optional role, 1 server, no SSD required. This process is mainly
used to collect and summarize cluster metrics, with a small load. It is
recommended to deploy it on one of the MetaServer servers.
+
+# Prepare Apache Zookeeper
+
+The Pegasus cluster relies on Zookeeper for metadata storage and MetaServer
leader election, therefore requiring a Zookeeper service.
+* It is recommended to deploy Zookeeper in the same server room as the Pegasus
cluster server.
+
+# Prepare configuration files
+
+Since 1.7.1, Pegasus has provided [configuration
file](https://github.com/apache/incubator-pegasus/blob/master/src/server/config.ini),
you need to modify the file to replace all variables in the form of `%{xxx}`
with appropriate values. As follows:
+
+| Variables | Description
| Example
|
+|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------|
+| %{cluster.name} | Cluster name. Indicator reporting labels that will be
used for collectors, etc
| my_cluster
|
+| %{home.dir} | Pegasus home path. It will store global configuration
files, such as disk blacklist configuration files
| /home/work
|
+| %{app.dir} | Program working path. The data files and log files
will be placed here by default
| /home/work/app/pegasus
|
+| %{slog.dir} | The path to store the shared-log files. It is
recommended to place it on an exclusive SSD drive. If there is no available SSD
disk, it can be set as an empty string, indicating that `%{app.dir}` is used by
default. Deprecate since version 2.6 |
/home/work/ssd1/pegasus |
+| %{data.dirs} | A list of paths to store user data, separated by
commas. Each path needs to be specified with a name in the format of
`name1:path1,name2:path2`. If there is no available SSD disk, it can be set as
an empty string, indicating that `%{app.dir}` is used by default |
ssd2:/home/work/ssd2/pegasus,ssd3:/home/work/ssd3/pegasus |
+| %{meta.server.list} | MetaServer address list, separated by commas. In the
format of `ip1:port1,ip2:port2`. **Note: Currently, only IP addresses are
supported and hostnames are not supported**
|
1.2.3.4:34601,1.2.3.5:34601 |
+| %{zk.server.list} | Zookeeper address list, separated by commas. In the
format of `ip1:port1,ip2:port2`
| 1.2.3.4:2181,1.2.3.5:2181
|
+
+> Please refer to the meaning of [configuration](/administration/config)。
+
+> Note: The same variable may appear in multiple places, so it is important to
ensure that all `%{xxx}` variables are replaced.
+
+## How to configure multiple SSD disks
+
+If there are multiple SSD disks, it is recommended to use one SSD disk
specifically for slog (i.e., shared-log) and the other disks to store user data
for each replicas.
+
+For example, suppose the server has 4 disks and the mounting path is
`/home/work/ssd{id}`, where {id}=1,2,3,4. So you can use ssd1 for slog, which
can be configured as follows:
+```ini
+[replication]
+ slog_dir = /home/work/ssd1/pegasus
+ data_dirs =
ssd2:/home/work/ssd2/pegasus,ssd3:/home/work/ssd3/pegasus,ssd4:/home/work/ssd4/pegasus
+```
+
+If there is only one SSD drive, then both slog and each replicas data share
this drive. Assuming the SSD disk mounting path is `/home/work/ssd`, it can be
configured as follows:
+```ini
+[replication]
+ slog_dir = /home/work/ssd/pegasus/{cluster.name}
+ data_dirs = ssd:/home/work/ssd/pegasus/{cluster.name}
+```
+
+## How to configure multiple network cards
+
+There is a section in the configuration file:
+```ini
+[network]
+ primary_interface =
+```
+Specify network card through `primary_interface`:
+* If there is only one network card, it can be set as an empty string to
automatically obtain the appropriate network card address. The specific
strategy is to search for the first address that complies with the
`10.\*.\*.\*/172.16.\*.\*/192.168.\*.\*` rule (i.e., the intranet address) in
the output list of the `ifconfig` command, which will ignore the loopback
address and virtual address.
+* If there are multiple network cards, please specify the network card name.
If not specified, the first address that complies with
`10.\*.\*.\*/172.16.\*.\*/192.168.\*.\*` rules will be used.
+
+For example, if there are multiple network cards and you want to use the
`eth2` network card, you can configure it as follows:
+```ini
+[network]
+ primary_interface = eth2
+```
+
+# Preparing to deploy packages
+
+The three roles of ReplicaServer, MetaServer, and Collector share the same
program binaries and configuration files.
+
+At first [build Pegasus](/overview/compilation). After building, run the
following command to package and generate a server-side deployment package:
+```
+./run.sh pack_server
+```
+After successful packed, a directory and a tar.gz package named
`pegasus-server-{version}-{gitSHA}-{platform}-{buildType}` will be generated in
the local path.
+There is a `bin/` directory that contains pegasus_server program binary and
dependency libraries, as well as the recently modified `config.ini` file.
+
+Copy the tar.gz package to the server that needs to be deployed and unzip it.
+
+# Start Service
+
+Before starting the server, it is necessary to add the path of the dynamic
link libraries that the program depends on to `LD_LIBRARY_PATH`:
+```
+export LD_LIBRARY_PATH=/path/to/your/pegasus/bin:$LD_LIBRARY_PATH
+```
+
+## Start MetaServer：
+```
+cd bin/
+./pegasus_server config.ini -app_list meta
+```
+
+## Start ReplicaServer：
+```
+cd bin/
+./pegasus_server config.ini -app_list replica
+```
+
+## Start Collector：
+```
+cd bin/
+./pegasus_server config.ini -app_list collector
+```
+
+* After the cluster is successfully started, a `temp` table will be created by
default, which is also used for the cluster availability detection by the
Collector.
+* Use [Shell tools](/overview/shell) to view various states of the cluster.
+* If the startup fails, you can check the logs in `%{app.dir}/log` to
troubleshoot the issue.
+
+# FAQ
+
+## Cluster cleaning
+
+If you want to completely redeploy the cluster and **clean up all data**, you
need to clean up the following environment, otherwise there may be issues when
starting a new cluster:
+* Remove the `%{app.dir}`, `%{slog.dir}` and `%{data.dirs}` directories of
MetaServer，ReplicaServer and Collector
+* Remove the `%{cluster_root}` path of Zookeeper
diff --git a/_docs/zh/administration/deployment.md
b/_docs/zh/administration/deployment.md
index b6feb120..c45ffabc 100644
--- a/_docs/zh/administration/deployment.md
+++ b/_docs/zh/administration/deployment.md
@@ -6,45 +6,44 @@ permalink: administration/deployment
Pegasus 分布式集群至少需要准备这些服务器：
* MetaServer：2~3 台服务器，无需 SSD 盘。
-* ReplicaServer：至少 3 台服务器，建议挂载 SSD 盘。多块磁盘能够提升节点的吞吐能力，各台服务器挂载相同数量和性能的磁盘来保证负载均衡。
-* Collector：可选角色，1 台服务器，无需 SSD 盘。该进程主要用于收集和汇总集群的统计信息，负载很小，建议放在 MetaServer
的其中一台服务器上。
+* ReplicaServer：至少 3 台服务器，建议挂载 SSD
盘。多块磁盘能够提升节点的吞吐能力，各台服务器挂载相同数量和性能（例如，IOPS，带宽和读写延迟等）的磁盘来保证负载均衡。
+* Collector：可选角色，1 台服务器，无需 SSD 盘。该进程主要用于收集和汇总集群的 metrics 信息，负载很小，建议部署在
MetaServer 的其中一台服务器上。
-# 准备 Zookeeper
+# 准备 Apache Zookeeper
-Pegasus 集群依赖 Zookeeper 进行元数据存储和 MetaServer 抢锁，因此需要一个 Zookeeper 服务。
+Pegasus 集群依赖 Zookeeper 进行元数据存储和 MetaServer 选主，因此需要一个 Zookeeper 服务。
* 建议在 Pegasus 集群服务器所在的同机房搭建。
-记下 Zookeeper 的服务地址列表，后面配置文件要用。
-
# 准备配置文件
-从 1.7.1 版本开始，Pegasus 提供了配置文件
[src/server/config.ini](https://github.com/apache/incubator-pegasus/blob/master/src/server/config.ini)，你需要修改该文件，替换所有
`%{xxx}` 形式的变量为合适的值。如下：
+从 1.7.1 版本开始，Pegasus 提供了
[配置文件](https://github.com/apache/incubator-pegasus/blob/master/src/server/config.ini)，你需要修改该文件，替换所有
`%{xxx}` 形式的变量为合适的值。如下：
-| 变量 | 说明
| 示例 |
-|---------------------|------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------|
-| %{cluster.name} | 集群名称。会用于 collector 的指标上报标签等
|
my_cluster |
-| %{home.dir} | Pegasus 主路径。会存放全局的配置文件，如磁盘黑名单配置文件
|
/home/work |
-| %{app.dir} | 程序工作路径。默认数据文件和日志文件都会放在这里
|
/home/work/app/pegasus |
-| %{slog.dir} | 存放 Shared Commit Log 文件的路径，建议放在一个独享的 SSD 盘上。如果没有可用的
SSD 盘，可以设置为空字符串，表示默认使用 %{app.dir}。2.6 版本之后已废弃 |
/home/work/ssd1/pegasus |
-| %{data.dirs} | 存放用户数据的路径列表，用逗号分隔。每个路径需要指定一个名称，格式为
`name1:path1,name2:path2`。如果没有可用的独立 SSD 盘，可以设置为空字符串，表示默认使用 %{app.dir} |
ssd2:/home/work/ssd2/pegasus,ssd3:/home/work/ssd3/pegasus |
-| %{meta.server.list} | MetaServer 地址列表，用逗号分隔。格式为 `ip1:port1,ip2:port2`。**
注意只能用 IP 地址，不能用 hostname** |
1.2.3.4:34601,1.2.3.5:34601 |
-| %{zk.server.list} | Zookeeper 地址列表，用逗号分隔。格式为 `ip1:port1,ip2:port2`
| 1.2.3.4:2181,1.2.3.5:2181 |
+| 变量 | 说明
| 示例
|
+|---------------------|----------------------------------------------------------------------------------------------------------|-----------------------------------------------------------|
+| %{cluster.name} | 集群名称。会用于 collector 的指标上报标签等
| my_cluster
|
+| %{home.dir} | Pegasus 主路径。会存放全局的配置文件，如磁盘黑名单配置文件
| /home/work
|
+| %{app.dir} | 程序工作路径。默认数据文件和日志文件都会放在这里
| /home/work/app/pegasus
|
+| %{slog.dir} | 存放 shared-log 文件的路径，建议放在一个独享的 SSD 盘上。如果没有可用的 SSD
盘，可以设置为空字符串，表示默认使用 `%{app.dir}`。2.6 版本之后已废弃 |
/home/work/ssd1/pegasus |
+| %{data.dirs} | 存放用户数据的路径列表，用逗号分隔。每个路径需要指定一个名称，格式为
`name1:path1,name2:path2`。如果没有可用的 SSD 盘，可以设置为空字符串，表示默认使用 `%{app.dir}` |
ssd2:/home/work/ssd2/pegasus,ssd3:/home/work/ssd3/pegasus |
+| %{meta.server.list} | MetaServer 地址列表，用逗号分隔。格式为
`ip1:port1,ip2:port2`。**注意：目前只支持 IP 地址，不支持 hostname**
| 1.2.3.4:34601,1.2.3.5:34601 |
+| %{zk.server.list} | Zookeeper 地址列表，用逗号分隔。格式为 `ip1:port1,ip2:port2`
| 1.2.3.4:2181,1.2.3.5:2181
|
> 配置的含义请参考 [配置说明](/administration/config)。
-> 注意：同一个变量可能出现在多个地方，要保证所有的 ``%{xxx}`` 变量都被替换掉。
+> 注意：同一个变量可能出现在多个地方，要保证所有的 `%{xxx}` 变量都被替换掉。
## 多个 SSD 盘如何配置
-如果有多个 SSD 盘，推荐使用一个 SSD 盘专门用于 slog，其他盘用于存储 replica 数据。
+如果有多个 SSD 盘，推荐使用一个 SSD 盘专门用于 slog（即 shared-log），其他盘用于存储各分片的用户数据。
-譬如，假设服务器有 4 个盘，挂载路径为 /home/work/ssd{id}，其中 {id}=1,2,3,4。那么可以将 ssd1 用于 shared
log，可配置如下：
+譬如，假设服务器有 4 个盘，挂载路径为 `/home/work/ssd{id}`，其中 {id}=1,2,3,4。那么可以将 ssd1 用于
slog，可配置如下：
```ini
[replication]
slog_dir = /home/work/ssd1/pegasus
data_dirs =
ssd2:/home/work/ssd2/pegasus,ssd3:/home/work/ssd3/pegasus,ssd4:/home/work/ssd4/pegasus
```
-如果只有一个 SSD 盘，那么就只能将 slog 和 data 共享这一块盘。假设 SSD 盘挂载路径为 /home/work/ssd，可配置如下：
+
+如果只有一个 SSD 盘，那么就只能将 slog 和 data 共享这一块盘。假设 SSD 盘挂载路径为 `/home/work/ssd`，可配置如下：
```ini
[replication]
slog_dir = /home/work/ssd/pegasus/{cluster.name}
@@ -77,9 +76,9 @@ ReplicaServer，MetaServer，Collector 三种角色共用一套二进制程序
```
./run.sh pack_server
```
-运行成功后，会在本地文件夹下产生 `pegasus-server-{version}-{gitSHA}-{platform}-{buildType}`
的目录以及 tar.gz 包。其中有个 `bin/` 目录，里面包含 pegasus_server 程序及依赖库，还包括 ** 刚刚修改好的
config.ini 文件 **。
+运行成功后，会在本地文件夹下产生 `pegasus-server-{version}-{gitSHA}-{platform}-{buildType}`
的目录以及 tar.gz 包。其中有个 `bin/` 目录，里面包含 pegasus_server 程序及依赖库，还包括刚刚修改好的 config.ini
文件。
-将 tar.gz 包拷贝到需要部署的服务器上并解压。你可以使用合适的分布式分发工具来完成这件事情。
+将 tar.gz 包拷贝到需要部署的服务器上并解压。
# 启动服务
@@ -106,14 +105,14 @@ cd bin/
./pegasus_server config.ini -app_list collector
```
-集群启动成功后，会默认创建一个 `temp` 表，该表也用于 Collector 的集群可用度检查，最好不要删除。
-
-你可以使用 [Shell 工具](/overview/shell) 查看集群的各种状态。如果启动失败，可以到 `%{app.dir}/log`
内查看日志，排查问题。
+* 集群启动成功后，会默认创建一个 `temp` 表，该表也用于 Collector 的集群可用度检查。
+* 可以使用 [Shell 工具](/overview/shell) 查看集群的各种状态。
+* 如果启动失败，可以到 `%{app.dir}/log` 内查看日志，排查问题。
# 常见问题
-## 集群清理重建
+## 集群清理
如果想完全重新部署集群，**并清理所有数据**，需要清理以下环境，否则可能会出现启动新集群失败的问题：
-* 清理 MetaServer，ReplicaServer 和 Collector 的 `%{app.dir}`，`%{slog.dir}` 和
`%{data.dirs}` 目录
-* 删除 Zookeeper 的 `/pegasus/%{cluster.name}` 节点
+* 删除 MetaServer，ReplicaServer 和 Collector 的 `%{app.dir}`，`%{slog.dir}` 和
`%{data.dirs}` 目录
+* 删除 Zookeeper 的 `%{cluster_root}` 路径
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
