This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 77e772e103 [enhancement](config) add some pre-process and pre-check
for BE storage config attentions in docs (#22486)
77e772e103 is described below
commit 77e772e1032876d05a2530b9cd8656fcc9dffa93
Author: Siyang Tang <[email protected]>
AuthorDate: Mon Aug 7 18:16:57 2023 +0800
[enhancement](config) add some pre-process and pre-check for BE storage
config attentions in docs (#22486)
---
be/src/olap/options.cpp | 10 +++++++++-
be/test/olap/options_test.cpp | 11 +++++++++++
docs/en/docs/install/standard-deployment.md | 8 +++-----
docs/zh-CN/docs/install/standard-deployment.md | 8 +++-----
4 files changed, 26 insertions(+), 11 deletions(-)
diff --git a/be/src/olap/options.cpp b/be/src/olap/options.cpp
index c55aa2fa21..9ea21928d8 100644
--- a/be/src/olap/options.cpp
+++ b/be/src/olap/options.cpp
@@ -148,6 +148,14 @@ Status parse_root_path(const string& root_path, StorePath*
path) {
Status parse_conf_store_paths(const string& config_path,
std::vector<StorePath>* paths) {
std::vector<string> path_vec = strings::Split(config_path, ";",
strings::SkipWhitespace());
+ if (path_vec.empty()) {
+ // means compute node
+ return Status::OK();
+ }
+ if (path_vec.back().empty()) {
+ // deal with the case that user add `;` to the tail
+ path_vec.pop_back();
+ }
for (auto& item : path_vec) {
StorePath path;
auto res = parse_root_path(item, &path);
@@ -157,7 +165,7 @@ Status parse_conf_store_paths(const string& config_path,
std::vector<StorePath>*
LOG(WARNING) << "failed to parse store path " << item << ", res="
<< res;
}
}
- if (paths->empty() || (path_vec.size() != paths->size() &&
!config::ignore_broken_disk)) {
+ if ((path_vec.size() != paths->size() && !config::ignore_broken_disk)) {
return Status::Error<INVALID_ARGUMENT>("fail to parse
storage_root_path config. value={}",
config_path);
}
diff --git a/be/test/olap/options_test.cpp b/be/test/olap/options_test.cpp
index d01a8a70aa..5bab61bdeb 100644
--- a/be/test/olap/options_test.cpp
+++ b/be/test/olap/options_test.cpp
@@ -19,10 +19,12 @@
#include <gtest/gtest-message.h>
#include <gtest/gtest-test-part.h>
+#include <gtest/gtest.h>
#include <stdlib.h>
#include <filesystem>
#include <string>
+#include <vector>
#include "gtest/gtest_pred_impl.h"
#include "olap/olap_define.h"
@@ -114,6 +116,15 @@ TEST_F(OptionsTest, parse_root_path) {
EXPECT_EQ(10 * GB_EXCHANGE_BYTE, path.capacity_bytes);
EXPECT_EQ(TStorageMedium::HDD, path.storage_medium);
}
+ {
+ // test tail `;`
+ std::string path = path1 + ";" + path2 + ";";
+ std::vector<StorePath> paths;
+ EXPECT_EQ(Status::OK(), parse_conf_store_paths(path, &paths));
+ EXPECT_EQ(paths.size(), 2);
+ EXPECT_STREQ(path1.c_str(), paths[0].path.c_str());
+ EXPECT_STREQ(path2.c_str(), paths[1].path.c_str());
+ }
}
} // namespace doris
diff --git a/docs/en/docs/install/standard-deployment.md
b/docs/en/docs/install/standard-deployment.md
index 64baef2738..33d277a16b 100644
--- a/docs/en/docs/install/standard-deployment.md
+++ b/docs/en/docs/install/standard-deployment.md
@@ -197,14 +197,12 @@ See the `lower_case_table_names` section in
[Variables](../advanced/variables.md
* Modify all BE configurations
- Modify be/conf/be.conf. Mainly configure `storage_root_path`: data storage
directory. By default, it is under be/storage. If you need to specify a
directory, you need to **pre-create the directory**. Multiple paths are
separated by a semicolon `;` in English (**do not add `;`** after the last
directory).
+ Modify be/conf/be.conf. Mainly configure `storage_root_path`: data storage
directory. By default, it is under be/storage. If you need to specify a
directory, you need to **pre-create the directory**. Multiple paths are
separated by a semicolon `;` in English.
The hot and cold data storage directories in the node can be distinguished
by path, HDD (cold data directory) or SSD (hot data directory). If you don't
need the hot and cold mechanism in the BE node, you only need to configure the
path without specifying the medium type; and you don't need to modify the
default storage medium configuration of FE
**Notice:**
- 1. If you specify the storage type of the storage path, at least one path
must have a storage type of HDD (cold data directory)!
- 2. If the storage type of the storage path is not specified, all are HDD
(cold data directory) by default.
- 3. The HDD and SSD here have nothing to do with the physical storage
medium, but only to distinguish the storage type of the storage path, that is,
you can mark a certain directory on the disk of the HDD medium as SSD (hot data
directory).
- 4. Here HDD and SSD **MUST** be capitalized!
+ 1. If the storage type of the storage path is not specified, all are HDD
(cold data directory) by default.
+ 2. The HDD and SSD here have nothing to do with the physical storage
medium, but only to distinguish the storage type of the storage path, that is,
you can mark a certain directory on the disk of the HDD medium as SSD (hot data
directory).
Example 1 is as follows:
diff --git a/docs/zh-CN/docs/install/standard-deployment.md
b/docs/zh-CN/docs/install/standard-deployment.md
index d82b05d9ad..cc1b34f73a 100644
--- a/docs/zh-CN/docs/install/standard-deployment.md
+++ b/docs/zh-CN/docs/install/standard-deployment.md
@@ -192,14 +192,12 @@ doris默认为表名大小写敏感，如有表名大小写不敏感的需求需
* 修改所有 BE 的配置
- 修改 be/conf/be.conf。主要是配置
`storage_root_path`：数据存放目录。默认在be/storage下，若需要指定目录的话，需要**预创建目录**。多个路径之间使用英文状态的分号
`;` 分隔（**最后一个目录后不要加 `;`**）。
+ 修改 be/conf/be.conf。主要是配置
`storage_root_path`：数据存放目录。默认在be/storage下，若需要指定目录的话，需要**预创建目录**。多个路径之间使用英文状态的分号
`;` 分隔。
可以通过路径区别节点内的冷热数据存储目录，HDD（冷数据目录）或 SSD（热数据目录）。如果不需要 BE
节点内的冷热机制，那么只需要配置路径即可，无需指定 medium 类型；也不需要修改FE的默认存储介质配置
**注意：**
- 1. 如果指定存储路径的存储类型，则最少要有一个路径的存储类型为 HDD（冷数据目录）！
- 2. 如果未指定存储路径的存储类型，则默认全部为 HDD（冷数据目录）。
- 3. 这里的 HDD 和 SSD 与物理存储介质无关，只为了区分存储路径的存储类型，即可以在 HDD 介质的盘上标记某个目录为 SSD（热数据目录）。
- 4. 这里的 HDD 和 SSD **必须**要大写！
+ 1. 如果未指定存储路径的存储类型，则默认全部为 HDD（冷数据目录）。
+ 2. 这里的 HDD 和 SSD 与物理存储介质无关，只为了区分存储路径的存储类型，即可以在 HDD 介质的盘上标记某个目录为 SSD（热数据目录）。
示例1如下：
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
