SteveYurongSu commented on code in PR #15181:
URL: https://github.com/apache/iotdb/pull/15181#discussion_r2014333768
##########
iotdb-core/node-commons/src/assembly/resources/conf/iotdb-system.properties.template:
##########
@@ -242,6 +242,27 @@ dn_data_dirs=data/datanode/data
# Datatype: String
dn_multi_dir_strategy=SequenceStrategy
+# load_disk_select_strategy
+# The strategy is used to choose a proper disk for loading tsfile.
+# The info of the two strategies are as follows:
+# 1. MIN_IO_FIRST: the system will try to choose the same disk as tsFileToLoad
for loading.
+# 2. DISK_STORAGE_BALANCE_FIRST: the system will make the disk choice that
balance storage as much as possible.
+# If this property is unset, system will use DISK_STORAGE_BALANCE_FIRST as
default strategy because storage-balance is ensured with higher priority.
+# effectiveMode: restart
+# Datatype: String
+load_disk_select_strategy=DISK_STORAGE_BALANCE_FIRST
+
+# load_disk_select_strategy_for_pipe_and_iotv2
+# The strategy is used to choose a proper disk for loading tsfile.
+# The info of the three strategies are as follows:
+# 1. MIN_IO_FIRST: the system will try to choose the same disk as tsFileToLoad
for loading.
+# 2. DISK_STORAGE_BALANCE_FIRST: the system will make the disk choice that
balance storage as much as possible.
+$ 3. EXTEND_LOAD: the system will use the same strategy as
load_disk_select_strategy
Review Comment:
```suggestion
$ 3. INHERIT_LOAD: the system will use the same strategy as
load_disk_select_strategy
```
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java:
##########
@@ -388,6 +394,28 @@ public DataRegion(
recover();
}
+ switch
(ILoadDiskSelector.LoadDiskSelectorType.fromValue(config.getLoadDiskSelectStrategy()))
{
+ case MIN_IO_FIRST:
+ ordinaryLoadDiskSelector = new MinIOSelector();
+ break;
+ case DISK_STORAGE_BALANCE_FIRST:
+ default:
+ ordinaryLoadDiskSelector = new StorageBalanceSelector();
+ }
+
+ switch (ILoadDiskSelector.LoadDiskSelectorType.fromValue(
+ config.getLoadDiskSelectStrategyForIoTV2AndPipe())) {
+ case MIN_IO_FIRST:
+ pipeAndIoTV2LoadDiskSelector = new MinIOSelector();
+ break;
+ case EXTEND_LOAD:
+ pipeAndIoTV2LoadDiskSelector = ordinaryLoadDiskSelector;
+ break;
+ case DISK_STORAGE_BALANCE_FIRST:
+ default:
Review Comment:
EXTEND_LOAD by default
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java:
##########
@@ -3051,17 +3101,28 @@ private boolean loadTsFileToUnSequence(
final boolean deleteOriginFile,
boolean isGeneratedByPipe)
throws LoadFileException, DiskSpaceInsufficientException {
- final File targetFile =
- fsFactory.getFile(
- TierManager.getInstance().getNextFolderForTsFile(0, false),
- databaseName
- + File.separatorChar
- + dataRegionId
- + File.separatorChar
- + filePartitionId
- + File.separator
- + tsFileResource.getTsFile().getName());
- tsFileResource.setFile(targetFile);
+ File targetFile = null;
Review Comment:
And targetFile can be final here
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java:
##########
@@ -401,6 +429,28 @@ public DataRegion(String databaseName, String id) {
partitionMaxFileVersions.put(0L, 0L);
upgradeModFileThreadPool = null;
this.metrics = new DataRegionMetrics(this);
+
+ switch
(ILoadDiskSelector.LoadDiskSelectorType.fromValue(config.getLoadDiskSelectStrategy()))
{
Review Comment:
Extract a method for init
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java:
##########
@@ -388,6 +394,28 @@ public DataRegion(
recover();
}
+ switch
(ILoadDiskSelector.LoadDiskSelectorType.fromValue(config.getLoadDiskSelectStrategy()))
{
+ case MIN_IO_FIRST:
Review Comment:
MIN_IO by default?
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/DataRegion.java:
##########
@@ -3051,17 +3101,28 @@ private boolean loadTsFileToUnSequence(
final boolean deleteOriginFile,
boolean isGeneratedByPipe)
throws LoadFileException, DiskSpaceInsufficientException {
- final File targetFile =
- fsFactory.getFile(
- TierManager.getInstance().getNextFolderForTsFile(0, false),
- databaseName
- + File.separatorChar
- + dataRegionId
- + File.separatorChar
- + filePartitionId
- + File.separator
- + tsFileResource.getTsFile().getName());
- tsFileResource.setFile(targetFile);
+ File targetFile = null;
Review Comment:
Using a ? b : c is more elegant
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java:
##########
@@ -2247,6 +2247,17 @@ private void loadLoadTsFileProps(TrimProperties
properties) {
properties.getProperty(
"load_active_listening_verify_enable",
Boolean.toString(conf.isLoadActiveListeningVerifyEnable()))));
+
+ conf.setLoadDiskSelectStrategy(
+ properties.getProperty(
+ "load_disk_select_strategy",
+
ConfigurationFileUtils.getConfigurationDefaultValue("load_disk_select_strategy")));
Review Comment:
Using value from var conf is enough. Why consider using
ConfigurationFileUtils here. May IO be caused in the invocation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
