Hi all,
The current default configurations and JVM options of IoTDB are mainly
designed for deployments where IoTDB runs exclusively on a machine.
However, in many scenarios, IoTDB needs to share the machine with other
services.
We plan to introduce an IoTDB Edge distribution for standalone deployment,
targeting a combined memory limit of 512 MiB for one ConfigNode and one
DataNode.
The initial proposal is as follows.
1.
Lightweight system configuration defaults
-
Limit RPC connections and internal client pools:
-
cn_rpc_max_concurrent_client_num=16
-
dn_rpc_max_concurrent_client_num=16
-
Selector thread counts set to 1
-
Limit query concurrency:
-
max_allowed_concurrent_queries=16
-
query_thread_count=1
-
degree_of_query_parallelism=1
-
MPP data exchange core/max pool sizes set to 4
-
Limit background workers:
-
Procedure, flush, compaction, compaction subtask, and compaction
scheduler thread counts set to 1
-
Reduce memory-related buffers:
-
One WAL node with a 1 MiB WAL buffer
-
64 KiB TsBlock size
-
Query batch size of 1,000
-
Sort and modification buffers limited to 1 MiB
2.
Lightweight JVM options
ConfigNode:
-Xms32M -Xmx64M -XX:MaxDirectMemorySize=8M -XX:MaxMetaspaceSize=48M
-XX:ReservedCodeCacheSize=24M
DataNode:
-Xms64M -Xmx128M -XX:MaxDirectMemorySize=24M -XX:MaxMetaspaceSize=64M
-XX:ReservedCodeCacheSize=32M
Common options:
-XX:ActiveProcessorCount=1 -XX:+UseSerialGC -Xss320k
-XX:TieredStopAtLevel=1 -XX:InitialCodeCacheSize=4M
3.
New release package
For every IoTDB release, we propose publishing an additional binary package:
apache-iotdb-edge-${version}-bin.zip
This package would contain dedicated lightweight configuration files, JVM
options, and a standalone startup script. The existing all-bin package and
its default behavior would remain unchanged.
Feedback is welcome, especially regarding the combined 512 MiB memory
target, configuration defaults, JVM options, and package naming.
Best regards,
----------------------
Yuan Tian
