[ https://issues.apache.org/jira/browse/IOTDB-5237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17682453#comment-17682453 ]
周沛辰 commented on IOTDB-5237: ---------------------------- 解决方法是:调参,减小Fast空间内合并出的文件大小,这样一次跨空间任务就可以选中较多的源文件参与合并。 > [fast compaction] cross/seq/unseq parameters are set to the value "fast" , > and the compaction performance is slow > ----------------------------------------------------------------------------------------------------------------- > > Key: IOTDB-5237 > URL: https://issues.apache.org/jira/browse/IOTDB-5237 > Project: Apache IoTDB > Issue Type: Improvement > Components: mpp-cluster > Reporter: 刘珍 > Assignee: 周沛辰 > Priority: Major > Attachments: fast_ip75_1.conf, fast_ip75_2.conf, fast_ip75_3.conf, > image-2022-12-19-16-02-15-108.png, image-2022-12-19-16-03-00-384.png, > image-2022-12-19-16-03-11-460.png, image-2022-12-19-16-03-26-674.png, > image-2022-12-19-16-03-42-117.png > > > master_1215_831a45e > compaction_write_throughput_mb_per_sec=1048576 (相当于合并 不限流) > 普通序列,使用模板,3个参数设置fast,比1/2个参数设置fast的 > 合并性能慢,且合出来的文件多,占用磁盘空间多。 > !image-2022-12-19-16-02-15-108.png|width=913,height=862! > 资源监控: > !image-2022-12-19-16-03-00-384.png! > !image-2022-12-19-16-03-11-460.png! > !image-2022-12-19-16-03-26-674.png! > !image-2022-12-19-16-03-42-117.png! > 测试环境 > 1. 192.168.10.75 48CPU384GB HDD > IoTDB 关合并 ,加大时间分区间隔(不跨分区) > 生成数据见配置1,2,3 > 串行运行BM,写入完成,cli执行flush。 > 2. 生成测试数据:IoTDB配置 > ConfigNode配置 > MAX_HEAP_SIZE="8G" > DataNode配置 > MAX_HEAP_SIZE="192G" > MAX_DIRECT_MEMORY_SIZE="32G" > Common配置 > time_partition_interval=60480000000 > query_timeout_threshold=36000000 > enable_seq_space_compaction=false > enable_unseq_space_compaction=false > enable_cross_space_compaction=false > compaction_write_throughput_mb_per_sec=1048576 > 3. 数据与日志信息 > 数据备份 /data/mpp_test/m_1215_831a45e/data_template > 顺序数据:2731 (267G) > 乱序数据:5276 (511G) > 序号1 /data/mpp_test/m_1215_831a45e_fast1 > 序号2 /data/mpp_test/m_1215_831a45e_fast2 > 序号3 /data/mpp_test/m_1215_831a45e_fast3 > 序号4 /data/mpp_test/m_1215_831a45e_comm -- This message was sent by Atlassian Jira (v8.20.10#820010)