周沛辰 created IOTDB-5500: -------------------------- Summary: Fast compaction is slow with template Key: IOTDB-5500 URL: https://issues.apache.org/jira/browse/IOTDB-5500 Project: Apache IoTDB Issue Type: Bug Reporter: 周沛辰 Assignee: 周沛辰
h4. 对齐序列+template FIT物理机 192.168.10.75 数据备份 /data/mpp_test/r_1214_b7516d4/data 顺序tsfile49(9.9GB),乱序tsfile186(39GB)。共48.9GB。 但是测试版本是 master 1214_2de2096 |序号|配置参数|合并耗时|数据一致性|合出的tsfile数量|合出的tsfile总大小|cpu/mem使用率率| |1|cross_performer=fast|2022-12-14 13:51:55 2022-12-14 15:56:02 2小时4分钟7秒|/ |48 |30GB |!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=MGY1Y2Y0ZTJlYjZmM2QxMGVkYjZkMDg2NDEwMTZhMzJfVFRncEVmVkZUTHYweFhaeU5FYjRlM1BmQ3E5cHp3ZkpfVG9rZW46Ym94Y243RW1Cd05LOWZvR0t4Uml3V1paaUVnXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! 序号1:fast1内存使用最高,序号2和3差不多。 默认参数:cpu使用率最高,io最高。| |2 |cross_performer=fast inner_unseq_performer=fast |2022-12-14 16:26:59 2022-12-14 18:27:46 2小时47秒|/ |49 |30GB | |3|cross_performer=fast inner_unseq_performer=fast inner_seq_performer=fast|2022-12-14 19:01:59 2022-12-14 21:07:28 2小时5分钟29秒|/|48 |30GB | |4|默认|2022-12-14 21:36:58 2022-12-14 23:43:14 2小时6分钟16秒|/ |47 |30GB | |5|结果描述 |差不多 |select min_time(s_477),max_time(s_477) ,count(s_477) from root.** align by device;|/ |/ | | |6|测试结果|待定|通过|通过|通过|序号1的内存使用略高。| h4. 普通序列+template 合并不限流 测试机器 fit 192.168.10.75 48CPU384GB 数据备份 /data/mpp_test/m_1215_831a45e/data 顺序数据:2731 (267G) 乱序数据:5276 (511G) 序号1 /data/mpp_test/m_1215_831a45e_fast1 序号2 /data/mpp_test/m_1215_831a45e_fast2 序号3 /data/mpp_test/m_1215_831a45e_fast3 序号4 /data/mpp_test/m_1215_831a45e_comm compaction_write_throughput_mb_per_sec=1048576 |序号|配置参数|合并耗时|数据一致性|合出的tsfile数量|合出的tsfile总大小|cpu/mem使用率率| |1|cross_performer=fast|2022-12-16 01:41:10 2022-12-16 10:51:20 33010秒|/ |494 |139GB |CPU !https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=Y2I2MWRjZTRjOTc3Y2NlMTBiODE2NGU4NzM0NTA2YmRfM01pODUxNGdvZVhtSjRNT3pzZmVFdEJIN0NwTXBvellfVG9rZW46Ym94Y25RRUR0V21KTmlQM0NGaDVlYllBZEtlXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! !https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=M2FjOTliZDZmNjYwMTJjYTlhZWU2YmU2YzdjZTMyODNfdDZoeEIyT3BKR25McGxhWU9weFJsWGZqMzNLNjB2d2RfVG9rZW46Ym94Y25sdzNoU3UybjJTemNqVFlzU0w2VThmXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! !https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=ZGQwYTg4OTc4MDZlYjBhMzMxNDFmOTRhOGI2MTc5MDVfSWJ6NFlMV0loaGRNVnB2WnZacksxbTloUzROWlVid1ZfVG9rZW46Ym94Y25lUTI1N2NqTUF3ZDFCeUE2VmJmVGxlXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! !https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=ZjY3MTM3NjRkNWFlNGJmZWI4MGJkMmM4NDgwYjQ3OTlfZGJLS3M5YkZ1VnB3b0xhWUNHRVdpRmVVYVNTdHNZSndfVG9rZW46Ym94Y250TnkxdjA5Szl4Z2ZQZkRxQ3hDZHpiXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! | |2 |cross_performer=fast inner_unseq_performer=fast |2022-12-16 11:18:32 2022-12-16 19:09:33 28261秒|/ |453 |138GB | |3|cross_performer=fast inner_unseq_performer=fast inner_seq_performer=fast|2022-12-16 19:35:36 2022-12-17 05:21:06 35130秒|/|546 |270GB | |4|默认|2022-12-17 05:35:44 2022-12-17 15:57:24 37300秒|/ |495 |138GB | |5|结果描述 |!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=OGQ0YjU0NjljYjBmZmRlMzA3NTMwZWRmMDk5NDRlNmNfVGVQQ2dCaFN5eHV4ZzVvN1FSRGFKUHc4dmJlUUs5VWRfVG9rZW46Ym94Y25vWHFwbGJiejRSekoyWnI3cGJyMkpjXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA! 纵轴越高,合并越慢 默认/fast1=1.13 默认/fast2=1.32 默认/fast3=1.06|select min_time(s_433),max_time(s_433),count(s_433) from root.** align by device|3个参数设置fast合并完成,tsfile数多。 |3个参数设置fast合并完成,tsfile占用空间大。 |序号1 和序号2的使用内存比默认略高。 序号3和默认的使用内存差不多。 | |6|测试结果|fast总体性能优势不明显。|通过|通过|不通过| | -- This message was sent by Atlassian Jira (v8.20.10#820010)