刘珍 created IOTDB-4200: ------------------------- Summary: DataRegion serial recovery, need to be optimized Key: IOTDB-4200 URL: https://issues.apache.org/jira/browse/IOTDB-4200 Project: Apache IoTDB Issue Type: Improvement Components: mpp-cluster Reporter: 刘珍 Assignee: Jialin Qiao
master_0819_9e0512a schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus 3副本3C3D,集群恢复,各DataRegion是串行恢复,可改为并行提升恢复性能。 各dataregion 开始恢复和结束恢复的日志: 2022-08-22 10:10:10,191 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 567 entries from segment file data_region/47474747-4747-4747-4747-00010000001e/current/log_799996-800562 2022-08-22 10:11:22,571 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 2 entries from segment file data_region/47474747-4747-4747-4747-00010000001e/current/log_inprogress_887810 2022-08-22 10:11:23,635 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 668 entries from segment file data_region/47474747-4747-4747-4747-00010000001a/current/log_799353-800020 2022-08-22 10:12:38,348 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 94 entries from segment file data_region/47474747-4747-4747-4747-00010000001a/current/log_inprogress_890298 2022-08-22 10:12:39,236 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 670 entries from segment file data_region/47474747-4747-4747-4747-00010000001d/current/log_799753-800422 2022-08-22 10:13:58,456 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 629 entries from segment file data_region/47474747-4747-4747-4747-00010000001d/current/log_inprogress_894897 2022-08-22 10:14:00,094 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 609 entries from segment file data_region/47474747-4747-4747-4747-000100000014/current/log_799551-800159 2022-08-22 10:15:13,888 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 2 entries from segment file data_region/47474747-4747-4747-4747-000100000014/current/log_inprogress_889501 2022-08-22 10:15:15,062 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 611 entries from segment file data_region/47474747-4747-4747-4747-000100000016/current/log_799699-800309 2022-08-22 10:16:24,215 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 411 entries from segment file data_region/47474747-4747-4747-4747-000100000016/current/log_inprogress_884237 2022-08-22 10:16:25,645 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 676 entries from segment file data_region/47474747-4747-4747-4747-000100000011/current/log_799350-800025 2022-08-22 10:17:44,174 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 348 entries from segment file data_region/47474747-4747-4747-4747-000100000011/current/log_inprogress_893749 2022-08-22 10:17:45,097 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 611 entries from segment file data_region/47474747-4747-4747-4747-000100000015/current/log_799514-800124 2022-08-22 10:19:01,113 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 357 entries from segment file data_region/47474747-4747-4747-4747-000100000015/current/log_inprogress_891069 2022-08-22 10:19:02,180 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 613 entries from segment file data_region/47474747-4747-4747-4747-000100000010/current/log_799802-800414 2022-08-22 10:20:20,786 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 345 entries from segment file data_region/47474747-4747-4747-4747-000100000010/current/log_inprogress_893926 2022-08-22 10:20:22,781 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 597 entries from segment file data_region/47474747-4747-4747-4747-000100000013/current/log_799934-800530 2022-08-22 10:21:39,803 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 353 entries from segment file data_region/47474747-4747-4747-4747-000100000013/current/log_inprogress_890063 2022-08-22 10:21:41,877 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 612 entries from segment file data_region/47474747-4747-4747-4747-000100000019/current/log_799991-800602 2022-08-22 10:22:53,057 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 409 entries from segment file data_region/47474747-4747-4747-4747-000100000019/current/log_inprogress_888402 2022-08-22 10:22:54,229 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 601 entries from segment file data_region/47474747-4747-4747-4747-00010000001f/current/log_799807-800407 2022-08-22 10:24:01,915 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 462 entries from segment file data_region/47474747-4747-4747-4747-00010000001f/current/log_inprogress_883049 2022-08-22 10:24:03,156 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 625 entries from segment file data_region/47474747-4747-4747-4747-000100000012/current/log_799686-800310 2022-08-22 10:25:24,340 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 472 entries from segment file data_region/47474747-4747-4747-4747-000100000012/current/log_inprogress_892498 2022-08-22 10:25:25,340 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 615 entries from segment file data_region/47474747-4747-4747-4747-000100000017/current/log_799591-800205 2022-08-22 10:26:36,761 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 468 entries from segment file data_region/47474747-4747-4747-4747-000100000017/current/log_inprogress_888099 2022-08-22 10:26:37,884 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 692 entries from segment file data_region/47474747-4747-4747-4747-00010000001c/current/log_799343-800034 2022-08-22 10:27:55,329 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 464 entries from segment file data_region/47474747-4747-4747-4747-00010000001c/current/log_inprogress_890994 2022-08-22 10:27:56,425 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 600 entries from segment file data_region/47474747-4747-4747-4747-000100000018/current/log_799635-800234 2022-08-22 10:29:11,990 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 377 entries from segment file data_region/47474747-4747-4747-4747-000100000018/current/log_inprogress_889529 2022-08-22 10:29:13,673 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 649 entries from segment file data_region/47474747-4747-4747-4747-00010000001b/current/log_799413-800061 2022-08-22 10:30:36,923 [pool-4-thread-1] INFO o.a.r.s.r.s.LogSegment:173 - Successfully read 521 entries from segment file data_region/47474747-4747-4747-4747-00010000001b/current/log_inprogress_897016 详细日志见附件。 复现流程: 1. 192.168.130.3/4/5 16核32G bm在ip2 iotdb路径:/data2/cluster_test/master_0819_9e0512a 2. 集群配置参数 confignode MAX_HEAP_SIZE="4G" schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus schema_replication_factor=3 data_replication_factor=3 DataNode MAX_HEAP_SIZE="16G" MAX_DIRECT_MEMORY_SIZE="6G" wal_buffer_size_in_byte=1048576 data_dirs=/data1/mpp_data/data max_waiting_time_when_insert_blocked=3600000 3. 启动benchmark,见附件 4. 重启恢复集群,查看日志 -- This message was sent by Atlassian Jira (v8.20.10#820010)