[jira] [Commented] (IOTDB-5111) [ ratis ] Data is distributed across disks ,after the cluster is restarted, all data is lost
[ https://issues.apache.org/jira/browse/IOTDB-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653841#comment-17653841 ] Houliang Qi commented on IOTDB-5111: Hi, data loss is *BIG* issue in the product environment, so when does the issue happen? if the user specifies multi dn_data_dirs which across many disks, will this happen? > [ ratis ] Data is distributed across disks ,after the cluster is restarted, > all data is lost > > > Key: IOTDB-5111 > URL: https://issues.apache.org/jira/browse/IOTDB-5111 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 1.0.0 >Reporter: 刘珍 >Assignee: Song Ziyang >Priority: Major > Attachments: image-2022-12-02-17-58-45-096.png, > image-2022-12-02-17-59-05-010.png > > > rel/1.0 > config/schema/data 3个协议均是ratis, > dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data > 跨盘存储, > 写入数据,重启集群,{color:#DE350B}*数据全部丢失*{color}。 > 还有1个问题,{color:#DE350B}snapshot目录下依然有.tmp.文件夹名称{color}: > !image-2022-12-02-17-59-05-010.png! > 测试环境-私有云1期 8C32GB > 1. 3副本3C7D > Common > data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > schema_replication_factor=3 > data_replication_factor=3 > wal_buffer_size_in_byte=1048576 > max_waiting_time_when_insert_blocked=360 > query_timeout_threshold=3600 > ConfigNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > DataNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data > 2. 启动BM 写入数据 > GROUP_NUMBER=1 > DEVICE_NUMBER=1000 > REAL_INSERT_RATE=1.0 > SENSOR_NUMBER=1000 > IS_SENSOR_TS_ALIGNMENT=true > IS_OUT_OF_ORDER=false > OUT_OF_ORDER_RATIO=0.5 > OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0 > CLIENT_NUMBER=50 > LOOP=1 > BATCH_SIZE_PER_WRITE=10 > START_TIME=2018-8-30T00:00:00+08:00 > POINT_STEP=200 > OP_MIN_INTERVAL=0 > OP_MIN_INTERVAL_RANDOM=false > INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1 > ENCODINGS=PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN > COMPRESSOR=SNAPPY > IS_DELETE_DATA=false > CREATE_SCHEMA=true > BENCHMARK_CLUSTER=false > !image-2022-12-02-17-58-45-096.png! > 3. 重启集群 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5111) [ ratis ] Data is distributed across disks ,after the cluster is restarted, all data is lost
[ https://issues.apache.org/jira/browse/IOTDB-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649532#comment-17649532 ] 刘珍 commented on IOTDB-5111: --- 可以等支持跨盘后,再resolved。 > [ ratis ] Data is distributed across disks ,after the cluster is restarted, > all data is lost > > > Key: IOTDB-5111 > URL: https://issues.apache.org/jira/browse/IOTDB-5111 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 1.0.0 >Reporter: 刘珍 >Assignee: Song Ziyang >Priority: Major > Attachments: image-2022-12-02-17-58-45-096.png, > image-2022-12-02-17-59-05-010.png > > > rel/1.0 > config/schema/data 3个协议均是ratis, > dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data > 跨盘存储, > 写入数据,重启集群,{color:#DE350B}*数据全部丢失*{color}。 > 还有1个问题,{color:#DE350B}snapshot目录下依然有.tmp.文件夹名称{color}: > !image-2022-12-02-17-59-05-010.png! > 测试环境-私有云1期 8C32GB > 1. 3副本3C7D > Common > data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > schema_replication_factor=3 > data_replication_factor=3 > wal_buffer_size_in_byte=1048576 > max_waiting_time_when_insert_blocked=360 > query_timeout_threshold=3600 > ConfigNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > DataNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data > 2. 启动BM 写入数据 > GROUP_NUMBER=1 > DEVICE_NUMBER=1000 > REAL_INSERT_RATE=1.0 > SENSOR_NUMBER=1000 > IS_SENSOR_TS_ALIGNMENT=true > IS_OUT_OF_ORDER=false > OUT_OF_ORDER_RATIO=0.5 > OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0 > CLIENT_NUMBER=50 > LOOP=1 > BATCH_SIZE_PER_WRITE=10 > START_TIME=2018-8-30T00:00:00+08:00 > POINT_STEP=200 > OP_MIN_INTERVAL=0 > OP_MIN_INTERVAL_RANDOM=false > INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1 > ENCODINGS=PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN > COMPRESSOR=SNAPPY > IS_DELETE_DATA=false > CREATE_SCHEMA=true > BENCHMARK_CLUSTER=false > !image-2022-12-02-17-58-45-096.png! > 3. 重启集群 -- This message was sent by Atlassian Jira (v8.20.10#820010)