[jira] [Commented] (IOTDB-5111) [ ratis ] Data is distributed across disks ,after the cluster is restarted, all data is lost

2023-01-02 Thread Houliang Qi (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653841#comment-17653841
 ] 

Houliang Qi commented on IOTDB-5111:


Hi, data loss is *BIG* issue in the product environment, so when does the issue 
happen? if the user specifies multi dn_data_dirs which across many disks, will 
this happen?

> [ ratis ] Data is distributed across disks ,after the cluster is restarted, 
> all data is lost
> 
>
> Key: IOTDB-5111
> URL: https://issues.apache.org/jira/browse/IOTDB-5111
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 1.0.0
>Reporter: 刘珍
>Assignee: Song Ziyang
>Priority: Major
> Attachments: image-2022-12-02-17-58-45-096.png, 
> image-2022-12-02-17-59-05-010.png
>
>
> rel/1.0
> config/schema/data 3个协议均是ratis,
> dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data
> 跨盘存储,
> 写入数据,重启集群,{color:#DE350B}*数据全部丢失*{color}。
> 还有1个问题,{color:#DE350B}snapshot目录下依然有.tmp.文件夹名称{color}:
>  !image-2022-12-02-17-59-05-010.png! 
> 测试环境-私有云1期  8C32GB
> 1. 3副本3C7D
> Common
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> schema_replication_factor=3
> data_replication_factor=3
> wal_buffer_size_in_byte=1048576
> max_waiting_time_when_insert_blocked=360
> query_timeout_threshold=3600
> ConfigNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> DataNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data
> 2. 启动BM 写入数据
> GROUP_NUMBER=1
> DEVICE_NUMBER=1000
> REAL_INSERT_RATE=1.0
> SENSOR_NUMBER=1000
> IS_SENSOR_TS_ALIGNMENT=true
> IS_OUT_OF_ORDER=false
> OUT_OF_ORDER_RATIO=0.5
> OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
> CLIENT_NUMBER=50
> LOOP=1
> BATCH_SIZE_PER_WRITE=10
> START_TIME=2018-8-30T00:00:00+08:00
> POINT_STEP=200
> OP_MIN_INTERVAL=0
> OP_MIN_INTERVAL_RANDOM=false
> INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
> ENCODINGS=PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN
> COMPRESSOR=SNAPPY
> IS_DELETE_DATA=false
> CREATE_SCHEMA=true
> BENCHMARK_CLUSTER=false
>  !image-2022-12-02-17-58-45-096.png! 
> 3. 重启集群



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5111) [ ratis ] Data is distributed across disks ,after the cluster is restarted, all data is lost

2022-12-19 Thread Jira


[ 
https://issues.apache.org/jira/browse/IOTDB-5111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17649532#comment-17649532
 ] 

刘珍 commented on IOTDB-5111:
---

可以等支持跨盘后,再resolved。

> [ ratis ] Data is distributed across disks ,after the cluster is restarted, 
> all data is lost
> 
>
> Key: IOTDB-5111
> URL: https://issues.apache.org/jira/browse/IOTDB-5111
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 1.0.0
>Reporter: 刘珍
>Assignee: Song Ziyang
>Priority: Major
> Attachments: image-2022-12-02-17-58-45-096.png, 
> image-2022-12-02-17-59-05-010.png
>
>
> rel/1.0
> config/schema/data 3个协议均是ratis,
> dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data
> 跨盘存储,
> 写入数据,重启集群,{color:#DE350B}*数据全部丢失*{color}。
> 还有1个问题,{color:#DE350B}snapshot目录下依然有.tmp.文件夹名称{color}:
>  !image-2022-12-02-17-59-05-010.png! 
> 测试环境-私有云1期  8C32GB
> 1. 3副本3C7D
> Common
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> schema_replication_factor=3
> data_replication_factor=3
> wal_buffer_size_in_byte=1048576
> max_waiting_time_when_insert_blocked=360
> query_timeout_threshold=3600
> ConfigNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> DataNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data
> 2. 启动BM 写入数据
> GROUP_NUMBER=1
> DEVICE_NUMBER=1000
> REAL_INSERT_RATE=1.0
> SENSOR_NUMBER=1000
> IS_SENSOR_TS_ALIGNMENT=true
> IS_OUT_OF_ORDER=false
> OUT_OF_ORDER_RATIO=0.5
> OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
> CLIENT_NUMBER=50
> LOOP=1
> BATCH_SIZE_PER_WRITE=10
> START_TIME=2018-8-30T00:00:00+08:00
> POINT_STEP=200
> OP_MIN_INTERVAL=0
> OP_MIN_INTERVAL_RANDOM=false
> INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
> ENCODINGS=PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN
> COMPRESSOR=SNAPPY
> IS_DELETE_DATA=false
> CREATE_SCHEMA=true
> BENCHMARK_CLUSTER=false
>  !image-2022-12-02-17-58-45-096.png! 
> 3. 重启集群



--
This message was sent by Atlassian Jira
(v8.20.10#820010)