[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-26 Thread Jialin Qiao (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638828#comment-17638828
 ] 

Jialin Qiao commented on IOTDB-4632:


Opeimize the info given to User

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
>  Labels: pull-request-available
> Attachments: more_ts.conf, screenshot-1.png, screenshot-2.png, 
> screenshot-3.png, screenshot-4.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-23 Thread Yukun Zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638145#comment-17638145
 ] 

Yukun Zhou commented on IOTDB-4632:
---

通过分析日志发现（附录截图screenshot-3、4），产生MeasurementAlreadyExist的时候，集群正在执行leader选举，而在leader选举过程中是有可能触发重试的，进而造成序列已存在

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
>  Labels: pull-request-available
> Attachments: more_ts.conf, screenshot-1.png, screenshot-2.png, 
> screenshot-3.png, screenshot-4.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-22 Thread Jira 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17637550#comment-17637550
 ] 

刘珍 commented on IOTDB-4632:
---

master_1123_32e2f98
元数据路径已存在问题还存在，部分元数据创建失败
 !screenshot-2.png! 

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
>  Labels: pull-request-available
> Attachments: more_ts.conf, screenshot-1.png, screenshot-2.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-17 Thread Jira 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635207#comment-17635207
 ] 

刘珍 commented on IOTDB-4632:
---

master_1117_92c6a57,
schema_region_ratis_request_timeout_ms=360
connection_timeout_ms=360
cn_connection_timeout_ms=360
有1万条MetaData error 
2022-11-17 16:30:00,756 [3@group-00020008-StateMachineUpdater] ERROR 
o.a.i.d.m.v.SchemaExecutionVisitor:108 - IoTDB: MetaData error:
org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
[root.test.g_8.d_328.s_8] already exist
at 
org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:234)
at 
org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:586)
at 
org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:105)
at 
org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:64)
at 
org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:156)
at 
org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:73)
at 
org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:137)
at 
org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1672)
at 
org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
at 
org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
at java.lang.Thread.run(Thread.java:748)


> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
>  Labels: pull-request-available
> Attachments: more_ts.conf, screenshot-1.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
>

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-02 Thread Yukun Zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628036#comment-17628036
 ] 

Yukun Zhou commented on IOTDB-4632:
---

一个可选的方案是，系统内部执行时，将超大的请求拆分为若干个较轻量的小请求然后进行稳定消费和执行。

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
> Attachments: more_ts.conf, screenshot-1.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-11-01 Thread Yukun Zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17627408#comment-17627408
 ] 

Yukun Zhou commented on IOTDB-4632:
---

复测结果中发生了10次的ratis转发的重试，导致产生了100w条序列已存在的报错日志。

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
> Attachments: more_ts.conf, screenshot-1.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4632) [ MetaData ] org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path [xx] already exist

2022-10-24 Thread Jira 


[ 
https://issues.apache.org/jira/browse/IOTDB-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623497#comment-17623497
 ] 

刘珍 commented on IOTDB-4632:
---

master_1023_2fea011 ，加大ratis request timeout ，依然有大量元数据path已存在的ERROR
2022-10-25 09:30:21,874 [5@group-00020006-StateMachineUpdater] ERROR 
o.a.i.d.m.v.SchemaExecutionVisitor:105 - IoTDB: MetaData error:
org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
[root.test.g_6.d_106.s_48388] already exist

  !screenshot-1.png! 

> [ MetaData ] 
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [xx] already exist
> -
>
> Key: IOTDB-4632
> URL: https://issues.apache.org/jira/browse/IOTDB-4632
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
> Attachments: more_ts.conf, screenshot-1.png
>
>
> master_1010_9917053
> schema region ： ratis
> 3副本
> 500dev，10万sensor/dev , enable_auto_create_schema=false
> 注册元数据，大量ERROR，部分dev没注册成功（ratis 超时？超时时间需可配置）
> ERROR 信息：
> 2022-10-13 11:37:36,676 
> [192.168.10.62_50010@group-00020001-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:109 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.MeasurementAlreadyExistException: Path 
> [root.test.g_1.d_221.s_48893] already exist
> at 
> org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.createTimeseries(MTreeBelowSGMemoryImpl.java:218)
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:622)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:106)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitCreateMultiTimeSeries(SchemaExecutionVisitor.java:63)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.CreateMultiTimeSeriesNode.accept(CreateMultiTimeSeriesNode.java:141)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:74)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:135)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1588)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.lang.Thread.run(Thread.java:748)
> 测试环境
> 1. 192.168.10.62/66/68  物理机  72CPU 256GB
> bm在64
> ConfigNode
> MAX_HEAP_SIZE="8G"
> MAX_DIRECT_MEMORY_SIZE="8G"
>  
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
>  data_replication_factor=3
> connection_timeout_ms=120
> DataNode
>  MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> connection_timeout_ms=120
> max_connection_for_internal_service=1100
>  enable_timed_flush_seq_memtable=true
> seq_memtable_flush_interval_in_ms=360
> seq_memtable_flush_check_interval_in_ms=60
> enable_timed_flush_unseq_memtable=true
>  unseq_memtable_flush_interval_in_ms=360
> unseq_memtable_flush_check_interval_in_ms=60
>  max_waiting_time_when_insert_blocked=360
>  query_timeout_threshold=3600
>  enable_auto_create_schema=false
> 2. bm配置见附件



--
This message was sent by Atlassian Jira
(v8.20.10#820010)