[jira] [Created] (IOTDB-4915) [trigger]must have a way to drop bad trigger

2022-11-10 Thread changxue (Jira)
changxue created IOTDB-4915:
---

 Summary: [trigger]must have a way to drop bad trigger
 Key: IOTDB-4915
 URL: https://issues.apache.org/jira/browse/IOTDB-4915
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: changxue
Assignee: Yuan Tian


[trigger]trigger could not drop if onDrop throws exception

 

environment:

3C3D cluster

 

consider this 2 scenarios below:

1. If I develop a trigger class and I make some mistakes in onDrop method,  I'd 
like to drop it and update it, but I could not.

2. If I send message in onDrop and it couldn't connect now, the trigger would 
be updated or dropped, but I could not.

 

So, it must be a way to drop useless or bad trigger.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4914) [trigger]drop stateful trigger failed: this trigger has not been created

2022-11-10 Thread Jira


 [ 
https://issues.apache.org/jira/browse/IOTDB-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

李伟豪 reassigned IOTDB-4914:
--

Assignee: 李伟豪  (was: liaolanyu)

> [trigger]drop stateful trigger failed: this trigger has not been created
> 
>
> Key: IOTDB-4914
> URL: https://issues.apache.org/jira/browse/IOTDB-4914
> Project: Apache IoTDB
>  Issue Type: Bug
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: changxue
>Assignee: 李伟豪
>Priority: Major
>
> [trigger]drop stateful trigger failed: this trigger has not been created
> environment:
> 3C3D cluster, Nov.11 master source codes
> 之前没有这个问题,今天突然出现的。
> reproduction:
> {code:java}
> CREATE STATEful TRIGGER `stateful-beforeInsert` before INSERT ON 
> root.stateful.** AS 'com.abc.trigger.stateful.test.StatefulTriggerTest' USING 
> URI 'file:///data/trigger/stateful-test-0.14-SNAPSHOT.jar' WITH ( 
> 'HOST'='172.20.70.44', 'data_type'='double', 'standard_value'=100, 
> 'trig_name'='stateful-beforeInsert');
> show triggers;
> drop trigger `stateful-beforeInsert`;
> {code}
> failed msg:
> Msg: 923: Failed to drop trigger [stateful-beforInsert], this trigger has not 
> been created
> datanode log:
> {code:java}
> 2022-11-11 14:37:06,709 
> [pool-25-IoTDB-ClientRPC-Processor-1$2022_063706_00031_1] ERROR 
> o.a.i.d.m.p.e.c.e.ClusterConfigTaskExecutor:553 - [TSStatus(code:923, 
> message:Failed to drop trigger [stateful-beforInsert], this trigger has not 
> been created)] Failed to drop trigger stateful-beforInsert. 
> 2022-11-11 14:37:06,710 [pool-21-IoTDB-MPPCoordinator-20] ERROR 
> o.a.i.d.m.p.e.c.ConfigExecution:132 - Failures happened during running 
> ConfigExecution. 
> org.apache.iotdb.commons.exception.IoTDBException: Failed to drop trigger 
> [stateful-beforInsert], this trigger has not been created
>     at 
> org.apache.iotdb.db.mpp.plan.execution.config.executor.ClusterConfigTaskExecutor.dropTrigger(ClusterConfigTaskExecutor.java:554)
>     at 
> org.apache.iotdb.db.mpp.plan.execution.config.metadata.DropTriggerTask.execute(DropTriggerTask.java:39)
>     at 
> org.apache.iotdb.db.mpp.plan.execution.config.ConfigExecution.start(ConfigExecution.java:107)
>     at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:150)
>     at 
> org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:215)
>     at 
> org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:390)
>     at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3453)
>     at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3433)
>     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
>     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
>     at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
>     at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>     at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>     at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4914) [trigger]drop stateful trigger failed: this trigger has not been created

2022-11-10 Thread changxue (Jira)
changxue created IOTDB-4914:
---

 Summary: [trigger]drop stateful trigger failed: this trigger has 
not been created
 Key: IOTDB-4914
 URL: https://issues.apache.org/jira/browse/IOTDB-4914
 Project: Apache IoTDB
  Issue Type: Bug
Affects Versions: 0.14.0-SNAPSHOT
Reporter: changxue
Assignee: liaolanyu


[trigger]drop stateful trigger failed: this trigger has not been created

environment:
3C3D cluster, Nov.11 master source codes

之前没有这个问题,今天突然出现的。

reproduction:
{code:java}
CREATE STATEful TRIGGER `stateful-beforeInsert` before INSERT ON 
root.stateful.** AS 'com.abc.trigger.stateful.test.StatefulTriggerTest' USING 
URI 'file:///data/trigger/stateful-test-0.14-SNAPSHOT.jar' WITH ( 
'HOST'='172.20.70.44', 'data_type'='double', 'standard_value'=100, 
'trig_name'='stateful-beforeInsert');

show triggers;

drop trigger `stateful-beforeInsert`;
{code}
failed msg:
Msg: 923: Failed to drop trigger [stateful-beforInsert], this trigger has not 
been created

datanode log:
{code:java}
2022-11-11 14:37:06,709 
[pool-25-IoTDB-ClientRPC-Processor-1$2022_063706_00031_1] ERROR 
o.a.i.d.m.p.e.c.e.ClusterConfigTaskExecutor:553 - [TSStatus(code:923, 
message:Failed to drop trigger [stateful-beforInsert], this trigger has not 
been created)] Failed to drop trigger stateful-beforInsert. 
2022-11-11 14:37:06,710 [pool-21-IoTDB-MPPCoordinator-20] ERROR 
o.a.i.d.m.p.e.c.ConfigExecution:132 - Failures happened during running 
ConfigExecution. 
org.apache.iotdb.commons.exception.IoTDBException: Failed to drop trigger 
[stateful-beforInsert], this trigger has not been created
    at 
org.apache.iotdb.db.mpp.plan.execution.config.executor.ClusterConfigTaskExecutor.dropTrigger(ClusterConfigTaskExecutor.java:554)
    at 
org.apache.iotdb.db.mpp.plan.execution.config.metadata.DropTriggerTask.execute(DropTriggerTask.java:39)
    at 
org.apache.iotdb.db.mpp.plan.execution.config.ConfigExecution.start(ConfigExecution.java:107)
    at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:150)
    at 
org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:215)
    at 
org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:390)
    at 
org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3453)
    at 
org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3433)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
    at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4913) [write] instert null to timeseries throws NPE

2022-11-10 Thread changxue (Jira)
changxue created IOTDB-4913:
---

 Summary: [write] instert null to timeseries throws NPE
 Key: IOTDB-4913
 URL: https://issues.apache.org/jira/browse/IOTDB-4913
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Server
Affects Versions: 0.14.0-SNAPSHOT
Reporter: changxue
Assignee: Haonan Hou


 [write] instert null to timeseries throws NPE

environment:
3C3D cluster, Nov.11 master source codes

reproduction:
{code}
CREATE TIMESERIES root.sg1.d1.s1 WITH DATATYPE=double, ENCODING=PLAIN;
INSERT INTO root.sg1.d1(timestamp, s1) VALUES (6, 
10),(7,12),(8,14),(9,160),(10,null),(11,58);

SQLSTATE : 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "INSERT INTO 
root.sg1.d1(timestamp, s1) VALUES (6, 
10),(7,12),(8,14),(9,160),(10,null),(11,58)". executeStatement failed. null

{code}

{code:java}
2022-11-11 13:51:03,628 [pool-25-IoTDB-ClientRPC-Processor-3] ERROR 
o.a.i.d.u.ErrorHandlingUtils:60 - Status code: INTERNAL_SERVER_ERROR(500), 
operation: "INSERT INTO root.sg1.d1(timestamp, s1) VALUES (6, 
10),(7,12),(8,14),(9,160),(10,null),(11,58)". executeStatement failed 
java.lang.NullPointerException: null
    at java.util.HashMap.merge(HashMap.java:1225)
    at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
    at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
    at 
java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    at 
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at 
org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertRowsOfOneDeviceNode.storeMeasurementsAndDataType(InsertRowsOfOneDeviceNode.java:212)
    at 
org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertRowsOfOneDeviceNode.setInsertRowNodeList(InsertRowsOfOneDeviceNode.java:121)
    at 
org.apache.iotdb.db.mpp.plan.planner.LogicalPlanVisitor.visitInsertRowsOfOneDevice(LogicalPlanVisitor.java:628)
    at 
org.apache.iotdb.db.mpp.plan.planner.LogicalPlanVisitor.visitInsertRowsOfOneDevice(LogicalPlanVisitor.java:79)
    at 
org.apache.iotdb.db.mpp.plan.statement.crud.InsertRowsOfOneDeviceStatement.accept(InsertRowsOfOneDeviceStatement.java:99)
    at 
org.apache.iotdb.db.mpp.plan.statement.StatementVisitor.process(StatementVisitor.java:107)
    at 
org.apache.iotdb.db.mpp.plan.planner.LogicalPlanner.plan(LogicalPlanner.java:42)
    at 
org.apache.iotdb.db.mpp.plan.execution.QueryExecution.doLogicalPlan(QueryExecution.java:293)
    at 
org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:199)
    at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:150)
    at 
org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementInternal(ClientRPCServiceImpl.java:215)
    at 
org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.executeStatementV2(ClientRPCServiceImpl.java:390)
    at 
org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3453)
    at 
org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$executeStatementV2.getResult(IClientRPCService.java:3433)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
    at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
2022-11-11 13:51:53,877 [pool-3-IoTDB-timedQuerySqlCount-1] INFO  
o.a.i.d.s.b.QueryFrequencyRecorder:44 - Query count in current 1 minute 1   
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4912) Support aggregating partition query result when using align by device

2022-11-10 Thread Yongzao Dan (Jira)
Yongzao Dan created IOTDB-4912:
--

 Summary: Support aggregating partition query result when using 
align by device
 Key: IOTDB-4912
 URL: https://issues.apache.org/jira/browse/IOTDB-4912
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Yongzao Dan
Assignee: Jinrui Zhang
 Fix For: 0.14.0
 Attachments: image-2022-11-11-12-08-01-912.png, 
image-2022-11-11-12-08-29-760.png, image-2022-11-11-12-08-36-384.png, 
image-2022-11-11-12-08-48-000.png, image-2022-11-11-12-08-51-813.png

Currently, the result of aggregating partition query is separated. Shown as the 
following pictures:

With `align by device`:
!image-2022-11-11-12-08-29-760.png!

 

!image-2022-11-11-12-08-36-384.png!

 

Without `align by device`:

!image-2022-11-11-12-08-48-000.png!

 

!image-2022-11-11-12-08-51-813.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4911) [0.13]After the operationSync function is synchronized, data on disks occupies different space

2022-11-10 Thread Zhijia Cao (Jira)
Zhijia Cao created IOTDB-4911:
-

 Summary: [0.13]After the operationSync function is synchronized, 
data on disks occupies different space 
 Key: IOTDB-4911
 URL: https://issues.apache.org/jira/browse/IOTDB-4911
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Server
Reporter: Zhijia Cao
 Attachments: image-2022-11-11-12-06-10-269.png, 
image-2022-11-11-12-06-28-651.png

Write data to IOTDB A with the Benchmark tool on OperationSync and stop writing 
after about 10 minutes. It is found that the space occupied by data in IOTDB A 
is 1.6G, and that in another IOTDB B is 3.1G. In theory, the amount of disk 
space used by data should be the same or very small.

!image-2022-11-11-12-06-10-269.png!!image-2022-11-11-12-06-28-651.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4556) [Remove-DataNode] ERROR o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:287 - The consensus group DataRegion[24] doesn't exist

2022-11-10 Thread Haiming Zhu (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632051#comment-17632051
 ] 

Haiming Zhu commented on IOTDB-4556:


DataRegion[24] exists on the ip73, we can demonstrate this by the log of 
creating new TsFileProcessor 

!ip73_dataregion24.png!

I assume that ip73、ip74、IP75 are one consensus group and ip73 is the leader 
because ip73 syncs data to ip74 and ip75 before the remove datanode 
operation(09-29 14:10)

!73to74.png!!73to75.png!

 But, after the remove datanode operation(09-29 14:10), we can see log below

!73to76.png!

Maybe this issue occurs when the leader is removed and there are some errors 
when migrating data from ip73 to ip76.

> [Remove-DataNode] ERROR o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:287 - 
> The consensus group DataRegion[24] doesn't exist
> ---
>
> Key: IOTDB-4556
> URL: https://issues.apache.org/jira/browse/IOTDB-4556
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Haiming Zhu
>Priority: Major
> Attachments: 73to74.png, 73to75.png, 73to76.png, 
> ip73_dataregion24.png, more_dev.conf
>
>
> m_0929_71d5f65
> SchemaRegion : ratis
> DataRegion : multiLeader
> 均为3副本,3C5D
> 启动客户端bm写入,缩容期间写入不停。
> bm运行40分钟,缩容节点1(ip72),1小时38分钟缩容成功。
> mv 节点1的data ,logs,再上线。
> 缩容节点2(ip73,开始缩容的时间09-29 14:10),此节点不包含DataRegion[24]
> *{color:#DE350B}DataRegion[24]在ip74,ip75,ip76{color}*
> 但是ip73 error :
> 2022-09-29 14:23:39,273 
> [pool-24-IoTDB-ClientRPC-Processor-2$20220929_062339_48081_4.1.0] 
> {color:#DE350B}*ERROR o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:287 - The 
> consensus group DataRegion[24] doesn't exist*{color}
> 2022-09-29 14:23:39,275 [MultiLeaderConsensusClientPool-selector-98] ERROR 
> o.a.i.c.m.l.IndexController:111 - {color:#DE350B}*failed to flush sync index. 
> cannot find previous version file. previous: 93500*{color}
> 2022-09-29 14:23:39,179 [pool-24-IoTDB-ClientRPC-Processor-45] WARN  
> o.a.i.d.u.ErrorHandlingUtils:62 - Status code: EXECUTE_STATEMENT_ERROR(400), 
> operation: insertTablet failed
> java.lang.RuntimeException: 
> org.apache.iotdb.commons.exception.IoTDBException: There are no available 
> RegionGroups currently, please check the status of cluster DataNodes
> at 
> org.apache.iotdb.db.mpp.plan.analyze.ClusterPartitionFetcher.getOrCreateDataPartition(ClusterPartitionFetcher.java:280)
> at 
> org.apache.iotdb.db.mpp.plan.analyze.AnalyzeVisitor.visitInsertTablet(AnalyzeVisitor.java:1236)
> at 
> org.apache.iotdb.db.mpp.plan.analyze.AnalyzeVisitor.visitInsertTablet(AnalyzeVisitor.java:150)
> at 
> org.apache.iotdb.db.mpp.plan.statement.crud.InsertTabletStatement.accept(InsertTabletStatement.java:121)
> at 
> org.apache.iotdb.db.mpp.plan.statement.StatementVisitor.process(StatementVisitor.java:98)
> at 
> org.apache.iotdb.db.mpp.plan.analyze.Analyzer.analyze(Analyzer.java:40)
> at 
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.analyze(QueryExecution.java:236)
> at 
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.(QueryExecution.java:138)
> at 
> org.apache.iotdb.db.mpp.plan.Coordinator.createQueryExecution(Coordinator.java:100)
> at 
> org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:133)
> at 
> org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:160)
> at 
> org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.insertTablet(ClientRPCServiceImpl.java:996)
> at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:3512)
> at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:3492)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: org.apache.iotdb.commons.exception.IoTDBException: There are no 
> available RegionGroups currently, please check the status of cluster DataNodes
> ... 20 common frames omitted
> 测试环境
> 1. 192.168.10.72/73/74/75/76  48CPU384GB
> 3C : 72,73,74
> 5D : 72 ,73,74,75,76
> 集群配置
> ConfigNode 
> MAX_HEAP_SIZE="8G"
> 

[jira] [Commented] (IOTDB-4858) [Update] Can not restart IoTDB with (old)data

2022-11-10 Thread Gaofei Cao (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632050#comment-17632050
 ] 

Gaofei Cao commented on IOTDB-4858:
---

Will be solved in https://issues.apache.org/jira/browse/IOTDB-4333

> [Update] Can not restart IoTDB with (old)data
> -
>
> Key: IOTDB-4858
> URL: https://issues.apache.org/jira/browse/IOTDB-4858
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: FengQingxin
>Assignee: Gaofei Cao
>Priority: Major
> Attachments: image-2022-11-07-11-18-18-380.png
>
>
> commit:abbb45b891dbbd3ca8c30a96fa5b858156117f1b
> Reproduce:
> 1.Build IoTDB with commit :abbb45b891dbbd3ca8c30a96fa5b858156117f1b
> 2. Copy data folder to new instance.
> 3.Modify the system properties file 
> 4.using start-confignode.sh and start-datanode.sh to start 1c1d with old data;
> got an error like below:
> Successfully update ConfigNode: [TEndPoint(ip:127.0.0.1, port:22277)].
> Successfully update ConfigNode: [TEndPoint(ip:0.0.0.0, port:22277)].
> !image-2022-11-07-11-18-18-380.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4858) [Update] Can not restart IoTDB with (old)data

2022-11-10 Thread Gaofei Cao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gaofei Cao reassigned IOTDB-4858:
-

Assignee: 伊丹翔  (was: Gaofei Cao)

> [Update] Can not restart IoTDB with (old)data
> -
>
> Key: IOTDB-4858
> URL: https://issues.apache.org/jira/browse/IOTDB-4858
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: FengQingxin
>Assignee: 伊丹翔
>Priority: Major
> Attachments: image-2022-11-07-11-18-18-380.png
>
>
> commit:abbb45b891dbbd3ca8c30a96fa5b858156117f1b
> Reproduce:
> 1.Build IoTDB with commit :abbb45b891dbbd3ca8c30a96fa5b858156117f1b
> 2. Copy data folder to new instance.
> 3.Modify the system properties file 
> 4.using start-confignode.sh and start-datanode.sh to start 1c1d with old data;
> got an error like below:
> Successfully update ConfigNode: [TEndPoint(ip:127.0.0.1, port:22277)].
> Successfully update ConfigNode: [TEndPoint(ip:0.0.0.0, port:22277)].
> !image-2022-11-07-11-18-18-380.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4910) [Benchmark] Fix the concurrency problem of GenerateQueryWorkLoad

2022-11-10 Thread Jira
张洪胤 created IOTDB-4910:
--

 Summary: [Benchmark] Fix the concurrency problem of 
GenerateQueryWorkLoad
 Key: IOTDB-4910
 URL: https://issues.apache.org/jira/browse/IOTDB-4910
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: 张洪胤
Assignee: 张洪胤






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4809) [Broadcast partitionCache on removed datanode] ConsensusGroupNotExistException: The consensus group DataRegion[11] doesn't exist

2022-11-10 Thread Gaofei Cao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gaofei Cao reassigned IOTDB-4809:
-

Assignee: Gaofei Cao  (was: 陈哲涵)

> [Broadcast partitionCache on removed datanode] 
> ConsensusGroupNotExistException: The consensus group DataRegion[11] doesn't 
> exist
> 
>
> Key: IOTDB-4809
> URL: https://issues.apache.org/jira/browse/IOTDB-4809
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Gaofei Cao
>Priority: Major
> Attachments: after_remove_regions_info.out, 
> before_remove_regions_info.out, more_dev.conf, screenshot-1.png, 
> screenshot-2.png
>
>
> m_1031_76b947f
> 3rep , 3C5D
> schema region : ratis
> data region : multiLeader
> {color:#DE350B}*This issue contains 3 bugs*{color}
> benchmark runs for 1 hour and execute remove (ip72) datanode ,  ip72 datanode 
> brushes ERROR logs :
> 2022-10-31 17:48:06,277 
> [pool-25-IoTDB-ClientRPC-Processor-85$20221031_094806_31457_3.1.0] ERROR 
> o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:234 - write locally failed. 
> TSStatus: TSStatus(code:412, 
> message:org.apache.iotdb.consensus.exception.ConsensusGroupNotExistException: 
> The consensus group DataRegion[13] doesn't exist), message: 
> org.apache.iotdb.consensus.exception.ConsensusGroupNotExistException: The 
> consensus group DataRegion[13] doesn't exist
> 2022-10-31 17:48:06,285 
> [pool-25-IoTDB-ClientRPC-Processor-93$20221031_094806_31458_3.1.0] ERROR 
> o.a.i.d.m.e.e.RegionWriteExecutor$WritePlanNodeExecutionVisitor:235 - 
> {color:#DE350B}*Something wrong happened while calling consensus layer's 
> write API.
> org.apache.iotdb.consensus.exception.ConsensusGroupNotExistException: The 
> consensus group DataRegion[11] doesn't exist*{color}
> at 
> org.apache.iotdb.consensus.multileader.MultiLeaderConsensus.write(MultiLeaderConsensus.java:155)
> at 
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor.fireTriggerAndInsert(RegionWriteExecutor.java:101)
> at 
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.executeDataInsert(RegionWriteExecutor.java:215)
> at 
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:163)
> at 
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:117)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertTabletNode.accept(InsertTabletNode.java:1085)
> at 
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor.execute(RegionWriteExecutor.java:83)
> at 
> org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchLocally(FragmentInstanceDispatcherImpl.java:232)
> at 
> org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:137)
> at 
> org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchWriteSync(FragmentInstanceDispatcherImpl.java:119)
> at 
> org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:90)
> at 
> org.apache.iotdb.db.mpp.plan.scheduler.ClusterScheduler.start(ClusterScheduler.java:102)
> at 
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.schedule(QueryExecution.java:283)
> at 
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:201)
> at 
> org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:146)
> at 
> org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:160)
> at 
> org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.insertTablet(ClientRPCServiceImpl.java:1198)
> at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:4078)
> at 
> org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:4058)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at 

[jira] [Assigned] (IOTDB-4752) some data is lost after the read-only node returns to normal

2022-11-10 Thread Gaofei Cao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gaofei Cao reassigned IOTDB-4752:
-

Assignee: 陈哲涵  (was: Gaofei Cao)

> some data is lost after the read-only node returns to normal
> 
>
> Key: IOTDB-4752
> URL: https://issues.apache.org/jira/browse/IOTDB-4752
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: 陈哲涵
>Priority: Major
> Attachments: disk space sample.png, 
> image-2022-10-25-16-22-00-198.png, iotdb_4752.conf, read-only check.png, 
> read-only reason.png, state machine lock.png, tsfile-content.png, 
> tsfiles.png, wal-content.png, 同步完成_ip72.out, 同步完成_ip73_ip74.out
>
>
> m_1025_cbc6225
> schema region : ratis
> data region : multiLeader
> 3rep , 3C3D
> During benchmark writing, ip72 is set to read-only due to "no space left on 
> device". 
> Wait for the benchmark write to complete, ip72 release disk space.
> ip72 : SET SYSTEM TO RUNNING
> Wait for synchronization to complete
> ip72 : flush
> Perform query comparison,ip72 {color:#DE350B}*lost some data*{color}.
> "select count(s_0) ,count(s_9),count(s_99),count(s_999),count(s_) from 
> root.** align by device"
>  !image-2022-10-25-16-22-00-198.png! 
> Test environment :
> 1. 192.168.10.72 / 73 /74  48CPU 384GB
> benchmark : ip75 /home/liuzhen/benchmark/bm_0620_7ec96c1
> iotdb_dir : /ssd_data/mpp_test/m_1025_cbc6225
> ConfigNode
> MAX_HEAP_SIZE="8G"
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=3
> data_replication_factor=3
> connection_timeout_ms=12
> schema_region_ratis_request_timeout_ms=120
> data_region_ratis_request_timeout_ms=120
> schema_region_ratis_max_retry_attempts=1
> data_region_ratis_max_retry_attempts=2
> DataNode
> MAX_HEAP_SIZE="256G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> avg_series_point_number_threshold=1
> max_waiting_time_when_insert_blocked=360
> enable_seq_space_compaction=false
> enable_unseq_space_compaction=false
> enable_cross_space_compaction=false
> query_timeout_threshold=3600
> 2. benchmark configuration
> see attachment .
> 3. During benchmark writing, ip72 is set to read-only due to "no space left 
> on device". 
> 4. Wait for the benchmark write to complete, ip72 release disk space.
> 5.  ip72 : SET SYSTEM TO RUNNING
> 6.  Wait for synchronization to complete
> 7.  ip72 : flush
> 8. Perform query comparison
> select count(s_0) ,count(s_9),count(s_99),count(s_999),count(s_) from 
> root.** align by device



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4830) [ confignode ] remove datanode that has stopped ,confignode executes “DELETE_OLD_REGION_PEER” on this datanode

2022-11-10 Thread Gaofei Cao (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gaofei Cao reassigned IOTDB-4830:
-

Assignee: Gaofei Cao  (was: 陈哲涵)

> [ confignode ] remove datanode that has stopped ,confignode executes 
> “DELETE_OLD_REGION_PEER” on this datanode
> --
>
> Key: IOTDB-4830
> URL: https://issues.apache.org/jira/browse/IOTDB-4830
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Gaofei Cao
>Priority: Major
> Fix For: 0.14.0-SNAPSHOT
>
> Attachments: image-2022-11-02-14-55-28-013.png, iotdb_4830.conf
>
>
> m_1102_09e2566
> 1. 启动3副本 , 3C5D集群
> 2.调用stop-datanode.sh脚本正常停止ip76的 datanode 
> 3. benchmark写入数据完成
> 4. 缩容下线的ip76的datanode
> confignode 会重试连接ip76 
> ,并且有DELETE_OLD_REGION_PEER重试操作,DELETE_OLD_REGION_PEER可以不执行,因为不是缩容开始后的重试 :
> 2022-11-02 14:34:23,637 [ProcExecWorker-9] ERROR 
> o.a.i.c.c.s.SyncDataNodeClientPool:113 - 
> {color:#DE350B}*DELETE_OLD_REGION_PEER*{color} failed on DataNode 
> TEndPoint(ip:192.168.10.76, port:9003) 
> 5. 启动 ip76 datanode , 可以看到remove开始在 ip76上执行 ,但此时此节点的状态却是Running, 应该是Removing。
> ip76 datanode log (已经在执行remove了):
> 2022-11-02 14:38:45,611 [pool-53-IoTDB-Region-Migrate-Pool-1] INFO  
> o.a.i.d.s.RegionMigrateService$DeleteOldRegionPeerTask:493 - succeed to 
> remove region DataRegion[12] consensus group
> 此时集群节点状态:
>  !image-2022-11-02-14-55-28-013.png! 
> TEST ENV
> 192.168.10.72~76



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4909) [0.14][cluster][datanode] log-all do not display sql statements

2022-11-10 Thread ZhangZhengming (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhangZhengming reassigned IOTDB-4909:
-

  Sprint: 2022-11-Cluster
Assignee: Yuan Tian

> [0.14][cluster][datanode] log-all do not display sql statements
> ---
>
> Key: IOTDB-4909
> URL: https://issues.apache.org/jira/browse/IOTDB-4909
> Project: Apache IoTDB
>  Issue Type: Bug
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: ZhangZhengming
>Assignee: Yuan Tian
>Priority: Minor
> Attachments: image-2022-11-11-10-18-11-520.png
>
>
>  
> !image-2022-11-11-10-18-11-520.png|width=1181,height=400!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4909) [0.14][cluster][datanode] log-all do not display sql statements

2022-11-10 Thread ZhangZhengming (Jira)
ZhangZhengming created IOTDB-4909:
-

 Summary: [0.14][cluster][datanode] log-all do not display sql 
statements
 Key: IOTDB-4909
 URL: https://issues.apache.org/jira/browse/IOTDB-4909
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: ZhangZhengming
 Attachments: image-2022-11-11-10-18-11-520.png

 

!image-2022-11-11-10-18-11-520.png|width=1181,height=400!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4908) Clear deprecayed code related to sync in old standalone

2022-11-10 Thread yanze chen (Jira)
yanze chen created IOTDB-4908:
-

 Summary: Clear deprecayed code related to sync in old standalone
 Key: IOTDB-4908
 URL: https://issues.apache.org/jira/browse/IOTDB-4908
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4907) Change show devices with storage group to database

2022-11-10 Thread Minghui Liu (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632008#comment-17632008
 ] 

Minghui Liu commented on IOTDB-4907:


https://github.com/apache/iotdb/pull/7946/commits/9a0ab8dd5634348a77c0c7aadd012fd0be419cbd

> Change show devices with storage group to database
> --
>
> Key: IOTDB-4907
> URL: https://issues.apache.org/jira/browse/IOTDB-4907
> Project: Apache IoTDB
>  Issue Type: Task
>Reporter: Yuan Tian
>Assignee: Minghui Liu
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Reopened] (IOTDB-4771) BufferedPipeDataQueueTest.testOfferWhileTakingWithDiscontinuousSerialNumber fail

2022-11-10 Thread Haonan Hou (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haonan Hou reopened IOTDB-4771:
---

> BufferedPipeDataQueueTest.testOfferWhileTakingWithDiscontinuousSerialNumber 
> fail
> 
>
> Key: IOTDB-4771
> URL: https://issues.apache.org/jira/browse/IOTDB-4771
> Project: Apache IoTDB
>  Issue Type: Bug
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: Haonan Hou
>Assignee: yusicheng
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2022-10-27-10-08-03-147.png
>
>
> !image-2022-10-27-10-08-03-147.png|width=556,height=95!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4864) [template]Active templates should not be unset directly

2022-11-10 Thread Yukun Zhou (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632006#comment-17632006
 ] 

Yukun Zhou commented on IOTDB-4864:
---

https://github.com/apache/iotdb/pull/7943

> [template]Active templates should not be unset directly
> ---
>
> Key: IOTDB-4864
> URL: https://issues.apache.org/jira/browse/IOTDB-4864
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: xiaozhihong
>Assignee: Yukun Zhou
>Priority: Major
>
> In 0.14.0 snapshot version,enter cli to execute sql about template.
> During the execution process, the template in the active state can still be 
> unset directly.
> The SQL is:
>  
> {code:java}
> create schema template t2 aligned (lat FLOAT encoding=Gorilla, lon FLOAT 
> encoding=Gorilla);{code}
> {code:java}
> create storage group root.sg1; 
> set schema template t2 to root.sg1.d2;
> create timeseries of schema template on root.sg1.d2;
> unset schema template t2 from root.sg1.d2;
> show paths using schema template t2;{code}
> Then Re-execute set and create operations:
> {code:java}
> IoTDB> set schema template t2 to root.sg1.d2;
> Msg: The statement is executed successfully.
> IoTDB> create timeseries of schema template on root.sg1.d2;
> Msg: 326: Template is in use on root.sg1.d2
> IoTDB> show paths using schema template t2
> +---+
> |      paths|
> +---+
> |root.sg1.d2|
> +---+
> Total line number = 1
> It costs 0.009s
> IoTDB> 
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4473) Support TSBS Timeseries Benchmark

2022-11-10 Thread Xingyu Liu (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632005#comment-17632005
 ] 

Xingyu Liu commented on IOTDB-4473:
---

user handbook preview in English: [tsbs/iotdb.md at iotdb · 
citrusreticulata/tsbs 
(github.com)|https://github.com/citrusreticulata/tsbs/blob/iotdb/docs/iotdb.md]

考虑到还需要阅读tsbs的总README,中文用户手册会包含更详细的内容,除了IoTDB部分还会包含tsbs的使用,正在编写中

> Support TSBS Timeseries Benchmark
> -
>
> Key: IOTDB-4473
> URL: https://issues.apache.org/jira/browse/IOTDB-4473
> Project: Apache IoTDB
>  Issue Type: New Feature
>Reporter: Julian Feinauer
>Assignee: Xingyu Liu
>Priority: Major
> Attachments: image-2022-11-06-19-37-51-033.png
>
>
> The tsbs Benchmark was initially created by InfluxDB and is now maintained by 
> timescale db and compares many state of the art timeseries databases: 
> [https://github.com/timescale/tsbs]
>  
> IMHO we should add support for IoTDB



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4907) Change show devices with storage group to database

2022-11-10 Thread Yuan Tian (Jira)
Yuan Tian created IOTDB-4907:


 Summary: Change show devices with storage group to database
 Key: IOTDB-4907
 URL: https://issues.apache.org/jira/browse/IOTDB-4907
 Project: Apache IoTDB
  Issue Type: Task
Reporter: Yuan Tian
Assignee: Minghui Liu






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4906) [0.14][cluster][load]change time_partition report tsfile cross partitions

2022-11-10 Thread ZhangZhengming (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhangZhengming reassigned IOTDB-4906:
-

Assignee: yusicheng

> [0.14][cluster][load]change time_partition report tsfile cross partitions
> -
>
> Key: IOTDB-4906
> URL: https://issues.apache.org/jira/browse/IOTDB-4906
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: ZhangZhengming
>Assignee: yusicheng
>Priority: Major
> Attachments: image-2022-11-10-18-43-18-211.png
>
>
> I write data use benchmark,
> dataset:1C3D,30sg,300device,30 series,10W points per series
> iotdb_config:
> enable_partition=true,time_partition_interval_for_storage=60480 (7 day)
>  
> Before load data, I change iotdb config:
> enable_partition=true,time_partition_interval_for_storage=8640 (1 day)
> Then report:
> *!image-2022-11-10-18-43-18-211.png|width=1650,height=265!*
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4906) [0.14][cluster][load]change time_partition will lost data

2022-11-10 Thread ZhangZhengming (Jira)
ZhangZhengming created IOTDB-4906:
-

 Summary: [0.14][cluster][load]change time_partition will lost data
 Key: IOTDB-4906
 URL: https://issues.apache.org/jira/browse/IOTDB-4906
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: ZhangZhengming
 Attachments: image-2022-11-10-18-43-18-211.png

I write data use benchmark,

dataset:1C3D,30sg,300device,30 series,10W points per series

iotdb_config:

enable_partition=true,time_partition_interval_for_storage=60480 (7 day)

 

Before load data, I change iotdb config:

enable_partition=true,time_partition_interval_for_storage=8640 (1 day)

 

*!image-2022-11-10-18-43-18-211.png|width=1650,height=265!*

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4904) [ ConfigNode ] When dynamically extending DataNode resources online, you need to optimize schemaregion allocation policies

2022-11-10 Thread Jira


[ 
https://issues.apache.org/jira/browse/IOTDB-4904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631573#comment-17631573
 ] 

刘珍 commented on IOTDB-4904:
---

另一个场景,不是在线动态扩展,1副本,3C5D, schema region分布在1个节点上,导致:
2022-11-10 18:09:57,109 [5@group-0002-StateMachineUpdater] ERROR 
o.a.i.d.m.v.SchemaExecutionVisitor:184 - IoTDB: MetaData error:
org.apache.iotdb.db.exception.metadata.SeriesOverflowException: There are too 
many timeseries in memory, please increase MAX_HEAP_SIZE in 
datanode-env.sh/bat, restart and create timeseries again.
at 
org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:575)
at 
org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.executeInternalCreateTimeseries(SchemaExecutionVisitor.java:176)
at 
org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitInternalCreateTimeSeries(SchemaExecutionVisitor.java:150)
at 
org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitInternalCreateTimeSeries(SchemaExecutionVisitor.java:64)
at 
org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.InternalCreateTimeSeriesNode.accept(InternalCreateTimeSeriesNode.java:105)
at 
org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:73)
at 
org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:137)
at 
org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1672)
at 
org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
at 
org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
at java.lang.Thread.run(Thread.java:748)


> [ ConfigNode ] When dynamically extending DataNode resources online, you need 
> to optimize schemaregion allocation policies
> --
>
> Key: IOTDB-4904
> URL: https://issues.apache.org/jira/browse/IOTDB-4904
> Project: Apache IoTDB
>  Issue Type: Improvement
>  Components: mpp-cluster
>Reporter: 刘珍
>Assignee: Gaofei Cao
>Priority: Major
> Attachments: all_online.sh, image-2022-11-10-15-06-01-931.png, 
> image-2022-11-10-15-07-15-997.png, ip26.conf, ip27.conf, ip28.conf, 
> ip29.conf, ip30.conf, ip31.conf, ip32.conf, online_exp_datanode.sh
>
>
> m_1109_87a416e, 3副本
> 1. 启动3C3D1Benchmark,写入数据,1小时。
> 2.集群在线扩展3DataNode,再启动1Benchmark,写入数据,1小时。
> ..
> 直至DataNode 扩展至21个,客户端为7Benchmark,会出现创建元数据失败(对照:3C21D 
> 1次全部启动,顺序间隔2s启动7Benchmark,元数据创建成功),因为schema region的分配策略不均衡,创建元数据报错日志:
> 2022-11-10 11:24:43,533 [3@group-0002-StateMachineUpdater] ERROR 
> o.a.i.d.m.v.SchemaExecutionVisitor:184 - IoTDB: MetaData error:
> org.apache.iotdb.db.exception.metadata.SeriesOverflowException: There are too 
> many timeseries in memory, please increase MAX_HEAP_SIZE in 
> datanode-env.sh/bat, restart and create timeseries again.
> at 
> org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.createTimeseries(SchemaRegionMemoryImpl.java:575)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.executeInternalCreateTimeseries(SchemaExecutionVisitor.java:176)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitInternalCreateTimeSeries(SchemaExecutionVisitor.java:150)
> at 
> org.apache.iotdb.db.metadata.visitor.SchemaExecutionVisitor.visitInternalCreateTimeSeries(SchemaExecutionVisitor.java:64)
> at 
> org.apache.iotdb.db.mpp.plan.planner.plan.node.metedata.write.InternalCreateTimeSeriesNode.accept(InternalCreateTimeSeriesNode.java:105)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.write(SchemaRegionStateMachine.java:73)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.applyTransaction(ApplicationStateMachineProxy.java:137)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1672)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.applyLog(StateMachineUpdater.java:239)
> at 
> org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:182)
> at java.base/java.lang.Thread.run(Thread.java:834)
> 在线扩展DataNode,最后状态的SchemaRegion :
>  !image-2022-11-10-15-06-01-931.png! 
> 不发生扩展,1次全部启动所有节点,7Benchmark运行完成的SchemaRegion :
>  !image-2022-11-10-15-07-15-997.png! 
> 测试环境,私有云1期
> 172.16.2.2 ~ 32 
> benchmark配置文件见附件ip*.conf
> 在线扩展脚本见online_exp_datanode.sh
> 不扩展脚本见all_online.sh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)