[jira] [Commented] (IOTDB-4350) [ MultiLeader Throttle Down] Performance does not return to normal after “Throttle Down“

2022-09-06 Thread Jira 


[ 
https://issues.apache.org/jira/browse/IOTDB-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601136#comment-17601136
 ] 

刘珍 commented on IOTDB-4350:
---

不制造故障，运行完成4546.75 second。
此issue场景（72故障3分钟，73故障2分钟），运行完成 ：13210.52 second，13210.52/4546.75=2.9。

> [ MultiLeader  Throttle Down] Performance does not return to normal after 
> “Throttle Down“
> -
>
> Key: IOTDB-4350
> URL: https://issues.apache.org/jira/browse/IOTDB-4350
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: 张洪胤
>Priority: Major
> Attachments: net_restart.conf, screenshot-1.png, screenshot-2.png, 
> screenshot-3.png, screenshot-4.png, screenshot-5.png, screenshot-6.png
>
>
> m_0905_0095eb3，3副本3C3D
> 3个dataregion ， 每个node上有1个leader。
> ip72 断网3分钟(16:52 ~ 16:55)，查看集群状态，切主成功后,
> ip73断网2分钟，之后不执行故障操作。
> 同步慢，multiLeader一直在合并限流，但是限流性能也回不去，如下，统计1分钟的写入数据量（bm中的batch）
>  !screenshot-6.png! 
> IoTDB> select count(latency) from 
> root.result.moresession_2022_09_06_04_47_03.INGESTION where okPoint>0 group 
> by ([1662454041076000186,1662459764764000179),1m);
> +---++
> |   
> Time|count(root.result.moresession_2022_09_06_04_47_03.INGESTION.latency)|
> +---++
> |2022-09-06T16:47:21.076000186+08:00| 
>5544|
> |2022-09-06T16:48:21.076000186+08:00| 
>6282|
> |2022-09-06T16:49:21.076000186+08:00| 
>5671|
> |2022-09-06T16:50:21.076000186+08:00| 
>4589|
> |2022-09-06T16:51:21.076000186+08:00| 
>5350|
> |2022-09-06T16:52:21.076000186+08:00| 
>1121|
> |2022-09-06T16:53:21.076000186+08:00| 
> 901|
> |2022-09-06T16:54:21.076000186+08:00| 
> 201|
> |2022-09-06T16:55:21.076000186+08:00| 
> 334|
> |2022-09-06T16:56:21.076000186+08:00| 
>3501|
> |2022-09-06T16:57:21.076000186+08:00| 
>3677|
> |2022-09-06T16:58:21.076000186+08:00| 
>3111|
> |2022-09-06T16:59:21.076000186+08:00| 
>1948|
> |2022-09-06T17:00:21.076000186+08:00| 
>3889|
> |2022-09-06T17:01:21.076000186+08:00| 
>2982|
> |2022-09-06T17:02:21.076000186+08:00| 
>4465|
> |2022-09-06T17:03:21.076000186+08:00| 
>4871|
> |2022-09-06T17:04:21.076000186+08:00| 
>4478|
> |2022-09-06T17:05:21.076000186+08:00| 
>3242|
> |2022-09-06T17:06:21.076000186+08:00| 
>2545|
> |2022-09-06T17:07:21.076000186+08:00| 
>2579|
> |2022-09-06T17:08:21.076000186+08:00| 
> 133|
> |2022-09-06T17:09:21.076000186+08:00| 
> 488|
> |2022-09-06T17:10:21.076000186+08:00| 
> 253|
> |2022-09-06T17:11:21.076000186+08:00| 
> 445|
> |2022-09-06T17:12:21.076000186+08:00| 
>2122|
> |2022-09-06T17:13:21.076000186+08:00| 
>1799|
> |2022-09-06T17:14:21.076000186+08:00| 
>1568|
> |2022-09-06T17:15:21.076000186+08:00| 
> 355|
> |2022-09-06T17:16:21.076000186+08:00|

[jira] [Created] (IOTDB-4356) Filter reduplicative device paths in PathPatternTree

2022-09-06 Thread Yongzao Dan (Jira) 
Yongzao Dan created IOTDB-4356:
--

 Summary: Filter reduplicative device paths in PathPatternTree
 Key: IOTDB-4356
 URL: https://issues.apache.org/jira/browse/IOTDB-4356
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Yongzao Dan
Assignee: yanze chen
 Fix For: 0.14.0
 Attachments: image-2022-09-07-11-42-50-076.png

!image-2022-09-07-11-42-50-076.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4355) Support index skip in PlanNodeIterator

2022-09-06 Thread Haiming Zhu (Jira) 
Haiming Zhu created IOTDB-4355:
--

 Summary: Support index skip in PlanNodeIterator
 Key: IOTDB-4355
 URL: https://issues.apache.org/jira/browse/IOTDB-4355
 Project: Apache IoTDB
  Issue Type: Improvement
  Components: Core/WAL
Reporter: Haiming Zhu
Assignee: Haiming Zhu


PlanNodeIterator reads plan node by index order, if index has gap, this 
iterator will block forever. We should move this gap process logic to consensus 
layer.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4354) [ confignode ] The leader of the schemaregion is displayed on the node that has been down

2022-09-06 Thread Jira 
刘珍 created IOTDB-4354:
-

 Summary: [ confignode ] The leader of the schemaregion is 
displayed on the node that has been down
 Key: IOTDB-4354
 URL: https://issues.apache.org/jira/browse/IOTDB-4354
 Project: Apache IoTDB
  Issue Type: Bug
  Components: mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Yongzao Dan
 Attachments: image-2022-09-07-10-36-30-819.png

m_0905_0095eb3， 3副本3C3D
下图，74节点的datanode已经down掉（72也down了），show regions|grep Leader 
，SchemaRegion的Leader还在74，应该切主。
 !image-2022-09-07-10-36-30-819.png! 

测试环境同https://issues.apache.org/jira/browse/IOTDB-4350



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4294) [ mem leak ] Request metadata timed out, causing memory leak

2022-09-06 Thread Jira 


[ 
https://issues.apache.org/jira/browse/IOTDB-4294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601086#comment-17601086
 ] 

刘珍 commented on IOTDB-4294:
---

OOM的issue ，见
https://issues.apache.org/jira/browse/IOTDB-4251

> [ mem leak ] Request metadata timed out, causing memory leak
> 
>
> Key: IOTDB-4294
> URL: https://issues.apache.org/jira/browse/IOTDB-4294
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Yuan Tian
>Priority: Major
>  Labels: pull-request-available
> Attachments: add_dn.conf, screenshot-1.png, screenshot-2.png, 
> screenshot-3.png
>
>
> master_0830_42fcbfc
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.standalone.{color:#DE350B}*StandAloneConsensus*{color}
> {color:#DE350B}*schemaregion 1副本*{color}
> dataregion 3副本
> 先启动1confignode，3datanode，启动benchmark，只有写入，{color:#DE350B}15小时后，大量写入失败{color}，下图是统计每小时的写入数据量：
>   !screenshot-1.png! 
> ip5 error
> 2022-08-30 18:01:04,256 [20220830_094446_42966_3.1.0-1068] ERROR 
> o.a.i.d.m.e.f.FragmentInstanceManager:157 - Execute error caused by
> org.apache.iotdb.db.mpp.exception.MemoryNotEnoughException: There is not 
> enough memory to execute current fragment instance, current remaining free 
> memory is 1014007, estimated memory usage for current fragment instance is 
> 1048576
> at 
> org.apache.iotdb.db.mpp.plan.planner.LocalExecutionPlanner.checkMemory(LocalExecutionPlanner.java:132)
> at 
> org.apache.iotdb.db.mpp.plan.planner.LocalExecutionPlanner.plan(LocalExecutionPlanner.java:104)
> at 
> org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceManager.lambda$execSchemaQueryFragmentInstance$3(FragmentInstanceManager.java:147)
> at 
> java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
> at 
> org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceManager.execSchemaQueryFragmentInstance(FragmentInstanceManager.java:133)
> at 
> org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.read(SchemaRegionStateMachine.java:94)
> at 
> org.apache.iotdb.consensus.standalone.StandAloneServerImpl.read(StandAloneServerImpl.java:72)
> at 
> org.apache.iotdb.consensus.standalone.StandAloneConsensus.read(StandAloneConsensus.java:135)
> at 
> org.apache.iotdb.db.service.thrift.impl.DataNodeInternalRPCServiceImpl.sendFragmentInstance(DataNodeInternalRPCServiceImpl.java:169)
> at 
> org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendFragmentInstance.getResult(IDataNodeRPCService.java:2136)
> at 
> org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendFragmentInstance.getResult(IDataNodeRPCService.java:2116)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 复现流程
> 1. 172.20.70.3/4/5   8c32G  datanode
> 172.20.70.31 confignode  8c32G
> benchmark在ip15  /data/benchmark/bm_0620_7ec96c1
> 集群与regions信息
>  !screenshot-2.png! 
> 2. 数据库配置参数
> confignode
> MAX_HEAP_SIZE="16G"
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.standalone.StandAloneConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_replication_factor=1
> data_replication_factor=3
> datanode
> MAX_HEAP_SIZE="16G"
> wal_buffer_size_in_byte=1048576
> max_waiting_time_when_insert_blocked=360
> 3. 启动bm 配置文件见附件
> 4. 后续还有add datanode（20分钟 加1个datanode，共加6个）
> ip 2/13/14/16/18/19
> 但是没有新的写入操作，所以这些新的datanode上没有数据。



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IOTDB-4176) Research Procedure Framework

2022-09-06 Thread Jinrui Zhang (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17601080#comment-17601080
 ] 

Jinrui Zhang commented on IOTDB-4176:
-

Please paste the doc link here

> Research Procedure Framework
> 
>
> Key: IOTDB-4176
> URL: https://issues.apache.org/jira/browse/IOTDB-4176
> Project: Apache IoTDB
>  Issue Type: Task
>Reporter: 伊丹翔
>Assignee: 伊丹翔
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4353) [Benchmark] Add new strategy of dividing device into client.

2022-09-06 Thread Jira 
张洪胤 created IOTDB-4353:
--

 Summary: [Benchmark] Add new strategy of dividing device into 
client.
 Key: IOTDB-4353
 URL: https://issues.apache.org/jira/browse/IOTDB-4353
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: 张洪胤
Assignee: 张洪胤






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4352) Fix PatialPath#overlapWith PartialPath contains MULTI_LEVEL_PATH_WILDCARD

2022-09-06 Thread yanze chen (Jira) 
yanze chen created IOTDB-4352:
-

 Summary: Fix PatialPath#overlapWith PartialPath contains 
MULTI_LEVEL_PATH_WILDCARD
 Key: IOTDB-4352
 URL: https://issues.apache.org/jira/browse/IOTDB-4352
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


PatialPath#overlapWith should not return true directly if PartialPath contains 
MULTI_LEVEL_PATH_WILDCARD.

For example, root.**.s1 is not overlap with root.**.s2



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IOTDB-4351) Enhance client-cpp for better performance

2022-09-06 Thread Bo Zhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bo Zhang reassigned IOTDB-4351:
---

Assignee: Bo Zhang

> Enhance client-cpp for better performance
> -
>
> Key: IOTDB-4351
> URL: https://issues.apache.org/jira/browse/IOTDB-4351
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: Bo Zhang
>Assignee: Bo Zhang
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4351) Enhance client-cpp for better performance

2022-09-06 Thread Bo Zhang (Jira) 
Bo Zhang created IOTDB-4351:
---

 Summary: Enhance client-cpp for better performance
 Key: IOTDB-4351
 URL: https://issues.apache.org/jira/browse/IOTDB-4351
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Bo Zhang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4350) [ MultiLeader Throttle Down] Performance does not return to normal after “Throttle Down“

2022-09-06 Thread Jira 
刘珍 created IOTDB-4350:
-

 Summary: [ MultiLeader  Throttle Down] Performance does not return 
to normal after “Throttle Down“
 Key: IOTDB-4350
 URL: https://issues.apache.org/jira/browse/IOTDB-4350
 Project: Apache IoTDB
  Issue Type: Bug
  Components: mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: 张洪胤


m_0905_0095eb3，3副本3C3D

3个dataregion ， 每个node上有1个leader。
ip72 断网2分钟，查看集群状态，切主成功后。
ip73断网2分钟，之后不执行故障操作。

同步慢，multiLeader一直在合并限流，但是限流性能也回不去，如下，统计1分钟的写入数据量（bm中的batch）

IoTDB> select count(latency) from 
root.result.moresession_2022_09_06_04_47_03.INGESTION where okPoint>0 group by 
([1662454041076000186,1662459764764000179),1m);

+---++
|   
Time|count(root.result.moresession_2022_09_06_04_47_03.INGESTION.latency)|
+---++
|2022-09-06T16:47:21.076000186+08:00|   
 5544|
|2022-09-06T16:48:21.076000186+08:00|   
 6282|
|2022-09-06T16:49:21.076000186+08:00|   
 5671|
|2022-09-06T16:50:21.076000186+08:00|   
 4589|
|2022-09-06T16:51:21.076000186+08:00|   
 5350|
|2022-09-06T16:52:21.076000186+08:00|   
 1121|
|2022-09-06T16:53:21.076000186+08:00|   
  901|
|2022-09-06T16:54:21.076000186+08:00|   
  201|
|2022-09-06T16:55:21.076000186+08:00|   
  334|
|2022-09-06T16:56:21.076000186+08:00|   
 3501|
|2022-09-06T16:57:21.076000186+08:00|   
 3677|
|2022-09-06T16:58:21.076000186+08:00|   
 3111|
|2022-09-06T16:59:21.076000186+08:00|   
 1948|
|2022-09-06T17:00:21.076000186+08:00|   
 3889|
|2022-09-06T17:01:21.076000186+08:00|   
 2982|
|2022-09-06T17:02:21.076000186+08:00|   
 4465|
|2022-09-06T17:03:21.076000186+08:00|   
 4871|
|2022-09-06T17:04:21.076000186+08:00|   
 4478|
|2022-09-06T17:05:21.076000186+08:00|   
 3242|
|2022-09-06T17:06:21.076000186+08:00|   
 2545|
|2022-09-06T17:07:21.076000186+08:00|   
 2579|
|2022-09-06T17:08:21.076000186+08:00|   
  133|
|2022-09-06T17:09:21.076000186+08:00|   
  488|
|2022-09-06T17:10:21.076000186+08:00|   
  253|
|2022-09-06T17:11:21.076000186+08:00|   
  445|
|2022-09-06T17:12:21.076000186+08:00|   
 2122|
|2022-09-06T17:13:21.076000186+08:00|   
 1799|
|2022-09-06T17:14:21.076000186+08:00|   
 1568|
|2022-09-06T17:15:21.076000186+08:00|   
  355|
|2022-09-06T17:16:21.076000186+08:00|   
 1127|
|2022-09-06T17:17:21.076000186+08:00|   
  803|
|2022-09-06T17:18:21.076000186+08:00|   
  674|
|2022-09-06T17:19:21.076000186+08:00|   
  621|
|2022-09-06T17:20:21.076000186+08:00|   
  361|
|2022-09-06T17:21:21.076000186+08:00|   
  367|
|2022-09-06T17:22:21.076000186+08:00|   
  999|

[jira] [Commented] (IOTDB-4328) Replica add/remove for multiLeader

2022-09-06 Thread Jinrui Zhang (Jira) 


[ 
https://issues.apache.org/jira/browse/IOTDB-4328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600695#comment-17600695
 ] 

Jinrui Zhang commented on IOTDB-4328:
-

The implementation is completed.

The basic test is passed.

!image-2022-09-06-17-23-35-762.png|width=2071,height=576!

> Replica add/remove for multiLeader
> --
>
> Key: IOTDB-4328
> URL: https://issues.apache.org/jira/browse/IOTDB-4328
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: Jinrui Zhang
>Assignee: Jinrui Zhang
>Priority: Major
> Attachments: image-2022-09-06-17-22-34-700.png, 
> image-2022-09-06-17-23-35-762.png
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IOTDB-4027) ERROR o.a.i.d.e.s.SnapshotLoader:94 - Exception occurs when creating links from snapshot directory to data directory

2022-09-06 Thread Liuxuxin (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liuxuxin reassigned IOTDB-4027:
---

Assignee: Song Ziyang  (was: Liuxuxin)

>  ERROR o.a.i.d.e.s.SnapshotLoader:94 - Exception occurs when creating links 
> from snapshot directory to data directory
> -
>
> Key: IOTDB-4027
> URL: https://issues.apache.org/jira/browse/IOTDB-4027
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Song Ziyang
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
> Attachments: image-2022-08-03-09-39-10-230.png, 
> image-2022-08-03-09-39-48-739.png, image-2022-09-06-17-05-21-387.png, 
> ip18_befor_stop_datanode_log.tar.gz, ip18_restart_with-error_log.tar.gz, 
> ip4_2000_config.properties, screenshot-1.png
>
>
> master_0801_55b5b17
> 问题描述
> RatisConsensus，3副本3C9D，1个bm连1个datanode执行并发写入，停止1个follower节点，5分钟后启动；{color:#DE350B}*然后停止另1个follower节点10分钟后启动，此节点启动过程中报错，此节点少数据*{color}：
> 2022-08-02 18:04:17,376 [pool-4-thread-1] ERROR o.a.i.d.e.s.SnapshotLoader:94 
> - Exception occurs when creating links from snapshot directory to data 
> directory
> java.io.IOException: Cannot find 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536/sequence/root.ip4.g_0
>  or 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536/unsequence/root.ip4.g_0
> at 
> org.apache.iotdb.db.engine.snapshot.SnapshotLoader.createLinksFromSnapshotDirToDataDir(SnapshotLoader.java:163)
> at 
> org.apache.iotdb.db.engine.snapshot.SnapshotLoader.loadSnapshotForStateMachine(SnapshotLoader.java:91)
> at 
> org.apache.iotdb.db.consensus.statemachine.DataRegionStateMachine.loadSnapshot(DataRegionStateMachine.java:93)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.loadSnapshot(ApplicationStateMachineProxy.java:188)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.lambda$initialize$0(ApplicationStateMachineProxy.java:73)
> at 
> org.apache.ratis.util.LifeCycle.startAndTransition(LifeCycle.java:270)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.initialize(ApplicationStateMachineProxy.java:69)
> at 
> org.apache.ratis.server.impl.ServerState.(ServerState.java:136)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.(RaftServerImpl.java:201)
> at 
> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$5(RaftServerProxy.java:274)
> at 
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2022-08-02 18:04:17,376 [pool-4-thread-1] ERROR 
> o.a.i.d.c.s.DataRegionStateMachine:95 - Fail to load snapshot from 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536
> ip18少数据，期望序列的count值是2点
>  !screenshot-1.png! 
> 1. 复现流程
> 私有云172.20.70.2/3/4/5/13/14/16/18/19
> benchmark 在ip15（连ip4）
> 停ip4/启动ip4  , 停ip18/启动ip18，ip18报错
>  !image-2022-08-03-09-39-10-230.png! 
>  !image-2022-08-03-09-39-48-739.png! 
> 2. 启动benchmark
> 2022-08-02 17:34:57 启动bm
> 3. 停止ip4的datanode
> 2022-08-02 17:45:42停止datanode
> sleep 300
> 启动ip4
> 4. 停止ip18的datanode
> 2022-08-02 17:54:11 停止ip18的datanode
> sleep 600
> 启动ip18
> {color:#DE350B}*启动过程中，报错*{color}：
> 见问题描述
> bm写入完成，各节点同步完成，{color:#DE350B}*ip18节点少数据*{color}，ip16，ip4 的数据正确。



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Reopened] (IOTDB-4027) ERROR o.a.i.d.e.s.SnapshotLoader:94 - Exception occurs when creating links from snapshot directory to data directory

2022-09-06 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

刘珍 reopened IOTDB-4027:
---

master_0906_0095eb3
在snapshot前 ，down follower ip2 ，ip2在线后，raft log不同步
ip2日志
2022-09-06 15:53:31,056 [null-request--thread1] INFO  
o.a.r.g.s.GrpcClientProtocolService$UnorderedRequestStreamObserver:284 - Failed 
RaftClientRequest:client-0DF34DD58B52->172.20.70.2_40010@group-00010002, 
cid=36, seq=0, RW, Message:000d0012726f6f74...(size=68644), 
reply=RaftClientReply:client-0DF34DD58B52->172.20.70.2_40010@group-00010002,
 cid=36, FAILED org.apache.ratis.protocol.exceptions.NotLeaderException: Server 
172.20.70.2_40010@group-00010002 is not the leader, logIndex=0, 
commits[172.20.70.2_40010:c202840, 172.20.70.14_40010:c556712, 
172.20.70.18_40010:c556711]


>  ERROR o.a.i.d.e.s.SnapshotLoader:94 - Exception occurs when creating links 
> from snapshot directory to data directory
> -
>
> Key: IOTDB-4027
> URL: https://issues.apache.org/jira/browse/IOTDB-4027
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Liuxuxin
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.14.0
>
> Attachments: image-2022-08-03-09-39-10-230.png, 
> image-2022-08-03-09-39-48-739.png, image-2022-09-06-17-05-21-387.png, 
> ip18_befor_stop_datanode_log.tar.gz, ip18_restart_with-error_log.tar.gz, 
> ip4_2000_config.properties, screenshot-1.png
>
>
> master_0801_55b5b17
> 问题描述
> RatisConsensus，3副本3C9D，1个bm连1个datanode执行并发写入，停止1个follower节点，5分钟后启动；{color:#DE350B}*然后停止另1个follower节点10分钟后启动，此节点启动过程中报错，此节点少数据*{color}：
> 2022-08-02 18:04:17,376 [pool-4-thread-1] ERROR o.a.i.d.e.s.SnapshotLoader:94 
> - Exception occurs when creating links from snapshot directory to data 
> directory
> java.io.IOException: Cannot find 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536/sequence/root.ip4.g_0
>  or 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536/unsequence/root.ip4.g_0
> at 
> org.apache.iotdb.db.engine.snapshot.SnapshotLoader.createLinksFromSnapshotDirToDataDir(SnapshotLoader.java:163)
> at 
> org.apache.iotdb.db.engine.snapshot.SnapshotLoader.loadSnapshotForStateMachine(SnapshotLoader.java:91)
> at 
> org.apache.iotdb.db.consensus.statemachine.DataRegionStateMachine.loadSnapshot(DataRegionStateMachine.java:93)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.loadSnapshot(ApplicationStateMachineProxy.java:188)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.lambda$initialize$0(ApplicationStateMachineProxy.java:73)
> at 
> org.apache.ratis.util.LifeCycle.startAndTransition(LifeCycle.java:270)
> at 
> org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.initialize(ApplicationStateMachineProxy.java:69)
> at 
> org.apache.ratis.server.impl.ServerState.(ServerState.java:136)
> at 
> org.apache.ratis.server.impl.RaftServerImpl.(RaftServerImpl.java:201)
> at 
> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$5(RaftServerProxy.java:274)
> at 
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2022-08-02 18:04:17,376 [pool-4-thread-1] ERROR 
> o.a.i.d.c.s.DataRegionStateMachine:95 - Fail to load snapshot from 
> /data/iotdb/master_0801_2de0dd8/datanode/./sbin/../data/consensus/data_region/47474747-4747-4747-4747-00010001/sm/1_354536
> ip18少数据，期望序列的count值是2点
>  !screenshot-1.png! 
> 1. 复现流程
> 私有云172.20.70.2/3/4/5/13/14/16/18/19
> benchmark 在ip15（连ip4）
> 停ip4/启动ip4  , 停ip18/启动ip18，ip18报错
>  !image-2022-08-03-09-39-10-230.png! 
>  !image-2022-08-03-09-39-48-739.png! 
> 2. 启动benchmark
> 2022-08-02 17:34:57 启动bm
> 3. 停止ip4的datanode
> 2022-08-02 17:45:42停止datanode
> sleep 300
> 启动ip4
> 4. 停止ip18的datanode
> 2022-08-02 17:54:11 停止ip18的datanode
> sleep 600
> 启动ip18
> {color:#DE350B}*启动过程中，报错*{color}：
> 见问题描述
> bm写入完成，各节点同步完成，{color:#DE350B}*ip18节点少数据*{color}，ip16，ip4 的数据正确。



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4349) Remove error status in data node

2022-09-06 Thread Haiming Zhu (Jira) 
Haiming Zhu created IOTDB-4349:
--

 Summary: Remove error status in data node
 Key: IOTDB-4349
 URL: https://issues.apache.org/jira/browse/IOTDB-4349
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: Haiming Zhu
Assignee: Haiming Zhu


Error status is useless, remove it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IOTDB-4343) [master]After starting mqtt, after executing Trigger's SQL, there is an NPE error in the log

2022-09-06 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

张洪胤 reassigned IOTDB-4343:
--

Assignee: 张洪胤  (was: liaolanyu)

> [master]After starting mqtt, after executing Trigger's SQL, there is an NPE 
> error in the log
> 
>
> Key: IOTDB-4343
> URL: https://issues.apache.org/jira/browse/IOTDB-4343
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Server
>Affects Versions: master branch
>Reporter: xiaozhihong
>Assignee: 张洪胤
>Priority: Major
> Attachments: log_all.log
>
>
> commit 518dcfbfa461ffe4912fba18cb29299416301588
> After enabling MQTT, execute the trigger use case, and find that there is an 
> NPE authorization authentication error in the log.
> Step 1: In the iotdb-datanode.properties ,modify parameter:
> enable_mqtt_service=true
> Step 2: Start server, Enter CLI
> Step 3: Execute SQL:
> {code:java}
> set storage group to root.sg1;
> CREATE TIMESERIES root.sg1.dev1.s_1 WITH DATATYPE=INT32, ENCODING=GORILLA;
> CREATE TIMESERIES root.sg1.dev1.s_2 WITH DATATYPE=INT64, ENCODING=PLAIN;
> CREATE TIMESERIES root.sg1.dev1.s_3 WITH DATATYPE=float, ENCODING=RLE;
> CREATE TIMESERIES root.sg1.dev1.s_4 WITH DATATYPE=double, ENCODING=TS_2DIFF;
> CREATE TIMESERIES root.sg1.dev1.s_5 WITH DATATYPE=text, ENCODING=DICTIONARY;
> CREATE TIMESERIES root.sg1.dev1.s_6 WITH DATATYPE=boolean, ENCODING=PLAIN;
> CREATE TRIGGER trig1
> AFTER INSERT
> ON root.sg1.dev1.s_1
> AS 'TriggerTest'
> WITH (
>   'ts_type' = 'int32',
>   'remote_ip' = '127.0.0.1',
>   'trig_name'='trig1'
> );
> CREATE TRIGGER trig2
> AFTER INSERT
> ON root.sg1.dev1.s_2
> AS 'TriggerTest'
> WITH (
>   'ts_type' = 'int64',
>   'remote_ip' = '127.0.0.1',
>   'trig_name'='trig2'
> );
> insert into root.sg1.dev1(time,s_1,s_2) values(1,100,200);
> insert into root.sg1.dev1(time,s_1,s_2) values(2,300,400);
> select s_1,s_2,s_3,s_4,s_5,s_6 from root.sg1.dev1;
> select local_trig1,local_trig2,remotetrig1,remotetrig2 from 
> root.target.alerting; 
>   
> delete timeseries root.sg1.dev1.s_1;
> insert into root.sg1.dev1(time,s_2) values(100,1600);{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IOTDB-4343) [master]After starting mqtt, after executing Trigger's SQL, there is an NPE error in the log

2022-09-06 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

张洪胤 reassigned IOTDB-4343:
--

Assignee: liaolanyu  (was: 张洪胤)

> [master]After starting mqtt, after executing Trigger's SQL, there is an NPE 
> error in the log
> 
>
> Key: IOTDB-4343
> URL: https://issues.apache.org/jira/browse/IOTDB-4343
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Server
>Affects Versions: master branch
>Reporter: xiaozhihong
>Assignee: liaolanyu
>Priority: Major
> Attachments: log_all.log
>
>
> commit 518dcfbfa461ffe4912fba18cb29299416301588
> After enabling MQTT, execute the trigger use case, and find that there is an 
> NPE authorization authentication error in the log.
> Step 1: In the iotdb-datanode.properties ,modify parameter:
> enable_mqtt_service=true
> Step 2: Start server, Enter CLI
> Step 3: Execute SQL:
> {code:java}
> set storage group to root.sg1;
> CREATE TIMESERIES root.sg1.dev1.s_1 WITH DATATYPE=INT32, ENCODING=GORILLA;
> CREATE TIMESERIES root.sg1.dev1.s_2 WITH DATATYPE=INT64, ENCODING=PLAIN;
> CREATE TIMESERIES root.sg1.dev1.s_3 WITH DATATYPE=float, ENCODING=RLE;
> CREATE TIMESERIES root.sg1.dev1.s_4 WITH DATATYPE=double, ENCODING=TS_2DIFF;
> CREATE TIMESERIES root.sg1.dev1.s_5 WITH DATATYPE=text, ENCODING=DICTIONARY;
> CREATE TIMESERIES root.sg1.dev1.s_6 WITH DATATYPE=boolean, ENCODING=PLAIN;
> CREATE TRIGGER trig1
> AFTER INSERT
> ON root.sg1.dev1.s_1
> AS 'TriggerTest'
> WITH (
>   'ts_type' = 'int32',
>   'remote_ip' = '127.0.0.1',
>   'trig_name'='trig1'
> );
> CREATE TRIGGER trig2
> AFTER INSERT
> ON root.sg1.dev1.s_2
> AS 'TriggerTest'
> WITH (
>   'ts_type' = 'int64',
>   'remote_ip' = '127.0.0.1',
>   'trig_name'='trig2'
> );
> insert into root.sg1.dev1(time,s_1,s_2) values(1,100,200);
> insert into root.sg1.dev1(time,s_1,s_2) values(2,300,400);
> select s_1,s_2,s_3,s_4,s_5,s_6 from root.sg1.dev1;
> select local_trig1,local_trig2,remotetrig1,remotetrig2 from 
> root.target.alerting; 
>   
> delete timeseries root.sg1.dev1.s_1;
> insert into root.sg1.dev1(time,s_2) values(100,1600);{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IOTDB-4347) NPE error when session is expired

2022-09-06 Thread Yifu Zhou (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-4347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yifu Zhou reassigned IOTDB-4347:


Assignee: Yifu Zhou

> NPE error when session is expired
> -
>
> Key: IOTDB-4347
> URL: https://issues.apache.org/jira/browse/IOTDB-4347
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: Yifu Zhou
>Assignee: Yifu Zhou
>Priority: Major
>  Labels: pull-request-available
> Fix For: master branch, 0.14.0-SNAPSHOT
>
> Attachments: image-2022-09-06-14-31-32-982.png
>
>
> When session is expired or session is closed, it will occurs NPE when 
> obtaining the username.
> !image-2022-09-06-14-31-32-982.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4348) NoSuchFileExeception when creating snapshot

2022-09-06 Thread Jinrui Zhang (Jira) 
Jinrui Zhang created IOTDB-4348:
---

 Summary: NoSuchFileExeception when creating snapshot
 Key: IOTDB-4348
 URL: https://issues.apache.org/jira/browse/IOTDB-4348
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: Jinrui Zhang
Assignee: Liuxuxin
 Attachments: image-2022-09-06-14-22-05-045.png

NoSuchFileException is threw when taking snapshot in DataRegion migration

!image-2022-09-06-14-22-05-045.png|width=1270,height=409!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IOTDB-4347) NPE error when session is expired

2022-09-06 Thread Yifu Zhou (Jira) 
Yifu Zhou created IOTDB-4347:


 Summary: NPE error when session is expired
 Key: IOTDB-4347
 URL: https://issues.apache.org/jira/browse/IOTDB-4347
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: Yifu Zhou
 Fix For: master branch, 0.14.0-SNAPSHOT






--
This message was sent by Atlassian Jira
(v8.20.10#820010)