[jira] [Created] (IOTDB-5712) Concurrent bug in memory pool caused by identitySink
Alima777 created IOTDB-5712: --- Summary: Concurrent bug in memory pool caused by identitySink Key: IOTDB-5712 URL: https://issues.apache.org/jira/browse/IOTDB-5712 Project: Apache IoTDB Issue Type: Bug Reporter: Alima777 Assignee: Alima777 Alls sinkChannels of identitySink shares the planNodeId of identitySinkNode. Therefore, when the first child closes the sharedTsBlockQueue, it will try to clear the queryMemoryMap related about planNodeId. However, the second child can reserve, free or clear the queryMemoryMap of the same planNodeId concurrently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (IOTDB-5686) [ Sequence InnerSpaceCompaction ] Devices with the same name but different alignment properties：Meet errors in inner space compaction.
[ https://issues.apache.org/jira/browse/IOTDB-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘珍 reopened IOTDB-5686: --- rc/1.1.0 0321_6616b0d 合并不报错，但是数据不一致 count(s_0)是110，s_0是80条。 > [ Sequence InnerSpaceCompaction ] Devices with the same name but different > alignment properties：Meet errors in inner space compaction. > --- > > Key: IOTDB-5686 > URL: https://issues.apache.org/jira/browse/IOTDB-5686 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction, mpp-cluster >Affects Versions: 1.1.0-SNAPSHOT >Reporter: 刘珍 >Assignee: 周沛辰 >Priority: Major > Labels: pull-request-available > Fix For: master branch, 1.1.0-SNAPSHOT > > Attachments: 0316.zip, image-2023-03-22-10-25-24-477.png > > > rc/1.1.0 0314_aeee0eb > 顺序空间内合并： > 100个非对齐设备，写入数据，删除2个非对齐设备，再创建2个与已deleted同名的对齐设备，合并不报错； > 再把这2个对齐设备删除，创建2个同名非对齐设备，写入数据，触发合并，报错： > 2023-03-16 14:38:18,945 [pool-16-IoTDB-Compaction-3] ERROR > o.a.i.d.e.c.e.t.InnerSpaceCompactionTask:269 - root.test.g_0-2 [Compaction] > Meet errors in inner space compaction. > org.apache.iotdb.commons.exception.IllegalPathException: root.test.g_0.d_0. > is not a legal path > at > org.apache.iotdb.commons.utils.PathUtils.splitPathToDetachedNodes(PathUtils.java:48) > at org.apache.iotdb.commons.path.PartialPath.(PartialPath.java:82) > at > org.apache.iotdb.db.engine.compaction.execute.performer.impl.ReadChunkCompactionPerformer.compactNotAlignedSeries(ReadChunkCompactionPerformer.java:174) > at > org.apache.iotdb.db.engine.compaction.execute.performer.impl.ReadChunkCompactionPerformer.perform(ReadChunkCompactionPerformer.java:87) > at > org.apache.iotdb.db.engine.compaction.execute.task.InnerSpaceCompactionTask.doCompaction(InnerSpaceCompactionTask.java:151) > at > org.apache.iotdb.db.engine.compaction.execute.task.AbstractCompactionTask.start(AbstractCompactionTask.java:80) > at > org.apache.iotdb.db.engine.compaction.schedule.CompactionWorker.run(CompactionWorker.java:65) > at > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > 数据库测试版本 rc/1.1.0 0314_aeee0eb > 私有云1期172.16.2.18 > bm：172.16.2.30 > 集群配置参数 > CN > MAX_HEAP_SIZE="2G" > cn_internal_address=172.16.2.18 > cn_target_config_node_list=172.16.2.18:10710 > DN > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > dn_rpc_address=172.16.2.18 > dn_internal_address=172.16.2.18 > dn_target_config_node_list=172.16.2.18:10710 > 运行脚本见附件0316.tar.gz -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [iotdb-client-go] HTHou merged pull request #81: Bump actions/setup-go from 3 to 4
HTHou merged PR #81: URL: https://github.com/apache/iotdb-client-go/pull/81 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (IOTDB-5711) Python API should support connecting multiple nodes
[ https://issues.apache.org/jira/browse/IOTDB-5711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-5711: - Assignee: Haonan Hou > Python API should support connecting multiple nodes > --- > > Key: IOTDB-5711 > URL: https://issues.apache.org/jira/browse/IOTDB-5711 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Haonan Hou >Assignee: Haonan Hou >Priority: Major > > Python API should support connecting multiple nodes just like Java Api does. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5711) Python API should support connecting multiple nodes
Haonan Hou created IOTDB-5711: - Summary: Python API should support connecting multiple nodes Key: IOTDB-5711 URL: https://issues.apache.org/jira/browse/IOTDB-5711 Project: Apache IoTDB Issue Type: New Feature Reporter: Haonan Hou Python API should support connecting multiple nodes just like Java Api does. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5708) Pipe: Define Data Capture Class on DataNode
[ https://issues.apache.org/jira/browse/IOTDB-5708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yusicheng reassigned IOTDB-5708: Assignee: yusicheng > Pipe: Define Data Capture Class on DataNode > --- > > Key: IOTDB-5708 > URL: https://issues.apache.org/jira/browse/IOTDB-5708 > Project: Apache IoTDB > Issue Type: Task >Reporter: yusicheng >Assignee: yusicheng >Priority: Major > > we need to find change data capture interface API on DN > includes: > 1. write tsfile > 2. write wal > 3. consensus write log > ignore: > 1. consensus snapshot > 2. load tsfile > 3. restore tsfile -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5710) Implement parser
lichenyu created IOTDB-5710: --- Summary: Implement parser Key: IOTDB-5710 URL: https://issues.apache.org/jira/browse/IOTDB-5710 Project: Apache IoTDB Issue Type: Task Components: Client/Python Reporter: lichenyu -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5709) Support MlNodeInteralRPCService in DataNode
yang caiyin created IOTDB-5709: -- Summary: Support MlNodeInteralRPCService in DataNode Key: IOTDB-5709 URL: https://issues.apache.org/jira/browse/IOTDB-5709 Project: Apache IoTDB Issue Type: New Feature Reporter: yang caiyin Assignee: yang caiyin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5600) Metadata between nodes is inconsistent
[
https://issues.apache.org/jira/browse/IOTDB-5600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703066#comment-17703066
]
刘珍 commented on IOTDB-5600:
---
rel/1.1 0320_1f605d5, 开启元数据持久化，所有元数据创建成功，副本间元数据一致。
> Metadata between nodes is inconsistent
> --
>
> Key: IOTDB-5600
> URL: https://issues.apache.org/jira/browse/IOTDB-5600
> Project: Apache IoTDB
> Issue Type: Improvement
>Reporter: 刘珍
>Assignee: Yukun Zhou
>Priority: Major
> Attachments: exec_iotdb_5516.sh, image-2023-03-01-13-47-52-418.png,
> iotdb.sh, iotdb_5516.conf, screenshot-1.png
>
>
> master_0228_3ef36de
> { drop database + create metadata , insert 1 point/sensor } , execute 10
> times in serial .
> benchmark execution complete，cli -e "flush"
> Metadata between nodes is inconsistent.
> !image-2023-03-01-13-47-52-418.png!
> 测试流程：
> 1.启动3副本3C5D . 把自动创建元数据设置为false
> ConfigNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> DataNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> COMMON：
> schema_replication_factor=3
> data_replication_factor=3
> enable_auto_create_schema=false
> 3ConfigNode ：
> 172.16.2.23
> 172.16.2.24
> 172.16.2.25
> 5DataNode：
> 172.16.2.2
> 172.16.2.3
> 172.16.2.4
> 172.16.2.5
> 172.16.2.6
> 2. 运行如下脚本（172.16.2.26机器，/data/iotdb/benchmark_v1下）：
> for i in {1..10}
> do
> /data/iotdb/m_0227_2bc29bb/sbin/start-cli.sh -h 172.16.2.2 -e "drop
> database root.test.*;"
> sh ./iotdb.sh 5516 > 5516_$i.out
> done
> 运行完成，cli 执行flush。
> 3. 验证各节点元数据与数据一致性
> 所有节点在线：count devices
> down掉ip2：count devices
> down 掉ip6：count devices
> 查询结果不一致。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-5664) [compaction] java.lang.OutOfMemoryError: Java heap space
[ https://issues.apache.org/jira/browse/IOTDB-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703065#comment-17703065 ] 刘珍 commented on IOTDB-5664: --- https://apache-iotdb.feishu.cn/docx/SJWvdzgXYottyxxlvSac33UpnAh 序号17，运行83小时，OOM > [compaction] java.lang.OutOfMemoryError: Java heap space > > > Key: IOTDB-5664 > URL: https://issues.apache.org/jira/browse/IOTDB-5664 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction, mpp-cluster >Affects Versions: 1.1.0-SNAPSHOT >Reporter: 刘珍 >Assignee: 周沛辰 >Priority: Major > Attachments: comp1_normal.conf, comp3_aligned.conf, run_comp.sh, > test.sh > > > rc/1.1.0 0306_32f9db0 > 启动1C1D，写入1个顺序tsfile，其他tsfile为乱序，每个序列(普通/对齐)写入1个点，flush一次。 > 46小时，OOM > 2023-03-12 14:53:49,951 [pool-24-IoTDB-DataNodeInternalRPC-Processor-24] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar > replaced objects > 2023-03-12 14:57:58,102 [pool-7-IoTDB-StorageEngine-1415] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=StorageEngine > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:58:03,694 [pool-24-IoTDB-DataNodeInternalRPC-Processor-24] > ERROR o.a.i.c.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread > pool-24-IoTDB-DataNodeInternalRPC-Processor-24-20462 > java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar > replaced objects > 2023-03-12 14:53:49,951 [pool-24-IoTDB-DataNodeInternalRPC-Processor-17] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:53:49,951 [pool-21-IoTDB-MPPCoordinator-1] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=MPPCoordinator > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:58:03,694 [pool-21-IoTDB-MPPCoordinator-2] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=MPPCoordinator > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:54:01,138 [pool-21-IoTDB-MPPCoordinator-4] ERROR > o.a.i.c.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread > pool-21-IoTDB-MPPCoordinator-4-62 > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:57:52,533 [pool-24-IoTDB-DataNodeInternalRPC-Processor-77] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:54:12,287 > [pool-25-IoTDB-ClientRPC-Processor-22385$20230312_064221_26096_1.1.0] ERROR > o.a.i.d.m.e.e.RegionWriteExecutor:92 - Java heap space > java.lang.OutOfMemoryError: Java heap space > at java.base/java.lang.StringLatin1.toLowerCase(StringLatin1.java:397) > at java.base/java.lang.String.toLowerCase(String.java:2503) > at java.base/java.lang.String.toLowerCase(String.java:2527) > at > org.apache.iotdb.commons.service.metric.enums.Metric.toString(Metric.java:96) > at > org.apache.iotdb.db.mpp.metric.PerformanceOverviewMetricsManager.recordScheduleSchemaValidateCost(PerformanceOverviewMetricsManager.java:88) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.executeDataInsert(RegionWriteExecutor.java:228) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:185) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:139) > at > org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertTabletNode.accept(InsertTabletNode.java:1062) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor.execute(RegionWriteExecutor.java:90) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchLocally(FragmentInstanceDispatcherImpl.java:321) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:211) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchWriteAsync(FragmentInstanceDispatcherImpl.java:182) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:101) >
[jira] [Assigned] (IOTDB-5664) [compaction] java.lang.OutOfMemoryError: Java heap space
[ https://issues.apache.org/jira/browse/IOTDB-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘珍 reassigned IOTDB-5664: - Assignee: 周沛辰 (was: Jinrui Zhang) > [compaction] java.lang.OutOfMemoryError: Java heap space > > > Key: IOTDB-5664 > URL: https://issues.apache.org/jira/browse/IOTDB-5664 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction, mpp-cluster >Affects Versions: 1.1.0-SNAPSHOT >Reporter: 刘珍 >Assignee: 周沛辰 >Priority: Major > Attachments: comp1_normal.conf, comp3_aligned.conf, run_comp.sh, > test.sh > > > rc/1.1.0 0306_32f9db0 > 启动1C1D，写入1个顺序tsfile，其他tsfile为乱序，每个序列(普通/对齐)写入1个点，flush一次。 > 46小时，OOM > 2023-03-12 14:53:49,951 [pool-24-IoTDB-DataNodeInternalRPC-Processor-24] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar > replaced objects > 2023-03-12 14:57:58,102 [pool-7-IoTDB-StorageEngine-1415] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=StorageEngine > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:58:03,694 [pool-24-IoTDB-DataNodeInternalRPC-Processor-24] > ERROR o.a.i.c.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread > pool-24-IoTDB-DataNodeInternalRPC-Processor-24-20462 > java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar > replaced objects > 2023-03-12 14:53:49,951 [pool-24-IoTDB-DataNodeInternalRPC-Processor-17] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:53:49,951 [pool-21-IoTDB-MPPCoordinator-1] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=MPPCoordinator > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:58:03,694 [pool-21-IoTDB-MPPCoordinator-2] ERROR > o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=MPPCoordinator > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:54:01,138 [pool-21-IoTDB-MPPCoordinator-4] ERROR > o.a.i.c.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread > pool-21-IoTDB-MPPCoordinator-4-62 > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:57:52,533 [pool-24-IoTDB-DataNodeInternalRPC-Processor-77] > ERROR o.a.i.c.c.t.WrappedThreadPoolExecutor:108 - Exception in thread pool > org.apache.iotdb.threadpool:type=DataNodeInternalRPC-Processor > java.lang.OutOfMemoryError: Java heap space > 2023-03-12 14:54:12,287 > [pool-25-IoTDB-ClientRPC-Processor-22385$20230312_064221_26096_1.1.0] ERROR > o.a.i.d.m.e.e.RegionWriteExecutor:92 - Java heap space > java.lang.OutOfMemoryError: Java heap space > at java.base/java.lang.StringLatin1.toLowerCase(StringLatin1.java:397) > at java.base/java.lang.String.toLowerCase(String.java:2503) > at java.base/java.lang.String.toLowerCase(String.java:2527) > at > org.apache.iotdb.commons.service.metric.enums.Metric.toString(Metric.java:96) > at > org.apache.iotdb.db.mpp.metric.PerformanceOverviewMetricsManager.recordScheduleSchemaValidateCost(PerformanceOverviewMetricsManager.java:88) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.executeDataInsert(RegionWriteExecutor.java:228) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:185) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:139) > at > org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertTabletNode.accept(InsertTabletNode.java:1062) > at > org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor.execute(RegionWriteExecutor.java:90) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchLocally(FragmentInstanceDispatcherImpl.java:321) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchOneInstance(FragmentInstanceDispatcherImpl.java:211) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatchWriteAsync(FragmentInstanceDispatcherImpl.java:182) > at > org.apache.iotdb.db.mpp.plan.scheduler.FragmentInstanceDispatcherImpl.dispatch(FragmentInstanceDispatcherImpl.java:101) > at > org.apache.iotdb.db.mpp.plan.scheduler.ClusterScheduler.start(ClusterScheduler
[jira] [Created] (IOTDB-5708) Pipe: Data Caputrue
yusicheng created IOTDB-5708: Summary: Pipe: Data Caputrue Key: IOTDB-5708 URL: https://issues.apache.org/jira/browse/IOTDB-5708 Project: Apache IoTDB Issue Type: Task Reporter: yusicheng -- This message was sent by Atlassian Jira (v8.20.10#820010)
