[jira] [Created] (IOTDB-5527) 0.13.4 Compaction OOM
周沛辰 created IOTDB-5527: -- Summary: 0.13.4 Compaction OOM Key: IOTDB-5527 URL: https://issues.apache.org/jira/browse/IOTDB-5527 Project: Apache IoTDB Issue Type: Bug Reporter: 周沛辰 Assignee: 周沛辰 Find a scenario where the 0.13.4 compaction will lead to memory OOM: the test environment for importing historical data, the JVM memory of about 10G, the write throughput is relatively high, and the out-of-order ratio is not low. After using this data to start in a small memory environment, it will directly OOM. Test Data：https://cloud.tsinghua.edu.cn/d/77627e3d9f48450caf0a/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5526) [Template]The timeseries with the same device path as the template cannot be deleted
xiaozhihong created IOTDB-5526:
--
Summary: [Template]The timeseries with the same device path as the
template cannot be deleted
Key: IOTDB-5526
URL: https://issues.apache.org/jira/browse/IOTDB-5526
Project: Apache IoTDB
Issue Type: Bug
Components: Core/Schema Manager, mpp-cluster
Affects Versions: master branch
Reporter: xiaozhihong
Assignee: Yukun Zhou
master branch
commit adf55e896ef55220785dcf72564cbb8a0e5a2280
First of all create a template, then mount the template, activate and insert
data;
Then create other timeseries under the same device and insert the data
Finally, delete the newly created time series, which cannot be deleted.
{code:java}
create schema template t1 (temperature FLOAT encoding=RLE, status BOOLEAN
encoding=PLAIN compression=SNAPPY);
show schema templates;
create database root.sg1;
set schema template t1 to root.sg1.d1;
show paths set schema template t1;
create timeseries of schema template on root.sg1.d1;
show paths using schema template t1;
show timeseries root.sg1.**;
show devices root.sg1.**;
show timeseries;
insert into root.sg1.d1(time, temperature, status) values(1, 1, 1);
insert into root.sg1.d1(time, temperature, status) values(2, 2, 0), (3, 3, 1);
select * from root.sg1.d1;
delete timeseries root.sg1.d1.temperature;
CREATE ALIGNED TIMESERIES root.sg1.d1.GPS(latitude FLOAT encoding=PLAIN
compressor=SNAPPY, longitude FLOAT encoding=PLAIN compressor=SNAPPY);
show timeseries root.sg1.d1.**;
insert into root.sg1.d1.GPS(time, latitude, longitude) aligned values(1, 1, 1);
insert into root.sg1.d1.GPS(time, latitude, longitude) aligned values(2, 2, 2),
(3, 3, 3);
select ** from root.sg1.d1;
delete timeseries root.sg1.d1.GPS.latitude;
show timeseries;
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5525) Can't find device's data partition after partition fetcher
Yuan Tian created IOTDB-5525:
Summary: Can't find device's data partition after partition fetcher
Key: IOTDB-5525
URL: https://issues.apache.org/jira/browse/IOTDB-5525
Project: Apache IoTDB
Issue Type: Bug
Components: Core/Engine
Reporter: Yuan Tian
Assignee: Yuan Tian
This issue records the following four things we meet in geely POC:
1. Previously, while calculating TTimeSlot for each SeriesSlot, we mistakenly
replace previous one with current one which cause the TTimeSlot is incomplete
for each SeriesSlot. In write case, we may get IndexOutOfBound error in the
later phase.
2. Accelerate the distribution plan phase for {{{}group by level query{}}},
reducing time complexity from O(N^2) to O(N) for each data region
3. Correct the memory estimation for AggregationOperator.
4. Correct the default value for {{seriesPartitionExecutorClass}} from
{{APHashExecutor}} to {{BKDRHashExecutor}} which is same as the one in config
file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5524) Add Metrics for Schema File
yanze chen created IOTDB-5524: - Summary: Add Metrics for Schema File Key: IOTDB-5524 URL: https://issues.apache.org/jira/browse/IOTDB-5524 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5524) Add Metrics for Schema File
[ https://issues.apache.org/jira/browse/IOTDB-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanze chen reassigned IOTDB-5524: - Assignee: yanze chen > Add Metrics for Schema File > --- > > Key: IOTDB-5524 > URL: https://issues.apache.org/jira/browse/IOTDB-5524 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: yanze chen >Assignee: yanze chen >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5472) [Atmos]The number of tsfiles went up between db6f17e 【02/01】and 5602d0e【02/05】
[ https://issues.apache.org/jira/browse/IOTDB-5472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687665#comment-17687665 ] Haiming Zhu commented on IOTDB-5472: In the 1C1D, datanode uses 20G memory, the number of dataregions changed from 10 to 20, so each region has smaller memory space, which causes tsfiles' size being smaller. As for the query, we haven't test the relationship between the number of regions and query efficiency. > [Atmos]The number of tsfiles went up between db6f17e 【02/01】and 5602d0e【02/05】 > -- > > Key: IOTDB-5472 > URL: https://issues.apache.org/jira/browse/IOTDB-5472 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Engine >Reporter: Qingxin Feng >Assignee: Haiming Zhu >Priority: Minor > Attachments: image-2023-02-06-09-18-27-596.png, > image-2023-02-06-09-22-50-304.png, image-2023-02-08-08-41-55-425.png, > image-2023-02-10-08-55-18-990.png, image-2023-02-10-17-31-06-102.png > > > The number of tsfiles went up between db6f17e 【02/01】and 5602d0e【02/05】 > Please refer to below picture. > > !image-2023-02-06-09-22-50-304.png|width=661,height=380! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5523) Definition of IdentiySinkNode and ShuffleSinkNode
liaolanyu created IOTDB-5523: Summary: Definition of IdentiySinkNode and ShuffleSinkNode Key: IOTDB-5523 URL: https://issues.apache.org/jira/browse/IOTDB-5523 Project: Apache IoTDB Issue Type: Improvement Reporter: liaolanyu Assignee: liaolanyu -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (IOTDB-5467) Execute query : ERROR o.a.i.d.m.e.e.RegionWriteExecutor:88 - Fetch Schema failed
[
https://issues.apache.org/jira/browse/IOTDB-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
刘珍 reopened IOTDB-5467:
---
不过这个异常，需要分析一下：
java.lang.RuntimeException: cannot fetch schema, status is: 301, msg is: There
is not enough memory to execute current fragment instance, current remaining
free memory is 10409, estimated memory usage for current fragment instance is
131072
> Execute query : ERROR o.a.i.d.m.e.e.RegionWriteExecutor:88 - Fetch Schema
> failed
>
>
> Key: IOTDB-5467
> URL: https://issues.apache.org/jira/browse/IOTDB-5467
> Project: Apache IoTDB
> Issue Type: Bug
> Components: mpp-cluster
>Affects Versions: 1.0.1-SNAPSHOT
>Reporter: 刘珍
>Assignee: Minghui Liu
>Priority: Major
> Attachments: auto_set_ttl_per_sg.sh,
> confignode_ip23_leader_logs.tar.gz, datanode_ip2_fetch_schema_failed.tar.gz,
> image-2023-02-03-14-39-21-109.png, image-2023-02-03-14-39-33-195.png, lt.conf
>
>
> 测试版本：rc/1.0.1 20230129 573097a
> 问题描述：
> 启动3副本3C21D集群，
> 2023-1-31 16:40:00 , 启动1个Benchmark连ip2（操作执行间隔OP_INTERVAL=1000）执行读写，
> 设置TTL 为1小时（脚本见附件）
> 2023-02-02 02:21:08 ConfigNode Leader（ip23）报错，连不上ip7的datanode（unkown）
> IP2 datanode log：
> 2023-02-02 02:51:04,625
> [pool-26-IoTDB-DataNodeInternalRPC-Processor-9]{color:red}* ERROR
> o.a.i.d.m.e.e.RegionWriteExecutor:88 - Fetch Schema failed.
> java.lang.RuntimeException: Fetch Schema failed.*{color}
> at
> org.apache.iotdb.db.mpp.plan.analyze.ClusterSchemaFetcher.executeSchemaFetchQuery(ClusterSchemaFetcher.java:202)
> at
> org.apache.iotdb.db.mpp.plan.analyze.ClusterSchemaFetcher.fetchSchema(ClusterSchemaFetcher.java:156)
> at
> org.apache.iotdb.db.mpp.plan.analyze.ClusterSchemaFetcher.fetchSchema(ClusterSchemaFetcher.java:98)
> at
> org.apache.iotdb.db.mpp.plan.analyze.ClusterSchemaFetcher.fetchSchemaWithAutoCreate(ClusterSchemaFetcher.java:265)
> at
> org.apache.iotdb.db.mpp.plan.analyze.SchemaValidator.validate(SchemaValidator.java:56)
> at
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.executeDataInsert(RegionWriteExecutor.java:202)
> at
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:174)
> at
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor$WritePlanNodeExecutionVisitor.visitInsertTablet(RegionWriteExecutor.java:128)
> at
> org.apache.iotdb.db.mpp.plan.planner.plan.node.write.InsertTabletNode.accept(InsertTabletNode.java:1086)
> at
> org.apache.iotdb.db.mpp.execution.executor.RegionWriteExecutor.execute(RegionWriteExecutor.java:86)
> at
> org.apache.iotdb.db.service.thrift.impl.DataNodeInternalRPCServiceImpl.sendPlanNode(DataNodeInternalRPCServiceImpl.java:288)
> at
> org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendPlanNode.getResult(IDataNodeRPCService.java:3607)
> at
> org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendPlanNode.getResult(IDataNodeRPCService.java:3587)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: org.apache.iotdb.commons.exception.IoTDBException:
> org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceFailureInfo$FailureException
> at
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.dealWithException(QueryExecution.java:428)
> at
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.getResult(QueryExecution.java:411)
> at
> org.apache.iotdb.db.mpp.plan.execution.QueryExecution.getBatchResult(QueryExecution.java:437)
> at
> org.apache.iotdb.db.mpp.plan.analyze.ClusterSchemaFetcher.executeSchemaFetchQuery(ClusterSchemaFetcher.java:200)
> ... 18 common frames omitted
> Caused by:
> org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceFailureInfo$FailureException:
> null
> at
> org.apache.iotdb.db.mpp.execution.fragment.FragmentInstanceManager.lambda$cancelTimeoutFlushingInstances$8(FragmentInstanceManager.java:288)
> at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at
> java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
> at
>
