[jira] [Commented] (IOTDB-5461) Is DataRegion related to numbers of threads in confignode
[ https://issues.apache.org/jira/browse/IOTDB-5461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696740#comment-17696740 ] Yongzao Dan commented on IOTDB-5461: No. You can see the [RegionGroup extension|https://apache-iotdb.feishu.cn/docs/doccnHuDkDD6RiwLjaej3hkA9Vb] doc for more details. > Is DataRegion related to numbers of threads in confignode > - > > Key: IOTDB-5461 > URL: https://issues.apache.org/jira/browse/IOTDB-5461 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Yuan Tian >Assignee: Xinyu Tan >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5628) SchemaRegion with 1 replica occurs error when restarting IoTDB server
Jinrui Zhang created IOTDB-5628: --- Summary: SchemaRegion with 1 replica occurs error when restarting IoTDB server Key: IOTDB-5628 URL: https://issues.apache.org/jira/browse/IOTDB-5628 Project: Apache IoTDB Issue Type: Improvement Reporter: Jinrui Zhang Assignee: Jinrui Zhang *Background:* IoTDB server runs for a very long time with lots of timeseries/data. *Operation* shutdown server and restart it. Start client to write data again. *Error* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5627) [Metric] Add Local/Remote Schedule Timer
Hongyin Zhang created IOTDB-5627: Summary: [Metric] Add Local/Remote Schedule Timer Key: IOTDB-5627 URL: https://issues.apache.org/jira/browse/IOTDB-5627 Project: Apache IoTDB Issue Type: Improvement Reporter: Hongyin Zhang Assignee: Hongyin Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5626) SchemaRegion with 1 replica occurs error when restarting IoTDB server
Jinrui Zhang created IOTDB-5626: --- Summary: SchemaRegion with 1 replica occurs error when restarting IoTDB server Key: IOTDB-5626 URL: https://issues.apache.org/jira/browse/IOTDB-5626 Project: Apache IoTDB Issue Type: Improvement Reporter: Jinrui Zhang Assignee: Jinrui Zhang Attachments: image-2023-03-06-15-43-48-809.png *Background:* IoTDB server runs for a very long time with lots of timeseries/data. *Operation* shutdown server and restart it. Start client to write data again. *Error* *!image-2023-03-06-15-43-48-809.png|width=1278,height=248!* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5625) SchemaRegion with 1 replica occurs error when restarting IoTDB server
Jinrui Zhang created IOTDB-5625: --- Summary: SchemaRegion with 1 replica occurs error when restarting IoTDB server Key: IOTDB-5625 URL: https://issues.apache.org/jira/browse/IOTDB-5625 Project: Apache IoTDB Issue Type: Improvement Reporter: Jinrui Zhang Assignee: Jinrui Zhang Attachments: image-2023-03-06-15-41-51-456.png *Background:* IoTDB server runs for a very long time with lots of timeseries/data. *Operation* shutdown server and restart it. Start client to write data again. *Error* *!image-2023-03-06-15-41-51-456.png|width=1278,height=248!* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5274) There is a problem that "show cluster details" is displayed repeatedly
[ https://issues.apache.org/jira/browse/IOTDB-5274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5274: -- Assignee: Yongzao Dan > There is a problem that "show cluster details" is displayed repeatedly > -- > > Key: IOTDB-5274 > URL: https://issues.apache.org/jira/browse/IOTDB-5274 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Server >Reporter: sunhao >Assignee: Yongzao Dan >Priority: Major > Fix For: 1.0.0 > > > 1.Start three iotdb clusters and set the datanodes to 6667, 6668 and 6669 > respectively. When starting the cluster, it is found that the port numbers > 6668 and 6669 have been occupied. Therefore, only one datanode is > successfully started > 2.Because the port number has been occupied, first clear the data directory > of the 6667 and 6668datanodes, modify the port number, and then restart. Then > execute the "show cluster details" command to add two more nodes (6668 and > 6669) to the datanode, which is the breakpoint number set before the > successful startup. That is to say, there were 3 datanodes and now there are > 5. 6668 and 6669 should not be displayed. > 3.Finally, delete all datanodes and confignodes' data directories, restart > them, and the display is normal > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5344) Catch exception thrown by ClientManager and give some friendly tips
[ https://issues.apache.org/jira/browse/IOTDB-5344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5344: -- Assignee: Yongzao Dan (was: Gaofei Cao) Resolution: Duplicate > Catch exception thrown by ClientManager and give some friendly tips > > > Key: IOTDB-5344 > URL: https://issues.apache.org/jira/browse/IOTDB-5344 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Gaofei Cao >Assignee: Yongzao Dan >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5385) Optimize the DataRegion leader calculation policy when ConfigNode leader changes
[ https://issues.apache.org/jira/browse/IOTDB-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5385: -- Assignee: Yongzao Dan (was: Gaofei Cao) Resolution: Won't Fix > Optimize the DataRegion leader calculation policy when ConfigNode leader > changes > > > Key: IOTDB-5385 > URL: https://issues.apache.org/jira/browse/IOTDB-5385 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jinrui Zhang >Assignee: Yongzao Dan >Priority: Major > > We found a phenomenon in recent testing. > That is, the leaders of DataRegions will be updated by new elected ConfigNode > leader. It adds unstable risks to our system sometimes and it will be > optimized. > See this doc for detials. > https://apache-iotdb.feishu.cn/docx/ZTlkdiPwRoXGi0xs2cacYpSlnYb?from=space_persnoal_filelist&pre_pathname=%2Fdrive%2Ffolder%2Ffldcnf4szpemAst96rw3XajU5jb -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4966) Remove datanode to support graceful exit of processes
[ https://issues.apache.org/jira/browse/IOTDB-4966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-4966: -- Assignee: Yongzao Dan (was: Gaofei Cao) > Remove datanode to support graceful exit of processes > - > > Key: IOTDB-4966 > URL: https://issues.apache.org/jira/browse/IOTDB-4966 > Project: Apache IoTDB > Issue Type: Improvement > Components: mpp-cluster >Reporter: 刘珍 >Assignee: Yongzao Dan >Priority: Major > > 缩容datanode，进程退出不够优雅，有一些报错（可加入处理计划，优先级可低）： > 2022-11-16 15:54:13,378 [pool-24-IoTDB-DataNodeInternalRPC-Processor-51] > ERROR o.a.i.c.s.ThriftService:168 - IoTDB: close MPP Data exchange manager > failed because: > java.lang.InterruptedException: null > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1302) > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231) > at > org.apache.iotdb.commons.service.ThriftService.stopService(ThriftService.java:163) > at > org.apache.iotdb.commons.service.ThriftService.stop(ThriftService.java:74) > at > org.apache.iotdb.db.mpp.execution.exchange.MPPDataExchangeService.stop(MPPDataExchangeService.java:128) > at > org.apache.iotdb.commons.service.IService.waitAndStop(IService.java:36) > at > org.apache.iotdb.commons.service.RegisterManager.deregisterAll(RegisterManager.java:60) > at org.apache.iotdb.db.service.DataNode.deactivate(DataNode.java:683) > at org.apache.iotdb.db.service.DataNode.stop(DataNode.java:651) > at > org.apache.iotdb.db.service.thrift.impl.DataNodeInternalRPCServiceImpl.stopDataNode(DataNodeInternalRPCServiceImpl.java:1493) > at > org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$stopDataNode.getResult(IDataNodeRPCService.java:4082) > at > org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$stopDataNode.getResult(IDataNodeRPCService.java:4062) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > 2022-11-16 15:54:13,388 [pool-24-IoTDB-DataNodeInternalRPC-Processor-51] > ERROR o.a.i.c.s.RegisterManager:63 - Failed to stop Storage Engine > ServerService because: > org.apache.iotdb.db.exception.runtime.StorageEngineFailureException: > StorageEngine failed to stop because of Timed-Flush-Unseq-Memtable. > at > org.apache.iotdb.db.utils.ThreadUtils.stopThreadPool(ThreadUtils.java:47) > at > org.apache.iotdb.db.engine.StorageEngineV2.stop(StorageEngineV2.java:389) > at > org.apache.iotdb.commons.service.IService.waitAndStop(IService.java:36) > at > org.apache.iotdb.commons.service.RegisterManager.deregisterAll(RegisterManager.java:60) > at org.apache.iotdb.db.service.DataNode.deactivate(DataNode.java:683) > at org.apache.iotdb.db.service.DataNode.stop(DataNode.java:651) > at > org.apache.iotdb.db.service.thrift.impl.DataNodeInternalRPCServiceImpl.stopDataNode(DataNodeInternalRPCServiceImpl.java:1493) > at > org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$stopDataNode.getResult(IDataNodeRPCService.java:4082) > at > org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$stopDataNode.getResult(IDataNodeRPCService.java:4062) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.InterruptedException: null > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2067) > at > java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475) > at > java.util.concurrent.Executors$DelegatedExecutorService.awaitTermination(Executors.java:675) > at > org.apache.iotdb.commons.concurrent.threadpool.WrappedSingleThreadScheduledExecutor.awaitTermination(WrappedSingleThreadScheduledExec
[jira] [Assigned] (IOTDB-5452) Add leastDataRegionGroupNumSetByUser examination to make that leastDataRegionGroupNum will not be changed by system
[ https://issues.apache.org/jira/browse/IOTDB-5452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5452: -- Assignee: Yongzao Dan (was: Gaofei Cao) Resolution: Won't Fix > Add leastDataRegionGroupNumSetByUser examination to make that > leastDataRegionGroupNum will not be changed by system > --- > > Key: IOTDB-5452 > URL: https://issues.apache.org/jira/browse/IOTDB-5452 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Gaofei Cao >Assignee: Yongzao Dan >Priority: Major > Labels: pull-request-available > > When user changed the conf value `least_data_region_group_num` in > iotdb-common.property, the system should not change this value in backgroud. > So when user changed the value `least_data_region_group_num`, we set > leastDataRegionGroupNumSetByUser as true, so system will not change this > value in backgroud. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5115) [ ConfigNode ] “ConfigNodeRPC-Processor” thread leaks
[ https://issues.apache.org/jira/browse/IOTDB-5115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5115: -- Assignee: Yongzao Dan (was: Gaofei Cao) > [ ConfigNode ] “ConfigNodeRPC-Processor” thread leaks > - > > Key: IOTDB-5115 > URL: https://issues.apache.org/jira/browse/IOTDB-5115 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 1.0.0 >Reporter: 刘珍 >Assignee: Yongzao Dan >Priority: Major > Attachments: image-2022-12-05-09-13-14-368.png > > > rel/1.0 fbbca3f ，1副本，schema/data region CUSTOM 策略 > 启动1个datanode，启动1个BM连接这个datanode创建元数据，每个序列写入1个点。 > 再启动到第15个datanode，15个BM创建原数据时，元数据不能继续创建（实际创建144799 设备，预期应该创建15），ConfigNode > Leader 的ConfigNodeRPC-Processor线程数在不断增长： > !image-2022-12-05-09-13-14-368.png|width=831,height=444! > 测试环境-私有云1期 > 1.172.16.2.2~22 是DataNode（共21台 8C32GB） > BM ：172.16.2.26~46 运行BM （共21台 4C16GB） > ConfigNode ：172.16.2.23~25 （共3台 8C32GB） > 配置参数 > # > config_node_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > # > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > # > data_region_consensus_protocol_class=org.apache.iotdb.consensus.iot.IoTConsensus > # schema_replication_factor=1 > # data_replication_factor=1 > schema_region_group_extension_policy=CUSTOM > schema_region_group_per_database=6 > data_region_group_extension_policy=CUSTOM > data_region_group_per_database=8 > time_partition_interval=6048000 > ConfigNode配置 > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > cn_target_config_node_list=172.16.2.23:22277 > DataNode配置 > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > dn_data_dirs=data/datanode/data,/data1/iotdb/datanode/data > dn_target_config_node_list=172.16.2.23:22277,172.16.2.24:22277,172.16.2.25:22277 > Common 配置 > # > config_node_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > # > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > # > data_region_consensus_protocol_class=org.apache.iotdb.consensus.iot.IoTConsensus > # schema_replication_factor=1 > # data_replication_factor=1 > schema_region_group_extension_policy=CUSTOM > schema_region_group_per_database=6 > data_region_group_extension_policy=CUSTOM > data_region_group_per_database=8 > time_partition_interval=6048000 > 2. BM配置 > DB_SWITCH=IoTDB-014-SESSION_BY_TABLET > GROUP_NUMBER=1 > DEVICE_NUMBER=1 > REAL_INSERT_RATE=1.0 > SENSOR_NUMBER=100 > IS_SENSOR_TS_ALIGNMENT=true > IS_OUT_OF_ORDER=false > OUT_OF_ORDER_RATIO=0.5 > OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0 > CLIENT_NUMBER=15 > LOOP=1 > BATCH_SIZE_PER_WRITE=1 > START_TIME=2018-8-29T00:00:00+08:00 > POINT_STEP=100 > OP_MIN_INTERVAL=0 > OP_MIN_INTERVAL_RANDOM=false > INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1 > ENCODINGS=PLAIN/PLAIN/PLAIN/PLAIN/PLAIN/PLAIN > COMPRESSOR=SNAPPY > IS_DELETE_DATA=false > CREATE_SCHEMA=true > 3. 日志 > ConfigNode Leader > 172.16.2.23 > /data/iotdb/rel_1202_fbbca3f/logs_confignode_thread_leak -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5519) [ confignode] ERROR o.a.i.c.m.p.PartitionManager:320 - Lacked some data partition allocation result in the response of getOrCreateDataPartition method
[ https://issues.apache.org/jira/browse/IOTDB-5519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5519: -- Assignee: Yongzao Dan (was: Xinyu Tan) > [ confignode] ERROR o.a.i.c.m.p.PartitionManager:320 - Lacked some data > partition allocation result in the response of getOrCreateDataPartition method > -- > > Key: IOTDB-5519 > URL: https://issues.apache.org/jira/browse/IOTDB-5519 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 1.1.0-SNAPSHOT >Reporter: 刘珍 >Assignee: Yongzao Dan >Priority: Major > Attachments: iotdb_5519.conf > > > master_0210_8757400 > 问题描述： > 启动3副本3C5D集群，benchmark创建元数据，写入数据。 > 设置集群为readonly （为了避免 insert + drop database 并发报错） > drop database root.** ; 全部drop成功。 > 这时客户端还在继续写入，集群设置为running > 应该能继续自动创建元数据，写入数据，实际是写入全部失败 > confignode报错： > 2023-02-10 17:50:54,128 [pool-11-IoTDB-ConfigNodeRPC-Processor-33] ERROR > o.a.i.c.m.p.PartitionManager:320 - Lacked some data partition allocation > result in the response of getOrCreateDataPartition method > DataNode报错: > 2023-02-10 17:54:37,461 [pool-25-IoTDB-ClientRPC-Processor-155] WARN > o.a.i.d.u.ErrorHandlingUtils:61 - Status code: EXECUTE_STATEMENT_ERROR(301), > operation: insertTablet failed > java.lang.RuntimeException: Failed to get replicaSet of consensus group[id= > TConsensusGroupId(type:DataRegion, id:14)] > at > org.apache.iotdb.db.mpp.plan.analyze.cache.PartitionCache.getRegionReplicaSet(PartitionCache.java:419) > at > org.apache.iotdb.db.mpp.plan.analyze.cache.PartitionCache.getTimeSlotDataPartition(PartitionCache.java:727) > at > org.apache.iotdb.db.mpp.plan.analyze.cache.PartitionCache.getDeviceDataPartition(PartitionCache.java:695) > at > org.apache.iotdb.db.mpp.plan.analyze.cache.PartitionCache.getStorageGroupDataPartition(PartitionCache.java:647) > at > org.apache.iotdb.db.mpp.plan.analyze.cache.PartitionCache.getDataPartition(PartitionCache.java:607) > at > org.apache.iotdb.db.mpp.plan.analyze.ClusterPartitionFetcher.getOrCreateDataPartition(ClusterPartitionFetcher.java:260) > at > org.apache.iotdb.db.mpp.plan.analyze.AnalyzeVisitor.getAnalysisForWriting(AnalyzeVisitor.java:2002) > at > org.apache.iotdb.db.mpp.plan.analyze.AnalyzeVisitor.visitInsertTablet(AnalyzeVisitor.java:1828) > at > org.apache.iotdb.db.mpp.plan.analyze.AnalyzeVisitor.visitInsertTablet(AnalyzeVisitor.java:183) > at > org.apache.iotdb.db.mpp.plan.statement.crud.InsertTabletStatement.accept(InsertTabletStatement.java:121) > at > org.apache.iotdb.db.mpp.plan.statement.StatementVisitor.process(StatementVisitor.java:113) > at > org.apache.iotdb.db.mpp.plan.analyze.Analyzer.analyze(Analyzer.java:48) > at > org.apache.iotdb.db.mpp.plan.execution.QueryExecution.analyze(QueryExecution.java:263) > at > org.apache.iotdb.db.mpp.plan.execution.QueryExecution.(QueryExecution.java:147) > at > org.apache.iotdb.db.mpp.plan.Coordinator.createQueryExecution(Coordinator.java:107) > at > org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:140) > at > org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:167) > at > org.apache.iotdb.db.service.thrift.impl.ClientRPCServiceImpl.insertTablet(ClientRPCServiceImpl.java:1357) > at > org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:4279) > at > org.apache.iotdb.service.rpc.thrift.IClientRPCService$Processor$insertTablet.getResult(IClientRPCService.java:4259) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at > org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:834) > 测试环境 私有云1期 > 1.3副本3C5D > 3confignode ：172.16.2.23/24 /25 /data/iotdb/m_0210_8757400/logs > 5datanode : 172.16.2.2/3/4/5/6 /data1/iotdb/m_0210_8757400/logs > benchmark 在 172.16.2.26 /data/iotdb/benchmark_v1 > 数据库配置 > ConfigNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > DataNode > MAX_HEAP_SIZE="20G" > MAX_DIRECT_MEMORY_SIZE="6G" > Common配置 > schema_replication_factor=3 > data_replication_factor=3 > 2. 启动benchmark 创建元数据，写入数据 > 3. 设置集群整体状态为readonly > 4.drop databas
[jira] [Assigned] (IOTDB-5566) Give a interface to show the configurations of IoTDB in command window
[ https://issues.apache.org/jira/browse/IOTDB-5566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5566: -- Assignee: Yongzao Dan (was: Xinyu Tan) > Give a interface to show the configurations of IoTDB in command window > -- > > Key: IOTDB-5566 > URL: https://issues.apache.org/jira/browse/IOTDB-5566 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: changxue >Assignee: Yongzao Dan >Priority: Major > > Give a interface to show the configurations of IoTDB in command window > The configurations of iotdb-common.properties, iotdb-confignode.properties > and iotdb-datanode.properties should be shown in command window(cli), because > users may update these configurations and would like to make sure whether the > modification is effective. > Like mysql: show variables mem% > > expect: > # show variables: should show all variables that set in IoTDB > # considering there are many configurations, should give a wildcard format > to show specified variables > > actual: > # show variables does not show all > # no wildcard > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5609) [ remove-confignode.sh/remove-datanode.sh ] -help / --help
[ https://issues.apache.org/jira/browse/IOTDB-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-5609: -- Assignee: Yongzao Dan (was: Xinyu Tan) > [ remove-confignode.sh/remove-datanode.sh ] -help / --help > --- > > Key: IOTDB-5609 > URL: https://issues.apache.org/jira/browse/IOTDB-5609 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 1.1.0-SNAPSHOT >Reporter: 刘珍 >Assignee: Yongzao Dan >Priority: Major > Attachments: image-2023-03-02-10-59-57-503.png, > image-2023-03-02-11-01-34-630.png > > > remove-confignode.sh/remove-datanode.sh > 支持-help/--help参数，打印HELP 信息 > !image-2023-03-02-10-59-57-503.png! > !image-2023-03-02-11-01-34-630.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5624) [Metric] Optimize the organization of ConfigNode Dashboard
Hongyin Zhang created IOTDB-5624: Summary: [Metric] Optimize the organization of ConfigNode Dashboard Key: IOTDB-5624 URL: https://issues.apache.org/jira/browse/IOTDB-5624 Project: Apache IoTDB Issue Type: Improvement Reporter: Hongyin Zhang Assignee: Hongyin Zhang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-5571) [trigger]trigger does not work properly when run with other testcases
[
https://issues.apache.org/jira/browse/IOTDB-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuan Tian reassigned IOTDB-5571:
Assignee: liaolanyu (was: Yuan Tian)
> [trigger]trigger does not work properly when run with other testcases
> --
>
> Key: IOTDB-5571
> URL: https://issues.apache.org/jira/browse/IOTDB-5571
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Query
>Reporter: changxue
>Assignee: liaolanyu
>Priority: Major
> Attachments: TriggerTest.java, conf-ext-logs.tar.gz,
> testcase-run.tar.gz
>
>
> [trigger]trigger does not work properly when run with other testcases
> env:
> 2023-2-21 master source codes
> version 1.1.0-SNAPSHOT (Build: d6e015e-dev)
> bug description:
> data_types.run line 236
> {code:java}
> select count(*) from root.target.alerting; {code}
> expect: 17
> actual: 16
> missing this value:
> {code:java}
> [1]this statement is too long to p|root.triggertest.nonaligned_dev1.s_5
> {code}
> reproduction:
> 1. start IoTDB with the config of attachment conf-ext-logs.tar.gz
> 2. run testcases of testcase-run.tar.gz (ask me about how to run)
> 3. check data_types.out
> ps:
> 1. It's all right when running data_types.run alone, but it fails when
> running with other test cases of testcase-run.tar.gz.
> 2. The Trigger Class using in tests are synchronized
> 3. It is not a trigger problem maybe but a problem related to cleanup.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-5623) [compaction] The optimized version compacts slowly
刘珍 created IOTDB-5623: - Summary: [compaction] The optimized version compacts slowly Key: IOTDB-5623 URL: https://issues.apache.org/jira/browse/IOTDB-5623 Project: Apache IoTDB Issue Type: Improvement Components: Core/Compaction, mpp-cluster Reporter: 刘珍 Assignee: 周沛辰 Attachments: comp1.conf, comp2.conf, image-2023-03-06-11-50-02-812.png, run_2_comp.sh, test.sh 对比balance合并调度策略优化后版本与优化前版本的合并性能，优化后版本慢80%： !image-2023-03-06-11-50-02-812.png! 只有合并，无其他操作，数据50%乱序。 测试环境：192.168.130.1 1C1D 16CPU 32G内存 1.关合并，生成数据备份 /data1/iotdb/m_0227_74c950d/data_close_comp_1partition CN ：MAX_HEAP_SIZE="2G" DN：MAX_HEAP_SIZE="20G" MAX_DIRECT_MEMORY_SIZE="6G" COMMON ： time_partition_interval=604800 max_waiting_time_when_insert_blocked=360 enable_seq_space_compaction=false enable_unseq_space_compaction=false enable_cross_space_compaction=false benchmark配置见附件 2. copy 同一份数据，对比2个版本的合并性能 2个版本的显式设置的参数： CN ： MAX_HEAP_SIZE="2G" DN ： MAX_HEAP_SIZE="20G" MAX_DIRECT_MEMORY_SIZE="6G" COMMON: time_partition_interval=604800 query_timeout_threshold=3600 max_waiting_time_when_insert_blocked=360 enable_seq_space_compaction=true enable_unseq_space_compaction=true enable_cross_space_compaction=true compaction_write_throughput_mb_per_sec=1024 2个版本的参数对比 优化版多了一个参数： opt_0302_33cd749/conf/iotdb-common.properties 优化版 max_number_of_points_in_page=1 优化前的版本此参数 max_number_of_points_in_page=1048576 3.两个 版本日志路径 /data1/iotdb/bef_opt_854e4c8/logs /data1/iotdb/opt_0302_33cd749/logs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5622) Create and use aligned template but devices is non-aligned
yanze chen created IOTDB-5622: - Summary: Create and use aligned template but devices is non-aligned Key: IOTDB-5622 URL: https://issues.apache.org/jira/browse/IOTDB-5622 Project: Apache IoTDB Issue Type: Bug Reporter: yanze chen Assignee: yanze chen Attachments: image-2023-03-06-10-30-00-008.png, image-2023-03-06-10-30-18-520.png !image-2023-03-06-10-30-00-008.png|width=608,height=215! !image-2023-03-06-10-30-18-520.png|width=607,height=205! devices root.sg1.d1 should be aligned -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5621) Eliminate the deprecated code in the template
yanze chen created IOTDB-5621: - Summary: Eliminate the deprecated code in the template Key: IOTDB-5621 URL: https://issues.apache.org/jira/browse/IOTDB-5621 Project: Apache IoTDB Issue Type: Improvement Reporter: yanze chen Assignee: yanze chen As the current templates only support single level templates that are either aligned or non-aligned, much of the deprecated code in template can be removed. In addition, the responsibility of the template should be managed for MeasurementSchema and not for IMNode. This has also been removed in this issue. To guarantee compatibility, the refactoring is done from the lower level of Statement. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-5192) [CrossSpaceCompaction] The level of compaction file is too large
[
https://issues.apache.org/jira/browse/IOTDB-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696649#comment-17696649
]
刘珍 commented on IOTDB-5192:
---
master_0302_33cd749 : balance合并任务调度优化，跨空间合并合出40+层级tsfile
!screenshot-1.png!
测试环境：3副本3C3D
192.168.10.62/66/68
ConfigNode ENV:
MAX_HEAP_SIZE="8G"
DataNode ENV:
MAX_HEAP_SIZE="192G"
MAX_DIRECT_MEMORY_SIZE="32G"
COMMON PROP:
schema_replication_factor=3
data_replication_factor=3
disk_space_warning_threshold=0.01
compaction_write_throughput_mb_per_sec=64
iot_consensus_throttle_threshold_in_byte=536870912000
benchmark配置文件见附件comp_2023.conf
> [CrossSpaceCompaction] The level of compaction file is too large
> -
>
> Key: IOTDB-5192
> URL: https://issues.apache.org/jira/browse/IOTDB-5192
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: mpp-cluster
>Reporter: 刘珍
>Assignee: 周沛辰
>Priority: Major
> Attachments: fast.conf, image-2022-12-13-16-06-23-875.png,
> screenshot-1.png
>
>
> master 1208 7f2218b
> 跨空间合并，合并文件层级过大（32层）：
> !image-2022-12-13-16-06-23-875.png!
> 日志：
> 2022-12-13 14:37:09,832 [pool-64-IoTDB-Compaction-1] INFO
> o.a.i.d.e.c.c.CrossSpaceCompactionTask:219 - root.test.g_1-10 [Compaction]
> CrossSpaceCompaction task finishes successfully, time cost is
> {color:#DE350B}*18029 *{color}s, compaction speed is 3.2967052791379006 MB/s
> 备份数据：
> fit 机器 192.168.10.72
> /data/mpp_test/m_1207_a0b2c8c/data
> 数据库配置：
> ConfigNode配置
> MAX_HEAP_SIZE="8G"
> DataNode配置
> MAX_HEAP_SIZE="256G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> Common配置（此版本合并全失败，可以设置为关合并）
> 备份数据生成方式：
> BM配置文件见附件。
> master 1208 7f2218b Common配置 （confignode和datanode配置同上）：
> cross_performer=fast
> inner_unseq_performer=fast
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IOTDB-5620) Cannot flush when there is a log of time partitions in each DataRegion
[ https://issues.apache.org/jira/browse/IOTDB-5620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-5620: - Assignee: Haonan Hou > Cannot flush when there is a log of time partitions in each DataRegion > -- > > Key: IOTDB-5620 > URL: https://issues.apache.org/jira/browse/IOTDB-5620 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 1.0.1 >Reporter: Haonan Hou >Assignee: Haonan Hou >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-5620) Cannot flush when there is a log of time partitions in each DataRegion
Haonan Hou created IOTDB-5620: - Summary: Cannot flush when there is a log of time partitions in each DataRegion Key: IOTDB-5620 URL: https://issues.apache.org/jira/browse/IOTDB-5620 Project: Apache IoTDB Issue Type: Bug Affects Versions: 1.0.1 Reporter: Haonan Hou -- This message was sent by Atlassian Jira (v8.20.10#820010)
