[jira] [Assigned] (IOTDB-2669) Improvement of the C++ client
[ https://issues.apache.org/jira/browse/IOTDB-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-2669: -- Assignee: ITAMI SHO (was: Jialin Qiao) > Improvement of the C++ client > - > > Key: IOTDB-2669 > URL: https://issues.apache.org/jira/browse/IOTDB-2669 > Project: Apache IoTDB > Issue Type: Improvement > Components: Client/C++ >Reporter: Yuan Tian >Assignee: ITAMI SHO >Priority: Major > Attachments: image-2022-03-06-14-28-56-027.png, > image-2022-03-06-14-29-24-821.png > > Original Estimate: 40h > Remaining Estimate: 40h > > Currently, in C++ client, we convert all types of data to string and then > save them in Tablet. While call the rpc interface to insert tablet, we > convert that from string to the corresponding dataType. > !image-2022-03-06-14-28-56-027.png! > !image-2022-03-06-14-29-24-821.png! > Actually, these conversions can be removed which will improve the C++ > client's ingest throughput. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3583) [ ConfigNode ] Failed to connect to ConfigNode leader
刘珍 created IOTDB-3583:
-
Summary: [ ConfigNode ] Failed to connect to ConfigNode leader
Key: IOTDB-3583
URL: https://issues.apache.org/jira/browse/IOTDB-3583
Project: Apache IoTDB
Issue Type: Bug
Components: mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Yongzao Dan
Attachments: config.properties
https://github.com/apache/iotdb/pull/6325
请求较多时无法转发给 ConfigNode leader，导致请求失败。
日志
2022-06-21 13:57:29,135 [main] WARN o.a.i.d.c.ConfigNodeClient:234 - Failed to
connect to ConfigNode TEndPoint(ip:192.168.10.68, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),{color:red}*because the current node is
not leader,try next node*{color}
2022-06-21 13:57:29,137 [main] WARN o.a.i.d.c.ConfigNodeClient:234 - Failed to
connect to ConfigNode TEndPoint(ip:192.168.10.68, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),because the current node is not
leader,try next node
2022-06-21 14:02:43,765 [pool-23-IoTDB-RPC-Client-1] WARN
o.a.i.d.c.ConfigNodeClient:234 - Failed to connect to ConfigNode
TEndPoint(ip:192.168.10.68, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),because the current node is not
leader,try next node
2022-06-21 14:02:43,766 [pool-23-IoTDB-RPC-Client-1] WARN
o.a.i.d.c.ConfigNodeClient:234 - Failed to connect to ConfigNode
TEndPoint(ip:192.168.10.66, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),because the current node is not
leader,try next node
2022-06-21 14:02:44,028 [20220621_060244_0_4-69] WARN
o.a.i.d.c.ConfigNodeClient:234 - Failed to connect to ConfigNode
TEndPoint(ip:192.168.10.68, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),because the current node is not
leader,try next node
2022-06-21 14:02:44,028 [20220621_060244_0_4-69] WARN
o.a.i.d.c.ConfigNodeClient:234 - Failed to connect to ConfigNode
TEndPoint(ip:192.168.10.66, port:22277) from DataNode
TEndPoint(ip:192.168.10.66, port:6667),because the current node is not
leader,try next node
~
复现步骤
1. 192.168.10.62/66/68 72C256G
data_region_consensus_protocol_class=org.apache.iotdb.consensus.standalone.StandAloneConsensus
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.standalone.StandAloneConsensus
schema_replication_factor=1
data_replication_factor=1
2. 100用户并发创建元数据
配置文件见附件。
DEVICE_NUMBER=10
SENSOR_NUMBER=6
CLIENT_NUMBER=100
GROUP_NUMBER=1
ip68 bm路径：/data/benchmark/weekly_shell/bm_0527_e7b23aa
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Assigned] (IOTDB-3582) Deleting a storage group with a ** path reports an error
[
https://issues.apache.org/jira/browse/IOTDB-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jialin Qiao reassigned IOTDB-3582:
--
Assignee: LuMing (was: Jialin Qiao)
> Deleting a storage group with a ** path reports an error
>
>
> Key: IOTDB-3582
> URL: https://issues.apache.org/jira/browse/IOTDB-3582
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Cluster, mpp-cluster
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: xiaozhihong
>Assignee: LuMing
>Priority: Major
> Attachments: logs.zip
>
>
> During the test, the storage group containing ** in the path was deleted many
> times, and the deletion of the storage group group failed occasionally.
> {code:java}
> IoTDB> delete storage group root.**;
> Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "delete storage
> group root.**". executeStatement failed. error code: TSStatus(code:500,
> message:Error in calling method deleteStorageGroups)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IOTDB-3582) Deleting a storage group with a ** path reports an error
xiaozhihong created IOTDB-3582:
--
Summary: Deleting a storage group with a ** path reports an error
Key: IOTDB-3582
URL: https://issues.apache.org/jira/browse/IOTDB-3582
Project: Apache IoTDB
Issue Type: Bug
Affects Versions: 0.14.0-SNAPSHOT
Reporter: xiaozhihong
Assignee: Jialin Qiao
Attachments: logs.zip
During the test, the storage group containing ** in the path was deleted many
times, and the deletion of the storage group group failed occasionally.
{code:java}
IoTDB> delete storage group root.**;
Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "delete storage
group root.**". executeStatement failed. error code: TSStatus(code:500,
message:Error in calling method deleteStorageGroups)
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IOTDB-3581) Unify the DataStruct between client and server to TsBlock
Yuan Tian created IOTDB-3581: Summary: Unify the DataStruct between client and server to TsBlock Key: IOTDB-3581 URL: https://issues.apache.org/jira/browse/IOTDB-3581 Project: Apache IoTDB Issue Type: New Feature Reporter: Yuan Tian Between different dataNodes, we use TsBlock as our transferring data structure. But between client and dataNodes, we still use original TSQueryDataSet. TSQueryDataSet doesn't have some encoding optimization like RLE etc which has been implemented in Serde Utils for TsBlock. So we want to add a new interface to support directly transferring TsBlock between client and DataNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3580) Support ORDER BY DEVICE in align by device query
Yuan Tian created IOTDB-3580: Summary: Support ORDER BY DEVICE in align by device query Key: IOTDB-3580 URL: https://issues.apache.org/jira/browse/IOTDB-3580 Project: Apache IoTDB Issue Type: New Feature Reporter: Yuan Tian In current mpp and original standalone version, for align by device query, we don't hava an order for the result set. Records in one device will be ordered by time, but different devices are in a hash order which depends on the sequence they are in the iteration of HashMap. Some users want to order all the devices by their alphabetical order, so we need to implement a new syntax like ORDER BY DEVICE. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3579) Effectively support alias in last query
Yuan Tian created IOTDB-3579: Summary: Effectively support alias in last query Key: IOTDB-3579 URL: https://issues.apache.org/jira/browse/IOTDB-3579 Project: Apache IoTDB Issue Type: New Feature Reporter: Yuan Tian Attachments: image-2022-06-21-13-31-45-287.png In current mpp version, last query doesn't support timesereis alias, like if you have a timeseries named root.sg.d1.s1 with an alias speed, then you do the query "select last s1, speed from root.sg.d1", you will only get one record, like the following: !image-2022-06-21-13-31-45-287.png! which should have two records. In the old standalone IoTDB, it will rightly return two records, but its implementation is not effective. It treat root.sg.d1.s1 and root.sg.d1.speed as two different timesereis, and query them twice. So we hope in new mpp version, we can support alias in last query effectively which means that we should only query once and return two records. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3536) [ flush ] ConfigNode : org.apache.thrift.TException: Error in calling method flush
[ https://issues.apache.org/jira/browse/IOTDB-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556646#comment-17556646 ] 任宇华 commented on IOTDB-3536: Previously, flush requests were sent synchronously, and each send will lead to blocking. Therefore, the failure of datanode connection may be due to too long waiting for syncdatanodeclientpool > [ flush ] ConfigNode : org.apache.thrift.TException: Error in calling method > flush > -- > > Key: IOTDB-3536 > URL: https://issues.apache.org/jira/browse/IOTDB-3536 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.14.0-SNAPSHOT >Reporter: 刘珍 >Assignee: 任宇华 >Priority: Major > Labels: pull-request-available > Attachments: image-2022-06-20-09-58-41-791.png, > image-2022-06-20-09-59-46-382.png, image-2022-06-20-09-59-54-181.png > > > master_0617_1840cd1 > 3节点3副本，写入数据，执行flush ： > !image-2022-06-20-09-58-41-791.png! > ConfigNode的日志 ： > 2022-06-17 17:47:05,929 [pool-2-IoTDB-ConfigNodeRPC-Client-74] ERROR > o.a.i.c.c.SyncDataNodeClientPool:135 - flush on DataNode > TEndPoint(ip:192.168.10.68, port:9003) failed > org.apache.thrift.TException: Error in calling method flush > at > org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94) > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.flush() > at > org.apache.iotdb.confignode.client.SyncDataNodeClientPool.flush(SyncDataNodeClientPool.java:130) > at > org.apache.iotdb.confignode.service.thrift.ConfigNodeRPCServiceProcessor.flush(ConfigNodeRPCServiceProcessor.java:440) > at > org.apache.iotdb.confignode.rpc.thrift.ConfigIService$Processor$flush.getResult(ConfigIService.java:2361) > at > org.apache.iotdb.confignode.rpc.thrift.ConfigIService$Processor$flush.getResult(ConfigIService.java:2341) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.thrift.TException: Error in calling method recv_flush > at > org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94) > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.recv_flush() > at > org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.flush(InternalService.java:591) > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.CGLIB$flush$61() > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79$$FastClassByCGLIB$$d4c42083.invoke() > at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) > at > org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55) > ... 11 common frames omitted > Caused by: org.apache.thrift.TException: Error in calling method receiveBase > at > org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94) > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.receiveBase() > at > org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_flush(InternalService.java:604) > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.CGLIB$recv_flush$56() > at > org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79$$FastClassByCGLIB$$d4c42083.invoke() > at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) > at > org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55) > ... 17 common frames omitted > Caused by: org.apache.thrift.transport.TTransportException: > java.net.SocketTimeoutException: Read timed out > at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:178) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) > at > org.apache.iotdb.rpc.TElasticFrame
[jira] [Assigned] (IOTDB-3558) Print the thread stack for all DataNode and ConfigNode in IT Framework
[ https://issues.apache.org/jira/browse/IOTDB-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Tian reassigned IOTDB-3558: Assignee: Eric Pai > Print the thread stack for all DataNode and ConfigNode in IT Framework > -- > > Key: IOTDB-3558 > URL: https://issues.apache.org/jira/browse/IOTDB-3558 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Assignee: Eric Pai >Priority: Major > > When some queries are time out, there may be deadlock or alive lock in > system, so iit will be better for our debuging that New Cluster IT Framework > can print the thread stack for all DataNode and ConfigNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3578) Support core requirements of template
Yukun Zhou created IOTDB-3578: - Summary: Support core requirements of template Key: IOTDB-3578 URL: https://issues.apache.org/jira/browse/IOTDB-3578 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou Assignee: Yukun Zhou Fix For: master branch The core requirements include: # deliver template from configNode to dataNode # mount template on specific path on involving dataNodes # activate template on specific path on involving dataNode -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3528) Filter DataNode which may not be ready in ConfigNode
[ https://issues.apache.org/jira/browse/IOTDB-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3528: -- Assignee: xiaojiang > Filter DataNode which may not be ready in ConfigNode > > > Key: IOTDB-3528 > URL: https://issues.apache.org/jira/browse/IOTDB-3528 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Assignee: xiaojiang >Priority: Major > > If data structure in DataNode A has not been initialized，but it has > registered itself into ConfigNode，and meanwhile its Internal RPC Service has > been set up. At this time if someone send request through another DataNode > B，then the request is dispatched to DataNode A，it will cause NPE. > So we should filter dataNodes which may not be ready in ConfigNode -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3560) Support basic requirements of template
[ https://issues.apache.org/jira/browse/IOTDB-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3560: -- Assignee: xiaojiang (was: Yukun Zhou) > Support basic requirements of template > -- > > Key: IOTDB-3560 > URL: https://issues.apache.org/jira/browse/IOTDB-3560 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yukun Zhou >Assignee: xiaojiang >Priority: Major > Fix For: master branch > > > The basic requirements includes: > # Create template and store it on configNode > # Deliver template to dataNode and cache it on dataNode > # Mount and activate template on involving dataNodes -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3564) Reduce the number of I/O threads using thrift asynchronous server mode for MultiLeaderConsensusRPC
[ https://issues.apache.org/jira/browse/IOTDB-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3564: -- Assignee: xiaojiang > Reduce the number of I/O threads using thrift asynchronous server mode for > MultiLeaderConsensusRPC > -- > > Key: IOTDB-3564 > URL: https://issues.apache.org/jira/browse/IOTDB-3564 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Xinyu Tan >Assignee: xiaojiang >Priority: Major > > Consider using selector mode or HSHA mode and abstracting out the > corresponding parameters -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3391) Judge ConfigNode status through heartbeat
[ https://issues.apache.org/jira/browse/IOTDB-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3391: -- Assignee: xiaojiang > Judge ConfigNode status through heartbeat > - > > Key: IOTDB-3391 > URL: https://issues.apache.org/jira/browse/IOTDB-3391 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Yongzao Dan >Assignee: xiaojiang >Priority: Major > Fix For: 0.14.0 > > > When we use "show cluster", the node status will be shown. However, we do not > have a heartbeat among leader confignode with follower confignodes. > This needs to be added and update the ConfigNode status. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3577) Boolean and TEXT perform linear null fill prompt error
xiaozhihong created IOTDB-3577:
--
Summary: Boolean and TEXT perform linear null fill prompt error
Key: IOTDB-3577
URL: https://issues.apache.org/jira/browse/IOTDB-3577
Project: Apache IoTDB
Issue Type: Bug
Components: Core/Cluster, mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: xiaozhihong
Assignee: Minghui Liu
Attachments: image-2022-06-21-11-42-41-224.png
The data type is not supported for boolean and text, linear null value filling,
and the error message is unclear during execution.
{code:java}
IoTDB> select status, temperature from root.ln.wf01.wt01 where time <
157308480 fill(linear);
Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "select status,
temperature from root.ln.wf01.wt01 where time < 157308480 fill(linear)".
executeStatement failed. error code: TSStatus(code:411, message:Fragment cannot
be dispatched) {code}
The log is:
!image-2022-06-21-11-42-41-224.png!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (IOTDB-3576) [ thread ] Controls the number of "ForkJoinPool.commonPool-worker"
刘珍 created IOTDB-3576: - Summary: [ thread ] Controls the number of "ForkJoinPool.commonPool-worker" Key: IOTDB-3576 URL: https://issues.apache.org/jira/browse/IOTDB-3576 Project: Apache IoTDB Issue Type: Bug Components: mpp-cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: 刘珍 Assignee: Yongzao Dan Attachments: 0621_stack_confignode.out MultiLeaderConsensus，72cpu机器，21个dataregion，1个schemaregion，单个confignode进程的"ForkJoinPool.commonPool-worker"线程数过多，有47个。需要线程控制。 stack详细信息见附件。 数据库配置参数： data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus schema_replication_factor=3 data_replication_factor=3 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3575) Fill should not check dataType for device column
[ https://issues.apache.org/jira/browse/IOTDB-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Tian reassigned IOTDB-3575: Attachment: image-2022-06-21-11-01-48-794.png Sprint: 2022-6-Cluster Assignee: Minghui Liu Description: select count(s1), sum(s2), avg(s1) from root.sg1.d1 where s3 > 5 and time < 30 GROUP BY ([1, 36), 5ms) FILL (3.14) align by device !image-2022-06-21-11-01-48-794.png! Summary: Fill should not check dataType for device column (was: Fill should not check dataType) > Fill should not check dataType for device column > > > Key: IOTDB-3575 > URL: https://issues.apache.org/jira/browse/IOTDB-3575 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Yuan Tian >Assignee: Minghui Liu >Priority: Major > Attachments: image-2022-06-21-11-01-48-794.png > > > select count(s1), sum(s2), avg(s1) from root.sg1.d1 where s3 > 5 and time < > 30 GROUP BY ([1, 36), 5ms) FILL (3.14) align by device > > !image-2022-06-21-11-01-48-794.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3575) Fill should not check dataType
Yuan Tian created IOTDB-3575: Summary: Fill should not check dataType Key: IOTDB-3575 URL: https://issues.apache.org/jira/browse/IOTDB-3575 Project: Apache IoTDB Issue Type: Bug Reporter: Yuan Tian -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3574) [ thread ] Controls the number of “IoTDB-ConfigNodeRPC-Client ”
刘珍 created IOTDB-3574: - Summary: [ thread ] Controls the number of “IoTDB-ConfigNodeRPC-Client ” Key: IOTDB-3574 URL: https://issues.apache.org/jira/browse/IOTDB-3574 Project: Apache IoTDB Issue Type: Bug Components: mpp-cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: 刘珍 Assignee: Yongzao Dan Attachments: 0621_stack_confignode.out MultiLeaderConsensus，72cpu机器，21个dataregion，1个schemaregion，单个confignode进程的IoTDB-ConfigNodeRPC-Client线程数过多，有72个，看起来=cpu数。需要线程控制。 stack详细信息见附件。 数据库配置参数： data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus schema_replication_factor=3 data_replication_factor=3 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3559) Add metrics for the consensus module
Xinyu Tan created IOTDB-3559: Summary: Add metrics for the consensus module Key: IOTDB-3559 URL: https://issues.apache.org/jira/browse/IOTDB-3559 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3573) Refine pom.xml in integration-test
Eric Pai created IOTDB-3573: --- Summary: Refine pom.xml in integration-test Key: IOTDB-3573 URL: https://issues.apache.org/jira/browse/IOTDB-3573 Project: Apache IoTDB Issue Type: Improvement Components: Core/Others Reporter: Eric Pai Assignee: Eric Pai Fix For: master branch # Refine codes in pom.xml to reduce duplicated plugin declearations. # Use maven-replacer-plugin to replace configs in different OS. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3572) NPE when executing ALTER TIMESERIES in new standalone
Minghui Liu created IOTDB-3572: -- Summary: NPE when executing ALTER TIMESERIES in new standalone Key: IOTDB-3572 URL: https://issues.apache.org/jira/browse/IOTDB-3572 Project: Apache IoTDB Issue Type: Bug Reporter: Minghui Liu Assignee: Jinrui Zhang ALTER timeseries root.sg.d2.s3 UPSERT ALIAS='powerNew' 2022-06-21 10:10:34,935 [pool-19-IoTDB-RPC-Client-12] ERROR o.a.i.d.u.ErrorHandlingUtils:60 - Status code: INTERNAL_SERVER_ERROR(500), operation: "ALTER timeseries root.sg.d2.s3 UPSERT ALIAS='powerNew'". executeStatement failed java.lang.NullPointerException: null at org.apache.iotdb.db.mpp.plan.planner.plan.FragmentInstance.setDataRegionAndHost(FragmentInstance.java:86) at org.apache.iotdb.db.mpp.plan.planner.distribution.WriteFragmentParallelPlanner.parallelPlan(WriteFragmentParallelPlanner.java:65) at org.apache.iotdb.db.mpp.plan.planner.distribution.DistributionPlanner.planFragmentInstances(DistributionPlanner.java:95) at org.apache.iotdb.db.mpp.plan.planner.distribution.DistributionPlanner.planFragments(DistributionPlanner.java:79) at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.doDistributedPlan(QueryExecution.java:233) at org.apache.iotdb.db.mpp.plan.execution.QueryExecution.start(QueryExecution.java:165) at org.apache.iotdb.db.mpp.plan.Coordinator.execute(Coordinator.java:133) at org.apache.iotdb.db.service.thrift.impl.DataNodeTSIServiceImpl.executeStatement(DataNodeTSIServiceImpl.java:551) at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2853) at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2833) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3571) Add IT for UDTF Non Align
liaolanyu created IOTDB-3571: Summary: Add IT for UDTF Non Align Key: IOTDB-3571 URL: https://issues.apache.org/jira/browse/IOTDB-3571 Project: Apache IoTDB Issue Type: Improvement Reporter: liaolanyu Assignee: liaolanyu -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3570) Extend the Peer structure of the consensus layer to embed the ID of the upper layer
Xinyu Tan created IOTDB-3570: Summary: Extend the Peer structure of the consensus layer to embed the ID of the upper layer Key: IOTDB-3570 URL: https://issues.apache.org/jira/browse/IOTDB-3570 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan Currently, the Peer returned by the getLeader interface only contains the IP address and port corresponding to the consensus layer of the Leader node. However, the port that the upper layer wants to obtain may be the service port that the upper layer RPC can connect to, such as internalService, etc. Therefore, we can consider extending the structure of the Peer so that it can be packed with a business custom ID structure, so that the upper layer can be returned at getLeader with the ID with business semantics defined for each Peer at AddConsensusGroup, thus reducing the coding burden on the upper layer. For example, DataNode can encode the TEndpoint of internalService into the ID -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3569) Use iterator batch interface under MultiLeaderConsensus to get logs from WAL logs at high speed
Xinyu Tan created IOTDB-3569: Summary: Use iterator batch interface under MultiLeaderConsensus to get logs from WAL logs at high speed Key: IOTDB-3569 URL: https://issues.apache.org/jira/browse/IOTDB-3569 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan Using the batch interface directly may result in OOM -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3568) Support linearizable read for RatisConsensus
Xinyu Tan created IOTDB-3568: Summary: Support linearizable read for RatisConsensus Key: IOTDB-3568 URL: https://issues.apache.org/jira/browse/IOTDB-3568 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan * We can contribute to the Ratis community to support linear consistent reading * It is also possible to add additional coordination logic on top of the RatisConsensus to satisfy linearizable read -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3567) Support Tree Structure template
Yukun Zhou created IOTDB-3567: - Summary: Support Tree Structure template Key: IOTDB-3567 URL: https://issues.apache.org/jira/browse/IOTDB-3567 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou Assignee: Yukun Zhou Fix For: master branch The template represents a logic subtree under the path it activated on. The inner structure will be extended from one level to multi level. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3566) Add IT for UDTFHybridQuery
liaolanyu created IOTDB-3566: Summary: Add IT for UDTFHybridQuery Key: IOTDB-3566 URL: https://issues.apache.org/jira/browse/IOTDB-3566 Project: Apache IoTDB Issue Type: Improvement Reporter: liaolanyu Assignee: liaolanyu -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3564) Reduce the number of I/O threads using thrift asynchronous server mode for MultiLeaderConsensusRPC
Xinyu Tan created IOTDB-3564: Summary: Reduce the number of I/O threads using thrift asynchronous server mode for MultiLeaderConsensusRPC Key: IOTDB-3564 URL: https://issues.apache.org/jira/browse/IOTDB-3564 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan Consider using selector mode or hahs mode and abstracting out the corresponding parameters -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3565) Support modify template
Yukun Zhou created IOTDB-3565: - Summary: Support modify template Key: IOTDB-3565 URL: https://issues.apache.org/jira/browse/IOTDB-3565 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou Assignee: Yukun Zhou Fix For: master branch # add measurement # delete measurement, notice that if the template is activated on some path, the data should be considered -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3563) Support query template
Yukun Zhou created IOTDB-3563: - Summary: Support query template Key: IOTDB-3563 URL: https://issues.apache.org/jira/browse/IOTDB-3563 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou # Query the info of template it self, including measurements and isAligned. # Query the mounted path of template # Query the path template activated on -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3563) Support query template
[ https://issues.apache.org/jira/browse/IOTDB-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yukun Zhou reassigned IOTDB-3563: - Fix Version/s: master branch Sprint: 2022-7-Cluster Assignee: Yukun Zhou > Support query template > -- > > Key: IOTDB-3563 > URL: https://issues.apache.org/jira/browse/IOTDB-3563 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yukun Zhou >Assignee: Yukun Zhou >Priority: Major > Fix For: master branch > > > # Query the info of template it self, including measurements and isAligned. > # Query the mounted path of template > # Query the path template activated on -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3562) Support unmount and delete template
Yukun Zhou created IOTDB-3562: - Summary: Support unmount and delete template Key: IOTDB-3562 URL: https://issues.apache.org/jira/browse/IOTDB-3562 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou Assignee: Yukun Zhou Fix For: master branch Unmount template includes the following requirements: # support soft unmount template, which means if this template is activated on some path, it should not be unmounted # support hard unmount template, which means unmount the template directly and delete all related data Delete template includes the similar requirements, the difference is that a unmount template can be used on another path. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3561) Support snapshot transfer under MultiLeaderConsensus
Xinyu Tan created IOTDB-3561: Summary: Support snapshot transfer under MultiLeaderConsensus Key: IOTDB-3561 URL: https://issues.apache.org/jira/browse/IOTDB-3561 Project: Apache IoTDB Issue Type: New Feature Reporter: Xinyu Tan * On the one hand, we can delete some unsynchronized wal after snapshot. On the other hand, we can make the old node catchup faster. * BTW, member changes must be transferred through snapshot because the corresponding WAL may have been deleted -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3560) Support basic requirements of template
Yukun Zhou created IOTDB-3560: - Summary: Support basic requirements of template Key: IOTDB-3560 URL: https://issues.apache.org/jira/browse/IOTDB-3560 Project: Apache IoTDB Issue Type: Improvement Reporter: Yukun Zhou Fix For: master branch The basic requirements includes: # Create template and store it on configNode # Deliver template to dataNode and cache it on dataNode # Mount and activate template on involving dataNode -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3560) Support basic requirements of template
[ https://issues.apache.org/jira/browse/IOTDB-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yukun Zhou reassigned IOTDB-3560: - Assignee: Yukun Zhou > Support basic requirements of template > -- > > Key: IOTDB-3560 > URL: https://issues.apache.org/jira/browse/IOTDB-3560 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yukun Zhou >Assignee: Yukun Zhou >Priority: Major > Fix For: master branch > > > The basic requirements includes: > # Create template and store it on configNode > # Deliver template to dataNode and cache it on dataNode > # Mount and activate template on involving dataNode -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3397) [cluster]Support template in new cluster
[ https://issues.apache.org/jira/browse/IOTDB-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yukun Zhou reassigned IOTDB-3397: - Assignee: Yukun Zhou > [cluster]Support template in new cluster > > > Key: IOTDB-3397 > URL: https://issues.apache.org/jira/browse/IOTDB-3397 > Project: Apache IoTDB > Issue Type: New Feature > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Yukun Zhou >Priority: Major > Attachments: image-2022-06-06-14-16-50-530.png > > > commit 565b14dbb104670b336bf928fdc9ebe3bf8608f0 > Author: liuminghui233 <36565497+liuminghui...@users.noreply.github.com> > Date: Mon Jun 6 09:22:32 2022 +0800 > [IOTDB-3383] Refactor calcInputLocationList() in LocalExecutionPlanner > (#6152) > > > Reproduce steps： > 1.Setup a cluster with 3C3D; > 2.Using iotdb-cli to connect a data node; > 3. Excute sql "create schema template t1 (temperature FLOAT encoding=RLE, > status BOOLEAN encoding=PLAIN compression=SNAPPY);" > got an error like below: > Msg: 400: [EXECUTE_STATEMENT_ERROR(400)] Exception occurred: checkAuthority > failed. null > > !image-2022-06-06-14-16-50-530.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3558) Print the thread stack for all DataNode and ConfigNode in IT Framework
[ https://issues.apache.org/jira/browse/IOTDB-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Tian reassigned IOTDB-3558: Assignee: (was: Eric Pai) > Print the thread stack for all DataNode and ConfigNode in IT Framework > -- > > Key: IOTDB-3558 > URL: https://issues.apache.org/jira/browse/IOTDB-3558 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Priority: Major > > When some queries are time out, there may be deadlock or alive lock in > system, so iit will be better for our debuging that New Cluster IT Framework > can print the thread stack for all DataNode and ConfigNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[GitHub] [iotdb-client-go] dependabot[bot] opened a new pull request, #34: Bump github.com/stretchr/testify from 1.7.2 to 1.7.4
dependabot[bot] opened a new pull request, #34: URL: https://github.com/apache/iotdb-client-go/pull/34 Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.7.2 to 1.7.4. Commits https://github.com/stretchr/testify/commit/48391ba5eb8c5f49132138e67ec112d180b88f63";>48391ba Fix panic in AssertExpectations for mocks without expectations (https://github-redirect.dependabot.com/stretchr/testify/issues/1207";>#1207) https://github.com/stretchr/testify/commit/840cb801497147a6d30a568e0874dfefb10867a9";>840cb80 arrays value types in a zero-initialized state are considered empty (https://github-redirect.dependabot.com/stretchr/testify/issues/1126";>#1126) https://github.com/stretchr/testify/commit/07dc7ee5abe6d1c0ef6449f731ca9bf08e1787f8";>07dc7ee Bump actions/setup-go from 3.1.0 to 3.2.0 (https://github-redirect.dependabot.com/stretchr/testify/issues/1191";>#1191) https://github.com/stretchr/testify/commit/c33fc8d30db6f6d2c5d270c5023eb0f2181bbab5";>c33fc8d Bump actions/checkout from 2 to 3 (https://github-redirect.dependabot.com/stretchr/testify/issues/1163";>#1163) https://github.com/stretchr/testify/commit/3c33e07c4c233bf61e1414f5acb3cda06ffef1d1";>3c33e07 Added Go 1.18.1 as a build/supported version (https://github-redirect.dependabot.com/stretchr/testify/issues/1182";>#1182) https://github.com/stretchr/testify/commit/e2b56b3a384eb40136b2cff13e07e932dd95bf28";>e2b56b3 Bump github.com/stretchr/objx from 0.1.0 to 0.4.0 See full diff in https://github.com/stretchr/testify/compare/v1.7.2...v1.7.4";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (IOTDB-3549) Support delete data for new standalone IoTDB
[ https://issues.apache.org/jira/browse/IOTDB-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-3549: - Assignee: Yukun Zhou (was: yanze chen) > Support delete data for new standalone IoTDB > > > Key: IOTDB-3549 > URL: https://issues.apache.org/jira/browse/IOTDB-3549 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Haonan Hou >Assignee: Yukun Zhou >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3558) Print the thread stack for all DataNode and ConfigNode
Yuan Tian created IOTDB-3558: Summary: Print the thread stack for all DataNode and ConfigNode Key: IOTDB-3558 URL: https://issues.apache.org/jira/browse/IOTDB-3558 Project: Apache IoTDB Issue Type: Improvement Reporter: Yuan Tian Assignee: Eric Pai When some queries are time out, there may be deadlock or alive lock in system, so iit will be better for our debuging that new Cluster Framework can print the thread stack for all DataNode and ConfigNode. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3551) [ thread ] Thread control is required
[ https://issues.apache.org/jira/browse/IOTDB-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘珍 reassigned IOTDB-3551: - Assignee: Jialin Qiao > [ thread ] Thread control is required > - > > Key: IOTDB-3551 > URL: https://issues.apache.org/jira/browse/IOTDB-3551 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: 刘珍 >Assignee: Jialin Qiao >Priority: Major > Attachments: stack_1.out > > > MultiLeaderConsensus，72cpu机器，21个dataregion，1个schemaregion，单个datanode进程的Threads: > 1200 total，需做好线程控制。 > TAsyncClientManager ： 378 > Compaction相关：72 > Flush 相关：76 > MultiLeaderConsensusRPC ： 65 > WAL 相关 ：43 > LogDispatcher ： 42 > grpc-default-worker-ELG ： 72 > 20220620_090240_53823线程名：161 > 详细见附件stack_1.out。 > 数据库配置参数： > data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > schema_replication_factor=3 > data_replication_factor=3 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-544) Apache IoTDB integration with more powerful aggregation index
[ https://issues.apache.org/jira/browse/IOTDB-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-544: - Assignee: (was: Zesong Sun) > Apache IoTDB integration with more powerful aggregation index > - > > Key: IOTDB-544 > URL: https://issues.apache.org/jira/browse/IOTDB-544 > Project: Apache IoTDB > Issue Type: Wish > Components: Core/Engine >Reporter: Xiangdong Huang >Priority: Major > Labels: IoTDB, gsoc2020, mentor, pull-request-available > > IoTDB is a highly efficient time series database, which supports high speed > query process, including aggregation query. > Currently, IoTDB pre-calculates the aggregation info, or called the summary > info, (sum, count, max_time, min_time, max_value, min_value) for each page > and each Chunk. The info is helpful for aggregation operations and some query > filters. For example, if the query filter is value >10 and the max value of a > page is 9, we can skip the page. For another example, if the query is select > max(value) and the max value of 3 chunks are 5, 10, 20, then the max(value) > is 20. > However, there are two drawbacks: > 1. The summary info actually reduces the data that needs to be scanned as 1/k > (suppose each page has k data points). However, the time complexity is still > O(N). If we store a long historical data, e.g., storing 2 years data with > 500KHz, then the aggregation operation may be still time-consuming. So, a > tree-based index to reduce the time complexity from O(N) to O(logN) is a good > choice. Some basic ideas have been published in [1], while it can just handle > data with fix frequency. So, improving it and implementing it into IoTDB is a > good choice. > 2. The summary info is helpless for evaluating the query like where value >8 > if the max value = 10. If we can enrich the summary info, e.g., storing the > data histogram, we can use the histogram to evaluate how many points we can > return. > This proposal is mainly for adding an index for speeding up the aggregation > query. Besides, if we can let the summary info be more useful, it could be > better. > Notice that the premise is that the insertion speed should not be slow down > too much! > You should know: > • IoTDB query process > • TsFile structure and organization > • Basic index knowledge > • Java > difficulty: Major > mentors: > h...@apache.org > Reference: > [1] [https://www.sciencedirect.com/science/article/pii/S0306437918305489] > > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Reopened] (IOTDB-544) Apache IoTDB integration with more powerful aggregation index
[ https://issues.apache.org/jira/browse/IOTDB-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reopened IOTDB-544: --- > Apache IoTDB integration with more powerful aggregation index > - > > Key: IOTDB-544 > URL: https://issues.apache.org/jira/browse/IOTDB-544 > Project: Apache IoTDB > Issue Type: Wish > Components: Core/Engine >Reporter: Xiangdong Huang >Assignee: Zesong Sun >Priority: Major > Labels: IoTDB, gsoc2020, mentor, pull-request-available > > IoTDB is a highly efficient time series database, which supports high speed > query process, including aggregation query. > Currently, IoTDB pre-calculates the aggregation info, or called the summary > info, (sum, count, max_time, min_time, max_value, min_value) for each page > and each Chunk. The info is helpful for aggregation operations and some query > filters. For example, if the query filter is value >10 and the max value of a > page is 9, we can skip the page. For another example, if the query is select > max(value) and the max value of 3 chunks are 5, 10, 20, then the max(value) > is 20. > However, there are two drawbacks: > 1. The summary info actually reduces the data that needs to be scanned as 1/k > (suppose each page has k data points). However, the time complexity is still > O(N). If we store a long historical data, e.g., storing 2 years data with > 500KHz, then the aggregation operation may be still time-consuming. So, a > tree-based index to reduce the time complexity from O(N) to O(logN) is a good > choice. Some basic ideas have been published in [1], while it can just handle > data with fix frequency. So, improving it and implementing it into IoTDB is a > good choice. > 2. The summary info is helpless for evaluating the query like where value >8 > if the max value = 10. If we can enrich the summary info, e.g., storing the > data histogram, we can use the histogram to evaluate how many points we can > return. > This proposal is mainly for adding an index for speeding up the aggregation > query. Besides, if we can let the summary info be more useful, it could be > better. > Notice that the premise is that the insertion speed should not be slow down > too much! > You should know: > • IoTDB query process > • TsFile structure and organization > • Basic index knowledge > • Java > difficulty: Major > mentors: > h...@apache.org > Reference: > [1] [https://www.sciencedirect.com/science/article/pii/S0306437918305489] > > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Reopened] (IOTDB-93) IoTDB Calcite integration
[ https://issues.apache.org/jira/browse/IOTDB-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reopened IOTDB-93: -- > IoTDB Calcite integration > - > > Key: IOTDB-93 > URL: https://issues.apache.org/jira/browse/IOTDB-93 > Project: Apache IoTDB > Issue Type: Task >Reporter: EJTTianyu >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3555) Show RegionNum when show datanodes
[ https://issues.apache.org/jira/browse/IOTDB-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3555: -- Assignee: 任宇华 (was: Yongzao Dan) > Show RegionNum when show datanodes > -- > > Key: IOTDB-3555 > URL: https://issues.apache.org/jira/browse/IOTDB-3555 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yongzao Dan >Assignee: 任宇华 >Priority: Major > Fix For: 0.14.0 > > > See [this > doc|https://apache-iotdb.feishu.cn/docs/doccnfObLV57B9XQ0Tebg8N6Dqf#mnHr4v] > for more details. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3557) Maintain onlineDataNodes by LoadManager
[ https://issues.apache.org/jira/browse/IOTDB-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3557: -- Assignee: (was: Yongzao Dan) > Maintain onlineDataNodes by LoadManager > --- > > Key: IOTDB-3557 > URL: https://issues.apache.org/jira/browse/IOTDB-3557 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Yongzao Dan >Priority: Major > Fix For: 0.14.0 > > > Currently, onlineDataNodes is maintained by NodeInfo, but NodeInfo should > only be used to maintain TDataNodeLocation(static) that has been added to the > cluster. The real-time DataNodeStatus(dynamic) should be maintained by > LoadManager through cluster heartbeat. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3556) Maintain onlineConfigNodes by LoadManager
[ https://issues.apache.org/jira/browse/IOTDB-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3556: -- Assignee: (was: Yongzao Dan) > Maintain onlineConfigNodes by LoadManager > - > > Key: IOTDB-3556 > URL: https://issues.apache.org/jira/browse/IOTDB-3556 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Yongzao Dan >Priority: Major > Fix For: 0.14.0 > > > Currently, onlineConfigNodes is maintained by NodeInfo, but NodeInfo should > only be used to maintain TConfigNodeLocation(static) that has been added to > the cluster. The real-time ConfigNodeStatus(dynamic) should be maintained by > LoadManager through cluster heartbeat. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3528) Filter DataNode which may not be ready in ConfigNode
[ https://issues.apache.org/jira/browse/IOTDB-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3528: -- Assignee: (was: Yongzao Dan) > Filter DataNode which may not be ready in ConfigNode > > > Key: IOTDB-3528 > URL: https://issues.apache.org/jira/browse/IOTDB-3528 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Priority: Major > > If data structure in DataNode A has not been initialized，but it has > registered itself into ConfigNode，and meanwhile its Internal RPC Service has > been set up. At this time if someone send request through another DataNode > B，then the request is dispatched to DataNode A，it will cause NPE. > So we should filter dataNodes which may not be ready in ConfigNode -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3391) Judge ConfigNode status through heartbeat
[ https://issues.apache.org/jira/browse/IOTDB-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3391: -- Assignee: (was: Yongzao Dan) > Judge ConfigNode status through heartbeat > - > > Key: IOTDB-3391 > URL: https://issues.apache.org/jira/browse/IOTDB-3391 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Yongzao Dan >Priority: Major > Fix For: 0.14.0 > > > When we use "show cluster", the node status will be shown. However, we do not > have a heartbeat among leader confignode with follower confignodes. > This needs to be added and update the ConfigNode status. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-2608) Meet exception when timestamp in Tsfile name is negative
[ https://issues.apache.org/jira/browse/IOTDB-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leping Huang reassigned IOTDB-2608: --- Assignee: Leping Huang > Meet exception when timestamp in Tsfile name is negative > > > Key: IOTDB-2608 > URL: https://issues.apache.org/jira/browse/IOTDB-2608 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0-SNAPSHOT >Reporter: Jialin Qiao >Assignee: Leping Huang >Priority: Minor > Labels: pull-request-available > Original Estimate: 8h > Remaining Estimate: 8h > > The machine time is 1929-11-26 08:07:09,688, which is less than 0 in > timestamp format. > Then, the generation time of TsFile is less than 0, which obeys the check > inside IoTDB. > This is not expected, we should allow a generation time that less than 0. > > ``` > 1929-11-26 08:07:09,688 [pool-14-IoTDB-RPC-Client-5] ERROR > o.a.i.d.e.s.StorageGroupProcessor:1097 - meet IOException when creating > TsFileProcessor, change system mode to read-only > java.io.IOException: tsfile file name format is > incorrect:-1265413970528-1-0-0.tsfile > at > org.apache.iotdb.db.engine.storagegroup.TsFileResource.getTsFileName(TsFileResource.java:860) > at > org.apache.iotdb.db.engine.storagegroup.TsFileResource.getMergeLevel(TsFileResource.java:874) > at > org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.add(LevelCompactionTsFileManagement.java:268) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.getOrCreateTsFileProcessorIntern(StorageGroupProcessor.java:1121) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.getOrCreateTsFileProcessor(StorageGroupProcessor.java:1086) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.insertTabletToTsFileProcessor(StorageGroupProcessor.java:944) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.insertTablet(StorageGroupProcessor.java:898) > at > org.apache.iotdb.db.engine.StorageEngine.insertTablet(StorageEngine.java:706) > at > org.apache.iotdb.db.qp.executor.PlanExecutor.insertTablet(PlanExecutor.java:1432) > at > org.apache.iotdb.db.qp.executor.PlanExecutor.processNonQuery(PlanExecutor.java:255) > at > org.apache.iotdb.db.service.TSServiceImpl.executeNonQuery(TSServiceImpl.java:1153) > at > org.apache.iotdb.db.service.TSServiceImpl.executeNonQueryPlan(TSServiceImpl.java:1891) > at > org.apache.iotdb.db.service.TSServiceImpl.insertTablet(TSServiceImpl.java:1547) > at > org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablet.getResult(TSIService.java:2823) > at > org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablet.getResult(TSIService.java:2803) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-2608) Meet exception when timestamp in Tsfile name is negative
[ https://issues.apache.org/jira/browse/IOTDB-2608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-2608: -- Assignee: (was: 周沛辰) > Meet exception when timestamp in Tsfile name is negative > > > Key: IOTDB-2608 > URL: https://issues.apache.org/jira/browse/IOTDB-2608 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0-SNAPSHOT >Reporter: Jialin Qiao >Priority: Minor > Labels: pull-request-available > Original Estimate: 8h > Remaining Estimate: 8h > > The machine time is 1929-11-26 08:07:09,688, which is less than 0 in > timestamp format. > Then, the generation time of TsFile is less than 0, which obeys the check > inside IoTDB. > This is not expected, we should allow a generation time that less than 0. > > ``` > 1929-11-26 08:07:09,688 [pool-14-IoTDB-RPC-Client-5] ERROR > o.a.i.d.e.s.StorageGroupProcessor:1097 - meet IOException when creating > TsFileProcessor, change system mode to read-only > java.io.IOException: tsfile file name format is > incorrect:-1265413970528-1-0-0.tsfile > at > org.apache.iotdb.db.engine.storagegroup.TsFileResource.getTsFileName(TsFileResource.java:860) > at > org.apache.iotdb.db.engine.storagegroup.TsFileResource.getMergeLevel(TsFileResource.java:874) > at > org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.add(LevelCompactionTsFileManagement.java:268) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.getOrCreateTsFileProcessorIntern(StorageGroupProcessor.java:1121) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.getOrCreateTsFileProcessor(StorageGroupProcessor.java:1086) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.insertTabletToTsFileProcessor(StorageGroupProcessor.java:944) > at > org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.insertTablet(StorageGroupProcessor.java:898) > at > org.apache.iotdb.db.engine.StorageEngine.insertTablet(StorageEngine.java:706) > at > org.apache.iotdb.db.qp.executor.PlanExecutor.insertTablet(PlanExecutor.java:1432) > at > org.apache.iotdb.db.qp.executor.PlanExecutor.processNonQuery(PlanExecutor.java:255) > at > org.apache.iotdb.db.service.TSServiceImpl.executeNonQuery(TSServiceImpl.java:1153) > at > org.apache.iotdb.db.service.TSServiceImpl.executeNonQueryPlan(TSServiceImpl.java:1891) > at > org.apache.iotdb.db.service.TSServiceImpl.insertTablet(TSServiceImpl.java:1547) > at > org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablet.getResult(TSIService.java:2823) > at > org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablet.getResult(TSIService.java:2803) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ``` -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3548) [cluster]can not create timeseries when start 3C2D
[ https://issues.apache.org/jira/browse/IOTDB-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyu Tan reassigned IOTDB-3548: Assignee: (was: Xinyu Tan) > [cluster]can not create timeseries when start 3C2D > -- > > Key: IOTDB-3548 > URL: https://issues.apache.org/jira/browse/IOTDB-3548 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Priority: Major > Attachments: iotdb-confignode.properties, iotdb-engine.properties, > log_all.log > > > > commit c42cfe5fbee50b24cc1a1078cd5af1ee69930881 > Author: YongzaoDan <33111881+crzbulab...@users.noreply.github.com> > Date: Mon Jun 20 13:54:12 2022 +0800 > [IOTDB-3510] Read/Write Routing policy (Routing to DataNode with the > lowest-loaded) (#6308) > Reproduce steps： > 1. Modify config file as 3C3D: > schema_replication_factor=3 > data_replication_factor=1 > 2. Start 3C > 3.Start 2D > 4. using iotdb-cli to execute below sql: > set storage group to root.sg; > create timeseries root.sg.d.s1 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s2 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s3 with > datatype=INT32,encoding=RLE,compression=snappy; > insert into root.sg.d(time,s1,s2,s3) values(1,1,2,3); > insert into root.sg.d(time,s1,s2,s3) values(2,1,2,3); > 5.Got below error msg: > Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "create timeseries > root.sg.d.s1 with datatype=INT32,encoding=RLE,compression=snappy". > executeStatement failed. null > !image-2022-06-20-17-13-31-196.png! > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3551) [ thread ] Thread control is required
[ https://issues.apache.org/jira/browse/IOTDB-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyu Tan reassigned IOTDB-3551: Assignee: (was: Xinyu Tan) > [ thread ] Thread control is required > - > > Key: IOTDB-3551 > URL: https://issues.apache.org/jira/browse/IOTDB-3551 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: 刘珍 >Priority: Major > Attachments: stack_1.out > > > 72cpu机器，21个dataregion，1个schemaregion，单个datanode进程的Threads: 1200 total，需做好线程控制。 > TAsyncClientManager ： 378 > Compaction相关：72 > Flush 相关：76 > MultiLeaderConsensusRPC ： 65 > WAL 相关 ：43 > LogDispatcher ： 42 > grpc-default-worker-ELG ： 72 > 20220620_090240_53823线程名：161 > 详细见附件stack_1.out。 > 数据库配置参数： > data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > schema_replication_factor=3 > data_replication_factor=3 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3557) Maintain onlineDataNodes by LoadManager
Yongzao Dan created IOTDB-3557: -- Summary: Maintain onlineDataNodes by LoadManager Key: IOTDB-3557 URL: https://issues.apache.org/jira/browse/IOTDB-3557 Project: Apache IoTDB Issue Type: New Feature Reporter: Yongzao Dan Assignee: Yongzao Dan Fix For: 0.14.0 Currently, onlineDataNodes is maintained by NodeInfo, but NodeInfo should only be used to maintain TDataNodeLocation(static) that has been added to the cluster. The real-time DataNodeStatus(dynamic) should be maintained by LoadManager through cluster heartbeat. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3556) Maintain onlineConfigNodes by LoadManager
Yongzao Dan created IOTDB-3556: -- Summary: Maintain onlineConfigNodes by LoadManager Key: IOTDB-3556 URL: https://issues.apache.org/jira/browse/IOTDB-3556 Project: Apache IoTDB Issue Type: New Feature Reporter: Yongzao Dan Assignee: Yongzao Dan Fix For: 0.14.0 Currently, onlineConfigNodes is maintained by NodeInfo, but NodeInfo should only be used to maintain TConfigNodeLocation(static) that has been added to the cluster. The real-time ConfigNodeStatus(dynamic) should be maintained by LoadManager through cluster heartbeat. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3555) Show RegionNum when show datanodes
[ https://issues.apache.org/jira/browse/IOTDB-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzao Dan reassigned IOTDB-3555: -- Assignee: Yongzao Dan > Show RegionNum when show datanodes > -- > > Key: IOTDB-3555 > URL: https://issues.apache.org/jira/browse/IOTDB-3555 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yongzao Dan >Assignee: Yongzao Dan >Priority: Major > > See [this > doc|https://apache-iotdb.feishu.cn/docs/doccnfObLV57B9XQ0Tebg8N6Dqf#mnHr4v] > for more details. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3555) Show RegionNum when show cluster
Yongzao Dan created IOTDB-3555: -- Summary: Show RegionNum when show cluster Key: IOTDB-3555 URL: https://issues.apache.org/jira/browse/IOTDB-3555 Project: Apache IoTDB Issue Type: Improvement Reporter: Yongzao Dan See [this doc|https://apache-iotdb.feishu.cn/docs/doccnfObLV57B9XQ0Tebg8N6Dqf#mnHr4v] for more details. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3554) Controls the number of rpc threads under the MultiLeaderConsensus
Xinyu Tan created IOTDB-3554: Summary: Controls the number of rpc threads under the MultiLeaderConsensus Key: IOTDB-3554 URL: https://issues.apache.org/jira/browse/IOTDB-3554 Project: Apache IoTDB Issue Type: Improvement Reporter: Xinyu Tan Assignee: Xinyu Tan * Make all regions share a clientManager * Reduce the number of pipelines because concurrency in the same region is generally not very large -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3553) Change the error that the new user does not set permission check when querying
任宇华 created IOTDB-3553: -- Summary: Change the error that the new user does not set permission check when querying Key: IOTDB-3553 URL: https://issues.apache.org/jira/browse/IOTDB-3553 Project: Apache IoTDB Issue Type: Task Components: Core/Authority Reporter: 任宇华 Fix For: 0.13.0 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3553) Change the error that the new user does not set permission check when querying
[ https://issues.apache.org/jira/browse/IOTDB-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 任宇华 reassigned IOTDB-3553: -- Assignee: 任宇华 > Change the error that the new user does not set permission check when querying > -- > > Key: IOTDB-3553 > URL: https://issues.apache.org/jira/browse/IOTDB-3553 > Project: Apache IoTDB > Issue Type: Task > Components: Core/Authority >Reporter: 任宇华 >Assignee: 任宇华 >Priority: Major > Fix For: 0.13.0 > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3552) ITs related to last query failed in new Standalone
Minghui Liu created IOTDB-3552: -- Summary: ITs related to last query failed in new Standalone Key: IOTDB-3552 URL: https://issues.apache.org/jira/browse/IOTDB-3552 Project: Apache IoTDB Issue Type: Bug Reporter: Minghui Liu Assignee: Minghui Liu -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3551) [ thread ] Thread control is required
刘珍 created IOTDB-3551: - Summary: [ thread ] Thread control is required Key: IOTDB-3551 URL: https://issues.apache.org/jira/browse/IOTDB-3551 Project: Apache IoTDB Issue Type: Bug Components: mpp-cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: 刘珍 Assignee: Xinyu Tan Attachments: stack_1.out 72cpu机器，21个dataregion，1个schemaregion，单个datanode进程的Threads: 1200 total，需做好线程控制。 TAsyncClientManager ： 378 Compaction相关：72 Flush 相关：76 MultiLeaderConsensusRPC ： 65 WAL 相关 ：43 LogDispatcher ： 42 grpc-default-worker-ELG ： 72 20220620_090240_53823线程名：161 详细见附件 -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3550) [cluster]Got a wrong result data set when using 'show child nodes root'
FengQingxin created IOTDB-3550: -- Summary: [cluster]Got a wrong result data set when using 'show child nodes root' Key: IOTDB-3550 URL: https://issues.apache.org/jira/browse/IOTDB-3550 Project: Apache IoTDB Issue Type: Bug Components: Core/Cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: FengQingxin Assignee: Jinrui Zhang Attachments: image-2022-06-20-17-49-51-365.png Reproduce steps: # Setup a cluster with 3C3D # create timeseries like below picture # using 'show child nodes root.**' , got a wrong result data set like below: !image-2022-06-01-17-38-54-614.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3260) [cluster]Got an error when delete root.** from 3 clients at the same time
[ https://issues.apache.org/jira/browse/IOTDB-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556298#comment-17556298 ] FengQingxin commented on IOTDB-3260: !screenshot-1.png! > [cluster]Got an error when delete root.** from 3 clients at the same time > - > > Key: IOTDB-3260 > URL: https://issues.apache.org/jira/browse/IOTDB-3260 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Yukun Zhou >Priority: Major > Attachments: image-2022-05-23-12-36-25-860.png, screenshot-1.png > > > Reproduce steps: > 1.set up a cluster with 3C3D > 2.set sg like this: > set storage group to root.a; > set storage group to root.b.b1; > set storage group to root.c.c1.c2; > set storage group to root.x; > 3.delete all sg at the same time in 3 cli(connect to 3 datanodes) > one of 3 got an error > Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "delete storage > group root.**". executeStatement failed. error code: TSStatus(code:500, > message:506: null) > > !image-2022-05-23-12-36-25-860.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3254) [cluster]Error messages need to be improved(create a duplicate sg)
[ https://issues.apache.org/jira/browse/IOTDB-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556297#comment-17556297 ] FengQingxin commented on IOTDB-3254: !screenshot-1.png! > [cluster]Error messages need to be improved(create a duplicate sg) > -- > > Key: IOTDB-3254 > URL: https://issues.apache.org/jira/browse/IOTDB-3254 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Yukun Zhou >Priority: Major > Attachments: image-2022-05-23-07-14-42-272.png, screenshot-1.png > > > Reproduce steps: > 1.Set up a cluster with 3C3D; > 2.using iotdb-cli create a sg named as root.sg4; > 3.using iotdb-cli create a sg named as root.sg4 again,we got below eror msg: > Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "set storage group > to root.sg4". executeStatement failed. error code: TSStatus(code:500, > message:902: StorageGroup root.sg4 is already set.) > !image-2022-05-23-07-14-42-272.png! > expect:just a msg :StorageGroup root.sg4 is already set > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3549) Support delete data for new standalone IoTDB
[ https://issues.apache.org/jira/browse/IOTDB-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-3549: - Assignee: yanze chen > Support delete data for new standalone IoTDB > > > Key: IOTDB-3549 > URL: https://issues.apache.org/jira/browse/IOTDB-3549 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Haonan Hou >Assignee: yanze chen >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3548) [cluster]can not create timeseries when start 3C2D
[ https://issues.apache.org/jira/browse/IOTDB-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-3548: - Assignee: Xinyu Tan (was: Haonan Hou) > [cluster]can not create timeseries when start 3C2D > -- > > Key: IOTDB-3548 > URL: https://issues.apache.org/jira/browse/IOTDB-3548 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Xinyu Tan >Priority: Major > Attachments: iotdb-confignode.properties, iotdb-engine.properties, > log_all.log > > > > commit c42cfe5fbee50b24cc1a1078cd5af1ee69930881 > Author: YongzaoDan <33111881+crzbulab...@users.noreply.github.com> > Date: Mon Jun 20 13:54:12 2022 +0800 > [IOTDB-3510] Read/Write Routing policy (Routing to DataNode with the > lowest-loaded) (#6308) > Reproduce steps： > 1. Modify config file as 3C3D: > schema_replication_factor=3 > data_replication_factor=1 > 2. Start 3C > 3.Start 2D > 4. using iotdb-cli to execute below sql: > set storage group to root.sg; > create timeseries root.sg.d.s1 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s2 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s3 with > datatype=INT32,encoding=RLE,compression=snappy; > insert into root.sg.d(time,s1,s2,s3) values(1,1,2,3); > insert into root.sg.d(time,s1,s2,s3) values(2,1,2,3); > 5.Got below error msg: > Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "create timeseries > root.sg.d.s1 with datatype=INT32,encoding=RLE,compression=snappy". > executeStatement failed. null > !image-2022-06-20-17-13-31-196.png! > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3549) Support delete data for new standalone IoTDB
Haonan Hou created IOTDB-3549: - Summary: Support delete data for new standalone IoTDB Key: IOTDB-3549 URL: https://issues.apache.org/jira/browse/IOTDB-3549 Project: Apache IoTDB Issue Type: New Feature Reporter: Haonan Hou -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3548) [cluster]can not create timeseries when start 3C2D
[ https://issues.apache.org/jira/browse/IOTDB-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengQingxin reassigned IOTDB-3548: -- Assignee: Haonan Hou > [cluster]can not create timeseries when start 3C2D > -- > > Key: IOTDB-3548 > URL: https://issues.apache.org/jira/browse/IOTDB-3548 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Haonan Hou >Priority: Major > Attachments: image-2022-06-20-17-13-31-196.png, > image-2022-06-20-17-13-34-029.png, log_all.log > > > > commit c42cfe5fbee50b24cc1a1078cd5af1ee69930881 > Author: YongzaoDan <33111881+crzbulab...@users.noreply.github.com> > Date: Mon Jun 20 13:54:12 2022 +0800 > [IOTDB-3510] Read/Write Routing policy (Routing to DataNode with the > lowest-loaded) (#6308) > Reproduce steps： > 1. Modify config file as 3C3D: > schema_replication_factor=3 > data_replication_factor=1 > 2. Start 3C > 3.Start 2D > 4. using iotdb-cli to execute below sql: > set storage group to root.sg; > create timeseries root.sg.d.s1 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s2 with > datatype=INT32,encoding=RLE,compression=snappy; > create timeseries root.sg.d.s3 with > datatype=INT32,encoding=RLE,compression=snappy; > insert into root.sg.d(time,s1,s2,s3) values(1,1,2,3); > insert into root.sg.d(time,s1,s2,s3) values(2,1,2,3); > 5.Got below error msg: > Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "create timeseries > root.sg.d.s1 with datatype=INT32,encoding=RLE,compression=snappy". > executeStatement failed. null > !image-2022-06-20-17-13-31-196.png! > > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3548) [cluster]can not create timeseries when start 3C2D
FengQingxin created IOTDB-3548: -- Summary: [cluster]can not create timeseries when start 3C2D Key: IOTDB-3548 URL: https://issues.apache.org/jira/browse/IOTDB-3548 Project: Apache IoTDB Issue Type: Bug Components: Core/Cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: FengQingxin Attachments: image-2022-06-20-17-13-31-196.png, image-2022-06-20-17-13-34-029.png, log_all.log commit c42cfe5fbee50b24cc1a1078cd5af1ee69930881 Author: YongzaoDan <33111881+crzbulab...@users.noreply.github.com> Date: Mon Jun 20 13:54:12 2022 +0800 [IOTDB-3510] Read/Write Routing policy (Routing to DataNode with the lowest-loaded) (#6308) Reproduce steps： 1. Modify config file as 3C3D: schema_replication_factor=3 data_replication_factor=1 2. Start 3C 3.Start 2D 4. using iotdb-cli to execute below sql: set storage group to root.sg; create timeseries root.sg.d.s1 with datatype=INT32,encoding=RLE,compression=snappy; create timeseries root.sg.d.s2 with datatype=INT32,encoding=RLE,compression=snappy; create timeseries root.sg.d.s3 with datatype=INT32,encoding=RLE,compression=snappy; insert into root.sg.d(time,s1,s2,s3) values(1,1,2,3); insert into root.sg.d(time,s1,s2,s3) values(2,1,2,3); 5.Got below error msg: Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "create timeseries root.sg.d.s1 with datatype=INT32,encoding=RLE,compression=snappy". executeStatement failed. null !image-2022-06-20-17-13-31-196.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3535) [cluster]A cross-space compaction has occurred when using benchmark to test iotdb with operation "IS_OUT_OF_ORDER=false"
[ https://issues.apache.org/jira/browse/IOTDB-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-3535: - Assignee: Quan Siyi (was: Haonan Hou) > [cluster]A cross-space compaction has occurred when using benchmark to test > iotdb with operation "IS_OUT_OF_ORDER=false" > > > Key: IOTDB-3535 > URL: https://issues.apache.org/jira/browse/IOTDB-3535 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: config.properties, image-2022-06-20-09-52-52-981.png, > iotdb-confignode.properties, iotdb-engine.properties, log-all-20220619.19.log > > > A cross-space compaction has occurred when using benchmark to test iotdb with > operation "IS_OUT_OF_ORDER=false" > commit 1309eef2b1fb27a8ce7c8911b9c71ee5a9ed7035 > Author: 刘威 <51618159+liu-wei-...@users.noreply.github.com> > Date: Sat Jun 18 08:50:37 2022 +0800 > [IOTDB-3388] Let MQTT Service could parse JSON array of existed format > (#6154) > > Reproduce steps： > # Setup a cluster with 3C3D > # Using 3BMs to insert data > BM - 》 IoTDB Node > 172.20.70.7 - 》 172.20.70.22 > 172.20.70.8 - 》 172.20.70.23 > 172.20.70.9 - 》 172.20.70.24 > > !image-2022-06-20-09-52-52-981.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3547) AddConfigNode support procedure
Chao Wang created IOTDB-3547: Summary: AddConfigNode support procedure Key: IOTDB-3547 URL: https://issues.apache.org/jira/browse/IOTDB-3547 Project: Apache IoTDB Issue Type: Task Reporter: Chao Wang Assignee: Chao Wang 1. The system can be started concurrently. 2. Supports leader switchover. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3398) Device not in TsFile
[ https://issues.apache.org/jira/browse/IOTDB-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3398: -- Assignee: yanze chen (was: Yuan Tian) > Device not in TsFile > > > Key: IOTDB-3398 > URL: https://issues.apache.org/jira/browse/IOTDB-3398 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.5 >Reporter: Jialin Qiao >Assignee: yanze chen >Priority: Major > Attachments: iotdblost.zip > > > > When querying a TsFile, we suspect that the device should be in this file. > Howerver, it doesn't. > > 2022-06-05 14:02:19,048 [pool-37-IoTDB-Sub_RawQuery-5] WARN > o.a.i.t.r.TsFileSequenceReader:336 - Device root.sg.track19.d_53810829218 is > not in /mntb/iotdb/data/sequence/root.sg/1/0/1654406867872-6737-0-0.tsfile -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3546) [ WAL ] Under multi-leader, the wal file of the storage group that accepts synchronized data cannot be deleted
刘珍 created IOTDB-3546: - Summary: [ WAL ] Under multi-leader, the wal file of the storage group that accepts synchronized data cannot be deleted Key: IOTDB-3546 URL: https://issues.apache.org/jira/browse/IOTDB-3546 Project: Apache IoTDB Issue Type: Bug Components: mpp-cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: 刘珍 Assignee: Haiming Zhu Attachments: config.properties.wal.1, config.properties.wal.2, image-2022-06-20-14-56-09-259.png 问题描述： multi-leader下，接受同步数据的存储组的wal文件无法删除。 wal文件夹过大： !image-2022-06-20-14-56-09-259.png! 测试流程 1. 测试环境 192.168.10.62/66/68 72C256G data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus schema_replication_factor=3 data_replication_factor=3 2. 68服务器bm 连62写入数据 启动配置文件1，30分钟后，启动配置文件2。 （见附件） 一直运行着，查看wal文件信息。 -- This message was sent by Atlassian Jira (v8.20.7#820007)
