[jira] [Created] (IOTDB-4649) Expression should never be thought as same just through their expressionString
Yuan Tian created IOTDB-4649: Summary: Expression should never be thought as same just through their expressionString Key: IOTDB-4649 URL: https://issues.apache.org/jira/browse/IOTDB-4649 Project: Apache IoTDB Issue Type: Bug Reporter: Yuan Tian Assignee: liaolanyu Attachments: image-2022-10-14-11-39-55-172.png, image-2022-10-14-11-40-04-631.png You can replay by the following: insert into root.sg1.d1(time, s1, s2) aligned values (1,1, "1"), (2,2,"2"); select s2 from root.** where s1 = 1 and s2 >= "1" and s2 <= "2"; !image-2022-10-14-11-39-55-172.png! 、 !image-2022-10-14-11-40-04-631.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4648) Start leader schedule threads when restart
Yongzao Dan created IOTDB-4648: -- Summary: Start leader schedule threads when restart Key: IOTDB-4648 URL: https://issues.apache.org/jira/browse/IOTDB-4648 Project: Apache IoTDB Issue Type: Bug Reporter: Yongzao Dan Assignee: Yongzao Dan Fix For: 0.14.0 We've found that in the scenario with only one ConfigNode, the leader schedule threads don't start properly after restart. Because the leadership remains the same after restart. Therefore, the notifyLeaderChanged interface in PartitionStateMachine is not called after a restart, so these leader schedule threads cannot be started. One possible solution is for ConfigNode to restart and check its own leadership and start the leader schedule threads automatically if necessary. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4647) Mapping for temperature not found, expected one of [Time, Device, hardware(TEXT), temperature(FLOAT)]
任宇华 created IOTDB-4647: -- Summary: Mapping for temperature not found, expected one of [Time, Device, hardware(TEXT), temperature(FLOAT)] Key: IOTDB-4647 URL: https://issues.apache.org/jira/browse/IOTDB-4647 Project: Apache IoTDB Issue Type: Bug Reporter: 任宇华 Attachments: image-2022-10-14-11-18-14-886.png, image-2022-10-14-11-19-20-757.png 当导入的数据携带数据类型时，会出错： !image-2022-10-14-11-18-14-886.png! 数据文件： !image-2022-10-14-11-19-20-757.png! 、 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4647) Mapping for temperature not found, expected one of [Time, Device, hardware(TEXT), temperature(FLOAT)]
[ https://issues.apache.org/jira/browse/IOTDB-4647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 任宇华 reassigned IOTDB-4647: -- Assignee: 任宇华 > Mapping for temperature not found, expected one of [Time, Device, > hardware(TEXT), temperature(FLOAT)] > - > > Key: IOTDB-4647 > URL: https://issues.apache.org/jira/browse/IOTDB-4647 > Project: Apache IoTDB > Issue Type: Bug >Reporter: 任宇华 >Assignee: 任宇华 >Priority: Major > Attachments: image-2022-10-14-11-18-14-886.png, > image-2022-10-14-11-19-20-757.png > > > 当导入的数据携带数据类型时，会出错： > > !image-2022-10-14-11-18-14-886.png! > 数据文件： > !image-2022-10-14-11-19-20-757.png! 、 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4646) Illegal privilege APPLY_TEMPLATE on seriesPath root.ln.**
[ https://issues.apache.org/jira/browse/IOTDB-4646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 任宇华 reassigned IOTDB-4646: -- Assignee: 任宇华 > Illegal privilege APPLY_TEMPLATE on seriesPath root.ln.** > - > > Key: IOTDB-4646 > URL: https://issues.apache.org/jira/browse/IOTDB-4646 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Authority >Reporter: 任宇华 >Assignee: 任宇华 >Priority: Major > Attachments: image-2022-10-14-10-10-36-708.png > > > !image-2022-10-14-10-10-36-708.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4646) Illegal privilege APPLY_TEMPLATE on seriesPath root.ln.**
任宇华 created IOTDB-4646: -- Summary: Illegal privilege APPLY_TEMPLATE on seriesPath root.ln.** Key: IOTDB-4646 URL: https://issues.apache.org/jira/browse/IOTDB-4646 Project: Apache IoTDB Issue Type: Bug Components: Core/Authority Reporter: 任宇华 Attachments: image-2022-10-14-10-10-36-708.png !image-2022-10-14-10-10-36-708.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4473) Support TSBS Timeseries Benchmark
[ https://issues.apache.org/jira/browse/IOTDB-4473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jinrui Zhang reassigned IOTDB-4473: --- Assignee: Xingyu Liu > Support TSBS Timeseries Benchmark > - > > Key: IOTDB-4473 > URL: https://issues.apache.org/jira/browse/IOTDB-4473 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Julian Feinauer >Assignee: Xingyu Liu >Priority: Major > > The tsbs Benchmark was initially created by InfluxDB and is now maintained by > timescale db and compares many state of the art timeseries databases: > [https://github.com/timescale/tsbs] > > IMHO we should add support for IoTDB -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4645) Optimization of estimating memory usage of cross space compaction task
Quan Siyi created IOTDB-4645: Summary: Optimization of estimating memory usage of cross space compaction task Key: IOTDB-4645 URL: https://issues.apache.org/jira/browse/IOTDB-4645 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Assignee: Quan Siyi Fix For: 0.13.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4616) Support serialization and deserialization for confignode standalone mode
[ https://issues.apache.org/jira/browse/IOTDB-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617371#comment-17617371 ] 刘俊男 commented on IOTDB-4616: 1、目前暂时没有standalone共识协议的持计划和重放的代码，重启后会丢失confignode相关数据。 2、write 方法最后会走到 PartitionStateMachine，再走到 ConfigExecutor，然后根据请求分流。 > Support serialization and deserialization for confignode standalone mode > - > > Key: IOTDB-4616 > URL: https://issues.apache.org/jira/browse/IOTDB-4616 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Gaofei Cao >Assignee: 刘俊男 >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4644) Update DataNode's IP:Port function
伊丹翔 created IOTDB-4644: -- Summary: Update DataNode's IP:Port function Key: IOTDB-4644 URL: https://issues.apache.org/jira/browse/IOTDB-4644 Project: Apache IoTDB Issue Type: Task Reporter: 伊丹翔 Assignee: 伊丹翔 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3936) Add an interface in IClientRPCService to directly use TsBlock instead of TSQueryDataSet
[ https://issues.apache.org/jira/browse/IOTDB-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617081#comment-17617081 ] yang caiyin commented on IOTDB-3936: To better solve this problem, besides executeStatementV2. We should also add fetchResultV2、executeRawDataQuery and executeLastDataQuery to replace the former method for a faster query without queryDataset. > Add an interface in IClientRPCService to directly use TsBlock instead of > TSQueryDataSet > --- > > Key: IOTDB-3936 > URL: https://issues.apache.org/jira/browse/IOTDB-3936 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Assignee: yang caiyin >Priority: Major > > Currently, we convert TsBlock to TSQueryDataSet to be compatible with > previous executeStatement() interface in IClientRPCService and in this way we > won't need to change all other language clients like go, C/C++ and python etc. > However, we can add a new interface like executeStatementV2 to directly > return serialized form of TsBlock to client and then do the deserialization > things in the client end. In this way, we can improve the query performance > if the result TsBlock is received from remote node. Because in such case, > what we received is already serialized TsBlock(in ByteBuffer form), we can > directly return that to client which will saving one deserializing TsBlock > and one serialzie to TSQueryDataSet. > We alse need to change the java client: jdbc and session to adapt to read > from TsBlock. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4643) Add unknown_dataNode_detector
李伟豪 created IOTDB-4643: -- Summary: Add unknown_dataNode_detector Key: IOTDB-4643 URL: https://issues.apache.org/jira/browse/IOTDB-4643 Project: Apache IoTDB Issue Type: Sub-task Reporter: 李伟豪 Assignee: 李伟豪 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4642) Failed to transfer the snapshot file.
Chao Wang created IOTDB-4642: Summary: Failed to transfer the snapshot file. Key: IOTDB-4642 URL: https://issues.apache.org/jira/browse/IOTDB-4642 Project: Apache IoTDB Issue Type: Bug Components: Core/Cluster Affects Versions: 0.12.6 Reporter: Chao Wang in 0.12 version, The number of snapshot files exceeds the thrift max size. As a result, data cannot be synchronized. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4637) [0.13.3]In the case of a large amount of data, perform an aggregate query to report NPE
[
https://issues.apache.org/jira/browse/IOTDB-4637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616881#comment-17616881
]
Yuan Tian commented on IOTDB-4637:
--
It can be seen that the root cause is that IoTDB has opened too many files. In
current query processing, we will keep all the related files opened until this
query finished and this will be optimized in the future which can eliminate
this error.
Before we do that optimization, you can try 2 methods:
# modify the system max allowed opened file handles
# set compaction to true and compaction can reduce the file numbers.
!image-2022-10-13-16-42-11-268.png!
> [0.13.3]In the case of a large amount of data, perform an aggregate query to
> report NPE
> ---
>
> Key: IOTDB-4637
> URL: https://issues.apache.org/jira/browse/IOTDB-4637
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Query
>Affects Versions: 0.13.3
>Reporter: xiaozhihong
>Assignee: Yuan Tian
>Priority: Major
> Attachments: image-2022-10-13-16-42-03-326.png,
> image-2022-10-13-16-42-11-268.png, log_all.log
>
>
> In Version V0.13.3, I prepared 208G data files: 1 storage group, 1 device,
> 500 timeseries.
> After started server, then enter CLI, executed aggregate sql.
> For a while, repored NPE message.
> {code:java}
> IoTDB> select count(s_0) from root.test.g_0.d_0 ;
> Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "select count(s_0)
> from root.test.g_0.d_0". executeStatement failed.
> java.lang.NullPointerException {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-4022) Adjust the storage_group_report_threshold according to memory for writing
[ https://issues.apache.org/jira/browse/IOTDB-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616878#comment-17616878 ] yanze chen commented on IOTDB-4022: --- 更新测试： 吞吐(point/s)，写入内存分配=8980481489 || ||aligned_seq_config.properties||seq_config.properties||unseq_config.properties||aligned_unseq_config.properties|| |storage_group_report_threshold=16777216|8912483.19|17433312.42/ 16564385.56|3345125.65 |半个小时只跑了10%| |write_memory_vriation_report_proportion=0.001|13742844.78|16521936.57 /13984814.52|5082498.77 | | 吞吐(point/s)，总内存分配=100M || ||seq_config.properties|| |storage_group_report_threshold=16777216|写入一部分后OOM| |write_memory_vriation_report_proportion=0.001|写入失败，reject| > Adjust the storage_group_report_threshold according to memory for writing > - > > Key: IOTDB-4022 > URL: https://issues.apache.org/jira/browse/IOTDB-4022 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Assignee: yanze chen >Priority: Major > Labels: pull-request-available > Attachments: 完整实验结果.md > > > Currently, we have a parameter called storage_group_report_threshold, which > means when a storage group report its memory usage, it should accumulate a > memory change of this number. > If this parameter is too small, may report frequently, which may decrease the > insertion performance. > If this parameter is too large, may report not in time, which may cause OOM. > > We need to change this parameter according to the memory allocated for > writing. > > (1) remove storage_group_report_threshold > (2) Add a new parameter write_memory_vriation_report_proportion > > To rel/0.13 and master -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4581) Compression Ratio compute error
[ https://issues.apache.org/jira/browse/IOTDB-4581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616877#comment-17616877 ] Haiming Zhu commented on IOTDB-4581: This is not a bug, but maybe we should find a better way to compute compression ratio. This phenomenon is caused by tsfile contains few series whose data can be compressed with high compression ratio. Take avg_series_point_num_threshold=5000 as an example: *Scenario 1.* Small TsFile with high compression ratio, It's *Scenario 2.* Small TsFile with low compression ratio > Compression Ratio compute error > --- > > Key: IOTDB-4581 > URL: https://issues.apache.org/jira/browse/IOTDB-4581 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Jialin Qiao >Assignee: Haiming Zhu >Priority: Minor > Attachments: high ratio data.png, high ratio log.png, low ratio > data.png, low ratio log.png > > > We try to read a tsfile and write it to IoTDB with different config but in > same version. If we set the avg_series_point_num_threshold to 5000, 1 and > 10, IoTDB computes compression rate as 100, 10 and 3. But the file size > in the disk is the same. The compression ratio must be computed wrong. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4641) New Standalone Sync Receiver
[ https://issues.apache.org/jira/browse/IOTDB-4641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yusicheng reassigned IOTDB-4641: Assignee: yusicheng > New Standalone Sync Receiver > > > Key: IOTDB-4641 > URL: https://issues.apache.org/jira/browse/IOTDB-4641 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: yusicheng >Assignee: yusicheng >Priority: Major > > Implement Sync Receiver of New Standalone > design doc: https://apache-iotdb.feishu.cn/docx/doxcnyQuQqZhfRCwdnoV7rNF6rT -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4641) New Standalone Sync Receiver
yusicheng created IOTDB-4641: Summary: New Standalone Sync Receiver Key: IOTDB-4641 URL: https://issues.apache.org/jira/browse/IOTDB-4641 Project: Apache IoTDB Issue Type: New Feature Reporter: yusicheng Implement Sync Receiver of New Standalone design doc: https://apache-iotdb.feishu.cn/docx/doxcnyQuQqZhfRCwdnoV7rNF6rT -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4640) [0.13.3]After canceling a data archiving task and resubmitting it again, its task status is not updated
xiaozhihong created IOTDB-4640:
--
Summary: [0.13.3]After canceling a data archiving task and
resubmitting it again, its task status is not updated
Key: IOTDB-4640
URL: https://issues.apache.org/jira/browse/IOTDB-4640
Project: Apache IoTDB
Issue Type: Bug
Components: Core/TsFile
Affects Versions: 0.13.3
Reporter: xiaozhihong
Assignee: Haiming Zhu
In Version 0.13.3, after canceling a data archiving task and resubmitting it
again, then task status is not updated, just shows CANCELED.
Like this:
{code:java}
IoTDB> SET ARCHIVING TO storage_group=root.test.g_0
start_time=3603-02-05T16:53:25 ttl=360 target_dir="/home/xzh/files";
Msg: The statement is executed successfully.
IoTDB> SHOW ARCHIVING
+---+-+-+--+---+---++
|task id| submit time|storage group|status|
start time|expire time(ms)|target directory|
+---+-+-+--+---+---++
| 0|2022-10-13T16:16:56.790+08:00|root.test.g_0|
READY|3603-2-5T16:53:25.000+08:00| 360| /home/xzh/files|
+---+-+-+--+---+---++
Total line number = 1
It costs 0.026s
IoTDB> SHOW ALL ARCHIVING
+---+-+-+--+---+---++
|task id| submit time|storage group|status|
start time|expire time(ms)|target directory|
+---+-+-+--+---+---++
| 0|2022-10-13T16:16:56.790+08:00|root.test.g_0|
READY|3603-2-5T16:53:25.000+08:00| 360| /home/xzh/files|
+---+-+-+--+---+---++
Total line number = 1
It costs 1.775s
IoTDB> SHOW ARCHIVING ON root.test.g_0;
+---+-+-+--+---+---++
|task id| submit time|storage group|status|
start time|expire time(ms)|target directory|
+---+-+-+--+---+---++
| 0|2022-10-13T16:16:56.790+08:00|root.test.g_0|
READY|3603-2-5T16:53:25.000+08:00| 360| /home/xzh/files|
+---+-+-+--+---+---++
Total line number = 1
It costs 0.031s
IoTDB> CANCEL ARCHIVING ON root.test.g_0;
Msg: The statement is executed successfully.
IoTDB> SHOW ARCHIVING ON root.test.g_0;
+---+---+-+--+--+---++
|task id|submit time|storage group|status|start time|expire time(ms)|target
directory|
+---+---+-+--+--+---++
+---+---+-+--+--+---++
Empty set.
It costs 0.033s
IoTDB> SET ARCHIVING TO storage_group=root.test.g_0
start_time=3603-02-05T16:53:25 ttl=360 target_dir="/home/xzh/files";
Msg: The statement is executed successfully.
IoTDB> SHOW ARCHIVING ON root.test.g_0;
+---+---+-+--+--+---++
|task id|submit time|storage group|status|start time|expire time(ms)|target
directory|
+---+---+-+--+--+---++
+---+---+-+--+--+---++
Empty set.
It costs 1.282s
IoTDB> SHOW ALL ARCHIVING
+---+-+-++---+---++
|task id| submit time|storage group| status|
start time|expire time(ms)|target directory|
+---+-+-++---+---++
|
0|2022-10-13T16:16:56.790+08:00|root.test.g_0|CANCELED|3603-2-5T16:53:25.000+08:00|
360| /home/xzh/files|
+---+-+-++---+---++
Total line number = 1
It costs 1.742s
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4639) Tag aggregation: limit and offset supported
Eric Pai created IOTDB-4639: --- Summary: Tag aggregation: limit and offset supported Key: IOTDB-4639 URL: https://issues.apache.org/jira/browse/IOTDB-4639 Project: Apache IoTDB Issue Type: Sub-task Reporter: Eric Pai Assignee: Eric Pai -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4638) Tag Aggregation Relatives
Eric Pai created IOTDB-4638: --- Summary: Tag Aggregation Relatives Key: IOTDB-4638 URL: https://issues.apache.org/jira/browse/IOTDB-4638 Project: Apache IoTDB Issue Type: New Feature Components: Core/Query Reporter: Eric Pai Assignee: Eric Pai -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4637) [0.13.3]In the case of a large amount of data, perform an aggregate query to report NPE
xiaozhihong created IOTDB-4637:
--
Summary: [0.13.3]In the case of a large amount of data, perform an
aggregate query to report NPE
Key: IOTDB-4637
URL: https://issues.apache.org/jira/browse/IOTDB-4637
Project: Apache IoTDB
Issue Type: Bug
Components: Core/Query
Affects Versions: 0.13.3
Reporter: xiaozhihong
Assignee: Yuan Tian
Attachments: log_all.log
In Version V0.13.3, I prepared 208G data files: 1 storage group, 1 device, 500
timeseries.
After started server, then enter CLI, executed aggregate sql.
For a while, repored NPE message.
{code:java}
IoTDB> select count(s_0) from root.test.g_0.d_0 ;
Msg: 500: [INTERNAL_SERVER_ERROR(500)] Exception occurred: "select count(s_0)
from root.test.g_0.d_0". executeStatement failed.
java.lang.NullPointerException {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4636) IndexOutOfBoundsException when compacting aligned series
Liuxuxin created IOTDB-4636: --- Summary: IndexOutOfBoundsException when compacting aligned series Key: IOTDB-4636 URL: https://issues.apache.org/jira/browse/IOTDB-4636 Project: Apache IoTDB Issue Type: Bug Affects Versions: master branch, 0.13.3 Reporter: Liuxuxin Assignee: Liuxuxin Fix For: master branch, 0.13.3 Attachments: image-2022-10-13-15-45-07-212.png !image-2022-10-13-15-45-07-212.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4635) Data cannot be queried when flushing a memtable
Liuxuxin created IOTDB-4635: --- Summary: Data cannot be queried when flushing a memtable Key: IOTDB-4635 URL: https://issues.apache.org/jira/browse/IOTDB-4635 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.13.3 Reporter: Liuxuxin Assignee: Liuxuxin Fix For: 0.13.3 The data in a memtable cannot be queried when it is flushing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
