[jira] [Created] (IOTDB-2702) 1 bug in IoTDB cutomer-doc

2022-03-07 Thread Bo Zhang (Jira) 
Bo Zhang created IOTDB-2702:
---

 Summary: 1 bug in IoTDB cutomer-doc 
 Key: IOTDB-2702
 URL: https://issues.apache.org/jira/browse/IOTDB-2702
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Document
Reporter: Bo Zhang


In IoTDB cutomer-doc :  
[https://iotdb.apache.org/zh/UserGuide/Master/Process-Data/Continuous-Query.html#sql-%E8%AF%AD%E5%8F%A5]

 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (IOTDB-2357) Support create timeseries interface with tags in Python client

2022-03-07 Thread Haonan Hou (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haonan Hou reassigned IOTDB-2357:
-

Assignee: Haonan Hou

> Support create timeseries interface with tags in Python client
> --
>
> Key: IOTDB-2357
> URL: https://issues.apache.org/jira/browse/IOTDB-2357
> Project: Apache IoTDB
>  Issue Type: New Feature
>  Components: Client/Python
>Reporter: Haonan Hou
>Assignee: Haonan Hou
>Priority: Minor
>  Labels: easy-fix, pull-request-available
>
> Java session api has the following interface.
> {code:java}
>   public void createMultiTimeseries(
>   List paths,
>   List dataTypes,
>   List encodings,
>   List compressors,
>   List> propsList,
>   List> tagsList,
>   List> attributesList,
>   List measurementAliasList)
> {code}
> We need to support this interface in Python client as well.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2701) User guide of new Sync-Tool

2022-03-07 Thread yanze chen (Jira) 
yanze chen created IOTDB-2701:
-

 Summary: User guide of new Sync-Tool
 Key: IOTDB-2701
 URL: https://issues.apache.org/jira/browse/IOTDB-2701
 Project: Apache IoTDB
  Issue Type: New Feature
  Components: Tools/Sync
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2700) Sync receiver adaption to transfer

2022-03-07 Thread yanze chen (Jira) 
yanze chen created IOTDB-2700:
-

 Summary: Sync receiver adaption to transfer
 Key: IOTDB-2700
 URL: https://issues.apache.org/jira/browse/IOTDB-2700
 Project: Apache IoTDB
  Issue Type: New Feature
  Components: Tools/Sync
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2699) [Benchmark] Add split record features into benchmark

2022-03-07 Thread Jira 
张洪胤 created IOTDB-2699:
--

 Summary: [Benchmark] Add split record features into benchmark
 Key: IOTDB-2699
 URL: https://issues.apache.org/jira/browse/IOTDB-2699
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: 张洪胤
Assignee: 张洪胤


When operation of benchmark is too big, we choose to split record.

For csv, it will split into many files.

For mysql, it will split into many tables.

For iotdb, it will not split.

Related params:
 # RECORD_SPLIT: whether use split
 # RECORD_SPLIT_MAX_LINE: max line of one split record.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (IOTDB-2698) IOException happened in IoTDBContinuousQueryIT.tearDown

2022-03-07 Thread Yuan Tian (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuan Tian reassigned IOTDB-2698:


Assignee: Steve Yurong Su

> IOException happened in IoTDBContinuousQueryIT.tearDown
> ---
>
> Key: IOTDB-2698
> URL: https://issues.apache.org/jira/browse/IOTDB-2698
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Query
>Reporter: Yuan Tian
>Assignee: Steve Yurong Su
>Priority: Major
> Attachments: image-2022-03-08-10-43-10-384.png
>
>
> You can see details in this 
> [link|https://github.com/apache/iotdb/runs/5458001155?check_suite_focus=true].
> !image-2022-03-08-10-43-10-384.png!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2698) IOException happened in IoTDBContinuousQueryIT.tearDown

2022-03-07 Thread Yuan Tian (Jira) 
Yuan Tian created IOTDB-2698:


 Summary: IOException happened in IoTDBContinuousQueryIT.tearDown
 Key: IOTDB-2698
 URL: https://issues.apache.org/jira/browse/IOTDB-2698
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Query
Reporter: Yuan Tian


You can see details in this 
[link|https://github.com/apache/iotdb/runs/5458001155?check_suite_focus=true].



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2697) Unexpected statistics reading of deleted timeseries

2022-03-07 Thread Eric Pai (Jira) 
Eric Pai created IOTDB-2697:
---

 Summary: Unexpected statistics reading of deleted timeseries
 Key: IOTDB-2697
 URL: https://issues.apache.org/jira/browse/IOTDB-2697
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Query
Reporter: Eric Pai
Assignee: Eric Pai
 Fix For: master branch
 Attachments: image-2022-03-08-10-17-52-776.png, 
image-2022-03-08-10-21-13-199.png

!image-2022-03-08-10-17-52-776.png!

Replay SQLs:

 
{code:java}
SET STORAGE GROUP TO root.sg;
CREATE TIMESERIES root.sg.d1.s1 with datatype=FLOAT,encoding=PLAIN;
CREATE TIMESERIES root.sg.d1.s2 with datatype=INT64,encoding=PLAIN;
INSERT INTO root.sg.d1(time, s1, s2) VALUES(11000, 10, 20);
FLUSH;delete timeseries root.sg.d1.s1;
CREATE TIMESERIES root.sg.d1.s1 with datatype=INT32,encoding=PLAIN;
select * from root.sg.d1 where s1 < 8;{code}
Background exception logs

!image-2022-03-08-10-21-13-199.png!

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2696) Handle the issue that the series count is too large in a query

2022-03-07 Thread Jinrui Zhang (Jira) 
Jinrui Zhang created IOTDB-2696:
---

 Summary: Handle the issue that the series count is too large in a 
query
 Key: IOTDB-2696
 URL: https://issues.apache.org/jira/browse/IOTDB-2696
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Jinrui Zhang






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2695) Discuss whether VSG should be shared by different SG under the new cluster architecture

2022-03-07 Thread Xinyu Tan (Jira) 
Xinyu Tan created IOTDB-2695:


 Summary: Discuss whether VSG should be shared by different SG 
under the new cluster architecture
 Key: IOTDB-2695
 URL: https://issues.apache.org/jira/browse/IOTDB-2695
 Project: Apache IoTDB
  Issue Type: Task
Reporter: Xinyu Tan
Assignee: Xinyu Tan


see 
[docs|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199537831]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (IOTDB-2694) Without Null Any Bug in raw query with value filter

2022-03-07 Thread Yuan Tian (Jira) 
Yuan Tian created IOTDB-2694:


 Summary: Without Null Any Bug in raw query with value filter
 Key: IOTDB-2694
 URL: https://issues.apache.org/jira/browse/IOTDB-2694
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Query
Reporter: Yuan Tian
Assignee: JiaXin Zhang


We won't get satisfied rows in raw query with value filter while using without 
null any.

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (IOTDB-2693) [compaction] org.apache.commons.io.FileExistsException: File element in parameter 'null' already exists

2022-03-07 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

周沛辰 reassigned IOTDB-2693:
--

Assignee: 周沛辰  (was: Jialin Qiao)

> [compaction] org.apache.commons.io.FileExistsException: File element in 
> parameter 'null' already exists
> ---
>
> Key: IOTDB-2693
> URL: https://issues.apache.org/jira/browse/IOTDB-2693
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Compaction
>Affects Versions: 0.13.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: 周沛辰
>Priority: Major
> Attachments: image-2022-03-07-15-19-26-903.png, 
> image-2022-03-07-15-29-25-036.png, q.sh, r.sh, screenshot-1.png, 
> screenshot-2.png, select_del-2rows.sh
>
>
> 被测iotdb信息，是个分支版本：
> https://github.com/choubenson/iotdb/tree/formatCompactionLog
> 测试内容：0.12合并中断 ( 有delete操作)，然后0.13恢复的正确性。
> 问题描述：
> {color:#DE350B}*问题1：合并过程中有异常org.apache.commons.io.FileExistsException: File 
> element in parameter 'null' already exists: 
> 问题2：0.12的备份日志compaction.log 在0.13没有被删除
> 问题3：0.12 deleted的2条记录，在0.13能查询到。
> 实际查询结果，所有的设备的count值为2。
> 期望结果是，每个设备的记录数为：199,999,998。
> 问题4：0.13的data中还有0.12的merge.mods文件。
> 问题5：java.nio.BufferUnderflowException: null。*{color}
> 复现流程：
> 1. 0.12.5-pre4 
> 关合并，生成数据，记作备份1（在/data/liuzhen_test/1205_pre4/data_bk_close-comp）。
> 在192.168.10.68 /data/liuzhen_test/1205_pre4/data_bk_close-comp路径。
> 共10sg/100设备 /100 序列，每个序列2亿点。
> unseq tsfile个数：50064
> seq tsfile个数：50076
> 2. 0.12 0305版本3154faf。copy 备份1 到此iotdb 
> （68机器/data/liuzhen_test/12_0305_3154faf下）。
> merge_thread_num=2
> 2022-03-07 09:49:42 启动iotdb成功。
> 2022-03-07 11:10:26 ：（启动数据库1小时20分钟）执行delete操作，脚本为：
> #!/bin/bash
> base_db_dir="/data/liuzhen_test/1205_pre4"
> out_dev="dev_name.txt"
> function expDevNameToFile()
> {
> ${base_db_dir}/sbin/start-cli.sh  -e "set max_display_num=0 ;show 
> devices;"|grep root|awk -F '|' '{print $2}'>${out_dev}
> }
> expDevNameToFile
> # delete 2 row  1 of seq ,1 of unseq
> function del2rows()
> {
>while read LINE
>do
>   ${base_db_dir}/sbin/start-cli.sh -e " select s_0 from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00 or time=2034-01-24T18:53:22.000+08:00" >> 
> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " delete from "$LINE" where 
> time=2034-01-24T18:53:22.000+08:00" >> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " delete from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00" >> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " select s_0 from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00 or time=2034-01-24T18:53:22.000+08:00" >> 
> sel_del_sel.out
>   echo $LINE
>done  < ${out_dev}
> }
> del2rows
> 执行脚本完成后，killall -u liuzhen
> 删除iotdb进程，当前的合并情况和mods情况：
> 342个mods文件。
> 2个merge，4个compaction：
>  !image-2022-03-07-15-29-25-036.png! 
> 备份此数据，记作备份2。
> 备份2 在：/data/liuzhen_test/12_0305_3154faf/data_0307_stop-merge2-comp3_del2rows
> 3. 被测分支：在68机器
> /data/liuzhen_test/pr_0304_formatCompactionLog
> copy备份2 到此路径。
> 启动数据库。
> 启动数据库后，执行脚本：
> nohup sh -x r.sh > r.log &
> 这个脚本串行调用2个脚本：
> sh -x ./select_del-2rows.sh > s.log
> sh -x ./q.sh > q.log
> 这3个脚本见附件。
> {color:#DE350B}*问题1：*{color}
> 2022-03-07 11:37:49 iotdb启动成功。2小时18分钟后有异常：
> 2022-03-07 13:55:51,443 [pool-7-IoTDB-Compaction-5] ERROR 
> o.a.i.t.f.f.LocalFSFactory:118 - Failed to move file from 
> /data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.inner
>  to 
> /data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.tsfile.
> {color:#DE350B}org.apache.commons.io.FileExistsException: File element in 
> parameter 'null' already exists: 
> {color}'/data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.tsfile'
> at org.apache.commons.io.FileUtils.requireAbsent(FileUtils.java:2587)
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2305)
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2283)
> at 
> org.apache.iotdb.tsfile.fileSystem.fsFactory.LocalFSFactory.moveFile(LocalFSFactory.java:116)
> at 
> org.apache.iotdb.db.engine.compaction.inner.utils.InnerSpaceCompactionUtils.moveTargetFile(InnerSpaceCompactionUtils.java:285)
> at 
> org.apache.iotdb.db.engine.compaction.inner.sizetiered.SizeTieredCompactionTask.doCompaction(SizeTieredCompactionTask.java:127)
> at 
> org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:66)
> at 
> org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:45)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at

[jira] [Assigned] (IOTDB-2693) [compaction] org.apache.commons.io.FileExistsException: File element in parameter 'null' already exists

2022-03-07 Thread Jira 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

刘珍 reassigned IOTDB-2693:
-

Assignee: Jialin Qiao

> [compaction] org.apache.commons.io.FileExistsException: File element in 
> parameter 'null' already exists
> ---
>
> Key: IOTDB-2693
> URL: https://issues.apache.org/jira/browse/IOTDB-2693
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Compaction
>Affects Versions: 0.13.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: Jialin Qiao
>Priority: Major
> Attachments: image-2022-03-07-15-19-26-903.png, 
> image-2022-03-07-15-29-25-036.png, q.sh, r.sh, screenshot-1.png, 
> screenshot-2.png, select_del-2rows.sh
>
>
> 被测iotdb信息，是个分支版本：
> https://github.com/choubenson/iotdb/tree/formatCompactionLog
> 测试内容：0.12合并中断 ( 有delete操作)，然后0.13恢复的正确性。
> 问题描述：
> {color:#DE350B}*问题1：合并过程中有异常org.apache.commons.io.FileExistsException: File 
> element in parameter 'null' already exists: 
> 问题2：0.12的备份日志compaction.log 在0.13没有被删除
> 问题3：0.12 deleted的2条记录，在0.13能查询到。
> 实际查询结果，所有的设备的count值为2。
> 期望结果是，每个设备的记录数为：199,999,998。
> 问题4：0.13的data中还有0.12的merge.mods文件。
> 问题5：java.nio.BufferUnderflowException: null。*{color}
> 复现流程：
> 1. 0.12.5-pre4 
> 关合并，生成数据，记作备份1（在/data/liuzhen_test/1205_pre4/data_bk_close-comp）。
> 在192.168.10.68 /data/liuzhen_test/1205_pre4/data_bk_close-comp路径。
> 共10sg/100设备 /100 序列，每个序列2亿点。
> unseq tsfile个数：50064
> seq tsfile个数：50076
> 2. 0.12 0305版本3154faf。copy 备份1 到此iotdb 
> （68机器/data/liuzhen_test/12_0305_3154faf下）。
> merge_thread_num=2
> 2022-03-07 09:49:42 启动iotdb成功。
> 2022-03-07 11:10:26 ：（启动数据库1小时20分钟）执行delete操作，脚本为：
> #!/bin/bash
> base_db_dir="/data/liuzhen_test/1205_pre4"
> out_dev="dev_name.txt"
> function expDevNameToFile()
> {
> ${base_db_dir}/sbin/start-cli.sh  -e "set max_display_num=0 ;show 
> devices;"|grep root|awk -F '|' '{print $2}'>${out_dev}
> }
> expDevNameToFile
> # delete 2 row  1 of seq ,1 of unseq
> function del2rows()
> {
>while read LINE
>do
>   ${base_db_dir}/sbin/start-cli.sh -e " select s_0 from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00 or time=2034-01-24T18:53:22.000+08:00" >> 
> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " delete from "$LINE" where 
> time=2034-01-24T18:53:22.000+08:00" >> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " delete from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00" >> sel_del_sel.out
>   ${base_db_dir}/sbin/start-cli.sh -e " select s_0 from "$LINE" where 
> time=2024-05-24T18:00:05.000+08:00 or time=2034-01-24T18:53:22.000+08:00" >> 
> sel_del_sel.out
>   echo $LINE
>done  < ${out_dev}
> }
> del2rows
> 执行脚本完成后，killall -u liuzhen
> 删除iotdb进程，当前的合并情况和mods情况：
> 342个mods文件。
> 2个merge，4个compaction：
>  !image-2022-03-07-15-29-25-036.png! 
> 备份此数据，记作备份2。
> 备份2 在：/data/liuzhen_test/12_0305_3154faf/data_0307_stop-merge2-comp3_del2rows
> 3. 被测分支：在68机器
> /data/liuzhen_test/pr_0304_formatCompactionLog
> copy备份2 到此路径。
> 启动数据库。
> 启动数据库后，执行脚本：
> nohup sh -x r.sh > r.log &
> 这个脚本串行调用2个脚本：
> sh -x ./select_del-2rows.sh > s.log
> sh -x ./q.sh > q.log
> 这3个脚本见附件。
> {color:#DE350B}*问题1：*{color}
> 2022-03-07 11:37:49 iotdb启动成功。2小时18分钟后有异常：
> 2022-03-07 13:55:51,443 [pool-7-IoTDB-Compaction-5] ERROR 
> o.a.i.t.f.f.LocalFSFactory:118 - Failed to move file from 
> /data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.inner
>  to 
> /data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.tsfile.
> {color:#DE350B}org.apache.commons.io.FileExistsException: File element in 
> parameter 'null' already exists: 
> {color}'/data/liuzhen_test/pr_0304_formatCompactionLog/./sbin/../data/data/sequence/root.ip5.ip53/0/0/1645065803917-37-1-1.tsfile'
> at org.apache.commons.io.FileUtils.requireAbsent(FileUtils.java:2587)
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2305)
> at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2283)
> at 
> org.apache.iotdb.tsfile.fileSystem.fsFactory.LocalFSFactory.moveFile(LocalFSFactory.java:116)
> at 
> org.apache.iotdb.db.engine.compaction.inner.utils.InnerSpaceCompactionUtils.moveTargetFile(InnerSpaceCompactionUtils.java:285)
> at 
> org.apache.iotdb.db.engine.compaction.inner.sizetiered.SizeTieredCompactionTask.doCompaction(SizeTieredCompactionTask.java:127)
> at 
> org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:66)
> at 
> org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:45)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> ja