[jira] [Created] (IOTDB-3051) javax.servelet dependency conflict

2022-04-29 Thread Xiangdong Huang (Jira) 
Xiangdong Huang created IOTDB-3051:
--

 Summary: javax.servelet dependency conflict
 Key: IOTDB-3051
 URL: https://issues.apache.org/jira/browse/IOTDB-3051
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Xiangdong Huang


see [https://github.com/apache/iotdb/discussions/5620]

 

when using `mvn dependency:tree`,  we can find many modules depends on javax.* 
(especially for javax.servelet). However, the versions of these dependencies 
are not compatible. 

 

I have did some preliminary work:
 * IoTDB-server module depends on Jetty for supporting Rest API, and jetty 
depends on javax.servlet-api-3.1.0;
 * hadoop module depends on  org.apache.hadoop:hadoop-mapreduce-client-core and 
it depends on javax.servlet-api-2.5

 

This issue wants to find a way to make a consensus on the dependency version, 
i.e., either upgrade to 3.1 or downgrade to 2.5, depends on which can guarantee 
the system works well.

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3050) Support expression evaluation with time column

2022-04-29 Thread Steve Yurong Su (Jira) 
Steve Yurong Su created IOTDB-3050:
--

 Summary: Support expression evaluation with time column
 Key: IOTDB-3050
 URL: https://issues.apache.org/jira/browse/IOTDB-3050
 Project: Apache IoTDB
  Issue Type: Task
Reporter: Steve Yurong Su
Assignee: Steve Yurong Su






--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3049) Sync Sender loss data when restart IoTDB with data in working memtable

2022-04-29 Thread yusicheng (Jira) 
yusicheng created IOTDB-3049:


 Summary: Sync Sender loss data when restart IoTDB with data in 
working memtable
 Key: IOTDB-3049
 URL: https://issues.apache.org/jira/browse/IOTDB-3049
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Tools/Sync
Affects Versions: master branch
Reporter: yusicheng
Assignee: yusicheng
 Fix For: master branch


  Data loss happens when sender shutdowns with data in working memtable.
  When restarting IoTDB, WAL will restore all data in unsealed tsfile and close 
it, but this tsfile is not collected by running pipe.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3048) Structure of BinaryTransformer can be optimized

2022-04-29 Thread Xi Zhang (Jira) 
Xi Zhang created IOTDB-3048:
---

 Summary: Structure of BinaryTransformer can be optimized
 Key: IOTDB-3048
 URL: https://issues.apache.org/jira/browse/IOTDB-3048
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: Xi Zhang
Assignee: Xi Zhang


A lot of switch case statements are used in BinaryTransformer, and the enum 

TransformerType seems to be redundant. We can rewrite it wilh sbstract methods



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3047) [sync] Path does not exist when delete storage group

2022-04-29 Thread yanze chen (Jira) 
yanze chen created IOTDB-3047:
-

 Summary: [sync] Path does not exist when delete storage group 
 Key: IOTDB-3047
 URL: https://issues.apache.org/jira/browse/IOTDB-3047
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


34062 [pool-19-IoTDB-Sync-Collector-1] INFO  o.a.i.d.s.r.collector.Collector - 
Start load pipeData with serialize number 25 and type 
SCHEMA,value=SchemaPipeData\{serialNumber=25, 
plan=DeleteTimeSeriesPlan{deletePathList=[root.sg2.d1, root.sg2.d1.**, 
root.sg2.d2, root.sg2.d2.**], results={}, partitionFilter=null}} 
34062 [pool-19-IoTDB-Sync-Collector-1] ERROR o.a.i.d.s.r.collector.Collector - 
Cannot load pipeData with serialize number 25 and type SCHEMA, because Path 
[root.sg2.d1] does not exist 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3046) ClientManagerTest can't bind port successfully

2022-04-29 Thread Eric Pai (Jira) 
Eric Pai created IOTDB-3046:
---

 Summary: ClientManagerTest can't bind port successfully
 Key: IOTDB-3046
 URL: https://issues.apache.org/jira/browse/IOTDB-3046
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Cluster
Reporter: Eric Pai
Assignee: Eric Pai
 Fix For: master branch
 Attachments: image-2022-04-29-17-07-01-714.png

I guess the reason is that the port 9003 is still in use after server closed. 
The TCP connections are on TIME_WAIT status.

https://www.zhihu.com/question/20129467

!image-2022-04-29-17-07-01-714.png!



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Assigned] (IOTDB-2797) [privilege] root.** doesn't work

2022-04-29 Thread Steve Yurong Su (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Yurong Su reassigned IOTDB-2797:
--

Assignee: Liu Wei

> [privilege] root.**   doesn't work
> --
>
> Key: IOTDB-2797
> URL: https://issues.apache.org/jira/browse/IOTDB-2797
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Server
>Affects Versions: 0.13.0
>Reporter: 刘珍
>Assignee: Liu Wei
>Priority: Major
> Attachments: image-2022-03-24-14-40-26-163.png
>
>
> 0.13.0 rc1
> 给用户root.**路径对应的权限，无效。
> delete storage group root.**;
> drop user lily_create_trig;
> create user lily_create_trig '123456';
> set storage group to root.sg1;
> CREATE TIMESERIES root.sg1.dev0.s_1 WITH DATATYPE=INT32, ENCODING=GORILLA;
> set storage group to root.sg2;
> CREATE TIMESERIES root.sg2.dev0.s_1 WITH DATATYPE=INT32, ENCODING=GORILLA;
> CREATE TIMESERIES root.sg1.dev1.s_1 WITH DATATYPE=INT32, ENCODING=GORILLA;
> grant user lily_create_trig privileges CREATE_TRIGGER on root.**;
> list privileges user lily_create_trig on root.**;
> list privileges user lily_create_trig on root.sg1;
> 存储组root.sg1上没有权限：
>  !image-2022-03-24-14-40-26-163.png! 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Assigned] (IOTDB-2957) Fix the bug of python client password parsing error

2022-04-29 Thread Steve Yurong Su (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Yurong Su reassigned IOTDB-2957:
--

Assignee: Xi Zhang

> Fix the bug of python client password parsing error
> ---
>
> Key: IOTDB-2957
> URL: https://issues.apache.org/jira/browse/IOTDB-2957
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: 周沛辰
>Assignee: Xi Zhang
>Priority: Major
> Attachments: 151650362578_.pic.jpg, 41650361129_.pic.jpg
>
>
> Using the python client, the session with the wrong password can still 
> connect to the server.
>  !41650361129_.pic.jpg! 
>   !151650362578_.pic.jpg! 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (IOTDB-3045) [ schema recovery ] The query result contains some timeseries that have been deleted

2022-04-29 Thread Jira 
刘珍 created IOTDB-3045:
-

 Summary: [ schema recovery ] The query result contains some 
timeseries that have been deleted
 Key: IOTDB-3045
 URL: https://issues.apache.org/jira/browse/IOTDB-3045
 Project: Apache IoTDB
  Issue Type: Bug
  Components: Core/Schema Manager
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Jialin Qiao
 Attachments: config.properties, d1.log

master_0428_ca4f3cf
问题描述：
50 sg，100 dev，20 sensors/dev, 共2000万序列。
100个设备名称记录在文件中，
遍历文件，10个用户并发 delete ts，1个设备上delete 1条序列。
重复10次，也就是这100个dev，每个dev共delete 10个ts。
delete 后，执行count timeseries dev.* ；结果符合预期。

备份数据。

{color:#DE350B}重启iotdb，恢复元数据后，再查询 100个dev的序列数，有14个设备的序列数都是20万，
期望是10，delete的ts，又查询到了。{color}

详细测试流程：
1. 私有云 192.168.130.2 
2. benchmark 配置见附件
3. 获取dev 名称，记录到文件dev_name.txt
 cat get_dev_name.sh
#!/bin/bash
./sbin/start-cli.sh  -e "show devices;" |grep root|awk -F '|' '{print $2}' > 
dev_name.txt

4. 并发delete
cat del_1_ts.sh
#!/bin/bash
function delete_ts()
{
   del_ts=$1
   dev_name=$2
   exp_count=$3
   count_ts ${exp_count} ${dev_name}
   ./sbin/start-cli.sh -e "delete timeseries ${del_ts};"
   let exp_count--
   count_ts ${exp_count} ${dev_name}
}

function count_ts()
{
exp_c_ts=$1
dev_name=$2
cur_c_ts=`./sbin/start-cli.sh -e "count timeseries ${dev_name};"|sed -n 
"4,4p"|awk -F '|' '{print $2}'`
if [ "${cur_c_ts}" -eq "${exp_c_ts}" ];then
echo "ok"
else
echo "fail. ${dev_name}  ${exp_c_ts} ${cur_c_ts}" >>./fail.log
fi
}

function execute()
{
in_file=$1
i=$2
exp_count=$3
n=0
cat ${in_file} | while read line
do
   del_ts="${line}.s_${i}"
   dev_name="${line}.*";
   delete_ts ${del_ts} ${dev_name} ${exp_count} &
   let i++
   let n++
   b=$(( $n % 10 ))
   if [ "$b" -eq "0" ];then
  wait
   fi
done
wait
}

# 循环几轮
count_ts_base=20

for loop in {1..10}
do
execute "dev_name.txt" ${loop} ${count_ts_base}
let count_ts_base--
done

并发delete脚本的日志文件见附件。
执行count_ts.sh，查看每个dev的count timeseries，是否都是10。
有14个dev的结果是20，不符合预期结果。




--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Assigned] (IOTDB-3044) [ concurrent delete sg ] There are some timeseries that cannot be deleted

2022-04-29 Thread ZhaoXin (Jira) 


 [ 
https://issues.apache.org/jira/browse/IOTDB-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoXin reassigned IOTDB-3044:
--

Assignee: ZhaoXin  (was: Jialin Qiao)

> [ concurrent delete sg ] There are some timeseries that cannot be deleted
> -
>
> Key: IOTDB-3044
> URL: https://issues.apache.org/jira/browse/IOTDB-3044
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Schema Manager
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: ZhaoXin
>Priority: Major
> Attachments: image-2022-04-29-13-45-06-033.png
>
>
> master_0428_ca4f3cf
> 问题描述：
> 50sg，500 dev, 20 sensor/dev
> show devices的结果输出到dev_name.txt
> 顺序读取dev_name.txt中的设备名称，一个设备上1次delete 1个ts，10用户并发执行delete。
> 有的ts delete不掉。
> 日志中记录delete ts的操作，show timeseries还可以看到：
>  !image-2022-04-29-13-45-06-033.png! 
> 测试流程
> 1. 192.168.10.68 机器
> benchmark生成元数据。
> DEVICE_NUMBER=500
> SENSOR_NUMBER=20
> CLIENT_NUMBER=50
> GROUP_NUMBER=50
> 2. 获取devices （放在${iotdb_dir}）
> 执行脚本
> get_dev_name.sh
> #!/bin/bash
> ./sbin/start-cli.sh  -e "show devices;" |grep root|awk -F '|' '{print $2}' > 
> dev_name.txt
> 3. 启动并行delete （放在${iotdb_dir}）
> cat del_10ts.sh
> #!/bin/bash
> function delete_ts()
> {
>del_ts=$1
>dev_name=$2
>exp_count=$3
>count_ts ${exp_count} ${dev_name}
>./sbin/start-cli.sh -e "delete timeseries ${del_ts};"
>let exp_count--
>count_ts ${exp_count} ${dev_name}
> }
> function count_ts()
> {
> exp_c_ts=$1
> dev_name=$2
> cur_c_ts=`./sbin/start-cli.sh -e "count timeseries ${dev_name};"|sed -n 
> "4,4p"|awk -F '|' '{print $2}'`
> if [ "${cur_c_ts}" -eq "${exp_c_ts}" ];then
> echo "ok"
> else
> echo "fail. ${dev_name}  ${exp_c_ts} ${cur_c_ts}" >>./fail.log
> fi
> }
> function execute()
> {
> in_file=$1
> i=$2
> exp_count=$3
> n=0
> cat ${in_file} | while read line
> do
>del_ts="${line}.s_${i}"
>dev_name="${line}.*";
>delete_ts ${del_ts} ${dev_name} ${exp_count} &
>let i++
>let n++
>b=$(( $n % 10 ))
>if [ "$b" -eq "0" ];then
>   wait
>fi
> done
> wait
> }
> count_ts_base=20
> for loop in {0..99}
> do
> idx=$(( $loop * 2000 ))
> execute "dev_name.txt" ${idx} ${count_ts_base}
> let count_ts_base--
> done
> idx=19
> execute "dev_name.txt" ${idx} ${count_ts_base}
> 不符合预期的结果会打印到fail.log
> 根据fail中记录的 ts名称，查看日志，有对应的delete ，show 可以看到没有被delete掉。



--
This message was sent by Atlassian Jira
(v8.20.7#820007)