[jira] [Created] (IOTDB-6271) PBTree Mode supports node-level read/write locking with concurrent flush and querying

2023-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-6271:
-

 Summary: PBTree Mode supports node-level read/write locking with 
concurrent flush and querying
 Key: IOTDB-6271
 URL: https://issues.apache.org/jira/browse/IOTDB-6271
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


PBTree Mode supports node-level read/write locking with concurrent flush and 
querying.

Docs: [https://apache-iotdb.feishu.cn/docx/U72AdRODgowAZCx76DRcWr8hned]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-6270) PBTree mode support logical view

2023-12-19 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-6270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798784#comment-17798784
 ] 

yanze chen commented on IOTDB-6270:
---

https://github.com/apache/iotdb/pull/11401

> PBTree mode support logical view
> 
>
> Key: IOTDB-6270
> URL: https://issues.apache.org/jira/browse/IOTDB-6270
> Project: Apache IoTDB
>  Issue Type: New Feature
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> In IoTDB, all features in the View User Manual should be supported in PBTree 
> mode.
> [https://timecho.com/docs/zh/UserGuide/V1.3.x/User-Manual/IoTDB-View_timecho.html#%E8%A7%86%E5%9B%BE]
>  
> Enable PBTree Mode:
> set schema_engine_mode to "PBTree"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6270) PBTree mode support logical view

2023-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-6270:
-

 Summary: PBTree mode support logical view
 Key: IOTDB-6270
 URL: https://issues.apache.org/jira/browse/IOTDB-6270
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


In IoTDB, all features in the View User Manual should be supported in PBTree 
mode.

[https://timecho.com/docs/zh/UserGuide/V1.3.x/User-Manual/IoTDB-View_timecho.html#%E8%A7%86%E5%9B%BE]

 

Enable PBTree Mode:

set schema_engine_mode to "PBTree"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-6231) SchemaCache supports precise eviction

2023-10-31 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-6231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781393#comment-17781393
 ] 

yanze chen commented on IOTDB-6231:
---

https://github.com/apache/iotdb/pull/11226

> SchemaCache supports precise eviction
> -
>
> Key: IOTDB-6231
> URL: https://issues.apache.org/jira/browse/IOTDB-6231
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> Now once deletion of data occurs, it clears all SchemaCache on the DataNode 
> it is on.
> Optimised to delete entries in the SchemaCache as accurately as possible if 
> the device does not contain wildcards.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6231) SchemaCache supports precise eviction

2023-10-31 Thread yanze chen (Jira)
yanze chen created IOTDB-6231:
-

 Summary: SchemaCache supports precise eviction
 Key: IOTDB-6231
 URL: https://issues.apache.org/jira/browse/IOTDB-6231
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Now once deletion of data occurs, it clears all SchemaCache on the DataNode it 
is on.

Optimised to delete entries in the SchemaCache as accurately as possible if the 
device does not contain wildcards.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6162) Enhanced accuracy in deleting data

2023-09-18 Thread yanze chen (Jira)
yanze chen created IOTDB-6162:
-

 Summary: Enhanced accuracy in deleting data
 Key: IOTDB-6162
 URL: https://issues.apache.org/jira/browse/IOTDB-6162
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2023-09-18-17-51-49-501.png, 
image-2023-09-18-17-53-47-557.png

There are currently a number of potential optimisation points when it comes to 
deleting data.

 
 # Time partitioning is not considered when generating distributed plans.  
!image-2023-09-18-17-51-49-501.png|width=597,height=397!
 # When taking the TsFIleResourceList, the unwanted partitions can be filtered 
out. !image-2023-09-18-17-53-47-557.png|width=1084,height=610!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6147) Alter view does not check the source path.

2023-09-11 Thread yanze chen (Jira)
yanze chen created IOTDB-6147:
-

 Summary: Alter view does not check the source path.
 Key: IOTDB-6147
 URL: https://issues.apache.org/jira/browse/IOTDB-6147
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-6130) Delete data by specific pattern didn't work

2023-09-04 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-6130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-6130:
-

Assignee: yanze chen

> Delete data by specific pattern didn't work
> ---
>
> Key: IOTDB-6130
> URL: https://issues.apache.org/jira/browse/IOTDB-6130
> Project: Apache IoTDB
>  Issue Type: Bug
>Affects Versions: 1.1.2
>Reporter: Haonan Hou
>Assignee: yanze chen
>Priority: Minor
> Attachments: image-2023-08-29-17-37-33-557.png
>
>
> Execute the following sql, you will see data of root.sg.d2.s1 doesn't delete 
> by `delete from root.\*.\*.s1`.
> {code:sql}
> IoTDB> insert into root.sg.d2(time,s1,s2) values(1,2,3)
> Msg: The statement is executed successfully.
> IoTDB> flush
> Msg: The statement is executed successfully.
> IoTDB> select ** from root
> +-+-+-+
> |                         Time|root.sg.d2.s1|root.sg.d2.s2|
> +-+-+-+
> |1970-01-01T08:00:00.001+08:00|          2.0|          3.0|
> +-+-+-+
> Total line number = 1
> It costs 0.153s
> IoTDB> delete from root.*.*.s1
> Msg: The statement is executed successfully.
> IoTDB> select ** from root
> +-+-+-+
> |                         Time|root.sg.d2.s1|root.sg.d2.s2|
> +-+-+-+
> |1970-01-01T08:00:00.001+08:00|          2.0|          3.0|
> +-+-+-+
> Total line number = 1
> It costs 0.013s
> IoTDB> delete from root.**.s1
> Msg: The statement is executed successfully.
> IoTDB> select ** from root
> +-+-+-+
> |                         Time|root.sg.d2.s1|root.sg.d2.s2|
> +-+-+-+
> |1970-01-01T08:00:00.001+08:00|         null|          3.0|
> +-+-+-+
> Total line number = 1
> It costs 0.018s{code}
> The following code causes this issue.
> !image-2023-08-29-17-37-33-557.png|width=364,height=261!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6124) Add scope filter parameter for tree visitor framework

2023-08-21 Thread yanze chen (Jira)
yanze chen created IOTDB-6124:
-

 Summary: Add scope filter parameter for tree visitor framework
 Key: IOTDB-6124
 URL: https://issues.apache.org/jira/browse/IOTDB-6124
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


With the introduction of the permissions module, traversal of the MTree may no 
longer be a search of the whole tree.

Fortunately, the PathPatternTree for a permission must consist of either a 
prefix or a fullPath. For example, root.db.d1.s1 or root.db.d2.**. Therefore, 
we only need access to the subtree identified by the permissions 
PathPatternTree.

The PathPatternTree of permissions can be easily converted to a deterministic 
automaton (DFA). In the Mtree access framework, we can introduce a new member 
variable ScopeDFA to restrict the search scope. Pruning will be done early when 
the search exceeds the ScopeDFA.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-6076) Add duplicate checking when upsert alias

2023-07-20 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-6076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745417#comment-17745417
 ] 

yanze chen commented on IOTDB-6076:
---

https://github.com/apache/iotdb/pull/10618

> Add duplicate checking when upsert alias
> 
>
> Key: IOTDB-6076
> URL: https://issues.apache.org/jira/browse/IOTDB-6076
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
> Attachments: image-2023-07-21-13-22-45-887.png, 
> image-2023-07-21-13-23-35-183.png
>
>
> Aliases should not conflict with other column names/aliases
>  
> !image-2023-07-21-13-23-35-183.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6076) Add duplicate checking when upsert alias

2023-07-20 Thread yanze chen (Jira)
yanze chen created IOTDB-6076:
-

 Summary: Add duplicate checking when upsert alias
 Key: IOTDB-6076
 URL: https://issues.apache.org/jira/browse/IOTDB-6076
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2023-07-21-13-22-45-887.png, 
image-2023-07-21-13-23-35-183.png

Aliases should not conflict with other column names/aliases

 

!image-2023-07-21-13-23-35-183.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-6074) Ignore error message when TagManager createSnapt

2023-07-20 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-6074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-6074:
-

  Sprint: Schema-Backlog
Assignee: yanze chen

> Ignore error message when TagManager createSnapt
> 
>
> Key: IOTDB-6074
> URL: https://issues.apache.org/jira/browse/IOTDB-6074
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
> Attachments: image-2023-07-20-14-37-45-736.png, 
> image-2023-07-20-14-37-55-986.png
>
>
> !image-2023-07-20-14-37-45-736.png!
> !image-2023-07-20-14-37-55-986.png|width=659,height=524!
>  
> Error message should not be printed when the tmp file does not exist.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6074) Ignore error message when TagManager createSnapt

2023-07-20 Thread yanze chen (Jira)
yanze chen created IOTDB-6074:
-

 Summary: Ignore error message when TagManager createSnapt
 Key: IOTDB-6074
 URL: https://issues.apache.org/jira/browse/IOTDB-6074
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
 Attachments: image-2023-07-20-14-37-45-736.png, 
image-2023-07-20-14-37-55-986.png

!image-2023-07-20-14-37-45-736.png!

!image-2023-07-20-14-37-55-986.png|width=1132,height=900!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-6065) Considering LastCacheContainer in the memory estimation of SchemaCacheEntry

2023-07-13 Thread yanze chen (Jira)
yanze chen created IOTDB-6065:
-

 Summary: Considering LastCacheContainer in the memory estimation 
of SchemaCacheEntry
 Key: IOTDB-6065
 URL: https://issues.apache.org/jira/browse/IOTDB-6065
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


{code:java}
/**
 * Total basic 100B
 *
 * 
 *   SchemaCacheEntry Object header, 8B
 *   isAligned, 1B
 *   LastCacheContainer reference, 8B
 *   MeasurementSchema
 *   
 * Reference, 8B
 * Object header, 8B
 * String measurementId basic, 8 + 8 + 4 + 8 + 4 = 32B
 * type, encoding, compressor, 3 B
 * encodingConverter, 8 + 8 + 8 = 24B
 * props, 8B
 *   
 * 
 */
public static int estimateSize(SchemaCacheEntry schemaCacheEntry) {
  // each char takes 2B in Java
  return 100 + 2 * 
schemaCacheEntry.getIMeasurementSchema().getMeasurementId().length();
} {code}
The current memory estimate for the SchemaCacheEntry does not include the 
lastCacheContainer's retained size.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5892) Parsing error : ”create database root.view; “

2023-05-17 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723392#comment-17723392
 ] 

yanze chen commented on IOTDB-5892:
---

https://github.com/apache/iotdb/pull/9875

> Parsing error : ”create database root.view; “
> -
>
> Key: IOTDB-5892
> URL: https://issues.apache.org/jira/browse/IOTDB-5892
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: mpp-cluster
>Reporter: 刘珍
>Assignee: yanze chen
>Priority: Minor
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> iotdb version 1.2.0-SNAPSHOT (Build: 1e6475d)
> IoTDB> create database root.timeseries;
> Msg: The statement is executed successfully.
> IoTDB>
> IoTDB>
> IoTDB> create database root.view;
> {color:#DE350B}*Msg: 700: Error occurred while parsing SQL to physical plan: 
> line 1:21 mismatched input 'view' expecting {ID, QUOTED_ID}*{color}
> IoTDB>
> {color:#DE350B}解析报错,需修复。{color}
>  !screenshot-2.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5881) Support path contains filter in show timeseries and show devices

2023-05-15 Thread yanze chen (Jira)
yanze chen created IOTDB-5881:
-

 Summary: Support path contains filter in show timeseries and show 
devices
 Key: IOTDB-5881
 URL: https://issues.apache.org/jira/browse/IOTDB-5881
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


Support path contains filter in show timeseries and show devices



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5826) Fix schema query with * cannot display satisfied template series

2023-04-26 Thread yanze chen (Jira)
yanze chen created IOTDB-5826:
-

 Summary: Fix schema query with * cannot display satisfied template 
series
 Key: IOTDB-5826
 URL: https://issues.apache.org/jira/browse/IOTDB-5826
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2023-04-27-09-52-04-565.png

!image-2023-04-27-09-52-04-565.png|width=811,height=217!

create schema template t1 (temperature FLOAT encoding=RLE, status BOOLEAN 
encoding=PLAIN compression=SNAPPY);

set schema template t1 to root.sg1;

create timeseries using schema template on root.sg1.d1;

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-5824) show devices with * cannot display satisfied devices

2023-04-26 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-5824:
-

Assignee: yanze chen  (was: Yukun Zhou)

> show devices with * cannot display satisfied devices
> 
>
> Key: IOTDB-5824
> URL: https://issues.apache.org/jira/browse/IOTDB-5824
> Project: Apache IoTDB
>  Issue Type: Bug
>  Components: Core/Schema Manager
>Reporter: Yuan Tian
>Assignee: yanze chen
>Priority: Major
>  Labels: pull-request-available
> Attachments: image-2023-04-26-15-23-52-166.png
>
>
> !image-2023-04-26-15-23-52-166.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5732) Add Device Schema Statistics

2023-03-27 Thread yanze chen (Jira)
yanze chen created IOTDB-5732:
-

 Summary: Add Device Schema Statistics
 Key: IOTDB-5732
 URL: https://issues.apache.org/jira/browse/IOTDB-5732
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


In order to better implement cluster quota management and enrich the metadata 
monitoring dashboard, it is necessary to add device-related statistical values 
when SchemaRegion creates and deletes device nodes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5730) Use a hard link when loading a SchemaFile snapshot

2023-03-26 Thread yanze chen (Jira)
yanze chen created IOTDB-5730:
-

 Summary: Use a hard link when loading a SchemaFile snapshot
 Key: IOTDB-5730
 URL: https://issues.apache.org/jira/browse/IOTDB-5730
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


Using a hard link when loading a SchemaFile snapshot on the current system can 
lead to corruption of snaptshot. Files.copy should be used instead.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5720) Release processor fail to release memory due to writer-preferred starvation

2023-03-23 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703949#comment-17703949
 ] 

yanze chen commented on IOTDB-5720:
---

https://github.com/apache/iotdb/pull/9422

> Release processor fail to release memory due to writer-preferred starvation
> ---
>
> Key: IOTDB-5720
> URL: https://issues.apache.org/jira/browse/IOTDB-5720
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
> Attachments: image-2023-03-23-15-26-27-207.png
>
>
> !image-2023-03-23-15-26-27-207.png|width=488,height=155!
> [https://apache-iotdb.feishu.cn/docx/WwdAd6SYLofsAHxEJeRcDuWBnPh]
>  
> The current stampedLock uses a writer-preferred rule, which may lead to 
> extreme cases where a waiting write lock request blocks the subsequent 
> release thread from acquiring the read lock, resulting in memory not being 
> released. To solve this problem, a high priority read lock request interface 
> needs to be introduced.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5720) Release processor fail to release memory due to writer-preferred starvation

2023-03-23 Thread yanze chen (Jira)
yanze chen created IOTDB-5720:
-

 Summary: Release processor fail to release memory due to 
writer-preferred starvation
 Key: IOTDB-5720
 URL: https://issues.apache.org/jira/browse/IOTDB-5720
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2023-03-23-15-26-27-207.png

!image-2023-03-23-15-26-27-207.png|width=488,height=155!

[https://apache-iotdb.feishu.cn/docx/WwdAd6SYLofsAHxEJeRcDuWBnPh]

 

The current stampedLock uses a writer-preferred rule, which may lead to extreme 
cases where a waiting write lock request blocks the subsequent release thread 
from acquiring the read lock, resulting in memory not being released. To solve 
this problem, a high priority read lock request interface needs to be 
introduced.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5719) Cluster-Wide Time Series Limit Control

2023-03-22 Thread yanze chen (Jira)
yanze chen created IOTDB-5719:
-

 Summary: Cluster-Wide Time Series Limit Control
 Key: IOTDB-5719
 URL: https://issues.apache.org/jira/browse/IOTDB-5719
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


https://apache-iotdb.feishu.cn/docx/NEa3d9er8oIDtsxe7Wzc8YHjnZf



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5688) ShcemaFile mode lost template schema after reboot in SimpleConsensus

2023-03-16 Thread yanze chen (Jira)
yanze chen created IOTDB-5688:
-

 Summary: ShcemaFile mode lost template schema after reboot in 
SimpleConsensus
 Key: IOTDB-5688
 URL: https://issues.apache.org/jira/browse/IOTDB-5688
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Fix For: 1.1.0, master
 Attachments: image-2023-03-16-17-44-06-034.png

{code:java}
create database root.sg1;
create schema template t1  (temperature FLOAT encoding=RLE, status BOOLEAN 
encoding=PLAIN compression=SNAPPY);
set schema template t1 to root.sg1;
create timeseries of schema template on root.sg1.d3;
create timeseries of schema template on root.sg1.d2; {code}
!image-2023-03-16-17-44-06-034.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5675) Metric of Schema Template

2023-03-14 Thread yanze chen (Jira)
yanze chen created IOTDB-5675:
-

 Summary: Metric of Schema Template
 Key: IOTDB-5675
 URL: https://issues.apache.org/jira/browse/IOTDB-5675
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5660) Rename StorageGroup related expcetion name to Database

2023-03-09 Thread yanze chen (Jira)
yanze chen created IOTDB-5660:
-

 Summary: Rename StorageGroup related expcetion name to Database
 Key: IOTDB-5660
 URL: https://issues.apache.org/jira/browse/IOTDB-5660
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Rename StorageGroup related expcetion name to Database



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5647) Multi-Model IMNode Management

2023-03-08 Thread yanze chen (Jira)
yanze chen created IOTDB-5647:
-

 Summary: Multi-Model IMNode Management
 Key: IOTDB-5647
 URL: https://issues.apache.org/jira/browse/IOTDB-5647
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


https://apache-iotdb.feishu.cn/docx/JZhudIX6Vob7kxx62t2cWeeLnbb



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5622) Create and use aligned template but devices is non-aligned

2023-03-05 Thread yanze chen (Jira)
yanze chen created IOTDB-5622:
-

 Summary: Create and use aligned template but devices is non-aligned
 Key: IOTDB-5622
 URL: https://issues.apache.org/jira/browse/IOTDB-5622
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2023-03-06-10-30-00-008.png, 
image-2023-03-06-10-30-18-520.png

!image-2023-03-06-10-30-00-008.png|width=608,height=215!

!image-2023-03-06-10-30-18-520.png|width=607,height=205!

 

devices root.sg1.d1 should be aligned



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5621) Eliminate the deprecated code in the template

2023-03-05 Thread yanze chen (Jira)
yanze chen created IOTDB-5621:
-

 Summary: Eliminate the deprecated code in the template
 Key: IOTDB-5621
 URL: https://issues.apache.org/jira/browse/IOTDB-5621
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


As the current templates only support single level templates that are either 
aligned or non-aligned, much of the deprecated code in template can be removed.

In addition, the responsibility of the template should be managed for 
MeasurementSchema and not for IMNode. This has also been removed in this issue. 
To guarantee compatibility, the refactoring is done from the lower level of 
Statement.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5578) Keep CacheMemoryManager monitor alive when exception

2023-02-23 Thread yanze chen (Jira)
yanze chen created IOTDB-5578:
-

 Summary: Keep CacheMemoryManager monitor alive when exception
 Key: IOTDB-5578
 URL: https://issues.apache.org/jira/browse/IOTDB-5578
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


flushTaskMonitor and releaseTaskMonitor should keep alive until 
CacheMemoryManager#clear() is invoked expecitly. So they should not throw 
exception in catch clause.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5577) Metric dashboard for Schema Module

2023-02-23 Thread yanze chen (Jira)
yanze chen created IOTDB-5577:
-

 Summary: Metric dashboard for Schema Module
 Key: IOTDB-5577
 URL: https://issues.apache.org/jira/browse/IOTDB-5577
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


[https://apache-iotdb.feishu.cn/docx/HD2hdIYS6oBSzsxpFw2cCNPFnoe]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5576) Some node missed out when concurrent flush release

2023-02-23 Thread yanze chen (Jira)
yanze chen created IOTDB-5576:
-

 Summary: Some node missed out when concurrent flush release 
 Key: IOTDB-5576
 URL: https://issues.apache.org/jira/browse/IOTDB-5576
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Fix bug occurs in SchemaStatisticsTest#testSchemaFileNodeStatistics



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5549) [Schema_File] Ensure concurrently memory release

2023-02-16 Thread yanze chen (Jira)
yanze chen created IOTDB-5549:
-

 Summary: [Schema_File] Ensure concurrently memory release 
 Key: IOTDB-5549
 URL: https://issues.apache.org/jira/browse/IOTDB-5549
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-5545) Heterogeneous schema_engine_mode among DataNodes

2023-02-16 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-5545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-5545:
-

Assignee: Yukun Zhou  (was: yanze chen)

> Heterogeneous schema_engine_mode among DataNodes
> 
>
> Key: IOTDB-5545
> URL: https://issues.apache.org/jira/browse/IOTDB-5545
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: Yukun Zhou
>Priority: Major
>
>  
> The snapshot file formats for SchemaRegion in Memory mode and Schema_File 
> mode are incompatible with each other. If some nodes of the cluster are in 
> Memory mode and some nodes are in Schema_File mode, errors may occur when 
> loading snapshots.
> There are two kinds of solution:
>  # Add a DataNode registration check to ensure that the schema_engine_mode 
> configuration of all DataNodes is consistent.
>  # Ensure compatibility when SchemaRegion loading snapshot, support for 
> datanode heterogeneous configurations.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5545) DataNode registration check for schema_engine_mode

2023-02-16 Thread yanze chen (Jira)
yanze chen created IOTDB-5545:
-

 Summary: DataNode registration check for schema_engine_mode
 Key: IOTDB-5545
 URL: https://issues.apache.org/jira/browse/IOTDB-5545
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


The snapshot file formats for SchemaRegion in Memory mode and Schema_File mode 
are incompatible with each other. If some nodes of the cluster are in Memory 
mode and some nodes are in Schema_File mode, errors may occur when loading 
snapshots. Therefore, it is necessary to add a DataNode registration check to 
ensure that the schema_engine_mode configuration of all DataNodes is consistent.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5528) Memory statistics for each schema region

2023-02-13 Thread yanze chen (Jira)
yanze chen created IOTDB-5528:
-

 Summary: Memory statistics for each schema region
 Key: IOTDB-5528
 URL: https://issues.apache.org/jira/browse/IOTDB-5528
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


The memory statistics for the current metadata are global. For better 
monitoring, the granularity of memory statistics needs to be refined per 
SchemaRegion.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5524) Add Metrics for Schema File

2023-02-12 Thread yanze chen (Jira)
yanze chen created IOTDB-5524:
-

 Summary: Add Metrics for Schema File
 Key: IOTDB-5524
 URL: https://issues.apache.org/jira/browse/IOTDB-5524
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-5524) Add Metrics for Schema File

2023-02-12 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-5524:
-

Assignee: yanze chen

> Add Metrics for Schema File
> ---
>
> Key: IOTDB-5524
> URL: https://issues.apache.org/jira/browse/IOTDB-5524
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5510) Block and wait to create timeseries when releasing memory

2023-02-08 Thread yanze chen (Jira)
yanze chen created IOTDB-5510:
-

 Summary: Block and wait to create timeseries when releasing memory
 Key: IOTDB-5510
 URL: https://issues.apache.org/jira/browse/IOTDB-5510
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


The current strategy when creating a time series is to simply throw a 
SeriesOverflowException if memory resources are running low. However, there may 
be background tasks on the system that are releasing memory at this time and 
memory resources may become available immediately.

This issue provides a mechanism to wait for a limited time when memory is low 
and a releasingTask is running.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5471) Add isDone() check in WrappedThreadPoolExecutor#afterExecute

2023-02-04 Thread yanze chen (Jira)
yanze chen created IOTDB-5471:
-

 Summary: Add isDone() check in 
WrappedThreadPoolExecutor#afterExecute
 Key: IOTDB-5471
 URL: https://issues.apache.org/jira/browse/IOTDB-5471
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


I want to use the thread pool created by the IoTDBThreadPoolFactory in 
CompleteFuture. However, I have found that this can cause problems with threads 
not being released.

Here is the example code.
{code:java}
import org.apache.iotdb.commons.concurrent.IoTDBThreadPoolFactory;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;

public class Tests {

  public static void main(String[] args) {
ExecutorService flushTaskExecutor1 = 
IoTDBThreadPoolFactory.newFixedThreadPool(2, "1");
CompletableFuture[] completableFutures = new CompletableFuture[5];
for (int j = 0; j < 5; j++) {
  completableFutures[j] =
  CompletableFuture.supplyAsync(
  () -> {
System.out.println("run");
return null;
  },
  flushTaskExecutor1);
}
CompletableFuture.allOf(completableFutures).join();
System.out.println("done");
flushTaskExecutor1.shutdown();
  }
}
 {code}
Only two “run” are printed, whereas the expectation is that 5 “run” should be 
printed. if I change the thread creation code to 
{code:java}
ExecutorService flushTaskExecutor1 = Executors.newFixedThreadPool(2); {code}
, it works as expected.

Using the thread dump, I found that the thread was blocking at 
WrappedThreadPoolExecutor#afterExecute after execution. This was due to a 
problem with the implementation of CompleteFuture#AsyncRun. One solution I 
found in 
[https://stackoverflow.com/questions/2248131/handling-exceptions-from-java-executorservice-tasks]
 is to add the isDone check before invoking get().  The {{isDone()}} check is 
necessary in certain cases to avoid blocking. See 
[bugs.openjdk.org/browse/JDK-8071638|https://bugs.openjdk.org/browse/JDK-8071638]
 and 
[bugs.openjdk.org/browse/JDK-7146994|https://bugs.openjdk.org/browse/JDK-7146994]
 for details.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5459) [SchemaFile] Memory control cross SchemaRegion

2023-02-02 Thread yanze chen (Jira)
yanze chen created IOTDB-5459:
-

 Summary: [SchemaFile] Memory control cross SchemaRegion 
 Key: IOTDB-5459
 URL: https://issues.apache.org/jira/browse/IOTDB-5459
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


[https://apache-iotdb.feishu.cn/docx/IVtgdbJNPoIjpdxlQTocOuTTnLh]

 

The shortcoming of the current implementation is that a schemaRegion can only 
trigger the release of its own occupied memory and cannot affect other 
schemaRegions, lacking global memory control.

 

Therefore, a global CacheMemoryManager needs to be introduced for memory 
control.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5442) Standardising the use of CachedMTreeStore read/write locks

2023-01-30 Thread yanze chen (Jira)
yanze chen created IOTDB-5442:
-

 Summary: Standardising the use of CachedMTreeStore read/write locks
 Key: IOTDB-5442
 URL: https://issues.apache.org/jira/browse/IOTDB-5442
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


https://apache-iotdb.feishu.cn/docx/WwdAd6SYLofsAHxEJeRcDuWBnPh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5414) Timeseries with alias deleted success but can still be queried by alias

2023-01-12 Thread yanze chen (Jira)
yanze chen created IOTDB-5414:
-

 Summary: Timeseries with alias deleted success but can still be 
queried by alias
 Key: IOTDB-5414
 URL: https://issues.apache.org/jira/browse/IOTDB-5414
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


{code:java}
IoTDB> create timeseries root.turbine.d1.s1(temprature) with datatype=FLOAT, 
encoding=RLE, compression=SNAPPY
Msg: The statement is executed successfully.
IoTDB> create timeseries root.turbine.d1.s2 with datatype=FLOAT, encoding=RLE, 
compression=SNAPPY
Msg: The statement is executed successfully.
IoTDB> SHOW TIMESERIES root.**
+--+--++++---++--++--+
|        Timeseries|     Alias|    
Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|
+--+--++++---++--++--+
|root.turbine.d1.s1|temprature|root.turbine|   FLOAT|     RLE|     SNAPPY|null| 
     null|    null|              null|
|root.turbine.d1.s2|      null|root.turbine|   FLOAT|     RLE|     SNAPPY|null| 
     null|    null|              null|
+--+--++++---++--++--+
Total line number = 2
It costs 0.109s
IoTDB> delete timeseries root.turbine.d1.s1
Msg: The statement is executed successfully.
IoTDB> SHOW TIMESERIES root.**
+--+-++++---++--++--+
|        Timeseries|Alias|    
Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|
+--+-++++---++--++--+
|root.turbine.d1.s2| null|root.turbine|   FLOAT|     RLE|     SNAPPY|null|      
null|    null|              null|
+--+-++++---++--++--+
Total line number = 1
It costs 0.022s
IoTDB> SHOW TIMESERIES root.turbine.d1.temprature
+--+--++++---++--++--+
|        Timeseries|     Alias|    
Database|DataType|Encoding|Compression|Tags|Attributes|Deadband|DeadbandParameters|
+--+--++++---++--++--+
|root.turbine.d1.s1|temprature|root.turbine|   FLOAT|     RLE|     SNAPPY|null| 
     null|    null|              null|
+--+--++++---++--++--+
Total line number = 1
It costs 0.019s {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5413) ReadWriteLock for CachedMTreeStore

2023-01-12 Thread yanze chen (Jira)
yanze chen created IOTDB-5413:
-

 Summary: ReadWriteLock for CachedMTreeStore
 Key: IOTDB-5413
 URL: https://issues.apache.org/jira/browse/IOTDB-5413
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


https://apache-iotdb.feishu.cn/docx/WwdAd6SYLofsAHxEJeRcDuWBnPh



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5398) Handle failure in schema query if there is exception occurs during the iteration

2023-01-10 Thread yanze chen (Jira)
yanze chen created IOTDB-5398:
-

 Summary: Handle failure in schema query if there is exception 
occurs during the iteration
 Key: IOTDB-5398
 URL: https://issues.apache.org/jira/browse/IOTDB-5398
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Handle failure in schema query if there is exception occurs during the iteration



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5396) Iterative interface for query about TagManager

2023-01-10 Thread yanze chen (Jira)
yanze chen created IOTDB-5396:
-

 Summary: Iterative interface for query about TagManager
 Key: IOTDB-5396
 URL: https://issues.apache.org/jira/browse/IOTDB-5396
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


The current query interface relating to TagManager returns a collection of type 
list. For better adaptation to SchemaReader, iterator should be returned.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5391) Implement SchemaReader using iterative Traverser

2023-01-09 Thread yanze chen (Jira)
yanze chen created IOTDB-5391:
-

 Summary: Implement SchemaReader using iterative Traverser
 Key: IOTDB-5391
 URL: https://issues.apache.org/jira/browse/IOTDB-5391
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


The current SchemaReader uses the result set returned from the original 
SchemaRegion to generate an iterator; however, the new Traverser supports the 
iterator pattern, allowing the SchemaReader to be implemented using an 
iterative Traverser, avoiding the overhead of repeated traversals.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-5235) [SchemaFile] support props

2023-01-06 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-5235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-5235:
-

Assignee: ZhaoXin  (was: yanze chen)

> [SchemaFile] support props
> --
>
> Key: IOTDB-5235
> URL: https://issues.apache.org/jira/browse/IOTDB-5235
> Project: Apache IoTDB
>  Issue Type: New Feature
>Reporter: yanze chen
>Assignee: ZhaoXin
>Priority: Major
>
> SchemaFile need to support props.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException

2023-01-06 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-4960:
-

Assignee: ZhaoXin  (was: yanze chen)

> [SchemaFile] Query large amounts of metadata throw ClosedChannelException
> -
>
> Key: IOTDB-4960
> URL: https://issues.apache.org/jira/browse/IOTDB-4960
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: yanze chen
>Assignee: ZhaoXin
>Priority: Major
> Attachments: image-2022-11-16-15-29-18-436.png, 
> image-2022-11-16-16-01-50-022.png
>
>
> reproduce:
> 1C1D Schema_File mode 128G for DataNode
>  
> run benchmark with config: 
> {*}DEVICE_NUMBER{*}=5000
> {*}SENSOR_NUMBER{*}=2
> {*}CLIENT_NUMBER{*}=5
> {*}GROUP_NUMBER{*}=10
> {*}BATCH_SIZE_PER_WRITE{*}=0
> {*}LOOP{*}=1
> {*}VECTOR{*}=true
>  
> execute SQL:
> count timeseries root.**
>  
> !image-2022-11-16-15-29-18-436.png|width=881,height=596!
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5359) Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset

2023-01-06 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655289#comment-17655289
 ] 

yanze chen commented on IOTDB-5359:
---

https://github.com/apache/iotdb/pull/8721

> Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset
> --
>
> Key: IOTDB-5359
> URL: https://issues.apache.org/jira/browse/IOTDB-5359
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5360) Release node and iterator in AbstractTreeVisitor

2023-01-06 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655288#comment-17655288
 ] 

yanze chen commented on IOTDB-5360:
---

https://github.com/apache/iotdb/pull/8721

> Release node and iterator in AbstractTreeVisitor
> 
>
> Key: IOTDB-5360
> URL: https://issues.apache.org/jira/browse/IOTDB-5360
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> To support pin and unpin in SchemaFile mode, AbstractTreeVisitor should 
> release node and iterator after using. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-5342) Separate filtering and business logic in AbstractTreeVisitor

2023-01-06 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-5342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655287#comment-17655287
 ] 

yanze chen commented on IOTDB-5342:
---

https://github.com/apache/iotdb/pull/8721

> Separate filtering and business logic in AbstractTreeVisitor
> 
>
> Key: IOTDB-5342
> URL: https://issues.apache.org/jira/browse/IOTDB-5342
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>  Labels: pull-request-available
>
> Separate filtering and business logic in AbstractTreeVisitor



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5360) Release node and iterator in AbstractTreeVisitor

2023-01-04 Thread yanze chen (Jira)
yanze chen created IOTDB-5360:
-

 Summary: Release node and iterator in AbstractTreeVisitor
 Key: IOTDB-5360
 URL: https://issues.apache.org/jira/browse/IOTDB-5360
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


To support pin and unpin in SchemaFile mode, AbstractTreeVisitor should release 
node and iterator after using. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5359) Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset

2023-01-04 Thread yanze chen (Jira)
yanze chen created IOTDB-5359:
-

 Summary: Use decorator pattern to refactor 
SchemaTreeVisitorWithLimitOffset
 Key: IOTDB-5359
 URL: https://issues.apache.org/jira/browse/IOTDB-5359
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Use decorator pattern to refactor SchemaTreeVisitorWithLimitOffset



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5342) Separate filtering and business logic in AbstractTreeVisitor

2023-01-03 Thread yanze chen (Jira)
yanze chen created IOTDB-5342:
-

 Summary: Separate filtering and business logic in 
AbstractTreeVisitor
 Key: IOTDB-5342
 URL: https://issues.apache.org/jira/browse/IOTDB-5342
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Separate filtering and business logic in AbstractTreeVisitor



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5329) Exception handler for AbstractTreeVisitor

2023-01-02 Thread yanze chen (Jira)
yanze chen created IOTDB-5329:
-

 Summary: Exception handler for AbstractTreeVisitor
 Key: IOTDB-5329
 URL: https://issues.apache.org/jira/browse/IOTDB-5329
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Add exception handler for AbstractTreeVisitor



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5277) [schema] SchemaRegion throws NPE when loading snapshot

2022-12-24 Thread yanze chen (Jira)
yanze chen created IOTDB-5277:
-

 Summary: [schema] SchemaRegion throws NPE when loading snapshot 
 Key: IOTDB-5277
 URL: https://issues.apache.org/jira/browse/IOTDB-5277
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: 9b23c97eca9eeb1adc11571607baa3f6.zip

reproduce: Launch 1C1D using the attached data directory

 
{code:java}
1972 [main] ERROR o.a.iotdb.commons.ServerCommandLine - Failed to execute 
system command 
java.util.concurrent.CompletionException: java.lang.NullPointerException: 
Cannot invoke 
"org.apache.iotdb.db.metadata.mnode.IMNode.getAsStorageGroupMNode()" because 
the return value of 
"org.apache.iotdb.db.metadata.mtree.store.MemMTreeStore.getRoot()" is null
    at 
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
    at 
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
    at 
java.base/java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1498)
    at 
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at 
java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
    at org.apache.ratis.util.ConcurrentUtils.accept(ConcurrentUtils.java:174)
    at 
org.apache.ratis.util.ConcurrentUtils.lambda$null$3(ConcurrentUtils.java:165)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException: Cannot invoke 
"org.apache.iotdb.db.metadata.mnode.IMNode.getAsStorageGroupMNode()" because 
the return value of 
"org.apache.iotdb.db.metadata.mtree.store.MemMTreeStore.getRoot()" is null
    at 
org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.(MTreeBelowSGMemoryImpl.java:133)
    at 
org.apache.iotdb.db.metadata.mtree.MTreeBelowSGMemoryImpl.loadFromSnapshot(MTreeBelowSGMemoryImpl.java:173)
    at 
org.apache.iotdb.db.metadata.schemaregion.SchemaRegionMemoryImpl.loadSnapshot(SchemaRegionMemoryImpl.java:457)
    at 
org.apache.iotdb.db.consensus.statemachine.SchemaRegionStateMachine.loadSnapshot(SchemaRegionStateMachine.java:67)
    at 
org.apache.iotdb.consensus.ratis.ApplicationStateMachineProxy.loadSnapshot(ApplicationStateMachineProxy.java:257)
    at org.a {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5247) [SchemaFile] Drop tags or attributes throw NPE

2022-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-5247:
-

 Summary: [SchemaFile] Drop tags or attributes throw NPE 
 Key: IOTDB-5247
 URL: https://issues.apache.org/jira/browse/IOTDB-5247
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


Reproduce:
{code:java}
// SchemaRegionAliasAndTagTest.java
@Test
public void testDropTagsOrAttributes() {
  try {
prepareTimeseries();
Set keySet = new HashSet<>(Arrays.asList("tag1", "tag2", "attr1", 
"attr2"));
List fullPaths =
Arrays.asList(
"root.sg.wf01.wt01.v1.s1",
"root.sg.wf01.wt01.v1.s2",
"root.sg.wf01.aligned_device1.s1",
"root.sg.wf01.aligned_device1.s2",
"root.sg.wf01.aligned_device2.s1",
"root.sg.wf01.aligned_device2.s2");
for (String fullPath : fullPaths) {
  schemaRegion.dropTagsOrAttributes(keySet, new PartialPath(fullPath));
  checkTags(fullPath, Collections.emptyMap());
  checkAttributes(fullPath, Collections.emptyMap());
}
  } catch (Exception e) {
logger.error(e.getMessage(), e);
Assert.fail(e.getMessage());
  }
} {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5242) UT related to MetaUtils

2022-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-5242:
-

 Summary: UT related to MetaUtils
 Key: IOTDB-5242
 URL: https://issues.apache.org/jira/browse/IOTDB-5242
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5239) UT related to delete timeseries

2022-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-5239:
-

 Summary: UT related to delete timeseries
 Key: IOTDB-5239
 URL: https://issues.apache.org/jira/browse/IOTDB-5239
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


UT related to delete timeseries



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5238) UT related to alias and tags and attributes

2022-12-19 Thread yanze chen (Jira)
yanze chen created IOTDB-5238:
-

 Summary: UT related to alias and tags and attributes
 Key: IOTDB-5238
 URL: https://issues.apache.org/jira/browse/IOTDB-5238
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


UT related to alias and tags and attributes



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5235) [SchemaFile] support props

2022-12-18 Thread yanze chen (Jira)
yanze chen created IOTDB-5235:
-

 Summary: [SchemaFile] support props
 Key: IOTDB-5235
 URL: https://issues.apache.org/jira/browse/IOTDB-5235
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


SchemaFile need to support props.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5223) Check data directories cross-disk under RatisConsensus

2022-12-14 Thread yanze chen (Jira)
yanze chen created IOTDB-5223:
-

 Summary: Check data directories cross-disk under RatisConsensus
 Key: IOTDB-5223
 URL: https://issues.apache.org/jira/browse/IOTDB-5223
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Configuring the data directories as cross-disk directories is not supported 
under RatisConsensus(it will be supported in a later version)

 

_https://stackoverflow.com/questions/56712909/get-a-folders-mount-point-using-java_



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5201) Extracting getConfig from the registerDataNode interface

2022-12-13 Thread yanze chen (Jira)
yanze chen created IOTDB-5201:
-

 Summary: Extracting getConfig from the registerDataNode interface
 Key: IOTDB-5201
 URL: https://issues.apache.org/jira/browse/IOTDB-5201
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2022-12-13-20-59-57-380.png

Extracting getConfig from the registerDataNode interface

 

!image-2022-12-13-20-59-57-380.png|width=591,height=308!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5133) [0.13] Export schema tool

2022-12-06 Thread yanze chen (Jira)
yanze chen created IOTDB-5133:
-

 Summary: [0.13] Export schema tool
 Key: IOTDB-5133
 URL: https://issues.apache.org/jira/browse/IOTDB-5133
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


Add an export schema function to export MTree and Tag from the current system 
into a non-deleted mlog.bin for loading mlog in a non-empty iotdb correctly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-5106) Parse PathPattern to Deterministic Finite Automate

2022-12-01 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-5106:
-

Assignee: yanze chen

> Parse PathPattern to Deterministic Finite Automate
> --
>
> Key: IOTDB-5106
> URL: https://issues.apache.org/jira/browse/IOTDB-5106
> Project: Apache IoTDB
>  Issue Type: New Feature
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> https://apache-iotdb.feishu.cn/docx/DjrAdDM3woXcn1x5Phlcyveznqe



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5106) Parse PathPattern to Deterministic Finite Automate

2022-12-01 Thread yanze chen (Jira)
yanze chen created IOTDB-5106:
-

 Summary: Parse PathPattern to Deterministic Finite Automate
 Key: IOTDB-5106
 URL: https://issues.apache.org/jira/browse/IOTDB-5106
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen


https://apache-iotdb.feishu.cn/docx/DjrAdDM3woXcn1x5Phlcyveznqe



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-5021) Using Junit parameterized test to optimize schema IT framework

2022-11-22 Thread yanze chen (Jira)
yanze chen created IOTDB-5021:
-

 Summary: Using Junit parameterized test to optimize schema IT 
framework
 Key: IOTDB-5021
 URL: https://issues.apache.org/jira/browse/IOTDB-5021
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


In order to make the tests cover Schema_File mode and Memory mode, we 
originally needed to create two classes for each IT, such as 
IoTDBMetadataFetchIT and IoTDBMetadataFetchSchemaFileIT, which would result in 
a large number of classes.

This ISSUE introduces a JUnit parametric test approach. An AbstractSchemaIT 
abstract class is created for all schema test, and its subclasses will 
automatically run tests in both schema engine modes.

 

reference:

[https://blog.csdn.net/w605283073/article/details/80051229]

[https://stackoverflow.com/questions/27745691/how-to-combine-runwith-with-runwithparameterized-class]

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4970) [sync] Enhanced sync safety management

2022-11-16 Thread yanze chen (Jira)
yanze chen created IOTDB-4970:
-

 Summary: [sync] Enhanced sync safety management
 Key: IOTDB-4970
 URL: https://issues.apache.org/jira/browse/IOTDB-4970
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Before the sync sender can send data, a handshake with the receiver is 
required. The current handshake policy is to verify the sender's IP address 
based on the receiver's white list. However, the current sender's ip address is 
provided by the sender and may be falsified leading to security issues.

 
 # This ISSUE will get the ip address from the thrift connection.
 # The default whitelist is set to 0.0.0.0/32, which rejects all the sender's 
ip.
 # Some optimizations have been made to handle cases of handshake failure.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException

2022-11-16 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634708#comment-17634708
 ] 

yanze chen commented on IOTDB-4960:
---

原因可能是mpp框架查询超时,interrupt掉了正在read的channel,导致channel关闭

!image-2022-11-16-16-01-50-022.png|width=827,height=407!

> [SchemaFile] Query large amounts of metadata throw ClosedChannelException
> -
>
> Key: IOTDB-4960
> URL: https://issues.apache.org/jira/browse/IOTDB-4960
> Project: Apache IoTDB
>  Issue Type: Bug
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
> Attachments: image-2022-11-16-15-29-18-436.png, 
> image-2022-11-16-16-01-50-022.png
>
>
> reproduce:
> 1C1D Schema_File mode 128G for DataNode
>  
> run benchmark with config: 
> {*}DEVICE_NUMBER{*}=5000
> {*}SENSOR_NUMBER{*}=2
> {*}CLIENT_NUMBER{*}=5
> {*}GROUP_NUMBER{*}=10
> {*}BATCH_SIZE_PER_WRITE{*}=0
> {*}LOOP{*}=1
> {*}VECTOR{*}=true
>  
> execute SQL:
> count timeseries root.**
>  
> !image-2022-11-16-15-29-18-436.png|width=881,height=596!
>  
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4960) [SchemaFile] Query large amounts of metadata throw ClosedChannelException

2022-11-15 Thread yanze chen (Jira)
yanze chen created IOTDB-4960:
-

 Summary: [SchemaFile] Query large amounts of metadata throw 
ClosedChannelException
 Key: IOTDB-4960
 URL: https://issues.apache.org/jira/browse/IOTDB-4960
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2022-11-16-15-29-18-436.png

reproduce:

1C1D Schema_File mode 128G for DataNode

 

run benchmark with config: 

*DEVICE_NUMBER*=5000

*SENSOR_NUMBER*=2

*CLIENT_NUMBER*=5

*GROUP_NUMBER*=10

*BATCH_SIZE_PER_WRITE*=0

{*}LOOP{*}=1

 

execute SQL:

count timeseries root.**

 

 

!image-2022-11-16-15-29-18-436.png!

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4941) Prevent compatibility problems in PipeData

2022-11-15 Thread yanze chen (Jira)
yanze chen created IOTDB-4941:
-

 Summary: Prevent compatibility problems in PipeData
 Key: IOTDB-4941
 URL: https://issues.apache.org/jira/browse/IOTDB-4941
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Prevent compatibility problems in PipeData beacause of ordinal of PipeDataType



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4908) Clear deprecayed code related to sync in old standalone

2022-11-10 Thread yanze chen (Jira)
yanze chen created IOTDB-4908:
-

 Summary: Clear deprecayed code related to sync in old standalone
 Key: IOTDB-4908
 URL: https://issues.apache.org/jira/browse/IOTDB-4908
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IOTDB-4209) [ schema recovery ] Aligned timeseries recovery is slow

2022-11-08 Thread yanze chen (Jira)


 [ 
https://issues.apache.org/jira/browse/IOTDB-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yanze chen reassigned IOTDB-4209:
-

Assignee: yanze chen  (was: ZhaoXin)

> [ schema recovery ] Aligned timeseries recovery is slow
> ---
>
> Key: IOTDB-4209
> URL: https://issues.apache.org/jira/browse/IOTDB-4209
> Project: Apache IoTDB
>  Issue Type: Improvement
>  Components: Core/Schema Manager
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: 刘珍
>Assignee: yanze chen
>Priority: Major
>
> 单机,sf_tree_seg,0819_207abf6 
> bm创建对齐序列耗时650.73 second
> 恢复这些序列耗时{color:#DE350B}*1716436ms*{color}, 需优化对齐序列的恢复性能。
> bm运行时间19分钟:
> 2022-08-22 09:56:01 到 2022-08-22 10:15:42
> {color:#00875A}*Create schema cost 650.73 second*{color}
> Test elapsed time (not include schema creation): 308.10 second
> iotdb重启恢复:
> 2022-08-22 13:19:19,195 [main] INFO o.a.i.db.service.IoTDB:140 - recover the 
> schema...
> 2022-08-22 13:47:55,631 [main] INFO o.a.i.db.service.IoTDB:234 - spend 
> {color:#DE350B}*1716436ms*{color} to recover schema.
> log_all.log 耗时38分钟
> 2022-08-22 13:19:18 到2022-08-22 13:57:52,856 [main] INFO 
> o.a.i.db.service.IoTDB:198 - Congratulation, IoTDB is set up successfully. 
> Now, enjoy yourself!
> 复现流程:
> 1. 192.168.10.64 72C256G
> MAX_HEAP_SIZE="192G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> mlog_buffer_size=10485760
> schema_engine_mode=Schema_File
> 2. benchmark运行附件中的脚本
> benchmark在本机路径/data/liuzhen_test/weektest/benchmark_tool_v13 (0620的benchmark)
> 主要配置
> DEVICE_NUMBER=500
> SENSOR_NUMBER=20
> CLIENT_NUMBER=20
> GROUP_NUMBER=100
> VECTOR=true
> LOOP=1
> BATCH_SIZE_PER_WRITE=1
> OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4854) [UT] Migrate SchemaBasicTest to SchemaRegionBasicTest

2022-11-05 Thread yanze chen (Jira)
yanze chen created IOTDB-4854:
-

 Summary: [UT] Migrate SchemaBasicTest to SchemaRegionBasicTest
 Key: IOTDB-4854
 URL: https://issues.apache.org/jira/browse/IOTDB-4854
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4840) [schema] Fix MLog writing logic about template

2022-11-02 Thread yanze chen (Jira)
yanze chen created IOTDB-4840:
-

 Summary: [schema] Fix MLog writing logic about template 
 Key: IOTDB-4840
 URL: https://issues.apache.org/jira/browse/IOTDB-4840
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen
 Attachments: image-2022-11-02-20-52-49-392.png

MLog entry should be splitted into actual fullPath and templateId

!image-2022-11-02-20-52-49-392.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4827) [SchemaFile] Adaptation for cluster template

2022-11-01 Thread yanze chen (Jira)
yanze chen created IOTDB-4827:
-

 Summary: [SchemaFile] Adaptation for cluster template
 Key: IOTDB-4827
 URL: https://issues.apache.org/jira/browse/IOTDB-4827
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


# activateSchemaTemplate

 # getPathsUsingTemplate

 # constructSchemaBlackListWithTemplate
 # rollbackSchemaBlackListWithTemplate
 # deactivateTemplateInBlackList
 # countPathsUsingTemplate
 # getAllTimeseriesCount, fetchSchema andshowTimeseriesWithoutIndex with 
template



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4808) Rewrite SyncTransportTest in new IT framework

2022-10-31 Thread yanze chen (Jira)
yanze chen created IOTDB-4808:
-

 Summary: Rewrite SyncTransportTest in new IT framework
 Key: IOTDB-4808
 URL: https://issues.apache.org/jira/browse/IOTDB-4808
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4799) [sync] Print more detailed pipe information in show pipe

2022-10-29 Thread yanze chen (Jira)
yanze chen created IOTDB-4799:
-

 Summary: [sync] Print more detailed pipe information in show pipe
 Key: IOTDB-4799
 URL: https://issues.apache.org/jira/browse/IOTDB-4799
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


* Add a column attributes to show pipe attributes
 * For receiver, show connection of sender



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4743) [sync] OperatePipeProcedure support rollback

2022-10-24 Thread yanze chen (Jira)
yanze chen created IOTDB-4743:
-

 Summary: [sync] OperatePipeProcedure support rollback
 Key: IOTDB-4743
 URL: https://issues.apache.org/jira/browse/IOTDB-4743
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


OperatePipeProcedure support rollback



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4628) [sync] Do not store pipe infomation after drop

2022-10-24 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17623479#comment-17623479
 ] 

yanze chen commented on IOTDB-4628:
---

https://github.com/apache/iotdb/pull/7581

> [sync] Do not store pipe infomation after drop
> --
>
> Key: IOTDB-4628
> URL: https://issues.apache.org/jira/browse/IOTDB-4628
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: yanze chen
>Assignee: yanze chen
>Priority: Major
>
> In old version, pipe infomation will be stored even if pipe has been dropped. 
> There are two advantages to not storing pipe infomation after drop can.
>  # Support rollback logic in procedure better. If failed to create pipe, drop 
> directlt.
>  # Other modules(e.g.Trigger) do the same thing. Doing so can keep the 
> semantics consistent in IoTDB.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4719) [SchemaFile] Adaptation for cluster delete timeseries

2022-10-23 Thread yanze chen (Jira)
yanze chen created IOTDB-4719:
-

 Summary: [SchemaFile] Adaptation for cluster delete timeseries
 Key: IOTDB-4719
 URL: https://issues.apache.org/jira/browse/IOTDB-4719
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


Support 3 interface in SchemaRegionSchemaFileImpl:
 # constructSchemaBlackList

 # rollbackSchemaBlackList

 # deleteTimeseriesInBlackList



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4714) [sync] Rename TPipeInfo to TCreatePipeReq

2022-10-21 Thread yanze chen (Jira)
yanze chen created IOTDB-4714:
-

 Summary: [sync] Rename TPipeInfo to TCreatePipeReq
 Key: IOTDB-4714
 URL: https://issues.apache.org/jira/browse/IOTDB-4714
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4692) SchemaFile adaptation for cluster metadata validation

2022-10-18 Thread yanze chen (Jira)
yanze chen created IOTDB-4692:
-

 Summary: SchemaFile adaptation for cluster metadata validation
 Key: IOTDB-4692
 URL: https://issues.apache.org/jira/browse/IOTDB-4692
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen


Support these two interface in SchemaRegionSchemaFileImpl:
 # checkMeasurementExistence()

 # fetchSchema() without template



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4678) [sync] Sync DataNode start-up process

2022-10-17 Thread yanze chen (Jira)
yanze chen created IOTDB-4678:
-

 Summary: [sync] Sync DataNode start-up process
 Key: IOTDB-4678
 URL: https://issues.apache.org/jira/browse/IOTDB-4678
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4677) [sync] Support sync DeleteData

2022-10-17 Thread yanze chen (Jira)
yanze chen created IOTDB-4677:
-

 Summary: [sync] Support sync DeleteData
 Key: IOTDB-4677
 URL: https://issues.apache.org/jira/browse/IOTDB-4677
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yusicheng


* Support collect DeleteData in sender
 * Add new PipeData type
 * Support load DeleteData in receiver



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4676) [sync] Support sync DeleteTimeseries

2022-10-17 Thread yanze chen (Jira)
yanze chen created IOTDB-4676:
-

 Summary: [sync] Support sync DeleteTimeseries
 Key: IOTDB-4676
 URL: https://issues.apache.org/jira/browse/IOTDB-4676
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yusicheng


* Support collect DeleteTimeseries in sender
 * Add new PipeData type
 * Support load DeleteTimeseries in receiver



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4022) Adjust the storage_group_report_threshold according to memory for writing

2022-10-13 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616878#comment-17616878
 ] 

yanze chen commented on IOTDB-4022:
---

更新测试:

吞吐(point/s),写入内存分配=8980481489
|| 
||aligned_seq_config.properties||seq_config.properties||unseq_config.properties||aligned_unseq_config.properties||
|storage_group_report_threshold=16777216|8912483.19|17433312.42/ 
16564385.56|3345125.65 |半个小时只跑了10%|
|write_memory_vriation_report_proportion=0.001|13742844.78|16521936.57 
/13984814.52|5082498.77 | |

吞吐(point/s),总内存分配=100M
|| ||seq_config.properties||
|storage_group_report_threshold=16777216|写入一部分后OOM|
|write_memory_vriation_report_proportion=0.001|写入失败,reject|

> Adjust the storage_group_report_threshold according to memory for writing
> -
>
> Key: IOTDB-4022
> URL: https://issues.apache.org/jira/browse/IOTDB-4022
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: Jialin Qiao
>Assignee: yanze chen
>Priority: Major
>  Labels: pull-request-available
> Attachments: 完整实验结果.md
>
>
> Currently, we have a parameter called storage_group_report_threshold, which 
> means when a storage group report its memory usage, it should accumulate a 
> memory change of this number.
> If this parameter is too small, may report frequently, which may decrease the 
> insertion performance.
> If this parameter is too large, may report not in time, which may cause OOM.
>  
> We need to change this parameter according to the memory allocated for 
> writing.
>  
> (1) remove storage_group_report_threshold
> (2) Add a new parameter write_memory_vriation_report_proportion
>  
> To rel/0.13 and master



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4628) [sync] Do not store pipe infomation after drop

2022-10-12 Thread yanze chen (Jira)
yanze chen created IOTDB-4628:
-

 Summary: [sync] Do not store pipe infomation after drop
 Key: IOTDB-4628
 URL: https://issues.apache.org/jira/browse/IOTDB-4628
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


In old version, pipe infomation will be stored even if pipe has been dropped. 
There are two advantages to not storing pipe infomation after drop can.
 # Support rollback logic in procedure better. If failed to create pipe, drop 
directlt.
 # Other modules(e.g.Trigger) do the same thing. Doing so can keep the 
semantics consistent in IoTDB.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4570) Move SerializationRunTimeException to node-commons

2022-10-04 Thread yanze chen (Jira)
yanze chen created IOTDB-4570:
-

 Summary: Move SerializationRunTimeException to node-commons
 Key: IOTDB-4570
 URL: https://issues.apache.org/jira/browse/IOTDB-4570
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4562) Optimize ITimeIndex#getDevices()

2022-09-29 Thread yanze chen (Jira)
yanze chen created IOTDB-4562:
-

 Summary: Optimize ITimeIndex#getDevices()
 Key: IOTDB-4562
 URL: https://issues.apache.org/jira/browse/IOTDB-4562
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IOTDB-4402) Avoid multiple IoTDB processes accessing same data dir

2022-09-28 Thread yanze chen (Jira)


[ 
https://issues.apache.org/jira/browse/IOTDB-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17610549#comment-17610549
 ] 

yanze chen commented on IOTDB-4402:
---

https://stackoverflow.com/questions/35842/how-can-a-java-program-get-its-own-process-id

> Avoid multiple IoTDB processes accessing same data dir 
> ---
>
> Key: IOTDB-4402
> URL: https://issues.apache.org/jira/browse/IOTDB-4402
> Project: Apache IoTDB
>  Issue Type: Improvement
>Reporter: Haonan Hou
>Assignee: yanze chen
>Priority: Major
>
> If multiple IoTDB processes operate the same data dirs, the data file may be 
> damaged. Need a method to protect the data dir.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4528) [sync] Pipe operation in cluster version

2022-09-26 Thread yanze chen (Jira)
yanze chen created IOTDB-4528:
-

 Summary: [sync] Pipe operation in cluster version
 Key: IOTDB-4528
 URL: https://issues.apache.org/jira/browse/IOTDB-4528
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4488) Implement ClusterSyncInfo snapshot by SyncLogWritter and SyncLogReader

2022-09-21 Thread yanze chen (Jira)
yanze chen created IOTDB-4488:
-

 Summary: Implement ClusterSyncInfo snapshot by SyncLogWritter and 
SyncLogReader
 Key: IOTDB-4488
 URL: https://issues.apache.org/jira/browse/IOTDB-4488
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4459) [sync] PipeSink operation in cluster version

2022-09-19 Thread yanze chen (Jira)
yanze chen created IOTDB-4459:
-

 Summary: [sync] PipeSink operation in cluster version
 Key: IOTDB-4459
 URL: https://issues.apache.org/jira/browse/IOTDB-4459
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4456) [sync] Sync cluster framework in ConfigNode

2022-09-19 Thread yanze chen (Jira)
yanze chen created IOTDB-4456:
-

 Summary: [sync] Sync cluster framework in ConfigNode 
 Key: IOTDB-4456
 URL: https://issues.apache.org/jira/browse/IOTDB-4456
 Project: Apache IoTDB
  Issue Type: New Feature
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4453) Fix the overflow of compareTo in TimeRange

2022-09-19 Thread yanze chen (Jira)
yanze chen created IOTDB-4453:
-

 Summary: Fix the overflow of compareTo in TimeRange
 Key: IOTDB-4453
 URL: https://issues.apache.org/jira/browse/IOTDB-4453
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4393) [docs] Detailed design of distributed SYNC

2022-09-13 Thread yanze chen (Jira)
yanze chen created IOTDB-4393:
-

 Summary: [docs] Detailed design of distributed SYNC
 Key: IOTDB-4393
 URL: https://issues.apache.org/jira/browse/IOTDB-4393
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen


Detail design of distributed SYNC, mainly about ConfigNode
 * Interface
 * Interaction between DataNode and ConfigNode
 * Procedure state for sync

https://apache-iotdb.feishu.cn/docx/doxcnpfFAOfJJGj6qmDM4VPEDNf



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4391) Apply PatternTreeMap to parse .mods file

2022-09-13 Thread yanze chen (Jira)
yanze chen created IOTDB-4391:
-

 Summary: Apply PatternTreeMap to parse .mods file
 Key: IOTDB-4391
 URL: https://issues.apache.org/jira/browse/IOTDB-4391
 Project: Apache IoTDB
  Issue Type: Improvement
Reporter: yanze chen
Assignee: yanze chen






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IOTDB-4352) Fix PatialPath#overlapWith PartialPath contains MULTI_LEVEL_PATH_WILDCARD

2022-09-06 Thread yanze chen (Jira)
yanze chen created IOTDB-4352:
-

 Summary: Fix PatialPath#overlapWith PartialPath contains 
MULTI_LEVEL_PATH_WILDCARD
 Key: IOTDB-4352
 URL: https://issues.apache.org/jira/browse/IOTDB-4352
 Project: Apache IoTDB
  Issue Type: Bug
Reporter: yanze chen
Assignee: yanze chen


PatialPath#overlapWith should not return true directly if PartialPath contains 
MULTI_LEVEL_PATH_WILDCARD.

For example, root.**.s1 is not overlap with root.**.s2



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


  1   2   >