[jira] [Created] (IOTDB-4954) remove enable_partition and time_partition_interval_for_storage
Quan Siyi created IOTDB-4954: Summary: remove enable_partition and time_partition_interval_for_storage Key: IOTDB-4954 URL: https://issues.apache.org/jira/browse/IOTDB-4954 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4896) Error in flushing a tsFileProcessor with an empty meltable
Quan Siyi created IOTDB-4896: Summary: Error in flushing a tsFileProcessor with an empty meltable Key: IOTDB-4896 URL: https://issues.apache.org/jira/browse/IOTDB-4896 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4784) Control total memory cost for last flush time
[ https://issues.apache.org/jira/browse/IOTDB-4784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-4784: Assignee: Quan Siyi > Control total memory cost for last flush time > - > > Key: IOTDB-4784 > URL: https://issues.apache.org/jira/browse/IOTDB-4784 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4784) Control total memory cost for last flush time
Quan Siyi created IOTDB-4784: Summary: Control total memory cost for last flush time Key: IOTDB-4784 URL: https://issues.apache.org/jira/browse/IOTDB-4784 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4783) Optimize memory usage for LastFlushTimeManager
Quan Siyi created IOTDB-4783: Summary: Optimize memory usage for LastFlushTimeManager Key: IOTDB-4783 URL: https://issues.apache.org/jira/browse/IOTDB-4783 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4645) Optimization of estimating memory usage of cross space compaction task
Quan Siyi created IOTDB-4645: Summary: Optimization of estimating memory usage of cross space compaction task Key: IOTDB-4645 URL: https://issues.apache.org/jira/browse/IOTDB-4645 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Assignee: Quan Siyi Fix For: 0.13.3 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4543) Fix error in calculating time partition slot
Quan Siyi created IOTDB-4543: Summary: Fix error in calculating time partition slot Key: IOTDB-4543 URL: https://issues.apache.org/jira/browse/IOTDB-4543 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4492) Add log and control total size for cross space compaction task
Quan Siyi created IOTDB-4492: Summary: Add log and control total size for cross space compaction task Key: IOTDB-4492 URL: https://issues.apache.org/jira/browse/IOTDB-4492 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (IOTDB-4249) Process unsequential data in sensor level
[ https://issues.apache.org/jira/browse/IOTDB-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reopened IOTDB-4249: -- > Process unsequential data in sensor level > - > > Key: IOTDB-4249 > URL: https://issues.apache.org/jira/browse/IOTDB-4249 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4399) Control chunk size for memtable
Quan Siyi created IOTDB-4399: Summary: Control chunk size for memtable Key: IOTDB-4399 URL: https://issues.apache.org/jira/browse/IOTDB-4399 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4249) Process unsequential data in sensor level
Quan Siyi created IOTDB-4249: Summary: Process unsequential data in sensor level Key: IOTDB-4249 URL: https://issues.apache.org/jira/browse/IOTDB-4249 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4036) The current default value of avg_series_point_number_threshold has a great impact on the write performance of aligned timeseries
[ https://issues.apache.org/jira/browse/IOTDB-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17583920#comment-17583920 ] Quan Siyi commented on IOTDB-4036: -- https://apache-iotdb.feishu.cn/docx/doxcn1X3VZYvQXf5f8TDnK9sbCg > The current default value of avg_series_point_number_threshold has a great > impact on the write performance of aligned timeseries > - > > Key: IOTDB-4036 > URL: https://issues.apache.org/jira/browse/IOTDB-4036 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Server, mpp-cluster >Reporter: 刘珍 >Assignee: Quan Siyi >Priority: Major > Attachments: NewIoTDB_aligned_shunxu_logs.tar.gz, > NewIoTDB_normal_shunxu_logs.tar.gz, image-2022-08-03-16-12-17-550.png, > image-2022-08-05-21-20-51-654.png, screenshot-1.png, screenshot-2.png > > > 周测场景，关合并，对齐序列，顺序数据 > 对比# avg_series_point_number_threshold=10最新默认值 > 与原默认值# avg_series_point_number_threshold=1 > 的写入性能（最新默认值10对写入性能影响大） > !screenshot-2.png! > !screenshot-1.png! > avg_series_point_number_threshold=10 数据库有system reject > （部分写入失败failOperation=30，okOperation=499970） > avg_series_point_number_threshold=1数据库没有system reject，全部写入成功。 > 测试机器192.168.10.64 72C256G，本地测试。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4182) Optimization of file selector of CrossSpaceCompaction
Quan Siyi created IOTDB-4182: Summary: Optimization of file selector of CrossSpaceCompaction Key: IOTDB-4182 URL: https://issues.apache.org/jira/browse/IOTDB-4182 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4101) Directly moving unsequence files when no overlap with sequence files
Quan Siyi created IOTDB-4101: Summary: Directly moving unsequence files when no overlap with sequence files Key: IOTDB-4101 URL: https://issues.apache.org/jira/browse/IOTDB-4101 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3971) Improve the process of writing chunks in compaction for aligned series
[
https://issues.apache.org/jira/browse/IOTDB-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572948#comment-17572948
]
Quan Siyi commented on IOTDB-3971:
--
I compare the performance of cross space copmaction and unseq inner space
compaction for aligned timeseries among three commits.
b309f71b959044cda2c5c37ca96bf89c69fbd640: {*}{*}Row-based data block +
row-based data process
70772cb8684315814959974811440f746e5c19c3: Column-based data block + row-based
data process
def138b5909e5df76440836719b62ebc4b48daeb: Column-based data block +
column-based data process
21 sequence tsfile and 21 unsequence tsfiles with one storage group, 100
devices, 300 sensors, and average 235470 points on each sensor.
unsequence inner space copmaction:
b3: 3.4932 Mb/s
70: 2.3602Mb/s
de: 4.4297Mb/s
cross space compaction:
b3: 1.9377 Mb/s
70: 2.1735Mb/s
de: 2.7183Mb/s
> Improve the process of writing chunks in compaction for aligned series
> --
>
> Key: IOTDB-3971
> URL: https://issues.apache.org/jira/browse/IOTDB-3971
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/Compaction
>Reporter: Quan Siyi
>Assignee: Quan Siyi
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0-SNAPSHOT
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-3884) [Performance]Cross_space and unseq_space compaction are slower at aligned timeseries
[ https://issues.apache.org/jira/browse/IOTDB-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572941#comment-17572941 ] Quan Siyi commented on IOTDB-3884: -- The issue IOTDB-3971 aims to process data in a column-based format from TsBlock to accelerate cross space compaction and unseq inner space compaction > [Performance]Cross_space and unseq_space compaction are slower at aligned > timeseries > > > Key: IOTDB-3884 > URL: https://issues.apache.org/jira/browse/IOTDB-3884 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-07-19-14-17-03-521.png, > image-2022-07-19-14-29-01-699.png, image-2022-07-19-14-30-03-267.png, > screenshot-1.png > > > Please refer to below picture: > Test scenario:Build and Start an iotdb instance of each commit id in master > branch with one dataset, statistics the compaction time in logs. > !image-2022-07-19-14-30-03-267.png|width=928,height=362! > !image-2022-07-19-14-29-01-699.png|width=931,height=187! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3884) [Performance]Cross_space and unseq_space compaction are slower at aligned timeseries
[ https://issues.apache.org/jira/browse/IOTDB-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572940#comment-17572940 ] Quan Siyi commented on IOTDB-3884: -- When executing cross sapce compaction and inner space compaction, we need to decode and decompress chunk of timesereis in the compaction task. We use sereisReader to get chunk of tsfile and decompress and decode it, and return BatchData for further process. Now We the new util "scanSeriesUtil" to get TsBlock of timeseries. The only difference between of BatchData and TsBlock is that Batchdata is row-based and TsBlock is column-based. Therefore, if we continue processing data in a row-based format, we spend much time converting tsBlock to rows. > [Performance]Cross_space and unseq_space compaction are slower at aligned > timeseries > > > Key: IOTDB-3884 > URL: https://issues.apache.org/jira/browse/IOTDB-3884 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-07-19-14-17-03-521.png, > image-2022-07-19-14-29-01-699.png, image-2022-07-19-14-30-03-267.png, > screenshot-1.png > > > Please refer to below picture: > Test scenario:Build and Start an iotdb instance of each commit id in master > branch with one dataset, statistics the compaction time in logs. > !image-2022-07-19-14-30-03-267.png|width=928,height=362! > !image-2022-07-19-14-29-01-699.png|width=931,height=187! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3971) Improve the process of writing chunks in compaction for aligned series
Quan Siyi created IOTDB-3971: Summary: Improve the process of writing chunks in compaction for aligned series Key: IOTDB-3971 URL: https://issues.apache.org/jira/browse/IOTDB-3971 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3535) [cluster]A cross-space compaction has occurred when using benchmark to test iotdb with operation "IS_OUT_OF_ORDER=false"
[ https://issues.apache.org/jira/browse/IOTDB-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570700#comment-17570700 ] Quan Siyi commented on IOTDB-3535: -- https://apache-iotdb.feishu.cn/docx/doxcnBbBgRHpttt9zaKLndpSAwb > [cluster]A cross-space compaction has occurred when using benchmark to test > iotdb with operation "IS_OUT_OF_ORDER=false" > > > Key: IOTDB-3535 > URL: https://issues.apache.org/jira/browse/IOTDB-3535 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: config.properties, image-2022-06-20-09-52-52-981.png, > iotdb-confignode.properties, iotdb-engine.properties, log-all-20220619.19.log > > > A cross-space compaction has occurred when using benchmark to test iotdb with > operation "IS_OUT_OF_ORDER=false" > commit 1309eef2b1fb27a8ce7c8911b9c71ee5a9ed7035 > Author: 刘威 <51618159+liu-wei-...@users.noreply.github.com> > Date: Sat Jun 18 08:50:37 2022 +0800 > [IOTDB-3388] Let MQTT Service could parse JSON array of existed format > (#6154) > > Reproduce steps： > # Setup a cluster with 3C3D > # Using 3BMs to insert data > BM - 》 IoTDB Node > 172.20.70.7 - 》 172.20.70.22 > 172.20.70.8 - 》 172.20.70.23 > 172.20.70.9 - 》 172.20.70.24 > > !image-2022-06-20-09-52-52-981.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3681) Replace RateLimiter with stable one
[ https://issues.apache.org/jira/browse/IOTDB-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564742#comment-17564742 ] Quan Siyi commented on IOTDB-3681: -- We have not found a completely stable version of rateLimiter, so we can use rateLimiter of guava temporaily. > Replace RateLimiter with stable one > --- > > Key: IOTDB-3681 > URL: https://issues.apache.org/jira/browse/IOTDB-3681 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Liuxuxin >Assignee: Quan Siyi >Priority: Major > Fix For: master branch > > > IoTDB use com.google.common.util.concurrent.RateLimiter to limit the rate of > compaction IO, but RateLimiter is unstable with a @Beta annotation. We should > find a stable one to replace it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3681) Replace RateLimiter with stable one
[ https://issues.apache.org/jira/browse/IOTDB-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564741#comment-17564741 ] Quan Siyi commented on IOTDB-3681: -- https://apache-iotdb.feishu.cn/docx/doxcnXO0pWemPa2UTG2IuMyifkG > Replace RateLimiter with stable one > --- > > Key: IOTDB-3681 > URL: https://issues.apache.org/jira/browse/IOTDB-3681 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Liuxuxin >Assignee: Quan Siyi >Priority: Major > Fix For: master branch > > > IoTDB use com.google.common.util.concurrent.RateLimiter to limit the rate of > compaction IO, but RateLimiter is unstable with a @Beta annotation. We should > find a stable one to replace it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3524) [cluster]The operations recorded on the logs of the three nodes are inconsistent
[ https://issues.apache.org/jira/browse/IOTDB-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17561531#comment-17561531 ] Quan Siyi commented on IOTDB-3524: -- This is a 3C3D cluster with 3 replica using ratis consensus protocol. In order to reduce the number of rpc, the client in benchmark will cache the leader of the ratis group, and forward the inserting request directly to the leader. When browsing the logs in all the three nodes, I find the 23 and 24 have become ratis leader for a while in turn. So the node 23 and 24 have to parse, analyse, plan the inserting request, and that's why 23 and 24 have more logs then 22. > [cluster]The operations recorded on the logs of the three nodes are > inconsistent > > > Key: IOTDB-3524 > URL: https://issues.apache.org/jira/browse/IOTDB-3524 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Fix For: 0.14.0-SNAPSHOT > > Attachments: 20220617_170909.mp4, image-2022-06-17-17-15-23-588.png, > log_22.tar.gz, log_23.tar.gz, log_24.tar.gz > > > commit a920737417330cb4b9fdb3d146af1c952057858b > Author: Chen YZ <43774645+cpau...@users.noreply.github.com> > Date: Thu Jun 16 23:01:49 2022 +0800 > IOTDB-3515 load empty TsFile error message (#6307) > > Reproduce： > 1.Setup a cluster with 3C3D > 2.Start 3 BMs to insert data > BM - 》 IoTDB Node > 172.20.70.7 - 》 172.20.70.22 > 172.20.70.8 - 》 172.20.70.23 > 172.20.70.9 - 》 172.20.70.24 > The operations recorded on the logs of the three nodes are inconsistent. > PS: It looks like 1 node(22) forward the insert operation to the other two. > Refer to below picture and video > !image-2022-06-17-17-15-23-588.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-3688) Error in initializing consensus implementation by processing heartbeat
Title: Message Title Quan Siyi created an issue Apache IoTDB / IOTDB-3688 Error in initializing consensus implementation by processing heartbeat Issue Type: Bug Assignee: Quan Siyi Created: 28/Jun/22 16:03 Priority: Major Reporter: Quan Siyi Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Commented] (IOTDB-3658) The datanode failed to start but successfully joined the cluster
Title: Message Title Quan Siyi commented on IOTDB-3658 Re: The datanode failed to start but successfully joined the cluster https://github.com/apache/iotdb/pull/6475 Add Comment This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9)
[jira] [Created] (IOTDB-3665) Serialize consensus protocol type to system.properties
Quan Siyi created IOTDB-3665: Summary: Serialize consensus protocol type to system.properties Key: IOTDB-3665 URL: https://issues.apache.org/jira/browse/IOTDB-3665 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3535) [cluster]A cross-space compaction has occurred when using benchmark to test iotdb with operation "IS_OUT_OF_ORDER=false"
[ https://issues.apache.org/jira/browse/IOTDB-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558944#comment-17558944 ] Quan Siyi commented on IOTDB-3535: -- This issue is caused by consensus layer. When transferring leader, consensus layer will retry inserting the same tablet, and this causes unsequence inserting. This problem is caused by a mechanism instead of a bug. > [cluster]A cross-space compaction has occurred when using benchmark to test > iotdb with operation "IS_OUT_OF_ORDER=false" > > > Key: IOTDB-3535 > URL: https://issues.apache.org/jira/browse/IOTDB-3535 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: config.properties, image-2022-06-20-09-52-52-981.png, > iotdb-confignode.properties, iotdb-engine.properties, log-all-20220619.19.log > > > A cross-space compaction has occurred when using benchmark to test iotdb with > operation "IS_OUT_OF_ORDER=false" > commit 1309eef2b1fb27a8ce7c8911b9c71ee5a9ed7035 > Author: 刘威 <51618159+liu-wei-...@users.noreply.github.com> > Date: Sat Jun 18 08:50:37 2022 +0800 > [IOTDB-3388] Let MQTT Service could parse JSON array of existed format > (#6154) > > Reproduce steps： > # Setup a cluster with 3C3D > # Using 3BMs to insert data > BM - 》 IoTDB Node > 172.20.70.7 - 》 172.20.70.22 > 172.20.70.8 - 》 172.20.70.23 > 172.20.70.9 - 》 172.20.70.24 > > !image-2022-06-20-09-52-52-981.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3507) Performance comparison between standalone IoTDB and Cluster IoTDB with client/partition cache
Quan Siyi created IOTDB-3507: Summary: Performance comparison between standalone IoTDB and Cluster IoTDB with client/partition cache Key: IOTDB-3507 URL: https://issues.apache.org/jira/browse/IOTDB-3507 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Assigned] (IOTDB-3435) Interface of migrate regions on data node
[ https://issues.apache.org/jira/browse/IOTDB-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-3435: Assignee: Quan Siyi > Interface of migrate regions on data node > - > > Key: IOTDB-3435 > URL: https://issues.apache.org/jira/browse/IOTDB-3435 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3435) Interface of migrate regions on data node
Quan Siyi created IOTDB-3435: Summary: Interface of migrate regions on data node Key: IOTDB-3435 URL: https://issues.apache.org/jira/browse/IOTDB-3435 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3231) Datanode can not be connected when all confignodes have been stoped.
[ https://issues.apache.org/jira/browse/IOTDB-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539589#comment-17539589 ] Quan Siyi commented on IOTDB-3231: -- The phenomenon is not a problem. When we use client to connect to data node, data node needs to check the authority of the user info which is managed by config nodes. So when the config nodes are all down, data node cannot open a session for users. > Datanode can not be connected when all confignodes have been stoped. > > > Key: IOTDB-3231 > URL: https://issues.apache.org/jira/browse/IOTDB-3231 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-05-18-10-33-39-333.png > > > 复线方法： > 1.启动一个3C3D的集群 > 2.使用cli命令登录（成功） > 3.使用stop-confignode.sh停止所有confignode（正常） > 4.使用处理登录（报错Can not establish connection with jdbc:iotdb://Chaos-5:6667/ : > Internal error processing openSession. Host is Chaos-5, port is 6667. > ） > > !image-2022-05-18-10-33-39-333.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (IOTDB-3231) Datanode can not be connected when all confignodes have been stoped.
[ https://issues.apache.org/jira/browse/IOTDB-3231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17539591#comment-17539591 ] Quan Siyi commented on IOTDB-3231: -- The phenomenon is not a problem. When we use client to connect to data node, data node needs check the authority of the user info which is managed by config nodes. So when the config nodes are all down, data node cannot open a session for users. > Datanode can not be connected when all confignodes have been stoped. > > > Key: IOTDB-3231 > URL: https://issues.apache.org/jira/browse/IOTDB-3231 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-05-18-10-33-39-333.png > > > 复线方法： > 1.启动一个3C3D的集群 > 2.使用cli命令登录（成功） > 3.使用stop-confignode.sh停止所有confignode（正常） > 4.使用处理登录（报错Can not establish connection with jdbc:iotdb://Chaos-5:6667/ : > Internal error processing openSession. Host is Chaos-5, port is 6667. > ） > > !image-2022-05-18-10-33-39-333.png! -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3108) Optimize register process of datanode
Quan Siyi created IOTDB-3108: Summary: Optimize register process of datanode Key: IOTDB-3108 URL: https://issues.apache.org/jira/browse/IOTDB-3108 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3092) Region client
Quan Siyi created IOTDB-3092: Summary: Region client Key: IOTDB-3092 URL: https://issues.apache.org/jira/browse/IOTDB-3092 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-3091) Stop datanode
Quan Siyi created IOTDB-3091: Summary: Stop datanode Key: IOTDB-3091 URL: https://issues.apache.org/jira/browse/IOTDB-3091 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-2988) Cache leader in session
Quan Siyi created IOTDB-2988: Summary: Cache leader in session Key: IOTDB-2988 URL: https://issues.apache.org/jira/browse/IOTDB-2988 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-2986) Standalone datanode
Quan Siyi created IOTDB-2986: Summary: Standalone datanode Key: IOTDB-2986 URL: https://issues.apache.org/jira/browse/IOTDB-2986 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-2985) ConsensusClient and proxy interface
Quan Siyi created IOTDB-2985: Summary: ConsensusClient and proxy interface Key: IOTDB-2985 URL: https://issues.apache.org/jira/browse/IOTDB-2985 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (IOTDB-2967) New writing process of cluster
Quan Siyi created IOTDB-2967: Summary: New writing process of cluster Key: IOTDB-2967 URL: https://issues.apache.org/jira/browse/IOTDB-2967 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Reopened] (IOTDB-2876) Definition of common thrift struct
[ https://issues.apache.org/jira/browse/IOTDB-2876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reopened IOTDB-2876: -- > Definition of common thrift struct > -- > > Key: IOTDB-2876 > URL: https://issues.apache.org/jira/browse/IOTDB-2876 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 8h > Remaining Estimate: 8h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2876) Definition of common thrift struct
Quan Siyi created IOTDB-2876: Summary: Definition of common thrift struct Key: IOTDB-2876 URL: https://issues.apache.org/jira/browse/IOTDB-2876 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2871) Data node client to connect with config node
Quan Siyi created IOTDB-2871: Summary: Data node client to connect with config node Key: IOTDB-2871 URL: https://issues.apache.org/jira/browse/IOTDB-2871 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2870) UT/IT of writing in new cluster
[ https://issues.apache.org/jira/browse/IOTDB-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2870: Assignee: Quan Siyi > UT/IT of writing in new cluster > --- > > Key: IOTDB-2870 > URL: https://issues.apache.org/jira/browse/IOTDB-2870 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2870) UT/IT of writing in new cluster
Quan Siyi created IOTDB-2870: Summary: UT/IT of writing in new cluster Key: IOTDB-2870 URL: https://issues.apache.org/jira/browse/IOTDB-2870 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2863) serialize and deserialize of insert node
Quan Siyi created IOTDB-2863: Summary: serialize and deserialize of insert node Key: IOTDB-2863 URL: https://issues.apache.org/jira/browse/IOTDB-2863 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2827) Batch insert in new cluster
Quan Siyi created IOTDB-2827: Summary: Batch insert in new cluster Key: IOTDB-2827 URL: https://issues.apache.org/jira/browse/IOTDB-2827 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2805) Implementation of management service on datanode
Quan Siyi created IOTDB-2805: Summary: Implementation of management service on datanode Key: IOTDB-2805 URL: https://issues.apache.org/jira/browse/IOTDB-2805 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2804) DataNode integrates consensus layer
Quan Siyi created IOTDB-2804: Summary: DataNode integrates consensus layer Key: IOTDB-2804 URL: https://issues.apache.org/jira/browse/IOTDB-2804 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2761) Basic framework and interfaces of coordinator
Quan Siyi created IOTDB-2761: Summary: Basic framework and interfaces of coordinator Key: IOTDB-2761 URL: https://issues.apache.org/jira/browse/IOTDB-2761 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2761) Basic framework and interfaces of coordinator
[ https://issues.apache.org/jira/browse/IOTDB-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2761: Assignee: Quan Siyi > Basic framework and interfaces of coordinator > - > > Key: IOTDB-2761 > URL: https://issues.apache.org/jira/browse/IOTDB-2761 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2748) Implementing Coordinator
Quan Siyi created IOTDB-2748: Summary: Implementing Coordinator Key: IOTDB-2748 URL: https://issues.apache.org/jira/browse/IOTDB-2748 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2748) Implementing Coordinator
[ https://issues.apache.org/jira/browse/IOTDB-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2748: Assignee: Quan Siyi > Implementing Coordinator > > > Key: IOTDB-2748 > URL: https://issues.apache.org/jira/browse/IOTDB-2748 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2747) Implement a simple start-up process of DataNode using a rpc server
[ https://issues.apache.org/jira/browse/IOTDB-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2747: Assignee: Quan Siyi > Implement a simple start-up process of DataNode using a rpc server > -- > > Key: IOTDB-2747 > URL: https://issues.apache.org/jira/browse/IOTDB-2747 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > * Move the TSServiceImpl from cluster to server module. > * Start the DataNode using a rpc server on a specific port. > * Implement a simple start-up process > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2747) Implement a simple start-up process of DataNode using a rpc server
Quan Siyi created IOTDB-2747: Summary: Implement a simple start-up process of DataNode using a rpc server Key: IOTDB-2747 URL: https://issues.apache.org/jira/browse/IOTDB-2747 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi * Move the TSServiceImpl from cluster to server module. * Start the DataNode using a rpc server on a specific port. * Implement a simple start-up process -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-2572) Design of writing process in new cluster
[ https://issues.apache.org/jira/browse/IOTDB-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501962#comment-17501962 ] Quan Siyi commented on IOTDB-2572: -- Writing document needs to be added a roadmap and interface definitions. > Design of writing process in new cluster > > > Key: IOTDB-2572 > URL: https://issues.apache.org/jira/browse/IOTDB-2572 > Project: Apache IoTDB > Issue Type: New Feature > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 24h > Remaining Estimate: 24h > > see document > [here|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199531619] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2572) Design of writing process in new cluster
Quan Siyi created IOTDB-2572: Summary: Design of writing process in new cluster Key: IOTDB-2572 URL: https://issues.apache.org/jira/browse/IOTDB-2572 Project: Apache IoTDB Issue Type: New Feature Components: Core/Cluster Reporter: Quan Siyi see document [here|https://docs.qq.com/doc/DU0xoVmxlemV5Ukpo] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2572) Design of writing process in new cluster
[ https://issues.apache.org/jira/browse/IOTDB-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2572: Assignee: Quan Siyi > Design of writing process in new cluster > > > Key: IOTDB-2572 > URL: https://issues.apache.org/jira/browse/IOTDB-2572 > Project: Apache IoTDB > Issue Type: New Feature > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 80h > Remaining Estimate: 80h > > see document [here|https://docs.qq.com/doc/DU0xoVmxlemV5Ukpo] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-1967) Decline in writing performance in cluster when enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17476109#comment-17476109 ] Quan Siyi commented on IOTDB-1967: -- Confluence document：https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199531762 > Decline in writing performance in cluster when enabling time partition > -- > > Key: IOTDB-1967 > URL: https://issues.apache.org/jira/browse/IOTDB-1967 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Labels: 0.13.0 > Fix For: master branch, 0.12.2, 0.12.3 > > Attachments: flink test.zip > > > When enabling time partition(enablePartition is true), the writing > performance in cluster mode has a serious decline. > We execute a test using flink to write into the cluster. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2392) Memory control of raftlog in cluster
Quan Siyi created IOTDB-2392: Summary: Memory control of raftlog in cluster Key: IOTDB-2392 URL: https://issues.apache.org/jira/browse/IOTDB-2392 Project: Apache IoTDB Issue Type: Improvement Components: Core/Cluster Reporter: Quan Siyi Fix For: master branch -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2392) Memory control of raftlog in cluster
[ https://issues.apache.org/jira/browse/IOTDB-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2392: Assignee: Quan Siyi > Memory control of raftlog in cluster > - > > Key: IOTDB-2392 > URL: https://issues.apache.org/jira/browse/IOTDB-2392 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Fix For: master branch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-1967) Decline in writing performance in cluster when enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17467649#comment-17467649 ] Quan Siyi commented on IOTDB-1967: -- document： https://docs.qq.com/doc/DV3F6VXp6dEhLa2Ja > Decline in writing performance in cluster when enabling time partition > -- > > Key: IOTDB-1967 > URL: https://issues.apache.org/jira/browse/IOTDB-1967 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Fix For: master branch, 0.12.2, 0.12.3 > > Attachments: flink test.zip > > > When enabling time partition(enablePartition is true), the writing > performance in cluster mode has a serious decline. > We execute a test using flink to write into the cluster. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2227) Fix connection refused using session when users forget to set client ip
[ https://issues.apache.org/jira/browse/IOTDB-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2227: Assignee: Quan Siyi > Fix connection refused using session when users forget to set client ip > --- > > Key: IOTDB-2227 > URL: https://issues.apache.org/jira/browse/IOTDB-2227 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Fix For: master branch, 0.12.4 > > Original Estimate: 2h > Remaining Estimate: 2h > > Users only set internal ip in iotdb-cluster.properties, and they often forget > to set the rpc_address (public ip) in iotdb-engine.properties. So the rpc > address is the default value(0.0.0.0). Session will cache the 0.0.0.0 > endpoint when an insertPlan is redirected. > We want internal ip as private ip and rpc_address as public ip. So if the > rpc_address is not set, set it the same with internal ip. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2227) Fix connection refused using session when users forget to set client ip
Quan Siyi created IOTDB-2227: Summary: Fix connection refused using session when users forget to set client ip Key: IOTDB-2227 URL: https://issues.apache.org/jira/browse/IOTDB-2227 Project: Apache IoTDB Issue Type: Bug Components: Core/Cluster Reporter: Quan Siyi Fix For: master branch, 0.12.4 Users only set internal ip in iotdb-cluster.properties, and they often forget to set the rpc_address (public ip) in iotdb-engine.properties. So the rpc address is the default value(0.0.0.0). Session will cache the 0.0.0.0 endpoint when an insertPlan is redirected. We want internal ip as private ip and rpc_address as public ip. So if the rpc_address is not set, set it the same with internal ip. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2193) lock on logManager
[ https://issues.apache.org/jira/browse/IOTDB-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2193: Assignee: Quan Siyi > lock on logManager > -- > > Key: IOTDB-2193 > URL: https://issues.apache.org/jira/browse/IOTDB-2193 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Fix For: master branch, 0.12.4 > > Original Estimate: 16h > Remaining Estimate: 16h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2193) lock on logManager
Quan Siyi created IOTDB-2193: Summary: lock on logManager Key: IOTDB-2193 URL: https://issues.apache.org/jira/browse/IOTDB-2193 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2115) Document of sharding strategy in new cluster architecture
Quan Siyi created IOTDB-2115: Summary: Document of sharding strategy in new cluster architecture Key: IOTDB-2115 URL: https://issues.apache.org/jira/browse/IOTDB-2115 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Using 8h to investigate data sharding in TDengine, TimescaleDB, OpenTsDB, and using 8h to desigin the sharding strategy in the new IoTDB cluster architecture and finish the document. Please refer to [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195729383] . -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2115) Document of sharding strategy in new cluster architecture
[ https://issues.apache.org/jira/browse/IOTDB-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2115: Assignee: Quan Siyi > Document of sharding strategy in new cluster architecture > -- > > Key: IOTDB-2115 > URL: https://issues.apache.org/jira/browse/IOTDB-2115 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 16h > Remaining Estimate: 16h > > Using 8h to investigate data sharding in TDengine, TimescaleDB, OpenTsDB, and > using 8h to > desigin the sharding strategy in the new IoTDB cluster architecture and > finish the document. > Please refer to > [https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195729383] . -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2090) Design of new generation MPP cluster architecture
Quan Siyi created IOTDB-2090: Summary: Design of new generation MPP cluster architecture Key: IOTDB-2090 URL: https://issues.apache.org/jira/browse/IOTDB-2090 Project: Apache IoTDB Issue Type: New Feature Reporter: Quan Siyi Assignee: Quan Siyi [https://cwiki.apache.org/confluence/display/IOTDB/refactor-2021-MPP-decoupling] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2029) Cluster high availability test
[ https://issues.apache.org/jira/browse/IOTDB-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2029: Assignee: Quan Siyi > Cluster high availability test > -- > > Key: IOTDB-2029 > URL: https://issues.apache.org/jira/browse/IOTDB-2029 > Project: Apache IoTDB > Issue Type: Test >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 40h > Remaining Estimate: 40h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2029) Cluster high availability test
Quan Siyi created IOTDB-2029: Summary: Cluster high availability test Key: IOTDB-2029 URL: https://issues.apache.org/jira/browse/IOTDB-2029 Project: Apache IoTDB Issue Type: Test Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2019) Support auto create schema for inserting aligned timeseries in cluster mode
[ https://issues.apache.org/jira/browse/IOTDB-2019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2019: Assignee: Quan Siyi > Support auto create schema for inserting aligned timeseries in cluster mode > --- > > Key: IOTDB-2019 > URL: https://issues.apache.org/jira/browse/IOTDB-2019 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 8h > Remaining Estimate: 8h > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2019) Support auto create schema for inserting aligned timeseries in cluster mode
Quan Siyi created IOTDB-2019: Summary: Support auto create schema for inserting aligned timeseries in cluster mode Key: IOTDB-2019 URL: https://issues.apache.org/jira/browse/IOTDB-2019 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-2007) Cluster architecture document
[ https://issues.apache.org/jira/browse/IOTDB-2007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-2007: Assignee: Quan Siyi > Cluster architecture document > -- > > Key: IOTDB-2007 > URL: https://issues.apache.org/jira/browse/IOTDB-2007 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Original Estimate: 36h > Remaining Estimate: 36h > > We need to write a cluster architecture document so that others can easily > understand the distributed implementation of IOTDB. The online documentation > is available [here|https://docs.qq.com/doc/DZWVpVmRIeEFPRUll] and will be > placed in confluence after the end of this issue. > > Needs to finish the following parts: > * Data model > * Data partition > * Data assignment > * Data consistency > * Data replication > * Data writing > * Cluster scalability > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-2007) Cluster architecture document
Quan Siyi created IOTDB-2007: Summary: Cluster architecture document Key: IOTDB-2007 URL: https://issues.apache.org/jira/browse/IOTDB-2007 Project: Apache IoTDB Issue Type: Improvement Components: Core/Cluster Reporter: Quan Siyi We need to write a cluster architecture document so that others can easily understand the distributed implementation of IOTDB. The online documentation is available [here|https://docs.qq.com/doc/DZWVpVmRIeEFPRUll] and will be placed in confluence after the end of this issue. Needs to finish the following parts: * Data model * Data partition * Data assignment * Data consistency * Data replication * Data writing * Cluster scalability -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (IOTDB-1967) Decline in writing performance in cluster when enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-1967: Assignee: Xinyu Tan > Decline in writing performance in cluster when enabling time partition > -- > > Key: IOTDB-1967 > URL: https://issues.apache.org/jira/browse/IOTDB-1967 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/Cluster >Reporter: Quan Siyi >Assignee: Xinyu Tan >Priority: Major > Fix For: master branch, 0.12.2, 0.12.3 > > Attachments: flink test.zip > > > When enabling time partition(enablePartition is true), the writing > performance in cluster mode has a serious decline. > We execute a test using flink to write into the cluster. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (IOTDB-1967) Decline in writing performance in cluster when enabling time partition
Quan Siyi created IOTDB-1967: Summary: Decline in writing performance in cluster when enabling time partition Key: IOTDB-1967 URL: https://issues.apache.org/jira/browse/IOTDB-1967 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi When enabling time partition(enablePartition is true), the writing performance in cluster mode has a serious decline. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (IOTDB-1847) Error writing data to cluster(n nodes 1 copy) while enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440478#comment-17440478 ] Quan Siyi commented on IOTDB-1847: -- The log reveals that there exists internal error when pulling timeseries schema. In fact, when the time series haven't been created, the node which is been pulled schema will throw a "path not exist" exception. But then the time series will be created automatically, so the data can be written to the cluster normally. > Error writing data to cluster(n nodes 1 copy) while enabling time partition > --- > > Key: IOTDB-1847 > URL: https://issues.apache.org/jira/browse/IOTDB-1847 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.2 >Reporter: Xinyu Tan >Assignee: Quan Siyi >Priority: Major > Attachments: logs.rar > > Original Estimate: 24h > Remaining Estimate: 24h > > Tests found that writing data to the cluster(n nodes 1 copy) failed when time > partitioning was enabled. There seems to be a problem with pulling time > series metadata, the log is attached, this bug needs to be fixed. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (IOTDB-1847) Error writing data to cluster(n nodes 1 copy) while enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reopened IOTDB-1847: -- > Error writing data to cluster(n nodes 1 copy) while enabling time partition > --- > > Key: IOTDB-1847 > URL: https://issues.apache.org/jira/browse/IOTDB-1847 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.2 >Reporter: Xinyu Tan >Assignee: Quan Siyi >Priority: Major > Attachments: logs.rar > > Original Estimate: 24h > Remaining Estimate: 24h > > Tests found that writing data to the cluster(n nodes 1 copy) failed when time > partitioning was enabled. There seems to be a problem with pulling time > series metadata, the log is attached, this bug needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-1847) Error writing data to cluster(n nodes 1 copy) while enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434417#comment-17434417 ] Quan Siyi commented on IOTDB-1847: -- I looked through the log, and I found out that all errors are from a single node. I guess such error are caused by some special situations instead of logical errors in the process of pulling schema. > Error writing data to cluster(n nodes 1 copy) while enabling time partition > --- > > Key: IOTDB-1847 > URL: https://issues.apache.org/jira/browse/IOTDB-1847 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.2 >Reporter: Xinyu Tan >Assignee: Quan Siyi >Priority: Major > Attachments: logs.rar > > Original Estimate: 24h > Remaining Estimate: 24h > > Tests found that writing data to the cluster(n nodes 1 copy) failed when time > partitioning was enabled. There seems to be a problem with pulling time > series metadata, the log is attached, this bug needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-1847) Error writing data to cluster(n nodes 1 copy) while enabling time partition
[ https://issues.apache.org/jira/browse/IOTDB-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434414#comment-17434414 ] Quan Siyi commented on IOTDB-1847: -- Using the method Session.InsertRecords() to write data to cluster(master branch, rel/0.12 branch, and 0.12.2 release version). I write data of different time partitions (enablePartition = True), but I can not recurrence the problem on all these versions. > Error writing data to cluster(n nodes 1 copy) while enabling time partition > --- > > Key: IOTDB-1847 > URL: https://issues.apache.org/jira/browse/IOTDB-1847 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.12.2 >Reporter: Xinyu Tan >Assignee: Quan Siyi >Priority: Major > Attachments: logs.rar > > Original Estimate: 24h > Remaining Estimate: 24h > > Tests found that writing data to the cluster(n nodes 1 copy) failed when time > partitioning was enabled. There seems to be a problem with pulling time > series metadata, the log is attached, this bug needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-1656) Design and document of look-up partition table
[ https://issues.apache.org/jira/browse/IOTDB-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424274#comment-17424274 ] Quan Siyi commented on IOTDB-1656: -- Investigate habse and tidb > Design and document of look-up partition table > -- > > Key: IOTDB-1656 > URL: https://issues.apache.org/jira/browse/IOTDB-1656 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Hai Liu >Assignee: Quan Siyi >Priority: Major > Original Estimate: 16h > Remaining Estimate: 16h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IOTDB-1656) Design and document of look-up partition table
[ https://issues.apache.org/jira/browse/IOTDB-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424273#comment-17424273 ] Quan Siyi commented on IOTDB-1656: -- Wait hongyin and sicheng to look through codes of IoTDB cluster > Design and document of look-up partition table > -- > > Key: IOTDB-1656 > URL: https://issues.apache.org/jira/browse/IOTDB-1656 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Hai Liu >Assignee: Quan Siyi >Priority: Major > Original Estimate: 16h > Remaining Estimate: 16h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IOTDB-1779) The network of one node is unreachable，write blocking problem for three node three replica cluster
[
https://issues.apache.org/jira/browse/IOTDB-1779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quan Siyi reassigned IOTDB-1779:
Assignee: Quan Siyi
> The network of one node is unreachable，write blocking problem for three node
> three replica cluster
> --
>
> Key: IOTDB-1779
> URL: https://issues.apache.org/jira/browse/IOTDB-1779
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/Cluster
>Reporter: 刘珍
>Assignee: Quan Siyi
>Priority: Major
> Attachments: 3节点3副本_拔网线_测试结果.docx, image-2021-09-29-15-49-33-180.png,
> image-2021-09-29-15-49-44-405.png, image-2021-09-29-15-51-24-108.png,
> 拔网线_v12_config.properties
>
>
> master 87e1ae444523c07b66b7bde1ec63d6b1924e3fca
> Three node three replica cluster，benchmark connects a node to write
> sequential data，remove the network cable from one node，Iotdb has the
> following problems：
> （3节点3副本集群，benchmark连接其中1个节点写入顺序数据，拔掉其他节点的网线火停止其他节点的网络服务--每次操作只影响1个节点，iotdb有如下几个问题）：
> 1.Symptom of pulling out a network cable（拔网线的表现）
> 问题1：Benchmark got exception (benchmark写入异常）：
> org.apache.iotdb.rpc.StatementExecutionException: {color:#DE350B}701: Request
> timed out.{color}
> org.apache.iotdb.rpc.StatementExecutionException: 702:{color:#DE350B}* Leader
> cannot be found.*{color}
> cli 执行insert ：
>
> 问题2：cli执行select耗时120秒：
> !image-2021-09-29-15-49-33-180.png!
> 问题3：node status 耗时120多秒
> !image-2021-09-29-15-49-44-405.png!
> 问题4：插上网线，网络恢复后
> Benchmark依然报错：
> org.apache.iotdb.rpc.IoTDBConnectionException: Fail to reconnect to server.
> Please check server status
> 停止网络服务（systemctl stop network）
> benchmark写入报同样的错误：
> org.apache.iotdb.rpc.IoTDBConnectionException: Fail to reconnect to server.
> Please check server status
> 但是命令行方式停止网络服务期间，cli 执行insert，select正常：
> insert into root.net.dev(col) values(now(),3);
> insert into root.net.dev(col) values(now(),4);
> insert into root.net.dev(col) values(now(),5);
>
> select * from root.net.dev;
> !image-2021-09-29-15-51-24-108.png!
> 详细结果见附件。
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IOTDB-1771) Look
Quan Siyi created IOTDB-1771: Summary: Look Key: IOTDB-1771 URL: https://issues.apache.org/jira/browse/IOTDB-1771 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-1770) Read and write operations in the cluster are not synchronized
Quan Siyi created IOTDB-1770: Summary: Read and write operations in the cluster are not synchronized Key: IOTDB-1770 URL: https://issues.apache.org/jira/browse/IOTDB-1770 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Fix For: master branch Some e2e tests fail when executing queries. Queries read error or incomplete data without synchronization. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IOTDB-1724) Cluster is stuck using benchmark insertRecords interface
[ https://issues.apache.org/jira/browse/IOTDB-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quan Siyi reassigned IOTDB-1724: Assignee: Quan Siyi > Cluster is stuck using benchmark insertRecords interface > > > Key: IOTDB-1724 > URL: https://issues.apache.org/jira/browse/IOTDB-1724 > Project: Apache IoTDB > Issue Type: Bug >Reporter: Quan Siyi >Assignee: Quan Siyi >Priority: Major > Fix For: 0.13.0, 0.12.3 > > Attachments: config.properties > > > When using benchmark to insert records into IoTDB cluster, the cluster will > be stuck, and schema of time series can't be automatically created. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-1724) Cluster is stuck using benchmark insertRecords interface
Quan Siyi created IOTDB-1724: Summary: Cluster is stuck using benchmark insertRecords interface Key: IOTDB-1724 URL: https://issues.apache.org/jira/browse/IOTDB-1724 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Fix For: 0.13.0, 0.12.3 Attachments: config.properties When using benchmark to insert records into IoTDB cluster, the cluster will be stuck, and schema of time series can't be automatically created. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-1589) Connection reset when using benchmark
Quan Siyi created IOTDB-1589: Summary: Connection reset when using benchmark Key: IOTDB-1589 URL: https://issues.apache.org/jira/browse/IOTDB-1589 Project: Apache IoTDB Issue Type: Bug Reporter: Quan Siyi Fix For: 0.12.1 some logs: 2021-08-25 17:20:32,509 [pool-8-IoTDB-RPC-Client-2] ERROR o.a.t.s.TThreadPoolServer$WorkerProcess:258 - Thrift Error occurred during processing of message. org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:178) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) at org.apache.iotdb.rpc.TElasticFramedTransport.readFrame(TElasticFramedTransport.java:112) at org.apache.iotdb.rpc.TElasticFramedTransport.read(TElasticFramedTransport.java:107) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:463) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:361) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:244) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:210) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:176) ... 12 common frames omitted -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IOTDB-1576) Update cluster setup document
Quan Siyi created IOTDB-1576: Summary: Update cluster setup document Key: IOTDB-1576 URL: https://issues.apache.org/jira/browse/IOTDB-1576 Project: Apache IoTDB Issue Type: Improvement Reporter: Quan Siyi -- This message was sent by Atlassian Jira (v8.3.4#803005)
