[jira] [Commented] (IOTDB-3920) [ 0.13.1 ] stop-server.sh , iotdb process does not exit
[
https://issues.apache.org/jira/browse/IOTDB-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573000#comment-17573000
]
yanze chen commented on IOTDB-3920:
---
as same as https://issues.apache.org/jira/browse/IOTDB-3813
> [ 0.13.1 ] stop-server.sh , iotdb process does not exit
> ---
>
> Key: IOTDB-3920
> URL: https://issues.apache.org/jira/browse/IOTDB-3920
> Project: Apache IoTDB
> Issue Type: Bug
>Affects Versions: 0.13.1-SNAPSHOT
>Reporter: 刘珍
>Assignee: yanze chen
>Priority: Major
> Attachments: insert_is_overflow.config-2.properties, log_all.log
>
>
> rel/0.13 23856d20a22ae3c3260d7b318151307a3f277bd9
> {color:#DE350B}对齐序列+ template{color} ，stop-server.sh，iotdb进程不退出：
> 1. benchmark运行附件中的脚本：insert_is_overflow.config.properties
> 2. benchmark写入完成，调用脚本停止数据库：
> 先执行flush
> start-cli.sh -e "flush"
> 再调用stop-server.sh，sleep 10秒，如果iotdb进程还存在，就kill -9 ：
> stop-server.sh
> sleep 10
> iotdb_pid=`jps|grep IoTDB |awk '
> {print $1}
> '`
> if [ "${iotdb_pid}x" = "x" ];then
> echo "stop iotdb server successed."
> else
> echo "stop iotdb server failed."
> echo "stop iotdb server failed." >> ${iotdb_dir}/logs/stop_iotdb.log
> kill -9 ${iotdb_pid}
> fi
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-3989) Server thread does not release
[ https://issues.apache.org/jira/browse/IOTDB-3989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572998#comment-17572998 ] yanze chen commented on IOTDB-3989: --- It has been verified that the thread count of IOTDB does not remain at a high level after closing the SessionPress program. During the execution of the SessionPress program, IOTDB creates threads for each session. When the SessionPress program is stopped, the resources are immediately released and the threads transfer to the TIMED_WAITING state( as shown in the attachment `rpc_client_thread_state.png` ), and by default these idle threads are released after 60 seconds. `IoTDBConfig.java` has a `thriftServerAwaitTimeForStopService` parameter which will be used when the Thrift service is created and eventually as the keepAliveTime parameter in the ThreadPoolExecutor. The release time of idle threads can be controlled by adjusting the thriftServerAwaitTimeForStopService. For example, if you set thriftServerAwaitTimeForStopService to 5, then 5 seconds after the SessionPress application exits, the idle threads will be released and a drop in the number of threads can be observed. > Server thread does not release > -- > > Key: IOTDB-3989 > URL: https://issues.apache.org/jira/browse/IOTDB-3989 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: 刘爽 >Assignee: yanze chen >Priority: Major > Attachments: SessionPress.java, image-2022-07-28-15-11-45-684.png, > image-2022-07-28-15-12-12-341.png, rpc_client_thread_state.png > > > (1) Run SessionPress > (2) Kill the SessionPress > The server thread does not release. > > The client stops abnormally，the Server thread does not release > !image-2022-07-28-15-12-12-341.png! > !image-2022-07-28-15-11-45-684.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3917) Restart keeps reporting errors，Failure to start when enabling Session Timeout
[ https://issues.apache.org/jira/browse/IOTDB-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572991#comment-17572991 ] Jialin Qiao commented on IOTDB-3917: Need reproduce and detailed description > Restart keeps reporting errors，Failure to start when enabling Session Timeout > - > > Key: IOTDB-3917 > URL: https://issues.apache.org/jira/browse/IOTDB-3917 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: zhenqqiang >Priority: Major > Attachments: image-2022-07-21-14-05-25-651.png, > image-2022-07-21-14-07-16-578.png > > > > [^image-2022-07-21-14-05-25-651.png] > !image-2022-07-21-14-05-25-651.png!!image-2022-07-21-14-07-16-578.png! > [^image-2022-07-21-14-07-16-578.png] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-3963) master、slave：Failed to insert data synchronously
[ https://issues.apache.org/jira/browse/IOTDB-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3963: -- Assignee: Gaofei Cao > master、slave：Failed to insert data synchronously > > > Key: IOTDB-3963 > URL: https://issues.apache.org/jira/browse/IOTDB-3963 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: zhenqqiang >Assignee: Gaofei Cao >Priority: Major > Attachments: log-error-20220722.7.log > > > 2022-07-22 17:44:48,501 [Thread-7] ERROR o.a.i.d.d.OperationSyncConsumer:76 - > OperationSyncConsumer can't transmit > org.apache.iotdb.rpc.StatementExecutionException: 304: Path > [root.test.g_test2.d_test4946] does not exist > at org.apache.iotdb.rpc.RpcUtils.verifySuccess(RpcUtils.java:94) > at > org.apache.iotdb.rpc.RpcUtils.verifySuccessWithRedirection(RpcUtils.java:115) > at > org.apache.iotdb.session.SessionConnection.executeOperationSync(SessionConnection.java:988) > at > org.apache.iotdb.session.Session.operationSyncTransmit(Session.java:2484) > at > org.apache.iotdb.session.pool.SessionPool.operationSyncTransmit(SessionPool.java:2305) > at > org.apache.iotdb.db.doublelive.OperationSyncConsumer.run(OperationSyncConsumer.java:69) > at java.lang.Thread.run(Thread.java:748) > 2022-07-22 17:44:48,503 [Thread-9] ERROR o.a.i.d.d.OperationSyncConsumer:76 - > OperationSyncConsumer can't transmit > org.apache.iotdb.rpc.StatementExecutionException: 304: Path > [root.test.g_test1.d_test953] does not exist > at org.apache.iotdb.rpc.RpcUtils.verifySuccess(RpcUtils.java:94) > at > org.apache.iotdb.rpc.RpcUtils.verifySuccessWithRedirection(RpcUtils.java:115) > at > org.apache.iotdb.session.SessionConnection.executeOperationSync(SessionConnection.java:988) > at > org.apache.iotdb.session.Session.operationSyncTransmit(Session.java:2484) > at > org.apache.iotdb.session.pool.SessionPool.operationSyncTransmit(SessionPool.java:2305) > at > org.apache.iotdb.db.doublelive.OperationSyncConsumer.run(OperationSyncConsumer.java:69) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4003) Remove duplicated dependency defination in pom.xml
Bo Zhang created IOTDB-4003: --- Summary: Remove duplicated dependency defination in pom.xml Key: IOTDB-4003 URL: https://issues.apache.org/jira/browse/IOTDB-4003 Project: Apache IoTDB Issue Type: Improvement Reporter: Bo Zhang Remove duplicated dependency defination in pom.xml files. == pom.xml ==> 2 dependency definations for jackson-databind library-udf/pom.xml ==> 2 dependency definations for iotdb-server -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4003) Remove duplicated dependency defination in pom.xml
[ https://issues.apache.org/jira/browse/IOTDB-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Zhang reassigned IOTDB-4003: --- Assignee: Bo Zhang > Remove duplicated dependency defination in pom.xml > -- > > Key: IOTDB-4003 > URL: https://issues.apache.org/jira/browse/IOTDB-4003 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Bo Zhang >Assignee: Bo Zhang >Priority: Minor > > Remove duplicated dependency defination in pom.xml files. > == > pom.xml ==> 2 dependency definations for jackson-databind > library-udf/pom.xml ==> 2 dependency definations for iotdb-server -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3971) Improve the process of writing chunks in compaction for aligned series
[
https://issues.apache.org/jira/browse/IOTDB-3971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572948#comment-17572948
]
Quan Siyi commented on IOTDB-3971:
--
I compare the performance of cross space copmaction and unseq inner space
compaction for aligned timeseries among three commits.
b309f71b959044cda2c5c37ca96bf89c69fbd640: {*}{*}Row-based data block +
row-based data process
70772cb8684315814959974811440f746e5c19c3: Column-based data block + row-based
data process
def138b5909e5df76440836719b62ebc4b48daeb: Column-based data block +
column-based data process
21 sequence tsfile and 21 unsequence tsfiles with one storage group, 100
devices, 300 sensors, and average 235470 points on each sensor.
unsequence inner space copmaction:
b3: 3.4932 Mb/s
70: 2.3602Mb/s
de: 4.4297Mb/s
cross space compaction:
b3: 1.9377 Mb/s
70: 2.1735Mb/s
de: 2.7183Mb/s
> Improve the process of writing chunks in compaction for aligned series
> --
>
> Key: IOTDB-3971
> URL: https://issues.apache.org/jira/browse/IOTDB-3971
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/Compaction
>Reporter: Quan Siyi
>Assignee: Quan Siyi
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.14.0-SNAPSHOT
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-3884) [Performance]Cross_space and unseq_space compaction are slower at aligned timeseries
[ https://issues.apache.org/jira/browse/IOTDB-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572941#comment-17572941 ] Quan Siyi commented on IOTDB-3884: -- The issue IOTDB-3971 aims to process data in a column-based format from TsBlock to accelerate cross space compaction and unseq inner space compaction > [Performance]Cross_space and unseq_space compaction are slower at aligned > timeseries > > > Key: IOTDB-3884 > URL: https://issues.apache.org/jira/browse/IOTDB-3884 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-07-19-14-17-03-521.png, > image-2022-07-19-14-29-01-699.png, image-2022-07-19-14-30-03-267.png, > screenshot-1.png > > > Please refer to below picture: > Test scenario:Build and Start an iotdb instance of each commit id in master > branch with one dataset, statistics the compaction time in logs. > !image-2022-07-19-14-30-03-267.png|width=928,height=362! > !image-2022-07-19-14-29-01-699.png|width=931,height=187! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3884) [Performance]Cross_space and unseq_space compaction are slower at aligned timeseries
[ https://issues.apache.org/jira/browse/IOTDB-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572940#comment-17572940 ] Quan Siyi commented on IOTDB-3884: -- When executing cross sapce compaction and inner space compaction, we need to decode and decompress chunk of timesereis in the compaction task. We use sereisReader to get chunk of tsfile and decompress and decode it, and return BatchData for further process. Now We the new util "scanSeriesUtil" to get TsBlock of timeseries. The only difference between of BatchData and TsBlock is that Batchdata is row-based and TsBlock is column-based. Therefore, if we continue processing data in a row-based format, we spend much time converting tsBlock to rows. > [Performance]Cross_space and unseq_space compaction are slower at aligned > timeseries > > > Key: IOTDB-3884 > URL: https://issues.apache.org/jira/browse/IOTDB-3884 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/Compaction >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Quan Siyi >Priority: Major > Attachments: image-2022-07-19-14-17-03-521.png, > image-2022-07-19-14-29-01-699.png, image-2022-07-19-14-30-03-267.png, > screenshot-1.png > > > Please refer to below picture: > Test scenario:Build and Start an iotdb instance of each commit id in master > branch with one dataset, statistics the compaction time in logs. > !image-2022-07-19-14-30-03-267.png|width=928,height=362! > !image-2022-07-19-14-29-01-699.png|width=931,height=187! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3940) Using bitmap in PageHeader to optimize ValuePageReader from row-base scanning to columnar-style scanning
[ https://issues.apache.org/jira/browse/IOTDB-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572931#comment-17572931 ] liaolanyu commented on IOTDB-3940: -- !image-2022-07-29-19-37-33-390.png! > Using bitmap in PageHeader to optimize ValuePageReader from row-base scanning > to columnar-style scanning > > > Key: IOTDB-3940 > URL: https://issues.apache.org/jira/browse/IOTDB-3940 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Yuan Tian >Assignee: liaolanyu >Priority: Major > Attachments: image-2022-07-29-19-37-33-390.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4002) Ratis frequently changes leader nodes
刘珍 created IOTDB-4002: - Summary: Ratis frequently changes leader nodes Key: IOTDB-4002 URL: https://issues.apache.org/jira/browse/IOTDB-4002 Project: Apache IoTDB Issue Type: Bug Components: mpp-cluster Affects Versions: 0.14.0-SNAPSHOT Reporter: 刘珍 Assignee: Song Ziyang Attachments: image-2022-07-29-17-26-58-690.png, ip3_config.properties, ip4_config.properties master_0727_975eaa5 问题描述 3副本3C7D ，2 benchmark连接2个datanode ，分别向自己的dataregion写入顺序数据。 监控机器的cpu发现，间隔cpu使用率接近100%，研发同事分析日志，ratis在频繁切主。 同一个节点的cpu和网络传输： !image-2022-07-29-17-26-58-690.png! 测试流程 1. 172.20.70.2/3/4/5/13/14/16 benchmark在15 2. 启动2个benchmark 配置文件见附件 3. 监控测试过程中的资源使用，及日志 资源使用 http://172.20.70.11/zabbix.php?action=dashboard.view=60 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-3914) [MultiLeaderConsensus] Generate out-of-order data, follower nodes have direct write
[ https://issues.apache.org/jira/browse/IOTDB-3914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3914: -- Assignee: Yongzao Dan (was: 张洪胤) > [MultiLeaderConsensus] Generate out-of-order data, follower nodes have direct > write > --- > > Key: IOTDB-3914 > URL: https://issues.apache.org/jira/browse/IOTDB-3914 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: 刘珍 >Assignee: Yongzao Dan >Priority: Major > Attachments: image-2022-07-21-11-34-23-458.png, > image-2022-07-21-11-34-37-254.png, ip3_config.properties, > ip4_config.properties, ip5_config.properties > > > MultiLeaderConsensus，写入的数据有乱序（客户端写入的是顺序数据），3副本中的2个follower节点都有直接写入。 > 复现流程 > 1. 192.168.130.1/2/3/4/5 > 3副本3C5D > schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus > data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus > schema_replication_factor=3 > data_replication_factor=3 > ConfigNode: > MAX_HEAP_SIZE="4G" > DataNode: > MAX_HEAP_SIZE="16G" > wal_buffer_size_in_byte=1048576 > max_waiting_time_when_insert_blocked=360 > query_timeout_threshold=3600 > 2. benchmark在ip2 > /home/benchmark/bm_0620_7ec96c1 > 启动3个benchmark，配置文件见附件 > ip1 > !image-2022-07-21-11-34-23-458.png! > 及对应的wal > !image-2022-07-21-11-34-37-254.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-3768) Support a new fill method FILL(NEXT)
[ https://issues.apache.org/jira/browse/IOTDB-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 任宇华 reassigned IOTDB-3768: -- Assignee: 任宇华 > Support a new fill method FILL(NEXT) > - > > Key: IOTDB-3768 > URL: https://issues.apache.org/jira/browse/IOTDB-3768 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: Yuan Tian >Assignee: 任宇华 >Priority: Major > > Now, we only support three fill methods, i.e. FILL(PREVIOUS), FILL(LINEAER), > FILL(VALUE). > FILL(NEXT) is also very common in fill case which use next non-null value to > fill current null value. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4001) Only unseq files reach a certain number can they be selected in cross compaction
周沛辰 created IOTDB-4001: -- Summary: Only unseq files reach a certain number can they be selected in cross compaction Key: IOTDB-4001 URL: https://issues.apache.org/jira/browse/IOTDB-4001 Project: Apache IoTDB Issue Type: Improvement Reporter: 周沛辰 Assignee: 周沛辰 When the number of unseq files reaches a certain number, they will be selected to participate in the cross space compaction, so as to avoid an unseq file being immediately selected to participate in the compaction, resulting in write amplification. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-3927) Fix incorrect cross compaction selector log
[ https://issues.apache.org/jira/browse/IOTDB-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572767#comment-17572767 ] 周沛辰 commented on IOTDB-3927: When no files can be selected in the cross space compaction, it is not because the memory exceeds the threshold, but the seq files or unseen files are being occupied by other compaction tasks. In addition, even if the memory exceeds the threshold, at least one group of unseq file and its overlapping seq files will be selected. > Fix incorrect cross compaction selector log > --- > > Key: IOTDB-3927 > URL: https://issues.apache.org/jira/browse/IOTDB-3927 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: 周沛辰 >Assignee: 周沛辰 >Priority: Major > Labels: pull-request-available > Fix For: master branch, 0.13.1 > > Attachments: image-2022-07-22-11-25-04-009.png > > > Since the first sēq file is being compacted, the source file cannot be > selected. It is not because the memory exceeds the threshold and the file > cannot be selected. > > !image-2022-07-22-11-25-04-009.png|width=2441,height=156! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4000) [MultiLeader] Add the control of wal size used by multiLeader Consensus
张洪胤 created IOTDB-4000: -- Summary: [MultiLeader] Add the control of wal size used by multiLeader Consensus Key: IOTDB-4000 URL: https://issues.apache.org/jira/browse/IOTDB-4000 Project: Apache IoTDB Issue Type: Improvement Reporter: 张洪胤 Assignee: 张洪胤 Add the control of wal size used by multiLeader Consensus -- This message was sent by Atlassian Jira (v8.20.10#820010)
