[jira] [Created] (IOTDB-4692) SchemaFile adaptation for cluster metadata validation
yanze chen created IOTDB-4692: - Summary: SchemaFile adaptation for cluster metadata validation Key: IOTDB-4692 URL: https://issues.apache.org/jira/browse/IOTDB-4692 Project: Apache IoTDB Issue Type: New Feature Reporter: yanze chen Assignee: yanze chen Support these two interface in SchemaRegionSchemaFileImpl: # checkMeasurementExistence() # fetchSchema() without template -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4691) How to check the time partition ID of Apache IoTDB?
soumyarani created IOTDB-4691: - Summary: How to check the time partition ID of Apache IoTDB? Key: IOTDB-4691 URL: https://issues.apache.org/jira/browse/IOTDB-4691 Project: Apache IoTDB Issue Type: Test Components: Connectors/Spark-IoTDB Reporter: soumyarani I found some of my original time partition in [Apache|[http://example.com|https://hkrtrainings.com/apache-spark-certification-training]] IoTDB not useful and therefore want to delete these time partition. I remember these time partition all have ID so I assume I need to type in these ID to delete, but I cannot remember the ID I used to give them. Any ways in IoTDB can I check these ID? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4690) Allow no cache for Netty threads
Song Ziyang created IOTDB-4690: -- Summary: Allow no cache for Netty threads Key: IOTDB-4690 URL: https://issues.apache.org/jira/browse/IOTDB-4690 Project: Apache IoTDB Issue Type: Sub-task Reporter: Song Ziyang Assignee: Song Ziyang [RATIS-1465. Use seperate channel for group heartbeat. (|https://github.com/apache/ratis/commit/208800a809630b27325ce039cf5312b69eac7bf9] [#561|https://github.com/apache/ratis/pull/561] [)|https://github.com/apache/ratis/commit/208800a809630b27325ce039cf5312b69eac7bf9] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4689) Use separate HB Channel
Song Ziyang created IOTDB-4689: -- Summary: Use separate HB Channel Key: IOTDB-4689 URL: https://issues.apache.org/jira/browse/IOTDB-4689 Project: Apache IoTDB Issue Type: Sub-task Reporter: Song Ziyang Assignee: Song Ziyang heartbeat won't timeout under heavy workload. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4688) Streaming MD5 replace blocking MD5
Song Ziyang created IOTDB-4688:
--
Summary: Streaming MD5 replace blocking MD5
Key: IOTDB-4688
URL: https://issues.apache.org/jira/browse/IOTDB-4688
Project: Apache IoTDB
Issue Type: Sub-task
Reporter: Song Ziyang
Currently in InstallSnapshot, statemachine has to pre-compute each file's md5
digest and provide it in {{{}FileInfo{}}}. When snapshot file is large, this
computing is time-consuming and may cause leader timeout.
In this PR I propose to move the computing task along within the
InstallSnapshot stream process. That is, for each SnapshotChunk, we can read
the chunk & compute its moving MD5 digest in the same time. This can enable us
to read file & compute file MD5 in one sequential scan and thus save IO costs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IOTDB-4688) Streaming MD5 replace blocking MD5
[
https://issues.apache.org/jira/browse/IOTDB-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Song Ziyang reassigned IOTDB-4688:
--
Assignee: Song Ziyang
> Streaming MD5 replace blocking MD5
> --
>
> Key: IOTDB-4688
> URL: https://issues.apache.org/jira/browse/IOTDB-4688
> Project: Apache IoTDB
> Issue Type: Sub-task
>Reporter: Song Ziyang
>Assignee: Song Ziyang
>Priority: Major
>
> Currently in InstallSnapshot, statemachine has to pre-compute each file's md5
> digest and provide it in {{{}FileInfo{}}}. When snapshot file is large, this
> computing is time-consuming and may cause leader timeout.
> In this PR I propose to move the computing task along within the
> InstallSnapshot stream process. That is, for each SnapshotChunk, we can read
> the chunk & compute its moving MD5 digest in the same time. This can enable
> us to read file & compute file MD5 in one sequential scan and thus save IO
> costs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IOTDB-4687) PRs related to Ratis 2.4.0 upgrade
Song Ziyang created IOTDB-4687: -- Summary: PRs related to Ratis 2.4.0 upgrade Key: IOTDB-4687 URL: https://issues.apache.org/jira/browse/IOTDB-4687 Project: Apache IoTDB Issue Type: Epic Reporter: Song Ziyang Assignee: Song Ziyang -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4537) [Trigger] Performance Test for Trigger
[ https://issues.apache.org/jira/browse/IOTDB-4537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619940#comment-17619940 ] liaolanyu commented on IOTDB-4537: -- Test doc: https://apache-iotdb.feishu.cn/docx/BtKCdgRV0oWTAMxHQvJcIC8ZnNh > [Trigger] Performance Test for Trigger > -- > > Key: IOTDB-4537 > URL: https://issues.apache.org/jira/browse/IOTDB-4537 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: liaolanyu >Assignee: liaolanyu >Priority: Major > > The output of this Jira will be a Performance Test Doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4670) [Trigger] Implement updateTriggerLocation interface
[ https://issues.apache.org/jira/browse/IOTDB-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619930#comment-17619930 ] liaolanyu commented on IOTDB-4670: -- This issue is done in pr: https://github.com/apache/iotdb/pull/7643 > [Trigger] Implement updateTriggerLocation interface > --- > > Key: IOTDB-4670 > URL: https://issues.apache.org/jira/browse/IOTDB-4670 > Project: Apache IoTDB > Issue Type: New Feature >Reporter: liaolanyu >Assignee: liaolanyu >Priority: Major > > Interface for transferring stateful trigger. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4686) SchemaReader for schema query operator-refactor traverser
[ https://issues.apache.org/jira/browse/IOTDB-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieqijun reassigned IOTDB-4686: --- Assignee: xieqijun > SchemaReader for schema query operator-refactor traverser > - > > Key: IOTDB-4686 > URL: https://issues.apache.org/jira/browse/IOTDB-4686 > Project: Apache IoTDB > Issue Type: Sub-task >Reporter: xieqijun >Assignee: xieqijun >Priority: Major > > Refactor traverser in MTree to support iteratable schema read -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4686) SchemaReader for schema query operator-refactor traverser
xieqijun created IOTDB-4686: --- Summary: SchemaReader for schema query operator-refactor traverser Key: IOTDB-4686 URL: https://issues.apache.org/jira/browse/IOTDB-4686 Project: Apache IoTDB Issue Type: Sub-task Reporter: xieqijun Refactor traverser in MTree to support iteratable schema read -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4685) Consensus heartbeat statistics
Yongzao Dan created IOTDB-4685: -- Summary: Consensus heartbeat statistics Key: IOTDB-4685 URL: https://issues.apache.org/jira/browse/IOTDB-4685 Project: Apache IoTDB Issue Type: Improvement Reporter: Yongzao Dan Assignee: Yongzao Dan Fix For: 0.14.0 Currently, all heartbeat samples are cached above the consensus layer of ConfigNode, which connotes that all load statistics will be discarded as long as the leader of ConfigNode is switched. Obviously, the following kinds of loadStatistics are worth to consensus: * The status of cluster Nodes and RegionGroups * The leader of RegionGroup * The loadScore of cluster Nodes and RegionGroups(will be implemented in the future) Therefore, I refactor the framework of cluster heartbeat related threads, and you can find the detailed desgin in the [LoadBalancing Framework|https://apache-iotdb.feishu.cn/docx/doxcnZ2tquzJKINQWi9nnYECPCb] doc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [iotdb-bin-resources] JackieTien97 merged pull request #43: Add pictures for cq
JackieTien97 merged PR #43: URL: https://github.com/apache/iotdb-bin-resources/pull/43 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@iotdb.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (IOTDB-4684) Devices with the same name but different alignment properties are compacted into the wrong alignment property
周沛辰 created IOTDB-4684: -- Summary: Devices with the same name but different alignment properties are compacted into the wrong alignment property Key: IOTDB-4684 URL: https://issues.apache.org/jira/browse/IOTDB-4684 Project: Apache IoTDB Issue Type: Bug Affects Versions: 0.13.0 Reporter: 周沛辰 Assignee: 周沛辰 *Description* After the nonAligned device is deleted, an aligned device with the same name is created, and it will be compacted into the nonAligned device after compaction. Similarly, after the aligned device is deleted, an nonAligned device with the same name is created, and it will be compacted into the aligned device after compaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (IOTDB-3671) [ confignode ] ERROR o.a.i.c.p.ProcedureExecutor:867 - ThreadGroup java.lang.ThreadGroup[name=ProcedureWorkerGroup,maxpri=10] contains running threads; null: See STDOUT
[ https://issues.apache.org/jira/browse/IOTDB-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] 刘珍 reopened IOTDB-3671: --- master_1017_0ebc42f ， 192.168.10.62 / 66 / 68机器 ip62 confignode还有这个ERROR, IP62开始是confignode的leader，缩容它，出现的这个ERROR 信息 > [ confignode ] ERROR o.a.i.c.p.ProcedureExecutor:867 - ThreadGroup > java.lang.ThreadGroup[name=ProcedureWorkerGroup,maxpri=10] contains running > threads; null: See STDOUT > > > Key: IOTDB-3671 > URL: https://issues.apache.org/jira/browse/IOTDB-3671 > Project: Apache IoTDB > Issue Type: Bug > Components: mpp-cluster >Affects Versions: 0.14.0-SNAPSHOT >Reporter: 刘珍 >Assignee: 伊丹翔 >Priority: Major > Labels: pull-request-available > Attachments: config.properties, ip13_confignode_log_all.zip > > > master_0627_0eb5010 > StandAloneConsensus, 3C3D > 启动顺序，bm连IP14 : > node1="172.20.70.13" > node2="172.20.70.16" > node3="172.20.70.14" > IP13 confignode error log : > 2022-06-28 04:47:51,229 [172.20.70.13_22278-server-thread2] ERROR > o.a.i.c.p.ProcedureExecutor:867 - ThreadGroup > java.lang.ThreadGroup[name=ProcedureWorkerGroup,maxpri=10] contains running > threads; null: See STDOUT > 完整日志见附件，benchmark配置见附件。 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4683) 100k sensors/dev ，write failed
刘珍 created IOTDB-4683:
-
Summary: 100k sensors/dev ，write failed
Key: IOTDB-4683
URL: https://issues.apache.org/jira/browse/IOTDB-4683
Project: Apache IoTDB
Issue Type: Bug
Components: mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Haonan Hou
Attachments: iotdb_13.conf, mpp.conf, mpp_ip76_dn_stack.out
m_1017_f33130b
1副本，1C1D
DEVICE_NUMBER=300
*SENSOR_NUMBER=10
CLIENT_NUMBER=1*
GROUP_NUMBER=3
BATCH_SIZE_PER_WRITE=1
写入超时时间设置为1小时，写入失败（{color:#DE350B}写入期间基本无CPU{color}）。
0.13 同样元数据配置，只是BATCH_SIZE_PER_WRITE=100
写入成功
测试环境
分布式1副本1C1D
1. 192.168.10.76 物理机 48CPU 384GB
ConfigNode
MAX_HEAP_SIZE="8G"
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
connection_timeout_ms=12
schema_region_ratis_request_timeout_ms = 120
data_region_ratis_request_timeout_ms = 120
schema_region_ratis_max_retry_attempts = 1
data_region_ratis_max_retry_attempts = 2
DataNode
MAX_HEAP_SIZE="256G"
MAX_DIRECT_MEMORY_SIZE="32G"
wal_buffer_size_in_byte=211424144
avg_series_point_number_threshold=100
max_waiting_time_when_insert_blocked=360
enable_seq_space_compaction=false
enable_unseq_space_compaction=false
enable_cross_space_compaction=false
query_timeout_threshold=3600
0.13 测试环境 （ commitID ：fe7cc90）
192.168.10.64 物理机 72CPU 256GB
MAX_HEAP_SIZE="192G"
MAX_DIRECT_MEMORY_SIZE="32G"
wal_buffer_size=211424144
avg_series_point_number_threshold=100
max_waiting_time_when_insert_blocked=360
enable_seq_space_compaction=false
enable_unseq_space_compaction=false
enable_cross_space_compaction=false
query_timeout_threshold=3600
bm配置文件见附件
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IOTDB-4682) [unload/remove] Support folders
[ https://issues.apache.org/jira/browse/IOTDB-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-4682: - Assignee: yusicheng (was: Haonan Hou) > [unload/remove] Support folders > --- > > Key: IOTDB-4682 > URL: https://issues.apache.org/jira/browse/IOTDB-4682 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/TsFile >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: yusicheng >Priority: Major > Attachments: image-2022-10-18-17-44-19-699.png, > image-2022-10-18-17-44-33-476.png > > > f33130b9fcaf35feef68538fed7f5a5d8724616e > Author: wanghui42 > <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> > Date: Mon Oct 17 17:00:09 2022 +0800 > IOTDB-4653 add time precision units when setting TTL (#7639) > > 期望 unload和remove能够也支持指定路径到文件夹级别，当前文件级别操作过多。 > !image-2022-10-18-17-44-33-476.png! > !image-2022-10-18-17-44-19-699.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4682) [unload/remove] Support folders
[ https://issues.apache.org/jira/browse/IOTDB-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengQingxin reassigned IOTDB-4682: -- Assignee: Haonan Hou (was: yusicheng) > [unload/remove] Support folders > --- > > Key: IOTDB-4682 > URL: https://issues.apache.org/jira/browse/IOTDB-4682 > Project: Apache IoTDB > Issue Type: Improvement > Components: Core/TsFile >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Haonan Hou >Priority: Major > Attachments: image-2022-10-18-17-44-19-699.png, > image-2022-10-18-17-44-33-476.png > > > f33130b9fcaf35feef68538fed7f5a5d8724616e > Author: wanghui42 > <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> > Date: Mon Oct 17 17:00:09 2022 +0800 > IOTDB-4653 add time precision units when setting TTL (#7639) > > 期望 unload和remove能够也支持指定路径到文件夹级别，当前文件级别操作过多。 > !image-2022-10-18-17-44-33-476.png! > !image-2022-10-18-17-44-19-699.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4682) [unload/remove] Support folders
FengQingxin created IOTDB-4682:
--
Summary: [unload/remove] Support folders
Key: IOTDB-4682
URL: https://issues.apache.org/jira/browse/IOTDB-4682
Project: Apache IoTDB
Issue Type: Improvement
Components: Core/TsFile
Affects Versions: 0.14.0-SNAPSHOT
Reporter: FengQingxin
Assignee: yusicheng
f33130b9fcaf35feef68538fed7f5a5d8724616e
Author: wanghui42
<105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]>
Date: Mon Oct 17 17:00:09 2022 +0800
IOTDB-4653 add time precision units when setting TTL (#7639)
2022-10-15 22:45:07,749 [pool-23-IoTDB-ClientRPC-Processor-2] INFO
o.a.i.d.s.t.i.ClientRPCServiceImpl:638 - Cost: {color:#de350b}1649732
{color}ms, sql is load '/data/load/data/sequence/root.test.g_0/0/0/'
There are 447 tsfiles in '/data/load/data/sequence/root.test.g_0/0/0/' ,their
size are almost 13M,and total size is 5.5G.
If I using two cli to load two folders like g_12 and g_13
_Cost: 3272731 ms, sql is load '/data/load/data/sequence/root.test.g_12/0/0/'_
_Cost: 3297271 ms, sql is load '/data/load/data/sequence/root.test.g_13/0/0/_
For more information about time consumption, see the attachement.
This performance is too low for user to use.Is there any way to accelerate that?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IOTDB-4679) [0.14][cluster] Unable to connect to iotdb in private-network using port-mapping
[ https://issues.apache.org/jira/browse/IOTDB-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619390#comment-17619390 ] ZhangZhengming commented on IOTDB-4679: --- !image-2022-10-18-17-32-01-496.png! > [0.14][cluster] Unable to connect to iotdb in private-network using > port-mapping > > > Key: IOTDB-4679 > URL: https://issues.apache.org/jira/browse/IOTDB-4679 > Project: Apache IoTDB > Issue Type: Bug >Affects Versions: 0.14.0-preview2 >Reporter: ZhangZhengming >Assignee: Haonan Hou >Priority: Major > Attachments: image-2022-10-18-16-17-43-132.png, > image-2022-10-18-16-18-32-594.png, image-2022-10-18-16-22-14-107.png, > image-2022-10-18-16-58-54-509.png, image-2022-10-18-16-59-48-604.png, > image-2022-10-18-17-09-48-843.png, image-2022-10-18-17-23-09-941.png, > image-2022-10-18-17-27-20-958.png, image-2022-10-18-17-32-01-496.png > > > I want to use port-mapping to expose rpc-port to the public network > I set rpc_address=0.0.0.0，internal_address=172.20.31.67(private_ip) > start iotdb-datanode.sh，the rpc_port listen to 172.20.31.67，instead of > 0.0.0.0，and this port cannot be connected by iotdb-client on the public > network > !image-2022-10-18-17-09-48-843.png|width=636,height=195! > I set rpc_address=111.xxx.xx.xxx (public-ip), The error is reported as > follows > !image-2022-10-18-17-27-20-958.png|width=1367,height=610! > What should I do? > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IOTDB-4680) [load] Lots of unnecessary error messages
[ https://issues.apache.org/jira/browse/IOTDB-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17619372#comment-17619372 ] yusicheng commented on IOTDB-4680: -- NoSuchFileException error has been fixed in https://github.com/apache/iotdb/pull/7354 > [load] Lots of unnecessary error messages > - > > Key: IOTDB-4680 > URL: https://issues.apache.org/jira/browse/IOTDB-4680 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/TsFile >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: yusicheng >Priority: Major > Labels: pull-request-available > Attachments: image-2022-10-18-16-50-22-344.png, > image-2022-10-18-16-51-29-557.png, log-all-20221015.0.log.gz, > log-all-20221016.0.log.gz, log-all-20221017.0.log.gz, log_all.log > > > f33130b9fcaf35feef68538fed7f5a5d8724616e > Author: wanghui42 > <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> > Date: Mon Oct 17 17:00:09 2022 +0800 > IOTDB-4653 add time precision units when setting TTL (#7639) > > When using the load function , lots error messages are displayed in the log. > 2022-10-18 00:00:16,101 > [org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileScheduler20221017_101107_00165_3.1.788-1873] > WARN o.a.i.c.utils.FileUtils:54 - > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51: > null > java.nio.file.NoSuchFileException: > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51 > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > at java.nio.file.Files.delete(Files.java:1126) > at > org.apache.iotdb.commons.utils.FileUtils.deleteDirectory(FileUtils.java:52) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.clearDir(LoadTsFileManager.java:175) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:171) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:159) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.lambda$writeToDataRegion$2(LoadTsFileManager.java:114) > at > java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.writeToDataRegion(LoadTsFileManager.java:113) > at > org.apache.iotdb.db.engine.StorageEngineV2.writeLoadTsFileNode(StorageEngineV2.java:711) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchLocally(LoadTsFileDispatcherImpl.java:164) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchOneInstance(LoadTsFileDispatcherImpl.java:113) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.lambda$dispatch$0(LoadTsFileDispatcherImpl.java:93) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > !image-2022-10-18-16-50-22-344.png|width=871,height=474! > > > There is also an error message with "%s" > !image-2022-10-18-16-51-29-557.png! > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4681) [load] Low performance
[
https://issues.apache.org/jira/browse/IOTDB-4681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haonan Hou reassigned IOTDB-4681:
-
Assignee: yusicheng (was: Haonan Hou)
> [load] Low performance
> --
>
> Key: IOTDB-4681
> URL: https://issues.apache.org/jira/browse/IOTDB-4681
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/TsFile
>Affects Versions: 0.14.0-SNAPSHOT
>Reporter: FengQingxin
>Assignee: yusicheng
>Priority: Major
> Attachments: log-all-20221015.0.log.gz, log-all-20221016.0.log.gz,
> log-all-20221017.0.log.gz, log_all.log
>
>
> f33130b9fcaf35feef68538fed7f5a5d8724616e
> Author: wanghui42
> <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]>
> Date: Mon Oct 17 17:00:09 2022 +0800
> IOTDB-4653 add time precision units when setting TTL (#7639)
> 2022-10-15 22:45:07,749 [pool-23-IoTDB-ClientRPC-Processor-2] INFO
> o.a.i.d.s.t.i.ClientRPCServiceImpl:638 - Cost: {color:#de350b}1649732
> {color}ms, sql is load '/data/load/data/sequence/root.test.g_0/0/0/'
>
> There are 447 tsfiles in '/data/load/data/sequence/root.test.g_0/0/0/'
> ,their size are almost 13M,and total size is 5.5G.
> If I using two cli to load two folders like g_12 and g_13
> _Cost: 3272731 ms, sql is load '/data/load/data/sequence/root.test.g_12/0/0/'_
> _Cost: 3297271 ms, sql is load '/data/load/data/sequence/root.test.g_13/0/0/_
> For more information about time consumption, see the attachement.
> This performance is too low for user to use.Is there any way to accelerate
> that?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IOTDB-3662) Add audit log
[ https://issues.apache.org/jira/browse/IOTDB-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jialin Qiao reassigned IOTDB-3662: -- Assignee: Zhijia Cao > Add audit log > - > > Key: IOTDB-3662 > URL: https://issues.apache.org/jira/browse/IOTDB-3662 > Project: Apache IoTDB > Issue Type: Improvement >Reporter: Jialin Qiao >Assignee: Zhijia Cao >Priority: Major > > Add audit log for insert/delete/query. > > For DDL: default is info level. > For DML: insert is debug level, delete and query are info level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4681) [load] Low performance
FengQingxin created IOTDB-4681: -- Summary: [load] Low performance Key: IOTDB-4681 URL: https://issues.apache.org/jira/browse/IOTDB-4681 Project: Apache IoTDB Issue Type: Bug Components: Core/TsFile Affects Versions: 0.14.0-SNAPSHOT Reporter: FengQingxin Assignee: Haonan Hou Attachments: log-all-20221015.0.log.gz, log-all-20221016.0.log.gz, log-all-20221017.0.log.gz, log_all.log f33130b9fcaf35feef68538fed7f5a5d8724616e Author: wanghui42 <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> Date: Mon Oct 17 17:00:09 2022 +0800 [IOTDB-4653] add time precision units when setting TTL (#7639) When using the load function , lots error messages are displayed in the log. 2022-10-18 00:00:16,101 [org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileScheduler20221017_101107_00165_3.1.788-1873] WARN o.a.i.c.utils.FileUtils:54 - /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51: null java.nio.file.NoSuchFileException: /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) at java.nio.file.Files.delete(Files.java:1126) at org.apache.iotdb.commons.utils.FileUtils.deleteDirectory(FileUtils.java:52) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.clearDir(LoadTsFileManager.java:175) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:171) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:159) at org.apache.iotdb.db.engine.load.LoadTsFileManager.lambda$writeToDataRegion$2(LoadTsFileManager.java:114) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.iotdb.db.engine.load.LoadTsFileManager.writeToDataRegion(LoadTsFileManager.java:113) at org.apache.iotdb.db.engine.StorageEngineV2.writeLoadTsFileNode(StorageEngineV2.java:711) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchLocally(LoadTsFileDispatcherImpl.java:164) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchOneInstance(LoadTsFileDispatcherImpl.java:113) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.lambda$dispatch$0(LoadTsFileDispatcherImpl.java:93) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4680) [load] Lots of unnecessary error messages
[ https://issues.apache.org/jira/browse/IOTDB-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haonan Hou reassigned IOTDB-4680: - Assignee: yusicheng (was: Haonan Hou) > [load] Lots of unnecessary error messages > - > > Key: IOTDB-4680 > URL: https://issues.apache.org/jira/browse/IOTDB-4680 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/TsFile >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: yusicheng >Priority: Major > Attachments: log-all-20221015.0.log.gz, log-all-20221016.0.log.gz, > log-all-20221017.0.log.gz, log_all.log > > > f33130b9fcaf35feef68538fed7f5a5d8724616e > Author: wanghui42 > <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> > Date: Mon Oct 17 17:00:09 2022 +0800 > [IOTDB-4653] add time precision units when setting TTL (#7639) > > When using the load function , lots error messages are displayed in the log. > 2022-10-18 00:00:16,101 > [org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileScheduler20221017_101107_00165_3.1.788-1873] > WARN o.a.i.c.utils.FileUtils:54 - > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51: > null > java.nio.file.NoSuchFileException: > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51 > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > at java.nio.file.Files.delete(Files.java:1126) > at > org.apache.iotdb.commons.utils.FileUtils.deleteDirectory(FileUtils.java:52) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.clearDir(LoadTsFileManager.java:175) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:171) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:159) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.lambda$writeToDataRegion$2(LoadTsFileManager.java:114) > at > java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.writeToDataRegion(LoadTsFileManager.java:113) > at > org.apache.iotdb.db.engine.StorageEngineV2.writeLoadTsFileNode(StorageEngineV2.java:711) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchLocally(LoadTsFileDispatcherImpl.java:164) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchOneInstance(LoadTsFileDispatcherImpl.java:113) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.lambda$dispatch$0(LoadTsFileDispatcherImpl.java:93) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IOTDB-4680) [load] Lots of unnecessary error messages
[ https://issues.apache.org/jira/browse/IOTDB-4680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] FengQingxin reassigned IOTDB-4680: -- Assignee: Haonan Hou > [load] Lots of unnecessary error messages > - > > Key: IOTDB-4680 > URL: https://issues.apache.org/jira/browse/IOTDB-4680 > Project: Apache IoTDB > Issue Type: Bug > Components: Core/TsFile >Affects Versions: 0.14.0-SNAPSHOT >Reporter: FengQingxin >Assignee: Haonan Hou >Priority: Major > Attachments: log-all-20221015.0.log.gz, log-all-20221016.0.log.gz, > log-all-20221017.0.log.gz, log_all.log > > > f33130b9fcaf35feef68538fed7f5a5d8724616e > Author: wanghui42 > <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> > Date: Mon Oct 17 17:00:09 2022 +0800 > [IOTDB-4653] add time precision units when setting TTL (#7639) > > When using the load function , lots error messages are displayed in the log. > 2022-10-18 00:00:16,101 > [org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileScheduler20221017_101107_00165_3.1.788-1873] > WARN o.a.i.c.utils.FileUtils:54 - > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51: > null > java.nio.file.NoSuchFileException: > /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51 > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > at java.nio.file.Files.delete(Files.java:1126) > at > org.apache.iotdb.commons.utils.FileUtils.deleteDirectory(FileUtils.java:52) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.clearDir(LoadTsFileManager.java:175) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:171) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:159) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.lambda$writeToDataRegion$2(LoadTsFileManager.java:114) > at > java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) > at > org.apache.iotdb.db.engine.load.LoadTsFileManager.writeToDataRegion(LoadTsFileManager.java:113) > at > org.apache.iotdb.db.engine.StorageEngineV2.writeLoadTsFileNode(StorageEngineV2.java:711) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchLocally(LoadTsFileDispatcherImpl.java:164) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchOneInstance(LoadTsFileDispatcherImpl.java:113) > at > org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.lambda$dispatch$0(LoadTsFileDispatcherImpl.java:93) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4680) [load] Lots of unnecessary error messages
FengQingxin created IOTDB-4680: -- Summary: [load] Lots of unnecessary error messages Key: IOTDB-4680 URL: https://issues.apache.org/jira/browse/IOTDB-4680 Project: Apache IoTDB Issue Type: Bug Components: Core/TsFile Affects Versions: 0.14.0-SNAPSHOT Reporter: FengQingxin Attachments: log-all-20221015.0.log.gz, log-all-20221016.0.log.gz, log-all-20221017.0.log.gz, log_all.log f33130b9fcaf35feef68538fed7f5a5d8724616e Author: wanghui42 <105700158+wanghui42@[users.noreply.github.com|http://users.noreply.github.com/]> Date: Mon Oct 17 17:00:09 2022 +0800 [IOTDB-4653] add time precision units when setting TTL (#7639) When using the load function , lots error messages are displayed in the log. 2022-10-18 00:00:16,101 [org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileScheduler20221017_101107_00165_3.1.788-1873] WARN o.a.i.c.utils.FileUtils:54 - /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51: null java.nio.file.NoSuchFileException: /data/cluster/first-rest-test/DataNode/apache-iotdb/datanode/sbin/../data/data/load/11ab173a-72fc-4686-8872-cf0909bb5c51 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) at java.nio.file.Files.delete(Files.java:1126) at org.apache.iotdb.commons.utils.FileUtils.deleteDirectory(FileUtils.java:52) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.clearDir(LoadTsFileManager.java:175) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:171) at org.apache.iotdb.db.engine.load.LoadTsFileManager$TsFileWriterManager.(LoadTsFileManager.java:159) at org.apache.iotdb.db.engine.load.LoadTsFileManager.lambda$writeToDataRegion$2(LoadTsFileManager.java:114) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at org.apache.iotdb.db.engine.load.LoadTsFileManager.writeToDataRegion(LoadTsFileManager.java:113) at org.apache.iotdb.db.engine.StorageEngineV2.writeLoadTsFileNode(StorageEngineV2.java:711) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchLocally(LoadTsFileDispatcherImpl.java:164) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.dispatchOneInstance(LoadTsFileDispatcherImpl.java:113) at org.apache.iotdb.db.mpp.plan.scheduler.load.LoadTsFileDispatcherImpl.lambda$dispatch$0(LoadTsFileDispatcherImpl.java:93) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IOTDB-4679) [0.14][cluster]iotdb-datanode.properties
ZhangZhengming created IOTDB-4679: - Summary: [0.14][cluster]iotdb-datanode.properties Key: IOTDB-4679 URL: https://issues.apache.org/jira/browse/IOTDB-4679 Project: Apache IoTDB Issue Type: Bug Reporter: ZhangZhengming Assignee: Haonan Hou Attachments: image-2022-10-18-16-17-43-132.png, image-2022-10-18-16-18-32-594.png 1. can't set rpc_address=0.0.0.0，it's not effective !image-2022-10-18-16-17-43-132.png|width=592,height=412! 2. can't set internal_address=0.0.0.0，register failed !image-2022-10-18-16-18-32-594.png|width=1302,height=577! -- This message was sent by Atlassian Jira (v8.20.10#820010)
