回复: [VOTE] Apache IoTDB 1.3.2 RC1 release

[冯 庆新]

Hello  all :
+1

The binary distribution:
version number in CLI [ok]
start in CentOS7, jdk11 [ok]
performance verification passed:  [ok]
### Server Configurations ###//
CPU=16
Memory=32G
Disk=1.8T HDD
Ethernet=1000Mbit
///
### IoTDB Configurations ###///
MAX_HEAP_SIZE="20G" in datanode-env.sh
///
### Client Mode ###
Insert Non-Aligned/Aligned timeseries with SESSION_BY_TABLET
CLIENT_NUMBER=10
GROUP_NUMBER=10
DEVICE_NUMBER=50
SENSOR_NUMBER=500
BATCH_SIZE_PER_WRITE=10
LOOP=86400
///

### Test Result ###
1C1D: Timeseries Num : 25,000；Loaded 21,600,000,000points；
Non-Aligned Timseries:
Throughput(points/s): 17,009,956.18  Cost Time(s): 1,269.84
Latency(ms): Avg 2.61, Min 0.67, MiddleAvg 0.83, Max 6,783.48
Aligned Timseries:
Throughput(points/s): 18,141,900.23  Cost Time(s): 1,190.61
Latency(ms): Avg 2.44, Min 0.79, MiddleAvg 0.99, Max 6,546.48
///


Avg: Average time cost of all ingestion operations. [ms]
MiddleAvg: Average time cost of ingestion operations without 5% head and tail. 
[ms]
Min: Min single operation time cost of all ingestion operations. [ms]
Max: Max single operation  time cost of all ingestion operations. [ms]
These results are tested with 
iot-benchmark(https://github.com/thulab/iot-benchmark)
That's all.
Please feel free to contact me if you have any questions
Thank you.
Best Regards!
Qingxin Feng

从 Windows 版邮件发送


发件人: Haonan Hou 
发送时间: Thursday, June 27, 2024 5:55:36 PM
收件人: dev@iotdb.apache.org 
主题: [VOTE] Apache IoTDB 1.3.2 RC1 release

Hi all,

Apache IoTDB 1.3.2 has been staged under [2] and it’s time to vote
on accepting it for release. All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.

Release tag: v1.3.2
Hash for the release tag: aa0ff4adf2fe00368b2145dd5f561d30df06a885

Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases. [3]

You can achieve the above by following [4].

[ ] +1 accept (indicate what you validated - e.g. performed the
non-RM items in [4])
[ ] -1 reject (explanation required)


[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1159
[2] https://dist.apache.org/repos/dist/dev/iotdb/1.3.2/rc1
[3] https://www.apache.org/dev/release.html#approving-a-release
[4] 
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release

Best,

Haonan Hou

回复: [VOTE] Apache IoTDB 1.3.0 RC1 release

[冯 庆新]

Hello  all :
+1

The binary distribution:
version number in CLI [ok]
start in CentOS7, jdk11 [ok]
performance verification [ok]
### Server Configurations ###//
CPU=16
Memory=32G
Disk=1.8T HDD
Ethernet=1000Mbit
///
### IoTDB Configurations ###///
MAX_HEAP_SIZE="20G" in datanode-env.sh
MAX_HEAP_SIZE="6G" in confignode-env.sh
schema_replication_factor=3
data_replication_factor=3
all_compaction=false
///
### Client Mode ###
Insert Non-Aligned/Aligned timeseries with SESSION_BY_TABLET
CLIENT_NUMBER=10
GROUP_NUMBER=10
DEVICE_NUMBER=(50 in 3C5D) (500 in 3C5D)
SENSOR_NUMBER=500
BATCH_SIZE_PER_WRITE=100
LOOP=1
///

### Test Result ###
3C3D: Timeseries Num : 25,000；Loaded 25,000,000,000points；
Non-Aligned Timseries:
Throughput(points/s): 15,371,100  Cost Time(s):1,678
Latency(ms): Avg 27.52, Min 3.83, MiddleAvg 5.04, Max 23,582.6
Aligned Timseries:
Throughput(points/s): 23,016,100 Cost Time(s):1,119
Latency(ms): Avg 17.92, Min 3.41, MiddleAvg 4.17, Max 15,849.5

3C5D:Timeseries Num : 250,000；Loaded 250,000,000,000points；
Non-Aligned Timseries:
Throughput(points/s):46,091,600  Cost Time(s):5,477
Latency(ms): Avg 9.84, Min 3.53, MiddleAvg 5.1, Max 5,409.47
Aligned Timseries:
Throughput(points/s):46,229,700  Cost Time(s):5,456
Latency(ms): Avg 9.5, Min 3.2, MiddleAvg 4.61, Max 5,389.18
///


Avg: Average time cost of all ingestion operations. [ms]
MiddleAvg: Average time cost of ingestion operations without 5% head and tail. 
[ms]
Min: Min single operation time cost of all ingestion operations. [ms]
Max: Max single operation  time cost of all ingestion operations. [ms]
There results are tested with 
iot-benchmark(https://github.com/thulab/iot-benchmark)
That’s all.
Please feel free to contact me if you have any questions
Thank you.
Best Reagrds!
Qingxin Feng


发件人: Haonan Hou
发送时间: 2023年12月27日 12:33
收件人: dev@iotdb.apache.org
主题: [VOTE] Apache IoTDB 1.3.0 RC1 release

Hi all,

Apache IoTDB 1.3.0 has been staged under [2] and it’s time to vote
on accepting it for release. All Maven artifacts are available under [1].
Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.

Release tag: v1.3.0
Hash for the release tag: fe787cac2ddd874b44482a80306d9d6ae333143c

Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases. [3]

You can achieve the above by following [4].

[ ] +1 accept (indicate what you validated - e.g. performed the
non-RM items in [4])
[ ] -1 reject (explanation required)


[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1154
[2] https://dist.apache.org/repos/dist/dev/iotdb/1.3.0/rc1
[3] https://www.apache.org/dev/release.html#approving-a-release
[4] 
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://github.com/apache/iotdb/blob/v1.3.0/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS

Best,

Haonan Hou

回复: [VOTE] Apache IoTDB 1.2.2 RC1 release

[冯 庆新]

Hello  all :
+1

The binary distribution:
version number in CLI [ok]
start in CentOS7, jdk8 [ok]
performance verification passed:  [ok]
### Server Configurations ###//
CPU=16
Memory=32G
Disk=1.8T HDD
Ethernet=1000Mbit
///
### IoTDB Configurations ###///
MAX_HEAP_SIZE="20G" in datanode-env.sh
MAX_HEAP_SIZE="6G" in confignode-env.sh
schema_replication_factor=3
data_replication_factor=3
all_compaction=false
///
### Client Mode ###
Insert Non-Aligned/Aligned timeseries with SESSION_BY_TABLET
CLIENT_NUMBER=10
GROUP_NUMBER=10
DEVICE_NUMBER=(50 in 3C5D) (500 in 3C5D)
SENSOR_NUMBER=500
BATCH_SIZE_PER_WRITE=100
LOOP=1
///

### Test Result ###
3C3D: Timeseries Num : 25,000；Loaded 25,000,000,000points；
Non-Aligned Timseries:
Throughput(points/s):16,276,600  Cost Time(s):1,573
Latency(ms): Avg 25.47, Min 3.77, MiddleAvg 4.78, Max 18,300.4
Aligned Timseries:
Throughput(points/s):23,380,500  Cost Time(s):1,103
Latency(ms): Avg 18.09, Min 3.55, MiddleAvg 4.47, Max 21,272

3C5D:Timeseries Num : 250,000；Loaded 250,000,000,000points；
Non-Aligned Timseries:
Throughput(points/s):40,000,800  Cost Time(s):5,396
Latency(ms): Avg 9.69, Min 3.47, MiddleAvg 5.08, Max 5,381.92
Aligned Timseries:
Throughput(points/s):48,026,300  Cost Time(s):5,256
Latency(ms): Avg 9.17, Min 3.09, MiddleAvg 4.47, Max 5,388.1
///


Avg: Average time cost of all ingestion operations. [ms]
MiddleAvg: Average time cost of ingestion operations without 5% head and tail. 
[ms]
Min: Min single operation time cost of all ingestion operations. [ms]
Max: Max single operation  time cost of all ingestion operations. [ms]
There results are tested with 
iot-benchmark(https://github.com/thulab/iot-benchmark)
That’s all.
Please feel free to contact me if you have any questions
Thank you.
Best Reagrds!
Qingxin Feng

发件人: Gaofei Cao
发送时间: 2023年10月13日 23:25
收件人: dev@iotdb.apache.org
主题: Re: [VOTE] Apache IoTDB 1.2.2 RC1 release

Hi,

+1 (binding)

The source release:
apache headers [ok]
signatures and hashes [ok]
LICENSE and NOTICE [ok]
no jar files [ok]
could compile from source: ./mvnw.sh clean install [ok]

The binary distribution:
version number in CLI [ok]
signatures and hashes [ok]
start in mac, jdk17 [ok]
statements executed successfully:  [ok]

CREATE DATABASE root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
insert into root.turbine.d1(timestamp,s0) values(1,1);
insert into root.turbine.d1(timestamp,s0) values(2,2);
insert into root.turbine.d1(timestamp,s0) values(3,3);
insert into root.turbine.d2(time,s0) values(now(),1);
insert into root.turbine.d2(time,s0) values(now(),2);
insert into root.turbine.d2(time,s0) values(now(),3);
show timeseries;
select ** from root.**;



Best,
-
Gaofei Cao

Jialin Qiao  于2023年10月12日周四 21:28写道：
>
> Hi,
>
> +1 (binding)
>
> The source release:
> apache headers [ok]
> signatures and hashes [ok]
> LICENSE and NOTICE [ok]
> no jar files [ok]
> could compile from source: ./mvnw.sh clean install [ok]
>
> The binary distribution:
> version number in CLI [ok]
> signatures and hashes [ok]
> start in mac, jdk8 [ok]
> statements executed successfully:  [ok]
>
> SET STORAGE GROUP TO root.turbine;
> CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
> insert into root.turbine.d1(timestamp,s0) values(1,1);
> insert into root.turbine.d1(timestamp,s0) values(2,2);
> select ** from root;
> select ** from root align by device;
>
> Thanks,
> ―
> Jialin Qiao
> Apache IoTDB PMC
>
> Yuan Tian  于2023年10月12日周四 20:52写道：
> >
> > Hi,
> >
> > +1 (binding)
> >
> > The source release:
> > apache headers [ok]
> > signatures and hashes [ok]
> > LICENSE and NOTICE [ok]
> > no jar files [ok]
> > could compile from source: ./mvnw.sh clean install [ok]
> >
> > The binary distribution:
> > version number in CLI [ok]
> > signatures and hashes [ok]
> > start in mac, jdk8 [ok]
> > statements executed successfully:  [ok]
> >
> > SET STORAGE GROUP TO root.turbine;
> > CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
> > insert into root.turbine.d1(timestamp,s0) values(1,1);
> > insert into root.turbine.d1(timestamp,s0) values(2,2);
> > insert into root.turbine.d1(timestamp,s0) values(3,3);
> > insert into root.turbine.d2(time,s0) values(now(),1);
> > insert into root.turbine.d2(time,s0) values(now(),2);
> > insert into root.turbine.d2(time,s0) values(now(),3);
> > select ** from root;
> > select 

回复: [DISCUSS] Incubating TsFile as an independent Apache project?

[冯 庆新]

Hi Steve Su

I think this is a great idea and I fully support it.
I believe Tsfile can generate greater influence and value.

Thanks
B.R.
Qingxin Feng


从 Windows 版邮件发送

发件人: Steve Yurong Su
发送时间: 2023年10月7日 0:53
收件人: dev@iotdb.apache.org
主题: [DISCUSS] Incubating TsFile as an independent Apache project?

Hi folks,

Recently, I've been thinking about something that has been on my mind:
I believe TsFile has the potential to develop as an independent Apache
project (note, not a subproject like apache-iotdb-tsfile), just like
ORC, Parquet, and Arrow. How about incubating TsFile as an independent
Apache project?

The main reason I came up with this idea is because I realized that
TsFile is not just the data file format of IoTDB, but it can also be a
standard format in the IoT field. It provides excellent support for
time series data, offering various encoding and compression methods.
It can be easily self-parsed and its statistical information enables
fast data querying by big data software. I believe that every IoT
software could benefit directly by supporting TsFile as a storage
format.

On the other hand, I hope to leverage the power of the community to
drive the development of the TsFile file format. Just imagine, TsFile
could have its own independent ecosystem: parsing tools in various
languages, connectors to various big data software, and so on. By
separating TsFile from IoTDB, we have the opportunity to attract more
community contributors and introduce more ideas, thus refining the
functionality and performance of TsFile.

What do you think about this idea?

In addition, how can TsFile, as a part of the current IoTDB project,
become an independent Apache project? Should it go through the Apache
Foundation incubation process again, or can it directly become a
top-level project? It may be too early to ask this question now, but I
would also like to hear your opinions.

Thanks,
Steve Su

回复: Re:[VOTE] New Repo: iotdb-client-csharp

[冯 庆新]

+1

发件人: Houliang Qi
发送时间: 2023年5月20日 19:20
收件人: dev@iotdb.apache.org
主题: Re:[VOTE] New Repo: iotdb-client-csharp

+1


And once the new repositories is created, perhaps it's best to contact the 
author(github id: eedalong) and see if they are willing to contribute code to 
iotdb-client-csharp





Thanks,
---
Houliang Qi
BONC, Ltd


 Replied Message 
| From | gitda...@163.com |
| Date | 05/20/2023 15:22 |
| To |  |
| Subject | Re:[VOTE] New Repo: iotdb-client-csharp |
+1
At 2023-05-20 14:57:56, "Jialin Qiao"  wrote:
Hi,

Currently, there are two Repositories of IoTDB Csharp client.

eedalong/Apache-IoTDB-Client-CSharp
eedalong/Apache-IoTDB-Client-CSharp-UseCase

I prefer to call for a vote for creating a new repo
iotdb-client-csharp for the csharp client.

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

The vote is open for the next 72 hours and passes if at least three +1
votes and more +1 votes than -1 votes.

Best,
―
Jialin Qiao
Apache IoTDB PMC

Re: Re：Remove UserGuide before 0.13 from website

[冯 庆新]

＋1

获取 Outlook for iOS

发件人: 张金瑞 <329920...@qq.com.INVALID>
发送时间: Thursday, February 23, 2023 7:11:00 PM
收件人: dev 
主题: Re：Remove UserGuide before 0.13 from website

+1



Thanks,
-
Jinrui
Apache IoTDB PMC



Original



From:"Jialin Qiao"< qiaojia...@apache.org ;

Date:2023/2/23 18:30

To:"dev"< dev@iotdb.apache.org ;

Subject:Remove UserGuide before 0.13 from website


Hi,

There are 8 versions UserGuide on our website[1], the old versions are
hardly used anymore.

I prefer to remove the history versions before 0.13 (not include).

[1] https://iotdb.apache.org/

Thanks,
―
Jialin Qiao
Apache IoTDB PMC

回复: [Discussion] Deprecate configuration parameter `max_deduplicated_path_num`

[冯 庆新]

Hi,

+ 1 for deprecate this parameter
But does the new version return the correct results by splitting the query into 
fewer series through memory control?
Or just reject the query to indicate that there is insufficient memory?

B.R.
Qingxin Feng

发件人: Minghui Liu
发送时间: 2023年2月15日 9:31
收件人: dev@iotdb.apache.org
主题: [Discussion] Deprecate configuration parameter `max_deduplicated_path_num`

In version 1.0, we have designed a finer query memory control
mechanism, which can ensure that the query will not be executed when
there are too many queried time series. Therefore, we no longer need
this parameter.

I plan to deprecate this parameter in the next version. What's your opinion?

Regards,
---
Minghui Liu
School of Software, Tsinghua University

刘明辉
清华大学 软件学院

Re: [VOTE] Create a repo for iotdb-backup-tool

[冯 庆新]

＋1

获取 Outlook for iOS

发件人: Xiangdong Huang 
发送时间: Wednesday, February 1, 2023 9:00:47 AM
收件人: dev@iotdb.apache.org 
主题: Re: [VOTE] Create a repo for iotdb-backup-tool

+1.
Though creating a new repo may reduce the focus, it will be helpful to
keep iotdb main repo concise and maintainable. And accepting codes
like iotdb-tool will be more convenient.

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院

Jialin Qiao  于2023年1月31日周二 21:49写道：
>
> Hi,
>
> Here is a tool for backup IoTDB data: 
> https://github.com/apache/iotdb/pull/7248
>
> As iotdb-backup-tool could be used independently, it could be a separate
> repository for users.
>
> So, I'd like to start a vote for creating a new Repo for iotdb-backup-tool.
>
> Please vote accordingly:
>
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove with the reason
>
> The vote is open for the next 72 hours and passes if at least three +1
> votes and more +1 votes than -1 votes.
>
> Thanks,
> ―
> Jialin Qiao
> Apache IoTDB PMC

回复: Add constraint to the length of database name

[冯 庆新]

Agree with  ‘add constraint to the length of database name’,but Can we choose a 
value greater than 64?

发件人: Jialin Qiao
发送时间: 2022年11月18日 16:15
收件人: dev@iotdb.apache.org
主题: Re: Add constraint to the length of database name

+1
―
Jialin Qiao
Apache IoTDB PMC

周钰坤  于2022年11月18日周五 15:44写道：
>
> Hi,
>
> We want to add constraint to the length of database name, as most
> popular database systems have such constraint as well, for example the
> length of database name in Mysql shall not exceed 64. Currently, the
> maximum length of database name, including "root.",  is *64* and it is
> immutable. Such constraint can help avoid some bugs of database and region
> management, since we use database names in the directory name, which shall
> not exceed the max name length defined by file system.
>
> best regards
> 
> Yukun Zhou, Tsinghua University

回复: [VOTE] Create a repo for TsFile-Viewer

[冯 庆新]

+1

发件人: Jialin Qiao
发送时间: 2022年11月7日 10:00
收件人: dev@iotdb.apache.org
主题: [VOTE] Create a repo for TsFile-Viewer

Hi,

Here is a Tool for viewing TsFile: https://github.com/apache/iotdb/pull/6995

As TsFile-Viewer could be used independently, it could be a separate
repository for users.

So, I'd like to start a vote for creating a new Repo for TsFile-Viewer.

Please vote accordingly:

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

The vote is open for the next 72 hours and passes if at least three +1
votes and more +1 votes than -1 votes.

Thanks,
―
Jialin Qiao
Apache IoTDB PMC

Re: About the standalone version

[冯 庆新]

Hi,
It sounds good.Maybe we can do an experiment to compare the performance 
differences between 1C1D and standalone.Perhaps 1C1D has a perfect performance.
B.R.
Qingxin Feng

获取 Outlook for iOS

发件人: Jialin Qiao 
发送时间: Sunday, October 30, 2022 10:29:19 AM
收件人: dev@iotdb.apache.org 
主题: About the standalone version

Hi,

In this new cluster version, we separate the IoTDB service to two
process: ConfigNode and DataNode, which could be deployed on multiple
machines and one machine.

So if users want to use a standalone IoTDB, they can deploy a 1C1D
architecture on one machine.

We are investigating if we need to support a one-process(combine
ConfigNode and DataNode) architecture for the standalone version.
which does not see much significant but introduces some learning
difficulties to users from the product complexity perspective.

In this case, I'd like to only release the ConfigNode and DataNode
Cluster, without another special standalone, and remove the
start-server.sh, start-new-server.sh.

Instead, we could introduce a start-all.sh, which start one ConfigNode
and one DataNode by default. And this could also be used to start the
whole cluster.

What do you think?

Thanks,
―
Jialin Qiao
Apache IoTDB PMC

回复: The changes of default write path in IoTDB metric reporter.

[冯 庆新]

Hi
SQLSTATE : 401: Error occurred while parsing SQL to physical plan: line 1:26 
mismatched input '' expecting {SYSTEM, '*', '**', STRING_LITERAL, 
INTEGER_LITERAL, ID, QUTOED_ID_IN_NODE_NAME}
 Is this change due to the following reasons?If so, then I recommend not doing 
it. I agree with Jialin, especially suggestion (2).
BR,
Qingxin Feng

发件人: Jialin Qiao
发送时间: 2022年10月28日 21:42
收件人: dev@iotdb.apache.org
主题: Re: The changes of default write path in IoTDB metric reporter.

+1 for an unified system path
Besides, some aspects need to be considered:

(1) Skip the system path when show timeseries or count timeseries to
avoid misguide.
(2) Better to enable users configure the default path, to avoid having
conflict with the path that users need.

Thanks,
―
Jialin Qiao
Apache IoTDB PMC

SpriCoder  于2022年10月27日周四 15:43写道：
>
> Hi all,
>
> There are some built-in storage, such as: root._metric(metrics), 
> root.system.audit(system audit log).
>
> Now, we need to unify the prefix of the built-int system storage group which 
> is `root.system`. So we will modify the default write path of iotdb metric 
> reporter from `root.${database}` to `root.system.${database}`. Related PR 
> ishttps://github.com/apache/iotdb/pull/7757.
>
> Any comments would be much appreciated.
>
>
>
>
>
> Best,
>
> ---
>
> Hongyin Zhang
>
> School of Software, Tsinghua University

Re: Change of configuration files

[冯 庆新]

＋1

获取 Outlook for iOS

发件人: Jialin Qiao 
发送时间: Friday, October 28, 2022 11:41:07 PM
收件人: dev@iotdb.apache.org 
主题: Re: Change of configuration files

+1
―
Jialin Qiao
Apache IoTDB PMC

Haonan Hou  于2022年10月28日周五 19:23写道：
>
> One more change I would like to mentions. I will remove the configurations of 
> HDFS, since we
> do not recommend users to use it on IoTDB.
>
> Best,
> Haonan Hou
>
> > On Oct 28, 2022, at 5:13 PM, Haonan Hou  wrote:
> >
> > Hi,
> >
> > In IoTDB 1.0, we have two roles of nodes and two major configuration files, 
> > iotdb-datanode.properties
> > and iotdb-confignode.properties. Some configurations are duplicated in 
> > these files such as the admin
> > username and password.
> >
> > Therefore, we are going to add a new configuration file, 
> > iotdb-common.properties, to solve this problems
> > The design doc is [1].
> >
> > [1] https://apache-iotdb.feishu.cn/docx/doxcnjzB2xd9sScIlkNYJUJA1zb
> >
> >
> > Thanks,
> > Haonan Hou
>

回复: Change default time partition interval from 1 day to 1 week

[冯 庆新]

Dear Haiming Zhu,

 If only refer to the explanation below mailing list, "No time partition" seems 
better.
So why is it 7 days?not 10 days. What experiments and analyses have you done to 
support this change.

Best Regards.
Qingxin Feng

发件人: Haiming Zhu<mailto:heimin...@gmail.com>
发送时间: 2022年10月10日 20:03
收件人: dev@iotdb.apache.org<mailto:dev@iotdb.apache.org>
主题: Re: Change default time partition interval from 1 day to 1 week

Three reasons to do this:

   1. 1 day is too small to enable time partition, there're too few tsfiles
   in one partition, which is bad for compaction.
   2. Avoid wal accumulation caused by too many time partitions.
   3. Avoid too many partition table entries and then trigger confignode
   snapshot frequently.



Best,
--------
Haiming Zhu


冯 庆新  于2022年10月10日周一 17:46写道：

> Hi,
>   But “enable_partition” is still “false”,right? According to this it
> doesn't affect anyone who using default value in
> ‘iotdb-datanode.properties’ to using standalone version apache-iotdb.
> If someone uses the distributed version, it will affect newly compiled and
> started instances,because it works by default in
> “iotdb-confignode.properties”.
> What is the expected impact? Degraded write performance?
>
> Best Regards.
> Thanks!
>
> 发件人: Haiming Zhu<mailto:heimin...@gmail.com>
> 发送时间: 2022年10月10日 15:06
> 收件人: iotdb<mailto:dev@iotdb.apache.org>
> 主题: Change default time partition interval from 1 day to 1 week
>
> Hi,
>
> We want to change default time partition interval from 1 day to 1 week,
> including
> time_partition_interval_for_routing in the confignode and
> time_partition_interval_for_storage in the datanode.
>
> Best,
> 
> Haiming Zhu
>
>

回复: Change default time partition interval from 1 day to 1 week

[冯 庆新]

Hi,
  But “enable_partition” is still “false”,right? According to this it doesn't 
affect anyone who using default value in ‘iotdb-datanode.properties’ to using 
standalone version apache-iotdb.
If someone uses the distributed version, it will affect newly compiled and 
started instances,because it works by default in “iotdb-confignode.properties”.
What is the expected impact? Degraded write performance?

Best Regards.
Thanks!

发件人: Haiming Zhu
发送时间: 2022年10月10日 15:06
收件人: iotdb
主题: Change default time partition interval from 1 day to 1 week

Hi,

We want to change default time partition interval from 1 day to 1 week,
including
time_partition_interval_for_routing in the confignode and
time_partition_interval_for_storage in the datanode.

Best,

Haiming Zhu

Re: Treat predicate whose time series doesn't exist as false

[冯 庆新]

Hi,
I support you in this improvement.

获取 Outlook for iOS

发件人: Zhou Yifu 
发送时间: Friday, October 7, 2022 3:48:43 PM
收件人: dev@iotdb.apache.org 
主题: 回复: Treat predicate whose time series doesn't exist as false

Yes, I agree with you. Query should not return an exception except for a syntax 
error.

Thanks,
Yifu Zhou

从 Windows 版邮件发送

发件人: Yuan Tian
发送时间: 2022年10月7日 14:56
收件人: dev
主题: Treat predicate whose time series doesn't exist as false

Hi,

Currently in IoTDB, if users write a non-exist time series in
select clause, we will simply ignore that and don't display it in
result columns instead of throwing exceptions. However, if users write
a non-exist time series in where clause, we will throw an exception
like: `the path 'root.test.g_0.d1.s2' in WHERE clause does not exist`.
It's inconsistent with the select clause, so we prepare to just treat
predicate whose time series doesn't exist as false instead of throwing
exception, same as InfluxDB.
How do you guys think?
The JIRA is https://issues.apache.org/jira/browse/IOTDB-4576


Best,
--
Yuan Tian

回复: Distribution: using id to delete nodes

[冯 庆新]

Hi ξ,

   I don't think so.We can obtain IP and port without logging in IOTDB, but if 
we want to get IDs, we must logging in IOTDB.
It looks like you make it easy how we input the commond, but complicates the 
acquisition.
If you are only adding a new way to remove nodes, then I support that.

B.R
Qingxin Feng

发件人: ξ
发送时间: 2022年9月23日 11:21
收件人: dev
主题: Distribution: using id to delete nodes

Hi!
We think it'll be inconvenient to remove data nodes and config nodes by their 
IPs and ports, so now we just require their ids to remove them.
Previous arguments:
remove-datanode.bat/sh: ip:ports, split by ','
remove-confignode.bat/sh: -r ip:port
Current arguments:
remove-datanode.bat/sh: id
remove-confignode.bat/sh: id

The documents are already updated.
Pull request url: https://github.com/apache/iotdb/pull/7417
Issue id: IOTDB-4482

回复: [VOTE] Apache IoTDB 0.13.1 RC1 release

[冯 庆新]

-1
I think you hava to prepare release RC2,because “[To rel/0.13][IOTDB-2736] 
DELETE_STORAGE_GROUP can not be granted to user (reporting 401)” maybe a 
criticlt issue too.


发件人: Haonan Hou
发送时间: 2022年8月3日 9:30
收件人: dev-iotdb
主题: Re: [VOTE] Apache IoTDB 0.13.1 RC1 release

Hi,

+1 (binding)

The source release:
apache headers [ok]
signatures and hashes [ok]
LICENSE and NOTICE [ok]
no jar files [ok]
could compile from source: ./mvnw.sh clean install


These two files are generated when compiling the source code, we should
omit them.

The binary distribution:
the version number in CLI [ok]
signatures and hashes [ok]
start in mac, jdk11 [ok]
statements executed successfully:  [ok]

SET STORAGE GROUP TO root.turbine;
CREATE TIMESERIES root.turbine.d1.s0 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
insert into root.turbine.d1(timestamp,s0) values(1,1);
insert into root.turbine.d1(timestamp,s0) values(2,2);
insert into root.turbine.d1(timestamp,s0) values(3,3);
select * from root.**;

Thanks,
Haonan Hou

> On Aug 3, 2022, at 9:14 AM, Yuan Tian  wrote:
>
> Hi,
>
> +1(binding)
>
> Best,
> -
> Yuan Tian
>
> On Mon, Aug 1, 2022 at 10:18 AM Jialin Qiao  wrote:
>>
>> Hi all,
>>
>> Apache IoTDB 0.13.1 is a bug-fix version from 0.13.0. You can get its
>> mainly changes from [5].
>>
>> Apache IoTDB v0.13.1 has been staged under [2] and it’s time to vote
>> on accepting it for release. All Maven artifacts are available under [1].
>> Voting will be open for 72hr.
>> A minimum of 3 binding +1 votes and more binding +1 than binding -1
>> are required to pass.
>>
>> Release tag: v0.13.1
>> Hash for the release tag: 8dd7aa670bf4ceabf8dd27dd9817de8cd0669e82
>>
>> Before voting +1, PMC members are required to download
>> the signed source code package, compile it as provided, and test
>> the resulting executable on their own platform, along with also
>> verifying that the package meets the requirements of the ASF policy
>> on releases. [3]
>>
>> You can achieve the above by following [4].
>>
>> [ ] +1 accept (indicate what you validated - e.g. performed the
>> non-RM items in [4])
>> [ ] -1 reject (explanation required)
>>
>>
>> [1] https://repository.apache.org/content/repositories/orgapacheiotdb-1081/
>> [2] https://dist.apache.org/repos/dist/dev/iotdb/0.13.1/rc1/
>> [3] https://www.apache.org/dev/release.html#approving-a-release
>> [4]
>> https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
>> [5] https://dist.apache.org/repos/dist/dev/iotdb/0.13.1/rc1/RELEASE_NOTES.md
>> [6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS
>>
>>
>> Best,
>> -
>> Jialin Qiao
>> Apache IoTDB PMC

回复: [IOTDB-4030] Simplify Privilege Operations SQL

[冯 庆新]

I support the change. It seems a lot less "confusing".
But can you list the result set of the statement before and after the change?


发件人: Zhou Yifu
发送时间: 2022年8月3日 11:42
收件人: dev
主题: [IOTDB-4030] Simplify Privilege Operations SQL

Hi all,


Currently, we have a plenty of privilege operations sql which are unfriendly 
for users. So I try to simplify these privileges operations sql because some of 
them are redundant and confusing.

Combine 'LIST ROLE PRIVILEGES ' and 'LIST PRIVILEGES ROLE  
on ' to 'LIST PRIVILEGES ROLE  (on )?

Combine 'LIST USER PRIVILEGES ' and 'LIST PRIVILEGES USER  
on ' to 'LIST PRIVILEGES USER  (on )?

Combine 'LIST USER' and 'LIST ALL USER OF ROLE ' to 'LIST USER (OF 
ROLE )?

Combine 'LIST ROLE' and 'LIST ALL ROLE OF USER ' to 'LIST ROLE (OF 
USER )?



Feel free to discuss here, thanks a lot!



Regards,
Yifu Zhou

从 Windows 版邮件发送

回复: Change rpc_address to internal_address in ConfigNode and refactor show cluster

[冯 庆新]

+1

发件人: Yuan Tian
发送时间: 2022年6月30日 11:23
收件人: dev
主题: Re: Change rpc_address to internal_address in ConfigNode and refactor show 
cluster

+1

Best,
-
Yuan Tian

On Thu, Jun 30, 2022 at 9:45 AM Jialin Qiao  wrote:
>
> Hi,
>
> As ConfigNode only serve DataNodes and other ConfigNode.
>
> The rpc_address in ConfigNode should be internal_address.
> The rpc_port in ConfigNode should be internal_port.
>
> After this, the show cluster function needs to be refactored.
>
> Previously, show cluster print the rpc_address of DataNode and ConfigNode,
> however, rpc_address for DataNode is the client_rpc_service.
>
> So it's better to print the internal_address of DataNode and ConfigNode.
>
> What do you think?
>
> Thanks,
> ―
> Jialin Qiao
> Apache IoTDB PMC

Re: Remove the old cluster module from master branch

[冯 庆新]

＋1

获取 Outlook for iOS

发件人: Jialin Qiao 
发送时间: Sunday, June 26, 2022 9:58:25 AM
收件人: dev@iotdb.apache.org 
主题: Re: Remove the old cluster module from master branch

+1
―
Jialin Qiao
Apache IoTDB PMC


Yuan Tian  于2022年6月26日周日 09:57写道：

> Hi,
>
> +1
>
>
> Best,
> -
> Yuan Tian
>
> On Sun, Jun 26, 2022 at 9:48 AM Haonan Hou 
> wrote:
> >
> > Hi,
> >
> > Since we are going to release the new cluster version 0.14.0-preview in
> few days,
> > I think we can consider removing the old cluster module before it.
> >
> > If we keep the old cluster in the source code, users who doesn’t
> familiar the code
> > may confused and may have trouble to understand the structure of the new
> cluster.
> >
> > How do you think?
> >
> > Best,
> > Haonan Hou
>

回复: Release of new cluster version

[冯 庆新]

+1 for 0.14.0-preview

发件人: Eric Pai
发送时间: 2022年6月22日 16:56
收件人: dev@iotdb.apache.org
主题: Re: Release of new cluster version

Good news! How about '0.14.0-preview' ? As in general development we always use 
*-alpha as an internal test version with all main features present. As the 
description of the version, it mainly contains the new cluster.

在 2022/6/22 16:44，“Jialin Qiao” 写入:

Hi all,

It has been 4~5 months since we started the new cluster version. Thanks for
the contribution of all contributors!

It's time to start the topic of releasing :)

The summary status of the master branch is as follows:

(1) The new cluster version has many advantages compared to the old
one(0.12~0.13).

* MPP Query Engine, we use volcano module with operators for better
extendibility in data processing.
* Configurable consensus protocol (Standalone, Raft, Multi-Leader) for
Partition, Schema and Data.
* Controlable partition table, so that we could control which node to
manage data.
* No whole cluster consensus group.
* More flexible and lightweight scale-out, we could add a node in a few
seconds without moving  data.
* Extensible load balance strategy.
* Built-in metric framework to monitor the status of the cluster.

(2) We have implement the basic function (read/write/schema/udf) of the new
Cluster/standalone version.

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapache-iotdb.feishu.cn%2Fmindnotes%2FbmncnEr7XPzmPvrZgYbcSEvMTIf%23mindmapdata=05%7C01%7C%7C14142baedc434ef11f1d08da542b722d%7C84df9e7fe9f640afb435%7C1%7C0%7C637914842832750235%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=HpaVs90uuGUsA4MhtHGyIltGwexNWcITWmcINs00rN8%3Dreserved=0

There are still some functions that need to be implemented in new
Cluster/standalone.
* Trigger
* Schema Template
* Continuous Query
* Select into
* Sync framework

(3) The performance  optimization of the new Cluster is still going.

https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapache-iotdb.feishu.cn%2Fdocx%2Fdoxcn6xEgTopkvsLsG3TXjwlx0cdata=05%7C01%7C%7C14142baedc434ef11f1d08da542b722d%7C84df9e7fe9f640afb435%7C1%7C0%7C637914842832750235%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=YDO5juXfrC0LZDmYJovmgAHw6LxZJANbLFcyobt4FOM%3Dreserved=0

* Write performance of one replication with standalone/multiLeader in new
cluster. [ok].
* Write performance of one replication with Ratis. [underway]
* Write performance of three replication with Ratis/MultiLeader. [underway]
* Query performance of one replication with standalone/ratis/multileader.
[underway]
* Query performance of three replication with Ratis/MultiLeader. [underway]

I suppose we could get a not-bad number by this weekend for all above
scenaios.

(4) More test needs to be done to improve the stability of the new
cluster/standalone, and the ITs are being added.

As this version does not contains the whole functions of 0.13, and the
system performance and stability need to be improved for some time before
produnction, I suggest retaining the 0.14.0 version number.

So, how about using 【0.14.0-alpha】, and we only release the binary
distribution of new cluster, without new/old standalone version.

Then, for the formal 0.14.0, we could release the new Cluster/Standalone
version and discard the old Standalone version.

Thanks,
—
Jialin Qiao
Apache IoTDB PMC

回复: JDBC vs Java API

[冯 庆新]

Hello Trevor Hart

Can you access this web link: 
http://111.202.73.147:13000/d/5ZvuEYE7z/atm-biao-zhun-da-qi-ya-huan-jing?orgId=1=26
Is this one ok?

Config items in benchmark:
DB_SWITCH=IoTDB-013-JDBC  / SESSION_BY_TABLET / SESSION_BY_RECORDS / 
SESSION_BY_RECORD
GROUP_NUMBER=10
LOOP=1000
DEVICE_NUMBER=50
SENSOR_NUMBER=500
BATCH_SIZE_PER_WRITE=100
POINT_STEP=1000
OP_INTERVAL=0
IS_OUT_OF_ORDER=false

B.R
qingxin.feng


发件人: Trevor Hart
发送时间: 2022年6月6日 8:46
收件人: dev
主题: JDBC vs Java API

Hello Team



Does anyone have any published benchmark results of JDBC vs the Java API?



Firstly Im aware of https://github.com/thulab/iotdb-benchmark but I dont see 
any published results for the various API methods.



I currently use JDBC for my non-realtime ingestion of data and while Ive never 
encountered any bottle necks I am aware that the documentation says that JDBC 
is not recommended for high velocity data.



Ive done some very basic ingestions benchmarking tests of inserting 1 million 
rows and the Java API is around 2x faster. Is this the typical improvement 
between JDBC and Java API?



For my simplistic test I am inserting 1 millions rows of timestamp & 
incrementing row id eg  insert into root.sg1.d1(timestamp,s1) 
values(${DateTime.Now}, ${n})



With JDBC I get around 6000 rows per second.



With the Java native API I get around 12000 rows per second using 
session.executeNonQueryStatement.



I assume insertTablets and insertRecord(s) would be even faster?



Thanks

Trevor Hart

回复: [VOTE] Apache IoTDB 0.11.2 RC1 release

[冯 庆新]

Hi all,


-1

We found a issue about merge when we try to restart iotdb service;

Error Message:
/data/fqx/incubator-iotdb/sbin/../data/data/sequence/root.group_0/0/1609818041167-1-1.tsfile
2021-01-05 13:48:33,085 [pool-4-IoTDB-Compaction-ServerServiceImpl-thread-2] 
ERROR 
org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement:640 
- cannot get tsfile resource path: 
/data/fqx/incubator-iotdb/data/data/sequence/root.group_0/0/1609818041167-1-1.tsfile


Thanks.
B.R


发件人: Xiangdong Huang
发送时间: 2020年12月31日 23:22
收件人: dev
主题: [VOTE] Apache IoTDB 0.11.2 RC1 release

Hi all,

I think this may be the last vote in 2020.  Thank all effort from you this
year once again.

Apache IoTDB 0.11.2 is a bug-fix version from 0.11.1.
You can get its main changes from [5].

Apache IoTDB 0.11.2 has been staged under [2] and it’s time to vote
on accepting it for release.  All Maven artifacts are available under [1].

Voting will be open for 72hr.
A minimum of 3 binding +1 votes and more binding +1 than binding -1
are required to pass.

Release tag: release/0.11.2
Hash for the release tag: d96876eaea7ac1e8c6b8576fd0958956afd32f68

Before voting +1, PMC members are required to download
the signed source code package, compile it as provided, and test
the resulting executable on their own platform, along with also
verifying that the package meets the requirements of the ASF policy
on releases.[3]

You can achieve the above by following [4].

[ ]  +1 accept (indicate what you validated)
[ ]  -1 reject (explanation required)


Bugfix list:

- IOTDB-990 cli parameter maxPRC shouldn't to be set zero
- IOTDB-993 Fix tlog bug
- IOTDB-994 Fix can not get last_value while doing the aggregation query
along with first_value
- IOTDB-1000 Fix read redundant data while select with value filter with
unseq data
- IOTDB-1007 Fix session pool concurrency and leakage issue when pool.close
is called
- IOTDB-1016 overlapped data should be consumed first
- IOTDB-1021 Fix NullPointerException when showing child paths of
non-existent path
- IOTDB-1028 add MAX_POINT_NUMBER format check
- IOTDB-1034 Fix Show timeseries error in Chinese on Windows
- IOTDB-1035 Fix bug in getDeviceTimeseriesMetadata when querying non-exist
device
- IOTDB-1038 Fix flink set storage group bug
- ISSUE-2179 fix insert partial tablet with binary NullPointer bug
- add reject status code
- Update compaction level list delete
- Fix query result is not correct
- Fix import errors in Session.py and SessionExample.py
- Fix modules can not be found when using pypi to pack client-py
- Fix Count timeseries group by level bug
- Fix desc batchdata count bug


[1] https://repository.apache.org/content/repositories/orgapacheiotdb-1051
[2] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc1
[3] https://www.apache.org/dev/release.html#approving-a-release
[4]
https://cwiki.apache.org/confluence/display/IOTDB/Validating+a+staged+Release
[5] https://dist.apache.org/repos/dist/dev/iotdb/0.11.2/rc1/RELEASE_NOTES.md
[6] https://dist.apache.org/repos/dist/dev/iotdb/KEYS

Best,
---
Xiangdong Huang
School of Software, Tsinghua University

 黄向东
清华大学 软件学院

Re: [jira] [Created] (IOTDB-255) Can't restart IoTDB service

[冯 庆新]

OK.
I have done as what you said.Now the IoTDB service has been restarted.
And I will try reproduce this issue. 
B.R

发自我的iPhone

> 在 2019年10月19日，下午8:09，Xiangdong Huang  写道：
> 
> Hi,
> 
> If you do not care losing some data, just remove the broken tsfile (and the
> corresponding tsfile.resource file, if exists).
> 
> My concern is, can the scenario be reproduced? If so, it is  really a
> critical BUG.
> 
> Best,
> ---
> Xiangdong Huang
> School of Software, Tsinghua University
> 
> 黄向东
> 清华大学 软件学院
> 
> 
> Jialin Qiao  于2019年10月18日周五 下午7:15写道：
> 
>> Hi,
>> 
>> This file has 4096 bytes. However, it only contains a Magic header, other
>> bytes are 0.
>> 
>> When we scan the TsFile from the beginning, It takes the first byte behind
>> the header as a marker, which is 0 (ChunkGroupFooter).
>> 
>> A possible solution is discarding the corrupted chunk group, then remove
>> the file if it is empty, otherwise close the remaining file.
>> 
>> Thanks,
>> --
>> Jialin Qiao
>> School of Software, Tsinghua University
>> 
>> 乔嘉林
>> 清华大学 软件学院
>> 
>>> -原始邮件-
>>> 发件人: "FengQingxin (Jira)" 
>>> 发送时间: 2019-10-18 19:04:00 (星期五)
>>> 收件人: dev@iotdb.apache.org
>>> 抄送:
>>> 主题: [jira] [Created] (IOTDB-255) Can't restart IoTDB service
>>> 
>>> FengQingxin created IOTDB-255:
>>> -
>>> 
>>> Summary: Can't restart IoTDB service
>>> Key: IOTDB-255
>>> URL: https://issues.apache.org/jira/browse/IOTDB-255
>>> Project: Apache IoTDB
>>>  Issue Type: Bug
>>>Reporter: FengQingxin
>>> Attachments: 1571300334171-6790.tsfile,
>> image-2019-10-18-19-03-49-959.png
>>> 
>>> Hi IoTDB Dev
>>> 
>>>   when I using iotdb [rel/0.8]，I got a issue.Someone reboot my server
>> without close IoTDB service.After the server restart,I tryed to restart my
>> IoTDB. I got an error message like this:
>>> {color:#FF}Caused by: java.lang.IllegalArgumentException: Given
>> chunkMetaDataList is null{color}
>>> 
>>> 
>>> Looks like I got a broken TsFile.
>>> 
>>> !image-2019-10-18-19-03-49-959.png!
>>> 
>>> Please help us to check this issue.Thanks!
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> This message was sent by Atlassian Jira
>>> (v8.3.4#803005)
>> 

答复: IoTDB Query Performance Test # 35

[冯 庆新]

Hi Chris



  The answer would be “no”.This is just a test for automatic notice test result 
of IoTDB.



BR



Feng







发送自 Windows 10 版邮件应用




发件人: Christofer Dutz 
发送时间: Friday, August 30, 2019 3:10:46 PM
收件人: dev@iotdb.apache.org 
主题: Re: IoTDB Query Performance Test # 35

Just a quick question ...

Are we going to be getting more of these emails? I had to moderate 2 of them 
this morning.

Chris

Am 30.08.19, 01:27 schrieb "dwf_tsing...@163.com" :

This e-mail is generated by Jenkins auto-test project. History query 
results are shown in attachment.

The first three group tests are based on the last release version which can 
be use as baseline for comparing the following test results. The last 6*3=18 
rows correspond to the 6 types of query test based on latest master branch. 
After ingestion test, three groups of query tests will be conducted. The first 
group gets the none-cache query performance results while the second and third 
groups get cached.  Since query test is after  ingestion query, the query 
result can be easily mapped to IoTDB version by looking at projectID.


queryResult.csv introduction (except obvious fields):
Field: Description

projectID: Unique test instance mark.

avg: Average time cost of all query operations. [ms]

midAvg: Average time cost of query operations without 5% head and tail. [ms]

min: Min time cost of all query operations. [ms]

max: Max time cost of all query operations. [ms]

p1: 1 percentile of  all  query operations time cost. [ms]

totalTimes: Max accumulative query operation time cost of the clients. [s]

totalPoints: Number of total query result points.

queryNumber: Number of query operation.

Eight Query Types:
1. aggTi: Aggregation Query with Time Filter
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
aggregation function: max_value
time filter range: 25 ms
500 query operations
example:
SELECT max_value(s_72) FROM root.perform.group_0.d_68 WHERE time >= 
2018-08-30 02:55:50 AND time <= 2018-08-30 03:00:00

2. aggVa: Aggregation Query with Value Filter
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
aggregation function: max_value
time filter range: 25 ms
500 query operations
example:
SELECT max_value(s_25) FROM root.perform.group_1.d_102 WHERE 
root.perform.group_1.d_102.s_25 > 1.0


3. criteri: Criteria Query (Simple Query with Time and Value Filter)
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
time filter range: 25 ms
value filter: > 1
500 query operations
example:
SELECT s_35 FROM root.perform.group_4.d_462 WHERE time >= 2018-08-30 
03:10:00 AND time <= 2018-08-30 03:14:10 AND root.perform.group_4.d_462.s_35 > 
1.0

4. exact: Exact Query
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
500 query operations
example:
SELECT s_75 FROM root.perform.group_7.d_731 WHERE time = 2018-08-30 02:25:00

5. group: Group-By Query
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
time filter range: 25 ms
time unit: 2 ms
aggregation function: max_value
500 query operations
example:
SELECT max_value(s_38) FROM root.perform.group_11.d_1171 GROUP BY(2ms, 
153555840,[2018-08-30 03:07:30,2018-08-30 03:11:40])

6. latest: MAX_TIME Query
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
aggregation function: max_time
500 query operations
example:
SELECT max_time(s_0) FROM root.performf.group_0.d_2

7. limit: limit Query
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
time filter range: 25 ms
500 query operations
example:
SELECT * FROM root.perform.group_0.d_0 limit 5 offset 5 slimit 5 soffset 5

8. range: Range Query
query workload:
number of involved sensor in each query: 1
number of involved device in each query: 1
time filter range: 25 ms
500 query operations
example:
SELECT s_0 FROM root.performf.group_0.d_2 WHERE time >= 2010-01-01 12:00:00 
AND time <= 2010-01-01 12:04:10


Configurations:

Data set is generated by ingestion test:
20 storage groups
2000 devices in total
100 sensors per device
50% out-of-order data

Hardware:
32G memory
8 threads CPU
1000Mbit ethernet

IoTDB:
MAX_HEAP_SIZE="20G"
meta_data_cache_enable=true(as default)
filemeta_chunkmeta_free_memory_proportion=3:6:10(as default)
The 

[#2]Re: Long term test on Apache VM is running

[冯 庆新]

Hello  all :

Last week we try to increase the cache number for evaluating the stability of 
IoTDB.

Now I will report the result to you:
Start Time: 2019-05-24 09:17:43,419
End Time: 2019-05-26 02:49:36,443
Total elapse time: 149506.27 second
Timeseries Num : 20,000 [200 Devices * 100 Sersors]
Loaded 320 points in 78543.09s with 4workers (mean rate 
407419.66points/s)
Total Operations 16; Latency(ms): Avg 1960.6918, MiddleAvg 1926.2808, Min 
1592, Max 25311
Total error num is 0.

///
### client mode ###
CLIENT_NUMBER=4
GROUP_NUMBER=10
DEVICE_NUMBER=200
SENSOR_NUMBER=100
CACHE_NUM=2000
LOOP=800
///
Next time we will try to increase the number of time series to 200,000[2000 
Devices and 100 Sensors].

That’s all.
Please feel free to contact me if you have any questions
Thank you.
B.R

FengQingxin
冯庆新 [qingxin.f...@hotmail.com]
大数据系统软件国家工程实验室

答复: Long term test on Apache VM is running

[冯 庆新]

Hello Kevin



That’s not our best on this enviroment.

Now We're going to increase the loading rate to verify our limits on this VM.



B.R




发件人: Kevin A. McGrail 
发送时间: Thursday, May 23, 2019 11:57:40 AM
收件人: dev@iotdb.apache.org; 冯 庆新
主题: Re: Long term test on Apache VM is running

I like the error rate.  Is the loading rate good?

On 5/22/2019 11:46 PM, 冯 庆新 wrote:
> Hello  all :
>
> Last week we started long term test for evaluating the stability of IoTDB.
>
> Now I will report the result to you:
> Start Time : 2019/5/18 18:35:23
> End  Time : 2019/5/20 19:6:28
> Timeseries Num : 20,000 [200 Devices * 100 Sersors]
> Loaded 327,903,000 points in 77,650.748816s (mean rate 4222.79 points/s).
> Total error num is 0.
>
> IoTDB can handel this stituation well in the VM whose configuration is low.[4 
> cores,16G Memory,200G Disk]
> Actually ,we ran both iotdb and the iotdb-benchmark in this VM.
> We have to stoped the test because there is no space left for new data.
> Next time we will try to increase the number of time series to 200,000[2000 
> Devices and 100 Sensors].
>
> That’s all.
> Please feel free to contact me if you have any questions
> Thank you.
> B.R
>
> FengQingxin
> 冯庆新 [qingxin.f...@hotmail.com]
> 大数据系统软件国家工程实验室
>
> 
>
> On 2019/05/19 12:35:23, Xiangdong Huang  wrote:
>> Hi all,>
>>
>> We have got a VM from ASF (iotdb-vm.apache.org), with 16GB memory, 4 cores>
>> and 200GB space.>
>>
>> Now Qingxin and I have started a long term test for evaluating the>
>> stability of IoTDB.>
>> The workload simulates 4 clients writing time series data as fast as>
>> possible. The data represents for  200 devices, while each device has 100>
>> sensors. The workload is generated by iotdb-benchmark[1].>
>>
>>
>> [1] https://github.com/thulab/iotdb-benchmark>
>>
>> Best,>
>> --->
>> Xiangdong Huang>
>> School of Software, Tsinghua University>
>>
>>  黄向东>
>> 清华大学 软件学院>
>>

--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Long term test on Apache VM is running

[冯 庆新]

Hello  all :

Last week we started long term test for evaluating the stability of IoTDB.

Now I will report the result to you:
Start Time : 2019/5/18 18:35:23
End  Time : 2019/5/20 19:6:28
Timeseries Num : 20,000 [200 Devices * 100 Sersors]
Loaded 327,903,000 points in 77,650.748816s (mean rate 4222.79 points/s).
Total error num is 0.

IoTDB can handel this stituation well in the VM whose configuration is low.[4 
cores,16G Memory,200G Disk]
Actually ,we ran both iotdb and the iotdb-benchmark in this VM.
We have to stoped the test because there is no space left for new data.
Next time we will try to increase the number of time series to 200,000[2000 
Devices and 100 Sensors].

That’s all.
Please feel free to contact me if you have any questions
Thank you.
B.R

FengQingxin
冯庆新 [qingxin.f...@hotmail.com]
大数据系统软件国家工程实验室



On 2019/05/19 12:35:23, Xiangdong Huang  wrote:
> Hi all,>
>
> We have got a VM from ASF (iotdb-vm.apache.org), with 16GB memory, 4 cores>
> and 200GB space.>
>
> Now Qingxin and I have started a long term test for evaluating the>
> stability of IoTDB.>
> The workload simulates 4 clients writing time series data as fast as>
> possible. The data represents for  200 devices, while each device has 100>
> sensors. The workload is generated by iotdb-benchmark[1].>
>
>
> [1] https://github.com/thulab/iotdb-benchmark>
>
> Best,>
> --->
> Xiangdong Huang>
> School of Software, Tsinghua University>
>
>  黄向东>
> 清华大学 软件学院>
>