IMO, some basic cluster functions should be taken into consideration, no matter which architecture we choose finally: 1. The ability of adding or removing the nodes. 2. How to migrate the data when adding/removing nodes. 3. If the data distribution is not even, can we do a data rebalance manually or automatically. 4. If the cluster is very large, the information sync cost may be huge.
在 2021/11/10 下午9:22，“[email protected]”<[email protected]> 写入: This is the features I concerned for the next architecture that I have proposed in my first discussion email: 1. the num of nodes in cluster can support, 2. the write and read performance in multi-replicas, 3. the linear ratio of read and write in cluster, 4. the efficiency and smartness of scheduler in cluster based on the dynamic workloads, 5. the joint query between time series data and relational data, 6. the federal cluster query across data centers, AZ and regions, 7. the spatio-temporal data analysis in one DB etc… 宋秉华 [email protected] 发件人： jianyun cheng 发送时间： 2021-11-10 21:06 收件人： [email protected] 主题： RE: What is the next architecture of IoTDB? Totally agree with the point that we should have some new architecture. But before get started, we should know that the challenges really can’t be overcome in current architecture and IoTDB really needs the features in the challenge list. Each architecture has its pro and weakness. Actually what I worry about is when we spend a lot time on the new architecture but finally we just get very a few benefit. I think we can follow the below steps to decide if we need a new architecture or not. 1. List the features expect to support. 2. Triage the features, filter out true requirements. (This can be done via vote.) 3. Can these true requirements be implemented in the current architecture? 4. If the answer in the 3 step is No. Then we really need a new architecture. Architecture evolution is an eternal topic, we should know where we want to go in each step. It would be very great if you can share some ideas so that we could have some discussion. How do you think? ---------------------------------------------------------- Jianyun Cheng Thanks! From: [email protected]<mailto:[email protected]> Sent: Wednesday, November 10, 2021 10:48 AM To: dev<mailto:[email protected]> Subject: What is the next architecture of IoTDB? What is the next architecture of IoTDB? As we know，IoTDB cluster version is based on the standalone architecture and almost reused all the basic components of standalone version and just add a multi-raft protocol based on the thrift. With the application spreading, the performance of the cluster version of IoTDB is changllaged for many aspects： the num of nodes in cluster can support, the write and read performance in multi-replicas, the linear ratio of read and write in cluster, the efficiency and smartness of scheduler in cluster based on the dynamic workloads, the joint query between time series data and relational data, the federal cluster query across data centers, AZ and regions and the spatio-temporal data analysis in one DB etc… So, I suggest we should make a blueprint for the next architecture of IoTDB and we can learn the advantage of the InfluxDB. The InfluxDB has designed the next architecture named IOx and use Fusion as the next query engine and adopt parquet as the file format etc. It's time to action, WDYT？ Bruce Song 宋秉华 icloudsong
