Hi Jincheng, Thank you! Looks good :)
jincheng sun <[email protected]> 于2020年1月16日周四 下午4:01写道： > Thanks for your explanation! @Jialin > > I thought about the composition of our components from the external system, > the user level, to the IoTDB bottom-level implementation to the auxiliary > tools, and I adjusted a little based on yours. as follows: > > 1. --Connectors------------- > Connectors components include all data integration with external systems, > The components are listed as follows: > > Connectors > Connectors/Grafana > Connectors/Spark-TsFile > Connectors/Spark-IoTDB > Connectors/Hive-tsfile > Connectors/Hadoop-tsfile > > 2. --Client------------- > Client components involves RPC between client and server, a command-line > interface（multi-languages).The components are listed as follows: > > Client > Client/Java > Client/Python > > 3. --Planner------------- > Planner components involves the query process in IoTDB, including multiple > types of queries, such as raw data query, aggregations, group by time, > group by device, and includes many optimized rules.The components are > listed as follows: > > Planner > Planner/sql parsar > Planner/sql optimizer > > 4. --Core------------- > Core components is the storage engine of IoTDB. The components are listed > as follows: > > Core > Core/TsFile > Core/Merge > Core/Storage > Core/WAL > > 5. --Tools------------- > Tools components involves many utils of IoTDB, Such as data Sync. The > components are listed as follows: > Tools > Tools/Sync > Tools/Other > > 6. --Authority--------- > Authority > > 7. --Distribution------ > Distribution > > 8. --Example----------- > Example > > 9. --WebSite----------- > WebSite > > 10. --Doc--------------- > Doc > > 11. --Other------------ > Other （I'm not sure if we really need the Other component, but there is no > harm in adding one） > > This does not change much from your solution. The core is to add a layer of > abstraction. > > Does this make sense to you? Anyway we can continue to improve if needed! > > Best, > Jincheng > > Jialin Qiao <[email protected]> 于2020年1月15日周三 下午4:28写道： > > > Hi, > > > > WAL has an independent physical layout that has nothing to do with > TsFile. > > It is used in the writing process and recovering in IoTDB. > > It could be merged into Storage components. > > > > I'm not sure whether I leave out some components, so I add an Other and > > therefore can not give an example... > > > > Thanks, > > > > jincheng sun <[email protected]> 于2020年1月15日周三 下午4:01写道： > > > > > Sorry for the late reply, I just came back from team outing. > > > > > > Thanks for adding more detail information. I have two questions then: > > > > > > - Is WAL a write strategy for TsFlie, can WAL be incorporated into > other > > > components and what is the reason for being a separate component? > > > - Can you give an example of what changes belongs to the Other > component? > > > > > > Best, > > > Jincheng > > > > > > > > > Jialin Qiao <[email protected]> 于2020年1月9日周四 下午6:39写道： > > > > > > > Hi, > > > > > > > > Sure, maybe a bit more... > > > > > > > > - TsFile : A columnar file format that has flexible schema and data > > > > management for time series. > > > > - Storage : The storage engine of IoTDB, including writing to > > memtable > > > > (buffer), flushing, system recovery, and memtable management. > > > > - Query : The query process in IoTDB, including multiple types of > > > > queries, such as raw data query, aggregations, group by time, group > by > > > > device. > > > > - Merge : Compaction of data files, including multiple compaction > > > > strategies, such as (1) merging unsequence TsFile with sequence > TsFile, > > > (2) > > > > merging small sequence TsFiles into a large TsFile. > > > > - Sync: Synchronize data files from one IoTDB instance to another. > > > > - Authority: Administration of IoTDB, including users, roles and > > > > privileges. > > > > - WAL : Write-ahead-log. > > > > - Client: Rpc between client and server, a command-line interface > and > > > > client of different languages. > > > > - Distribution : The shared-nothing distribution of IoTDB, > including > > > raft > > > > protocol, data partition, and system information management. > > > > - Grafana : The connector for using IoTDB as a data source in > > Grafana. > > > > - Spark-TsFile : A connector that allows users read/write TsFile > > > through > > > > SparkSQL (treat TsFile as an external data source for SparkSQL). > > > > - Spark-IoTDB : A connector that allows users read/write IoTDB > > through > > > > SparkSQL (treat IoTDB as an external data source for SparkSQL). > > > > - Hive-tsfile : A connector that maps TsFile as a table in Hive for > > the > > > > query. > > > > - Hadoop-tsfile : A connector that allows users read/write TsFile > > > through > > > > MapReduce. > > > > - Tools : Other tools for IoTDB, such as watermark, memory > > estimation, > > > > JMX, query history visualization. > > > > - Example : Examples of different modules, such as JDBC, Session, > > > TsFile. > > > > - WebSite : IoTDB official website. > > > > - Doc: documents of IoTDB, docs folder under root directory. > > > > - Other: Those do not belong to the above components. > > > > > > > > > > > > jincheng sun <[email protected]> 于2020年1月9日周四 下午5:28写道： > > > > > > > > > Hi, > > > > > > > > > > @Xiangdong thank you for helping to clarify my original intention. > > you > > > > are > > > > > right, I really want to list some components to let users choose > > which > > > > > component the given issue belongs to. > > > > > > > > > > @Jialin I think approach 4 would be more meaningful for IoTDB. > Could > > > you > > > > > please add the detailed description for each component, such as, > > > describe > > > > > the relationship between `tsfile` and > > > > > (`tsfile/hive-tsfile/spark-tsfile/Hadoop-tsfile`), and which > changes > > > > belong > > > > > to `Other` component, etc., which can help us better understand the > > > > reasons > > > > > for such classification. On the other hand, we will further discuss > > > > whether > > > > > there is optimization space for this component classification. > > > > > > > > > > What do you think? > > > > > > > > > > Best, > > > > > Jincheng > > > > > > > > > > Xiangdong Huang <[email protected]> 于2020年1月9日周四 下午4:46写道： > > > > > > > > > > > > No that is not the Apache Way. No one should be in charge of > an > > > area > > > > > > > > > > > > I think Jincheng's mean is, we can list some components to let > > users > > > > > > choose which component the given issue belongs to. > > > > > > > > > > > > As for which components should be listed, we need to have a > > > discussion. > > > > > > > > > > > > > No one should be in charge of an area > > > > > > > > > > > > Yes everyone can assign an issue to himself/herself. > > > > > > > > > > > > Best, > > > > > > ---------------------------------- > > > > > > Xiangdong Huang > > > > > > School of Software, Tsinghua University > > > > > > > > > > > > 黄向东 > > > > > > 清华大学 软件学院 > > > > > > > > > > > > > > > > > > jincheng sun <[email protected]> 于2020年1月9日周四 下午4:38写道： > > > > > > > > > > > > > > No that is not the Apache Way. No one should be in charge of > an > > > > area. > > > > > > > > > > > > > > Oh, great thanks for correcting me Justin, maybe it's my > language > > > > > > > expression problems. I mean some contributors and committers > will > > > be > > > > > very > > > > > > > interested in some components, then spend more time, and be > more > > > > > familiar > > > > > > > with the code. > > > > > > > > > > > > > > Best, > > > > > > > Jincheng > > > > > > > > > > > > > > > > > > > > > Justin Mclean <[email protected]> 于2020年1月9日周四 > 下午4:27写道： > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > When our community grows stronger, each\ module may have > some > > > > > > specific > > > > > > > > person in charge. > > > > > > > > > > > > > > > > No that is not the Apache Way. No one should be in charge of > an > > > > area. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Justin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > ————————————————— > > > > Jialin Qiao > > > > School of Software, Tsinghua University > > > > > > > > 乔嘉林 > > > > 清华大学 软件学院 > > > > > > > > > > > > > -- > > ————————————————— > > Jialin Qiao > > School of Software, Tsinghua University > > > > 乔嘉林 > > 清华大学 软件学院 > > > -- ————————————————— Jialin Qiao School of Software, Tsinghua University 乔嘉林 清华大学 软件学院
