jt2594838 commented on code in PR #365: URL: https://github.com/apache/tsfile/pull/365#discussion_r1903751462
########## docs/src/UserGuide/latest/QuickStart/Data-Model-zh.md: ########## Review Comment: Separate the zh and en docs into two directories. ########## docs/src/UserGuide/latest/QuickStart/Data-Model.md: ########## @@ -24,42 +24,45 @@ To manage industrial IoT timing data, the measurement point data model of TsFile includes the following information Review Comment: To manage industrial IoT time series data, the data model of TsFile includes the following concepts ########## docs/src/UserGuide/latest/QuickStart/Data-Model.md: ########## @@ -24,42 +24,45 @@ To manage industrial IoT timing data, the measurement point data model of TsFile includes the following information -- DeviceId（String）：Device Name -- MeasurementSchema：Measurement points - - measurementId（String）：Measurement Point Name - - tsDataType（TSDataType）：Data Type - -For the above detailed introduction, please refer to：[Entering Time Series Data](https://tsfile.apache.org/UserGuide/latest/QuickStart/Navigating_Time_Series_Data.html) - -## Example - - - -In the above example, the metadata (Scheme) of TsFile contains 2 devices and 5 time series, and is established as a table structure as shown in the following figure: +- TAG：Tag Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type +- FIELD：Field Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type <table> <tr> - <th rowspan="1">Device ID</th> - <th rowspan="1">Measurement points</th> + <th rowspan="1">concept</th> + <th rowspan="1">definition</th> </tr> <tr> - <th rowspan="2">Solar panel 1</th> - <th>Voltage（FLOAT）</th> + <th rowspan="1">table</th> + <th>A collection of devices with the same pattern.The storage table defined during modeling consists of three parts: identification column, time column, and physical quantity column.</th> </tr> <tr> - <th>Current（FLOAT）</th> + <th rowspan="1">TAG</th> + <th>The unique identifier of a device, which can contain 0 to multiple tag columns in a table. The composite value formed by combining the values of the tag columns in the column order when the table was created is called the identifier, and tags with the same composite value are called the same identifier.The data type of the tag column can currently only be String, which can be left unspecified and defaults to StringThe values of the identification column can all be emptyWhen writing, all tag columns must be specified (unspecified identity columns are filled with null by default)</th> </tr> <tr> - <th rowspan="4">Fan1</th> + <th rowspan="1">Time</th> + <th>A table must have a time column, and data with the same identifier value is sorted by time by default.The values in the time column cannot be empty and must be in sequence.</th> </tr> <tr> - <th>Voltage（FLOAT）</th> - </tr> - <tr> - <th>Current（FLOAT）</th> + <th rowspan="1">FIELD</th> + <th>The field column defines the measurement point names and data types for time-series data.</th> </tr> <tr> - <th>Wind Speed（FLOAT）</th> + <th rowspan="1">row</th> + <th>A row of data in the table</th> </tr> </table> + +## Example + +A table is a collection of devices with the same pattern. As shown in the figure below, it is a modeling management of factory equipment, and the physical quantity collection of each device has certain commonalities (such as collecting temperature and humidity physical quantities, collecting physical quantities of the same device on the same frequency, etc.), so it can be managed on a device by device basis. + +At this point, a physical device can be uniquely identified through [Region] - [Factory] - [Equipment] (orange column in the figure below, also known as device identification information). The final indicators collected by the device are [Temperature], [Humidity], [Status], and [Arrival Time] (blue column in the figure below). + Review Comment: identified through -> identified through 3 Tags The final indicators -> The Fields ########## docs/src/UserGuide/latest/QuickStart/Interface-Definitions.md: ########## @@ -0,0 +1,240 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> +# Interface Definitions + +## Write Interface + +### ITsFileWriter + +Used to write data to tsfile + +```shell Review Comment: Java ########## docs/src/Download/README-LATEST.md: ########## Review Comment: Only one readme is enough for “download.” ########## docs/src/UserGuide/latest/QuickStart/Data-Model.md: ########## @@ -24,42 +24,45 @@ To manage industrial IoT timing data, the measurement point data model of TsFile includes the following information -- DeviceId（String）：Device Name -- MeasurementSchema：Measurement points - - measurementId（String）：Measurement Point Name - - tsDataType（TSDataType）：Data Type - -For the above detailed introduction, please refer to：[Entering Time Series Data](https://tsfile.apache.org/UserGuide/latest/QuickStart/Navigating_Time_Series_Data.html) - -## Example - - - -In the above example, the metadata (Scheme) of TsFile contains 2 devices and 5 time series, and is established as a table structure as shown in the following figure: +- TAG：Tag Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type +- FIELD：Field Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type <table> <tr> - <th rowspan="1">Device ID</th> - <th rowspan="1">Measurement points</th> + <th rowspan="1">concept</th> + <th rowspan="1">definition</th> </tr> <tr> - <th rowspan="2">Solar panel 1</th> - <th>Voltage（FLOAT）</th> + <th rowspan="1">table</th> + <th>A collection of devices with the same pattern.The storage table defined during modeling consists of three parts: identification column, time column, and physical quantity column.</th> </tr> <tr> - <th>Current（FLOAT）</th> + <th rowspan="1">TAG</th> + <th>The unique identifier of a device, which can contain 0 to multiple tag columns in a table. The composite value formed by combining the values of the tag columns in the column order when the table was created is called the identifier, and tags with the same composite value are called the same identifier.The data type of the tag column can currently only be String, which can be left unspecified and defaults to StringThe values of the identification column can all be emptyWhen writing, all tag columns must be specified (unspecified identity columns are filled with null by default)</th> </tr> <tr> - <th rowspan="4">Fan1</th> + <th rowspan="1">Time</th> + <th>A table must have a time column, and data with the same identifier value is sorted by time by default.The values in the time column cannot be empty and must be in sequence.</th> </tr> <tr> - <th>Voltage（FLOAT）</th> - </tr> - <tr> - <th>Current（FLOAT）</th> + <th rowspan="1">FIELD</th> + <th>The field column defines the measurement point names and data types for time-series data.</th> </tr> <tr> - <th>Wind Speed（FLOAT）</th> + <th rowspan="1">row</th> + <th>A row of data in the table</th> </tr> </table> + +## Example + +A table is a collection of devices with the same pattern. As shown in the figure below, it is a modeling management of factory equipment, and the physical quantity collection of each device has certain commonalities (such as collecting temperature and humidity physical quantities, collecting physical quantities of the same device on the same frequency, etc.), so it can be managed on a device by device basis. Review Comment: A table is -> A schema describes it is a modeling management -> it models the management physical quantity -> physical variable device by device -> device-by-device ########## docs/src/UserGuide/latest/QuickStart/Interface-Definitions.md: ########## @@ -0,0 +1,240 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> +# Interface Definitions + +## Write Interface + +### ITsFileWriter + +Used to write data to tsfile + +```shell +interface ITsFileWriter extends AutoCloseable { + // Write data + void write(Tablet tablet); + + // Close Write + void close(); +} +``` + +### TsFileWriterBuilder + +Used to construct ITsFileWriter + +```shell +class TsFileWriterBuilder { + // Build ITsFileWriter object + public ITsFileWriter build(); + + // target file + public TsFileWriterBuilder file(File file); + + // Used to construct table structures + public TsFileWriterBuilder tableSchema(TableSchema schema); + + // Used to limit the memory size of objects + public TsFileWriterBuilder memoryThreshold(long memoryThreshold); +} +``` + +### TableSchema + +Describe the data structure of the table schema + +```shell +class TableSchema { + // Constructor function + public TableSchema(String tableName, List<ColumnSchema> columnSchemaList); +} + +class ColumnSchema { + // Constructor function + public ColumnSchema(String columnName, TSDataType dataType, ColumnCategory columnCategory); + + // Get column names + public String getColumnName(); + + // Get the data type of the column + public TSDataType getDataType(); + + // Get column category + public Tablet.ColumnCategory getColumnCategory(); +} + +class ColumnSchemaBuilder { + // Build ColumnSchema object + public ColumnSchema build(); + + // Column Name + public ColumnSchemaBuilder name(String columnName); + + // The data category of the column Review Comment: category -> data type ########## docs/src/UserGuide/latest/QuickStart/Interface-Definitions.md: ########## @@ -0,0 +1,240 @@ +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> +# Interface Definitions + +## Write Interface + +### ITsFileWriter + +Used to write data to tsfile + +```shell +interface ITsFileWriter extends AutoCloseable { + // Write data + void write(Tablet tablet); + + // Close Write + void close(); +} +``` + +### TsFileWriterBuilder + +Used to construct ITsFileWriter + +```shell +class TsFileWriterBuilder { + // Build ITsFileWriter object + public ITsFileWriter build(); + + // target file + public TsFileWriterBuilder file(File file); + + // Used to construct table structures + public TsFileWriterBuilder tableSchema(TableSchema schema); + + // Used to limit the memory size of objects + public TsFileWriterBuilder memoryThreshold(long memoryThreshold); +} +``` + +### TableSchema + +Describe the data structure of the table schema + +```shell +class TableSchema { + // Constructor function + public TableSchema(String tableName, List<ColumnSchema> columnSchemaList); +} + +class ColumnSchema { + // Constructor function + public ColumnSchema(String columnName, TSDataType dataType, ColumnCategory columnCategory); + + // Get column names + public String getColumnName(); + + // Get the data type of the column + public TSDataType getDataType(); + + // Get column category + public Tablet.ColumnCategory getColumnCategory(); +} + +class ColumnSchemaBuilder { + // Build ColumnSchema object + public ColumnSchema build(); + + // Column Name + public ColumnSchemaBuilder name(String columnName); + + // The data category of the column + public ColumnSchemaBuilder dataType(TSDataType columnType); + + // Column category + public ColumnSchemaBuilder category(ColumnCategory columnCategory); + + // Supported types + enum TSDataType { + BOOLEAN, + INT32, + INT64, + FLOAT, + DOUBLE, + TIMESTAMP, + TEXT, + DATE, + BLOB, + STRING; + } + + // Supported column categories + enum ColumnCategory { + TAG, + FIELD + } +} +``` + +### Tablet + +Write column memory structure + +```shell +class Tablet { + // Constructor function + public Tablet(List<String> columnNameList, List<TSDataType> dataTypeList); + public Tablet(List<String> columnNameList, List<TSDataType> dataTypeList, int maxRowNum); + + // Interface for adding timestamps + void addTimestamp(int rowIndex, long timestamp) + + // Interface for adding values + // Add values based on column names + void addValue(int rowIndex, String columnName, int val) + void addValue(int rowIndex, String columnName, long val) + void addValue(int rowIndex, String columnName, float val) + void addValue(int rowIndex, String columnName, double val) + void addValue(int rowIndex, String columnName, boolean val) + void addValue(int rowIndex, String columnName, String val) + void addValue(int rowIndex, String columnName, byte[] val) + void addValue(int rowIndex, String columnName, LocalDate val) + // Add values based on index position + void addValue(int rowIndex, int columnIndex, int val) + void addValue(int rowIndex, int columnIndex, long val) + void addValue(int rowIndex, int columnIndex, float val) + void addValue(int rowIndex, int columnIndex, double val) + void addValue(int rowIndex, int columnIndex, boolean val) + void addValue(int rowIndex, int columnIndex, String val) + void addValue(int rowIndex, int columnIndex, byte[] val) + void addValue(int rowIndex, int columnIndex, LocalDate val) +} +``` + +## Read Interface + +### ITsFileReader + +Used to query data in tsfile + +```shell +interface ITsFileReader extends AutoCloseable { + // Used to execute queries and return results + ResultSet query(String tableName, List<String> columnNames, long startTime, long endTime); + + // Return the schema of the table named tableName in tsfile + Optional<TableSchema> getTableSchemas(String tableName); + + // Retrieve schema information for all tables in the tsfile + List<TableSchema> getAllTableSchema(); + + // Close query + void close(); +} +``` + +### TsFileReaderBuilder + +Used to construct ITsFileWriter + +```shell +class TsFileReaderBuilder { + // Build ITsFileReader object + public ITsFileReader build(); + + // target file + public TsFileReaderBuilder file(File file); +} +``` + +### ResultSet + +The result set of the query + +```shell +interface ResultSet extends AutoCloseable { + // Move the cursor to the next line and return whether there is still data + boolean next(); Review Comment: line -> row ########## docs/src/UserGuide/latest/QuickStart/Data-Model.md: ########## @@ -24,42 +24,45 @@ To manage industrial IoT timing data, the measurement point data model of TsFile includes the following information -- DeviceId（String）：Device Name -- MeasurementSchema：Measurement points - - measurementId（String）：Measurement Point Name - - tsDataType（TSDataType）：Data Type - -For the above detailed introduction, please refer to：[Entering Time Series Data](https://tsfile.apache.org/UserGuide/latest/QuickStart/Navigating_Time_Series_Data.html) - -## Example - - - -In the above example, the metadata (Scheme) of TsFile contains 2 devices and 5 time series, and is established as a table structure as shown in the following figure: +- TAG：Tag Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type +- FIELD：Field Column + - name（String）：Column Name + - dataType（TSDataType）：Data Type Review Comment: - Tag: (注意使用英文冒号) The description of an IoT device of one dimension in the form of a string key-value pair. For example, "category=XT451" describes the category of a wind turbine is "XT451" and "year=2021" says it is built in the year 2021. - Field: A variable that is being measured by a device. Like fan speed, voltage, temperature, wind speed, and so on. Unlike a Tagthat is currently fixed to STRING data type, Fields are supported by various data types. - Timestamp and Timeseries: Each value of a FIELD measured by a device is associated with a unique timestamp. The sequence of the timestamps and values of a FIELD is a Timeseries. - Schema: A set of Tags that can uniquely identify a kind of device, and the set of Fields that are measured by these devices. E.g., a wind turbine can be uniquely located by "Province," "City," "WindField," and "SerialNumber," so the TAG set of SCHEMA "wind_turbine" can be "Province," "City," "WindField," and "SerialNumber." While its the FIELD set contains "FanSpeed," "Voltage," "Temperature," "WindSpeed," and so on. - DeviceId: The unique identifier of an IoT device, which consists of the schema name and all tag values. For example, for the Schema "wind_turbine" above, a DeviceId can be ("wind_turbine", "Beijing", "Beijing", "Field_1135", "T1523678"). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
