rok commented on code in PR #624:
URL: https://github.com/apache/parquet-format/pull/624#discussion_r4471584482


##########
LogicalTypes.md:
##########
@@ -1097,6 +1097,75 @@ optional group my_map (MAP_KEY_VALUE) {
 }
 ```
 
+### Finite Vectors
+
+`FINITE_VECTOR` is used to annotate fixed-length ordered sequences of finite 
and
+non-null elements.
+
+`FiniteVectorType` annotation has one required parameter, `num_elements`,
+which is the number of elements in each non-null vector and MUST be greater
+than zero.
+
+`FINITE_VECTOR` MUST always annotate the canonical 3-level structure:
+
+```
+<vector-repetition> group <name> (FINITE_VECTOR(<num_elements>)) {
+  repeated group list {
+    required <element-type> element;
+  }
+}
+```
+
+* The outer level must be a group annotated with `FINITE_VECTOR`. Its
+  repetition must be either `optional` or `required` and determines whether
+  the vector may be null. It must contain a single field named `list`.
+* The middle level must be a repeated group named `list` with a single field
+  named `element`.
+* The element MUST be a `required` primitive field, individual elements MUST
+  NOT be null.
+
+The element types supported are:
+
+* unannotated `BOOLEAN`, `FLOAT` or `DOUBLE`
+* `INT32` or `INT64`, either unannotated or annotated with `INTEGER` or
+  `DECIMAL`
+* `FIXED_LEN_BYTE_ARRAY` annotated with `FLOAT16` or `DECIMAL`.
+
+Every numeric element MUST be finite. For `FLOAT`, `DOUBLE`, and `FLOAT16`,
+this excludes NaN, positive infinity, and negative infinity.
+
+For example, a nullable vector containing 768 required `FLOAT` elements is:
+
+```
+optional group embedding (FINITE_VECTOR(768)) {
+  repeated group list {
+    required float element;
+  }
+}
+```
+
+Every non-null vector must contain exactly `num_elements` elements.
+Because `num_elements` is greater than zero, a non-null vector cannot be empty.
+For example, with `num_elements = 3` and a `required float element`,
+`[1.0, 2.0, 3.0]` is valid. `[]`, `[1.0, 2.0]`, `[1.0, 2.0, 3.0, 4.0]`,
+`[1.0, null, 3.0]`, `[null, null, null]`, `[1.0, NaN, 3.0]`, and
+`[1.0, +Infinity, 3.0]` are invalid. A null vector is valid only when the
+outer group is `optional`.
+
+The sort order of `FINITE_VECTOR` values is undefined and no statistics are
+defined for whole vectors.
+
+#### Compatibility
+
+Readers that do not support `FINITE_VECTOR` can use the 3-level structure to 
read
+it as ordinary list-like data, as they already when they encounter a similar
+non-annotated structure.
+
+`FINITE_VECTOR` does not change how values are encoded. A `FINITE_VECTOR`
+column has the same physical representation as a `LIST` column with the same
+structure and element type. The annotation only declares the fixed element
+count and the non-null, finite-element guarantees described above.
+

Review Comment:
   ```suggestion
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to