alamb commented on code in PR #244:
URL: https://github.com/apache/parquet-format/pull/244#discussion_r1618578403
##########
src/main/thrift/parquet.thrift:
##########
@@ -625,7 +631,11 @@ struct DataPageHeaderV2 {
/** Number of NULL values, in this data page.
Number of non-null = num_values - num_nulls which is also the number of
values in the data section **/
2: required i32 num_nulls
- /** Number of rows in this data page. which means pages change on record
boundaries (r = 0) **/
+ /**
+ * Number of rows in this data page. Every page must begin at a
+ * record boundary (repetition_level = 0): records must **not** be
+ * split across page boundaries when using V2 data pages.
Review Comment:
yes
##########
src/main/thrift/parquet.thrift:
##########
@@ -578,7 +578,13 @@ enum BoundaryOrder {
/** Data page header */
struct DataPageHeader {
- /** Number of values, including NULLs, in this data page. **/
+ /**
+ * Number of values, including NULLs, in this data page.
+ *
+ * If a OffsetIndex is present, a page must begin at a record
Review Comment:
I agree row/record seem to mean the same thing
I double checked and it appears the rest of the file is inconsistent in the
terminology as well
For example the repetition level documentation refers to records
https://github.com/apache/parquet-format/blob/8d59c7d8c0c01da1a435af36caa9f470e9bd3b67/src/main/thrift/parquet.thrift#L180-L192
but there are several fields that are named `num_rows` that clearly refer to
rows.
https://github.com/apache/parquet-format/blob/8d59c7d8c0c01da1a435af36caa9f470e9bd3b67/src/main/thrift/parquet.thrift#L623-L631
In this PR I followed the term used in the repetition level documentation as
I think it is the most relevant.
I will start a conversation on the mailing list about using consistent
terminology
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
