rdblue commented on code in PR #17609: URL: https://github.com/apache/iceberg/pull/17609#discussion_r3770819625
########## format/spec.md: ########## @@ -822,10 +822,12 @@ Each stats struct holds statistics for one table field. It may contain the follo | _optional_ | 2 | `upper_bound` | Field type or `geo_upper` | all primitives or `variant` | Upper bound stored as the field's type, or `geo_upper` for geo types | | _optional_ | 3 | `tight_bounds` | `boolean` | all primitives except for `geometry` and `geography` | When true, `lower_bound` and `upper_bound` must be equal to the min and max values | | _optional_ | 4 | `value_count` | `long` | all | Number of values in the column (including null and NaN values) | -| _optional_ | 5 | `null_value_count` | `long` | optional fields | Number of null values in the column | +| _optional_ | 5 | `null_value_count` | `long` | nullable fields | Number of null values in the column | Review Comment: This is a breaking change from the current behavior of our statistics. Right now, our null count matches what Parquet produces, which is a null count that is up to the number of values for the field. If a parent is null, it doesn't show up in the field-level count. It shows up in that the field has a different value count than the parent or the number of records in the data file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
