xndai commented on code in PR #17609: URL: https://github.com/apache/iceberg/pull/17609#discussion_r3770995142
########## format/spec.md: ########## @@ -822,10 +822,12 @@ Each stats struct holds statistics for one table field. It may contain the follo | _optional_ | 2 | `upper_bound` | Field type or `geo_upper` | all primitives or `variant` | Upper bound stored as the field's type, or `geo_upper` for geo types | | _optional_ | 3 | `tight_bounds` | `boolean` | all primitives except for `geometry` and `geography` | When true, `lower_bound` and `upper_bound` must be equal to the min and max values | | _optional_ | 4 | `value_count` | `long` | all | Number of values in the column (including null and NaN values) | -| _optional_ | 5 | `null_value_count` | `long` | optional fields | Number of null values in the column | +| _optional_ | 5 | `null_value_count` | `long` | nullable fields | Number of null values in the column | Review Comment: > our null count matches what Parquet produces, which is a null count that is up to the number of values for the field. If a parent is null, it doesn't show up in the field-level count. I thought this was a bug, and even created a PR for that (https://github.com/apache/iceberg/pull/17560). We do see cases where planner was misleaded and produced wrong results. I will take a look at #17413 and discuss there. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
