xndai commented on code in PR #17560:
URL: https://github.com/apache/iceberg/pull/17560#discussion_r3786602849
##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueWriters.java:
##########
@@ -494,9 +515,31 @@ public Stream<FieldMetrics<?>> metrics() {
}
}
- // skipping updating null stats for non-primitive types since we don't
use them today, to
- // avoid unnecessary work
- return writer.metrics();
+ // A null value here is also null for every descendant column, but those
columns are written
+ // directly and never see it, so their writers cannot count it. Add it
to the metrics of
+ // optional descendants; null counts are not tracked for required fields.
+ return writer
+ .metrics()
+ .map(
+ metrics ->
+ optionalFieldIds.contains(metrics.id()) ?
addNullValues(metrics) : metrics);
Review Comment:
You were reviewing an old diff. I have removed this in the latest PR to
align with our latest discussion. Please check again. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
