nastra commented on PR #17412: URL: https://github.com/apache/iceberg/pull/17412#issuecomment-5143659925
> > Parquet produces metrics for leaf columns only in ParquetMetrics.MetricsVisitor.struct and returns just the concatenation of its children and never emits a FieldMetrics for the struct — so no Parquet file yields those entries. > > This is true for Parquet, but do we have anything in the spec which would prevent other file formats fill out these metrics? Shall we enforce this behaviour through the spec, or handle the struct metrics when they exist in the stats correctly? That is a good point but we don't have anything in the spec or in the implementation that would prevent this. The spec and the read path don't forbid struct-level value/null counts. This PR matches Parquet (and v4 Java writer) reality and the motivation is to extend this test class and use the content stats evaluator and verify that the same results are produced, which isn't the case for this particular scenario because content stats don't keep stats for structs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
