nssalian commented on code in PR #17424:
URL: https://github.com/apache/iceberg/pull/17424#discussion_r3685770930
##########
parquet/src/main/java/org/apache/iceberg/parquet/VariantShreddingAnalyzer.java:
##########
@@ -51,10 +51,10 @@
*
* <ul>
* <li>Object fields are emitted in alphabetical order in the shredded
schema.
- * <li>Type selection picks the most common type with explicit tie-break
priority (see {@link
- * FieldInfo#TIE_BREAK_PRIORITY}), not enum ordinal.
+ * <li>A field is admitted only if all observations fall into a single type
family after numeric
+ * widening; mixed-type fields remain in the residual {@code value}.
* <li>Integer types (INT8/16/32/64) and decimal types (DECIMAL4/8/16) are
each promoted to the
- * widest observed before competing with other types.
+ * widest observed within their family.
* <li>Fields below {@code MIN_FIELD_FREQUENCY} are pruned. Above {@code
MAX_SHREDDED_FIELDS}, the
Review Comment:
FLOAT -> DOUBLE widening is lossless, but Iceberg's writer preserves the
exact original type via the residual. Keeping them separate keeps that contract
consistent, and matches TIMESTAMPTZ vs TIMESTAMPTZ_NANOS. I added a
`testMixedFloatAndDoubleNotShredded` for it. Widening-on-write would be its own
change, but not sure I'm seeing the benefit yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
