RussellSpitzer commented on code in PR #17424:
URL: https://github.com/apache/iceberg/pull/17424#discussion_r3679142695
##########
parquet/src/main/java/org/apache/iceberg/parquet/VariantShreddingAnalyzer.java:
##########
@@ -51,10 +51,10 @@
*
* <ul>
* <li>Object fields are emitted in alphabetical order in the shredded
schema.
- * <li>Type selection picks the most common type with explicit tie-break
priority (see {@link
- * FieldInfo#TIE_BREAK_PRIORITY}), not enum ordinal.
+ * <li>A field is admitted only if all observations fall into a single type
family after numeric
+ * widening; mixed-type fields remain in the residual {@code value}.
* <li>Integer types (INT8/16/32/64) and decimal types (DECIMAL4/8/16) are
each promoted to the
- * widest observed before competing with other types.
+ * widest observed within their family.
* <li>Fields below {@code MIN_FIELD_FREQUENCY} are pruned. Above {@code
MAX_SHREDDED_FIELDS}, the
Review Comment:
If we don't want to include them we probably should just have an explict
mixed float/double column isn't shredded test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
