wombatu-kun commented on code in PR #16606:
URL: https://github.com/apache/iceberg/pull/16606#discussion_r3691312975
##########
kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/SchemaUtils.java:
##########
@@ -306,7 +306,15 @@ Type inferIcebergType(Object value) {
return BooleanType.get();
} else if (value instanceof BigDecimal) {
BigDecimal bigDecimal = (BigDecimal) value;
- return DecimalType.of(bigDecimal.precision(), bigDecimal.scale());
+ int scale = bigDecimal.scale();
+ int precision = bigDecimal.precision();
+ // BigDecimal may use a negative scale (e.g. "1E+2" has scale -2)
+ if (scale < 0) {
+ precision -= scale;
+ scale = 0;
+ }
+ // a value < 1 may have precision < scale (e.g. "0.001"); widen
precision to the scale
Review Comment:
Spark does not pad. `Decimal.set(scala.math.BigDecimal)` sets `_precision =
scale` when `precision < scale`, so `0.001` is decimal(3, 3) there too, and for
`scale < 0` it sets `_precision = precision - scale, _scale = 0` - the same
normalization on both branches as this PR. `JsonInferSchema` also uses
`max(precision, scale)`, falling back to `DoubleType` past `MAX_PRECISION`
rather than padding. Keeping the tight type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
