wombatu-kun commented on code in PR #16606:
URL: https://github.com/apache/iceberg/pull/16606#discussion_r3691335518
##########
kafka-connect/kafka-connect/src/test/java/org/apache/iceberg/connect/data/TestSinkWriter.java:
##########
@@ -235,6 +237,26 @@ public void testDynamicNoRoute() {
assertThat(writerResults).hasSize(0);
}
+ @Test
+ public void testEvolveAddsFractionalDecimalColumn() {
+ IcebergSinkConfig config = mock(IcebergSinkConfig.class);
+ when(config.tableConfig(any())).thenReturn(mock(TableSinkConfig.class));
+
when(config.tables()).thenReturn(ImmutableList.of(TABLE_IDENTIFIER.toString()));
+ when(config.evolveSchemaEnabled()).thenReturn(true);
+
+ // a new column whose value is a fractional decimal < 1:
BigDecimal("0.001") has precision 1
+ // and scale 3, so before the fix the column evolves to a malformed
decimal(1, 3) and the
+ // write below fails; after the fix it evolves to decimal(3, 3) and the
record is written.
Review Comment:
`DecimalType.of(1, 3)` does not throw. The constructor carries a single
precondition, `precision <= 38`
(api/src/main/java/org/apache/iceberg/types/Types.java:526-529) - no `scale <=
precision` check and no negative-scale check, which is why this reaches the
writer at all. `Invalid DECIMAL scale: 3 cannot be greater than precision: 1`
is Parquet's message: reverting the fix locally fails at
`parquet.schema.Types$BasePrimitiveBuilder.decimalMetadata`, via
`iceberg.parquet.TypeToMessageType.primitive`, so the evolution does commit and
the write is reached. Reworded in e081d1513 to say why the failure is deferred,
since that is easy to misread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
