hililiwei commented on code in PR #4246:
URL: https://github.com/apache/iceberg/pull/4246#discussion_r995502246
##########
flink/v1.15/flink/src/main/java/org/apache/iceberg/flink/sink/FlinkSink.java:
##########
@@ -495,19 +502,27 @@ private DataStream<RowData>
distributeDataStream(DataStream<RowData> input,
}
static RowType toFlinkRowType(Schema schema, TableSchema requestedSchema) {
- if (requestedSchema != null) {
+ return toFlinkRowType(schema, requestedSchema, null);
+ }
+
+ static RowType toFlinkRowType(Schema schema, TableSchema tableSchema,
ResolvedSchema resolvedTableSchema) {
+ if (resolvedTableSchema != null) {
// Convert the flink schema to iceberg schema firstly, then reassign ids
to match the existing iceberg schema.
- Schema writeSchema =
TypeUtil.reassignIds(FlinkSchemaUtil.convert(requestedSchema), schema);
+ Schema writeSchema =
TypeUtil.reassignIds(FlinkSchemaUtil.convert(resolvedTableSchema), schema);
TypeUtil.validateWriteSchema(schema, writeSchema, true, true);
// We use this flink schema to read values from RowData. The flink's
TINYINT and SMALLINT will be promoted to
// iceberg INTEGER, that means if we use iceberg's table schema to read
TINYINT (backend by 1 'byte'), we will
// read 4 bytes rather than 1 byte, it will mess up the byte array in
BinaryRowData. So here we must use flink
// schema.
- return (RowType) requestedSchema.toRowDataType().getLogicalType();
- } else {
- return FlinkSchemaUtil.convert(schema);
+ return (RowType)
resolvedTableSchema.toPhysicalRowDataType().getLogicalType();
Review Comment:
We don't seem to need computed, and metadata columns. As toRowDataType
comments, toPhysicalRowDataType() might be more appropriate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
