ConeyLiu commented on code in PR #4577:
URL: https://github.com/apache/iceberg/pull/4577#discussion_r851855315
##########
core/src/main/java/org/apache/iceberg/avro/BuildAvroProjection.java:
##########
@@ -106,10 +106,16 @@ public Schema record(Schema record, List<String> names,
Iterable<Schema.Field> s
"Missing required field: %s", field.name());
// Create a field that will be defaulted to null. We assign a unique
suffix to the field
// to make sure that even if records in the file have the field it is
not projected.
+ String origFieldName = field.name();
Review Comment:
This problem only happens on metadata table, because we use a common
partition schema for metadata table. For example:
```scala
// create table
spark.sql("create table local.db.test_table(id long, data struct<id: long>)
using iceberg partitioned by (identity(id))").show()
// manifest 0
spark.range(10).map(i => Item(i,
Data(i))).writeTo("local.db.test_table").append()
// update table to add a partiition field on the nested field
spark.sql("alter table local.db.test_table add partition field
identity(data.id)").show()
// manifest 1
spark.range(10).map(i => Item(i,
Data(i))).writeTo("local.db.test_table").append()
spark.sql("select * from local.db.test_table.files").show()
```
The common partition schema is `(Identity(id), Identity(data.id))` when we
used the common schema projecting on the manifest 0, we could not find the
field(data.id) from the Avro schema which read from the manifest file. Then we
need to create a new Avro field from the missing one which is why we reached
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
