uros-b commented on code in PR #17578:
URL: https://github.com/apache/iceberg/pull/17578#discussion_r3966042374
##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetSchemaUtil.java:
##########
@@ -159,6 +159,12 @@ public static MessageType pruneColumnsFallback(MessageType
fileSchema, Schema ex
int ordinal = 1;
for (Type type : fileSchema.getFields()) {
if (selectedIds.contains(ordinal)) {
+ Types.NestedField expectedField = expectedSchema.findField(ordinal);
Review Comment:
Nested types in the fallback path - ParquetSchemaUtil.java:162-171
(pruneColumnsFallback) validates only top-level primitives: the
type.isPrimitive() && expectedField.type().isPrimitiveType() guard skips any
top-level struct/list/map that contains a nested geometry/geography field, so
CRS/algorithm go unchecked on the id-less path (the id-based PruneColumns
visitor reaches nested types fine via recursive descent). Either recurse
positionally against the projected type, or add an explicit comment documenting
the top-level-only scope.
##########
parquet/src/main/java/org/apache/iceberg/parquet/PruneColumns.java:
##########
@@ -162,9 +164,22 @@ public Type variant(
@Override
public Type primitive(
org.apache.iceberg.types.Type.PrimitiveType expected, PrimitiveType
primitive) {
+ validatePrimitive(expected, primitive);
return null;
}
+ static void validatePrimitive(
+ org.apache.iceberg.types.Type.PrimitiveType expected, PrimitiveType
primitive) {
+ if (expected != null
+ && (expected.typeId() == TypeID.GEOMETRY || expected.typeId() ==
TypeID.GEOGRAPHY)) {
+ Preconditions.checkArgument(
+
TypeUtil.isPromotionAllowed(MessageTypeToType.convertPrimitive(primitive),
expected),
+ "Cannot read Parquet type %s as Iceberg type %s",
Review Comment:
Column name in the error - PruneColumns.java:179: "Cannot read Parquet type
%s as Iceberg type %s" omits the column name, so on a wide table the user can't
tell which field mismatched. TypeWithSchemaVisitor exposes currentPath() (the
fieldNames deque, ~line 254) that primitive() can use to include the path, per
AGENTS.md:119 (actionable messages).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
