dwsmith1983 commented on code in PR #5654:
URL: https://github.com/apache/datafusion-comet/pull/5654#discussion_r4100255410
##########
native/core/src/parquet/schema_adapter.rs:
##########
@@ -1009,10 +1083,84 @@ impl PhysicalExprAdapterFactory for
SparkPhysicalExprAdapterFactory {
id_duplicate_roots,
logical_folded,
physical_folded,
+ nested_mappings,
+ root_id_ambiguities,
}))
}
}
+/// Per logical field name, the mapping of its nested type against its
physical counterpart,
+/// or the ambiguity Spark reports for it. Only fields whose type holds a
struct are listed.
+type NestedMappings = HashMap<String, Result<Arc<FieldMapping>, SparkError>>;
+
+/// Per logical field name, the `_LEGACY_ERROR_TEMP_2094` ambiguity of a root
field whose id
+/// matches more than one physical root field. Only ambiguous fields are
listed.
+type RootIdAmbiguities = HashMap<String, SparkError>;
Review Comment:
The root change is dropped, so `remap_physical_schema` raises the duplicate
root id for the whole schema as it does on main and `root_id_ambiguities` is
gone. If a follow-up narrows that to referenced columns it will reuse
`id_duplicate_roots` as the one map and check each referenced column in one
pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
