QuakeWang commented on code in PR #54:
URL: https://github.com/apache/paimon-mosaic/pull/54#discussion_r3371016399
##########
core/src/bucket_reader.rs:
##########
@@ -393,16 +393,184 @@ fn reassemble_list_columns(
// where the first child's element_field is NOT itself a complex type
// (complex element_field means it's a nested ARRAY/MAP intermediate).
+ // Skip children that are part of a STRUCT group (handled when we
reach the __struct_null__ child)
+ let is_struct_field_child = idx > 0
+ && children.iter().any(|c| {
+ c.parent_logical_col == parent
+ && c.physical_index < phys_idx
+ && c.element_field.name() == "__struct_null__"
+ && !processed[children
+ .iter()
+ .position(|x| std::ptr::eq(x, c))
+ .unwrap_or(usize::MAX)]
+ });
+ if is_struct_field_child {
+ continue;
+ }
+
+ // Check if this is a __struct_null__ child — if so, reassemble the
STRUCT
+ if child.element_field.name() == "__struct_null__" {
+ // Collect all subsequent children with same parent that aren't
processed
+ let mut struct_field_indices: Vec<usize> = Vec::new();
+ for si in (idx + 1)..children.len() {
Review Comment:
Collecting “all following unprocessed children with the same parent” works
for a flat `ARRAY<STRUCT<a, b>>`, but it flattens nested STRUCT fields.
For `ARRAY<STRUCT<name, addr: STRUCT<city, zip>>>`, the child layout is
roughly:
```rust
__struct_null__, name, addr.__struct_null__, city, zip
```
This code collects `addr.__struct_null__`, `city`, and `zip` as fields of
the outer STRUCT, so readback becomes `STRUCT<name, __struct_null__, city,
zip>` instead of `STRUCT<name, addr: STRUCT<city, zip>>`, and
`RecordBatch::try_new` fails with a schema mismatch.
I think this should rebuild STRUCT elements by recursively consuming child
columns from the original `DataType::Struct`, and add regressions for
`ARRAY<STRUCT<nested STRUCT>>` and `MAP<K, STRUCT<nested STRUCT>>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
