kosiew commented on code in PR #22960:
URL: https://github.com/apache/datafusion/pull/22960#discussion_r3427718360
##########
datafusion/datasource-parquet/src/row_filter.rs:
##########
@@ -858,36 +852,25 @@ fn prune_struct_type(dt: &DataType, paths: &[&[String]])
-> DataType {
return dt.clone();
};
- let needed = paths
- .iter()
- .filter_map(|p| p.first().map(|s| s.as_str()))
- .collect::<BTreeSet<_>>();
+ let paths_by_field = paths.iter().filter_map(|path|
path.split_first()).fold(
Review Comment:
This grouping logic now shows up at both schema levels with similar
`BTreeMap<_, Vec<_>>` construction. It might be worth adding a small local
helper for grouping paths by their next component, so the invariant, group once
per schema level and then recurse, lives in one place. I think that would also
make the `fold` with the mutable accumulator a bit easier to scan.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
