Thanks, David for the confirmation. I went ahead and created the related PR: https://github.com/apache/parquet-format/pull/487 Anyone would have an opinion on this topic, feel free to comment in the PR.
Cheers, Gabor David Cashman <[email protected]> ezt írta (időpont: 2025. márc. 4., K, 16:28): > I agree, it is valid to omit `typed_value` anywhere in the shredding > schema, whether for an array element or object field (or at the top > level, which would degenerate the column to being unshredded). The > second sentence about value being required does not apply to objects: > object fields can have an optional value even if there's no > typed_value, since a missing value implies that the field doesn't > exist in that object. > > Thanks, > > David > > On Tue, Mar 4, 2025 at 4:50 AM Gábor Szádovszky <[email protected]> wrote: > > > > Hey, > > > > Sorry if I bring up something that has already been discussed but I could > > not find an answer to this. > > > > We have added the following to the spec of array shredding: > > "The typed_value field may be omitted when not shredding elements as a > > specific type. When typed_value is omitted, value must be required." > > > > But we did not do the same for objects. Is it deliberate? I think, > > similarly to the array case, there is value to allow shredding an object > > without shredding the value of some of its fields. > > > > Thanks, > > Gabor >
