stefan-grafberger commented on code in PR #635:
URL: https://github.com/apache/parquet-format/pull/635#discussion_r4470139634


##########
LogicalTypes.md:
##########
@@ -712,8 +713,9 @@ If set and non-zero, readers must seek to this offset to 
retrieve the referenced
 
 The byte length of the referenced data. Must be zero or a positive integer if 
set; a
 value of 0 indicates empty referenced data. `size` must be set whenever 
`offset` is set.
-It may be omitted only for a whole-file external reference (`uri` set, 
`offset` not set),
-in which case the range runs to the end of the referenced file.
+It may be omitted for a whole-file external reference (`uri` set, `offset` not 
set),
+in which case the range runs to the end of the referenced file. `size` may 
also be set
+with `inline` and without `uri`, in which case it must equal the length of 
`inline`.

Review Comment:
   Yes, good point, that applies when size is missing for some values. But I 
think the main benefit is that writers get the option to write size, and then a 
simple user query like `SELECT file.size FROM t` cheaply answers the size 
question for inline values too, without touching inline. Today that's not 
possible, since size isn't allowed with inline alone.
   
   An engine that wants to offer a function like `FILE_SIZE(file)` that always 
returns the size could rewrite it internally to `file.size` or 
`COALESCE(file.size, LENGTH(file.inline))`, depending on the null count of the 
size column. But that's optional; the spec change only makes cheap size queries 
possible for engines and writers that want them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to