alkis commented on code in PR #603:
URL: https://github.com/apache/parquet-format/pull/603#discussion_r3814405625
##########
LogicalTypes.md:
##########
@@ -735,41 +732,35 @@ only.
A value resolves to bytes based on which of `inline`, `uri`, `offset`, and
`size` are
set:
-| `inline` | `uri` | `offset` | `size` | Resolves to
|
-|----------|-------|----------|--------|-------------------------------------------------------|
-| set | - | - | - | the inline bytes
|
-| - | set | - | - | whole external file at `uri`
|
-| - | set | set | - | invalid
|
-| - | set | - | set | external `uri`, `[0, size)`
|
-| - | set | set | set | external `uri`, `[offset, offset +
size)` |
-| - | - | set | - | invalid
|
-| - | - | - | set | invalid
|
-| - | - | set | set | this file, `[offset, offset + size)`
(self-reference) |
-| - | - | - | - | nothing - invalid
|
+| `inline` | `uri` | `offset` | `size` | Resolves to
|
+|----------|-------|----------|--------|-------------------------------------------|
+| set | - | - | - | the inline bytes
|
Review Comment:
Yes, allowed — specified in f4b79fe. This was a real gap: the text said
locator fields alongside `inline` are "provenance only" but the resolution
table did not cover the combination, so it was underspecified.
@rok no, not a preview slice. When both are set they must denote the *same*
bytes; the locator records where those bytes came from. Anything else would
give a value two different meanings depending on which representation you read.
The table row is now:
| `inline` | `uri` | `offset` | `size` | Resolves to |
|----------|-------|----------|--------|------------------|
| set | any | any | any | the inline bytes |
with the `inline` field description saying the two must denote the same
bytes and that a locator must not be a partial or otherwise different
representation, and a matching validation bullet. `inline` always wins for
reading, so a reader never has to reconcile the two.
##########
LogicalTypes.md:
##########
@@ -735,41 +732,35 @@ only.
A value resolves to bytes based on which of `inline`, `uri`, `offset`, and
`size` are
set:
-| `inline` | `uri` | `offset` | `size` | Resolves to
|
-|----------|-------|----------|--------|-------------------------------------------------------|
-| set | - | - | - | the inline bytes
|
-| - | set | - | - | whole external file at `uri`
|
-| - | set | set | - | invalid
|
-| - | set | - | set | external `uri`, `[0, size)`
|
-| - | set | set | set | external `uri`, `[offset, offset +
size)` |
-| - | - | set | - | invalid
|
-| - | - | - | set | invalid
|
-| - | - | set | set | this file, `[offset, offset + size)`
(self-reference) |
-| - | - | - | - | nothing - invalid
|
+| `inline` | `uri` | `offset` | `size` | Resolves to
|
+|----------|-------|----------|--------|-------------------------------------------|
+| set | - | - | - | the inline bytes
|
+| - | set | - | - | whole external file at `uri`
|
+| - | set | set | - | invalid
|
+| - | set | - | set | external `uri`, `[0, size)`
|
+| - | set | set | set | external `uri`, `[offset, offset +
size)` |
+| - | - | set | - | invalid
|
+| - | - | - | set | invalid
|
+| - | - | set | set | invalid
|
+| - | - | - | - | nothing - invalid
|
`size` must be set whenever `offset` is set, so any offset-based read always
carries an
-explicit `size`. A self-reference (`uri` not set) must set `offset`, and
therefore also
-`size`. `size` may be omitted only for a whole-file external reference, where
the range
-runs to the end of the referenced file.
+explicit `size`. `size` may be omitted only for a whole-file external
reference, where
+the range runs to the end of the referenced file. A byte range within the
current file
+cannot be referenced: `offset` and `size` apply only to data referenced by
`uri`.
-A self-reference points within the same Parquet file using `offset` and `size`
(both
-required). A self-reference is when `uri` is not set. A file containing
self-references
-can be renamed or relocated as a single unit.
-
-Parquet files containing self-references must not use Parquet modular
encryption.
-Self-referenced byte ranges are not Parquet encryption modules and therefore
cannot
-be encrypted or authenticated independently. Encryption of external files
referenced
-by `uri` is outside the scope of the Parquet format.
+Encryption of external files referenced by `uri` is outside the scope of the
Parquet
+format.
#### Validation
-* A value must resolve to some referenced data. It resolves only if `inline`,
`uri`, or
- `offset` is set; if none of them are set, the value does not resolve and is
invalid, even
- if `size` is set.
-* A self-reference (`uri` not set) must set `offset`. A value with neither
`uri` nor
- `offset` set (and not `inline`) does not resolve and is invalid.
+* A value must resolve to some referenced data. It resolves only if `inline`
or `uri` is
+ set; if neither is set, the value does not resolve and is invalid, even if
`offset` or
+ `size` is set.
+* `offset` may only be set together with `uri`. A value that sets `offset`
without `uri`
+ (and not `inline`) does not resolve and is invalid.
Review Comment:
Accepted in f4b79fe. Redundant given the bullet above it already establishes
that `inline` alone resolves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
