laskoviymishka commented on code in PR #13879:
URL: https://github.com/apache/iceberg/pull/13879#discussion_r3685044197
##########
open-api/rest-catalog-open-api.yaml:
##########
@@ -3833,6 +3843,317 @@ components:
additionalProperties:
type: string
+ ReadRestrictions:
+ type: object
+ description: >
+ Read restrictions for a table.
+
+ A reader evaluates the row filter against original, untransformed
column
+ values, then applies required-column-projections to the surviving
rows.
+ Each action must produce a value of the same type as the input
column.
+ If a reader that supports read-restrictions cannot apply any returned
+ restriction (a filter expression or an action), it must fail the
query
+ and must not silently return raw, partial, or empty results.
+
+ If a projection targets a nested-typed field (struct, list, or map),
+ other projections in the same ReadRestrictions must not target any
+ nested field-id (struct subfields, list elements, or map keys/values)
+ at any depth. This specification does not define how such actions
+ combine. A reader that receives such a response must fail the query.
+
+ A missing or empty ReadRestrictions object (no
required-column-projections and no
+ required-row-filter) imposes no restrictions.
+ example:
+ required-column-projections:
+ - field-id: 4
+ action: show-last-4
+ - field-id: 6
+ action: replace-with-null
+ - field-id: 8
+ action: truncate-to-year
+ - field-id: 10
+ action: sha-256-global
+ - field-id: 12
+ action: mask-alphanum
+ required-row-filter:
+ type: eq
+ left:
+ type: reference
+ id: 14
+ right:
+ type: literal
+ value: "US"
+ properties:
+ required-column-projections:
+ description: >
+ A list of columns that require specific actions to be applied when
reading.
+ A server must not return an action for a column whose type is not
listed in
+ that action's "Applicable to" set. If absent or empty, no required
actions
+ apply; columns not listed are not subject to any required action.
+
+ 1. For each column listed, the reader must apply the specified
action before
+ returning values for that column.
+
+ 2. The reader must replace all output references to the column
with the result
+ of the action, presenting the result under the original
field-id. For
+ example, if the action for field-id `9` is mask-alphanum, the
reader must
+ return the masked value as field-id `9` in the query output.
+
+ 3. A server must not return more than one projection for the same
field-id
+ in required-column-projections. If a duplicate field-id appears,
the reader
+ must fail the query.
+
+ 4. A projection must not target a map's key field-id. Applying an
action
+ to keys can produce duplicate or null keys, which readers
silently
+ coalesce or reject, causing data loss.
+
+ 5. A reader must enforce projections on the columns it is actually
reading.
+ Projections referencing columns that are not being read do not
apply.
+ type: array
+ items:
+ $ref: '#/components/schemas/Action'
+ required-row-filter:
+ description: >
+ An expression that limits which rows the reader may return.
+
+ 1. The expression must evaluate to a boolean (TRUE or FALSE;
Iceberg predicates
+ never produce NULL). A reader must discard any row for which the
filter
+ evaluates to FALSE, and no information derived from discarded
rows may be
+ included in the query result.
+
+ 2. If this property is absent, null, or always true then no
mandatory filtering is required.
+
+ 3. Column references within the expression must use field IDs
(IdReference),
Review Comment:
Right, and if “can’t interpret” already includes “a referenced field ID does
not exist in the schema being read,” then this is already covered. That is
mostly what I’m trying to confirm.
The case I had in mind is:
`required-row-filter = country_id (field 14) = 'US'`
The policy is valid against the current schema, but a reader time-travels to
an older snapshot from before field 14 was added, or reads a schema where field
14 has since been dropped.
The expression is still well-formed and parseable. The question is what the
reader does when it cannot resolve field 14:
* fail the read
* treat the missing field as null, so the predicate evaluates to false
* ignore the term, which would return all rows
The last option is the one I’m worried about.
It sounds like your expectation is that the read fails, which matches mine.
I just wasn’t sure the current “can’t interpret” wording clearly covers a known
expression type that references a field missing from the read schema.
Am i reading this right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
