danielcweeks commented on code in PR #16777: URL: https://github.com/apache/iceberg/pull/16777#discussion_r3463370440
########## format/spec.md: ########## @@ -330,7 +331,15 @@ The `initial-default` is set only when a field is added to an existing schema. T The `initial-default` and `write-default` produce SQL default value behavior, without rewriting data files. SQL default value behavior when a field is added handles all existing rows as though the rows were written with the new field's default value. Default value changes may only affect future records and all known fields are written into data files. Omitting a known field when writing a data file is never allowed. The write default for a field must be written if a field is not supplied to a write. If the write default for a required field is not set, the writer must fail. -All columns of `unknown`, `variant`, `geometry`, and `geography` types must default to null. Non-null values for `initial-default` or `write-default` are invalid. +Starting in v4, a field's `write-default` may be a [value expression](expressions-spec.md) rather than a literal. This allows defaults such as `current_timestamp()` to be evaluated when a row is written. A value expression `write-default` is subject to the following requirements: + +* The expression must be a constant or a function application (apply); [field references](expressions-spec.md#field-reference) (bound or unbound) are not allowed in a default value expression +* The expression must produce a value of the field's type, subject to [type promotion](#schema-evolution) +* The `write-default` expression is evaluated to populate the field for any record written after the field was added when the writer does not supply the field's value + +The `initial-default` must always be a constant (a single value); it is not allowed to be a function application. A literal is itself a value expression, so existing `write-default` values remain valid value expressions. In format version 3, both `initial-default` and `write-default` must be constants. + +All columns of `unknown`, `variant`, `geometry`, and `geography` types must default to null. Non-null `initial-default` or `write-default` values, including value expressions, are invalid. Review Comment: The rational for the `initial-default` as a constant is that you cannot accurately reproduce the value of an expression evaluation (like CURRENT_DATE). The behavior would be that if someone used that as the default, the result of evaluating the function would be captured and stored as the `initial-default` (as opposed to always reevaluating it on read). The expression would be stored as the `write-default` because that should be evaluated on write. The the type restrictions, it's mostly to limit scope, but we could relax this for geo types. I'd like to know better what the utility would be for `unknown` or `variant` types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
