[
https://issues.apache.org/jira/browse/FLINK-40298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40298:
-----------------------------------
Labels: pull-request-available (was: )
> Opt-in name-based state schema evolution for RowData
> ----------------------------------------------------
>
> Key: FLINK-40298
> URL: https://issues.apache.org/jira/browse/FLINK-40298
> Project: Flink
> Issue Type: Sub-task
> Components: API / Type Serialization System, Table SQL / Runtime
> Reporter: Weiqing Yang
> Priority: Major
> Labels: pull-request-available
>
> The core of FLIP-527. Adds:
> * A new option table.exec.state.schema-evolution.enabled (Boolean, default
> false).
> * Name-based compatibility in
> RowDataSerializer.RowDataSerializerSnapshot.resolveSchemaCompatibility: when
> the field layouts differ but the difference is a supported
> backward-compatible change, it returns compatibleAfterMigration() instead of
> incompatible(). Fields are matched by the names persisted by FLINK-40120.
> Added fields must be nullable, removed fields and changed leaf types are
> rejected, and nested ROW fields are validated by recursion.
> * An override of migrate that remaps a row into the new layout: fields
> placed by name, added fields null-filled, row kind preserved, nested ROW
> values remapped recursively.
> The option is carried as an in-memory, non-persisted flag on the RowData
> serializer rather than as anything written into the snapshot. The flag is set
> only on a serializer that is a state's own value serializer, at the point the
> state descriptor initializes it. Consequently a RowData serializer that sits
> below a composite serializer (List or Tuple, as in interval and outer join
> buffers), or one an operator pre-builds itself, never carries the flag and is
> rejected on restore rather than migrated. This matters for correctness:
> migration is not propagated into nested serializers by composite snapshots,
> so those shapes must fail closed. A test asserts this rejection explicitly.
> Snapshots written before field names were persisted have no names and are
> likewise rejected.
> Depends on FLINK-40296.
> FLIP-527:
> https://cwiki.apache.org/confluence/spaces/FLINK/pages/353601981/FLIP-527+State+Schema+Evolution+for+RowData
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
