raminqaf commented on code in PR #28123: URL: https://github.com/apache/flink/pull/28123#discussion_r3206884143
########## docs/content/release-notes/flink-2.3.md: ########## @@ -27,19 +27,282 @@ These release notes discuss important aspects, such as configuration, behavior o that changed between Flink 2.2 and Flink 2.3. Please read these notes carefully if you are planning to upgrade your Flink version to 2.3. +### Table SQL / API -### Core +#### FROM_CHANGELOG and TO_CHANGELOG built-in PTFs -#### Set security.ssl.algorithms default value to modern cipher suite +##### [FLINK-39258](https://issues.apache.org/jira/browse/FLINK-39258) (FLIP-564) -### [FLINK-39022](https://issues.apache.org/jira/browse/FLINK-39022) +The DataStream API has long offered `toChangelogStream()` and `fromChangelogStream()` for working +with changelog streams; Flink 2.3 brings equivalent functionality to SQL via two new built-in +Process Table Functions: -A JDK update (affecting JDK 11.0.30+, 17.0.18+, 21.0.10+, and 24+) disabled `TLS_RSA_*` cipher suites. -This was done to support forward-secrecy (RFC 9325) and comply with the IETF Draft on *Deprecating Obsolete Key Exchange Methods in TLS*. +- `FROM_CHANGELOG` converts an append-only stream that carries an operation column (and optional + before/after row descriptors) into a dynamic table. A configurable `op_mapping` makes it + straightforward to plug in custom CDC formats (e.g. Debezium-style `c`/`u`/`d` codes), and + `invalid_op_handling` (`FAIL`/`LOG`/`SKIP`) controls how rows with unmapped operation codes + are treated. +- `TO_CHANGELOG` is the inverse: it materializes a dynamic table back into an append-only + changelog stream. This is the first SQL-level operator that lets users convert + retract/upsert streams into append form, which is useful for archival, audit and writing to + append-only sinks. `produces_full_deletes` controls whether `-D` records carry the full row. -To support these and future JDK versions, the default value for the Flink configuration option `security.ssl.algorithms` has been changed to a modern, widely available cipher suite: +The two PTFs are designed to be symmetric, so `FROM_CHANGELOG(TO_CHANGELOG(table))` round-trips +correctly. Both support `PARTITION BY` for parallel execution and `uid` for query evolution, and +they expose a `state_ttl` parameter for state retention. Review Comment: Hey @twalthr — confirmed against `release-2.3`. ### `FROM_CHANGELOG` ([FLINK-39261]) - Append-only stream with an op column → updating dynamic table. - Default op codes (`INSERT`, `UPDATE_BEFORE`, `UPDATE_AFTER`, `DELETE`). - Custom op column name via `op => DESCRIPTOR(my_op)`. - Custom `op_mapping` with comma-separated keys. - Unmapped or `NULL` op codes fail with `TableRuntimeException`. No `error_handling` argument (the master-only follow-up that adds `FAIL`/`SKIP` was not cherry-picked). - Java `Table#fromChangelog(...)` and PyFlink `Table.from_changelog(...)` ([FLINK-39479]). - Row semantics only — `PARTITION BY` is rejected ([FLINK-39537] is open). - Retract output only — `UPDATE_AFTER` without `UPDATE_BEFORE` (upsert) is rejected. ### `TO_CHANGELOG` ([FLINK-39259], [FLINK-39349], [FLINK-39419]) - Dynamic table → append-only stream with an op column. - Custom op column name and `op_mapping`; deletion-flag pattern via comma-separated keys. - Filtering: only mapped operations forwarded. - Row semantics, full deletes, requires `UPDATE_BEFORE` upstream. - Java `Table#toChangelog(...)` and PyFlink `Table.to_changelog(...)` ([FLINK-39442]). - No `PARTITION BY` — `TO_CHANGELOG` set semantics ([FLINK-39614]) landed on master after the 2.3 cut. - Retract input only. ### Not in 2.3 - `state_ttl` argument — not supported yet, not declared on either PTF in any branch. - `FROM_CHANGELOG` `error_handling` ([FLINK-39495] + follow-up, master only). The can be back-ported to 2.3 - `FROM_CHANGELOG` `PARTITION BY` ([FLINK-39537], open). We are done with the work and we need to merge to master and backport to 2.3 - `TO_CHANGELOG` `PARTITION BY` ([FLINK-39614], master). Merged to master and needs to be backported to 2.3 - Conditional traits for PTFs ([FLINK-39392], master). needs to be backported to 2.3. - Upsert input/output for either PTF. ### Release-notes adjustments to consider - Drop the `state_ttl` mention — not declared on either PTF. - Drop `invalid_op_handling` / `LOG` — the parameter doesn't exist; even on master only `error_handling` (`FAIL`/`SKIP`) exists, and it's not in 2.3. - Drop `produces_full_deletes` — `TO_CHANGELOG` in 2.3 is locked to full deletes; no such argument is exposed. - Soften "Both support `PARTITION BY`" — neither does in 2.3 yet, unless we backport it and we will let you know. - The "round-trips correctly" claim holds only for the row-semantics retract case. [FLINK-39259]: https://issues.apache.org/jira/browse/FLINK-39259 [FLINK-39261]: https://issues.apache.org/jira/browse/FLINK-39261 [FLINK-39349]: https://issues.apache.org/jira/browse/FLINK-39349 [FLINK-39392]: https://issues.apache.org/jira/browse/FLINK-39392 [FLINK-39419]: https://issues.apache.org/jira/browse/FLINK-39419 [FLINK-39442]: https://issues.apache.org/jira/browse/FLINK-39442 [FLINK-39479]: https://issues.apache.org/jira/browse/FLINK-39479 [FLINK-39495]: https://issues.apache.org/jira/browse/FLINK-39495 [FLINK-39537]: https://issues.apache.org/jira/browse/FLINK-39537 [FLINK-39576]: https://issues.apache.org/jira/browse/FLINK-39576 [FLINK-39614]: https://issues.apache.org/jira/browse/FLINK-39614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
