SEZ9 commented on issue #13223: URL: https://github.com/apache/seatunnel/issues/13223#issuecomment-6842079748
@goutamadwant Please open the PR now; the `byte[]` content hash and the `primaryKey()` re-read after the schema-change barrier are independent of the capability contract, so there is no reason to wait for Phase 1. Two asks for #13242: (1) keep the routing-path changes in `MultiTableSinkWriter` minimal so the rebase onto #13243 is mechanical, since #13243 replaces the queue-selection logic; (2) in the tests, cover a rename of the key column and a column added/dropped before the key column, so we see the recomputed index is used for the first record after the barrier, and show that the `byte[]` hash gives the same queue for equal content across replicas. @SEPURI-SAI-KRISHNA Agreed on the three rows, with these conditions: - **Iceberg.** Delta writers as `KEYED` on the identifier/primary-key column is the right call; the replica-2 numbers you measured on `535d4237dd` are exactly the failure mode the contract is meant to reject. For the append writers, do not declare `RANDOM` unless you can show there is no per-key ordering dependency and no delete path at all; if there is any doubt, leave them at the default `UNSUPPORTED` and note it in the PR. Please include the `IcebergSinkCDCIT` replica-2 case and the repeated-changes-of-one-key case, and rebase on #12881 before marking ready. - **JDBC / exactly-once (XA).** The separate XIDs and index-scoped `recoverAndRollback` are necessary but not sufficient; the writer still has no routing key, so it must stay `UNSUPPORTED` until #13242 and #13109 land and it can declare `KEYED` with a real `primaryKey()`. The restore test with the same layout at replica 2 is required for this row. - **JDBC / normal, UNIQUE-key gap.** Routing by the key the sink actually upserts by is correct; the writer's `primaryKey()` must be derived from the same `primary_keys` value the factory sets, not re-read from the schema primary key. Please also add a negative case: a table with neither a primary key nor a UNIQUE key must be rejected at replica > 1 and still work at replica 1. The E2E matrix you listed (successive updates, delete then reinsert, repeated upserts, replicas 1, 2 and 4) is what I want to see, built on top of #13242. Sequencing: #13243 first, then the connector rows. Please update the row table in this issue as each PR opens so we can track per-connector status. <!-- streview-comment:2309 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
