CryoThrust commented on issue #13223: URL: https://github.com/apache/seatunnel/issues/13223#issuecomment-6919213948
I'd like to take the **Hudi / current AT_LEAST_ONCE writer** row, covering INSERT, UPSERT and BULK_INSERT through both the direct-client and multi-table resource-manager paths. I found no existing Hudi claim in this thread or separate replica-capability PR. I checked `dev` at `37309e1a8` before proposing the scope: - `HudiSinkWriter` inherits the empty `primaryKey()` default. Its resource manager caches a separate write client per table and queue index; it does not serialize all writers to one table through one client. - `HudiRecordConverter` derives identity from `record_key_fields` and the partition path, rather than the catalog primary key. INSERT without configured key fields generates UUIDs, while configured keys can contain multiple fields. - `HudiRecordWriter` handles DELETE and UPDATE_BEFORE as deletes independently of `op_type`. Consequently, INSERT/BULK_INSERT alone is not evidence that unordered RANDOM routing is safe. Each replica also buffers changes independently and starts its own Hudi commits. I will verify both record ordering and Hudi's concurrent-client/timeline behavior before enabling a mode. Adding a routing key alone would not establish that concurrent commits are safe. The destination checks will compare complete final values and deleted keys at replicas 1, 2 and 4, including repeated updates, delete/reinsert, multiple tables, configured composite keys and same-layout recovery where supported. An append-only RANDOM declaration would need its own enforced input contract and destination evidence. I'll build the connector declaration on the merged Phase 1 API, keep unverified modes UNSUPPORTED, and document the results and any remaining prerequisite rather than enabling them speculatively. This does not take over #12440 or #13224's exactly-once/state work, and it stays separate from #13264's authentication changes. Please record this Hudi audit under my name if the scope fits. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
