Hi devs, RFC-69 proposes some exciting features and in line with that vision, I would like to propose support for multi-table transactions in Hudi.
As the name suggests, this would enable transactional consistency across multiple tables, i.e. a set of changes to multiple tables either completely succeeds or completely fails. This could be helpful for use cases such as updating details about a sales order that affects 2 or more tables, deleting records for a customer across 2 or more tables, etc. Hudi already provides ACID guarantees on a single table and tunable concurrency control. We would need to build additional orchestration or consistency mechanisms on top of existing mechanisms. I would like to put more details in a separate RFC. However, the high-level goal is to provide the same guarantees as Hudi provides for a single table and should work with both kinds of concurrency control OCC and MVCC. Looking forward to hearing some thoughts from you all. Regards, Sagar