Hi Pulsar community, I'm planning to work on adding transactional support to Pulsar Functions and would like to start a discussion about the approach before creating a formal PIP.
Background Currently, Pulsar Functions cannot publish to multiple topics transactionally, which is a significant limitation for use cases requiring atomic multi-topic publishing. As noted in the Pulsar Transactions documentation<https://pulsar.apache.org/docs/4.0.x/txn-what/#use-case>, "Support for Pulsar Functions and other connectors will be added in future releases." Proposed Approach I've outlined a context-based approach in GitHub issue #24588<https://github.com/apache/pulsar/issues/24588> that would add transaction support through the existing Function Context interface. The key ideas include: 1. Explicit Transaction Management 2. context.beginTransaction(); try { context.newOutputMessage("topic-a", Schema.STRING).value("abc").send(); context.newOutputMessage("topic-b", Schema.INT32).value(1).send(); context.transactionallyAcknowledgeRecord(); context.commitTransaction(); } catch (Exception e) { context.rollbackTransaction(); throw e; } This approach would require turning off automatic acknowledgment and letting the context handle record acknowledgment transactionally. 3. Automatic Transaction Wrapping A configuration option to automatically wrap function execution in a transactional context, eliminating the need for explicit transaction management in user code. Any feedback, suggestions, or experiences from the community would be greatly appreciated before I proceed with the formal proposal. Has anyone else been working on similar functionality or have thoughts on this approach? Best regards, Ramzan Magomadow This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to ? 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien). Classification: GENERAL
