The proposal looks great to me, I also prefer the "Automatic Transaction Wrapping" approach.
For the "Explicit Transaction Management", we may use the `ProcessingGuarantees.MANUAL` to disable the automatic acknowledgment？ Regards, Pengcheng Jiang Lari Hotari <[email protected]> 于2025年8月5日周二 00:47写道： > This proposal sounds very useful. Thank you for taking the initiative to > address this gap in Pulsar Functions. > > When thinking of a phased approach for implementation, I'd recommend > prioritizing the "automatic transaction wrapping" approach since it > wouldn't require changes to existing public APIs other than adding a > configuration option for enabling this. After the "automatic transaction > wrapping" has been implemented, the second phase could be to add the > explicit transaction management support. > > The benefit of implementing in phases would be that we could simplify the > implementation and get the first phase delivered more quickly than > implementing both approaches at once. There might also be a chance to > backport this feature to Pulsar 4.0.x LTS if we don't change public > interfaces. > > -Lari > > On 2025/07/31 13:06:19 Ramzan MAGOMADOW wrote: > > Hi Pulsar community, > > > > I'm planning to work on adding transactional support to Pulsar Functions > and would like to start a discussion about the approach before creating a > formal PIP. > > > > Background > > > > Currently, Pulsar Functions cannot publish to multiple topics > transactionally, which is a significant limitation for use cases requiring > atomic multi-topic publishing. As noted in the Pulsar Transactions > documentation<https://pulsar.apache.org/docs/4.0.x/txn-what/#use-case>, > "Support for Pulsar Functions and other connectors will be added in future > releases." > > > > Proposed Approach > > > > I've outlined a context-based approach in GitHub issue #24588< > https://github.com/apache/pulsar/issues/24588> that would add transaction > support through the existing Function Context interface. > > > > The key ideas include: > > > > 1. > > Explicit Transaction Management > > 2. > > > > context.beginTransaction(); > > try { > > context.newOutputMessage("topic-a", > Schema.STRING).value("abc").send(); > > context.newOutputMessage("topic-b", Schema.INT32).value(1).send(); > > context.transactionallyAcknowledgeRecord(); > > context.commitTransaction(); > > } catch (Exception e) { > > context.rollbackTransaction(); > > throw e; > > } > > > > This approach would require turning off automatic acknowledgment and > letting the context handle record acknowledgment transactionally. > > 3. > > Automatic Transaction Wrapping > > A configuration option to automatically wrap function execution in a > transactional context, eliminating the need for explicit transaction > management in user code. > > > > Any feedback, suggestions, or experiences from the community would be > greatly appreciated before I proceed with the formal proposal. Has anyone > else been working on similar functionality or have thoughts on this > approach? > > > > Best regards, > > Ramzan Magomadow > > This message and any attachment ("the Message") are confidential. If you > have received the Message in error, please notify the sender immediately > and delete the Message from your system, any use of the Message is > forbidden. Correspondence via e-mail is primarily for information purposes. > RBI neither makes nor accepts legally binding statements via e-mail unless > explicitly agreed otherwise. Information pursuant to ? 14 Austrian > Companies Code: Raiffeisen Bank International AG; Registered Office: Am > Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN 122119m at > the Commercial Court of Vienna (Handelsgericht Wien). > > > > Classification: GENERAL > > > -- This email and any attachments are intended solely for the recipient(s) named above and may contain confidential, privileged, or proprietary information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reproduction of this information is strictly prohibited. If you have received this email in error, please notify the sender immediately by replying to this email and delete it from your system.
