Hi Lari! I've seen that you have already approved my PR, but I forgot to change the title to "PIP-439: Adding Transaction Support to Pulsar Functions Through Managed Transaction Wrapping" in the PIP-439.md. I'll add a comment about making the "pulsar_function_txn_latency" configurable as well, as you mentioned in your GitHub review. Finally, I'll squash the commits in my PR.
Is it okay to start the PIP voting even while I do those things? -Ramzan Classification: GENERAL On 2025/09/09 11:47:49 Lari Hotari wrote: > Great work, Ramzan! > > I think that we are ready to proceed to the PIP voting stage. Please initiate > a new vote thread. > > -Lari > > On 2025/09/08 08:41:54 Ramzan MAGOMADOW wrote: > > Hello Lari! > > > > Thank you for your valuable feedback and for merging the PR to support my > > proposal! > > > > As you suggested, I will rename the feature to "managed transactions" , > > which better reflects its nature, and will separate implementation details > > into a dedicated PR later. The proposal will also maintain consistency > > between transaction handling and the existing processing model to properly > > support asynchronous functions as well, thanks for pointing it out! > > > > I'll have the updated proposal ready soon. > > > > - Ramzan > > > > > > > > Classification: GENERAL > > > > On 2025/09/05 13:05:49 Lari Hotari wrote: > > > Thank you Ramzan for driving this improvement to Pulsar! Good work! > > > > > > Naming Suggestion: Could we call this "managed transactions" for > > > Pulsar Functions instead of "automatic transactions"? > > > > > > I provided a review with some suggestions: > > > https://github.com/apache/pulsar/pull/24704 > > > > > > It could be easier to review code in a separate implementation PR, > > > however, covering all implementation level details isn't necessary. > > > > > > It is useful to handle the main changes in the implementation and > > > describe the implementation strategy. > > > > > > For example, I added a comment about keeping the transactional > > > handling similar to current processing so that asynchronous functions > > > would also be supported. > > > > > > A poorly documented feature of Pulsar Functions is that it supports > > > asynchronous processing. This is really important in achieving higher > > > performance since it allows having more than 1 outstanding message in > > > processing. Async function support was added in > > > https://github.com/apache/pulsar/pull/6684. By default there can be > > > 1000 outstanding messages in processing and this can be controlled > > > with the "maxPendingAsyncRequests" setting. Supporting transactions > > > for asynchronous processing should also be part of the scope of this > > > PIP so that the implementation doesn't unnecessarily divert between > > > "managed transactions" and non-transactional processing. > > > > > > One interesting implementation detail is that this work requires the > > > PIP-407 changes > > > https://github.com/apache/pulsar/blob/master/pip/pip-407.md that just > > > landed in the Pulsar master branch with > > > https://github.com/apache/pulsar/pull/23942. > > > > > > Looking forward to seeing PIP-439 move forward! > > > > > > -Lari > > > > > > > > > On Fri, 5 Sept 2025 at 11:53, Ramzan MAGOMADOW > > > > > > <[email protected]<mailto:[email protected]>>> wrote: > > > > > > > > Dear Pulsar Community! > > > > > > > > I would like to start a formal discussion on PIP-439. > > > > > > > > This proposal aims to enhance Pulsar Functions with transaction > > > > capabilities, enabling atomic operations across multiple topics without > > > > requiring code changes to function implementations. The proposal builds > > > > on Pulsar's existing transaction system to provide exactly-once > > > > processing semantics for Functions. > > > > > > > > Key aspects of the proposal include: > > > > 1. Automatic transaction wrapping for Functions through configuration > > > > settings > > > > 2. Support for transactional publishing to multiple output topics > > > > 3. Transactional acknowledgment of input messages > > > > 4. Transaction timeout configuration > > > > > > > > This proposal continues the discussion started in the previous thread: > > > > https://lists.apache.org/thread/rll8qyovpd7t9v5yxth25qo44zksbgkn > > > > > > > > The complete proposal is available at: > > > > https://github.com/apache/pulsar/pull/24704 > > > > > > > > I welcome your feedback, questions, and suggestions on this proposal. > > > > Please share your thoughts on the design approach, implementation > > > > details, or any considerations I may have overlooked. > > > > > > > > Best regards, > > > > Ramzan Magomadow > > > > This message and any attachment ("the Message") are confidential. If > > > > you have received the Message in error, please notify the sender > > > > immediately and delete the Message from your system, any use of the > > > > Message is forbidden. Correspondence via e-mail is primarily for > > > > information purposes. RBI neither makes nor accepts legally binding > > > > statements via e-mail unless explicitly agreed otherwise. Information > > > > pursuant to ? 14 Austrian Companies Code: Raiffeisen Bank International > > > > AG; Registered Office: Am Stadtpark 9, 1030 Vienna, Austria; Company > > > > Register Number: FN 122119m at the Commercial Court of Vienna > > > > (Handelsgericht Wien). > > > > > > > > Classification: GENERAL > > > > > > > Sent from Outlook for Mac > > > > This message and any attachment ("the Message") are confidential. If you > > have received the Message in error, please notify the sender immediately > > and delete the Message from your system, any use of the Message is > > forbidden. Correspondence via e-mail is primarily for information purposes. > > RBI neither makes nor accepts legally binding statements via e-mail unless > > explicitly agreed otherwise. Information pursuant to ? 14 Austrian > > Companies Code: Raiffeisen Bank International AG; Registered Office: Am > > Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN 122119m at > > the Commercial Court of Vienna (Handelsgericht Wien). > > > Sent from Outlook for Mac This message and any attachment ("the Message") are confidential. If you have received the Message in error, please notify the sender immediately and delete the Message from your system, any use of the Message is forbidden. Correspondence via e-mail is primarily for information purposes. RBI neither makes nor accepts legally binding statements via e-mail unless explicitly agreed otherwise. Information pursuant to ? 14 Austrian Companies Code: Raiffeisen Bank International AG; Registered Office: Am Stadtpark 9, 1030 Vienna, Austria; Company Register Number: FN 122119m at the Commercial Court of Vienna (Handelsgericht Wien).
