merlimat opened a new pull request, #25772: URL: https://github.com/apache/pulsar/pull/25772
## Summary P4 of the [PIP-473](https://github.com/apache/pulsar/blob/master/pip/pip-473.md) metadata-driven transactions stack. Adds the ack-side complement to [#25768](https://github.com/apache/pulsar/pull/25768) — a `PendingAckStore` implementation for subscriptions on `segment://` topics that reads truth from the metadata-store layout. - `MetadataPendingAckStore`: - `appendIndividualAck` / `appendCumulativeAck` write `TxnOp(ACK, segment, sub, ledgerId, entryId, cumulative)` records via the `TxnMetadataStore` façade. - `appendCommitMark` / `appendAbortMark` are **no-ops** — in v5 the TC owns the lifecycle, the store consumes its events. - Subscription-event listener (`/txn-subscription-events/<seg>:<sub>-*`) uses the same header-truthing reconcile pattern as the P3 TB: re-read headers for every currently-open txn this subscription is involved in, then for terminals call `PendingAckHandleImpl.commitTxn` / `abortTxn` (no SPI change) and delete the corresponding `/txn-op` ack records. - Recovery via scan (Option C): subscribe → scan `listAcksBySegmentSubscription` → group by txnId → fetch headers → seed open-txn set → terminal txns discovered mid-scan are processed inline → drain post-recovery reconcile. - `MetadataPendingAckStoreProvider` + `DispatchingTransactionPendingAckStoreProvider` (routes by `TopicName.isSegment()`, falls through to legacy `MLPendingAckStore` otherwise). ### Supporting changes - `TxnOp` gains a nullable `Boolean cumulative` field (only set on cumulative acks). - `TxnMetadataStore.deleteAckOpsForSegmentSubscriptionAndTxn` mirrors the P3 write-op cleanup helper; the two flavors share a private `scanAndDeleteOpsForTxn`. ### Not in this PR (deferred to P5.4) - **No default-config flip.** `transactionPendingAckStoreProviderClassName` stays `MLPendingAckStoreProvider`. The new store is opt-in until the v5 TC is wired up. - **MLPendingAckStore segment-workaround stays in place.** Removing it now would break segment subscriptions while the legacy store is still the default. P5.4 will remove it as part of the same PR that flips all three providers. ## Test plan - [x] `pulsar-broker:test --tests MetadataPendingAckStoreTest` — 6 cases on the metadata store (Memory backend, mocked `PersistentSubscription` + `PendingAckHandleImpl`): individual ack append, cumulative ack append, no-op marks, commit event drives `handle.commitTxn` + ack-op cleanup, abort event drives `handle.abortTxn` + cleanup, recovery rebuilds state + processes terminal txns discovered mid-scan. - [x] `pulsar-broker:test --tests DispatchingTransactionPendingAckStoreProviderTest` — 2 routing tests. - [x] P2 + P3 unit tests still green after the `TxnOp.cumulative` addition. - [x] Checkstyle clean on `pulsar-broker`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
