merlimat opened a new pull request, #25878: URL: https://github.com/apache/pulsar/pull/25878
## Summary Final implementation PR for [PIP-475: Regular-to-Scalable Topic Migration](https://github.com/apache/pulsar/blob/master/pip/pip-475.md). Builds on the synthetic-layout lookup (#25822), the V5 SDK support (#25850), and the migration command (#25875). Adds an end-to-end test of the full operator timeline against a live broker, and fixes three real bugs the test surfaced that prevented a connected V5 client from transitioning across the migration boundary. ### Tests (`V5MigrationEndToEndTest`) - **produce-through-migration:** a V5 producer publishes via the synthetic layout (mod-N routing to the legacy segments); the topic is migrated while only that (marked) V5 producer is attached, so the pre-check passes without `--force`; the producer transparently follows the layout-change push to the real DAG and range-routes new messages to the active children; a V5 queue consumer then drains *every* pre- and post-migration message. - **v4 lockout:** after migration the old topic is terminated, so a legacy v4 producer can no longer write to it. ### Fixes surfaced by the E2E test - **`DagWatchSession` (the transition gap):** compute `segment://` URIs from the canonical `topic://` name, not the session's raw input. A session opened with a `persistent://` name (synthetic layout) previously threw *"Parent topic must have domain 'topic'"* inside `buildResponse` once the topic was migrated, so the real DAG was never pushed and connected clients never transitioned. - **Migration pre-check:** inspect per-partition stats instead of the aggregate. Aggregated partitioned stats merge publishers by name into fresh stat objects that drop per-connection metadata, which hid the V5-managed marker and made every V5 connection look like a legacy v4 one (so a fully-V5 topic was wrongly rejected). - **`ScalableTopicProducer`:** the send-retry now also covers per-segment producer *creation* (not just `send()`), since a migration terminates the old partition between routing and creation; detect the "segment gone" condition by unwrapping causes (type or message) and give the DAG-watch layout update a larger budget to arrive. ## Test plan - [x] `V5MigrationEndToEndTest` (2): produce-through-migration, v4 lockout. - [x] Regression: `ScalableTopicMigrationTest` 7/7, `DagWatchSessionTest` 18/18, `ScalableTopicControllerTest` 35/35, `V5RegularTopicInteropTest` 4/4, full `pulsar-client-v5` unit suite. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
