merlimat opened a new pull request, #25887: URL: https://github.com/apache/pulsar/pull/25887
## Motivation PIP-475 added regular-to-scalable topic migration on the broker and the V5 client SDK. The V5 client transparently routes against both regular and scalable topics. Pulsar's perf CLI (`pulsar-perf produce / consume / read / transaction`) still used the v4 client API and therefore could not benchmark scalable topics at all. This PR migrates all four `pulsar-perf` commands to the V5 client API so the same binary now works against regular and scalable topics out of the box. A small V5 client-side fix is bundled in because it is required for the perf migration: `PulsarClientBuilderV5#tlsPolicy` was a stub that dropped every TlsPolicy field on the floor (only setting `useTls=true`). The perf commands take `--trust-cert-file` and the related flags and expect them to land, so the policy is now wired through to `ClientConfigurationData` end-to-end. ## Modifications **`pulsar-client-v5`** - `PulsarClientBuilderV5#tlsPolicy` now wires `trustCertsFilePath`, `keyFilePath`, `certificateFilePath`, `allowInsecureConnection`, `enableHostnameVerification` through to the underlying v4 conf. Two new unit tests in `PulsarClientBuilderV5Test` cover field propagation and the `TlsPolicy.ofInsecure()` shortcut. **`pulsar-testclient`** — depends on `pulsar-client-v5` / `pulsar-client-api-v5`. - `PerfClientUtils` gets `createV5ClientBuilderFromArguments()` mirroring the v4 helper. Important detail: the TLS predicate only enables TLS when the user genuinely wants it (URL is `pulsar+ssl://`, a trust cert path is supplied, or a flag is explicitly TRUE). Earlier wiring that treated non-null Booleans as intent caused TLS to flip on against plaintext brokers because picocli / default-value resolution coerces those flags to `Boolean.FALSE` even when not passed. - `PerformanceProducer`: V5 `Producer` / `AsyncProducer.async().send()`. `BatchingPolicy` / `ChunkingPolicy` / `CompressionPolicy` / `ProducerAccessMode`. `ProducerEncryptionPolicy` backed by `PemFileKeyProvider` for `--encryption-key-name` + `--encryption-key-value-file`. V5 client-level `TransactionPolicy` + `Transaction.async().commit()`. - `PerformanceConsumer`: V5 `QueueConsumer`. All subscription types fold into `QueueConsumer` semantics (work distribution); `Exclusive` / `Failover` trigger a runtime warning. V5 has no `MessageListener`, so each consumer gets one dedicated poll thread driving `receive(Duration)` and invoking the same handler the v4 listener did. V5 `acknowledge(...)` is synchronous void — wrapped in try / catch with the existing failure counters. - `PerformanceReader` → V5 `CheckpointConsumer`. Same poll-thread emulation as `PerformanceConsumer` replaces `ReaderListener`. `--start-message-id` now only accepts `earliest` / `latest` — V5 `Checkpoint` has no `lid:eid` factory and rejecting that form explicitly is cleaner than silently falling back to `earliest`. - `PerformanceTransaction`: combined producer+consumer over V5 `Transaction`. V5 puts `.transaction(txn)` on the message builder rather than `producer.newMessage(txn)`. Same semantics, different ergonomics. ### V5 features without a direct v4 equivalent The following CLI flags survive for back-compat but are logged as no-ops at start-up: `--max-outstanding`, `--max-outstanding-across-partitions`, `--stats-interval-seconds`, `--max-lookup-request`, `--ssl-factory-plugin*`, `--busy-wait`, `--auto-scaled-receiver-queue-size`, `--batch-index-ack`, `--pool-messages`, chunked-message knobs (`--max_chunked_msg`, `--expire_time_incomplete_chunked_messages`, `--auto_ack_chunk_q_full`), `--receiver-queue-size-across-partitions`, `--replicated`, `--use-tls`. Per-transaction `--txn-timeout` moves to client-level `TransactionPolicy`. ## Verifying this change - New unit tests in `PulsarClientBuilderV5Test` cover the TlsPolicy wiring. This change is already covered by existing tests, so it does not need additional tests. ## Does this pull request potentially affect one of the following parts: - The public API: yes — the four `pulsar-perf` subcommands. CLI flag UX is preserved end-to-end; behavior for flags listed above is logged as a no-op. - The schema: no - The default values of configurations: no - The wire protocol: no - The rest endpoints: no - The admin cli options: no - Anything that affects deployment: no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
