aliehsaeedii opened a new pull request, #22775: URL: https://github.com/apache/kafka/pull/22775
Adds the integration test coverage for the streams group topology description plugin (KIP-1331) requested in [KAFKA-20627](https://issues.apache.org/jira/browse/KAFKA-20627): ### Broker-level RPC tests (`core`) - **`StreamsGroupTopologyDescriptionRequestTest`** (plugin configured with `InMemoryTopologyDescriptionPlugin`): - heartbeat solicits the push via `TopologyDescriptionRequired`, the push → describe lifecycle returns `AVAILABLE` with the stored topology, and there is no re-solicitation once stored - validation errors: empty group/member id → `INVALID_REQUEST`, unknown group → `GROUP_ID_NOT_FOUND`, unknown member → `UNKNOWN_MEMBER_ID`, stale topology epoch → `INVALID_REQUEST` - `DeleteGroups` succeeds on a group with a stored description and the group is gone afterwards - **`StreamsGroupTopologyDescriptionNoPluginRequestTest`** (no plugin configured): the push RPC returns `UNSUPPORTED_VERSION`, heartbeats never solicit a push, and describe reports `NOT_STORED` - `GroupCoordinatorBaseRequestTest`: new `streamsGroupTopologyDescriptionUpdate` helper and `IncludeTopologyDescription` support on `streamsGroupDescribe` ### End-to-end tests with real `KafkaStreams` clients (`streams/integration-tests`) - **`TopologyDescriptionPluginIntegrationTest`** — 4 tests covering the core push lifecycle on a live cluster: - topology description reaches the plugin after join - two-member group deduplicates: only one push happens - permanent plugin failure ratchets the failed epoch and stops re-solicitation (call count stays at one, the application keeps running, describe reports `NOT_STORED`) - explicit `DeleteGroups` triggers `plugin.deleteTopology` - **`TopologyDescriptionPluginExpirationIntegrationTest`** — 2 tests covering natural group expiry: - group with stored topology: the cleanup cycle calls `plugin.deleteTopology` and clears the stored epoch before the expiration sweep tombstones the group - group whose push permanently failed (stored epoch `-1`): the tombstone fires directly without any `deleteTopology` call - Both observe the broker through a new `TrackingTopologyDescriptionPlugin` test plugin (extends `InMemoryTopologyDescriptionPlugin`, records per-group `setTopology`/`deleteTopology` calls in static state and can inject permanent failures) — the embedded brokers run in the test JVM ### CLI and Admin API - `DescribeStreamsGroupTest#testDescribeStreamsGroupWithTopologyOption`: exercises `kafka-streams-groups.sh --describe --topology` end-to-end, waiting until the output contains `Topologies:` and the source topic name - `PlaintextAdminIntegrationTest#testDescribeStreamsGroupsWithTopologyDescription`: `DescribeStreamsGroupsOptions.includeTopologyDescription` wiring (`NOT_REQUESTED` / `NOT_STORED` on a plugin-less cluster; the `AVAILABLE` path is covered via the Admin API in `TopologyDescriptionPluginIntegrationTest`) ### Infrastructure - `EmbeddedKafkaCluster.addDefaultBrokerPropsIfAbsent` now configures `InMemoryTopologyDescriptionPlugin` via `putIfAbsent`, so all streams integration tests exercise the topology description path and catch regressions without per-test changes - `streams:integration-tests` gains a `group-coordinator-api` test dependency; import-control allows `org.apache.kafka.server.streams` in streams integration tests ### Testing All 14 new tests pass locally, plus checkstyle on the touched modules. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
