mjsax commented on code in PR #20029:
URL: https://github.com/apache/kafka/pull/20029#discussion_r2165066794
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -775,6 +776,20 @@ <h4><a class="toc-backref"
href="#id46">ensure.explicit.internal.resource.naming
</div>
</blockquote>
</div>
+ <div class="section" id="group-protocol">
+ <h4><a class="toc-backref" href="#id47">group.protocol</a><a
class="headerlink" href="#group-protocol" title="Permalink to this
headline"></a></h4>
+ <blockquote>
+ <div>
+ <p>
+ The group protocol used by the Kafka Streams client used for
coordination.
+ This is used to determine how the client will communicate with
the Kafka brokers and other clients in the same group.
+ The default value is <code class="docutils literal"><span
class="pre">"classic"</span></code>, which is the classic consumer group
protocol.
+ Set to <code class="docutils literal"><span
class="pre">"streams"</span></code> to enable the new Kafka Streams group
protocol.
+ Note that the streams rebalance protocol is an Early Access
feature and should not be used in production.
Review Comment:
```suggestion
Note that the "streams" rebalance protocol is an Early
Access feature and should not be used in production.
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -80,6 +80,7 @@
<li><a class="reference internal"
href="#deserialization-exception-handler"
id="id7">deserialization.exception.handler</a></li>
<li><a class="reference internal" href="#enable-metrics-push"
id="id43">enable.metrics.push</a></li>
<li><a class="reference internal"
href="#ensure-explicit-internal-resource-naming"
id="id46">ensure.explicit.internal.resource.naming</a></li>
+ <li><a class="reference internal" href="#group-protocol"
id="id47">group.protocol</a></li>
Review Comment:
Random thought: I just checked and priority is set to MEDIUM in
`StreamsConfig`. Should we reduce it to `LOW` for EA?
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -775,6 +776,20 @@ <h4><a class="toc-backref"
href="#id46">ensure.explicit.internal.resource.naming
</div>
</blockquote>
</div>
+ <div class="section" id="group-protocol">
+ <h4><a class="toc-backref" href="#id47">group.protocol</a><a
class="headerlink" href="#group-protocol" title="Permalink to this
headline"></a></h4>
+ <blockquote>
+ <div>
+ <p>
+ The group protocol used by the Kafka Streams client used for
coordination.
+ This is used to determine how the client will communicate with
the Kafka brokers and other clients in the same group.
Review Comment:
```suggestion
It determines how the client will communicate with the Kafka
brokers and other clients in the same group.
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -775,6 +776,20 @@ <h4><a class="toc-backref"
href="#id46">ensure.explicit.internal.resource.naming
</div>
</blockquote>
</div>
+ <div class="section" id="group-protocol">
+ <h4><a class="toc-backref" href="#id47">group.protocol</a><a
class="headerlink" href="#group-protocol" title="Permalink to this
headline"></a></h4>
+ <blockquote>
+ <div>
+ <p>
+ The group protocol used by the Kafka Streams client used for
coordination.
+ This is used to determine how the client will communicate with
the Kafka brokers and other clients in the same group.
+ The default value is <code class="docutils literal"><span
class="pre">"classic"</span></code>, which is the classic consumer group
protocol.
+ Set to <code class="docutils literal"><span
class="pre">"streams"</span></code> to enable the new Kafka Streams group
protocol.
Review Comment:
Or something like this -- maybe also a link to the details how to enable it
broker side?
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> No automated migration is
available between the classic and new streams protocols.</li>
+ </ul>
+
+ <p><strong>Why Use the Streams Rebalance Protocol?</strong></p>
+
+ <ul>
+ <li>
+ <strong>Broker-Driven Coordination:</strong>
+ Centralizes task assignment logic on brokers instead of the
client. This provides consistent,
+ authoritative task assignment decisions from a single coordination
point and reduces the potential for
+ split-brain scenarios.
+ </li>
+ <li>
+ <strong>Faster, More Stable Rebalances:</strong>
+ Reduces rebalance duration and impact by removing the global
synchronization point. This minimizes
+ application downtime during membership changes or failures.
+ </li>
+ <li>
+ <strong>Better Observability:</strong>
+ Provides dedicated metrics and admin interfaces that separate
streams from consumer groups, leading to
+ clearer troubleshooting with broker-side observability.
+ </li>
+ </ul>
+
+ <p>
+ To enable the protocol, set
<code>unstable.feature.versions.enable=true</code> for controllers and brokers,
and
+ set <code>unstable.api.versions.enable=true</code> on the brokers as
well. In your Kafka Streams application
+ configuration, set <code>group.protocol=streams</code>.
+ </p>
+
+ <p>
+ Migration between the classic consumer group protocol and the Streams
Rebalance Protocol is not supported in
+ either direction. An application using this protocol must use a new
<code>application.id</code> that has not
+ been used by any application on the classic protocol. Furthermore,
this ID must not be in use as a
+ <code>group.id</code> by any consumer application.
+ </p>
+
+ <p>
+ Please provide feedback on this feature via the Kafka mailing lists or
by filing JIRA issues.
Review Comment:
`Kafka mailing` -> make this a link to the https://kafka.apache.org/contact
`JIRA issues` -> link to https://kafka.apache.org/contributing or
https://issues.apache.org/jira/browse/KAFKA ?
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> No automated migration is
available between the classic and new streams protocols.</li>
+ </ul>
+
+ <p><strong>Why Use the Streams Rebalance Protocol?</strong></p>
+
+ <ul>
+ <li>
+ <strong>Broker-Driven Coordination:</strong>
+ Centralizes task assignment logic on brokers instead of the
client. This provides consistent,
+ authoritative task assignment decisions from a single coordination
point and reduces the potential for
+ split-brain scenarios.
+ </li>
+ <li>
+ <strong>Faster, More Stable Rebalances:</strong>
+ Reduces rebalance duration and impact by removing the global
synchronization point. This minimizes
+ application downtime during membership changes or failures.
+ </li>
+ <li>
+ <strong>Better Observability:</strong>
+ Provides dedicated metrics and admin interfaces that separate
streams from consumer groups, leading to
+ clearer troubleshooting with broker-side observability.
+ </li>
+ </ul>
+
+ <p>
+ To enable the protocol, set
<code>unstable.feature.versions.enable=true</code> for controllers and brokers,
and
+ set <code>unstable.api.versions.enable=true</code> on the brokers as
well. In your Kafka Streams application
+ configuration, set <code>group.protocol=streams</code>.
+ </p>
+
+ <p>
+ Migration between the classic consumer group protocol and the Streams
Rebalance Protocol is not supported in
Review Comment:
It should be possible to delete a group, to free up the `group.id` to switch
between protocols, right? Should we add a sentence about it somewhere in this
paragraph?
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> No automated migration is
available between the classic and new streams protocols.</li>
Review Comment:
```suggestion
<li><strong>Protocol Migration:</strong> Group migration is not
available between the classic and new streams protocols.</li>
```
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -775,6 +776,20 @@ <h4><a class="toc-backref"
href="#id46">ensure.explicit.internal.resource.naming
</div>
</blockquote>
</div>
+ <div class="section" id="group-protocol">
+ <h4><a class="toc-backref" href="#id47">group.protocol</a><a
class="headerlink" href="#group-protocol" title="Permalink to this
headline"></a></h4>
+ <blockquote>
+ <div>
+ <p>
+ The group protocol used by the Kafka Streams client used for
coordination.
+ This is used to determine how the client will communicate with
the Kafka brokers and other clients in the same group.
+ The default value is <code class="docutils literal"><span
class="pre">"classic"</span></code>, which is the classic consumer group
protocol.
+ Set to <code class="docutils literal"><span
class="pre">"streams"</span></code> to enable the new Kafka Streams group
protocol.
Review Comment:
```suggestion
Can be set to <code class="docutils literal"><span
class="pre">"streams"</span></code> (requires broker side enablement) to enable
the new Kafka Streams group protocol.
```
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
Review Comment:
This might to get updated, in case we get this feature into 4.1. (Just
comment as a reminder in case we cherry-pick this PR).
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> No automated migration is
available between the classic and new streams protocols.</li>
+ </ul>
+
+ <p><strong>Why Use the Streams Rebalance Protocol?</strong></p>
+
+ <ul>
+ <li>
+ <strong>Broker-Driven Coordination:</strong>
+ Centralizes task assignment logic on brokers instead of the
client. This provides consistent,
+ authoritative task assignment decisions from a single coordination
point and reduces the potential for
+ split-brain scenarios.
+ </li>
+ <li>
+ <strong>Faster, More Stable Rebalances:</strong>
+ Reduces rebalance duration and impact by removing the global
synchronization point. This minimizes
+ application downtime during membership changes or failures.
+ </li>
+ <li>
+ <strong>Better Observability:</strong>
+ Provides dedicated metrics and admin interfaces that separate
streams from consumer groups, leading to
+ clearer troubleshooting with broker-side observability.
+ </li>
+ </ul>
+
+ <p>
+ To enable the protocol, set
<code>unstable.feature.versions.enable=true</code> for controllers and brokers,
and
Review Comment:
We should add "requires broker version 4.1".
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> No automated migration is
available between the classic and new streams protocols.</li>
+ </ul>
+
+ <p><strong>Why Use the Streams Rebalance Protocol?</strong></p>
+
+ <ul>
+ <li>
+ <strong>Broker-Driven Coordination:</strong>
+ Centralizes task assignment logic on brokers instead of the
client. This provides consistent,
+ authoritative task assignment decisions from a single coordination
point and reduces the potential for
+ split-brain scenarios.
+ </li>
+ <li>
+ <strong>Faster, More Stable Rebalances:</strong>
+ Reduces rebalance duration and impact by removing the global
synchronization point. This minimizes
+ application downtime during membership changes or failures.
+ </li>
+ <li>
+ <strong>Better Observability:</strong>
+ Provides dedicated metrics and admin interfaces that separate
streams from consumer groups, leading to
+ clearer troubleshooting with broker-side observability.
+ </li>
+ </ul>
+
+ <p>
+ To enable the protocol, set
<code>unstable.feature.versions.enable=true</code> for controllers and brokers,
and
+ set <code>unstable.api.versions.enable=true</code> on the brokers as
well. In your Kafka Streams application
+ configuration, set <code>group.protocol=streams</code>.
+ </p>
+
+ <p>
+ Migration between the classic consumer group protocol and the Streams
Rebalance Protocol is not supported in
+ either direction. An application using this protocol must use a new
<code>application.id</code> that has not
+ been used by any application on the classic protocol. Furthermore,
this ID must not be in use as a
+ <code>group.id</code> by any consumer application.
Review Comment:
```suggestion
<code>group.id</code> by any consumer ("classic" or "consumer") not
share-group application.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
