jsancio commented on code in PR #12682:
URL: https://github.com/apache/kafka/pull/12682#discussion_r980449119
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
Review Comment:
```suggestion
<p>Kafka servers support listening for connections on multiple ports.
This is configured through
the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
list of the listeners to enable. At least one listener must be defined
on each server. The format
of each listener defined in <code>listeners</code> is given below:</p>
```
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
+
+ <p>Among the listeners in this list, it is possible to declare the
listener to be used for
+ inter-broker communication by setting the
<code>inter.broker.listener.name</code> configuration
+ to the name of the listener. If not defined, then the inter-broker
listener is determined
+ by the security protocol defined by
<code>security.inter.broker.protocol</code>, which
+ defaults to <code>PLAINTEXT</code>.</p>
+
+ <p>For legacy clusters which rely on Zookeeper to store cluster metadata,
it is possible to
+ declare a separate listener to be used for metadata propagation from the
active controller
+ to the brokers. This is defined by
<code>control.plane.listener.name</code>. The active controller
+ will use this listener when it needs to push metadata updates to the
brokers in the cluster.
+ The benefit of using a control plane listener is that it uses a separate
processing thread,
+ which makes it less likely for application traffic to impede timely
propagation of metadata changes
+ (such as partition leader and ISR updates). Note that the default value
is null, which
+ means that the controller will use the same listener as defined by
<code>inter.broker.listener</code></p>
+
+ <p>In a KRaft cluster, the listener defined by
<code>inter.broker.listener.name</code> is used
+ exclusively for requests sent to nodes which have the "broker" role
enabled in <code>process.roles</code>.
+ For legacy clusters, the active controller will connect to all brokers
in the cluster
+ to send metadata updates. In KRaft, it is the other way around. The
broker is responsible
+ for finding the current controller and connecting to it in order to
receive metadata updates.
+ Hence the configuration <code>control.plane.listener.name</code> cannot
be used in KRaft clusters.</p>
+
+ <p>Instead, nodes with the "controller" role must define separate
listeners through the
+ <code>controller.listener.names</code> configuration. Any node running
with both the
+ "broker" and "controller" roles must use separate listeners for the
broker and controller.
+ For example, a minimal configuration for a combined node might look like
the following:</p>
+
+ <pre class="line-numbers"><code
class="language-text">process.roles=broker,controller
+listeners=BROKER://localhost:9092,CONTROLLER://localhost:9093
+inter.broker.listener.name=BROKER
+controller.listener.names=CONTROLLER
+listener.security.protocol.map=BROKER:SASL_SSL,CONTROLLER:SASL_SSL</code></pre>
+
+ <p>This defines two listeners, one for the controller and one for the
broker. It is conventional
+ to use a separate listener for client traffic as well.</p>
+
+ <p>The controller will accept requests on all listeners defined by
<code>controller.listener.names</code>.
+ For outbound requests to other controller nodes, the first listener
listed in <code>controller.listener.names</code>
+ will be used. The list of listeners provides a way to change the active
listener from one port
+ to another through a roll of the cluster. Basically do one roll to
expose the new listener,
+ and one roll to remove the old listener.</p>
+
+ <p>In the following section, this document covers how to enable SSL on a
listener for encryption
+ as well as authentication. The subsequent section will then cover
additional authentication
+ mechanism using SASL.</p>
Review Comment:
Should we have these relative references "In the following section" and "The
subsequent section"? Should we instead use the anchor id to provide a direct
reference to them.
Feel free to ignore this comment if it is not possible with the current HTML
documentation implementation.
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
+
+ <p>Among the listeners in this list, it is possible to declare the
listener to be used for
+ inter-broker communication by setting the
<code>inter.broker.listener.name</code> configuration
+ to the name of the listener. If not defined, then the inter-broker
listener is determined
+ by the security protocol defined by
<code>security.inter.broker.protocol</code>, which
+ defaults to <code>PLAINTEXT</code>.</p>
+
+ <p>For legacy clusters which rely on Zookeeper to store cluster metadata,
it is possible to
+ declare a separate listener to be used for metadata propagation from the
active controller
+ to the brokers. This is defined by
<code>control.plane.listener.name</code>. The active controller
+ will use this listener when it needs to push metadata updates to the
brokers in the cluster.
+ The benefit of using a control plane listener is that it uses a separate
processing thread,
+ which makes it less likely for application traffic to impede timely
propagation of metadata changes
+ (such as partition leader and ISR updates). Note that the default value
is null, which
+ means that the controller will use the same listener as defined by
<code>inter.broker.listener</code></p>
+
+ <p>In a KRaft cluster, the listener defined by
<code>inter.broker.listener.name</code> is used
+ exclusively for requests sent to nodes which have the "broker" role
enabled in <code>process.roles</code>.
+ For legacy clusters, the active controller will connect to all brokers
in the cluster
+ to send metadata updates. In KRaft, it is the other way around. The
broker is responsible
+ for finding the current controller and connecting to it in order to
receive metadata updates.
+ Hence the configuration <code>control.plane.listener.name</code> cannot
be used in KRaft clusters.</p>
Review Comment:
```suggestion
<p>In a KRaft cluster, the listener defined by
<code>inter.broker.listener.name</code> is used
exclusively for requests sent by brokers to brokers. A Kafka server is
a broker if the <code>broker</code> role enabled in <code>process.roles</code>.
In KRaft, the broker is responsible
for finding the current controller and connecting to it in order to
receive metadata updates.
The broker determines the listener and security protocol based the
configuration in <code>controller.listener.names</code> and
<code>listener.security.protocol.map</code></p>
```
I think it is easier to read if we document how legacy cluster are configure
in one paragraph/section and how kraft clusters are configured in another
paragraph/section. I think that comparing the two makes it harder to read and
understand.
I would argue that at this point the user it trying to configure their
cluster correctly. They are less interest in the reason of why the
configuration are different. I think they can read the KIPs and discussion for
that kind of detail. What do you think?
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
Review Comment:
```suggestion
communicate with the servers. Each server must define the set of
listeners that are used to
```
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
+
+ <p>Among the listeners in this list, it is possible to declare the
listener to be used for
+ inter-broker communication by setting the
<code>inter.broker.listener.name</code> configuration
+ to the name of the listener. If not defined, then the inter-broker
listener is determined
+ by the security protocol defined by
<code>security.inter.broker.protocol</code>, which
+ defaults to <code>PLAINTEXT</code>.</p>
Review Comment:
Should we mentioned that inter-broker communication is mainly used for the
replication of topic partitions?
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
Review Comment:
I say we remove this if we don't want to recommend this type of
configuration.
If a user is ready this section to understand an existing configuration
maybe we can move this to the end of the section as a note.
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
+
+ <p>Among the listeners in this list, it is possible to declare the
listener to be used for
+ inter-broker communication by setting the
<code>inter.broker.listener.name</code> configuration
+ to the name of the listener. If not defined, then the inter-broker
listener is determined
+ by the security protocol defined by
<code>security.inter.broker.protocol</code>, which
+ defaults to <code>PLAINTEXT</code>.</p>
+
+ <p>For legacy clusters which rely on Zookeeper to store cluster metadata,
it is possible to
+ declare a separate listener to be used for metadata propagation from the
active controller
+ to the brokers. This is defined by
<code>control.plane.listener.name</code>. The active controller
+ will use this listener when it needs to push metadata updates to the
brokers in the cluster.
+ The benefit of using a control plane listener is that it uses a separate
processing thread,
+ which makes it less likely for application traffic to impede timely
propagation of metadata changes
+ (such as partition leader and ISR updates). Note that the default value
is null, which
+ means that the controller will use the same listener as defined by
<code>inter.broker.listener</code></p>
+
+ <p>In a KRaft cluster, the listener defined by
<code>inter.broker.listener.name</code> is used
+ exclusively for requests sent to nodes which have the "broker" role
enabled in <code>process.roles</code>.
+ For legacy clusters, the active controller will connect to all brokers
in the cluster
+ to send metadata updates. In KRaft, it is the other way around. The
broker is responsible
+ for finding the current controller and connecting to it in order to
receive metadata updates.
+ Hence the configuration <code>control.plane.listener.name</code> cannot
be used in KRaft clusters.</p>
+
+ <p>Instead, nodes with the "controller" role must define separate
listeners through the
+ <code>controller.listener.names</code> configuration. Any node running
with both the
+ "broker" and "controller" roles must use separate listeners for the
broker and controller.
+ For example, a minimal configuration for a combined node might look like
the following:</p>
Review Comment:
```suggestion
<p>Instead, servers with the <code>controlle</code> role must define
separate listeners through the
<code>controller.listener.names</code> configuration. Any server
running with both the
<code>broker</code> and <code>controller</code> roles must use
separate listeners for the broker and controller.
For example, a minimal configuration for a combined server might look
like the following:</p>
```
The broker also needs to the controller listener name so that it knows how
which security protocol to use.
##########
docs/security.html:
##########
@@ -36,7 +36,104 @@ <h3 class="anchor-heading"><a id="security_overview"
class="anchor-link"></a><a
The guides below explain how to configure and use the security features in
both clients and brokers.
- <h3 class="anchor-heading"><a id="security_ssl" class="anchor-link"></a><a
href="#security_ssl">7.2 Encryption and Authentication using SSL</a></h3>
+ <h3 class="anchor-heading"><a id="listener_configuration"
class="anchor-link"></a><a href="#listener_configuration">7.2 Listener
Configuration</a></h3>
+
+ <p>In order to secure a Kafka cluster, it is necessary to secure the
channels that are used to
+ communicate with the servers. Each node must define the set of listeners
that are used to
+ receive requests from clients as well as other servers. Each listener
may be configured
+ to authenticate clients using various mechanisms and to ensure traffic
between the
+ server and the client is encrypted. This section provides a primer for
the configuration
+ of listeners.</p>
+
+ <p>Kafka brokers support listening for connections on multiple ports. This
is configured through
+ the <code>listeners</code> property in the server configuration, which
accepts a comma-separated
+ list of the listeners to enable. At least one listener must be defined
on each node. The format
+ of each listener defined in <code>listeners</code> is given below:</p>
+
+ <pre class="line-numbers"><code
class="language-text">{LISTENER_NAME}://{hostname}:{port}</code></pre>
+
+ <p>The <code>LISTENER_NAME</code> is usually a descriptive name which
defines the purpose of
+ the listener. For example, many configurations use a separate listener
for client traffic,
+ so they might refer to the corresponding listener as <code>CLIENT</code>
in the configuration:</p
+
+ <pre class="line-numbers"><code
class="language-text">listeners=CLIENT://localhost:9092</code></pre>
+
+ <p>The security protocol of each listener is defined in a separate
configuration:
+ <code>listener.security.protocol.map</code>. The value is a
comma-separated list
+ of each listener mapped to its security protocol. For example, the
follow value
+ configuration specifies that the <code>CLIENT</code> listener will use
SSL while the
+ <code>BROKER</code> listener will use plaintext.</p>
+
+ <pre class="line-numbers"><code
class="language-text">listener.security.protocol.map=CLIENT:SSL,BROKER:PLAINTEXT</code></pre>
+
+ <p>Possible options for the security protocol are given below:</p>
+ <ol>
+ <li>PLAINTEXT</li>
+ <li>SSL</li>
+ <li>SASL_PLAINTEXT</li>
+ <li>SASL_SSL</li>
+ </ol>
+
+ <p>The plaintext protocol provides no security and does not require any
additional configuration.
+ In the following sections, this document covers how to configure the
remaining protocols.</p>
+
+ <p>If each required listener uses a separate security protocol, it is also
possible to use the
+ security protocol name as the listener name in <code>listeners</code>.
Using the example above,
+ we could skip the definition of the <code>CLIENT</code> and
<code>BROKER</code> listeners
+ using the following definition:</p>
+
+ <pre class="line-numbers"><code
class="language-text">listeners=SSL://localhost:9092,PLAINTEXT://localhost:9093</code></pre>
+
+ <p>However, we recommend users to provide explicit names for the listeners
since it
+ makes the intended usage clearer.</p>
+
+ <p>Among the listeners in this list, it is possible to declare the
listener to be used for
+ inter-broker communication by setting the
<code>inter.broker.listener.name</code> configuration
+ to the name of the listener. If not defined, then the inter-broker
listener is determined
+ by the security protocol defined by
<code>security.inter.broker.protocol</code>, which
+ defaults to <code>PLAINTEXT</code>.</p>
+
+ <p>For legacy clusters which rely on Zookeeper to store cluster metadata,
it is possible to
+ declare a separate listener to be used for metadata propagation from the
active controller
+ to the brokers. This is defined by
<code>control.plane.listener.name</code>. The active controller
+ will use this listener when it needs to push metadata updates to the
brokers in the cluster.
+ The benefit of using a control plane listener is that it uses a separate
processing thread,
+ which makes it less likely for application traffic to impede timely
propagation of metadata changes
+ (such as partition leader and ISR updates). Note that the default value
is null, which
+ means that the controller will use the same listener as defined by
<code>inter.broker.listener</code></p>
+
+ <p>In a KRaft cluster, the listener defined by
<code>inter.broker.listener.name</code> is used
+ exclusively for requests sent to nodes which have the "broker" role
enabled in <code>process.roles</code>.
+ For legacy clusters, the active controller will connect to all brokers
in the cluster
+ to send metadata updates. In KRaft, it is the other way around. The
broker is responsible
+ for finding the current controller and connecting to it in order to
receive metadata updates.
+ Hence the configuration <code>control.plane.listener.name</code> cannot
be used in KRaft clusters.</p>
+
+ <p>Instead, nodes with the "controller" role must define separate
listeners through the
+ <code>controller.listener.names</code> configuration. Any node running
with both the
+ "broker" and "controller" roles must use separate listeners for the
broker and controller.
+ For example, a minimal configuration for a combined node might look like
the following:</p>
+
+ <pre class="line-numbers"><code
class="language-text">process.roles=broker,controller
+listeners=BROKER://localhost:9092,CONTROLLER://localhost:9093
+inter.broker.listener.name=BROKER
+controller.listener.names=CONTROLLER
+listener.security.protocol.map=BROKER:SASL_SSL,CONTROLLER:SASL_SSL</code></pre>
+
+ <p>This defines two listeners, one for the controller and one for the
broker. It is conventional
+ to use a separate listener for client traffic as well.</p>
+
+ <p>The controller will accept requests on all listeners defined by
<code>controller.listener.names</code>.
+ For outbound requests to other controller nodes, the first listener
listed in <code>controller.listener.names</code>
+ will be used. The list of listeners provides a way to change the active
listener from one port
+ to another through a roll of the cluster. Basically do one roll to
expose the new listener,
+ and one roll to remove the old listener.</p>
Review Comment:
```suggestion
<p>The controller will accept requests on all listeners defined by
<code>controller.listener.names</code>.
For outbound requests to other controller servers, the first listener
listed in <code>controller.listener.names</code>
will be used. The list of listeners provides a way to change the
active listener from one port
to another through a roll of the cluster. Basically do one roll to
expose the new listener,
and one roll to remove the old listener.</p>
```
Hmm. Regarding the last sentence, maybe we can add a section at the end
describing the process of changing the controller listener. What do you think?
I think that last sentence is hiding a lot of details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
