Repository: kafka Updated Branches: refs/heads/0.10.0 4f3966cec -> 542f7af7b
MINOR: Move `Incorporating Security Features in a Running Cluster` to its own section under `Security` It was previously in the SASL section (probably by mistake). Author: Ismael Juma <[email protected]> Reviewers: Sriharsha Chintalapani <[email protected]> Closes #1405 from ijuma/fix-security-upgrade-location-in-docs (cherry picked from commit 2ce3285297a8cf20d801c7ac479603dd5926851c) Signed-off-by: Ismael Juma <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/kafka/repo Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/542f7af7 Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/542f7af7 Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/542f7af7 Branch: refs/heads/0.10.0 Commit: 542f7af7ba7e562b57b5fd50bfd67a8320bf8963 Parents: 4f3966c Author: Ismael Juma <[email protected]> Authored: Thu May 19 17:16:37 2016 +0100 Committer: Ismael Juma <[email protected]> Committed: Thu May 19 17:16:50 2016 +0100 ---------------------------------------------------------------------- docs/security.html | 145 ++++++++++++++++++++++++------------------------ 1 file changed, 73 insertions(+), 72 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kafka/blob/542f7af7/docs/security.html ---------------------------------------------------------------------- diff --git a/docs/security.html b/docs/security.html index 3e5085b..bd483e5 100644 --- a/docs/security.html +++ b/docs/security.html @@ -448,74 +448,6 @@ Apache Kafka allows clients to connect over SSL. By default SSL is disabled but and <a href="#security_sasl_plain_brokerconfig">PLAIN</a> to configure SASL for the enabled mechanisms.</li> </ol> </li> - <li><h4><a id="security_rolling_upgrade" href="#security_rolling_upgrade">Incorporating Security Features in a Running Cluster</a></h4> - You can secure a running cluster via one or more of the supported protocols discussed previously. This is done in phases: - <p></p> - <ul> - <li>Incrementally bounce the cluster nodes to open additional secured port(s).</li> - <li>Restart clients using the secured rather than PLAINTEXT port (assuming you are securing the client-broker connection).</li> - <li>Incrementally bounce the cluster again to enable broker-to-broker security (if this is required)</li> - <li>A final incremental bounce to close the PLAINTEXT port.</li> - </ul> - <p></p> - The specific steps for configuring SSL and SASL are described in sections <a href="#security_ssl">7.2</a> and <a href="#security_sasl">7.3</a>. - Follow these steps to enable security for your desired protocol(s). - <p></p> - The security implementation lets you configure different protocols for both broker-client and broker-broker communication. - These must be enabled in separate bounces. A PLAINTEXT port must be left open throughout so brokers and/or clients can continue to communicate. - <p></p> - - When performing an incremental bounce stop the brokers cleanly via a SIGTERM. It's also good practice to wait for restarted replicas to return to the ISR list before moving onto the next node. - <p></p> - As an example, say we wish to encrypt both broker-client and broker-broker communication with SSL. In the first incremental bounce, a SSL port is opened on each node: - <pre> - listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092</pre> - - We then restart the clients, changing their config to point at the newly opened, secured port: - - <pre> - bootstrap.servers = [broker1:9092,...] - security.protocol = SSL - ...etc</pre> - - In the second incremental server bounce we instruct Kafka to use SSL as the broker-broker protocol (which will use the same SSL port): - - <pre> - listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092 - security.inter.broker.protocol=SSL</pre> - - In the final bounce we secure the cluster by closing the PLAINTEXT port: - - <pre> - listeners=SSL://broker1:9092 - security.inter.broker.protocol=SSL</pre> - - Alternatively we might choose to open multiple ports so that different protocols can be used for broker-broker and broker-client communication. Say we wished to use SSL encryption throughout (i.e. for broker-broker and broker-client communication) but we'd like to add SASL authentication to the broker-client connection also. We would achieve this by opening two additional ports during the first bounce: - - <pre> - listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093</pre> - - We would then restart the clients, changing their config to point at the newly opened, SASL & SSL secured port: - - <pre> - bootstrap.servers = [broker1:9093,...] - security.protocol = SASL_SSL - ...etc</pre> - - The second server bounce would switch the cluster to use encrypted broker-broker communication via the SSL port we previously opened on port 9092: - - <pre> - listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093 - security.inter.broker.protocol=SSL</pre> - - The final bounce secures the cluster by closing the PLAINTEXT port. - - <pre> - listeners=SSL://broker1:9092,SASL_SSL://broker1:9093 - security.inter.broker.protocol=SSL</pre> - - ZooKeeper can be secured independently of the Kafka cluster. The steps for doing this are covered in section <a href="#zk_authz_migration">7.5.2</a>. - </li> <li><h4><a id="saslmechanism_rolling_upgrade" href="#saslmechanism_rolling_upgrade">Modifying SASL mechanism in a Running Cluster</a></h4> <p>SASL mechanism can be modified in a running cluster using the following sequence:</p> <ol> @@ -673,8 +605,77 @@ Suppose you want to add an acl "Principals User:Bob and User:Alice are allowed t In order to remove a principal from producer or consumer role we just need to pass --remove option. </li> </ul> -<h3><a id="zk_authz" href="#zk_authz">7.5 ZooKeeper Authentication</a></h3> -<h4><a id="zk_authz_new" href="#zk_authz_new">7.5.1 New clusters</a></h4> +<h3><a id="security_rolling_upgrade" href="#security_rolling_upgrade">7.5 Incorporating Security Features in a Running Cluster</a></h3> + You can secure a running cluster via one or more of the supported protocols discussed previously. This is done in phases: + <p></p> + <ul> + <li>Incrementally bounce the cluster nodes to open additional secured port(s).</li> + <li>Restart clients using the secured rather than PLAINTEXT port (assuming you are securing the client-broker connection).</li> + <li>Incrementally bounce the cluster again to enable broker-to-broker security (if this is required)</li> + <li>A final incremental bounce to close the PLAINTEXT port.</li> + </ul> + <p></p> + The specific steps for configuring SSL and SASL are described in sections <a href="#security_ssl">7.2</a> and <a href="#security_sasl">7.3</a>. + Follow these steps to enable security for your desired protocol(s). + <p></p> + The security implementation lets you configure different protocols for both broker-client and broker-broker communication. + These must be enabled in separate bounces. A PLAINTEXT port must be left open throughout so brokers and/or clients can continue to communicate. + <p></p> + + When performing an incremental bounce stop the brokers cleanly via a SIGTERM. It's also good practice to wait for restarted replicas to return to the ISR list before moving onto the next node. + <p></p> + As an example, say we wish to encrypt both broker-client and broker-broker communication with SSL. In the first incremental bounce, a SSL port is opened on each node: + <pre> + listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092</pre> + + We then restart the clients, changing their config to point at the newly opened, secured port: + + <pre> + bootstrap.servers = [broker1:9092,...] + security.protocol = SSL + ...etc</pre> + + In the second incremental server bounce we instruct Kafka to use SSL as the broker-broker protocol (which will use the same SSL port): + + <pre> + listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092 + security.inter.broker.protocol=SSL</pre> + + In the final bounce we secure the cluster by closing the PLAINTEXT port: + + <pre> + listeners=SSL://broker1:9092 + security.inter.broker.protocol=SSL</pre> + + Alternatively we might choose to open multiple ports so that different protocols can be used for broker-broker and broker-client communication. Say we wished to use SSL encryption throughout (i.e. for broker-broker and broker-client communication) but we'd like to add SASL authentication to the broker-client connection also. We would achieve this by opening two additional ports during the first bounce: + + <pre> + listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093</pre> + + We would then restart the clients, changing their config to point at the newly opened, SASL & SSL secured port: + + <pre> + bootstrap.servers = [broker1:9093,...] + security.protocol = SASL_SSL + ...etc</pre> + + The second server bounce would switch the cluster to use encrypted broker-broker communication via the SSL port we previously opened on port 9092: + + <pre> + listeners=PLAINTEXT://broker1:9091,SSL://broker1:9092,SASL_SSL://broker1:9093 + security.inter.broker.protocol=SSL</pre> + + The final bounce secures the cluster by closing the PLAINTEXT port. + + <pre> + listeners=SSL://broker1:9092,SASL_SSL://broker1:9093 + security.inter.broker.protocol=SSL</pre> + + ZooKeeper can be secured independently of the Kafka cluster. The steps for doing this are covered in section <a href="#zk_authz_migration">7.6.2</a>. + + +<h3><a id="zk_authz" href="#zk_authz">7.6 ZooKeeper Authentication</a></h3> +<h4><a id="zk_authz_new" href="#zk_authz_new">7.6.1 New clusters</a></h4> To enable ZooKeeper authentication on brokers, there are two necessary steps: <ol> <li> Create a JAAS login file and set the appropriate system property to point to it as described above</li> @@ -683,7 +684,7 @@ To enable ZooKeeper authentication on brokers, there are two necessary steps: The metadata stored in ZooKeeper is such that only brokers will be able to modify the corresponding znodes, but znodes are world readable. The rationale behind this decision is that the data stored in ZooKeeper is not sensitive, but inappropriate manipulation of znodes can cause cluster disruption. We also recommend limiting the access to ZooKeeper via network segmentation (only brokers and some admin tools need access to ZooKeeper if the new consumer and new producer are used). -<h4><a id="zk_authz_migration" href="#zk_authz_migration">7.5.2 Migrating clusters</a></h4> +<h4><a id="zk_authz_migration" href="#zk_authz_migration">7.6.2 Migrating clusters</a></h4> If you are running a version of Kafka that does not support security or simply with security disabled, and you want to make the cluster secure, then you need to execute the following steps to enable ZooKeeper authentication with minimal disruption to your operations: <ol> <li>Perform a rolling restart setting the JAAS login file, which enables brokers to authenticate. At the end of the rolling restart, brokers are able to manipulate znodes with strict ACLs, but they will not create znodes with those ACLs</li> @@ -704,7 +705,7 @@ Here is an example of how to run the migration tool: <pre> ./bin/zookeeper-security-migration --help </pre> -<h4><a id="zk_authz_ensemble" href="#zk_authz_ensemble">7.5.3 Migrating the ZooKeeper ensemble</a></h4> +<h4><a id="zk_authz_ensemble" href="#zk_authz_ensemble">7.6.3 Migrating the ZooKeeper ensemble</a></h4> It is also necessary to enable authentication on the ZooKeeper ensemble. To do it, we need to perform a rolling restart of the server and set a few properties. Please refer to the ZooKeeper documentation for more detail: <ol> <li><a href="http://zookeeper.apache.org/doc/r3.4.6/zookeeperProgrammers.html#sc_ZooKeeperAccessControl";>Apache ZooKeeper documentation</a></li>
