Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe merged PR #14160: URL: https://github.com/apache/kafka/pull/14160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on code in PR #14160: URL: https://github.com/apache/kafka/pull/14160#discussion_r1406786870 ## docs/ops.html: ## @@ -3778,6 +3784,14 @@ Migrating brokers to KRaft Each broker is restarted with a KRaft configuration until the entire cluster is running in KRaft mode. + Reverting to ZooKeeper mode During the Migration +While the cluster is still in migration mode, it is possible to revert to ZK mode. In order to do this: + + One by one, take each KRaft broker down. Remove the __cluster_metadata directory on the broker. Then, restart the broker as ZooKeeper. Review Comment: done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on code in PR #14160: URL: https://github.com/apache/kafka/pull/14160#discussion_r1406786425 ## docs/ops.html: ## @@ -3603,39 +3603,45 @@ Supporting JBOD configurations with multiple storage directories Modifying certain dynamic configurations on the standalone KRaft controller -Delegation tokens ZooKeeper to KRaft Migration ZooKeeper to KRaft migration is considered an Early Access feature and is not recommended for production clusters. +Please report issues with ZooKeeper to KRaft migration using the +https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - The following features are not yet supported for ZK to KRaft migrations: - + Terminology -Downgrading to ZooKeeper mode during or after the migration -Other features not yet supported in KRaft +Brokers that are in ZK mode store their metadata in Apache ZooKepeer. This is the old mode of handling metadata. +Brokers that are in KRaft mode store their metadata in a KRaft quorum. This is the new and improve mode of handling metadata. +Migration is the process of moving cluster metadata from ZooKeeper into a KRaft quorum. - -Please report issues with ZooKeeper to KRaft migration using the -https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - + Migration Phases + In general, the migration process passes through several phases. - Terminology - -We use the term "migration" here to refer to the process of changing a Kafka cluster's metadata -system from ZooKeeper to KRaft and migrating existing metadata. An "upgrade" refers to installing a newer version of Kafka. It is not recommended to -upgrade the software at the same time as performing a metadata migration. - + +In the initial phase, all the brokers are in ZK mode, and there is a ZK-based controller. +During the initial metadata load, a KRaft quorum loads the metadata from ZooKeeper, +In hybrid phase, some brokers are in ZK mode, but there is a KRaft controller. +In dual-write phase, all brokers are KRaft, but the KRaft controller is continuing to write to ZK. +When the migration has been finalized, we no longer write metadata to ZooKeeper. + - -We also use the term "ZK mode" to refer to Kafka brokers which are using ZooKeeper as their metadata -system. "KRaft mode" refers Kafka brokers which are using a KRaft controller quorum as their metadata system. - + Limitations + +While a cluster is being migrated from ZK mode to KRaft mode, we do not support changing the metadata + version (also known as the inter.broker.protocol version.) Please do not attempt to do this during + a migration, or you may break the cluster. +After the migration has been finalized, it is not possible to revert back to ZooKeeper mode. +As noted above, some features are not fully implemented in KRaft mode. If you are + using one of those features, you will not be able to migrate to KRaft yet. + + Review Comment: removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on code in PR #14160: URL: https://github.com/apache/kafka/pull/14160#discussion_r1406786099 ## docs/ops.html: ## @@ -3603,39 +3603,45 @@ Supporting JBOD configurations with multiple storage directories Modifying certain dynamic configurations on the standalone KRaft controller -Delegation tokens ZooKeeper to KRaft Migration ZooKeeper to KRaft migration is considered an Early Access feature and is not recommended for production clusters. +Please report issues with ZooKeeper to KRaft migration using the +https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - The following features are not yet supported for ZK to KRaft migrations: - + Terminology -Downgrading to ZooKeeper mode during or after the migration -Other features not yet supported in KRaft +Brokers that are in ZK mode store their metadata in Apache ZooKepeer. This is the old mode of handling metadata. +Brokers that are in KRaft mode store their metadata in a KRaft quorum. This is the new and improve mode of handling metadata. +Migration is the process of moving cluster metadata from ZooKeeper into a KRaft quorum. - -Please report issues with ZooKeeper to KRaft migration using the -https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - + Migration Phases + In general, the migration process passes through several phases. - Terminology - -We use the term "migration" here to refer to the process of changing a Kafka cluster's metadata -system from ZooKeeper to KRaft and migrating existing metadata. An "upgrade" refers to installing a newer version of Kafka. It is not recommended to -upgrade the software at the same time as performing a metadata migration. - + +In the initial phase, all the brokers are in ZK mode, and there is a ZK-based controller. +During the initial metadata load, a KRaft quorum loads the metadata from ZooKeeper, +In hybrid phase, some brokers are in ZK mode, but there is a KRaft controller. +In dual-write phase, all brokers are KRaft, but the KRaft controller is continuing to write to ZK. +When the migration has been finalized, we no longer write metadata to ZooKeeper. + Review Comment: I think distinguishing "hybrid phase" from "dual-write phase" is useful, even though they both map to the same `MigrationState` enum. "migration phases" aren't "migration states" (and actually, we don't discuss migration states in this doc) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
github-actions[bot] commented on PR #14160: URL: https://github.com/apache/kafka/pull/14160#issuecomment-1822035606 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch) If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe merged PR #14682: URL: https://github.com/apache/kafka/pull/14682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
lzyLuke commented on PR #14682: URL: https://github.com/apache/kafka/pull/14682#issuecomment-1789818408 > Thanks for the PR, @lzyLuke . > > Please remove these two lines from "The following features are not fully implemented in KRaft mode": > > > ``` > > Modifying certain dynamic configurations on the standalone KRaft controller > > ``` > > > > > > > > > > > > > > > > > > > > > > > >* Delegation tokens I could not find "delegation tokens" related lines. Probably it has been deleted already? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on PR #14682: URL: https://github.com/apache/kafka/pull/14682#issuecomment-1789520190 Thanks for the PR, @lzyLuke . Please remove these two lines from "The following features are not fully implemented in KRaft mode": > Modifying certain dynamic configurations on the standalone KRaft controller >Delegation tokens -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379209724 ## docs/ops.html: ## @@ -3613,23 +3613,22 @@ https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - - Terminology -Brokers that are in ZK mode store their metadata in Apache ZooKeeper. This is the old mode of handling metadata. +Brokers that are in ZK mode store their metadata in Apache ZooKepeer. This is the old mode of handling metadata. Brokers that are in KRaft mode store their metadata in a KRaft quorum. This is the new and improved mode of handling metadata. Migration is the process of moving cluster metadata from ZooKeeper into a KRaft quorum. Migration Phases In general, the migration process passes through several phases. + + For -In the initial phase, all the brokers are in ZK mode, and there is a ZK-based controller. -During the initial metadata load, a KRaft quorum loads the metadata from ZooKeeper, -In hybrid phase, some brokers are in ZK mode, but there is a KRaft controller. -In dual-write phase, all brokers are KRaft, but the KRaft controller is continuing to write to ZK. -When the migration has been finalized, we no longer write metadata to ZooKeeper. +In the pre-migration phase, all the brokers are in ZK mode, and there is a ZK-based controller. But a KRaft controller has been provisioned and has migration enabled Review Comment: Change "controller" to "controller quorum" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
cmccabe commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379209314 ## docs/ops.html: ## @@ -3613,23 +3613,22 @@ https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - - Terminology -Brokers that are in ZK mode store their metadata in Apache ZooKeeper. This is the old mode of handling metadata. +Brokers that are in ZK mode store their metadata in Apache ZooKepeer. This is the old mode of handling metadata. Review Comment: This adds a typo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
lzyLuke commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379152694 ## docs/ops.html: ## @@ -3787,7 +3786,7 @@ Migrating brokers to KRaft Reverting to ZooKeeper mode During the Migration While the cluster is still in migration mode, it is possible to revert to ZK mode. In order to do this: - One by one, take each KRaft broker down. Remove the __cluster_metadata directory on the broker. Then, restart the broker as ZooKeeper. + One by one, take each KRaft broker down. Remove the __cluster_metadata directory on the broker. Then, restart the broker as zookeeper mode. Review Comment: maybe `in` would be better :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]
ex172000 commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1378815861 ## docs/ops.html: ## @@ -3641,7 +3641,6 @@ Limitations using one of those features, you will not be able to migrate to KRaft yet. Review Comment: Maybe we can phrase like this here: `If you are using one of those features, you are not able to migrate to KRaft yet.` ## docs/ops.html: ## @@ -3787,7 +3786,7 @@ Migrating brokers to KRaft Reverting to ZooKeeper mode During the Migration While the cluster is still in migration mode, it is possible to revert to ZK mode. In order to do this: - One by one, take each KRaft broker down. Remove the __cluster_metadata directory on the broker. Then, restart the broker as ZooKeeper. + One by one, take each KRaft broker down. Remove the __cluster_metadata directory on the broker. Then, restart the broker as zookeeper mode. Review Comment: `with` zookeeper mode? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
