[
https://issues.apache.org/jira/browse/KAFKA-15990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viktor Nikitash updated KAFKA-15990:
------------------------------------
Description:
h2. Background
There is a miss in Kafka versions [3.1.x; 3.6.0) what we can enable tiering,
which is not working on that versions at all, and when we upgrade kafka to
version 3.6.0 we have crash restart loop with message "You have to delete all
topics with the property remote.storage.enable=true before disabling tiered
storage cluster-wide".
[https://github.com/apache/kafka/blob/43c635f3a48fcb16dd34eac16def379141912e77/storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java#L564]
h2. The Idea
The idea is to create new parameter which allows to wipe configurations related
to remote storage on topic level, with default value set to false (which won't
change logic from the current one), but in case we manually set parameter to
true, those remote topic parameters will be wiped.
Of course additional checks required. Like do it only on Kafka startup and
ensure that there is no data on remote storage etc...
h2. Motivation
Since tiering in to working on version 3.6.0-, there is no any reason to keep
this configs during migration from older 3rd version to 3.6.0+. When it comes
to managing lots of clusters it can be a problem. So enabling this new
parameter before migration to 3.6.+ and disabling it after, makes migration
easier.
was:
h2. Background
There is a miss in Kafka versions [3.1.x; 3.6.0) what we can enable tiering,
which is not working on that versions at all, and when we upgrade kafka to
version 3.6.0 we have crash restart loop with message "You have to delete all
topics with the property remote.storage.enable=true before disabling tiered
storage cluster-wide".
https://github.com/apache/kafka/blob/43c635f3a48fcb16dd34eac16def379141912e77/storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java#L564
h3.
The Idea
The idea is to create new parameter which allows to wipe configurations related
to remote storage on topic level, with default value set to false (which won't
change logic from the current one), but in case we manually set parameter to
true, those remote topic parameters will be wiped.
Of course additional checks required. Like do it only on Kafka startup and
ensure that there is no data on remote storage etc...
h3. Motivation
Since tiering in to working on version 3.6.0-, there is no any reason to keep
this configs during migration from older 3rd version to 3.6.0+. When it comes
to managing lots of clusters it can be a problem. So enabling this new
parameter before migration to 3.6.+ and disabling it after, makes migration
easier.
> Add additional parameter which allows parameter remote.enabled and its
> relatives to be wiped if server tiering is disabled
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15990
> URL: https://issues.apache.org/jira/browse/KAFKA-15990
> Project: Kafka
> Issue Type: Improvement
> Components: Tiered-Storage
> Reporter: Viktor Nikitash
> Priority: Major
>
> h2. Background
> There is a miss in Kafka versions [3.1.x; 3.6.0) what we can enable tiering,
> which is not working on that versions at all, and when we upgrade kafka to
> version 3.6.0 we have crash restart loop with message "You have to delete all
> topics with the property remote.storage.enable=true before disabling tiered
> storage cluster-wide".
> [https://github.com/apache/kafka/blob/43c635f3a48fcb16dd34eac16def379141912e77/storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java#L564]
> h2. The Idea
> The idea is to create new parameter which allows to wipe configurations
> related to remote storage on topic level, with default value set to false
> (which won't change logic from the current one), but in case we manually set
> parameter to true, those remote topic parameters will be wiped.
> Of course additional checks required. Like do it only on Kafka startup and
> ensure that there is no data on remote storage etc...
> h2. Motivation
> Since tiering in to working on version 3.6.0-, there is no any reason to keep
> this configs during migration from older 3rd version to 3.6.0+. When it comes
> to managing lots of clusters it can be a problem. So enabling this new
> parameter before migration to 3.6.+ and disabling it after, makes migration
> easier.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
