clolov commented on code in PR #14528: URL: https://github.com/apache/kafka/pull/14528#discussion_r1354990908
########## docs/ops.html: ########## @@ -3984,27 +3984,73 @@ <h5 class="anchor-heading"><a id="tiered_storage_config_topic" class="anchor-lin If unset, The value in <code>retention.ms</code> and <code>retention.bytes</code> will be used. </p> -<h4 class="anchor-heading"><a id="tiered_storage_config_ex" class="anchor-link"></a><a href="#tiered_storage_config_ex">Configurations Example</a></h4> +<h4 class="anchor-heading"><a id="tiered_storage_config_ex" class="anchor-link"></a><a href="#tiered_storage_config_ex">Quick Start Example</a></h4> + +<p>Apache Kafka doesn't provide an out-of-the-box RemoteStorageManager implementation. To have a preview of the tiered storage + feature, the `LocalTieredStorage` implemented for integration test can be used, which will create a temporary directory in local storage + to simulate the remote storage. +</p> + +<p>To adopt the `LocalTieredStorage`, the test library needs to be built locally</p> +<pre>./gradlew :storage:testJar</pre> +<p>After build successfully, there should be a `kafka-storage-x.x.x-SNAPSHOT-test.jar` file under `storage/build/libs`. +Next, setting configurations in the broker side to enable tiered storage feature.</p> -<p>Here is a sample configuration to enable tiered storage feature in broker side: <pre> # Sample Zookeeper/Kraft broker server.properties listening on PLAINTEXT://:9092 remote.log.storage.system.enable=true -# Please provide the implementation for remoteStorageManager. This is the mandatory configuration for tiered storage. -# remote.log.storage.manager.class.name=org.apache.kafka.server.log.remote.storage.NoOpRemoteStorageManager + # Using the "PLAINTEXT" listener for the clients in RemoteLogMetadataManager to talk to the brokers. remote.log.metadata.manager.listener.name=PLAINTEXT + +# Please provide the implementation info for remoteStorageManager. This is the mandatory configuration for tiered storage. +# Here, we use the `LocalTieredStorage` built above. +remote.log.storage.manager.class.name=org.apache.kafka.server.log.remote.storage.LocalTieredStorage +remote.log.storage.manager.class.path=/Users/luke/dev/kafka/storage/build/libs/kafka-storage-x.x.x-SNAPSHOT-test.jar + +# Configure the directory used for `LocalTieredStorage` +rsm.config.dir=/tmp/kafka-remote-storage + +# This needs to be changed if broker numbers are more than 1 +rlmm.config.remote.log.metadata.topic.replication.factor=1 </pre> </p> -<p>After broker is started, creating a topic with tiered storage enabled, and a small log time retention value to try this feature: -<pre>bin/kafka-topics.sh --create --topic tieredTopic --bootstrap-server localhost:9092 --config remote.storage.enable=true --config local.retention.ms=1000 +<p>After broker is started, creating a topic with tiered storage enabled with configs: + +<pre> +# remote.storage.enable=true -> enabling tiered storage +# local.retention.ms=1000 -> the time before local segments are moving to remote storage (and then deleted) Review Comment: ```suggestion # local.retention.ms=1000 -> the time after which local copies of segments moved to remote storage are legible for deletion ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
