Demogorgon314 commented on code in PR #23033: URL: https://github.com/apache/pulsar/pull/23033#discussion_r1680615906
########## pip/pip-366.md: ########## @@ -0,0 +1,70 @@ +# PIP-366: Support to specify different config for Configuration and Local Metadata Store + +# Background knowledge + +Pulsar metadata store maintains all the metadata, configuration, and coordination of a Pulsar cluster, such as topic metadata, schema, broker load data, and so on. + +The metadata store of each Pulsar instance should contain the following two components: + +- A local metadata store ensemble (`metadataStoreUrl`) that stores cluster-specific configuration and coordination, such as which brokers are responsible for which topics as well as ownership metadata, broker load reports, and BookKeeper ledger metadata. +- A configuration store quorum (`configurationMetadataStoreUrl`) stores configuration for clusters, tenants, namespaces, topics, and other entities that need to be globally consistent. + +# Motivation + +When using Geo-Replication and global configuration store for configuration global consistency, the configuration store's config may be different from the local metadata store's config. For example, the configuration store may have a different set of ZooKeeper servers than the local metadata store. + +The global configuration store may deploy in a different data center, and the local metadata store may be deployed in the same data center as the Pulsar broker. In this case, the global configuration store may need to use TLS and authentication to protect the connection to metadata store server, while the local metadata store may not need to use TLS and authentication. + +However, the current implementation of Pulsar only supports configuring different metadata store url for the local metadata store and the configuration store. This limitation makes it impossible to support the above scenario. + +# Goals + +## In Scope + +- Support specifying different configurations for the local metadata store and the configuration store. + +# Detailed Design + +## Design & Implementation Details + +Pulsar support `metadataStoreConfigPath` configuration, but it only supports for `RocksdbMetadataStore`, and it is not able to specify different configuration for Configuration Metadata Store. + +```java + @FieldContext( + category = CATEGORY_SERVER, + doc = "Configuration file path for local metadata store. It's supported by RocksdbMetadataStore for now." + ) + private String metadataStoreConfigPath = null; +``` + +Therefore, we need to add a new configuration `configurationMetadataStoreConfigPath` for `ConfigurationMetadataStore`, and `metadataStoreConfigPath` for `LocalMetadataStore`. + +```java + @FieldContext( + category = CATEGORY_SERVER, + doc = "Configuration file path for configuration metadata store." + ) + private String configurationMetadataStoreConfigPath = null; Review Comment: Updated, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
