daric93 opened a new issue, #13751: URL: https://github.com/apache/apisix/issues/13751
### Description As a user running Valkey/Redis-compatible infrastructure, I want to run APISIX data-plane nodes that sync their configuration from Valkey, so that I can operate a decoupled data plane without also running an etcd cluster. **Context** APISIX already supports multiple `config_provider` options for the data-plane role (`etcd`, `yaml`, `json`, `xds`). This proposal adds one more — `valkey` — following the same read-only sync pattern that `json` and `xds` established. It is purely additive: etcd remains the default and nothing about the existing providers changes. Many teams already standardize on Valkey/Redis for their infrastructure (APISIX's own redis-backed plugins are widely used against Valkey today, e.g. #13584, #12282). Letting a data plane sync config from Valkey removes etcd as a hard dependency for those deployments. **Why Valkey can satisfy the config-provider contract** The historical concern with non-etcd config stores (see the [FAQ](https://github.com/apache/apisix/blob/master/docs/en/latest/FAQ.md)) is the need for resumable watch streams, prefix reads, and atomic updates. Valkey now provides primitives that map cleanly onto these: - **Change notification** → Valkey Streams (`XREADGROUP` with consumer groups): resumable, at-least-once delivery from the last processed ID, which is exactly the "watch stream" semantic etcd provides. This is stronger than the `yaml` provider's 1s polling. - **Prefix / directory reads** → one Hash per collection (`{apisix}:routes`, `{apisix}:services`, ...): `HGETALL` returns all items in a collection atomically. - **Revision tracking** → an application-managed counter key (`{apisix}:revision`), `INCR`ed on each change, used to synthesize the `conf_version` / `modifiedIndex` that `core.config` consumers rely on. - Hash-tag `{apisix}` keeps keys co-located on one shard so the design is Valkey-cluster-safe. **Proposed design (data-plane, read-only)** - New provider module `apisix/core/config_valkey.lua` implementing the existing `core.config` read contract (`new`, `get`, `fetch_created_obj`, `close`, `init`, `init_worker`, plus `values` / `values_hash` / `conf_version` / `clean_handlers` and `filter` / `checker` / `item_schema` handling) — the same surface `config_yaml.lua` and `config_xds.lua` satisfy today. - Connect via the existing `api7-lua-resty-redis-connector` dependency (already used by the redis plugins). - Add `"valkey"` to the data-plane `config_provider` enum + a `valkey` connection schema (host, port, password, database, tls) in `apisix/cli/schema.lua`, with the corresponding edits in `apisix/cli/file.lua`, `apisix/cli/ops.lua`, and the `config_ready_check` allowlist in `apisix/init.lua`. - Config gets written to Valkey by an external process (or a Valkey-backed standalone controller); the data plane only reads and watches. **Admin API writes are explicitly out of scope for this issue** — this is data-plane sync only, matching how `xds` works. Example configuration: ```yaml # conf/config.yaml deployment: role: data_plane role_data_plane: config_provider: valkey valkey: host: "valkey.example.com" port: 6379 password: "secret" database: 0 tls: enabled: true ``` **Testing** - New `t/config-center-valkey/` directory mirroring `t/config-center-yaml/` and `t/config-center-json/`, using `--- yaml_config` blocks to set the data-plane role + `config_provider: valkey`. - Add a `valkey/valkey` service to `ci/pod/docker-compose.common.yml` and wire `t/config-center-valkey` into the corresponding test group in `.github/workflows/build.yml`. **Scope note** This is deliberately scoped to the data-plane read path — the pattern with clear precedent. Extending the option to the Admin API / control-plane write path is a larger, separate discussion (it would need a write-side storage abstraction, since the Admin API calls `core.etcd` directly today) and is not part of this proposal. I'd like to gauge maintainer interest before writing code, per CONTRIBUTING.md. Does this direction sound acceptable, and are there constraints I should account for in the design? I'm happy to implement it. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
