timmy-mathew-ah opened a new pull request, #6697: URL: https://github.com/apache/camel-k/pull/6697
# Summary This PR is intended primarily as a proof of concept and a source of inspiration rather than something that is expected to be merged in its current form even though the implementation has been fully tested with the code before the release of this fix: https://github.com/apache/camel-k/issues/6616 which solves the issue but with a static list of namespaces rather than dynamic. This PR introduces support for running a namespaced Camel K operator across multiple namespaces without requiring cluster-wide resource access. In addition to supporting a statically configured list of namespaces, the operator can dynamically discover namespaces based on Kubernetes labels and automatically adjust the set of watched namespaces as labels are added or removed. Because `controller-runtime` does not allow the watched namespace set to be modified after the manager has been created, the implementation uses a graceful self-restart strategy whenever the effective namespace set changes. This allows the operator to rebuild its caches with the updated namespace configuration while preserving the security benefits of namespaced RBAC. The implementation is fully compatible with the existing operator sharding mechanism and supports multiple operator shards watching the same dynamically discovered namespaces. # Detailed Design This PR introduces two new, fully backward-compatible capabilities for the namespaced (non-global) Camel K operator: 1. **Static multi-namespace support** – `WATCH_NAMESPACE` now accepts a comma-separated list of namespaces. 2. **Dynamic namespace discovery** – a new `WATCH_NAMESPACE_SELECTOR` environment variable enables the operator to automatically discover namespaces based on Kubernetes labels and update the watched namespace set as namespaces are added or removed. ## Motivation On large multi-tenant clusters, running the operator in global mode requires cluster-wide access to workloads, secrets, and other resources, significantly increasing the blast radius if the operator is compromised. Conversely, the existing single-namespace deployment model does not scale well when many teams require their own namespaces. This proposal provides a middle ground by allowing the operator to watch only explicitly authorized namespaces while keeping RBAC scoped to those namespaces instead of cluster level. Static namespace lists and dynamic label-based discovery are both supported. ## Controller-runtime constraint The design is primarily driven by a limitation of `controller-runtime`. The namespace scope configured through `cache.Options.DefaultNamespaces` is fixed when the manager is created and cannot be modified while the process is running. Although multiple namespaces are already supported, there is no supported mechanism for dynamically adding or removing namespaces from an existing cache. Several approaches were evaluated: 1. **Cluster-scoped cache with filtering** Rejected because it requires cluster-wide resource permissions, defeating the primary security objective. 2. **One cache per namespace managed dynamically** Rejected because it would require substantial changes to controller wiring and significantly increase implementation complexity. 3. **Graceful restart when the watched namespace set changes** Chosen because it preserves scoped RBAC while allowing dynamic namespace discovery. When the effective watched namespace set changes, the operator gracefully shuts down, rebuilds its caches, and resumes reconciliation using the updated namespace configuration. Since reconciliation is level-based, no work is lost during the restart. # Implementation Overview ## Static multi-namespace support When `WATCH_NAMESPACE` contains multiple comma-separated namespaces (for example `team-a,team-b,team-c`), the operator watches those namespaces together with its own installation namespace. - Existing single-namespace behavior remains unchanged. - Leaving `WATCH_NAMESPACE` empty or unset continues to enable global mode. - No additional cluster-scoped RBAC is required beyond the existing namespaced `Role` and `RoleBinding` resources in each watched namespace. ## Dynamic namespace discovery When `WATCH_NAMESPACE_SELECTOR` is configured (for example `camel-k-enabled=true`), the operator computes its watched namespace set as the union of: - the operator namespace, - namespaces specified in `WATCH_NAMESPACE`, and - all namespaces matching the configured label selector. A dedicated cluster-scoped namespace watcher monitors namespace events. When a namespace begins matching or stops matching the selector, the operator performs a graceful restart so that `controller-runtime` rebuilds its caches using the updated namespace set. Before adding a namespace to the watched set, the operator performs a `SelfSubjectAccessReview` to verify that it has permission to watch Camel K resources in that namespace. Namespaces lacking the required RBAC are skipped with a warning rather than preventing the operator from starting or synchronizing its caches. To avoid excessive restarts, namespace changes are debounced and periodically re-evaluated. This also allows deployments where RBAC is installed after labels have already been applied to recover automatically without requiring any manual intervention. The namespace watcher itself is leader-election aware, ensuring that only the active leader monitors namespace changes and initiates restarts. # Operator sharding compatibility The implementation is fully compatible with the existing operator sharding model. Multiple operator shards can independently discover and watch the same label-selected namespaces while reconciling only resources assigned to their respective `operator.id`. The accompanying RBAC helper also supports granting permissions to multiple operator ServiceAccounts within a single namespace, simplifying deployments that use multiple shards. # What this PR includes - Support for static multi-namespace watching through `WATCH_NAMESPACE`. - Support for dynamic namespace discovery through `WATCH_NAMESPACE_SELECTOR`. - A cluster-scoped namespace watcher that gracefully restarts the operator when the effective watched namespace set changes. - Pre-flight RBAC validation using `SelfSubjectAccessReview`. - Debouncing and periodic reconciliation of namespace membership changes. - Helm chart support for both static and dynamic configuration. - Minimal additional cluster-scoped RBAC limited to namespace discovery. - Documentation covering configuration and installation. - Unit tests covering namespace computation and watcher behavior. The remaining sections describe the implementation details, installation examples, sharded deployment configuration, and related Helm changes. ## Installing an operator that watches multiple namespaces by label Install into the operator namespace (e.g. `camel-k`) in dynamic mode: ```bash helm install camel-k ./helm/camel-k -n camel-k --create-namespace \ --set-string operator.global=false \ --set operator.watchNamespaceSelector="camel-k-enabled=true" ``` This renders the operator + namespaced RBAC in `camel-k`, plus a minimal cluster-scoped `namespaces`-read `ClusterRole` bound to the operator SA. > `--set-string` matters for `operator.global`: the chart compares it as a string, so a bare > `--set operator.global=false` (parsed as a boolean) fails template rendering. Onboard each team namespace — **install the RBAC first, then label it** (the label is the "ready" signal): ```bash # 1. Grant the operator its namespaced rights in the target namespace, # bound to the operator's ServiceAccount in the operator namespace. ./script/install_namespace_rbac.sh team-a camel-k camel-k-operator # 2. Label the namespace so the operator starts watching it. kubectl label namespace team-a camel-k-enabled=true ``` The operator detects the change and self-restarts to watch `team-a`. If you label before installing RBAC, the namespace is skipped with a warning and self-heals on the next periodic re-evaluation — no re-labelling needed. Removing the label (or deleting the namespace) makes the operator stop watching it; Integrations already running there keep running. **Static alternative** (no cluster RBAC, fixed set, changes require `helm upgrade`): ```bash helm install camel-k ./helm/camel-k -n camel-k --create-namespace \ --set-string operator.global=false \ --set "operator.watchNamespaces={team-a,team-b}" ./script/install_namespace_rbac.sh team-a camel-k camel-k-operator ./script/install_namespace_rbac.sh team-b camel-k camel-k-operator ``` ### Combining with multiple operator shards (optional) Multiple operator releases can run side by side (each with a distinct `operator.operatorId`, hence a distinct leader lease) and all watch the same label-selected namespaces, reconciling only the resources whose `camel.apache.org/operator.id` annotation matches their own id. For this model, grant **every** shard's ServiceAccount access in each watched namespace with a single multi-subject binding: ```bash ./script/install_namespace_rbac.sh team-a camel-k camel-k-shard-1-operator,camel-k-shard-2-operator ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
