This is an automated email from the ASF dual-hosted git repository.
emaynard pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push:
new 3844fae96 fix(helm): Fix features configuration section & enhance docs
(#1638)
3844fae96 is described below
commit 3844fae96559e506135fc424381d9ea5a322d37e
Author: Alexandre Dutra <[email protected]>
AuthorDate: Wed May 21 19:04:10 2025 +0200
fix(helm): Fix features configuration section & enhance docs (#1638)
Fixes #1618 .
Fixes #1634.
This change adapts the `Values.features` section to the changes from #1572,
also fixing incorrect configmap entries produced by the template.
It also re-syncs the README.md file with README.md.gotmpl (the sync was
lost a few commits ago), and enhances the instructions for running Helm tests
locally.
---
helm/polaris/README.md | 121 ++++++++++++++++++++++++++--------
helm/polaris/README.md.gotmpl | 116 ++++++++++++++++++++++++--------
helm/polaris/ci/features-values.yaml | 31 +++++++++
helm/polaris/templates/configmap.yaml | 2 +
helm/polaris/values.yaml | 13 ++--
5 files changed, 221 insertions(+), 62 deletions(-)
diff --git a/helm/polaris/README.md b/helm/polaris/README.md
index e6707091f..74b3e55ad 100644
--- a/helm/polaris/README.md
+++ b/helm/polaris/README.md
@@ -46,33 +46,27 @@ A Helm chart for Apache Polaris (incubating).
## Installation
-### Optional
+### Prerequisites
-When using a custom `persistence.xml`, a Kubernetes Secret must be created for
it. Below is a sample command:
-```bash
-kubectl create secret generic polaris-secret -n polaris
--from-file=persistence.xml
-```
-
-### Running the unit tests
-
-Helm unit tests do not require a Kubernetes cluster. To run the unit tests
from the Polaris repo
-root:
+When using the (deprecated) EclipseLink-backed metastore, a custom
`persistence.xml` is required,
+and a Kubernetes Secret must be created for it. Below is a sample command:
```bash
-helm unittest helm/polaris
+kubectl create secret generic polaris-secret -n polaris
--from-file=persistence.xml
```
### Running locally with a Kind cluster
The below instructions assume Kind and Helm are installed.
-Simply run the `run.sh` script from the Polaris repo root, making sure to
specify the
-`--eclipse-link-deps` option:
+Simply run the `run.sh` script from the Polaris repo root:
```bash
./run.sh
```
+If using the EclipseLink-backed metastore, make sure to specify the
`--eclipse-link-deps` option.
+
This script will create a Kind cluster, deploy a local Docker registry, build
the Polaris Docker
images with support for Postgres and load them into the Kind cluster. (It will
also create an
example Deployment and Service with in-memory storage.)
@@ -100,27 +94,24 @@ The below instructions assume a local Kubernetes cluster
is running and Helm is
#### Common setup
-Create and populate the target namespace:
+Create the target namespace:
```bash
kubectl create namespace polaris
-kubectl apply --namespace polaris -f helm/polaris/ci/fixtures/
-
-kubectl wait --namespace polaris --for=condition=ready pod
--selector=app.kubernetes.io/name=postgres --timeout=120s
```
-The `helm/polaris/ci` contains a number of values files that can be used to
install the chart with
-different configurations.
-
-You can also run `ct` (chart-testing):
-
-```bash
-ct lint --charts helm/polaris
-ct install --namespace polaris --debug --charts ./helm/polaris
-```
+Create all the required resources in the `polaris` namespace. This usually
includes a Postgres
+database and a Kubernetes Secret for the `persistence.xml` file. The Polaris
chart does not create
+these resources automatically, as they are not required for all Polaris
deployments. The chart will
+fail if these resources are not created beforehand.
Below are two sample deployment models for installing the chart: one with a
non-persistent backend and another with a persistent backend.
+> [!WARNING]
+> The examples below use values files located in the `helm/polaris/ci`
directory.
+> **These files are intended for testing purposes primarily, and may not be
suitable for production use**.
+> For production deployments, create your own values files based on the
provided examples.
+
#### Non-persistent backend
Install the chart with a non-persistent backend. From Polaris repo root:
@@ -131,6 +122,16 @@ helm upgrade --install --namespace polaris \
polaris helm/polaris
```
+Note: if you are running the tests on a Kind cluster started with the `run.sh`
command explained
+above, then you need to run `helm upgrade` as follows:
+
+```bash
+helm upgrade --install --namespace polaris \
+ --debug --values helm/polaris/ci/simple-values.yaml \
+ --set=image.repository=localhost:5001/apache/polaris \
+ polaris helm/polaris
+```
+
#### Persistent backend
> [!WARNING]
@@ -184,6 +185,69 @@ kubectl delete --namespace polaris -f
helm/polaris/ci/fixtures/
kubectl delete namespace polaris
```
+## Development & Testing
+
+This section is intended for developers who want to run the Polaris Helm chart
tests.
+
+### Prerequisites
+
+The following tools are required to run the tests:
+
+* [Helm Unit Test](https://github.com/helm-unittest/helm-unittest)
+* [Chart Testing](https://github.com/helm/chart-testing)
+
+Quick installation instructions for these tools:
+
+```bash
+helm plugin install https://github.com/helm-unittest/helm-unittest.git
+brew install chart-testing
+```
+
+The integration tests also require some fixtures to be deployed. The
`ci/fixtures` directory
+contains the required resources. To deploy them, run the following command:
+
+```bash
+kubectl apply --namespace polaris -f helm/polaris/ci/fixtures/
+kubectl wait --namespace polaris --for=condition=ready pod
--selector=app.kubernetes.io/name=postgres --timeout=120s
+```
+
+The `helm/polaris/ci` contains a number of values files that will be used to
install the chart with
+different configurations.
+
+### Running the unit tests
+
+Helm unit tests do not require a Kubernetes cluster. To run the unit tests,
execute Helm Unit from
+the Polaris repo root:
+
+```bash
+helm unittest helm/polaris
+```
+
+You can also lint the chart using the Chart Testing tool, with the following
command:
+
+```bash
+ct lint --charts helm/polaris
+```
+
+### Running the integration tests
+
+Integration tests require a Kubernetes cluster. See installation instructions
above for setting up
+a local cluster.
+
+Integration tests are run with the Chart Testing tool:
+
+```bash
+ct install --namespace polaris --debug --charts ./helm/polaris
+```
+
+Note: if you are running the tests on a Kind cluster started with the `run.sh`
command explained
+above, then you need to run `ct install` as follows:
+
+```bash
+ct install --namespace polaris --debug --charts ./helm/polaris \
+ --helm-extra-set-args "--set=image.repository=localhost:5001/apache/polaris"
+```
+
## Values
| Key | Type | Default | Description |
@@ -219,8 +283,7 @@ kubectl delete namespace polaris
| extraServices | list | `[]` | Additional service definitions. All service
definitions always select all Polaris pods. Use this if you need to expose
specific ports with different configurations, e.g. expose polaris-http with an
alternate LoadBalancer service instead of ClusterIP. |
| extraVolumeMounts | list | `[]` | Extra volume mounts to add to the polaris
container. See https://kubernetes.io/docs/concepts/storage/volumes/. |
| extraVolumes | list | `[]` | Extra volumes to add to the polaris pod. See
https://kubernetes.io/docs/concepts/storage/volumes/. |
-| features | object | `{"defaults":{},"realmOverrides":{}}` | Polaris features
configuration. |
-| features.defaults | object | `{}` | Features to enable or disable globally.
If a feature is not present in the map, the default built-in value is used. |
+| features | object | `{"realmOverrides":{}}` | Polaris features
configuration. |
| features.realmOverrides | object | `{}` | Features to enable or disable per
realm. This field is a map of maps. The realm name is the key, and the value is
a map of feature names to values. If a feature is not present in the map, the
default value from the 'defaults' field is used. |
| fileIo | object | `{"type":"default"}` | Polaris FileIO configuration. |
| fileIo.type | string | `"default"` | The type of file IO to use. Two
built-in types are supported: default and wasb. The wasb one translates WASB
paths to ABFS ones. |
@@ -285,7 +348,7 @@ kubectl delete namespace polaris
| persistence.eclipseLink.secret | object |
`{"key":"persistence.xml","name":null}` | The secret name to pull
persistence.xml from. |
| persistence.eclipseLink.secret.key | string | `"persistence.xml"` | The key
in the secret to pull persistence.xml from. |
| persistence.eclipseLink.secret.name | string | `nil` | The name of the
secret to pull persistence.xml from. If not provided, the default built-in
persistence.xml will be used. This is probably not what you want. |
-| persistence.type | string | `"relational-jdbc"` | Three built-in types are
available: "relational-jdbc", "in-memory", "eclipse-link". The in-memory type
is not recommended for production use. The eclipse-link type is deprecated and
will be unsupported in a future release. |
+| persistence.type | string | `"eclipse-link"` | The type of persistence to
use. Two built-in types are supported: in-memory and eclipse-link. |
| podAnnotations | object | `{}` | Annotations to apply to polaris pods. |
| podLabels | object | `{}` | Additional Labels to apply to polaris pods. |
| podSecurityContext | object |
`{"fsGroup":10001,"seccompProfile":{"type":"RuntimeDefault"}}` | Security
context for the polaris pod. See
https://kubernetes.io/docs/tasks/configure-pod-container/security-context/. |
diff --git a/helm/polaris/README.md.gotmpl b/helm/polaris/README.md.gotmpl
index d478c6518..94a233ada 100644
--- a/helm/polaris/README.md.gotmpl
+++ b/helm/polaris/README.md.gotmpl
@@ -47,33 +47,27 @@
## Installation
-### Optional
+### Prerequisites
-When using a custom `persistence.xml`, a Kubernetes Secret must be created for
it. Below is a sample command:
-```bash
-kubectl create secret generic polaris-secret -n polaris
--from-file=persistence.xml
-```
-
-### Running the unit tests
-
-Helm unit tests do not require a Kubernetes cluster. To run the unit tests
from the Polaris repo
-root:
+When using the (deprecated) EclipseLink-backed metastore, a custom
`persistence.xml` is required,
+and a Kubernetes Secret must be created for it. Below is a sample command:
```bash
-helm unittest helm/polaris
+kubectl create secret generic polaris-secret -n polaris
--from-file=persistence.xml
```
### Running locally with a Kind cluster
The below instructions assume Kind and Helm are installed.
-Simply run the `run.sh` script from the Polaris repo root, making sure to
specify the
-`--eclipse-link-deps` option:
+Simply run the `run.sh` script from the Polaris repo root:
```bash
./run.sh
```
+If using the EclipseLink-backed metastore, make sure to specify the
`--eclipse-link-deps` option.
+
This script will create a Kind cluster, deploy a local Docker registry, build
the Polaris Docker
images with support for Postgres and load them into the Kind cluster. (It will
also create an
example Deployment and Service with in-memory storage.)
@@ -101,27 +95,24 @@ The below instructions assume a local Kubernetes cluster
is running and Helm is
#### Common setup
-Create and populate the target namespace:
+Create the target namespace:
```bash
kubectl create namespace polaris
-kubectl apply --namespace polaris -f helm/polaris/ci/fixtures/
-
-kubectl wait --namespace polaris --for=condition=ready pod
--selector=app.kubernetes.io/name=postgres --timeout=120s
```
-The `helm/polaris/ci` contains a number of values files that can be used to
install the chart with
-different configurations.
-
-You can also run `ct` (chart-testing):
-
-```bash
-ct lint --charts helm/polaris
-ct install --namespace polaris --debug --charts ./helm/polaris
-```
+Create all the required resources in the `polaris` namespace. This usually
includes a Postgres
+database and a Kubernetes Secret for the `persistence.xml` file. The Polaris
chart does not create
+these resources automatically, as they are not required for all Polaris
deployments. The chart will
+fail if these resources are not created beforehand.
Below are two sample deployment models for installing the chart: one with a
non-persistent backend and another with a persistent backend.
+> [!WARNING]
+> The examples below use values files located in the `helm/polaris/ci`
directory.
+> **These files are intended for testing purposes primarily, and may not be
suitable for production use**.
+> For production deployments, create your own values files based on the
provided examples.
+
#### Non-persistent backend
Install the chart with a non-persistent backend. From Polaris repo root:
@@ -132,6 +123,16 @@ helm upgrade --install --namespace polaris \
polaris helm/polaris
```
+Note: if you are running the tests on a Kind cluster started with the `run.sh`
command explained
+above, then you need to run `helm upgrade` as follows:
+
+```bash
+helm upgrade --install --namespace polaris \
+ --debug --values helm/polaris/ci/simple-values.yaml \
+ --set=image.repository=localhost:5001/apache/polaris \
+ polaris helm/polaris
+```
+
#### Persistent backend
> [!WARNING]
@@ -185,4 +186,67 @@ kubectl delete --namespace polaris -f
helm/polaris/ci/fixtures/
kubectl delete namespace polaris
```
+## Development & Testing
+
+This section is intended for developers who want to run the Polaris Helm chart
tests.
+
+### Prerequisites
+
+The following tools are required to run the tests:
+
+* [Helm Unit Test](https://github.com/helm-unittest/helm-unittest)
+* [Chart Testing](https://github.com/helm/chart-testing)
+
+Quick installation instructions for these tools:
+
+```bash
+helm plugin install https://github.com/helm-unittest/helm-unittest.git
+brew install chart-testing
+```
+
+The integration tests also require some fixtures to be deployed. The
`ci/fixtures` directory
+contains the required resources. To deploy them, run the following command:
+
+```bash
+kubectl apply --namespace polaris -f helm/polaris/ci/fixtures/
+kubectl wait --namespace polaris --for=condition=ready pod
--selector=app.kubernetes.io/name=postgres --timeout=120s
+```
+
+The `helm/polaris/ci` contains a number of values files that will be used to
install the chart with
+different configurations.
+
+### Running the unit tests
+
+Helm unit tests do not require a Kubernetes cluster. To run the unit tests,
execute Helm Unit from
+the Polaris repo root:
+
+```bash
+helm unittest helm/polaris
+```
+
+You can also lint the chart using the Chart Testing tool, with the following
command:
+
+```bash
+ct lint --charts helm/polaris
+```
+
+### Running the integration tests
+
+Integration tests require a Kubernetes cluster. See installation instructions
above for setting up
+a local cluster.
+
+Integration tests are run with the Chart Testing tool:
+
+```bash
+ct install --namespace polaris --debug --charts ./helm/polaris
+```
+
+Note: if you are running the tests on a Kind cluster started with the `run.sh`
command explained
+above, then you need to run `ct install` as follows:
+
+```bash
+ct install --namespace polaris --debug --charts ./helm/polaris \
+ --helm-extra-set-args "--set=image.repository=localhost:5001/apache/polaris"
+```
+
{{ template "chart.valuesSection" . }}
diff --git a/helm/polaris/ci/features-values.yaml
b/helm/polaris/ci/features-values.yaml
new file mode 100644
index 000000000..cf44ce0c0
--- /dev/null
+++ b/helm/polaris/ci/features-values.yaml
@@ -0,0 +1,31 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+image:
+ pullPolicy: Never
+
+features:
+ ENFORCE_PRINCIPAL_CREDENTIAL_ROTATION_REQUIRED_CHECKING: false
+ SUPPORTED_CATALOG_STORAGE_TYPES:
+ - S3
+ realmOverrides:
+ POLARIS:
+ ENFORCE_PRINCIPAL_CREDENTIAL_ROTATION_REQUIRED_CHECKING: true
+ SUPPORTED_CATALOG_STORAGE_TYPES:
+ - GCS
diff --git a/helm/polaris/templates/configmap.yaml
b/helm/polaris/templates/configmap.yaml
index ae732760c..6d12fbd7e 100644
--- a/helm/polaris/templates/configmap.yaml
+++ b/helm/polaris/templates/configmap.yaml
@@ -37,8 +37,10 @@ data:
{{- /* Features */ -}}
{{- range $k, $v := .Values.features -}}
+ {{- if ( ne $k "realmOverrides" ) -}}
{{- $_ = set $map (printf "polaris.features.\"%s\"" $k) (toJson $v) -}}
{{- end -}}
+ {{- end -}}
{{- range $realm, $overrides := .Values.features.realmOverrides -}}
{{- range $k, $v := $overrides -}}
{{- $_ = set $map (printf "polaris.features.realm-overrides.\"%s\".\"%s\""
$realm $k) (toJson $v) -}}
diff --git a/helm/polaris/values.yaml b/helm/polaris/values.yaml
index 04a1013cf..5cf964f20 100644
--- a/helm/polaris/values.yaml
+++ b/helm/polaris/values.yaml
@@ -507,13 +507,12 @@ realmContext:
features:
# -- Features to enable or disable globally. If a feature is not present in
the map, the default
# built-in value is used.
- defaults: {}
- # ENFORCE_PRINCIPAL_CREDENTIAL_ROTATION_REQUIRED_CHECKING: false
- # SUPPORTED_CATALOG_STORAGE_TYPES:
- # - S3
- # - GCS
- # - AZURE
- # - FILE
+ # ENFORCE_PRINCIPAL_CREDENTIAL_ROTATION_REQUIRED_CHECKING: false
+ # SUPPORTED_CATALOG_STORAGE_TYPES:
+ # - S3
+ # - GCS
+ # - AZURE
+ # - FILE
# -- Features to enable or disable per realm. This field is a map of maps.
The realm name is the key, and the value is a map of
# feature names to values. If a feature is not present in the map, the
default value from the 'defaults' field is used.
realmOverrides: {}
