This is an automated email from the ASF dual-hosted git repository. morhidi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push: new 66953e01 Add OLM bundle generator 66953e01 is described below commit 66953e01dcfa9ed1b31a7b6b854373cce477a8db Author: ted chang <htch...@us.ibm.com> AuthorDate: Wed Nov 30 13:37:13 2022 -0800 Add OLM bundle generator Signed-off-by: ted chang <htch...@us.ibm.com> --- tools/olm/AutomatedOLMBuildAndDeploy.sh | 86 ++++++++ tools/olm/README.md | 144 ++++++++++++++ ...-kubernetes-operator.clusterserviceversion.yaml | 216 +++++++++++++++++++++ tools/olm/csv-template/bases/initContainer.yaml | 40 ++++ tools/olm/csv-template/bundle.Dockerfile | 37 ++++ tools/olm/csv-template/catalog.Dockerfile | 33 ++++ tools/olm/docker-entry.sh | 114 +++++++++++ tools/olm/env.sh | 42 ++++ tools/olm/generate-olm-bundle.sh | 153 +++++++++++++++ tools/olm/install-prereq.sh | 97 +++++++++ tools/olm/utils.Dockerfile | 43 ++++ 11 files changed, 1005 insertions(+) diff --git a/tools/olm/AutomatedOLMBuildAndDeploy.sh b/tools/olm/AutomatedOLMBuildAndDeploy.sh new file mode 100755 index 00000000..359388a5 --- /dev/null +++ b/tools/olm/AutomatedOLMBuildAndDeploy.sh @@ -0,0 +1,86 @@ +#!/bin/bash +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +set +x +# Script that Automates the OLM Creation and test of Flink Operator + +# Run the generate-olm-bundle.sh script +./generate-olm-bundle.sh + +# Reset Cluster +kind delete cluster +kind create cluster +kubectl cluster-info --context kind-kind +# operator-sdk olm install +curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh | bash -s v0.22.0 + +# Deploy the catalog src +cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha1 +kind: CatalogSource +metadata: + name: olm-flink-operator-catalog + namespace: default +spec: + sourceType: grpc + image: "${DOCKER_REGISTRY}/${DOCKER_ORG}/flink-op-catalog:${BUNDLE_VERSION}" +EOF + +# sleep 40 seconds wait for the catalog serving pod to start +echo "Sleeping 40 seconds" +sleep 40 +# Check that the image is up: +kubectl get pods -n default |grep flink + +# Deploy the subscription in the default namespace: +cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha2 +kind: OperatorGroup +metadata: + name: default-og + namespace: default +spec: + # if not set, default to watch all namespaces + targetNamespaces: + - default +--- +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: flink-kubernetes-operator + namespace: default +spec: + channel: alpha + name: flink-kubernetes-operator + source: olm-flink-operator-catalog + sourceNamespace: default + # For testing upgrade from previous version + #installPlanApproval: Automatic # Manual + #startingCSV: "flink-kubernetes-operator.v${PREVIOUS_BUNDLE_VERSION}" +EOF + +# sleep 40 seconds +echo "Sleeping 40 seconds" +sleep 40 + +# Check the image is running +kubectl get pods,csv,ip,sub + +# Deploy the sample: +echo "Run a Flink job, use: kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.2/examples/basic.yaml" + diff --git a/tools/olm/README.md b/tools/olm/README.md new file mode 100644 index 00000000..06110f65 --- /dev/null +++ b/tools/olm/README.md @@ -0,0 +1,144 @@ +# OLM Integration + +The document describes the automated flink-kubernetes-operator OLM bundle generation process for each release. For every release, we publish new bundle to two online catalogs. Each catalog has a Github repository that uses similar publishing process. +![image](https://user-images.githubusercontent.com/7155778/202823924-8d08561f-1b5f-4e96-9f87-adc171e699c9.png) +The [community-operators-prod catalog](https://github.com/redhat-openshift-ecosystem/community-operators-prod) comes with OCP (Openshift Container Platform) by default. The [community-operator catalog](https://github.com/k8s-operatorhub/community-operators) can be optionally installed on kubernetes clusters. + +## Prerequisite + +- [minikube](https://minikube.sigs.k8s.io/docs/)(recommended), [kind](https://kind.sigs.k8s.io/), or [OpenShift Local](https://developers.redhat.com/products/openshift-local/overview) +- [kubectl](https://kubernetes.io/docs/tasks/tools/)(Kubernetes) or [oc](https://docs.openshift.com/container-platform/4.11/cli_reference/openshift_cli/getting-started-cli.html#installing-openshift-cli)(OpenShift) +- [Docker](https://www.docker.com/) + +## Generate the bundle locally +Clone the olm repo into the flink-kubernetes-operator repo: +```sh +git clone https://github.com/apache/flink-kubernetes-operator.git +cd flink-kubernetes-operator/tools/olm +``` + +Change the variables in `env.sh` if needed: +```sh +vi env.sh +``` + +Generate bundle and catalog images and push them to repos specified in env.sh. +```sh +./generate-olm-bundle.sh +``` + +Verify the bundle is generated in the target folder. We only need the bundle folder which has the following files: +``` +1.3.0/ +├── bundle.Dockerfile +├── manifests +│ ├── flink.apache.org_flinkdeployments.yaml +│ ├── flink.apache.org_flinksessionjobs.yaml +│ ├── flink-kubernetes-operator.clusterserviceversion.yaml +│ ├── flink-operator-config_v1_configmap.yaml +│ ├── flink-operator-webhook-secret_v1_secret.yaml +│ ├── flink_rbac.authorization.k8s.io_v1_role.yaml +│ ├── flink-role-binding_rbac.authorization.k8s.io_v1_rolebinding.yaml +│ └── flink_v1_serviceaccount.yaml +└── metadata + └── annotations.yaml +``` + +## Deploy bundle using Subscription +With the bundle and catalog images pushed to a image registry, we can now deploy the operator by +creating a Subscription. + +Kubernetes does not come with OLM. To deploy the OLM operator on a cluster, run the following: +```sh +curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh | bash -s v0.22.0 +``` +Deploy private catalog to serves the bundle on a cluster: +```sh +# Deploy the catalog src +cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha1 +kind: CatalogSource +metadata: + name: olm-flink-operator-catalog + namespace: default +spec: + sourceType: grpc + image: "${DOCKER_REGISTRY}/${DOCKER_ORG}/flink-op-catalog:${BUNDLE_VERSION}" +EOF +# wait for the catalog server pod to be ready +kubectl get po -w +``` +Create subscription to deploy the flink-kubernetes-operator: +```sh +# Create a OperatorGroup and subscription in the default namespace: +cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha2 +kind: OperatorGroup +metadata: + name: default-og + namespace: default +spec: + # if not set, default to watch all namespaces + targetNamespaces: + - default +--- +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: flink-kubernetes-operator + namespace: default +spec: + channel: alpha + name: flink-kubernetes-operator + source: olm-flink-operator-catalog + sourceNamespace: default + # For testing upgrade from previous version + #installPlanApproval: Automatic # Manual + #startingCSV: "flink-kubernetes-operator.v${PREVIOUS_BUNDLE_VERSION}" +EOF +# wait for the flink-kubernetes-operator pod to be ready +kubectl get po -w +``` + +Deploy a Flink job to verify the operator: +```sh +kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.2/examples/basic.yaml +``` + +After verifying, the bundle image and catalog image are no longer needed. We only need the bundle folder to run the CI test suits. + + +## Run CI test suits before creating PR + +Clone your forked community-operators repo +```sh +git clone https://github.com/tedhtchang/community-operators.git +``` +The bundle(i.e. 1.3.0) folder is all we need to commit into the community-operators repo. Copy the new bundle into the forked community-operators repo +```sh +cp -r flink-kubernetes-operator/tools/olm/target/1.3.0 community-operators/operators/flink-kubernetes-operator/ +``` + +Run test suites +```sh +cd community-operators +OPP_PRODUCTION_TYPE=k8s OPP_AUTO_PACKAGEMANIFEST_CLUSTER_VERSION_LABEL=1 bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) all operators/flink-kubernetes-operator/1.3.0 +``` +The expected output: +``` +... +Test 'kiwi' : [ OK ] +Test 'lemon_latest' : [ OK ] +Test 'orange_latest' : [ OK ] +``` + +After the tests pass, commit and push the new bundle to a branch and then create a PR in the community-operator repo. + +Repeat the above steps for adding new bundle to the [community-operator-prod](https://github.com/redhat-openshift-ecosystem/community-operators-prod) repo. For this repo, run the CI test suits using: +```sh +cd community-operators-prod +OPP_PRODUCTION_TYPE=ocp OPP_AUTO_PACKAGEMANIFEST_CLUSTER_VERSION_LABEL=1 bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) all operators/flink-kubernetes-operator/1.3.0 +``` +After the tests pass, commit and push the new bundle to a branch and then create a PR in the community-operator-prod repo. + +See detail for running CI test suits [here](https://k8s-operatorhub.github.io/community-operators/operator-test-suite/). diff --git a/tools/olm/csv-template/bases/flink-kubernetes-operator.clusterserviceversion.yaml b/tools/olm/csv-template/bases/flink-kubernetes-operator.clusterserviceversion.yaml new file mode 100644 index 00000000..41bbc265 --- /dev/null +++ b/tools/olm/csv-template/bases/flink-kubernetes-operator.clusterserviceversion.yaml @@ -0,0 +1,216 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# Base ClusterServiceVersion template containing static information +apiVersion: operators.coreos.com/v1alpha1 +kind: ClusterServiceVersion +metadata: + annotations: + alm-examples: |- + [ + { + "apiVersion":"flink.apache.org/v1beta1", + "kind":"FlinkDeployment", + "metadata":{ + "name":"basic-example" + }, + "spec":{ + "image":"flink:1.15", + "flinkVersion":"v1_15", + "flinkConfiguration":{ + "taskmanager.numberOfTaskSlots":"2" + }, + "serviceAccount":"flink", + "jobManager":{ + "resource":{ + "memory":"2048m", + "cpu":1 + } + }, + "taskManager":{ + "resource":{ + "memory":"2048m", + "cpu":1 + } + }, + "job":{ + "jarURI":"local:///opt/flink/examples/streaming/StateMachineExample.jar", + "parallelism":2, + "upgradeMode":"stateless" + } + } + }, + { + "apiVersion":"flink.apache.org/v1beta1", + "kind":"FlinkSessionJob", + "metadata":{ + "name":"basic-session-job-example2" + }, + "spec":{ + "deploymentName":"basic-session-cluster", + "job":{ + "jarURI":"https://repo1.maven.org/maven2/org/apache/flink/flink-examples-streaming_2.12/1.15.0/flink-examples-streaming_2.12-1.15.0.jar", + "parallelism":2, + "upgradeMode":"stateless", + "entryClass":"org.apache.flink.streaming.examples.statemachine.StateMachineExample" + } + } + } + ] + alm-examples-metadata: |- + { + "basic-example": { + "description": "Example flink deployment" + }, + "basic-session-job-example2": { + "description": "Example flink session job" + } + } + capabilities: Deep Insights + categories: Big Data, Streaming & Messaging + containerImage: docker.io/apache/flink-kubernetes-operator:1.1.0 + createdAt: "2022-07-01T00:00:00Z" + description: Flink Kubernetes Operator manages the complete deployment lifecycle + of Apache Flink applications. + operators.operatorframework.io/builder: operator-sdk-v1.23.0 + operators.operatorframework.io/project_layout: unknown + repository: https://github.com/apache/flink-kubernetes-operator.git + support: Apache Software Foundation + name: flink-kubernetes-operator.v1.1.0 +spec: + apiservicedefinitions: {} + customresourcedefinitions: + owned: + - kind: FlinkDeployment + name: flinkdeployments.flink.apache.org + version: v1beta1 + displayName: Flink Deployment + description: The FlinkDeployment CR defines Flink Application and Session cluster deployments + - kind: FlinkSessionJob + name: flinksessionjobs.flink.apache.org + version: v1beta1 + displayName: Flink Session Job + description: The FlinkSessionJob CR defines the session job on the Session cluster and each Session cluster can run multiple FlinkSessionJob. + description: |- + Flink Kubernetes Operator acts as a control plane to manage the complete deployment lifecycle of Apache Flink applications. + ## Installation + This operator can be installed from [OperatorHub.io](https://operatorhub.io/operator/flink-kubernetes-operator). + + By default, the Flink operator monitors and run Flink applications in the same namespace as the operator. + To run Flink jobs in another namespace, users are responsible for setting up the RBAC in that namespace. + + For example, users need to do the following for each additional namespace that runs the Flink applications: + + 1. Switch to the namespace by running: + + ```sh + kubectl config set-context --current --namespace=CHANGEIT + ``` + + 2. Create the service account, role, and role binding in the namespace using the commands below: + + ```sh + kubectl apply -f - <<EOF + apiVersion: v1 + kind: ServiceAccount + metadata: + labels: + app.kubernetes.io/name: flink-kubernetes-operator + app.kubernetes.io/version: 1.2.0 + name: flink + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: Role + metadata: + labels: + app.kubernetes.io/name: flink-kubernetes-operator + app.kubernetes.io/version: 1.2.0 + name: flink + rules: + - apiGroups: + - "" + resources: + - pods + - configmaps + verbs: + - '*' + - apiGroups: + - apps + resources: + - deployments + - deployments/finalizers + verbs: + - '*' + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: RoleBinding + metadata: + labels: + app.kubernetes.io/name: flink-kubernetes-operator + app.kubernetes.io/version: 1.2.0 + name: flink-role-binding + roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: flink + subjects: + - kind: ServiceAccount + name: flink + EOF + ``` + + 3. Proceed deploying Flink applications using the the Custom Resource Definition below. + + ```sh + kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.1/examples/basic.yaml + ``` + + See [Flink custom resources](https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/overview/#overview) for more detail. + displayName: Flink Kubernetes Operator + icon: + - base64data: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPCEtLQpMaWNlbnNlZCB0byB0aGUgQXBhY2hlIFNvZnR3YXJlIEZvdW5kYXRpb24gKEFTRikgdW5kZXIgb25lCm9yIG1vcmUgY29udHJpYnV0b3IgbGljZW5zZSBhZ3JlZW1lbnRzLiAgU2VlIHRoZSBOT1RJQ0UgZmlsZQpkaXN0cmlidXRlZCB3aXRoIHRoaXMgd29yayBmb3IgYWRkaXRpb25hbCBpbmZvcm1hdGlvbgpyZWdhcmRpbmcgY29weXJpZ2h0IG93bmVyc2hpcC4gIFRoZSBBU0YgbGljZW5zZXMgdGhpcyBmaWxlCnRvIHlvdSB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUKIkxpY2Vuc2UiKTsgeW91IG1heSBub3QgdXNlIHRo [...] + mediatype: image/svg+xml + install: + spec: null + strategy: deployment + installModes: + - supported: true + type: OwnNamespace + - supported: true + type: SingleNamespace + - supported: true + type: MultiNamespace + - supported: true + type: AllNamespaces + keywords: + - flink + - streaming + maintainers: + - email: mbala...@apache.org + name: Marton Balassi + - email: htch...@us.ibm.com + name: Ted Chang + - email: jbus...@us.ibm.com + name: James Busche + maturity: beta + provider: + name: " " + url: https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/try-flink-kubernetes-operator/quick-start/ + replaces: + version: 1.1.0 + minKubeVersion: 1.16.0 + diff --git a/tools/olm/csv-template/bases/initContainer.yaml b/tools/olm/csv-template/bases/initContainer.yaml new file mode 100644 index 00000000..a20805f9 --- /dev/null +++ b/tools/olm/csv-template/bases/initContainer.yaml @@ -0,0 +1,40 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +initContainers: + - name: import-cert + image: docker.io/apache/flink-kubernetes-operator:latest + imagePullPolicy: IfNotPresent + command: [ "sh", "-ce","/usr/bin/openssl pkcs12 -export -in /apiserver.local.config/certificates/apiserver.crt -inkey /apiserver.local.config/certificates/apiserver.key -name 1 -passout pass:${WEBHOOK_KEYSTORE_PASSWORD} -out /certs/keystore.p12" ] + volumeMounts: + - name: keystore + mountPath: "/certs" + # ToDo: The name: apiservice-cert volume is injected by the OLM into deployment at runtime. The import-cert initContainer may + # fail if this mechanism is removed in the future. + # https://github.com/operator-framework/operator-lifecycle-manager/blob/master/doc/design/building-your-csv.md#apiservice-serving-certs + - name: apiservice-cert + # This is the default location for OLM managed cert. + mountPath: /apiserver.local.config/certificates + env: + - name: WEBHOOK_KEYSTORE_PASSWORD + valueFrom: + secretKeyRef: + name: flink-operator-webhook-secret + key: password + + + diff --git a/tools/olm/csv-template/bundle.Dockerfile b/tools/olm/csv-template/bundle.Dockerfile new file mode 100644 index 00000000..3c7ac07d --- /dev/null +++ b/tools/olm/csv-template/bundle.Dockerfile @@ -0,0 +1,37 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +FROM scratch + +ARG ARCH=multiarch +ARG USER=1001 + +# Core bundle labels. +LABEL operators.operatorframework.io.bundle.mediatype.v1=registry+v1 +LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/ +LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/ +LABEL operators.operatorframework.io.bundle.package.v1=flink-kubernetes-operator +LABEL operators.operatorframework.io.bundle.channels.v1=latest +LABEL operators.operatorframework.io.metrics.builder=operator-sdk-v1.23.0 +LABEL operators.operatorframework.io.metrics.mediatype.v1=metrics+v1 +LABEL operators.operatorframework.io.metrics.project_layout=unknown + +USER ${USER} + +# Copy files to locations specified by labels. +COPY ./manifests /manifests/ +COPY ./metadata /metadata/ diff --git a/tools/olm/csv-template/catalog.Dockerfile b/tools/olm/csv-template/catalog.Dockerfile new file mode 100644 index 00000000..f929410c --- /dev/null +++ b/tools/olm/csv-template/catalog.Dockerfile @@ -0,0 +1,33 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# The base image is expected to contain +# /bin/opm (with a serve subcommand) and /bin/grpc_health_probe +# This image is used to start a local catalog sever for validating bundle deployment. +FROM quay.io/operator-framework/opm:latest + +# Configure the entrypoint and command +ENTRYPOINT ["/bin/opm"] +CMD ["serve", "/configs", "--cache-dir=/tmp/cache"] + +# Copy declarative config root into image at /configs and pre-populate serve cache +ADD ./config.yaml /configs/config.yaml +RUN ["/bin/opm", "serve", "/configs", "--cache-dir=/tmp/cache", "--cache-only"] + +# Set DC-specific label for the location of the DC root directory +# in the image +LABEL operators.operatorframework.io.index.configs.v1=/configs diff --git a/tools/olm/docker-entry.sh b/tools/olm/docker-entry.sh new file mode 100755 index 00000000..6005b670 --- /dev/null +++ b/tools/olm/docker-entry.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# +# Generates OLM bundle using existing CRDs +# +# +set -euox pipefail + +. env.sh +BUNDLE_VERSION=${BUNDLE_VERSION:-1.2.0} +PREVIOUS_BUNDLE_VERSION=${PREVIOUS_BUNDLE_VERSION:-1.1.0} +BASEDIR="$(dirname "$(realpath "$0")")" +TARGET="${BASEDIR}/target" +BUNDLE="${TARGET}/${BUNDLE_VERSION}" +OPERATOR_IMG=${OPERATOR_IMG:-"docker.io/apache/flink-kubernetes-operator:1.2.0"} + +CHANNELS=alpha +PACKAGE_NAME=flink-kubernetes-operator +CSV_TEMPLATE_DIR="$(dirname "$(realpath "$0")")/csv-template" +# Base ClusterServiceVersion resource containing static information +CSV_TEMPLATE=${CSV_TEMPLATE_DIR}/bases/${PACKAGE_NAME}.clusterserviceversion.yaml +INIT_CONT=${CSV_TEMPLATE_DIR}/bases/initContainer.yaml +CRD_DIR="${BASEDIR}/helm/${PACKAGE_NAME}/crds" +MANIFESTS=${BUNDLE}/manifests +ROLE_BINDING=${MANIFESTS}/flink-role-binding_rbac.authorization.k8s.io_v1_rolebinding.yaml +CRDDIR="${TARGET}/crds" + +# CSV name may contain versioned or constant +# CSV_FILE="${MANIFESTS}/${PACKAGE_NAME}.v${BUNDLE_VERSION}.clusterserviceversion.yaml" +CSV_FILE="${MANIFESTS}/${PACKAGE_NAME}.clusterserviceversion.yaml" + +# Generates bundle from existing CRDs and Resources +generate_olm_bundle() { + rm -rf "${BUNDLE}" + rm -rf "${CRDDIR}" + mkdir -p "${CRDDIR}" + cp -R "${CRD_DIR}" "${CRDDIR}" + + helm template flink-kubernetes-operator helm/flink-kubernetes-operator > "$CRDDIR/template.yaml" + + operator-sdk generate bundle \ + --input-dir="$CRDDIR" \ + --output-dir="$BUNDLE" \ + --kustomize-dir="$CSV_TEMPLATE_DIR" \ + --package="$PACKAGE_NAME" \ + --version="$BUNDLE_VERSION" \ + --channels="$CHANNELS" + + # Remove Dockerfile generated by operator-sdk + rm ./bundle.Dockerfile + cp "${CSV_TEMPLATE_DIR}/bundle.Dockerfile" "${BUNDLE}" + + # Remove extra files added by operator-sdk + rm "${MANIFESTS}"/*webhook-service*.yaml + + # Update CSV filename to name traditionally used for OperatorHub. + # Is this step necessary ? + # mv "${MANIFESTS}"/*.clusterserviceversion.yaml "${CSV_FILE}" + + yq ea -i 'select(fi==0).metadata.annotations = select(fi==1).metadata.annotations | select(fi==0)' "${CSV_FILE}" "${CSV_TEMPLATE}" + yq ea -i ".spec.install.spec.deployments[0].spec.template.spec.securityContext = {}" "${CSV_FILE}" + yq ea -i '.spec.install.spec.deployments[0].spec.template.spec.containers[0].resources = {"requests": {"cpu": "10m", "memory": "100Mi"}}' "${CSV_FILE}" + yq ea -i '.spec.install.spec.deployments[0].spec.template.spec.containers[1].resources = {"requests": {"cpu": "10m", "memory": "100Mi"}}' "${CSV_FILE}" + + # Only needed until this PR gets merged: https://github.com/apache/flink-kubernetes-operator/pull/420 + # yq ea -i '.spec.webhookdefinitions[0].rules[0].apiGroups=["flink.apache.org"]' "${CSV_FILE}" + # yq ea -i '.spec.webhookdefinitions[1].rules[0].apiGroups=["flink.apache.org"]' "${CSV_FILE}" + # yq ea -i '.spec.webhookdefinitions[0].generateName="validationwebhook.flink.apache.org"' "${CSV_FILE}" + # yq ea -i '.spec.webhookdefinitions[1].generateName="mutationwebhook.flink.apache.org"' "${CSV_FILE}" + + # The OLM lets operator to watch specified namespace with i.e.: spec.targetNamespaces:[bar, foo] in the OperatorGroup crd. + # The OLM then automatically injects template.metadata.annotations.olm.targetNamespaces:[bar, foo] into deployment.apps/flink-kubernetes-operator + # See PR https://github.com/apache/flink-kubernetes-operator/pull/420 see how the watchedNamespaces is assigned to targetNamespaces. + yq ea -i '(... | select(has("env"))).env += {"name": "WATCH_NAMESPACES", "valueFrom": {"fieldRef": {"fieldPath": "metadata.annotations['\''olm.targetNamespaces'\'']"}}}' "${CSV_FILE}" + + yq ea -i ".spec.install.spec.deployments[0].spec.template.spec.initContainers = load(\"${INIT_CONT}\").initContainers" "${CSV_FILE}" + + yq ea -i '.spec.install.spec.deployments[0].spec.template.spec.volumes[1] = {"name": "keystore","emptyDir": {}}' "${CSV_FILE}" + + yq ea -i "(... | select(has(\"image\"))).image =\"${OPERATOR_IMG}\"" "${CSV_FILE}" + + yq ea -i ".metadata.annotations.createdAt = \"$(date +'%Y-%m-%d %H:%M:%S')\"" "${CSV_FILE}" + + yq ea -i ".metadata.annotations.containerImage = \"${OPERATOR_IMG}\"" "${CSV_FILE}" + + yq ea -i ".spec.replaces = \"${PACKAGE_NAME}.v${PREVIOUS_BUNDLE_VERSION}\" | .spec.replaces style=\"\"" "${CSV_FILE}" + + yq ea -i "del(.subjects[0].namespace)" "${ROLE_BINDING}" +} + +validate_olm_bundle() { + operator-sdk bundle validate "${BUNDLE}" --select-optional name=operatorhub + operator-sdk bundle validate "${BUNDLE}" --select-optional suite=operatorframework +} + +generate_olm_bundle +validate_olm_bundle + diff --git a/tools/olm/env.sh b/tools/olm/env.sh new file mode 100644 index 00000000..9e01d210 --- /dev/null +++ b/tools/olm/env.sh @@ -0,0 +1,42 @@ +#!/bin/bash +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# +# Change this file to your need + +# Specify the release version for this bundle. +BUNDLE_VERSION=1.3.0 + +# This is the previous bundle release. This is used to generate catalog image +# which includes the previous version for testing operator upgrade. +PREVIOUS_BUNDLE_VERSION=1.2.0 + +# Specify an image registry repo to push the bundle and catalog temporarily. +# For example: +# docker.io/your_org/flink-op-bundle:1.3.0 +# docker.io/your_org/flink-op-catalog:1.3.0 +# NOTE: If you specify a public registry such as docker.io, you need to login first. +# i.e. docker login docker.io -u your_docker_org +DOCKER_REGISTRY="ttl.sh" +DOCKER_ORG="$(head -c24 < /dev/random|base64|tr -dc "[:lower:]"|head -c24)" +IMAGE_TAG="10m" + +# Specify the operator image which is used in this bundle. +# ie.: OPERATOR_IMG="docker.io/apache/flink-kubernetes-operator:${BUNDLE_VERSION}" +# https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/development/guide/#building-docker-images +OPERATOR_IMG=ghcr.io/apache/flink-kubernetes-operator:main diff --git a/tools/olm/generate-olm-bundle.sh b/tools/olm/generate-olm-bundle.sh new file mode 100755 index 00000000..31fbe191 --- /dev/null +++ b/tools/olm/generate-olm-bundle.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# +# Generates OLM bundle from existing helm chart +# +# +set -euo pipefail + +. env.sh +BASEDIR=$(dirname "$(realpath "$0")") +TARGET=${BASEDIR}/target +BUNDLE_VERSION=${BUNDLE_VERSION} +DOCKER_ORG=${DOCKER_ORG} +DOCKER_REGISTRY=${DOCKER_REGISTRY} +TAG=${IMAGE_TAG} + +BUNDLE_IMG=${DOCKER_REGISTRY}/${DOCKER_ORG}/flink-op-bundle:${TAG} +BUNDLE=${TARGET}/${BUNDLE_VERSION} +CATALOG_IMG=${DOCKER_REGISTRY}/${DOCKER_ORG}/flink-op-catalog:${TAG} +CATDIR="${TARGET}/cat" +OLMTOOL_IMG="olm-utils" +CSV_TEMPLATE_DIR="${BASEDIR}/csv-template" + +# Generate bundle in a docker container +generate_olm_bundle() { + uid="$(id -g ${USER})" + gid="$(id -u ${USER})" + cp -r ../../helm ./ + docker build -t "${OLMTOOL_IMG}" -f utils.Dockerfile ${BASEDIR} + docker run --user="${uid}:${gid}" -v ${BASEDIR}:/olm "${OLMTOOL_IMG}" + rm -rf helm +} + +# Push bundle and catalog images for testing olm subscription +build_push_bundle(){ + docker build -f ${CSV_TEMPLATE_DIR}/bundle.Dockerfile -t "${BUNDLE_IMG}" ${BUNDLE} + docker push "${BUNDLE_IMG}" +} +build_push_catalog(){ + docker build -f ${CSV_TEMPLATE_DIR}/catalog.Dockerfile -t ${CATALOG_IMG} ${CATDIR} + docker push ${CATALOG_IMG} +} + +# Simplest way to build local index(catalog) image without any upgrade path +# opm index add --bundles "${BUNDLE_IMG}" --tag "${CATALOG_IMG}" -c docker +# But we also want to test upgrade from the previus version +# Therefor, building local catalog image using latest file-based-catalog format +# https://olm.operatorframework.io/docs/reference/file-based-catalogs/#opm-generate-dockerfile +# We assume someone runing an older olm bundle. +# This enables testing upgrade from the previous version using startingCSV in Subscription +generate_fbc_config() { +rm -rf "${CATDIR}" +mkdir -p "${CATDIR}" +CATCONF="${CATDIR}/config.yaml" +OPM_IMAGE="quay.io/operator-framework/opm:latest" +cat <<EOF > "${CATCONF}" +--- +defaultChannel: alpha +icon: + base64data: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4KPCEtLQpMaWNlbnNlZCB0byB0aGUgQXBhY2hlIFNvZnR3YXJlIEZvdW5kYXRpb24gKEFTRikgdW5kZXIgb25lCm9yIG1vcmUgY29udHJpYnV0b3IgbGljZW5zZSBhZ3JlZW1lbnRzLiAgU2VlIHRoZSBOT1RJQ0UgZmlsZQpkaXN0cmlidXRlZCB3aXRoIHRoaXMgd29yayBmb3IgYWRkaXRpb25hbCBpbmZvcm1hdGlvbgpyZWdhcmRpbmcgY29weXJpZ2h0IG93bmVyc2hpcC4gIFRoZSBBU0YgbGljZW5zZXMgdGhpcyBmaWxlCnRvIHlvdSB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUKIkxpY2Vuc2UiKTsgeW91IG1heSBub3QgdXNlIHRoaX [...] + mediatype: image/svg+xml +name: flink-kubernetes-operator +schema: olm.package +--- +name: alpha +package: flink-kubernetes-operator +schema: olm.channel +entries: +- name: flink-kubernetes-operator.v${PREVIOUS_BUNDLE_VERSION} +- name: flink-kubernetes-operator.v${BUNDLE_VERSION} + replaces: flink-kubernetes-operator.v${PREVIOUS_BUNDLE_VERSION} + skipRange: ">=1.0.1 <${BUNDLE_VERSION}" +EOF + docker run ${OPM_IMAGE} render quay.io/operatorhubio/flink-kubernetes-operator:v${PREVIOUS_BUNDLE_VERSION} --output=yaml >> "${CATCONF}" + docker run ${OPM_IMAGE} render ${BUNDLE_IMG} --output=yaml >> "${CATCONF}" + docker run -v ${CATDIR}:/cat ${OPM_IMAGE} validate /cat +} + +print_helper(){ +echo "### First deploy the catalog server that serves the bundle" +echo "The OLM operator does not come with Kubernetes by default. To install OLM, run:" +echo "curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.22.0/install.sh | bash -s v0.22.0" +echo "cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha1 +kind: CatalogSource +metadata: + name: olm-flink-operator-catalog + namespace: default +spec: + sourceType: grpc + image: "${DOCKER_REGISTRY}/${DOCKER_ORG}/flink-op-catalog:${TAG}" +EOF" + +echo +echo "### After the catalog server is ready, create a subscription" +echo "cat <<EOF | kubectl apply -f - +apiVersion: operators.coreos.com/v1alpha2 +kind: OperatorGroup +metadata: + name: default-og + namespace: default +spec: + # if not set, default to watch all namespaces + targetNamespaces: + - default +--- +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: flink-kubernetes-operator + namespace: default +spec: + channel: alpha + name: flink-kubernetes-operator + source: olm-flink-operator-catalog + sourceNamespace: default + # For testing upgrade from previous version + #installPlanApproval: Automatic # Manual + #startingCSV: "flink-kubernetes-operator.v${PREVIOUS_BUNDLE_VERSION}" +EOF" + +echo +echo "After the operator is ready, deploy a flink job using: +kubectl create -f ../../examples/basic.yaml +or +kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.2/examples/basic.yaml +" + + +} +generate_olm_bundle +build_push_bundle + +generate_fbc_config +build_push_catalog + +print_helper diff --git a/tools/olm/install-prereq.sh b/tools/olm/install-prereq.sh new file mode 100644 index 00000000..bcc7f594 --- /dev/null +++ b/tools/olm/install-prereq.sh @@ -0,0 +1,97 @@ +#!/bin/bash +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +# Install RedHat Prereqs: +if [ -f /etc/redhat-release ] +then + yum install make podman git tree jq unzip -y + ln -s /usr/bin/podman /usr/bin/docker +fi + +# Install kubectl +wget -qO /usr/local/bin/kubectl "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" +chmod a+x /usr/local/bin/kubectl + + +# Install helm +cd /root +wget https://get.helm.sh/helm-v3.10.1-linux-amd64.tar.gz +tar -zxvf helm-v3*-linux-amd64.tar.gz +mv linux-amd64/helm /usr/local/bin/helm +chmod 600 /root/auth/kubeconfig + +# Install the latest version of operator-sdk: +export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac) +export OS=$(uname | awk '{print tolower($0)}') +cd /root +export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/v1.25.0 +curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH} +gpg --keyserver keyserver.ubuntu.com --recv-keys 052996E2A20B5C7E +curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt +curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt.asc +gpg -u "Operator SDK (release) <cncf-operator-...@cncf.io>" --verify checksums.txt.asc +grep operator-sdk_${OS}_${ARCH} checksums.txt | sha256sum -c - +chmod +x operator-sdk_${OS}_${ARCH} && sudo mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk +operator-sdk version + +# Install OPM +curl -o opm-checksums.txt -LO https://github.com/operator-framework/operator-registry/releases/download/v1.26.2/checksums.txt +wget https://github.com/operator-framework/operator-registry/releases/download/v1.26.2/linux-amd64-opm +grep linux-amd64-opm opm-checksums.txt | sha256sum -c - +chmod +x linux-amd64-opm && sudo mv linux-amd64-opm /usr/local/bin/opm +opm version + +# Install YQ +cd /root +export VERSION=v4.29.2 +export BINARY=yq_linux_amd64 +wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\ + tar xz && mv ${BINARY} /usr/bin/yq +yq --version + +# Install Docker +cd /root +#curl -fsSL https://get.docker.com -o get-docker.sh +#chmod +x ./get-docker.sh +#./get-docker.sh +apt-get install -y \ + ca-certificates \ + curl \ + gnupg \ + lsb-release +mkdir -p /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +echo \ + "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +apt-get update +apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin + +# Install KIND +cd /root +curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64 +chmod +x ./kind +sudo mv ./kind /usr/local/bin/kind +kind delete cluster +kind create cluster +kubectl cluster-info --context kind-kind +operator-sdk olm install +kubectl config set-context --current --namespace=default + + diff --git a/tools/olm/utils.Dockerfile b/tools/olm/utils.Dockerfile new file mode 100644 index 00000000..db5ef2df --- /dev/null +++ b/tools/olm/utils.Dockerfile @@ -0,0 +1,43 @@ +################################################################################ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ +# Generate Bundle +FROM ubuntu:22.04 +WORKDIR /olm +RUN apt-get update && apt-get -y install curl +# helm +RUN curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && \ + chmod 700 get_helm.sh && \ + ./get_helm.sh +# yq +RUN export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac); \ + export OS=$(uname | awk '{print tolower($0)}'); \ + curl -L https://github.com/mikefarah/yq/releases/latest/download/yq_${OS}_${ARCH} -o /usr/local/bin/yq && \ + chmod +x /usr/local/bin/yq + +# operator-sdk +RUN export ARCH=$(case $(uname -m) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(uname -m) ;; esac); \ + export OS=$(uname | awk '{print tolower($0)}'); \ + export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/v1.25.3 && \ + curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH} && \ + chmod +x operator-sdk_${OS}_${ARCH} && \ + mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk + +RUN operator-sdk version; \ + yq --version; \ + helm version; +ENTRYPOINT [ "/olm/docker-entry.sh" ]