GitHub user pykenny edited a discussion: Kuberentes integration settings/commands for Breeze environment?
I want to reproduce several issues I encountered when using operator classes under "[cncf-kuberentes](https://github.com/apache/airflow/discussions/52131#discussion-8487393)" provider. To better provide proof to these problems, I want to make a minimum example with Breeze development environment, but now I'm totally stuck and cannot figure out how to integrate a local Kubernetes cluster to a running Breeze environment (having very limited knowledge about Docker and Kubernetes, while both K8s environment setup and settings required to run these K8s tasks are mostly managed by other teams). Any idea or suggested guide to help with this scenario? Also, there's no documentation on `breeze-k8s` subcommand, from help doc it seems to be utility that runs Airflow in local Kubernetes cluster (such as using Kuberentes Executor?), and I'm not certain if it does any help to this scenario. I also found this issue thread but seems this has been suspended for more than a year after it's proposed: - https://github.com/apache/airflow/issues/40005 Here's an unsuccessful attempt to create a K8s cluster locally and let Airflow to gain access to the cluster (with part of the instructions/settings suggested by ChatGPT, if you find them erroneous in some way): 0. We've already have Docker (along with local K8s utilities) installed when installing Breeze. 1. Install [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) and run `kind create cluster --name airflow-testing` to create a cluster named `kind-airflow-testing`. Check cluster information with `kubectl cluster-info --context kind-airflow-testing` command. 2. Now we have the cluster's configuration stored in `~/.kube/config`. Copy the configuration file to the shared mount location under repository root: ```bash cp ~/.kube/config ./files/kubeconfig ``` 3. Add environment variable to `files/airflow-breeze-config/environment_variables.env`: ```bash KUBECONFIG=/files/kubeconfig ``` By default, K8s utilities will use what `KUBECONFIG` points to to initialize configuration. 4. Start Breeze environment and run Airflow: ```bash breeze --python 3.9 --backend mysql --mysql-version 8.0 start-airflow ``` 5. Build a DAG that uses any operator under "cncf-kuberentes" provider (well, one supposed to know where to place DAG files, if one, patiently, read through the Breeze doc, right to the [end chapter](https://github.com/apache/airflow/blob/207c4b09cf7c34b740472aca79dadc19c6f69db4/dev/breeze/doc/10_advanced_breeze_topics.rst)), enable it, run it, and welcome all kinds of `NewConnectionError` appearing on the console/Web-UI! GitHub link: https://github.com/apache/airflow/discussions/52131 ---- This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
