potiuk commented on a change in pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#discussion_r365083603
########## File path: BREEZE.rst ########## @@ -528,6 +509,65 @@ As soon as you enter the Breeze environment, you can run Airflow unit tests via For supported CI test suites, types of unit tests, and other tests, see `TESTING.rst <TESTING.rst>`_. +Running Tests with Kubernetes in Breeze +======================================= + +In order to run Kubernetes in Breeze you can start Breeze with ``--start-kind-cluster`` switch. This will +automatically create a Kind Kubernetes cluster in the same ``docker`` engine that is used to run Breeze +Setting up the Kubernetes cluster takes some time so the cluster continues running +until the cluster is stopped with ``--stop-kind-cluster`` switch or until ``--recreate-kind-cluster`` +switch is used rather than ``--start-kind-cluster``. + +The cluster name follows the pattern ``airflow-python-X.Y.Z-vA.B.C`` where X.Y.Z is Python version +and A.B.C is kubernetes version. This way you can have multiple clusters setup and running at the same +time for different python versions and different kubernetes versions. + +The Control Plane is available from inside the docker image via <CLUSTER_NAME>-control-plane:6443 +host:port, the worker of the kind cluster is available at <CLUSTER_NAME>-worker +and webserver port for the worker is 30809. + +The Kubernetes Cluster is started but in oder to deploy airflow to Kubernetes cluster you need to: + +1. Build the image. +2. Load it to Kubernetes cluster. +3. Deploy airflow application. + +It can be done with single script: ``./scripts/ci/in_container/kubernetes/deploy_airflow_to_kubernetes.sh`` + +You can however work separately on image in Kubernetes and the deploying the Airflow app in the cluster. + +Building Airflow Images and Loading them to Kubernetes cluster +-------------------------------------------------------------- + +This is done using ``./scripts/ci/in_container/kubernetes/docker/rebuild_airflow_image.sh`` script: + +1. Latest ``apache/airflow:master-pythonX.Y-ci`` images are rebuilt using latest sources. +3. New kubernetes image based on the ``apache/airflow:master-pythonX.Y-ci`` is build with + necessary scripts added to run in kubernetes. The image is tagged with + ``apache/airflow:master-pythonX.Y-ci-kubernetes`` tag. +4. The image is loaded to the kind cluster using ``kind load`` command from kind. Review comment: ```suggestion 4. The image is loaded to the kind cluster using ``kind load`` command with``airflow:latest tag``. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
