mik-laj commented on a change in pull request #12755: URL: https://github.com/apache/airflow/pull/12755#discussion_r534266991
########## File path: docs/apache-airflow/kubernetes.rst ########## @@ -20,31 +20,62 @@ Kubernetes ---------- +Apache Airflow aims to be a very Kubernetes-friendly project, and many users run Airflow +from within a Kubernetes cluster in order to take advantage of the increased stability +and autoscaling options that Kubernetes provides. + + +Helm Chart +^^^^^^^^^^ + +`Helm https://helm.sh/`_ provides a simple mechanism to deploy software to a +Kubernetes cluster, and the community is working on an +`offical Helm chart https://github.com/apache/airflow/tree/master/chart`_ for Airflow +which will be released with Airflow 2.1. The chart has +`KEDA integration https://github.com/apache/airflow/tree/master/chart#autoscaling-with-keda`_ +which allows you to autoscale your Celery workers. More information is available +`in this blog post https://www.astronomer.io/blog/the-keda-autoscaler`_. + +Other Helm charts in the wild: + +* https://github.com/airflow-helm/charts +* https://bitnami.com/stack/apache-airflow/helm +* https://github.com/astronomer/airflow-chart + + Kubernetes Executor ^^^^^^^^^^^^^^^^^^^ -The :doc:`Kubernetes Executor <executor/kubernetes>` allows you to run all the Airflow tasks on -Kubernetes as separate Pods. +The :doc:`Kubernetes Executor <executor/kubernetes>` allows you to run all the Airflow +tasks on Kubernetes as separate Pods. + KubernetesPodOperator ^^^^^^^^^^^^^^^^^^^^^ -The :ref:`KubernetesPodOperator <howto/operator:KubernetesPodOperator>` allows you to create -Pods on Kubernetes. +The :ref:`KubernetesPodOperator <howto/operator:KubernetesPodOperator>` allows you to +create Pods on Kubernetes. + Pod Mutation Hook ^^^^^^^^^^^^^^^^^ -The Airflow local settings file (``airflow_local_settings.py``) can define a ``pod_mutation_hook`` function -that has the ability to mutate pod objects before sending them to the Kubernetes client -for scheduling. It receives a single argument as a reference to pod objects, and -is expected to alter its attributes. +The Airflow local settings file (``airflow_local_settings.py``) can define a +``pod_mutation_hook`` function that has the ability to mutate pod objects before sending +them to the Kubernetes client for scheduling. It receives a single argument as a +reference to pod objects, and is expected to alter its attributes. This could be used, for instance, to add sidecar or init containers to every worker pod launched by KubernetesExecutor or KubernetesPodOperator. - .. code-block:: python def pod_mutation_hook(pod: Pod): pod.annotations['airflow.apache.org/launched-by'] = 'Tests' + + +Airflow on K8s Operator Review comment: Has Apache made an official release? Is this sentence violating Apache release policy? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
