Re: Regarding airflow 1.10

2018-08-22 Thread Taylor Edmiston
Hemanth - To add to what Fokko mentioned... You can find the KubernetesExecutor implementation in KubernetesExecutor and AirflowKubernetesScheduler

Re: Regarding airflow 1.10

2018-08-21 Thread Driesprong, Fokko
Hi Hemanth, Thanks for the question. There is the KubernetesExecutor which just uses a container with Airflow to schedule a task on the kubernetes cluster. This container will then execute a single task, and then the Executor checks the exitcode of the container. There is also the

Regarding airflow 1.10

2018-08-21 Thread Musunuru, Hemanth
Hi Team, Does Kubernetes executor in 1.10 create new pod for every task ? (Which means won’t there be any active workers running except for new Tasks) We are trying to containerize our existing to airflow environment, your reply Will help us to understand it more. Thanks in Advance . Thanks