Running Airflow in K8S Cluster with Istio and Mutual TLS?

2018-07-03 Thread Kevin Lam
Hi, We run airflow as a set of k8s deployments inside of a GKE cluster, similar to the way specified in Mumoshu's github repo: https://github.com/ mumoshu/kube-airflow. We are currently investigating securing our clusters using istio [1]. Specifically we are interested in adding additional

Re: K8S deployment operator proposal

2018-07-03 Thread Daniel Imberman
An example of creating a deployment using the k8s model architecture can be found here: https://github.com/kubernetes-client/python/blob/master/examples/deployment_examples.py def create_deployment_object(): # Configureate Pod template container container = client.V1Container(

K8S deployment operator proposal

2018-07-03 Thread Daniel Imberman
Hi all, Enclosed is a proposal for a kubernetes deployment management operator. I think this would be a good addition to current k8s offerings s.t. users can actually launch persistent applications from airflow DAGs. * What?* Add an operator that monitors a k8s deployment, declaring the task