Hi all, *TL;DR* Airflow doesn't have adequate built-in support for managing per-task credentials, the concept of connection helps to certain extent but is not very satisfactory. The current Airflow KubernetesExecutor work opens up the possibility to handle task credentials at the framework level and separate workflow business logic from credential/account management by leveraging the Kubernetes initializer mechanism. At the end of the day, a task/dag only needs to specify an account name and everything else is taken care by the Airflow framework in a secure fashion.
Detailed design: https://cwiki.apache.org/confluence/display/AIRFLOW/Managing+Per-task+Credentials+in+KubernetesExecutor Critics and comments are welcome :-) Thank you. Feng