StackOverflow synonym proposal

2018-01-15 Thread Baghino, Stefano(AWF)
Hi everybody, on StackOverflow many people regularly come to get help about Airflow. Currently on the site there are both the `airflow` and `apache-airflow` tags, which both mean the same but with the former having more tagged questions and history behind it. To make it easier for people to

Close SqlSensor Connection

2018-01-15 Thread Alexis Rolland
Hello everyone, I'm reaching out to discuss / suggest a small improvement in the class SqlSensor: https://pythonhosted.org/airflow/_modules/airflow/operators/sensors.html We are currently using SqlSensors on top of Teradata in several DAGs. When the DAGs execute we receive the following error

Re: Airflow + Kubernetes + Git Sync

2018-01-15 Thread jordan.zuc...@gmail.com
On 2018-01-12 16:17, Anirudh Ramanathan wrote: > > Any good way to debug this? > > One way might be reading the events from "kubectl get events". That should > reveal some information about the pod removal event. > This brings up another question - should

Re: Airflow + Kubernetes + Git Sync

2018-01-15 Thread jordan.zuc...@gmail.com
On 2018-01-13 08:12, Daniel Imberman wrote: > @jordan can you turn delete mode off and post the kubectl describe results > for the workers? Already had delete mode turned off. This was a really useful command. I can see the basic logs in the k8s dashboard: + airflow

Re: Credentials for accessing endpoints

2018-01-15 Thread Shah Altaf
It might depend on who you're trying to secure it from. My understanding is (correct me) that the airflow connections are encrypted in the DB so it should be good enough -- just make sure you've got the crypto package and you've generated a FERNET key. See :

Credentials for accessing endpoints

2018-01-15 Thread Veeranagouda Mukkanagoudar
Hi, We have been using .conf/.ini files to store the credentials to access endpoints. Wondering if there is a better and secure way to store the credentials besides as Airflow connections in Admin console. -Veera

Re: Fix on_kill command for operators

2018-01-15 Thread EKC (Erik Cederstrand)
I'm not sure if it's related, but there's an additional issue with attempting to kill processes, that doesn't always kill the process: https://issues.apache.org/jira/browse/AIRFLOW-949 Kind regards, Erik Cederstrand From: Milan van der Meer