Re: Task middleware or task hooks as a way to increase observability

2019-04-01 Thread Stachurski, Stephan
nup... > > You can write a whole little handler thingy to handle all the messages and > get whatever you want done, with almost no code in airflow or the dags. > > B > > Sent from a device with less than stellar autocorrect > > > On Mar 31, 2019, at 3:29 PM, Stachu

Re: Task middleware or task hooks as a way to increase observability

2019-04-01 Thread Stachurski, Stephan
lib that does “things” with the messages... logs in dynamo for > > starters, some carry on to slack notifications, most get passed to the > > logging/evening service after some cleanup... > > > > You can write a whole little handler thingy to handle all the messages > and >

Task middleware or task hooks as a way to increase observability

2019-03-31 Thread Stachurski, Stephan
Hi- I'm pretty new to airflow, and I'm trying to work on getting visibility/observability into what airflow is doing. I would like to be able to observe things about dag runs and task instances. I would like to be able to send metrics to time series databases (possibly extending the existing airf

Re: Hangout: Learn how to hack on Airflow

2019-03-27 Thread Stachurski, Stephan
Please add me! On Wed, Mar 27, 2019 at 8:24 AM Thao Ha Thi Phuong wrote: > Hi. I'm in. Please add me to the hangout. Thanks a lot :) > > On Tue, 26 Mar 2019 at 18:34, Priyanka Singh (Reco 2.0) > wrote: > > > I am in > > > > On Tue, Mar 26, 2019 at 10:14 PM Shiyuan wrote: > > > > > I am in. Ple