And let me add to it - we laid some foundations for it with Melodie Ezeani - the Outreachy intern where we did some internal integration work that let us understand the challenges and state of open-telemetry.
I am super excited about what open-telemetry can bring to Airflow - both short term (in metrics and instrumentation) and longer term - in logging when logging is mature enough. The open-telemetry mov is at the heart of the principles we have - making Airflow "modern" but at the same time delegating what's not "core" to those who can do it better. Open Telemetry is precisely about that, Howard particularly brought a great experience from using Open Telemetry before and making some good judgements and POC on the metrics produced by Airflow and how they can be useful from the "maintainer value" side. I looked at it from the technical integration POV - and Melodie helped to validate some of the assumptions and expose some of the technical challenges. The composite result is good, but we are looking with Howard on some insightful comments and critique - especially from the users of Airflow! I look forward to more cool stuff on Airflow! J. On Fri, Mar 25, 2022 at 9:39 PM Howard Yoo <howard...@gmail.com> wrote: > > Hi all, > > I am pleased to announce the start of the discussion for the new AIP draft > that was recently been published: > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-49+OpenTelemetry+Support+for+Apache+Airflow > > Jarek Potuik and I have been discussing about this proposal since early this > year. During that time, we worked together on drafting this proposal, as well > as doing another round of mini-POC to refresh and test on feasibility of > OpenTelemetry on Apache Airflow. > > As the POC was successful in terms of testing the OpenTelemetry on Airflow, > we would like to expand the discussion to a wider user community here this > mailing list to gather more consensus, comments, and feedbacks regarding this > AIP. > > Sincerely, > Howard and Jarek
