Yeah, no strength or bandwidth right now for a discussion. And would very much prefer we focus on getting 3.0.0 out and any bug fixes after that
On Tue, 15 Apr 2025 at 14:08, Jarek Potiuk <ja...@potiuk.com> wrote: > Very much agree with Ash here. > > On Tue, Apr 15, 2025 at 10:30 AM Ash Berlin-Taylor <a...@apache.org> wrote: > > > Hi Howard, > > > > Your timing is…. Interesting given we are focusing on getting 3.0 shipped > > and are in the final throws of that. > > > > Let's revisit this in May or June once the dust has settled from 3.0, any > > immediate bug fixes we might need, and give people a little bit of time > to > > unwind after the crunch. > > > > -ash > > > > > On 14 Apr 2025, at 16:22, Howard Yoo <howard...@gmail.com> wrote: > > > > > > Hi Airflow community members, > > > > > > Apache Airflow has been moving steadily from conventional statsd based > > > telemetry into OpenTelemetry (opentelemetry.io), which has gained > > > popularity and support for the last few years. There has been AIP > > proposals > > > which had a 'phased' approach to incorporate > > > > > > 1. Metrics > > > 2. Traces > > > 3. Logs > > > > > > Based on the current availability and maturity of Otel's support. Since > > the > > > metrics emission based on statsd were the fastest low-hanging fruit > that > > we > > > could implement OTEL metrics support relatively easy, which we did as > our > > > first try. > > > > > > Second was the Traces. OTEL support on traces for Python was very > mature > > > and well established. Even though Airflow did not have any prior > > > implementation on emitting Traces, we were able to instrument it and > > > produce traces on both Airflow specific processes (jobs) (e.g. > > schedulers, > > > triggers, executors), as well as task runs. > > > > > > However, the area for Logs were not done yet, due at that time a > > relatively > > > early and immature state of logs support in Otel. > > > > > > Now, it seems like Logging support on Python has become stable and > > reliable > > > that we can use it to emit log messages in Otel format. So, the timing > > > seems to be quite perfect for us to start up a discussion on it. > > > > > > Logging support in Airflow is basically provided via 'provider' model > > where > > > Airflow itself does not have a specific implementation of how logging > is > > > done, but requests contributors to provide provider that can plug-in > > their > > > logging support. > > > > > > Would we want to move to that direction, of having 'opentelemetry' > > provider > > > ? > > > Or do we want to have logging support as more of a native support way? > > > > > > The reason why I would like to start this discussion thread was to gain > > > opinions from the community on whether Airflow has the interest to > > support > > > OTEL logging first, and then any opinions on specifics of what would be > > the > > > best way to support OTEL logging. > > > > > > Please feel free to reply and add any comments to the thread to voice > > your > > > opinions. If there isn't any substantial interest in OTEL logging, > well, > > we > > > can live without it. > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
