aeroyorch commented on PR #73044: URL: https://github.com/apache/airflow/pull/73044#issuecomment-6720796058
> > [@aeroyorch](https://github.com/aeroyorch) are you still working on this PR? > > Hi Shahar, sorry I haven't been able to dedicate much time to this over the last few weeks... but yes, I'm still working on it. > > I moved the PR back to draft after Potiuk's review because I ran a benchmark that included log emission, and it confirmed that logging could affect the event loop. > > That said, we could merge the PR as it is and address log emission in a followup. IMHO it deserves a deeper look at the best approach and the tradeoff between performance and complexity. In fact, I was thinking of using a queue on a separate thread, so the pod's async coroutines can enqueue logs without the overhead of `to_thread`. @shahar1 I've moved the PR back to ready for review. What do you think about reviewing the impact of log emission on the main event loop in a follow-up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
