uranusjr commented on PR #68708: URL: https://github.com/apache/airflow/pull/68708#issuecomment-4847730835
Hmm, for OL context we need `get_template_context()` and `extract_metadata()`, which use Comms with a threading lock. This has been fine since forking makes the locking per-process. But with threading, the lock would apply throughout all the threads, making it much easier to deadlock and kill the entire process. Maybe it would be worthwhile to add some sort of timeout mechanism so the threads can go away if they live for too long? This would also help with the OOM issue mentioned above. Also, as Kacper mentioned, a doc addition would be nice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
