wjddn279 commented on PR #56044: URL: https://github.com/apache/airflow/pull/56044#issuecomment-3426111038
@potiuk Thank you for your response. I should have based my thinking on actual use cases rather than abstract concerns—I was lacking in that regard. Thank you for the detailed explanation. To summarize what you've mentioned (for the benefit of others who may review this later), we can classify the cases as follows: 1. Parent process does not use DB → No connections are created in the first place, so no issue arises 2. Parent uses DB but child process does not use DB → Since the child doesn't use it, there's no need to dispose() on fork 3. Both parent and child processes use DB → Race conditions 'could potentially occur' (not if query execution is synchronous) For workers, dag-processor, and triggerer, cases 1 or 2 clearly apply, and for the api-server, case 1 also clearly applies since it spawns workers. Therefore, only the scheduler when using Local Executor, which falls under case 3, requires verification. However, since all queries executed in the main process (scheduler) are performed synchronously, it's evident that no race condition exists. Ultimately, I understand you're saying that concerns about race conditions in this case are not warranted within the Airflow system. The solution will be to not dispose on fork for MySQL, but instead apply engine.dispose(close=True) for Local Executor. If you had a different intention in mind, please let me know. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
